Cognitive Productivity with macOS: 7 Principles for Getting Smarter with Knowledge
Whether you are a professional knowledge worker, a student or a lifelong learner, if you value your time and competence,
1,069 105 9MB
English Pages (224 pages) : illustrations [406] Year 2018
Polecaj historie
Citation preview
Cognitive Productivity Using Knowledge to Become Profoundly Effective Luc P. Beaudoin This book is for sale at http://leanpub.com/cognitiveproductivity This version was published on 2016-10-27
This is a Leanpub book. Leanpub empowers authors and publishers with the Lean Publishing process. Lean Publishing is the act of publishing an in-progress ebook using lightweight tools and many iterations to get reader feedback, pivot until you have the right book and build traction once you do. © 2013 - 2016 Luc P. Beaudoin
Tweet This Book! Please help Luc P. Beaudoin by spreading the word about this book on Twitter! The suggested tweet for this book is: I just bought the book Cognitive Productivity by Luc Beaudoin of CogZest! The suggested hashtag for this book is #CognitiveProductivity. Find out what other people are saying about the book by clicking on this link to search for this hashtag on Twitter: https://twitter.com/search?q=#CognitiveProductivity
In memory of Pierre-Elliot Trudeau, Sir Winston Churchill, Jacques Brel, and their cognitive zest
Contents List of Figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
i
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
iii
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
vi
I
1
Challenges and opportunities . . . . . . . . . . . . . . . . . . .
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Broad cognitive science . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Updating how we think about knowledge and ourselves . . . . . . . . . . . . . 1.2.1 The designer stance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.2 Mindware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.3 Adult mental development . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2.4 Effectance: motivation for competence . . . . . . . . . . . . . . . . . . . . 1.2.5 Meta-effectiveness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Example knowledge resources referenced in this book . . . . . . . . . . . . . . 1.3.1 Keith Stanovich (2009). What Intelligence Tests Miss: The Psychology of Rational Thought . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.2 John Gottman: Seven Principles for Making Marriage Work and The Relationship Cure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.3 Ries (2011): The Lean Startup . . . . . . . . . . . . . . . . . . . . . . . . . 1.3.4 The work of Aaron Sloman and other cognitive scientists . . . . . . . . . . 1.4 Three vignettes: Disasters avertable by applying knowledge . . . . . . . . . . . 1.4.1 Being taken to the trough but choosing not to partake . . . . . . . . . . . . 1.4.2 The applied science of marital failure . . . . . . . . . . . . . . . . . . . . . 1.4.3 Project failures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 The imperative of meta-effectiveness . . . . . . . . . . . . . . . . . . . . . . . . 1.6 Overview of this book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2. Psychological contributors to effectiveness . 2.1 Effectiveness: The master objective . . . 2.2 Mastering objective knowledge . . . . . 2.2.1 Developing implicit understanding
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . . . . . . .
2 6 10 13 15 16 17 18 19
.
20
. . . . . . . . .
21 23 24 24 25 27 27 28 29
. . . .
32 34 35 37
CONTENTS
2.3 2.4 2.5 2.6 2.7 2.8 2.9 2.10
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
39 40 42 43 46 47 48 49
3. Challenges to meta-effectiveness . . . . . . . . . . . . . . . . . . . . . . 3.1 Information technology: Lack of support for cognitive productivity 3.1.1 Tools designed for surfing, not delving . . . . . . . . . . . . . 3.1.2 Inadequate support for annotation . . . . . . . . . . . . . . . 3.1.3 The need to annotate entire resources . . . . . . . . . . . . . . 3.1.4 The need for synchronized annotation services . . . . . . . . 3.1.5 Where’s the productive practice app? . . . . . . . . . . . . . . 3.1.6 Where’s the glossary manager and instiller? . . . . . . . . . . 3.1.7 Drawbacks of smartphones and tablets . . . . . . . . . . . . . 3.1.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Challenging circumstances . . . . . . . . . . . . . . . . . . . . . . 3.2.1 Demands on our time . . . . . . . . . . . . . . . . . . . . . . 3.2.2 Sequestered and ill-presented information . . . . . . . . . . . 3.2.3 Cognitive productivity training . . . . . . . . . . . . . . . . . 3.3 Psychological challenges . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 Cognitive science in the realm of knowledge work . . . . . . . 3.3.2 Illusions of meta-effectiveness . . . . . . . . . . . . . . . . . . 3.3.2.1 Illusions of helpfulness of information . . . . . . . . . . 3.3.2.2 Illusions of comprehension . . . . . . . . . . . . . . . . 3.3.2.3 Illusions of (future) recall . . . . . . . . . . . . . . . . . 3.3.2.4 Illusions of rationality: transfer reframed . . . . . . . . 3.3.3 Cognitive miserliness and its antagonists . . . . . . . . . . . . 3.3.3.1 Effectance as a propensity to develop competence . . . 3.3.3.2 Perceived self-efficacy . . . . . . . . . . . . . . . . . . 3.3.4 Cognitive aging . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.5 Distractibility and the mind’s design . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . .
50 52 52 53 54 55 59 60 61 62 62 62 63 64 67 67 68 69 70 71 74 78 79 83 85 85
Cognitive science . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
87
4. Introduction to Part 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
88
5. Your mind and its wares (the mind’s design) . . . . . . . . . . . . . . . . . . . . . . . 5.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Functional characterization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
89 90 93
II
Developing skills . . . . . . . . . . . . . . . . . . . . Mastering norms . . . . . . . . . . . . . . . . . . . . Developing attitudes . . . . . . . . . . . . . . . . . . Developing propensities, habits and other dispositions Developing mentally . . . . . . . . . . . . . . . . . . Countering cognitive aging . . . . . . . . . . . . . . . Becoming more meta-effective . . . . . . . . . . . . . Back to the top: Excelling . . . . . . . . . . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
CONTENTS
. . . . . . . . . . .
93 94 96 97 99 101 103 107 109 110 113
6. Adult mental development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 Objective knowledge (World 3), virtual machines (World 2’) and the rest (World 1) 6.1.1 Mindware as World 2’: Virtual machinery . . . . . . . . . . . . . . . . . . . 6.2 Understanding understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.3 Developing monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.4 Developing motivators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.5 Developing long-term working memory . . . . . . . . . . . . . . . . . . . . . . . 6.6 Developing representational machinery . . . . . . . . . . . . . . . . . . . . . . . 6.6.1 Growth of component processes . . . . . . . . . . . . . . . . . . . . . . . . . 6.6.2 Taking child and adult development seriously . . . . . . . . . . . . . . . . . 6.6.2.1 Some phenomena that highlight mental representations . . . . . . . . 6.6.2.2 Representational redescription (RR) . . . . . . . . . . . . . . . . . . . 6.6.3 RR in reverse: The problem of instilling mindware . . . . . . . . . . . . . . .
119 121 123 125 127 129 132 135 137 139 139 142 146
7. Deliberate practice: A source of effectiveness . . . . . . . . . . . . . . . 7.1 Practice enhances factual learning and memory . . . . . . . . . . . 7.1.1 Practicing slows forgetting . . . . . . . . . . . . . . . . . . . 7.1.2 Practicing trumps reviewing . . . . . . . . . . . . . . . . . . . 7.1.3 Why practicing works: Explanations of test-enhanced learning 7.2 Developing cognitive skills with practice . . . . . . . . . . . . . . . 7.2.1 Three phases of cognitive skill acquisition . . . . . . . . . . . 7.2.2 Beyond Ericsson’s theory of expertise . . . . . . . . . . . . . . 7.3 Reflective practice and deliberate performance . . . . . . . . . . . 7.4 Enter productive practice . . . . . . . . . . . . . . . . . . . . . . .
150 151 152 153 155 159 159 161 164 166
5.3
III
5.2.1 Reactive mechanisms . . . . . . . . . . . . . . . . . 5.2.2 Internal motivators . . . . . . . . . . . . . . . . . . . 5.2.3 Management processes (Deliberative processes) . . . 5.2.4 Motive generators . . . . . . . . . . . . . . . . . . . 5.2.5 Meta-management . . . . . . . . . . . . . . . . . . . 5.2.6 Interrupt filters and perturbance (tertiary emotions) . 5.2.7 Alarm systems and emotions . . . . . . . . . . . . . 5.2.8 Long-term memory abilities . . . . . . . . . . . . . . 5.2.9 (Short-term) working memory . . . . . . . . . . . . 5.2.10 Long-term working memory . . . . . . . . . . . . . Microcognition: Monitors, parallelism and mental reflexes
Solutions
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
8. Introduction to Part 3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169 9. Learn your way around your R&D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172 9.1 Learn your way around levels of processing . . . . . . . . . . . . . . . . . . . . . 173
CONTENTS
9.2 9.3
Learn your way around your meta-information . . . . . . 9.2.1 Appreciate the meta-access problem . . . . . . . . . 9.2.2 Address the meta-access problem . . . . . . . . . . . Learn your way around your R&D projects and activities 9.3.1 Identify your projects . . . . . . . . . . . . . . . . . 9.3.2 Classify your R&D tasks . . . . . . . . . . . . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
. . . . . .
181 184 184 188 192 194
10. Inspect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 11. Assess . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 About Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 CUPA: Caliber, utility, potency and appeal . . . . . . . . . . . . . 11.3 C: Gauge its caliber . . . . . . . . . . . . . . . . . . . . . . . . . 11.3.1 Rhetorical and rational compellingness . . . . . . . . . . . . 11.3.2 General epistemic criteria . . . . . . . . . . . . . . . . . . . 11.3.3 Assessing explanatory theories . . . . . . . . . . . . . . . . 11.4 U: Gauge its usefulness . . . . . . . . . . . . . . . . . . . . . . . 11.5 P: Gauge its potency . . . . . . . . . . . . . . . . . . . . . . . . . 11.5.1 Potency as the potential for mental development . . . . . . . 11.6 A: Gauge its appeal and analyze your intuitions . . . . . . . . . . 11.7 CUPA: Helpful information . . . . . . . . . . . . . . . . . . . . . 11.8 Other minds: Their recommendations, reviews and commentary .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
201 203 206 207 208 209 211 213 215 217 220 223 224
12. Delve . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1 Effective delving . . . . . . . . . . . . . . . . . 12.2 Annotation concepts and tools . . . . . . . . . 12.3 Tag entire resources . . . . . . . . . . . . . . . 12.4 Tag snips of text and images . . . . . . . . . . 12.5 Write meta-docs . . . . . . . . . . . . . . . . . 12.5.1 An elaborate meta-doc template . . . . . . 12.6 A template for conceptual understanding . . . 12.7 Quickly create and access meta-docs . . . . . . 12.8 Delve ebooks, audio and other media . . . . . 12.8.1 Delve audio on the go . . . . . . . . . . . 12.8.2 Delve e-books . . . . . . . . . . . . . . . 12.8.3 Delve other media on your computer . . . 12.8.4 Productive laziness (cognitive parsimony) 12.9 Computer monitors and other hardware . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
226 226 228 230 235 241 243 249 255 257 257 259 261 262 263
13. Productive practice: A master maker . . . . . . . . . . . . 13.1 Productive practice in a nutshell . . . . . . . . . . . 13.2 An example: Learning the motive generator concept 13.3 Co-opt flashcard software . . . . . . . . . . . . . . . 13.4 Capture and instillerize . . . . . . . . . . . . . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
266 266 269 271 275
CONTENTS
13.5
Design Instillers . . . . . . . . . . . . . . . . . . . . . . . . . 13.5.1 Instiller types and challenge templates . . . . . . . . . . 13.5.2 Grow your understanding . . . . . . . . . . . . . . . . . 13.5.3 Divide and conquer . . . . . . . . . . . . . . . . . . . . 13.5.4 RD cue mnemonic system: From free recall to cued recall 13.5.5 Instiller design rules . . . . . . . . . . . . . . . . . . . . 13.6 Practice with these general considerations in mind . . . . . . 13.6.1 Set your practice time . . . . . . . . . . . . . . . . . . . 13.6.2 Respond to challenges . . . . . . . . . . . . . . . . . . . 13.6.3 Be efficient and effective . . . . . . . . . . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
276 276 279 280 283 286 287 288 289 293
14. Practice productively . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.1 Aim for effectiveness with knowledge: Rationality and transfer . . . . . . 14.2 Grow monitors through review and reflection . . . . . . . . . . . . . . . . 14.3 Master concepts and vocabulary . . . . . . . . . . . . . . . . . . . . . . . 14.3.1 Some basic distinctions . . . . . . . . . . . . . . . . . . . . . . . . . 14.3.2 Structure concept instillers . . . . . . . . . . . . . . . . . . . . . . . 14.3.3 Instill mindware about mindware, for example . . . . . . . . . . . . . 14.3.4 Develop effective (affective) bid monitors . . . . . . . . . . . . . . . 14.4 Master collections of information . . . . . . . . . . . . . . . . . . . . . . . 14.4.1 Apply the RD cue system . . . . . . . . . . . . . . . . . . . . . . . . 14.5 Develop propensities to apply rules: Self-regulate with productive practice 14.5.1 Consider the opposite . . . . . . . . . . . . . . . . . . . . . . . . . . 14.5.2 Andon cord . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14.5.3 Avoid harsh startups with your new mindware . . . . . . . . . . . . 14.6 Develop attitudes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
. . . . . . . . . . . . . . .
295 295 296 298 300 301 303 304 307 310 313 314 316 318 321
IV
Conclusion
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324
15. Meta-effectiveness framework and clinical psychology . . . . . . . . . . . . . . . 15.1 The pertinence of psychotherapy concepts and methods to meta-effectiveness 15.2 The practical relevance of meta-effectiveness to psychotherapy . . . . . . . . 15.3 H-CogAff (mental architecture) and ACT as complementary . . . . . . . . . .
. . . .
. . . .
325 326 327 328
16. Delve and instill the knowledge of your choice . . . . . . . . . . . . . . . . . . . . . . 331 Postscript . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
References
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
List of Figures Figure 1.1 Figure 2.1 Figure 5.1 Figure 5.2 Figure 5.3 Figure 6.1 Figure 6.2 Figure 9.1 Figure 9.2 Figure 9.3 Figure 9.4 Figure 9.5 Figure 9.6 Figure 9.7 Figure 9.8 Figure 12.1 Figure 12.2 Figure 12.3 Figure 12.4 Figure 12.5 Figure 12.6 Figure 12.7 Figure 14.1 Figure 14.2
Processing knowledge for effectiveness Uses of Information H-CogAff (Human Mental Architecture) Internal Motivators Sloman’s depiction of Jerry Fodor’s modular architecture as sunflower-like Developing Motivators Microdevelopment (Beyond Modularity) Levels of Information Processing The Information to Effectiveness Funnel Shallow vs. Deep Processing A Focal Resource and its Meta-Information Sample Areas of Responsibility Example Development Activities as OmniFocus Contexts R&D activities (surf, process, develop) as OmniFocus contexts OmniFocus Task Capture Mavericks Tag Input Window OmniFocus Project View for this Book A Portion of a Meta-doc An Example Empty Meta-doc A Template for Understanding Concepts Schematic of Notational Velocity User Interface Example Transcription in Scrivener Cue Overload RD Cue for Avoiding Harsh-Startups
List of Figures
ii
Copyright (2013-2016). Luc P. Beaudoin Cognitive Productivity: Using Knowledge to Become Profoundly Effective Published by CogZest¹ of British Columbia. Prior to July 10, 2014, this book was published as Cognitive Productivity: The Art and Science of Using Knowledge to Become Profoundly Effective. Release notes for this book (errata of previous revisions) are published on the CogZest web site². If you discover an error in this book please email [email protected] about it. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Mac OS®, iPod®, iPhone®, iPad®, and Applescript®, iBooks® are registered trademarks of Apple® Inc. registered in the U.S. and other countries. App Store is a service mark of Apple Inc. GTD® and Getting Things Done® are registered trademarks of David Allen & Co. OmniFocus®, OmniOutliner®, and OmniGraffle® are trademarks of OmniGroup®. RIM and Blackberry are trademarks of Blackberry Limited. There are other marks not listed in this paragraph but mentioned below that are trademarks of their respective owners. Examples Given that people learn better with examples than without, I use many different types of examples in this book. For example, I use fictional characters in this book to illustrate the cognitive productivity framework. However, any resemblance between these fictional characters and real persons is strictly coincidental. I also refer to a few, diverse well-known, high-caliber concepts and books to illustrate my framework. My descriptions of example theories are terse; however, I encourage readers to consult the original works in order to benefit from them. Rather than merely refer to product categories, I mention specific products. For example, when talking about book holders, I refer to Book Gem®. Disclaimers. This book is not meant to provide legal, medical, psychological or any other type of advice in regulated domains. The Acknowledgments below constitute an extension to the copyright page.
¹http://cogzest.com ²http://cogzest.com/books/release-notes-for-cognitive-productivity-book-by-luc-p-beaudoin/
Preface We’ve all had this experience: We’ve read a factual or practical book that had the potential to make us more effective in some specific respects. Several months later, however, we can hardly remember the content. Worse, years go by and we have yet to apply the gems of knowledge it contains. There is no simple solution to this problem; but there are things we can do to address it. I have written this book primarily for self-directed learners and those who study them. My objective is to help effective people systematically use knowledge and technology to become increasingly effective. This is the problem of “knowledge potentiation”: How to release the potential of knowledge in ourselves. This book addresses this problem by leveraging the most progressive attempt humanity has made to understand the human mind: cognitive science, broadly speaking. Broad cognitive science is not restricted to the narrow, classical concept of “dry” information processing. It also seeks to explain affective information processing: motivation, emotions, moods and attitudes. To prepare you for this book, let me briefly recapitulate the journey of which it is a part. While taking a high school course on history, I “discovered” a simple yet potent algorithm to master a body of knowledge: 1. Review the materials to ensure full comprehension. 2. Formulate a collection of questions that can only be adequately answered by someone who sufficiently understands the matter. 3. Practice answering these questions, at spaced intervals until and beyond the point of manifest mastery. Of course, the system worked like a charm. It helped me gain a deep understanding of all kinds of academic problem spaces. I aced papers and exams. It helped me to earn more scholarships and Ph.D. placement offers than I could accept. I got to study in one of the finest cognitive science programs with my top choice of a Ph.D. thesis supervisor, Prof. Aaron Sloman. Cognitive psychology had decades earlier formally discovered some key data and principles that lend credence to my “algorithm”. Some of the keywords to that literature are test-enhanced learning, test-effect, distributed recall practice, self-regulated learning and deliberate practice. I contributed to the “Cognition and Affect Project” at the universities of Sussex and Birmingham. We developed a deep, new theory of how minds process goals, motives and emotional states.³ This has informed my understanding of all aspects of psychology and this book. Like that of many other knowledge workers, my career has required that I develop expertise in a wide variety of areas. I have been an Assistant Professor of Military Psychology and Leadership, a semiconductor technical writer (Tundra Semiconductor), an element-management software developer and team lead (Abatis Systems),⁴ a project manager, an Adjunct Professor of ³Beaudoin & Sloman (1993), Beaudoin (1994). See Hawes (2011) for a review of this theory. ⁴Tundra Semiconductor Corp. and Abatis Systems Corp. were both co-founded by Sir Terry Matthews, Newbridge Networks Corp. and their
employees. I was an employee of Tundra Semiconductor and Abatis Systems at the founding of these companies.
Preface
iv
Education and the founder of two businesses that apply cognitive science, CogZest⁵, which provides publications and services, and CogSci Apps Corp., which develops software. Each one of my roles has called upon me to rapidly transform myself with knowledge and technology. In each role, I tried to understand how what I knew could help me to learn more effectively. As I began to rely mainly on electronic documents, it struck me that the potential for technology to support learning with cognitive science was scarcely exploited. I had written, in 1991, a little Smalltalk program to help me acquire technical concepts and (being French Canadian) augment my English vocabulary, using principles of test-enhanced learning (as described in chapter 7). Web browsers and PDF readers later made it possible to read about most of what I needed to learn. Then came software to listen to podcasts and audiobooks, read ebooks, participate in conferences, and more. Yet, none of these applications included support for test-enhanced learning! Annotation mechanisms were (and still are) very rudimentary and fragmentary. It also struck me that whereas public performance experts engage in deliberate practice, knowledge workers seem to ignore deliberate practice and many other cognitive potentiators. Cognitive science was, and still is, not sufficiently exploited. So, after my second exhilarating (and successful) experience in high-tech startups, I decided to tackle, head on, the problems we knowledge workers face in learning with technology. I contributed my prior analyses to Phil Winne’s Learning Kit and nStudy projects, and worked with him from 2002 to 2009. We built a couple of general-purpose learning platforms and learned a great deal. In 2010, I struck out on my own again, founding CogZest and becoming Adjunct Professor at Simon Fraser University. I continued to focus on the cognitive productivity problems addressed in this book. In January 2010, just before Apple’s much anticipated tablet was announced, I wrote a blog article for SharpBrains⁶ detailing the cognitive productivity requirements I felt it should address.⁷ When the iPad was announced it received mixed reviews; but I was truly impressed! I could see its potential to improve cognitive productivity. So, I wrote another blog post for SharpBrains⁸, briefly assessing the iPad and pointing out ways in which it could be improved to further augment cognitive productivity. Wanting to put a “dent in the universe”, as Steve Jobs used to say, I emailed Jobs to congratulate him on Apple’s most recent innovation and suggest ways in which Apple could better support cognitive productivity. I offered Apple a white paper on the subject; Steve Jobs asked me to send him one. This book expands considerably on the 30+ page document I sent to Steve Jobs in February 2010. While I am still not satisfied with today’s technology, we must use the tools we have at our disposal. This book is meant to help self-directed learners do that. • Part 1 describes the problems and opportunities we face when trying to use knowledge to become more effective people. I refer to the ability and propensity to use knowledge for this purpose as “meta-effectiveness”. Meta-effectiveness is one of the most significant contributors ⁵http://cogzest.com/ ⁶http://cogzest.com/2011/10/steve-jobs-and-the-topic-of-cognitive-productivity/ ⁷On the CogZest web site, I’ve collated the two posts and some notes about them. ⁸http://cogzest.com/2011/10/steve-jobs-and-the-topic-of-cognitive-productivity/
Preface
v
to personal success and happiness. It is the key aspect of cognitive productivity with which this book is concerned. • Part 2 describes cognitive science that is pertinent to addressing meta-effectiveness. If you are only interested in applications, skip this part of the book and go straight to Part 3. In order to benefit from what we read (and other information we process), we need to learn to see the world in new ways with knowledge and to respond with the right motivation and emotions. Self-directed learning involves mental development. We develop “monitors” (internal and external perceptual mechanisms), “motive generators” (mechanisms to generate new evaluations, goals, wishes, wants and desires) and other mental mechanisms. If I am successful, then by delving this chapter, you will think of your mind and the learning you do in a new, more powerful way. • Part 3 provides concepts and guidance for using knowledge to become more effective. This framework will, I hope, help you – “know your way around” information and your information processing tasks, – systematically evaluate knowledge resources (ebooks, podcasts, videos, etc.), – delve knowledge resources, and – practice with knowledge gems in order to perceive, understand and respond to the world with the knowledge you acquire. I believe that self-directed learning from high-caliber, potent information requires more effort than people typically realize. This leads me, in the conclusion of this book, to elaborate on an important overlap between self-directed learning, education, self-help and clinical psychology. I hope this book helps us to discharge our privilege and duty—to further understand and improve the most sophisticated power in the world, the human mind.
Acknowledgements I have many people to thank for Cognitive Productivity. I am grateful to my peers who kindly reviewed parts of this book: Sharon Bratt (MacEwan University), Eva Hudlicka (University of Massachusetts-Amherst), Jeffrey Karpicke (Purdue University), Mary Pyc (Washington University in St-Louis) and Aaron Sloman (University of Birmingham). Christopher Stone (Harvey Mudd College) also provided helpful feedback. Students, friends and family also reviewed parts of this book: Mark Beaty, Alissa Ehrenkranz, Judy Garner, Sheryl Guloy, Marilyn Medén, Jeannine Malo, Jeanie Morton, Heather Morton, Christine Pan, Al Sather and Carol Woodworth. Claude Lamontagne, Phil Winne, Steve Leach, and Peter Brems provided feedback on ideas presented here. I appreciate the feedback I have received from readers of this book, clients, participants in my workshops, and lecture and seminar attendees. I’ve been encouraged by knowledge workers across disciplines who agreed with me that cognitive productivity is critical to the modern pursuit of excellence. Thanks to Lisa N. Eisen for deep insights into some of the psychological principles alluded to in chapter 15. Damien Elmes kindly reviewed the text in chapters 13 and 14 about his deliberate practice application, Anki⁹. Lam Wong¹⁰ created the fabulous front and back covers. He also convinced me to apply the principle of parsimony by removing “The science and art of” from the subtitle of this book. Carrie Spencer of Royal Rhodes University was instrumental to me choosing to develop the eponymous concept of this book, cognitive productivity, in 2010. Thanks to James Cullin¹¹ for carefully reviewing all the citations and correcting the bibliography. Thanks also to Brian Holmes of GradeAEdits, for proofreading this book. Several ideas in Part 3 have their roots in R&D projects led by Phil Winne at Simon Fraser University, where I was research associate and software development leader. We developed the StatStudy, gStudy and nStudy applications to understand and address learners’ cognitive requirements. Some of the ideas presented in this book have also been implemented in software by my colleagues at CogZest and CogSci Apps Corp.¹² (of British Columbia). I’m grateful to all the contributors to these projects. The Leanpub team¹³ has reliably provided an amazing platform to evolve this book according to lean principles. I greatly appreciate the support of Simon Fraser University, where I am Adjunct Professor of Cognitive Science and of Education. ⁹http://ankisrs.net ¹⁰http://www.lamwong.com ¹¹https://www.linkedin.com/profile/view?id=182612691 ¹²http://CogSciApps.com ¹³http://leanpub.com
Acknowledgements
vii
Some of the theoretical roots of this book are in my Ph.D. research, which was part of the Cognition and Affect Project at Sussex University and the University of Birmingham¹⁴ in England. Hence my gratitude to all contributors to that project. I am also grateful to my external Ph.D. thesis examiner, Prof. Margaret Boden¹⁵, for encouraging me to publish my thesis research in the form of a book. Thanks also to Prof. Aaron Sloman¹⁶ for encouraging me, in 2008, to resume research on the intersection of cognition and affect. This book contains extensions and applications of our “H-CogAff” framework and the perturbance theory of emotion. Thank you to Ian Hand (Managing Director, VentureLabs), John Siu (Engineering Director, In Motion Technology), Paul Terry (Entrepreneur in Residence at SFU Venture Labs) and Renwei Li (Senior Director, Software Engineering at Huawei) for permission to refer to them personally in this book. Interactions with them helped me shape this book. Carol Woodworth was a sounding board, editor and companion throughout this project. I am deeply grateful to countless authors who shaped my thinking, whether or not I have cited them in this book.
¹⁴http://www.cs.bham.ac.uk/research/projects/cogaff/ ¹⁵http://en.wikipedia.org/wiki/Margaret_Boden ¹⁶http://en.wikipedia.org/wiki/Aaron_Sloman
I Challenges and opportunities Only the ideas that we actually live are of any value. Hermann Hesse
1. Introduction The essence of knowledge is, having it, to apply it; not having it, to confess your ignorance. Confucius We live in an era of ineffable opportunities to use knowledge to become more effective. The information cornucopia is at our finger tips. We are served the latest knowledge in print, ebooks, audiobooks, web pages, podcasts, videos, screen casts, webinars, and other forms. For example, books by relationship expert Dr. John Gottman can improve your marriage and other close relationships. The principles of rationality conveyed by cognitive scientists like Dr. Keith Stanovich can help you avoid costly mistakes. Agile product-development principles conveyed by the likes of Eric Ries can help you develop products customers will actually like and pay for. High caliber investment advice from writers like TSI Network’s Pat McKeough can protect and grow your investments. Applying health and nutrition information from Center for Science in the Public Interest¹’s Nutrition Action² newsletter might help you live a healthier and longer life. The openaccess movement provides public access to information hitherto only available to select knowledge workers. Many universities are now even offering massive, open (free) online courses—MOOCs! To be sure, there is more irrelevant information than text worth reading, let alone delving. But there is no denying the abundance of potent knowledge to help us solve problems and develop ourselves. This bodes well for the exercise of the seventh habit of highly effective people, which— according to the late Stephen R. Covey—is to “sharpen the saw”®. It is to improve ourselves—our productive capacity—through regular reading and related pro-active activity (Covey, 2004). If we properly conduct our research and apply ourselves, then we can develop personal effectiveness: understanding, skills, attitudes, habits and dispositions. I agree with Aristotle, who laid the foundations for Western ethics, that in the balanced pursuit of excellence lies the route to happiness. Alas “the shallows”, intellectual defeatism, naive optimism and cognitive miserliness each in their own way threaten our knowledge-based and technology-enabled pursuit of effectiveness. In his best selling book, The Shallows, Nicolas Carr laments the effects he supposes the Internet has on our brains, minds and behavior. He suggests that our usage of information technology causes us to have shorter “attention spans” and more difficulty learning. He claims that the distractions, hyperlinks and other features of technology (and our way of using it) not only interfere with our productive use of technology, they alter our brains and minds. “The tools of the mind amplify and in turn numb the most intimate, the most human, of our natural capacities—those for reason, perception, memory, emotion.” From the neuroplasticity bandwagon, Carr professes that our new technological vices “rewire our brains”. We are, he seems to believe, becoming inextricably stuck in the shallows. However, Carr’s apparent defeatism overshadow his legitimate concerns. Let us “consider the opposite”, a reasoning strategy discussed below. If plasticity (i.e., modifiability) is as important a ¹http://www.cspinet.org ²http://www.nutritionaction.com
Introduction
3
characteristic of the brain as Carr believes, then an opposite conclusion might just as well be right: We can “mold” our brains to become more focused and productive by habitually using the Internet in focused, productive ways. Carr alludes to this possibility, but he shuns it. “It’s possible to think deeply while surfing the Net, just as it’s possible to think shallowly while reading a book, but that’s not the type of thinking the technology encourages and rewards.” One might as well say that automobiles encourage us to speed and so we should stay out of them. In the spirit of the people to whose memory this book is dedicated, I reject cognitive defeatism in favour of informed, productive ways of using technology to improve ourselves. Carr’s book is part of a trend amongst popular writers to try to describe, explain and predict psychological processes using neuroscience. Of course, understanding the brain is ultimately essential to understanding the mind. Alas, it is very difficult even for neuroscientists to make detailed sense of human behavior in neurological terms. Neuroscientist Seth Grant defines systems biology as “a new branch of biology aimed at understanding biological complexity” (2003). Grant has identified eight interacting layers in the system to consider. The bottom layer is genetics and the top layer is behaviour. Synaptic connectivity is just one of the components of systems biology. Synapses themselves are now considered as complex computers (Grant, 2007). We can expect learning to happen at multiple layers and not to be faithfully approximated by any “hard wiring”. The mind itself must be considered as having multiple layers capable of learning. Between the brain and behavior there are complex virtual machines—“the mind”.³ Mapping mental phenomena to brain mechanisms is a challenging task for scientists. As Stephen Pinker put it “Psychology, the analysis of mental software, will have to burrow a considerable way into the mountain before meeting the neurobiologists tunneling through from the other side.” (Pinker, 1999) Many popular “brain-based” claims originated in psychology—whether it be folk or scientific psychology. They mainly concern psychological matters. For example, many of the principles in John Medina’s popular Brain Rules book, such as the importance of repetition, are mainly psychological matters. The neuroscience of distributed practice effects has a long way to go — as does its cognitive science. These matters usually need to be assessed, if at all, with the rigorous research methods of empirical psychology. We need to be as careful when we draw inferences from neuroscience as other sciences; however, the luster of neuroscience can be particularly distracting.⁴ In particular, I reject the notion that the Internet is “rewiring our brains”. As Pinker put it: Critics of new media sometimes use science itself to press their case, citing research that shows how “experience can change the brain.” But cognitive neuroscientists roll their eyes at such talk. Yes, every time we learn a fact or skill the wiring of the brain changes; it’s not as if the information is stored in the pancreas. But the existence of ³Thus, multi-scale modeling of the brain must include virtual machines. See Sloman (2009a) for a description of the mind as a layered virtual machine that is itself layered on top of physical machines (themselves layered). The concept of layering is well understood in telecommunications (the Internet Protocol being one of several examples http://en.wikipedia.org/wiki/Internet_protocol_suite ) and computer software. However, it is still rarely explicitly invoked in relation to the mind. Yet to think in terms of “wiring” obscures the many layers at which learning may flexibly occur. Compare also Section 8-4 of Minsky (2006). ⁴While neuroscience is an important contributor to cognitive science, too many people are duped into thinking we understand more about mindbrain interactions than we do. One of the difficulties with neuroscience is statistical power, linked to low sample sizes (Button et al., 2013.) There are also problems with frequent non-blind studies. Button et al. lament the lack of reproducibility in swaths of neuroscience. See also Stix (2013) on the subject. Satel & Lilienfeld (2013) warn their readers about the seductive appeal of mindless neuroscience,, particularly given the psychological (if not rational) compellingness of neuroimaging. See also the discussion of “neuromania” in Changeux & McGinn (2013). Epistemic exuberance needs to be bridled by skeptical thinking (compare chapter 11, “Assess”.)
Introduction
4
neural plasticity does not mean the brain is a blob of clay pounded into shape by experience. (Pinker, 2010) Cognitive neuroscience is a difficult discipline. It is an important member of and contributor to cognitive science, one that is indirectly represented in this book. Carr is right to call our attention to the shallow use of technology and information. We face real challenges to our cognitive productivity⁵; many of them predate the web. To have studied cognitive psychology is to know that our perception, understanding, attention, ability to recall and utilize information, indeed all mental functioning is biased, limited and error prone. Furthermore, ostensibly learning something in one domain or context is no guarantee of being able or disposed to apply it when one should in another. For example, a person who aced mechanical physics may fail to realize (or value) that she is not keeping a safe distance from the car ahead of her. Likewise, we may read the work of Gottman, Stanovich and Ries, which I describe below, and yet still be blind to too many of our partners’ bids, make too many biased decisions and be insufficiently agile. Psychologists refer to these issues as problems of “inert knowledge”⁶ and “transfer”. They have been studying them at least since 1901 (Haskell, 2000). In chapter 3, I describe our cognitive productivity challenges so that we may remedy them with the rest of the book. The Internet is not the root cause of human information-processing fallibility. Nor are our limitations a fluke of evolution discovered by attentive empirical psychologists.⁷ I do not believe natural selection (or any intelligent mechanism) could evolve a machine that meets the awesome requirements of the human mind without this resulting machine having severe challenges to cognitive productivity described above. Design, human or Darwinian, is a matter of trade-offs.⁸ But we, intelligent machines, can nevertheless improve. The majority of people who read this book, I assume, are knowledge workers. Knowledge workers are people who spend a significant portion of their lives understanding, assessing, modifying, building and using knowledge. They solve problems with knowledge and often create and share knowledge in so doing. One can be a golfer without earning one’s living as a golfer. One can be a knowledge worker without being a scientist. Explicit knowledge-intense work need not occupy all of one’s time for one to be considered a knowledge worker. A surgeon may spend most of his time delivering services and administering his business. But the portion of time he spends acquiring and building knowledge provides significant value. A lawyer creates and processes knowledge as argument in service to her clients. An effective trades person reads about his profession, communicates with colleagues about it, and develops and shares new techniques and strategies. All these people are knowledge workers. The staggering abundance of knowledge has increased expectations for many of us to exploit knowledge to develop our own effectiveness, more effective products, and better solutions for our clients. In order to meet this challenge, one needs a propensity to develop effectiveness and consequently competence. This is something that the late psychologist of Harvard University, Robert ⁵Chapter 3 discusses the obstacles we face. ⁶Below, I describe a new way of thinking about the so called “transfer” problems and how to address them. ⁷Empirical psychologists are research psychologists who attempt to resolve psychological problems of understanding by collecting, analyzing
and interpreting data in studies involving real animals, whether human or not. ⁸Dawkins (1996), Sloman, (1978).
Introduction
5
White, referred to as “effectance” (White, 1959). Effectance plays a large role in determining which of two people of equal intelligence will be more effective. It pushes people to develop expertise to overcome limitations in fluid intelligence.⁹ Effectance drives one to develop thinking dispositions and skills to become increasingly effective. White articulated his concept of effectance in relation to children and before our transition to a knowledge society. In this book, I improve his critical but largely overlooked concept¹⁰. But even effectant people may be unsettled by the pressures to tame an exponentially expanding knowledge base. Faced with the cognitive demands of the knowledge economy, they often turn to productivity systems and software. Ironically, these categories of solutions are themselves expanding so fast productivity experts are finding them hard to track. That expanse, however, is not the major obstacle between effectant people and the effectiveness they pursue. Unfortunately, productivity systems, like David Allen’s popular Getting Things Done® (GTD®), and productivity software, are not for the most part designed to meet the specific requirements of cognitive productivity. A cognitive productivity solution is one that addresses the constitutive problems of knowledge work: to understand, assess, modify, create and apply knowledge. While I believe the GTD system contains useful general productivity concepts, it clearly was not designed specifically for knowledge-intense work. For example, Allen’s seminal book contains examples of managing grocery lists and cleaning one’s garage. GTD is supposed to free its user’s mind for cognitive work, but it has little to say about the particularities of mental processes or cognitive work. In contrast, the framework I develop in this book is specifically targeted at cognitive productivity challenges: to exploit knowledge to productively develop products, solutions, and oneself. Steve Jobs said of Apple, “We believe that it’s technology married with the humanities that yields us the result that makes our heart sing” (Isaacson, 2011). As I suggested in a white paper and email exchange with him in 2010¹¹, cognitive science—the interdisciplinary, information processing study of mind—also needs to be included in the intersection. Medicine is informed by biology. Mechanical engineering by physics. Likewise, we cannot adequately address difficult problems of cognitive productivity without exploiting the results of cognitive science. Consider an example of how we suffer as a result of such neglect. Today, we read documents in web browsers, ebook readers and other applications that in many respects are worse than paper. For example, no operating system yet provides a uniform way for users to annotate text across diverse applications—such as email, PDFs, and web pages. Their designers do not seem to consider basic principles of cognitive science. I will describe these problems in chapter 3 and show how to work around them in Part 3. The opposite of Carr’s intellectual-technical defeatism, a macho attitude towards learning, is no better. The implicit idea here is that everyone who has proven their intellectual capabilities at ⁹Fluid intelligence is the ability to solve novel problems using general purpose reasoning without depending on specialized knowledge. Crystallized intelligence is composed of our abilities to use what we have learned (skills, factual knowledge, etc.). However high one’s fluid intelligence, it is necessarily limited, and it tends to decrease in adulthood. ¹⁰See in particular the Section on Effectance, below. My extensions are based on Bereiter & Scardamalia (1993), Sloman (2009b) and Stanovich (2009). For example, White focused on the implicit motivation for competence. He did not explore other targets of effectiveness (developing better products, solutions and self). He restricted his analyses to children. He did not explore the creation and use of objective knowledge for effectance. He could not explore the architectural, information-processing bases of motivation. He did not frame effectance as a propensity, a key concept in this book. ¹¹Steve Jobs, like Winston Churchill, extended himself by asking other people to help him accomplish his goals and he said no to fear. He kindly repaid the favor by responding to emails from people he did not know, such as myself. Cf. McBurney (2013).
Introduction
6
university or work knows how to read, and more generally process knowledge resources, in such a way that they can derive the benefits they seek. Provided the information is well presented, they will understand it after processing it once or twice. Thereafter, they will be able to use it. They need not systematically and effortfully apply themselves to master knowledge gems. Their skills, understanding, attitudes, propensities, habits, etc. will follow from their own unaided abilities to learn. To be sure, students and public performers (musicians, athletes) may need to practice and rehearse. But competent professionals do not. Such are the beliefs floating in the bubble atop my cartoon of the intellectual macho. Alas, cognitive nonchalance flies in the face of cognitive science. I suspect that such cavalier attitudes, and the superficial strategies they entail, are the main causes of what Carr described as “the shallows”. However, contra defeatism, the “shallows”, where they exist, are correctible. Productive strategies can be learned. How? We cannot solely rely on motivational books or productivity systems. Cognitive science provides relevant material for our problems, though scholarly books are either too technical or general to satisfy the needs addressed by this book. Several recent popular books have drawn attention to the cognitive science of expertise¹². This primes my reader to the importance of effortful practice. These books, however, do not deal with specific problems of exploiting knowledge for enhanced effectiveness. Expertise is an important technical concept in cognitive psychology that is related to, but different from the fundamental concept of effectiveness. Nor are the abundant study-strategy books aimed at college students particularly relevant to knowledge workers. There is a need for a coherent, cognitive-science based framework specifically to help selfdirected learners exploit knowledge and technology to improve their effectiveness. The novelty of this quest partly explains why I have had to coin several terms, utilize several others that have yet to reach their memetic potential, and develop new concepts. I have in this introduction referred to a critical quality of people who pursue excellence with knowledge. Like the concept of acceleration in physics, this concept is a second-order (derivative) one. I call it meta-effectiveness: abilities and dispositions to use knowledge to become more effective. To a first approximation, meta-effectiveness is simply what it takes to be an effective lifelong learner. Naming, characterizing and applying this concept may help people become more meta-effective. If we are to draw deeply from the cornucopia of knowledge and be transformed by it, if we are to systematically develop effectiveness from knowledge rather than merely become vaguely familiar with information, then we need a meta-effectiveness framework—one that is informed by cognitive science and that, in turn, informs it. One that is designed to meet the requirements of effectant people in the Knowledge Age. It must eschew defeatism and machismo in favor of effectance. Those are the objectives of the framework I have set out to describe in this book; they are the standards by which I would like this book to be judged.
1.1 Broad cognitive science There are no subject matters; no branches of learning–or, rather, of inquiry: there are only problems, and the urge to solve them. ¹²For example Gladwell (2008), Coyle (2009), Foer (2011).
Introduction
7
Karl Popper It now behooves me to explain what I mean by saying that “our cognitive productivity tools need to be informed by cognitive science”. The expression “cognitive science” is an umbrella term that denotes the study and body of knowledge—concepts, theories, findings, principles, and so on—about the mind and mental (i.e., information) processing. It has since the 1970s been the dominant, and most progressive, scientific framework for posing and answering questions about mental processes and intelligent systems. A definition of cognitive science from Wikipedia is in this instance as useful as what you will find in scholarly texts: Cognitive science is the interdisciplinary scientific study of the mind and its processes. It examines what cognition is, what it does and how it works. It includes research on intelligence and behavior, especially focusing on how information is represented, processed, and transformed (in faculties such as perception, language, memory, reasoning, and emotion) within nervous systems (human or other animal) and machines (e.g. computers). Cognitive science consists of multiple research disciplines, including psychology, artificial intelligence, philosophy, neuroscience, linguistics, and anthropology. (Thagard, 2007). It spans many levels of analysis, from low-level learning and decision mechanisms to high-level logic and planning; from neural circuitry to modular brain organization. The fundamental concept of cognitive science is “that thinking can best be understood in terms of representational structures in the mind and computational procedures that operate on those structures.” (2012) Cognitive science is inclusive and interdisciplinary. We consider as pertinent to our endeavors data and ideas that pertain to our problems, whether or not they emanate from researchers who think of themselves as cognitive scientists. (Two eminent examples of the latter whose work I capitalize on in this book are philosopher of knowledge, Imre Lakatos and psychologist, Albert Bandura.) More than many other sciences, cognitive science is an eclectic collection of ways of thinking, of posing problems and questions and addressing them.¹³ To merit the name cognitive scientist, one should appreciate and be able to adopt the various perspectives.¹⁴ One should work with the critical assumption that the mind is an information processing system (also known as a “computational system”).¹⁵ One need not be a professional cognitive scientist to think like one. This book will attempt to demonstrate the learnability and value of such thinking. Unfortunately, the expression “cognitive science” has lead some to a lopsided concern with classical cognitive processes (such as perceiving, reasoning, storing information, remembering and problem solving). Conversely, some students of affect seem to fail to realize that affect is itself ¹³Sloman (1993a) sketches three approaches amongst others. A good way to get a sense of cognitive science is to peruse proceedings of the Cognitive Science Society and the AAAI Digital Library. ¹⁴The singular is preferred to the occasionally used “cognitive sciences” because it highlights the fact that it involves many different disciplines. (Cf. Boden, 2006). In my terminology, one can contribute to cognitive science without being a cognitive scientist. A cognitive scientist has a good understanding of computer modelling of mental processes and is able to think like (and understand the work of) the members of various disciplines in cognitive science, including Artificial Intelligence. Artificial Intelligence is the core discipline of cognitive science (Sloman, 1993a). ¹⁵As Pinker put it “the mind is a system of organs of computation designed by natural selection to solve the problems faced by our evolutionary ancestors in their foraging way of life.” (Pinker, 1999). See also Simon & Kaplan (1989) and Sloman (1978).
Introduction
8
information processing. Margaret Boden, the pre-eminent historian of cognitive science documented that the original appellation was not meant to exclude the study of motivation and emotion. It will be made clear through this book that rigid distinctions between cognition (knowing, thinking, etc.), conation (motivation and volition) and affect (value-laden states) must be abandoned. The members of this triad are intertwined and sometimes inseparable aspects of processing information. Margaret Boden’s first book and subsequent work addressed all kinds of mental phenomena (e.g., emotions, motivation and cognition.) I view cognition, emotion and motivation as blended rather than merely intertwined or interacting.¹⁶ To remind my readers, I therefore sometimes write in terms of broad cognitive science.¹⁷ To designate more traditional cognition, I refer to classical cognitive science or dry cognition. Cognition to me is central information-processing. In any event, do not let terminology distract you. As indicated by the opening quotation, what matters is understanding the problem and using an information-processing framework to address it. By addressing the core problems of this book with a cognitive-science mindset, we may make substantial progress towards resolving them. However, this is not to say that we can mechanically derive practical applications from cognitive science. As Donald Schön persuasively argued in The Reflective Practitioner, applying science is not a simple process of drawing logical implications from theory. Practice is unfortunately messy. Thus, I have had to frame problems effectant people face. I’ve had to invent new concepts, principles and workflows. These inventions, in turn, need to be adapted by the reflective practitioner, if they are to be used at all. Cognitive science is no panacea. Some of the problems we face in understanding learning are partly due to false starts in cognitive science. For example, the term transfer, used in educational psychology to denote cases in which learners fail to apply what they have learnt, is problematic as we shall see. Reductionist fervor induces some to accept the myth that empirical evidence from neuroscience will arbitrate disputes about how to define terms such as “consciousness”, “emotion” and “intention”. Further, the cognitive revolution in psychology is still not complete: behaviorism¹⁸ shows up in various forms. For example, compare the emphasis in the expertise literature on performance as opposed to objective knowledge building, and its scant information processing models. However, broad cognitive science offers the best set of tools we have. For productivity frameworks to improve, they need to be subjected to empirical tests and conceptual criticism by neutral, professional researchers.¹⁹ Unfortunately, productivity frameworks for knowledge workers have not been scientifically driven. For example, far as I have been able to gather, the book Getting Things Done has not been systematically assessed by academic researchers. It does not have a bibliography. My main concern with Getting Things Done is its relation to criticism, the mechanism by which science functions and knowledge is developed. I have not found the framework to be permeated with the hallmarks of science, such as: communicating in terms ¹⁶The classical distinction between cognition, emotion, and desire dates back to Plato’s The Republic. Still & Dryden (1999) explain how these classical distinctions are relevant in a modern psychological context: rational emotive behavior therapy. My view is that cognitive, affective and motivational processes are not merely interactive, they are blended information processing. To speak of affective processes is to emphasize evaluation. To speak of motivation is to emphasize the processing of motivators, which generally arise from and involve further evaluations. Compare Sloman (1991). ¹⁷The label affective cognitive science is also often used (even by myself). However, strictly speaking, the classical notion of affect excludes “volition” (a subset of executive functions), so the term “broad cognitive science” is more inclusive than “affective cognitive science”. ¹⁸Classical behaviorism is a school of thought according to which psychology should base its theories only observable phenomena (the organism’s behavior and its environment.) See Boden (2006) for a detailed history of cognitive science. ¹⁹One would expect some of this to research to be conducted by organizational psychologists and cognitive scientists.
Introduction
9
of theories, hypotheses, and tests; an abundance of caveats; attempts at refutation; references to empirical research; suggestions for future research;²⁰ or progressive reworking of the theory to deal with problematic findings.²¹ Nearly eight years after publishing Getting Things Done, Allen published another book in which, he boasted, he didn’t change any of the information in the earlier book (2008, p.19). Alas, in science, no change means no progress. Productivity frameworks aimed at knowledge workers should, in contrast, meet the epistemic criteria listed above.²² Are there recent examples of that framework itself being significantly altered by evidence or conceptual criticism? Allen writes “Making It All Work is not intended to supplant or change any of the information in Getting Things Done” (Allen, 2008). He is quoted as saying “nobody’s got the answers but us” and “I’ve proven it works” (Keegan, 2007). Yet we must in assessing and deciding upon frameworks ensure that they can evolve.²³ Despite GTD being pitched as having been “nailed”, there are accounts of people needing to modify the framework. The framework has many good things going for it, particularly when combined with good software, such as OmniGroup’s OmniFocus. It needs to be assessed empirically by researchers. In developing the meta-effectiveness framework, I have taken a different approach to productivity. This framework focuses on deriving effectiveness from knowledge resources. It exploits cognitive science and is a contribution to cognitive science. As such, it embraces the tentative, conjectural nature of knowledge while placing its faith in the processes of science. The meta-effectiveness framework is not an end-point. It is improvable. It encourages people to propose and experiment with ways of building and applying knowledge. Much of its value comes from its (a) specification of the problem of meta-effectiveness; and (b) presentation of concepts, findings and principles from cognitive science. Other differences will become apparent to the reader. The reader who is merely concerned with applications can skip to Part 3 of the book. While my proposals are informed by cognitive science, they also need rigorous testing. For example, the vast majority of empirical work in cognitive science is conducted on students. On the one hand, some of the findings I rely upon may not generalize. On the other hand, many of the principles I leverage are arguably universal.²⁴ Most of the research on expertise deals with readily observable performance. In contrast, I have written this book for and about effectant people to help them exploit knowledge for improved personal effectiveness. I have attempted to systematically analyze and address the distinctive reality of knowledge. It will be difficult to test my proposals. Knowledge workers are busy people, technology changes very fast, and the benefits I aim to provide are long-term and hard to measure.²⁵ However, there is an ²⁰In relation to his book, Allen reportedly said “Nobody’s going to write another one of those. I nailed it.” (Keegan, 2007). ²¹Lakatos (1980) systematically describes both what makes for a progressive research program and what distinguishes science from faith-based
frameworks. If a framework is not part of a research program, it will fail and be superseded by one that is; if it is not progressive, it will also be superseded. For tips on how to think about psychology and psychological research, see (Stanovich, 2010). See Sloman (1993a) for a brief overview of the approaches to cognitive science. ²²Criteria for assessing knowledge resources are discussed in chapter 11. ²³The concept of “knowledge shields”, proposed by Paul Feltovich, is a pertinent admonishment against being too quick to reject evidence that contradicts our beliefs (Hoffman, 2005). Compare the concept of evaluation-disabling properties of mindware described by Stanovich (2009). See Lakatos (1980) and Popper (1959, 1983) on the role of criticism. Compare, also, McCulloch’s (1953) criticism of psychoanalysis. ²⁴See chapter 7 of Stanovich (2010) for a response to the “artificiality” and “sophomore population” criticisms of psychology. ²⁵Hoffman, Marx, Amin, & McDermott (2010) wrote about the “moving target problem” and related issues. For example, the “the time frame for effective programmatic experimentation on hypotheses about cognitive work cannot match the pace of change in the work and the technology of sociotechnical systems.” (p. 562).
Introduction
10
exciting possibility. It is to build cognitive productivity software that collects data from consenting users and can easily be modified by researchers. This would allow R&D teams—consisting of cognitive scientists, software developers and others–to gather and analyze data on hundreds of thousands of users. They could perform split-tests of theories (Christian, 2012). They could gradually improve, in tandem, their theories and their software. Phil Winne and I developed such tools together for many years at Simon Fraser University.²⁶ This type of tool will allow cognitive scientists to investigate and enhance meta-effectiveness like never before.
1.2 Updating how we think about knowledge and ourselves [Confucius] was asked what he would do first if he were given power. He replied that he would rectify the names. In other words, he would work on the way society talks about itself to ensure that the terms and concepts were accurate. “If the names aren’t right, what you say will sound unreasonable. If what you say is unreasonable, what you try to do will fail.” John Ralston Saul And so we’re quite careful with the words we use, because those can determine the path that you go down. Jony Ives (quoted by T. Davenport) We have only recently transitioned to being a knowledge society. As a result, the English language still lacks important cognitive terms and idioms. Even many knowledge workers lack explicit knowledge of the foundational concepts of knowledge work. In a sense, they lack explicit knowledge of what they are implicitly doing with knowledge. Yet many cognitive psychologists believe that developing expertise involves developing explicit knowledge of things we know tacitly. I believe that resolving conceptual issues about cognitive work can help one improve one’s effectiveness. So, in this book, drawing upon and extending cognitive science, I attempt to provide a coherent, powerful framework for thinking about one’s knowledge work, with a particular focus on one’s research and development. The major activities to which I draw your attention in this book are the following: • • • •
Processing knowledge resources Developing products Developing custom-solutions to problems Developing ourselves
²⁶See Winne (2006, 2012). Ries’ concept of innovation accounting and the concept of split-testing are germane to this. Several companies routinely collect data about application usage. Winne and I are interested not only in improving technology but extending cognitive science, which in turn would improve technology.
Introduction
11
Take the concept of processing knowledge resources. English provides the verb “to read” which takes a document as an object. One reads a web page or a book. However, one attends a lecture or seminar. One listens to a podcast. One watches an educational video. One participates in or attends a workshop, meeting or course. There is no received expression for processing information or knowledge resources that does not commit the speaker to a particular type of resource. Many people speak in terms of “consuming” information. Alas, consumption is an inadequate metaphor. For example, to say that one consumes something implies that when one is done, the thing no longer exists. (Compare: “The house was consumed by fire” and “the meal was eaten.”) Moreover, digestion in contrast with information processing, is a passive, autonomous, process. Clay Johnson demonstrated the inadequacy of the nutrition metaphor by exhausting it in the first few chapters of The Information Diet.²⁷ Steve Whittaker proposes that “curation” is a better metaphor than consumption (Whittaker, 2011). The most powerful metaphor we have to date for understanding our dealing with information, however, is the information processing metaphor. This is the one that is central to cognitive science (described above) and the designer stance (described in the next section). In this book, I use the expression “processing knowledge resources”. This includes aspects of knowledge work (of which curation is a small subset), developing understanding and developing oneself. Because conceptual artifacts come in so many forms these days—e.g., as books, podcasts, screen casts, presentations, etc.—we need a term to refer to them. English teachers have generalized the term “text” for this. The risk with that tactic is to neglect important technical features of new media and technology. The educational technology community speaks of knowledge objects. But there is an equally important class of knowledge objects, namely the objects in knowledge itself (concepts, theories, algorithms, proofs, etc.) I favor the expression “knowledge resource” to draw our attention to conceptual artifacts as conveying information to be exploited. Processing knowledge resources (e.g., through reading), while it involves interacting with technology and knowledge resources, is largely a mental activity. In contrast, developing products, while it of course normally hinges on mental processes, is largely a public activity. It is public in the sense that it creates real, external and often negotiable products. Some of these products are physical. One can create a particular, concrete table, for instance. But many of them are abstract. ²⁷Like all metaphors, the digestion metaphor breaks down at a certain point. If you ingest nutritious food, your digestive system autonomously takes care of the rest. Food is automatically broken down into basic components. Your body creates and reconstructs what you need from it: blood cells, organs, etc. It even works while you sleep. In contrast, processing a potent document requires deliberate and systematic attention. You can’t passively extract and comprehend concepts, arguments, theories or principles. Of course, sleep is essential for consolidation, and most mental work is unconscious. But insightful problem-solving and creativity require effortful conscious work, even for the greatest minds. We are better served by information-processing metaphors than digestive ones. For a thoughtful attempt to relate cognitive processes to biological ones, see Piaget (1967). MacNamara (1976) criticizes Piaget’s attempt. Sziraki (1978) counters MacNamara’s criticism of Piaget. MacNamara (1978) soundly rebuts Szkiraki’s counter, ultimately with reference to Franz Brentano’s writings: “in a psychological act there is necessarily an intentional reference to some type of object. In a physical event or object there is not. For example when I know or I see or I fear there is of necessity an intentional reference to something known, seen or feared. In the physical world, by contrast, things just are. As Gertrude Stein once remarked: “a rose is a rose is a rose.”” MacNamara’s success does not eviscerate Piaget’s entire work. (Compare the foregoing references to Karmiloff-Smith’s reflective abstraction of some of Piaget’s ideas, and mine of hers.) I agree with Whittaker (2011) that curation is a better metaphor than consumption for information management requirements. Whittaker emphasizes the exploitation of information, a concept that I employ too. In this book, I also use a mining metaphor. I speak of “mining” knowledge “resources”, “extracting” knowledge “gems”, “developing” mastery, and “exploiting” knowledge. This helps direct our attention towards the fact that knowledge is an objective, artificial resource to be exploited. I will also refer to this as “World 3”. We can capitalize on knowledge (derive value) by processing it. I further speak of “instilling”. But all these metaphors have a breaking point. Johnson’s book nevertheless provides a useful antidote to the defeatism conveyed in The Shallows. I return to Johnson’s book in chapter 3. Our books address overlapping problems.
Introduction
12
One can create (and patent) a design for a table, for instance. Designs are abstract. Knowledge is a particular kind of abstract product or “stuff”. It is a conceptual artifact. “Conceptual artifact” is an expression you will often encounter in this book. (See also Bereiter, 2002a). Examples of conceptual artifacts are theories, designs and jurisprudence.²⁸ The core of knowledge work involves creating, assessing and improving conceptual artifacts. Knowledge processing (including learning) would have little to go on if objective knowledge was not created, perpetually improved and shared or traded. Knowledge work involves knowledge processing, of course. It is a uniquely human contributor to learning. The idea of knowledge-building as a powerful form of education was developed by Carl Bereiter and Maria Scardamalia (Scardamalia & Bereiter, 2006). Developing solutions to problems is another major thing we do with knowledge. Solutions are products in the sense that they are the application of a service. Some solutions are knowledge products. For example, the problem at hand might be to design a table. The design is your solution and of course an abstract product. However, solutions are usually not recorded as knowledge. For example, you might be tasked to custom build a table or to repair an existing table. This might not generate a re-usable record of your solution. Of course, an important productivity principle is to make build knowledge in the process of solving problems, such that one can re-use and improve the solution. Creating such knowledge is knowledge work. Another major use of knowledge is to develop ourselves. I specify this problem in the next two sections. It is the central problem addressed by this book. Ask sportsmen or sportswomen what they do for a living and you’ll rapidly be answered in terms of their sport, such as “I’m a hockey player”, or “I’m a baseball player” . Or perhaps “I’m an athlete.” Ask a musician and they’ll answer in terms of their particular instrument, such as “I’m a cellist”. But they might, similarly, simply state “I’m a musician.” None will have difficulty identifying with the superordinate category (musician or athlete). But ask a scientist, engineer, lawyer or someone else who develops and applies knowledge what they do and you’re unlikely to be told “I’m a knowledge worker”. Knowledge workers do not tend to think of themselves as knowledge workers. Why is this? It is not that knowledge work is a polymorphous concept, meaning that it refers to so many different kind of things. It is no doubt partly that knowledge work is only some of the many things these professionals do. Different members of the same profession engage various proportions of their time in knowledge work. The fact that knowledge workers don’t tend to see themselves as knowledge workers is, in my opinion, a transitory condition in the nascent knowledge economy. As time progresses knowledge workers, and society at large, will come to better understand themselves and their work. As I alluded to above, scientific and personal progress both hinge on conceptual development. Physicists benefited from folk physics—the intuitive knowledge of physics—and of course went beyond it. Newton fundamentally altered the meaning of colloquial terms, such as “force”, “mass”, “acceleration”, and “momentum”. Einstein returned the favor. Darwin revolutionized our understanding of evolution by relating it to his concepts of natural selection in relation to minor variations. This progress enabled educated lay people to understand physical and biological ²⁸They are normally expressed in a language that involves the application of predicates to objects. The latter are called “Fregean”, after Gottlob Frege who invented axiomatic predicate calculus. However, as Sloman (1975b) explained, they can also be expressed analogically, e.g., with diagrams. See also Funt (1980).
Introduction
13
phenomena. It enabled applied scientists and engineers to create technology we enjoy.²⁹ I believe that understanding foundational concepts and processes of knowledge work, particularly those involved in developing oneself with knowledge, will ultimately help people improve how they improve themselves. Cognitive scientists have also benefited from the deep wells of folk knowledge, e.g. regarding learning, emotion, and knowledge. Alas, it has been more difficult to evolve from folk psychology than naïve physics. Modern English, and our own ways of thinking, are still burdened by ancient concepts. To thrive in the Knowledge Age, we need to recognize and overcome these impediments. That calls for abandoning some expressions, assigning different meanings to old terms and adopting new terms and concepts. Given my claim that we need to improve how we think about our interactions with knowledge, it is worth pausing to reflect on the problematics of conceptual progress. It involves developing, analyzing, criticizing and reasoning about and with concepts— knowledge work and knowledge processing. We do not ask of concepts whether they are true or false. There is often no simple sense in which a concept is right or wrong. There are many ways to characterize a concept, not all of which are classical, analytical definitions (by genus and species), and none of which is universally the best. For a given term, there is not normally a single best definition. Defining a term is not always necessary; nor is producing or comprehending a definition the better part of analyzing or understanding a concept. It is often preferable to present concept specifications rather than definitions. Searching for the meaning of a term is not a productive enterprise. Deep conceptual understanding involves surveying the space of possible concepts, what Sloman calls “logical topography” (Sloman, 2010a). Within the space of possibilities one can search for productive³⁰ concepts and theories that make use of them. By this I mean that some concepts taken together are more effective than others at helping to resolve problems of understanding and practical problems. Being short a few terms can make it difficult to talk and think about things. This book analyzes several key concepts. It also provides practical recommendations on conceptual analysis. In this context, the next few sections briefly address the notions of personal development, mindware and adult mental development.
1.2.1 The designer stance Through culture and our own personal experience, children become “folk psychologists”, meaning psychologists with no formal training in the discipline. The English dictionary catalogs reams of useful concepts to describe, explain and predict emotions, moods, attitudes, feelings, motives, cognition, decisions and behavior. Many of these terms refer to mental events and processes. Cognitive scientists have adopted some of these terms and re-interpreted them. I claimed above that knowledge workers outside of cognitive science can reap some of the benefits of cognitive science to better understand and improve themselves. This will lead them to modify some of their mental ²⁹The public and perhaps most scientists believe that science progresses mainly through empirical discoveries. Certainly, most scientific work is empirical (experiments, quasi-experiments, etc.) However, the sense scientists make of all these data, through the development of concepts and explanations, is equally necessary. We think with concepts and theories. ³⁰Imre Lakatos spoke in terms of a theory’s heuristic power and progressive vs. degenerative research programs. See also chapter 11 for a discussion of how to assess knowledge resources on conceptual grounds.
Introduction
14
concepts. It will also introduce them to mechanisms in their minds and the organization of these mechanisms in an overall “architecture”. Before moving forward on this claim, I note that some researchers would disagree with it³¹. J. B. Watson felt that mentalistic concepts should altogether be dropped by scientific psychology (Watson, 1913). In more sophisticated proposals, Gilbert Ryle and Daniel Dennett, argued that putatively mental constructs can be interpreted (at least for the most part) without reference to internal mental processes (Dennett, 1987; Ryle, 1949). In The Concept of Mind, Ryle was particularly disparaging of mental concepts that have not evolved as part of folk psychology but instead are produced by psychologists and philosophers. For example, he (quite rightly) disparaged the classical idea that the mind has three parts: cognition, affect and volition. Many scholars would expect Ryle to disagree with cognitive science (as described in this book). In The Intentional Stance, Dennett took a similar position. He explicitly dealt with concepts from cognitive science that Ryle, his erstwhile professor, would not have considered. Dennett argued that when people use intentional concepts (chiefly, beliefs and desires), they are not really (or at least correctly) referring to internal states and processes. Intentional talk (and hence an intentional stance) is mainly used to describe and predict behavior. It can be applied in similar ways to various systems without one knowing the least bit about how the systems are internally designed or implemented. For example, when you put the right amount of money in a vending machine, normally you can correctly say that it believes that you have given it the right change. If it’s an honest machine, then when you tell it what particular product you want, it will want to give it to you. Thus, you predict it will give you your selection. You don’t need to know the specific mechanisms that operate in the machine in order to use intentional concepts to predict its behaviors. Dennett claimed the same was true for understanding each other’s behavior with intentional terminology. Thus, in the books mentioned above, intentional concepts and terms are for the most part quite coherent and useful ways of understanding and predicting people’s behaviors, though they don’t tell us much about what is happening inside their heads. While folk science and intuitive understanding are fundamental in physical sciences and psychology, they of course have all kinds of limitations that scientific approaches are designed to overcome. In particular, folk psychology is not particularly useful at making sense of errors, pathology and fine-grained data.³² Indeed, in order to develop a deeper understanding of each other’s behaviors and minds, cognitive scientists must transcend psychology by adopting the designer stance³³. The designer stance comes from Artificial Intelligence (AI) research (Sloman, 1993a). It is a form of reverse engineering of the mind that overcomes the legitimate concerns of behaviorism. In a nutshell, AI researchers in this tradition proceed as follows. (a) They attempt to specify the capabilities they wish to understand (e.g., some aspects of vision, language or reasoning). They consider informal ³¹For (a) important warnings about the dangers of interpreting folk psychology idioms as referring to real mental phenomena, and (b) a map to avoid the dangers, see chapter 3 of Clark (1989). ³²In his later publications, Dennett himself has taken some of the steps of the designer-stance; e.g., in Dennett (1991) and Hurley, Dennett, & Adams (2011). ³³Daniel Dennett famously introduced the expression the “design stance” to describe what AI researchers do. John McCarthy began to use the expression designer stance based on feedback from Aaron Sloman (McCarthy, 2008). Here, an engineering mindset dominates. See also Beaudoin (1994; Sloman, 1993a, 2005)
Introduction
15
observations and data from phenomena-based researchers.³⁴ Ideally, they subject the concepts of their requirements analyses to conceptual analysis. (b) They produce detailed designs of systems to meet these requirements. These designs specify an overall architecture and component mechanisms. (c) They implement as much of their designs as they can in computer simulations and possibly robotic systems. (d) They analyze the extent to which their designs and implementations meet (and fail to meet) their requirements. (e) They explore and study the space of possible designs that might satisfy these requirements. This leads to the gradual re-interpretation, pruning and replacement of folk psychological concepts and the creation of new concepts. Phenomena-based researchers test conjectures produced from the designer stance, folk psychology, and various cognitive science research programs. This book leverages pertinent empirical research. The second part of this book describes mechanisms of mind and their development from a designer-stance. Here are some of the concepts described there that are particularly important to personal development. • Monitors that help us recognize potentially pertinent information (for example, a child develops monitors to recognize when her name is spoken); • Inner motivators that generate new goals and evaluations (for example, a goal to slow down the vehicle as one approaches a red light); • Long-term working memory, a form of secondary memory that people develop as they gain expertise in a domain. It has some of the properties of normal long-term memory and some of the properties of working memory (rapid access). • Deliberate practice, a form of practice in which people engage in order to develop expertise. We are all intuitively familiar with this concept. However, outside of public performance disciplines, people tend to forget its importance. They also don’t necessarily understand how it relates to long-term working memory. Productive practice is a form of deliberate practice designed specifically for knowledge workers. The first three of these concepts are forms of mindware, whereas productive practice is a way to develop effectiveness, which hinges on mindware.
1.2.2 Mindware Information-processing is not simply a metaphor we use to understand the mind. It is what the mind does that is functionally important. It is what needs to be understood and modeled in order for us to make sense of mental phenomena and explain overt behavior. If we are to describe, in information-processing terms, the structures that the mind develops as people learn, we will need a ³⁴Phenomena-based research is by far the most commonly used family of research methods in psychology. It consists of various methods to test hypotheses produced through ad hoc theory development or, alternatively, through the designer stance. Undergraduate textbooks on research methods in psychology, for example Shaughnessy, Zechmeister & Zechmeister (2009), occasionally briefly describe theory construction. But they do not provide guidance on how to develop theories. Nor do they, to my knowledge, even mention designer-based research. I was accepted in the Ph.D. programme in the department of Psychology at McGill University in 1990. My prospective advisor, Prof. Thomas Shultz, accepted that I intended to do computational research. I asked him whether anyone had ever defended a theoretical Ph.D. thesis in his department. He said no! So, I turned down their offer and accepted a Commonwealth Scholarship to study in the Cognitive Science programme at Sussex University, which had an established tradition of theoretical research. Rigorous empirical research is required for progress in psychology; but it is not sufficient.
Introduction
16
concise expression to refer to them abstractly. The folk psychological term “memory” has culturally loaded connotations that disqualify it. We need a term for the active processors we develop, such as our monitors, motive generators, long-term working memory, and reactive processes (as described in Part 2). The generic term must not commit us to a specific type of component. Yet it must be more compact than “information-processing mechanism”. It must also be more theoretically neutral than the “agents” described in one of Marvin Minsky’s contributions to the canon of cognitive science, The Society of Mind. I opt for the expression “mindware”, which was coined by David Perkins and elaborated by Keith Stanovich. Mindware is the brain’s analog to a computer’s software. Like software, it comes in very different forms. It includes information processed in the mind, mechanisms to process information, mental representations, and even information architectures. Of course, mindware is only metaphorically related to software. Please do not assume I am (or that any serious AI researcher is) drawing a naive analogy between computers and minds. Mindware is cognitive science’s analog to matter in physics. Some physicists indicate that they are not really sure what matter is. Their models of matter change. But (so far as I know) they do not give up on there being matter. Cognitive science (as an information processing science) is a much younger science than physics. It seeks to explain higher-level phenomena. And so the concept of mindware, understandably, is still quite nebulous. It can, however, already benefit from the work of tens of thousands of cognitive scientists in addition to computer scientists and software developers outside of AI. The latter have studied countless types of information-processing systems that may be relevant to understanding mindware. I find it strange that the term “mindware” has not yet been widely adopted in cognitive science. However, I suppose it is just a matter of time before it or some other candidate for the concept takes off. For it is very convenient to have a term to refer to this important concept.
1.2.3 Adult mental development adult intellect is expected to grow over early and middle adulthood Phillip L. Ackerman This book describes a way of thinking about the development of competence. This usually falls under the umbrella term “learning”. However, Carl Bereiter has convincingly argued that the term “learning” is over-used and misleading (Bereiter, 2002a). There is such a variety of changes called learning that the expression is meaningless. A unicellular organism can learn in some sense. The most important distinction the term blurs is between the creation of objective knowledge and changes in mindware.³⁵ The distinction between objective knowledge and mental representations is not that knowledge is unbiased—knowledge can even be false. Rather, it’s that objective knowledge is potentially public. In some cases, objective knowledge is also negotiable. For example, one can buy, sell and license patents and copyright material. You can’t do that with your mental states, processes and mechanisms. While this may seem like an esoteric distinction, Bereiter has shown that blurring it is the source of much confusion. ³⁵See Popper (1979). I elaborate on these distinctions in chapter 5. They relate to Bereiter’s distinctions between learning and knowledge building.
Introduction
17
People often behave as if processing information guarantees they will be able to use it later. Yet new competencies do not often develop from knowledge resources as quickly or as passively as is common under associative learning paradigms such as classical (Pavlovian) conditioning and (Skinnerian) operant conditioning. (See the section Illusions of Meta-effectiveness.) Whether we are developing skills, understanding, habits, or simply an ability to recall information, a large number of mental changes must occur. Thinking about this as some kind of amorphous learning, or merely using the behavioral concepts of skills and habits, draws our attention away from the mechanisms that change as we learn. Thinking of learning in terms of mental (i.e., mindware) development draws us back inside. We don’t know exactly what happens mentally, let alone neurally, as we learn. But I suggest that we can benefit from using a broad and deepening theory of what happens when we grasp knowledge. This theory will draw our attention to the varied constituents of our mental development. It will involve hypothesized mental components such as monitors, inner motivators and long-term working memory. This way of thinking is also meant to help us choose ways of processing information that are more likely to deliver desired “learning outcomes”. Whatever theory of mind we espouse, many of us are accustomed to thinking of child development. People are much less apt to speak of adult mental development. We think of child development as a genetically unfolding program. However, developmental cognitive psychologist Annett Karmiloff-Smith has shown that epigenetic factors are very important in child development (Karmiloff-Smith, 2012). In a knowledge society, variability in adult development is largely a function of people’s interactions with knowledge. The minds of effectant people develop significantly over their lifetime, as a function of the knowledge resources they delve and master. The differences in mental functioning and performance between an expert and a novice (at least with respect to their domains of expertise) are as remarkable as the most striking differences between a young and older child. Effectant people’s mindware is programmed, and their mental architecture developed, with the knowledge resources they master.
1.2.4 Effectance: motivation for competence Sustein is one of the foremost legal scholars in the United States, and shares with other leaders of his profession the attribute of intellectual fearlessness. He knows he can master any body of knowledge quickly and thoroughly and he has mastered many, including both the psychology of judgment and choice and issues of regulation and risk policy. Daniel Kahneman As I mentioned above, Robert White (1959) coined the term “effectance” to make sense of the cognitive properties of children’s play. He attempted to fill conceptual gaps of two schools of thought, behaviorism and psychodynamics, that continue to limit our thinking about cognition. There is a competence motivation as well as competence in its more familiar sense of achieved capacity (p. 318) […] Such activities in the ultimate service of competence must therefore be conceived to be
Introduction
18
motivated in their own right. It is proposed to designate this motivation by the term effectance, and to characterize the experience produced as a feeling of efficacy. Expounding the concept of effectance, White emphasized the child’s need to learn to systematically influence the environment. He drew attention to the interest and curiosity displayed by children. The concept of effectance is of tremendous importance to knowledge work and personal development. White’s term never made its way into dictionaries and is only infrequently cited in the psychology literature. Waytz et al. (2010) are an exception. They defined effectance as “the motivation to attain control, predictability, and understanding, and to reduce uncertainty, unpredictability, and randomness.” (p. 424). They noted that the concept is important to make sense of much research, including on: sense-making, need for closure, desire for control, locus of control and, I would add, thinking dispositions and perceived self-efficacy. Absent a term for effectance, however, we are prone to overlook some of the major reasons why some people progress more than others (and more at some times than they did previously.) While White’s concept of effectance is a useful starting point for understanding the factors that drive people to improve themselves, it has a weakness. In chapter 3, I put forward a more subtle, parsimonious and powerful concept of effectance.
1.2.5 Meta-effectiveness Human language, and human culture, are not instincts— but they are instincts to learn W. Tecumseh Fitch With these concepts in place, we can revisit meta-effectiveness. Meta-effectiveness refers to the skills, dispositions and manifold underlying information-processing mechanisms that enable and drive people to improve themselves. It includes both fluid expertise and effectance. Fluid expertise is the ability to develop expertise (Bereiter & Scardamalia, 1993); it includes learning skills as distinct from one’s inclination to apply them. The concept of meta-effectiveness exemplifies a key tenet of this book: the folk psychological distinction between motivation and abilities fades when we adopt the designer-stance. This will become clearer when we take a closer look at information processing in Part 2. Many knowledge workers have easy access to useful high caliber knowledge. The major bottleneck in the development of personal excellence, for them and many others, is converting this knowledge into mindware. Meta-effectiveness is the width of this bottleneck. It enables and motivates individuals to release the potential of objective knowledge in themselves. It involves mindware that potentiates objective knowledge, further generating and developing mindware. The most potent ways in which knowledge workers improve themselves are through delving knowledge, progressive problem-solving, knowledge building, reflecting-in-action, deliberate practice, deliberate performance. • Delving refers to attentive, deliberate processing of knowledge resources (e.g., reading, attending seminars meetings, lectures and workshops, listening to podcasts, watching videos).
Introduction
19
• Progressive problem-solving refers to addressing and attempting to resolve increasingly difficult problems (Bereiter & Scardamalia, 1993). It entails working at the edge of one’s competence, as opposed to simply trying to reduce effort, be efficient, and “get things done” in the short run. • Knowledge building refers to creating, improving and assessing objective knowledge in response to problems of understanding (Scardamalia & Bereiter, 2006). • Deliberate performance refers to deliberately practicing skills on the job, i.e., while accomplishing things. • Deliberate practice involves purposefully practicing, offline. • Reflecting-in-action involves thinking about one’s work, and improving it, while one is doing it (Schön, 1983). • Productive practice is a form of deliberate practice in which one uses (and potentially builds) knowledge to develop personally (to become more effective). These activities are not completely orthogonal. For example, productive practice and progressive problem solving often involve knowledge building. Schön also described multiple types of practice (Schön, 1982). More importantly, the mental processes involving these activities overlap in ways that can be analyzed. (For example, the process of representational redescription (Karmiloff-Smith, 1995) can be invoked in all of these activities to develop mindware and improve one’s effectiveness.) Of these activities, this book focuses mainly on delving and deliberate practice (in the form of productive practice).
1.3 Example knowledge resources referenced in this book As we will see in chapter 7, meta-effective people like to use examples to drive their learning. They also process them more carefully (VanLehn, 1996). Therefore, I’ve loaded this book with examples. In particular, I refer to four sets of knowledge resources that are likely to be pertinent to my readers. They illustrate a wide variety of types of effectiveness you might seek. They are the following: • Some concepts expressed in two of John Gottman’s books: With N. Silver (1999). The Seven Principles for Making Marriage Work.³⁶ With J. DeClaire (2001). The Relationship Cure: A Five-Step Guide For Building Better Connections with Family, Friends, and Lovers³⁷. • Some concepts expressed in Keith Stanovich’s (2009) book, What Intelligence Tests Miss: The Psychology of Rational Thought³⁸. • Concepts developed in Eric Ries’s (2011) book: The Lean Startup: How Today’s Entrepreneurs Use Continuous Innovation to Create Radically Successful Businesses³⁹. • Various works by Aaron Sloman and myself (including this book). ³⁶http://www.gottman.com/shop/7-principles-for-making-marriage-work-2/ ³⁷http://www.gottman.com/shop/the-relationship-cure/ ³⁸http://www.keithstanovich.com/Site/Books.html ³⁹http://theleanstartup.com/book
Introduction
20
I will use these example resources to illustrate the concepts and techniques described herein.⁴⁰ For example, chapter 14 explains how to become able and disposed to apply Gottman’s concept of bids through productive practice. My book is of course not about that knowledge but about the problems of mastering knowledge. Therefore, while I highly recommend these other resources for their own sake, they are not required reading for this book.
1.3.1 Keith Stanovich (2009). What Intelligence Tests Miss: The Psychology of Rational Thought As I have said, ingratitude does not surprise me. What does startle me, in retrospect, is my lack of curiosity. Hermann Hesse’s Emil Sinclair character Keith Stanovich’s 2009 book, What Intelligence Tests Miss: The Psychology of Rational Thought⁴¹, is an important and very readable contribution to cognitive science. Many others had previously written about patterns of thinking and decision making, “cognitive biases”, that systematically predispose us to error. But Keith Stanovich and his colleagues have identified and aptly characterized some of the most fundamental sources of human irrationality. They have pointed, in general terms, to avenues for people to avoid making costly errors. Stanovich proposes that the apparent paradox of “smart” people, like George W. Bush, doing “stupid” things (like invading Iraq) can be resolved by separating the concepts of intelligence and rationality. Let intelligence be defined as mental abilities measured by intelligence tests, i.e., fluid intelligence and crystallized intelligence. Rationality refers to abilities and tendencies to reason, make judgments and decisions that optimally achieve one’s goals and solve one’s problems. Rationality is a different and broader concept than intelligence. To behave rationally, one needs not only to have the abilities to think productively, but to be inclined to activate them. Cognitive misers may be intelligent (have a high IQ) but they tend to be too intellectually lazy to bother to think rigorously when faced with even an important problem. They also don’t tend to bother to acquire the “mindware of rational thought”, which Stanovich refers to as cognitive rules, strategies, and belief systems for reasoning properly. An example of a cognitive strategy is “consider the opposite”. That is, when given a proposition, instead of simply assuming it to be true, to consider reasons why it might be false. Stanovich grounds the distinction between intelligence and rationality in a three-layer architecture of the mind. The three layers of the human mind are: 1. The autonomous mind, which handles all kinds of quick, well-trained, automatic routines (e.g., tying one’s shoes, playing a well rehearsed piece of music.) 2. The algorithmic mind, which handles the slower thinking that is characteristic of fluid intelligence (e.g., learning a piece of music, solving an unfamiliar problem). ⁴⁰My examples pertain to the knowledge conveyed by the resources rather than the resources themselves. And so I utilize related literature on the same knowledge. For examples: the concept of mindware referenced by Keith Stanovich was first proffered in Perkins (1995); Wile (1993) inspired Gottman’s own theories including the roles of bids (see, for example, Gottman and Driver, 2005); there are several scholarly papers on the two major relationship concepts used as examples in this book (e.g., Herrin, 2009; Barnacle & Abbott, 2009). ⁴¹http://www.keithstanovich.com/Site/Books.html
Introduction
21
3. The reflective mind, which monitors and controls the two other layers. Many costly errors in life may be avoided by (a) acquiring the mindware of rational thought, (b) acquiring the thinking dispositions that prevent overly miserly cognition. Keith’s Stanovich book is relevant to our concerns in many ways. It proposes a high-level, easy to understand, model of the mind that is compatible with the one described in Part 2 of this book. The very assumption that it is useful for practical and theoretical purposes to utilize such an architecture is itself a similarity between his book and this one. His book also proposes concepts— such as thinking dispositions, cognitive miserliness and mindware—that are important to this book. Moreover, Stanovich is concerned with improving rationality. The meta-effectiveness framework aims to help its readers be rational with knowledge.
1.3.2 John Gottman: Seven Principles for Making Marriage Work and The Relationship Cure Divorce, family strife, sour friendships, work relations gone bad. These problems can often be traced to the same root causes. Relationship problems can impact one’s affect, health, productivity at work, income, likelihood of being hired, promoted and fired. Fortunately, many relationships can be repaired by exploiting the same principles that when neglected lead to problems. Alas, finding a sufficiently helpful relationship framework is not easy. Fortunately, there is the scientific work of John Gottman.⁴² He and his colleagues have carefully studied thousands of relationships and discovered the “E=mc²” of relationships. Gottman and Silver claim to be able to predict, 91 percent of the time, the outcome of a relationship after observing the couple interacting for as little as five minutes. Gottman and colleagues have published extensively on the prediction of marital outcomes (e.g., Gottman, 2002; Gottman & Levenson, 2002; Gottman et al., 2002). In their book, Gottman and Silver document four reliable predictors of relationship failure and seven principles to set things right. The ominous behaviors are criticizing, showing contempt, defensiveness and stonewalling. For example, if you literally sneer at your partner (or vice versa), your relationship is in big trouble. In this book, I will focus on two of their principles and concepts. Einstein simplified physics by isolating, characterizing and relating three key concepts: energy, mass and the speed of light. Similarly, Gottman noticed that a core construct at the heart of interpersonal relations is the bid. A bid is a request, of any size, that one person makes to another. In relationships, some bids implicitly solicit connection with the other. For example, “How was your meeting with your client?”, “I’d love to go dancing Saturday night” and “I’m tired [while staring at the kitchen, meaning, please help with the housework]” are all bids. Gottman found that bids are ⁴²Several authors have examined the quality, effectiveness and rhetoric of psychological self-help literature (bibliotheraphy), including Gottman’s work (e.g., Hill, 2007; Redding, Herbert, Formann & Guadiano, 2008; Schrank & Engels, 1981). Marano, 1997 and Hill (2007), for example, are (correctly) highly critical of John Gray’s popular Men Are from Mars, Women Are from Venus in contrast with Gottman’s. We need to distinguish the potential helpfulness of literature from its actual effectiveness which varies as a function of the individual’s meta-effectiveness. For more on the assessment of knowledge resources, see Ch. 11. Science is a social enterprise. Several of Gottman’s books and scholarly papers were co-authored; some of his research builds on and has been replicated by others. However, I will generally refer to his work simply as Gottman’s work. The same principle applies to the other knowledge resource examples used in this book.
Introduction
22
issued very frequently in relationships; and more frequently in healthy ones. He identified three ways to respond to bids: • turn towards your partner (whether you do it very energetically or not), • turn away from your partner (e.g., change the subject or counter-bid), • turn against your partner (e.g., mock or criticize them). He also discovered that how people respond to each others’ bids determines the course of their relationship. In retrospect, this is quite obvious. Even behaviorism can make sense of this through their concept of “extinction”. If you habitually ignore someone’s bids, they will eventually get the message that you’re not interested. People learn this very quickly. Of course, turning against bids sends a stronger message. One can quickly drive a relationship into the ground by systematically turning away and against the other’s bids. Hence the first of Gottman’s principle addressed here is for members of a couple to turn-towards each other’s bids for connection. This fascinating and powerful principle holds for all kinds of relationships. Some relationship problems will never go away. But others are solvable. One needs to classify one’s problems and ensure that one addresses the tractable ones. The second Gottman principle I use as an example in this book is based on that premise: “Solve your solvable problems.” Gottman proposes five steps to do this. We’ll focus on the first one, which is to avoid “harsh-startups”. When one partner communicates their discontent in a provocative manner, a harsh startup may ensue. Both partners can avoid poisonous interactions by responding to frustrations with the right affect (attitude, emotion and motives) and with the right behavior. Gottman provides several specific tips for avoiding harsh startups. It is more obvious with practical knowledge than with purely factual knowledge resources that you can’t simply read a book to master its principles. Thus, Gottman’s work is germane to the challenges addressed by the meta-effectiveness framework. It addresses problems that everyone faces. Fortunately, Gottman doesn’t just explain his findings. His work provides copious exercises to help people master the concepts, principles and strategies that can help them excel where it matters most, in relationships. That jives with research from cognitive science presented in chapter 3 and Part 2 of this book: reading is not enough, one needs to practice productively. So his books provide clues as to how to respond to any knowledge one might want to master. Moreover, the kinds of mindware Gottman deals with is not just classically cognitive (thinking, reasoning, deciding). The mindware supports multiple purposes of learning: skills, self-regulation, habits, propensities, attitudes and genuine understanding. In this book, I will show you how the meta-effectiveness concepts can be used to understand what happens to people mentally as they master the concept of bids and the principle of avoiding harsh startups. I will also illustrate how the meta-effectiveness framework can help you master these ideas. This is not meant to be a substitute for reading the original work. I hope it will instead encourage you to read this and the other core books to which I refer.
Introduction
23
1.3.3 Ries (2011): The Lean Startup Eric Ries’s The Lean Startup⁴³ book attempts to explain why startup projects fail and what can be done about it. He exposes the myth according to which startups success is due to genius, perseverance and timing. He argues that some entrepreneurial projects fail because their leaders put too much faith in a good business plan, market research, and predictive strategy. Others fail because their leaders shoot from the hip. Ries proposes a success strategy that lies in Aristotle’s golden mean. A common, but flawed product management assumption is “build it, and they will come”. Ries argues instead that leaders need to follow a rigorous process in which products are built in small increments. Each increment must be assessed by the market or its proxies. The first release should be a minimum viable product—devoid of anything that may be optional. Each new product increment represents a hypothetical value proposition. Each such hypothesis must be submitted to a test. For customers or stakeholders might not like any of it; and they may, explicitly or implicitly, convey meaningful information about what they do or would value. The results of each test must be analyzed to discover whether the value proposition is correct. One must then learn, and either “pivot”, i.e., put forth another conjecture, or persevere with one’s plan. Product development in this model comprises an empirical, Build-Measure-Learn loop. This innovation process is not predicated on genius, luck, blood, sweat or tears. Ries in effect applies agile⁴⁴ product development ideas to entire projects and businesses. Agile methods have become increasingly prevalent in software development⁴⁵. Ries’s proposal, however, apply not only to high-tech projects. The general ideas are applicable to all kinds of organizational and personal projects. The lean approach applies (some) scientific methods to the projects of one’s life. I say “some” because there is more to science than these empirical processes. Along the way, Ries proffers important concepts such as: • • • •
innovation accounting (measuring the value of product revisions) validated learning (a process predicated on the former, empirically tracking progress) learning milestones (milestones in projects are now viewed in terms of the learning cycle). genchi genbutsu (a concept from Toyota meaning “Go see for yourself”, don’t assume you understand the situation) • andon cord (another concept from Toyota,⁴⁶ meaning to detect and respond to process failures quickly and resolutely, such that you minimize their nefarious effects and efficiently produce value, i.e., be productive). My main purpose in referring to Ries’s book is to invite you to test my hypothesis that the ideas I describe in this book apply to all kinds of knowledge resources, including this one. Like the other books I use as examples, Ries’s book is challenging because in order to master it, one must change substantially: i.e., to develop new conceptual understanding, new skills, new habits, new norms, new attitudes and so forth. ⁴³http://theleanstartup.com/book ⁴⁴http://agilemanifesto.org ⁴⁵http://www.ambysoft.com/essays/agileManifesto.html ⁴⁶When an issue is discovered on an assembly line, a worker can pull this cord to call for assistance. The worker can stop the entire line to allow
the issue to be investigated immediately.
Introduction
24
In addition, the concepts and principles in Ries’s book are relevant to the subject of this book. They had not previously been applied to the problems of meta-effectiveness. Lean methods are predicated on information processing. When one seeks to learn from any knowledge resource, it is not sufficient to read. One must learn and try to gauge one’s success over a period of time. When one fails in one’s application of knowledge, one should consider “pulling the andon cord”. This means one needs to find ways of measuring the quality of what one reads and our abilities to learn. The lean approach also suggests that readers should not expect to master knowledge resources in one fell swoop, but that it will require iterations of testing and reformulating one’s mindware. The lean principles can also be applied to the workflows described in Part 3. That is, the system you use for learning itself can be subject to tests.
1.3.4 The work of Aaron Sloman and other cognitive scientists Finally, I use the concepts that are proposed in this book as examples of things to learn. I use concepts from the cognitive science literature, with a particular emphasis on the work of Aaron Sloman, Carl Bereiter, Kurt VanLehn, Keith Stanovich and mine. I also exploit the expertise literature and the test-enhanced learning literature. Recursion is a powerful technique, and as you will see, a favorite of mine; but it can be taxing. That is another reason that I used the three other sets of examples mentioned above instead of just sticking with examples from this book. However, given that this book is the only book I can assume that my readers will be familiar with, it makes sense for me to draw some examples from it. This ought to help you master my book.
1.4 Three vignettes: Disasters avertable by applying knowledge You can’t think about thinking without thinking about thinking about something. Seymour Papert The major objective of this book is to help you identify and develop from potentially helpful knowledge resources. To accomplish this goal, I first must convince you that it is possible and worthwhile to become more meta-effective. This is not something that can easily be done in the abstract, without reference to particular examples, including knowledge and outcomes that matter to you. So, please take a moment to look at your physical and electronic libraries (web browsing history, ebooks, PDF files, etc.) Consider also courses, workshops, webinars and meetings you have attended. Choose documents of various caliber and usefulness. Was the time you spent processing them worthwhile? Would additional effort have paid-off sufficiently? Odds are there are resources that made a huge difference for you professionally or personally. Equally likely, there are promising resources that would have made a big difference in your life, if only you had taken the time to master their knowledge gems. There are also resources that would have improved your life to a lesser extent. Many little increments can add up (a lean principle). Your library also surely contains a raft of information that is, and ought to remain, mainly reference material. There is the stuff not worth remembering let alone applying.
Introduction
25
There are as many possible improvements to be made as there are problems addressed by knowledge. Better relationships, investments, career moves, project management decisions, documents, health outcomes, etc. So, take a moment to think of what those outcomes would have been, using particular resources you have processed. To really get a feel for this, you might need to treat this little task as a project to execute, now and again, over several weeks. To be sure, “to err is human”. However, if you recall and ponder specific errors that reasonable people around you have made, you will likely find that many of these errors could have been avoided if only the actors had applied what they actually already, in a sense, “knew”. The newspaper is filled with examples. This is because there is a world of difference between knowing something abstractly and being disposed to apply that knowledge when it matters. Bridging this gap is what the metaeffectiveness framework is about. To improve is divine. I mentioned above that research in cognitive psychology suggests that learners like to learn from examples. Furthermore, students learn more from studying examples than from trying to solving the same problems themselves. For these reasons, in the previous section, I presented several example theories that I will use throughout the book, two of which are leveraged directly in the framework I have developed (i.e., those of Keith Stanovich and Aaron Sloman). That’s also why I assigned the previous task. As this is a book and not a one-on-one coaching session, I can’t draw from your examples. Instead, I use the following pedagogic device. The sub-sections below present three specific hypothetical but realistic scenarios and a cast of hypothetical characters. I will illustrate how these characters could benefit from mastering the theories described above. Throughout this book, I will refer to these characters and scenarios. This will save me the trouble of conjuring up new characters and scenarios every time I want to illustrate a point. I hope this will help you master the metaeffectiveness framework.
1.4.1 Being taken to the trough but choosing not to partake High-tech startups are created by and attract intelligent, ambitious people who are willing to work hard in exchange for an unparalleled journey of learning, camaraderie and financial opportunity.⁴⁷ Their employees are usually young. There are drawbacks to being smart, having a track record of cognitive success and knowing it. One can become overconfident about one’s competence outside of one’s area of expertise. Expertise fundamentally hinges on having knowledge. Many startup upstarts lack financial savvy. They don’t necessarily appreciate the implications that for every startup that reaches great heights, there are many more that fail. Now for the plausible fiction. Jack and Chrissy signed up to work for a promising high-tech startup, NewCo. They both were very intelligent, competent R&D staff. They each had received 10,000 options⁴⁸ to buy stock in NewCo at $1 per share. They were typical of the speculative high-tech bubble of 1997-2000, in which this story is set, except that they had sound financial guidance: They were subscribers to Pat McKeough’s The Successful Investor⁴⁹ newsletter. McKeough consistently ⁴⁷Chapter 8 of Nesheim (2000) describes the personal rewards and costs of founding and participating in a high-tech startup. ⁴⁸http://en.wikipedia.org/wiki/Stock_options ⁴⁹http://www.tsinetwork.ca/publications/the-successful-investor/the-successful-investor/
Introduction
26
explains and extols reasonable principles of investing. For example, he advises to invest mainly in a diversified basket of consistently successful companies that have under-appreciated value. NewCo was purchased by ExtantCo in an all stock deal. Each single NewCo stock (option) could be exchanged for one ExtantCo stock (option). This was a big boon for Jack and Chrissy, because ExtantCo was trading at $200. On paper, they were each instantly worth almost $2M. This was complicated somewhat by periods during which employee trading was prohibited (closed trading windows) and other factors. But these two particular employees could in principle navigate those waters. Clearly, McKeough’s principles entailed that they should exercise their options and sell ExtantCo stock ASAP. For example, he advises people to diversify their holdings across the five sectors of the economy and then to diversify within them. He also advises people mainly to invest in well-established companies. However, at ExtantCo, there reigned an exuberant and optimistic atmosphere. After all, you have to be an optimist to thrive in a startup. Moreover, as Stanovich has since extensively documented, being intelligent does not put one above psychological principles and cognitive biases. One of these biases is the tendency to ignore disconfirming evidence. So, when ExtantCo shares began to decline, rather than allow this to shake her conviction in ExtantCo, Chrissy assumed that the market was missing out on a buying opportunity. By her reckoning, ExtantCo should be worth $400/share. Chrissy therefore decided to hold onto her ExtantCo shares as the stock price began to decline. Chrissy even used her theory to justify buying more ExtantCo stock, on margin (i.e., borrowed money), at $60, $30 and $15 per share. Alas, ExtantCo went into receivership and laid her off, as did the next two companies for which she toiled with waning enthusiasm. In contrast, using McKeough’s investment principles as his rudder, Jack sold his ExtantCo stock as soon as he could. He paid off his mortgage. He invested in McKeough’s recommendations, which have gone up consistently since then, staunchly recovering from the 2000 and 2008-2009 crash. Jack went from working for stock options to work being an option. Of course, this is a radical contrast. However, the general principle that mundane and significant life outcomes depend on our decisions, in general, and our using knowledge to make decisions, in particular, is indisputable. Keith Stanovich’s 2009 book⁵⁰ describes how a mathematics professor, John Allen Paulos made parallel mistakes with WorldCom, as did thousands of investors during that era. Paulos bought WorldCom at a high price, then bought all the way down, even on margin. We will come back to Chrissy and Jack later in this book. Fictional Chrissy later read Keith Stanovich’s 2009 book. She understood that her experience could partly be explained by Stanovich’s account of Paulos. She vowed to apply Stanovich’s ideas about rationality and thinking dispositions throughout her life. Stanovich’s books, however, do not tell the full story of these fictional characters. The metaeffectiveness framework provides additional psychological theory and practical guidance to help people systematically apply all kinds of knowledge they acquire, including and beyond successful investing. ⁵⁰http://www.keithstanovich.com/Site/Books.html
Introduction
27
1.4.2 The applied science of marital failure Ralph listens uneasily as the judge reads her decision. She awards custody, child support and spousal support to his now ex-wife Karen. His heart and finances are broken. That evening, he paces in his apartment, reflecting on the decline of his marriage. His gaze falls on a relationship book by John Gottman that she had bought them. “Seven principles? What were they again?” He leafs through the pages whose crispness betrays their neglect. “Turn towards bids? Oh, right, I remember. Yes … she used to ask to do things together. I was always too busy or tired. Then she stopped asking…” “Harsh-startup?” He re-reads the section and reflects: “Yep. Karen became very judgmental about my housekeeping and my schedule. I don’t think she used these tips to express her complaints. So, I just tried to defend myself and then I guess I stonewalled. No wonder I didn’t want to go out on dates… with a porcupine.” As Gottman points out, troubled couples forget the good times whereas thriving couples relive and repeat them. But Ralph comes across a picture of him and Karen together in their early days. “I wonder what would have happened if we had applied this book?” Over the next few weeks, Ralph rereads Gottman’s books. “This is how I’m going to do it next time. And I’ll only date a woman who is determined to relate according to these principles.” But is he sufficiently meta-effective to systematically apply what he reads?
1.4.3 Project failures Chrissy’s woes aren’t all due to her failure to apply McKeough’s principles. On exiting ExtantCo, for years she obsessed about the fact that she foolishly failed to apply what she superficially knew (McKeough’s investment principles). She has taken the knowledge to heart and invested wisely since then. Her subsequent woes are instead partly attributable to working for poor project managers, such as Janet of Waterflop Co. Like many other high-tech managers, Janet is familiar with agile project management methods. She has read several agile books, including The Lean Startup⁵¹. She could probably pass a pencil and paper test about agile. However, the exams of life are not of that nature: Life judges us by our propensity to apply helpful knowledge. Janet has been working with “waterfall” methods all of her working life. She uses many “knowledge shields” to rationalize her approach.⁵² At crucial decision making moments, she recalls the title, if not the substance of Frederic F. Brooks’s article “No Silver Bullet”.⁵³ She wrongly interprets as “Changing our general approach will not significantly improve our outcomes.” True, agile methods are not a silver bullet. However, there is no denying the pertinence of many of its key concepts. Robert N. Charette provocatively argued in IEEE Spectrum that most software project failures are preventable: For example, in 2004, the U.S. government spent $60 billion on software (not counting the embedded software in weapons systems); a 5 percent failure rate means $3 billion was probably wasted. However, after several decades as an IT consultant, I am ⁵¹http://theleanstartup.com/book ⁵²Hoffman et al. (2009) define knowledge shields as “Ill-formed counterarguments that let learners who are confronted with inaccuracies in their
knowledge or reasoning, preserve their simplistic understanding in the face of contradictory evidence.” They are cognitive defense mechanisms. ⁵³Brooks (1995).
Introduction
28
convinced that the failure rate is 15 to 20 percent for projects that have budgets of $10 million or more. Looking at the total investment in new software projects–both government and corporate–over the last five years, I estimate that project failures have likely cost the U.S. economy at least $25 billion and maybe as much as $75 billion. Of course, that $75 billion doesn’t reflect projects that exceed their budgets–which most projects do. Nor does it reflect projects delivered late–which the majority are. It also fails to account for the opportunity costs of having to start over once a project is abandoned or the costs of bug-ridden systems that have to be repeatedly reworked. (Charette, 2005) I will not assess Charette’s specific figures. However, clearly, the impact of project failures is tremendous. Surely the R&D community is generating knowledge about how to run projects more effectively. Janet is not alone in failing to apply pertinent knowledge. Consider Charette’s further point: The UK even has a government department charged with preventing IT failures, but as the report noted, more than half of the agencies the department oversees routinely ignore its advice. I call this type of behavior irrational project escalation–the inability to stop a project even after it’s obvious that the likelihood of success is rapidly approaching zero. Sadly, such behavior is in no way unique. This is known as “escalation of commitment”⁵⁴. These scenarios are meant to illustrate the concept and possible consequences of failing to apply what we, in a sense, “know”.
1.5 The imperative of meta-effectiveness The pursuit of individual happiness and collective welfare depend not only on reading but on developing lasting, active understanding through and with knowledge. This is in accord with Aristotle’s insight that happiness is not just knowing in an intellectual sense but being disposed to act in accordance with our best knowledge.⁵⁵ No previous era has faced problems of the magnitude and complexity that menace humanity today. Yet in no other era have people had the benefit of the knowledge and technology we now have at our disposal. It seems to me that we have a moral obligation to pursue excellence. We must not squander our inheritance. To address this imperative, we need to understand the impediments to our meta-effectiveness. Information technology, for all its benefits, has serious flaws we must work around. We are harried. We often implicitly assume that reading effectively “sharpens the saw”; i.e., that it is trivial to “translate” knowledge into competence and action tendencies. Thus, we fail to apply what we know. To overcome these challenges, to produce in ourselves the lasting effects we seek, to effectively ⁵⁴Staw (1981). Heng, Tan & Wei (2003) describe how this phenomenon plays out in information technology projects. ⁵⁵Aristotle claimed that happiness lies in the development and exercise of virtue (i.e., practical excellence) and intellectual contemplation (Aristotle,
2000).
Introduction
29
“sharpen the saw”® requires that we process knowledge in ways that reflect how the mind works. We must effectively extend and apply cognitive science and technology.⁵⁶
1.6 Overview of this book Given the concept of mental development sketched above, it would be reasonable of you, the reader, to demand two things of cognitive scientists and technologists: 1. To provide a mental development manual. This manual would provide you with a sketch of the mind. It would also provide concepts, workflows and principles, if not instructions, for developing mindware from knowledge. 2. To provide you with a mental development kit. This would contain, in addition, software and hardware tools that you could use to develop your mind. The manual would reference these tools. I started researching and developing solutions to these problems full-time in 2001 and have been at it ever since. I worked for about eight years with Prof. Phil Winne and colleagues at SFU on a related project. Since then, I’ve worked on this in private enterprise. I have extended and applied my prior academic training in cognitive science (including information processing modelling of motivation and emotion) to these problems. For we cannot adequately enhance meta-effectiveness unless we consider the intersection of cognition and affect; and we must use the informationprocessing metaphor—the foundation of cognitive science. Figure 1.1 summarizes the process of developing effectiveness from information. Meta-effectiveness is a matter of optimizing this process. ⁵⁶I agree with Donald Schön that rational practical conduct is not a simple matter of logically deriving applications from science (Schön, 1982). However, in a knowledge-rich society such as ours, scientific principles and concepts should guide conduct, including the recursive problem with which this book is concerned (i.e., applying knowledge). I agree with Carl Bereiter that education is not merely factual, but a design-based discipline. It requires “innovativeness, responsiveness to evidence, connectivity to basic science, and dedication to continual improvement.” (Bereiter, 2002b). Thus, this book adapts and extends existing practical applications of cognitive science, such as deliberate practice itself.
30
Introduction
Figure 1.1 Processing knowledge for effectiveness
The rest of this part of the book is concerned with meta-effectiveness. The next chapter analyzes our tacit objectives in processing knowledge resources. In order to make progress towards any goal, including mastering knowledge resources, we also need to have a sufficiently detailed understanding of the challenges we face. Knowledge workers are not immune to illusions of competence. Moreover, they need to contend with technology that is designed for surfing rather than for personal development. chapter 3 analyzes these challenges. Part 2 presents you with a brief overview of the cognitive science that is relevant to metaeffectiveness. Chapter 4 gives you new ways of thinking about your mind. It presents a model of the mind that you can use to interpret your own functioning and that of others. Chapter 5 explains how to use the previous ideas to think about personal development. Backed by fundamental empirical research, chapter 7 argues that deliberate practice is an effective way of becoming more effective. If you’re not particularly interested in cognitive science, these are chapters you can simply inspect rather than delve. Part 3 presents workflows, tools and tips to capitalize on knowledge with information technology. The framework is specifically designed to help you reach the objectives described in chapter 2, while avoiding the problems described in chapter 3 while leveraging the science of Part 2. “Capitalizing on knowledge”, itself, is an important way of relating to information. It may motivate
Introduction
31
one to approach information systematically. That means to assess knowledge resources (chapter 11); to delve the most promising material (chapter 12); and to practice the knowledge gems (Chapters 13 and 14. Depending on reader feedback, future editions may have a chapter to deal with weekly reviews and planning.
2. Psychological contributors to effectiveness Why do they do the things they do? David Francey To become more meta-effective, we need to understand the high-level psychological contributors of effectiveness. They figure prominently in our learning objectives. We ought not simply seek to read or comprehend something, but to deeply understand it, to develop norms, attitudes, habits, and so on. We also need to situate these learning objectives within the set of uses to which we put knowledge. This will help us to ensure that we are solving the right problem in tackling metaeffectiveness. Analyzing tasks and abilities that most people take for granted is a hallmark of expertise. To develop expertise, we need to treat as novel tasks at which we are already quite adept—e.g., reading. As Guadagnoli (2009) explained, Tiger Woods often reworks his already expert swing, to make it even better. Mental processes may seem obvious…until you try to replicate them in a computer program, as Artificial Intelligence researchers attempt to do. Hence, this book examines information processing and learning up close, beginning with the end, i.e., the purposes of learning. As figure 2.1 below illustrates, we process knowledge resources to different extents and for various purposes. Given that we are exposed to more resources than we can process, when faced with a knowledge resource, we need to be attuned to the distinct possibility that it is irrelevant or junk. Avid readers have a natural tendency to process documents before them. It’s particularly easy for them to be distracted by irrelevant information. If a resource passes the first filter, the next step is to add it to our indexes. That could be to tag it, bookmark it, file it or to make a note of it for future reference. We might then decide to inspect it. This could be to sample it here and there, or to read (listen, or view) it quickly all the way through. Some resources are promising enough to delve into. That is, we “actively” or “elaborately” read them, as educationalists say. English has not yet firmly established a term for “elaborate, purposeful processing” that is neutral to the type of information being processed. So I chose the term “to delve”, which people occasionally use in this way. Normally, when we delve a resource, we decide, or ought to decide, whether the resource is worth developing (knowledge or oneself) with. We might decide to develop a solution to a problem with. In other words, we provide a service with it. Or we may decide to use the information to build a re-usable product. This could be some new bit of objective knowledge, a document, a program or something else. Carl Bereiter and Marlene Scardamalia refer to this as “knowledge building”. We will return to the concept of knowledge building throughout this book.
33
Psychological contributors to effectiveness
Figure 2.1 Uses of Information
Sometimes, however, we not only want to use knowledge for external purposes, we want to develop ourselves with it. That is the final path in the figure above. We want to be able to use it without needing to refer back to it. In this case, we want to be changed by it in a lasting way. It is only rarely that we want to instill all the knowledge in a resource. We will normally only wish to extract its knowledge gems. Often, it is not the objective knowledge conveyed by the resource that we want to master, but our own interpretation, extension or conclusions. To say that one has read a particular document, or processed some other type of resource, is not very precise. We may have “learned” more or less than we think. After reading information people are more likely to use it, and can read it more quickly, even if they cannot even recognize it as information they have encountered. People implicitly know more than they explicitly “remember”, which goes to show that the concept of remembering is polymorphous. This phenomenon is referred to as “implicit memory” and “priming” (Tulving & Schacter, 1990). After processing the information a bit more deeply, people may be able to recognize it without being able to recall it.¹ Eventually, they may not recognize it as something they have processed before, but their more rapid processing of the same information demonstrates their implicit memory. Of course, sometimes recognition is what is needed; but often it is more. This chapter examines a wide range of personal, beneficial changes we may seek to derive from knowledge. These changes are implicit in the “become more effective” rectangle in the foregoing figure. One of the major contributors to effectiveness is the development of understanding. We will see that understanding is a concept that goes beyond the individual in that it is a relation between the individual and knowledge (compare Bereiter, 2002a). This chapter does not deal with the innumerable possible outcomes or external consequences of learning, such as promotion and ¹However, our ability to use information is not always proportional to the effort we put it into it. Different measures of memory do not correlate and are hard to predict (Roediger, 2008). This shows that even technical concepts of memory need to be refined (Tulving, 2007).
Psychological contributors to effectiveness
34
other forms of success.² It focuses on internal, individual benefits. The Uses of Information figure above implicitly depicts cognitive productivity and metaeffectiveness. Cognitive productivity can be cast as one’s efficiency and productiveness in passing through the various branches of the figure. Meta-effectiveness is the last branch of cognitive productivity within that figure: our disposition and ability to improve ourselves with knowledge.
2.1 Effectiveness: The master objective Thinking and reading are fluid, amorphous processes. Thus, as we shall see in the next chapter, it is all too easy to process information without deriving any lasting benefit from it. To aim to “learn” is not sufficiently precise or compelling. It doesn’t move us in any specific direction. It does not incline us towards applying knowledge. It is no surprise that effectiveness figures so prominently in some of the most respected self-help books, such as Steven R. Covey’s The 7 Habits of Highly Effective People. To aim for improved effectiveness sets the bar high. It raises the question “How can I use this information to become more effective?” Knowledge resources can be approached as instruments to improve personal effectiveness. We will consider the following categories of learning objectives in terms of effectiveness. • • • • • • • • •
To master objective knowledge, To develop skills, To master norms, To develop attitudes, To develop propensities, habits and other dispositions, To develop mentally, To become more meta-effective (better at using knowledge to improve our effectiveness), To counter cognitive aging, To excel.
Think of the taxonomy as an enticing menu of high-level results you can purchase with your time and mental resources. The categories are all drawn from folk psychology. As such, they are neither precise nor mutually exclusive. However, one cannot transcend folk psychology without reflecting upon it. The educational psychology literature has produced several taxonomies of learning objectives, the most popular of which are the original and revised “Bloom’s taxonomy” (Anderson et al., 2000; Bloom, 1956). My goal is not to convince you to adopt the one I propose here. It uses some of Bloom’s elements; however, it is more closely aligned with the one proposed by Bereiter (2002a). More important than the particular taxonomy itself is to have a schema, i.e., to be reflective about one’s pursuit of excellence. Such a schema should be geared towards effectiveness. Part 2 presents a high-level architecture of the mind in terms of which you may express your learning objectives. For example, you may think in terms of developing specific monitors and specific ²Another way to organize learning purposes is in terms of the types of knowledge that a person might seek to master. See “Objective knowledge”, below.
Psychological contributors to effectiveness
35
long-term working memory, in response to a knowledge resource. In other words, you can map types of effectiveness described below to mindware changes (mental development).
2.2 Mastering objective knowledge When [a Grade 5 student was] asked how she would know when she had learned something, she said: “I think that I can tell if I’ve learned something when I’m able to form substantial theories that seem to fit in with the information that I’ve already got; so it’s not necessarily that I have everything, that I have all the information, but that I’m able to piece things in that make sense and then to form theories on the questions that would all fit together.” Carl Bereiter & Marlene Scardamalia The traditional intellectual purposes of learning from knowledge resources can be lumped together into one over-arching category: to master objective knowledge. This, and cognitive skills, are what Anderson and colleagues mainly consider in their revision of Bloom’s taxonomy of educational objectives. Anderson et al. propose six levels³ of mastery of knowledge: Remembering, understanding, applying, analyzing, evaluating, and creating knowledge. In their view, to fully master knowledge is to be able to competently process it in each of these ways. Their taxonomy is meant to capture the objectives that schools typically focus on. They believe that the lower levels support the higher levels, and are therefore essential, but that schooling as a whole should promote attaining the highest levels. To master the first of the foregoing levels, “remembering”, you need to be able to recognize and recall instances of the knowledge in question. When reading, you will often encounter knowledge that you do wish to utilize in this way. For example, to master Gottman’s relationship theory, you’ll want to know what a harsh-startup is. (It’s mentioned over 30 times in his book.) However, it’s surprising both how little of what we read we remember and how little we need to remember. If you are in any doubt about this, test your recall of documents you have read (e.g., from books, PDFs, or your web browser’s history). Much of what we read at best fades into a kind of background knowledge. Nevertheless, one of the most important features of experts is the amount of knowledge they acquire. Experts recognize, recall and can explain more domain information than others. Their understanding depends upon their memory of domain content and recall skills. I expand on this in chapter 5. In Part 3, I describe ways of instilling long-term memories. The second level of mastery proposed by Anderson and colleagues is to understand what one has read. Of course, there is normally no point in remembering something that you have not understood. In order to meaningfully remember something you need to understand it. Sir Frederick Bartlett, an early cognitive psychologist, argued that remembering is actually more a matter of inferring and reconstructing than retrieving elaborate facts from long-term memory (Bartlett, 1938). The distinction between remembering and other cognitive processes is blurry. ³It, unfortunately, refers to the levels as “dimensions”.
Psychological contributors to effectiveness
36
Given the critical importance of understanding for learning and problem solving, it behooves us to ask, What does it mean to understand knowledge? This is critical because, even professional knowledge workers can run into a couple of noteworthy difficulties. The first one is the illusion of competence. That is to act as if one will be able to remember and understand in the future what one comprehends today. (See “Illusions of Meta-effectiveness” below.) The second issue is that the concept of understanding is a tough one to characterize. The Oxford English Dictionary’s definitions of the word is surprisingly bare. Bereiter’s insightful treatment of understanding, leveraged here, is a demanding read. Max Wertheimer’s posthumously published book, Productive Thinking, presents good examples of understanding. He attempts to characterize productive thinking—thinking with understanding—and to distinguish it from shallower thought. He focuses our attention on some of the core properties of understanding, including that it is driven by a correct understanding of requirements. Later research, alluded to above, demonstrated that when presented with a novel problem, experts spend proportionally more time than others thinking about the requirements of the problem (Sternberg, 1981). That enables them to ensure both that they are addressing the right problem and applying the most pertinent knowledge to it. Ironically, Wertheimer’s understanding of understanding remained largely tacit. He was hampered by his explicit use of concepts from physics—he did not have access to computer science. Freud was similarly hindered in explaining motivation because his theoretical concepts were based on the concepts of his youth—i.e., the steam engine. (What limiting concepts will future scholars describe us as having in our understanding of understanding?) In his book, Knowledge as Design, David Perkins analyzed the concept of understanding knowledge (e.g., a document, theory or concept.) He proposed that to explicitly understand an artifact, one must know: 1. Its purpose (or requirements), meaning the problem that it was designed to solve (or address).⁴ For example, Gottman proposed a bid-processing theory to explain how close interpersonal relationships work, what makes them work well and what tears them apart. 2. Its structure (or architecture), meaning its components and how they relate to each other. For example, the bid-processing theory proposes the concept of bids, bid responses, bid busting, and so on. And it explains the relations between these and other concepts. 3. Its models, meaning examples and instances of the knowledge. Gottman’s books, for example, brim with examples from his research and clinical experience. He also invites you to provide examples of your own experience. If you can’t provide such examples and if you can’t identify them, then you don’t understand his theory. Many people find it difficult to produce examples, which is one reason why conceptual analysis and teaching are both difficult. 4. The arguments that explain how the product addresses its requirements. This can be used to assess the knowledge. The argument must ultimately make sense in a priori terms (i.e., in terms of what one already knows). For example, Gottman & DeClaire (2001)⁵ say Turning toward leads to fewer conflicts, because the partners in a relationship are having the conversations they need to have–the conversations where they demonstrate ⁴Compare chapter 4 of Popper (1979). ⁵http://www.gottman.com/shop/the-relationship-cure/
Psychological contributors to effectiveness
37
their interest and concern for each other. With such high levels of interest expressed there is simply less static in the air. (p. 40). The word “because” introduces the argument. There is no such thing as “static in the air”; but we have an intuitive sense of what that means and how the ideas relate to each other, and so we understand and agree with the point. Explicitly understanding any product—be it a screwdriver, software, the theory of relativity, this book, or any thing else—requires having good answers to those questions. Conversely, one can criticize knowledge with respect to all four of these criteria. As Max Wertheimer emphasized, the most significant error we make in problem solving (including product development) is getting the requirements wrong—i.e., solving the wrong problem. While Perkins’ engineering-based characterization of understanding while useful, is incomplete. I will add to it throughout this book. In chapter 6 I characterize understanding as a relational concept. Understanding is not in your head. It is a set of relations between yourself and that which you understand out there (i.e., objective knowledge). In chapter 11, I introduce the concept of potency of knowledge resources, which I define in terms of its potential to modify your understanding. There, I provide a schema for analyzing concepts that extends Perkins’s schema of understanding. The next section deals with the better part of understanding, which is tacit. Another aspect of mastering objective knowledge, alluded to by the fifth grader in the opening quotation, is the ability to work with the knowledge. To master a theory or concept is to be able to assess, update, edit or repair the knowledge; to create new knowledge with it; to reason with and about it. It means being able to reflect upon the implications of changing parts of the knowledge (e.g., one of the propositions).
2.2.1 Developing implicit understanding The foregoing criteria don’t fully capture what it means to understand something. A good student might be able to remember and utilize knowledge, in the sense that she can accurately talk about it, reason with it and solve typical problems with it, and so on, without being sufficiently knowledgeable. Bloom’s concept of understanding is concerned with statable knowledge and the cognitive skills to use it. Our student might pass typical exams with flying colors and yet unless she had extensive and involved experience with the theory, which involves solving deep and varied problems of understanding over a period of time, she might not have developed sufficient implicit understanding. (Compare Bereiter, 2002.) Implicit understanding refers to a rather amorphous, intuitive understanding that is difficult for one to describe. Bereiter provides an intuitive physics example that I will adapt to illustrate this concept. Suppose you are presented with three different colored cups (one made of pottery, the other crystal and the last plastic), and three different floors (ceramic, linoleum, rubber) and were asked to state for each cup whether it would shatter if dropped onto the floor from a given height. You get to touch each object. You would likely answer these questions well. You intuitively understand brittleness, and you can detect brittleness as a property of crystal, not of plastic. However, if you are like most people, you would have a hard time explaining precisely why this would happen. Even with relevant physics education, you might find it difficult to explain this unless you have
Psychological contributors to effectiveness
38
sufficiently dealt with related problems of understanding and developed an intuitive grasp of the underlying principles. In sum, you might not have connected your statable knowledge with your intuitive knowledge. The current book is not directly concerned with the kind of implicit understanding that arises naturally without processing objective knowledge. However, to understand understanding we must recognize the importance of intuitive understanding that predates understanding any particular conceptual artifact. Prior understanding is always in some respects helpful, but it may of course in some respects be harmful (some of our intuitions can be wrong). We must also recognize that implicit understanding itself can develop. To master a theory means that one can interpret other information in terms of the theory. Implicit understanding is a way of seeing the world and thinking about it that feels natural. It is the result of experience but also leverages innate or prior dispositions⁶. It can be informed by self-directed education and reflection. For example, if you make a habit of applying Gottman’s bidprocessing theory to your daily life, with friends, colleagues and family, you will start to see bid processes at work. You will be able to perceive relationship problems as they unfold in terms of this theory. If you saw them together enough, you would notice that Karen habitually turns away from her partner, Ralph. That is, you would see this behavior as turning away. You would sense Ralph starting to distance himself from Karen. You would see a bid-response pattern at work between the two. Someone else might be blind to this and lack an implicit understanding of what is happening between Ralph and Karen, an understanding that lines up with a statable theory. They would have difficulty prescribing a solution to the problem. Knowledge workers need to be capable of switching from one mode of understanding to another, to deliberately see and interpret the world in different ways. For example, a competent psychologist can switch between seeing Ralph and Karen’s behavior from the perspective of bid-response theory, behaviorism and the Human Cognition and Affect theory. Their shift is not just in how they explain behavior, but in their implicit understanding (i.e., how they perceive the interactions). All knowledge workers must sport eclectic modes of explicit and implicit understanding. To be sure, not every one is easily able to quickly alternate from one statable theory to another (taking a different point of view), let alone from one implicit understanding to another. The ability and tendency to shift themselves entail particular thinking dispositions. Deliberate practice of different theories can help one shift between them. Developing implicit understanding is an important challenge for people who frequently change assignments and domains. Implicit understanding develops slowly. The expectation of “hitting the ground running”—often expressed in job advertisements—is not always realistic. Knowledge workers need ways to accelerate the development of implicit understanding—i.e., to be meta-effective. This book is meant to help you understand and improve your ability to develop implicit understanding from a knowledge resource. Cognitive scientists have found that many students use learning strategies that do not give them the results that they expect. This may also be a risk for knowledge workers with respect to implicit understanding in particular. This is important because implicit understanding is one of the most important benefits we all ought to strive for when we are ⁶Immanuel Kant argued that the human mind is structured in ways that favor understanding (Kant 1787/1905). Thomas R. Shultz provides powerful arguments about the insufficiency of purely empirical explanations of our causal reasoning powers (Shultz, 1982; Shultz, Fisher, Pratt, & Rulf, 1986). See also Pinker (1999) and Sloman (1996, 2009c).
Psychological contributors to effectiveness
39
dealing with very potent knowledge. Developing implicit understanding of knowledge requires that you develop an explicit understanding of its statable core (e.g., some core facts, principles, concepts, and how they work). This in turn requires that you learn how to solve a variety of problems using the knowledge. Varied problem solving will help you detect that the knowledge applies. This needs to be done over a sufficient period of time; otherwise, the understanding will not set and may fade. This is what happens to young people who stop thinking with concepts that they previously understood (e.g., flushing differential calculus from their minds after high school). To emphasize the temporal dimension of understanding, I often use the term “comprehension” to refer to apprehension of recently processed information. I try to reserve “understanding” for a longer lasting state. For additional clarity, I qualify “understanding” with a temporal epithet, such as “lasting”, “enduring” or “transient”.
2.3 Developing skills A major purpose of knowledge processing is to develop skills, or “know how”. Practical knowledge resources describe principles of behavior, rules of conduct, and ways of achieving goals with or without technology. There are such “how to” resources for most goals you want to achieve. Some deal with very simple tasks, such as quitting an iPhone® application, some with very complicated ones. Just about every practical book dealing with complex practical matters is not purely practical. They present arguments to justify their proposals. They also present factual knowledge (as defined in The Computer Revolution in Philosophy⁷.) For example, the Relationship Cure⁸ —a practical book— provides arguments to explain how systematically turning away from bids for connection vitiates relationships. Also, it puts forth concepts—i.e., factual knowledge—such as the concept of bid for connection. Once you comprehend important practical knowledge, you might want to develop skills to enable you to follow the rules and prescriptions. That is, you want to know how to act on the knowledge, untethered from the original documentation. Gottman’s explanations of how to avoid a “harsh startup” with your partner are of little use if you don’t ensure you can apply them before it is too late. You can’t normally excuse yourself from a tense situation with your partner to look up Gottman’s tips for averting an argument. Alas, practical books are neither tutors nor coaches. Some of them, like Gottman’s, contain many training exercises. Most of them don’t. Some provide instructions that won’t necessarily work for you, because they are written for a wide and varied audience. Furthermore, the author is not necessarily an instructional designer. Even in the best of cases, however, it’s up to you to set an explicit goal to master the skills you wish to develop. And then to follow up with the right training regimen. Procedural knowledge is thought by many to be a distinct type of knowledge underlying skills. However, some of us are skeptical of this idea. One reason for this skepticism is that the same information can be used procedurally and declaratively. Another problem is that the term ⁷http://www.cs.bham.ac.uk/research/projects/cogaff/crp/chap2.html ⁸http://www.gottman.com/shop/the-relationship-cure/
Psychological contributors to effectiveness
40
“knowledge” has an objective connotation, meaning that it exists outside the mind. Using the same word for inside and outside the mind is confusing though sometimes unavoidable. The expression “procedural mindware” may help avoid the confusion—to those who know the term “mindware”, meaning the mental mechanisms underlying what one has learned. So, when you set your sights on acquiring a skill, you may want to reflect on the mindware you will need to develop in order to acquire the skills. (Compare chapter 6).
2.4 Mastering norms Another type of effectiveness we seek to derive from knowledge is to understand, adopt and respect or obey norms. These are also known as standards, prescriptions, prohibitions and imperatives. They are often introduced by “should”, “must”, or “ought”. Let’s unpack what it means to adopt standards. Normative knowledge intersects practical, “how to”, knowledge. There are differences in the knowledge itself and occasionally in the mindware that implements it. Whereas both involve rules, norms are different kinds of propositional attitudes. The same proposition can be treated as a prescriptive norm or as a practical rule of thumb. Norms are meant to regulate overt and covert behavior. Norms normally are derived from practical and factual knowledge. For example, a woman might believe she ought to be faithful in a relationship because she believes that her overall happiness is promoted through faithfulness to a worthy spouse. In this case, she adopts the principle as a standard based on her egocentric assessment. She could also, however, adopt the standard based on consideration of a higher-order goal to maximize happiness within her environment. Regardless, what makes knowledge normative is that we treat it as indicating a standard, independently of current instrumental (means-ends) considerations. Norms indicate what is right or wrong. They can conflict with our desires, our attitudes and each other. For example, the aforementioned woman may internalize a standard to be faithful to her husband while finding other men sexually appealing (an attitude) and even wishing to be intimate with them (a motivational state).⁹ Norms can also conflict with higher order motivators and with reality—compare dogmatic religion and contaminated mindware. Norms can, of course, sometimes be revised in light of utilitarian considerations. Psychopaths, and some people with underdeveloped or injured brains, lack the mental machinery to treat normative knowledge differently from instrumental knowledge. Practical books normally contain many normative rules. We sometimes disagree with proposed rules, when they are not self-evident or when we feel that they do not serve the purposes they should. Even then, we may adopt them as standards and we may choose to do so without resentment—i.e., we can adopt rules that we disagree with (e.g., out of respect for others), which further adds to the mental complexity of implementing standards. Knowledge workers process a huge number of norms to guide their conduct. Here are a few popular books that illustrate the preponderance and variety of rules to which we are exposed. • Stephen R. Covey’s The 7 habits of highly effective people contains seven major rules¹⁰ each of which has several derivatives. For example, Covey’s seventh rule is “Sharpen the saw”™ which ⁹See chapters 3 and 4 of Beaudoin (1994) for analyses and designs of goal processing. ¹⁰Covey calls them “habits”. Habits can be expressed as rules to be executed under appropriate conditions.
Psychological contributors to effectiveness
• •
• •
41
means that one ought to regularly “renew the four dimensions of [one’s] nature—physical, spiritual, mental, and social/emotional”. My book explains and suggests ways you can do this by processing and practicing knowledge with cognitive science and technology. Joshua Bloch’s, Effective Java contains 57 rules (plus other sub-rules), such as “Use exceptions only in exceptional circumstances” (Bloch, 2001, p. 169). Chip Conley’s book, Peak: How Great Companies Get Their Mojo from Maslow, contains dozens of prescriptions, including “Practice honest, direct, and regular communication with your investors¹¹” (Conley, 2007). Richard Templar’s, The Rules of Life, contains 100 rules,¹² such as to be flexible in one’s thinking (Templar, 2006). The rules in Gottman and Silver’s The Seven Principles for Making Marriage Work¹³ are not all standards, but the book contains many standards. For example, Gottman implores members of a couple to turn towards their partners instead of against or away and to avoid harsh startups.
These books illustrate the variety of ways in which reasonable rules can be organized. For example, Conley’s prescriptions are organized in a stunning pyramid of pyramids. They can be used to get a sense of the number and complexity of rules we learn as we age. Study the foregoing sources, or survey your own libraries, and ask yourself whether you agree with the following statements. • • • • • • • • • •
Before reading the documents, you already followed some of the standards they propose. You would like to do a better job of applying the rules. You find the standards very compelling. There is not much point in reading such books if you’re not going to adopt some of their worthy standards. Reading standards is not sufficient to actually change your thinking or behavior. It takes time and effort to live up to any individual standard, let alone to adopt dozens of them. You don’t really know what changes in your mind (what mindware is established) when you do successfully manage to abide by a standard. Each author “teaches” standards in his or her own way (which might not work for you). You don’t have a clear system (let alone an algorithm) to ensure that you can productively adopt new standards. You would get more value out of the time you invest processing information if you had a reliable, productive way of implementing new standards.
To adopt a new standard your mind needs to change in many ways. You need to understand the ideas behind the standards. For example, to learn to respond properly to bids, you need to: understand the concept of bid and how it relates to other concepts; to practice perceiving bids and responses in others; etc. Implementing the bid response rule requires that you ¹¹Conley (2007) is a good example of how norms are sometimes inter-related in a network and hierarchy. See the hierarchical figure on his book’s web site and in the book itself. Bloch (2001) also organizes his prescriptions in a thought provoking taxonomy. ¹²Not all the rules in this book are standards, strictly speaking; but many of them contain more than one standard. ¹³http://www.gottman.com/shop/7-principles-for-making-marriage-work-2/
Psychological contributors to effectiveness
42
• learn to monitor your own thinking and motive processing, • detect and deal with conflicts in your own mind (for example, standards often conflict with desires and attitudes), • detect when you are about to behave in ways that violate standards, • learn to respond to situations by setting goals rather than by allowing your previous habitual responses to be enacted, • consider different behaviors that respond to the goals and situation to ensure that they do not conflict with your new standards. Eventually, these internal responses may become automatic (reflex-like). However, we all know that standards are often hard to live up to, even with attention. It is one thing to have a standard come to mind. It is another thing to abide by it in the light of multiple, independent and conflicting sources of motivation. In Part 3, I propose ways of developing: to help you walk the talk of your norms. It will remain, however, a difficult task. Psychologists refer to self-monitoring and self-modification as “self-regulation”.¹⁴ Humans aren’t the only machines that self-regulate. General-purpose computer operating systems have processes called “daemons” that monitor other processes and can interrupt ongoing processes, modify them, start new ones, etc. Daemons can even monitor other daemons. Daemons can run on the same physical computer that they are monitoring, or in networked computers (possibly through “backdoors”). In The Computer Revolution in Philosophy¹⁵, Aaron Sloman illustrates the plausibility and importance of self-monitoring in artificial and natural minds. Artificial Intelligence has proven better than any traditional philosophical argument can that there is nothing mystical or mysterious about self-awareness; it is just another set of design challenges.
2.5 Developing attitudes Occasionally, when processing knowledge, you want to develop new attitudes or change existing ones. By “attitudes”, I mean likes, dislikes, tastes and preferences. There are many things you might want to like more: parts of your job, people you need to work or live with, places you live in, aspects of people including yourself, certain foods, exercise, play, etc. Sometimes, you may even want to like what you want (perhaps because it’s good for you), or dislike what you want (perhaps because it violates some of your standards.) You might not like reading and learning with computers and tablets but you might want to do it anyway, because it’s so much more efficient. Purposefully developing attitudes is easier said than done. This is partly because the concept of attitude is itself difficult to understand. The concept cannot be fully understood without reference to a mechanistic theory of the mind. Part 2 sketches such a theory. Understanding a phenomenon, such as attitudes, is not sufficient to control it. One can understand why only one side of the moon faces the earth without being able to change this fact. Many attitudes, e.g., tastes for certain foods, are quite intransigent. ¹⁴See Kuhl (1992). ¹⁵http://www.cs.bham.ac.uk/research/projects/cogaff/crp/
Psychological contributors to effectiveness
43
However, some attitudes can change as well. Much advertising is predicated on this assumption. Gottman suggests that for a marriage to work, both partners must nurture their fondness and admiration for each other. They need to learn to like each other again and more. Thus, if the world’s foremost marital expert is right—and I believe he is—then people can influence their attitudes. Gottman provides tips in the marital sphere that we can adapt and generalize to other spheres. Social psychologists also have provided principles of attitude change that we can leverage. We will return to the topic of attitude change in chapter 14.
2.6 Developing propensities, habits and other dispositions A [personal disposition], then, is identifiable not by sharp contours or boundaries, but by a nuclear and quality. The nuclear quality will be some important goal or meaning, or sometimes a style of expression. All these betray the individual’s effort at survival and mastery, and give shape or form to his personality. Gordon Allport Alas, one can master objective knowledge, develop skills and comprehend standards without being inclined to use them. In other words, having knowledge does not guarantee that one will act with that knowledge. Such stunted effectiveness— illustrated by Chrissy, Jack, and Ralph in chapter 1— is well documented in the annals of cognitive science. Of course, one need not (and cannot) always apply what one reads, even the high caliber material one agrees with. Yet there is much knowledge that one would like to apply when circumstances call for it. Why don’t people develop habits that execute the knowledge they “have”? • They don’t tend to distinguish between familiarity, ability and propensity. • They don’t tend to use a system to instill abilities for knowledge they have processed (beyond their core work competencies, hobbies and problem solving). • They might not recognize that their knowledge and skills apply to the current case—this could be due to haste, competing interpretations or inadequate training. • They don’t tend to use a system that instills propensities. They lack the propensity to systematically develop propensities. This is a more systematic problem. Each of these bullet items itself covers a range of cases that need to be distinguished. We will only consider some of the distinctions in this section.¹⁶ A disposition is a state that tends to produce a certain behavioral pattern under certain circumstances. The term is used to make causal ascriptions. For example, to say that an object is brittle is to make a dispositional statement. The object is likely to shatter if dropped from a distance on a hard surface, or struck by a hard object with sufficient force. (“Hard” itself is a dispositional concept.) To say that someone is open-minded is to say that she tends to consider different theories ¹⁶Allport (1961) contains a noteworthy analysis of personality traits in relation to habits (and other dispositions). See also Ryle (1949).
Psychological contributors to effectiveness
44
or decision paths when they are presented to her or when they ought to be sought. Being openminded is a thinking disposition. Unlike Janet of Waterflop Co., a lean startup expert, as I described in the introduction, is disposed to develop products in small increments, test them, and learn from her tests. Dispositions don’t necessarily cause the behavior they are associated with. Janet might have a strong disposition to drink coffee but refrain from doing so because it prevents her from sleeping. She (her mind) has self-regulatory mechanisms that override the disposition. Knowing of a disposition, one can make counter-factual statements. One might say “Fortunately, the glass didn’t shatter because the rock was too light.” “Didn’t shatter” is counter-factual, meaning it is not what happened. Or “Janet would drink if she weren’t allergic to it.” Or one might say “Janet had a coffee after the party because she had been drinking.” Drinking removes inhibitions (weakens self-regulatory mechanisms). To think in dispositional terms itself requires certain thinking dispositions, such as to consider the opposite, as we will see below. Propensities are dispositions that one tends to activate or that have a strong motivational component. One wouldn’t normally say that Janet has a propensity to drink alcohol unless she in fact regularly did. But she might have a normally dormant disposition to drink. When people develop brain diseases some of their hidden (and often unconscious) dispositions (including attitudes) may manifest themselves. For example, there was a person who consistently evinced impeccable multicultural attitudes; but with Alzheimer’s expressed great fear of people of a different race.¹⁷ His former selves would have been ashamed. Whereas the term “propensity” is used to focus attention on the subjective impetus to act, the term “habit” is used to draw attention to the behavior itself. To say that Janet is in the habit of drinking draws our attention to her behavior as opposed to its motive. It leaves open, or ought to leave open, the question of what motivates Janet to drink. Perhaps she does it for an instrumental reason, e.g., in order to ingratiate herself with her employer’s executives who have cocktails after work. To say that Janet has a propensity to drink, in contrast, suggests that she has intrinsic reasons to do so. It assigns the locus of control to herself (as influenced by her past execution of the now habitual behavior). When a habit is formed, situations and goals can trigger potentially complex action without requiring careful control from higher order executive (or “management”) mental processes. This distinction has practical implications: one must not merely design systems to instill habits, but propensities. The terms disposition, propensity and habit have a noteworthy history in psychology. The term disposition was not frequently used in cognitive psychology until recently.¹⁸ Now it is most commonly invoked in relation to thinking dispositions (Kirby & Lawson, 2012). As noted in the previous chapter, it is used by Keith Stanovich to explain why smart people fail to solve problems that are within their intellectual grasp: They are cognitive misers, i.e., they lack dispositions to apply their fluid intelligence, knowledge and thinking strategies. They also lack the motivation to improve their cognition. To say that psychologists didn’t study the phenomenon is not to say they didn’t use ¹⁷There is a growing literature on implicit attitudes (Petty, Wheeler, & Tormala, 2003). See Gawronski, Hofmann & Wilbur (2006) for an analysis of the relations between the different types of awareness that people might have about their own attitudes: i.e., of their origins, their existence, and their effects. Part 2 tries to account for the existence of multiple, inconsistent motivational states. See also Minsky (1986). Attitude change is discussed in chapter 14. ¹⁸This may be partly due to its association with philosophical behaviorism in Ryle (1949). The concept has been used extensively by Aaron Sloman since the 1970s (Sloman, 1978). This book further develops the concept.
Psychological contributors to effectiveness
45
the concept. One cannot explain human behavior without it. The concept of habit has a more intriguing history. It was an important part of Aristotle’s psychology. As a result, it permeated western intellectual discourse until the 1800’s. Much of what William James said about habits in the 1800’s remains pertinent today. Clark L. Hull, a behaviorist, invoked mathematical concepts of habits to explain behavior; but he failed to persuade his peers (Boden, 2008 pp. 262-263.) Habits are not easily amenable to rigorous empirical research. These facts, combined with the cognitive revolution in psychology, explain why the concept of habit is rarely invoked by empirical psychologists.¹⁹ The concept of drive had a similar fate. Cognitive scientists, in particular, tend to focus more on classical cognition than on the broader conception that includes affective processes. The name of the field has had a significant impact on problems cognitive scientists consider to be within their professional scope. However, if one wants to understand and influence behavior, one can’t avoid thinking about habits. The term habit keeps being used in ordinary parlance, along with related terms like “tendency”. Several popular books have drawn public attention to the concept of habits. In The 7 Habits of Highly Effective People Covey defined habits as internalized principles and patterns of behaviors. He wrote: While practices are situationally specific, principles are deep, fundamental truths that have universal application. They apply to individuals, to marriages, to families, to private and public organizations of every kind. When these truths are internalized into habits, they empower people to create a wide variety of practices to deal with different situations. (Covey, 2004) In so doing, he departed from the usual concept of habit, which is a manner of behaving in sets of circumstances. His landmark book is arguably more about principles of conduct than habits. Martin Grunburg’s The Habit Factor® induces its readers to consider the importance of developing habits as instruments of achieving their goals. Charles Duhigg’s book, The Power of Habit, like Grunburg’s, draws readers’ attention to the role of cues in triggering habitual behavior. The concept of cue has for over 100 years been leveraged in psychology. Grunburg and Duhigg invoke neuroscience on basal ganglia to make sense of habits. Basal ganglia are highly interconnected nuclei situated at the base of the forebrain. The basal ganglia have been implicated in action selection. (Most of the research has been conducted on non-primates.) While pointing at a brain structure as being essential to a psychological function may provide a comforting feeling of knowing, despite widespread belief to the contrary, it does little to explain that function. One might as well simply say “the brain is essential for that function”. Of course, there is always some brain structure involved in cognition. These popular books on habits draw little from cognitive science. This lacuna is unfortunate because cognitive science is relevant to habits (even though cognitive scientists don’t use the term habit technically.) For example, as William James recognized, habits essentially involve the automation of frequently performed activities. The vast literature on expertise, memory and proceduralization examines this in detail. As William James recognized, ¹⁹Aarts & Dijksterhuis (2000) attribute the fate of habits to additional considerations.
Psychological contributors to effectiveness
46
habitual behavior is of course executed with little supervision by attentional processes. Thus the vast literature on attention, working memory, executive processes and consciousness is also pertinent to habits. In addition, some cognitive scientists have focused specifically on habits. Shallice, Norman and colleagues, for example, have provided an information-processing framework to explain how routine and controlled behaviors are effected and regulated.²⁰ They settled on an architecture that is similar to the one presented in Part 2. (They have modeled complex behavior with a computer program based on their theory.) They assume that the mind contains a rich collection of monitors and feedback mechanisms that detect applicability conditions for actions and that supervise ongoing action. Some of that research is covered in Part 2. In Artificial Intelligence, the role of internal monitors has long been emphasized.²¹ These are mechanisms that observe the internal (mental) and external environments and respond by recording their observations or emitting internal control signals. In order to form new habits, one must somehow grow new monitors that trigger new (internal and possibly external) behavioral patterns. The concept of monitors is central to the framework presented in this book. Developing the tendency to apply the best knowledge we encounter is one of the biggest metaeffectiveness challenges we face. Failing to apply what one knows, i.e., “the transfer problem”, is discussed in the next chapter. It is also one of the biggest problems in education research. The framework presented in this book is my attempt at solving the theoretical and practical problems of transfer.
2.7 Developing mentally In this chapter, I have fleshed out common purposes of learning—to master knowledge and norms, to develop skills, attitudes, and propensities. While these purposes are personal (they are about you, as a learner), they are not expressed in terms of how you want your mind to change. The purposes are expressed mainly from an intentional stance. The intentional stance, as Dennett (1987) described it, is a way of interpreting folk psychological terms, such as “belief” and “desire”, that eschews conjectures about mental or biological states, processes, mechanisms and architecture.²² This stance is compatible with much educational practice, where teachers and institutions set behaviorally measured learning objectives with little concern for information processing mechanisms. Examiners can even measure your attitudes and your ability to self-regulate, based on your performance. However, clearly, in order for externally measured competence to develop, something must change in underlying mental processes. The languages you speak and your culture provide readymade concepts to express some of these internal changes (e.g., in terms of memory, feelings, wants, ²⁰Cooper & Shallice (2000; 2006); Norman & Shallice (1986). ²¹These are sometimes referred to as “daemons”. They have been modeled with production systems, blackboard systems (Hayes-Roth, 1985, 1990)
and procedural reasoning systems (Georgeff & Lansky, 1986). They can also be modelled with formal neural networks. ²²Ryle (1949) and Dennett (1987) argue that folk psychological expressions ought to be interpreted without reference to internal mental states. However, see Park (1994) for a cogent argument that Ryle was not an ontological or logical behaviorist. Ryle acknowledged the existence of mental acts and events. Sloman (2010a) also claims that Ryle was no behaviorist. “[Ryle] discovered the same notion of ‘information processing virtual machine’ as began in later years to emerge in computer science, software engineering and AI, which was also crucial to my 1978 book (and much of the work of Daniel Dennett).” Chapter 2 of Beaudoin (1994) contains a critique of Dennett’s Intentional Stance (Beaudoin, 1994). As Sloman points out, Dennett seems to have distanced himself from the intentional stance and has increasingly adopted a design stance. His recent book on humor for instance is designer-based and emphasizes mental architecture as did Sloman in publications from 1978 onwards.
Psychological contributors to effectiveness
47
and so forth). While this language is error-prone and not scientific, it does reflect immense tacit knowledge about psychology. Thus, some of your learning objectives are mental. You want to be able to notice things, remember things, to think in a certain way, etc. You also have an inner language that uses representations that are not quite verbal—indeed, most of our self-knowledge is non-verbal, tacit or unconscious.²³ This book provides additional concepts from cognitive science which may alter your way of setting your learning objectives. Some of these concepts, on the one hand, are extrasomatic (i.e., distinct from you). For example, knowledge consists of external artifacts (theories, concepts, facts, etc). The concept of understanding proposed by Bereiter and articulated above is partly extrasomatic (it’s about your relations to what you know). On the other hand, many important concepts are mentalistic (e.g., mindware, effectance, internal motivators, filters, and long-term working memory). To avoid confusing these objects with colloquial mental concepts, I sometimes refer to them as mindware (as noted in the previous chapter). If this book is successful, you will begin to set some of your learning objectives in terms of mental development. For example, you might want to grow new motive generators or expand your longterm working memory. This will lead you to think and perhaps speak differently about yourself and others, even beyond the realm of knowledge processing. You can use these concepts, for instance, to interpret your affect in new, and deeper ways.
2.8 Countering cognitive aging Beyond early adulthood, our bodies gradually become less effective. The brain is no exception. Given that the brain is the physical layer of the collection of virtual machines comprising our minds, it stands to reason that some of our mental capabilities gradually deteriorate as we age. This includes our ability to focus in the light of distractions, our working memory functions, our fluid intelligence and our ability to retrieve information from long-term memory in the absence of cues.²⁴ There is a growing market of products claiming to measure cognitive abilities and enhance “cognitive fitness” (SharpBrains, 2013). SharpBrains attempts to educate its customers on the science and technology of cognitive fitness and brain health. It also promotes R&D in the area. The specific effects and effectiveness of each such application needs to be assessed carefully. If technology does have potential to improve brain structures underlying (say) fluid intelligence in a generalizable way, the effort required to achieve the desired effect needs to be considered. Also to be considered is whether the effect is important and desired. The developmental objectives considered in this chapter are all knowledge-based. Expertise rests on factual and practical knowledge. With regard to the purposes of this book, it is noteworthy that current cognitive fitness don’t currently tend to target knowledge-based personal-development. That is, they don’t engage users with expert, domain-relevant knowledge resources. They don’t involve progressive problem-solving in experts’ domains. This is not a criticism of the best R&D in that field. It is simply to point out a gap in the market. It is also an invitation to that industry to explore ²³Aaron Sloman and others have argued that intrapersonal languages are necessary pre-requisites for interpersonal communication. (Fodor, 1975; Sloman, 1979, 2008b) ²⁴See Naveh-Benjamin & Old (2008) and Craik & Salthouse (2008) for reviews of research on cognition and aging.
Psychological contributors to effectiveness
48
applications that enhance meta-effectiveness, training users with the specific content that matters to them. The productive practice software I mention in Part 3 can be interpreted in that light. It engages the user to work with pertinent knowledge. I propose that attempting to improve one’s effectiveness, through deliberate practice, problemsolving and knowledge building, as described below, is a promising non-biological means to counterbalance, if not to counter-act, cognitive aging. One needs to systematically select the best knowledge resources to process, to delve them, and to apply them reflectively. One needs to monitor one’s progress, build new knowledge and engage in progressive problem-solving. Cognitive science has shown beyond reasonable doubt that practicing is a potent, and necessary, route to expertise. I believe productive practice, described in Part 3²⁵, is a reliable way to address one of the most frequent of cognitive complaints, forgetting. To pursue effectiveness through productive practice enables us to build on that which is more resilient to aging, that which in many respects can improve to a much later apotheosis, namely the mindware underlying our understanding, skills, attitudes, habits²⁶ and norms.²⁷ This approach is not predicated on changing that which research in psychology suggests is very hard to change (e.g., fluid intelligence).
2.9 Becoming more meta-effective We also sometimes use knowledge to become better at developing ourselves—to be more metaeffective. Here are two ways in which learning can make you a more meta-effective learner. First, there is the “Matthew effect”: initial advantages have compound benefits (Rigney, 2010). The more you know, the easier it is to learn, because you can use prior knowledge to make sense of new information. This is not just a trite observation about side-effects of learning. It bears directly on the problem of “transfer”. The problem of “transfer” is manifest when people acquire knowledge that is pertinent to solving a problem, but then fail to apply it where they should. Schwartz, Bransford & Sears (2005) argued that in assessing “transfer”, we should consider two benefits of learning. One is that learning-activities can help us directly solve certain problems. That is what psychologists typically focus on when they assess whether subjects have successfully learned something. The other is that learning activities can prepare one for future learning and future knowledge building (including invention and problem solving). I take the latter to be a Matthew effect of learning. A second type of meta-effectiveness is more explicit. Here, one researches, adopts and implements concepts, workflows, heuristics and dispositions to take knowledge as input and develop mentally (mindware) as an output. This type of meta-accelerator is not as intimately bound with the ²⁵Productive practice, discussed in Part 3, systematically combines distributed recall and elaborative practice. Distributed recall practice is an effective means to enhance memory for information across the lifespan (Morris, Fritz, Jackson, Nichol, & Roberts 2005; McDaniel, Einstein, & Jacoby, 2008). ²⁶Whereas this book focuses on the personal application of knowledge, it is important to note that cognitive aging is a complex function of genetics, cognitive activity and biological health. It has been proposed that some “frontal” people naturally are genetically better equipped to age than others (McDaniel et al., 2008). Cognitive misers might also accelerate their cognitive decline, over and beyond the Matthew effect as described above. (This claim is known as the “disuse hypothesis”.) Cognitive processes can be understood as activities of a virtual machine that runs on a physical machine: the brain. Optimal mental functioning is predicated on biological health. See, for example, Section 5 of McDaniel & Bugg (2012) for a modest proposal that aerobic exercise supports cognitive functioning. ²⁷For a science-based but very readable review of memory and aging see Einstein & McDaniel (2004). For a review of cognitive aging in relation to knowledge acquisition and use, see Ackerman (2008). For a more technical review of memory and aging see Naveh-Benjamin & Old (2008).
Psychological contributors to effectiveness
49
content, or domain of knowledge, as the first one. That is, of course, what this entire book is about. Part 2 provides the cognitive science. Part 3 proposes ways of using information technology and cognitive science for meta-effectiveness. The information is designed to be useful for any knowledgeintense attempt to develop yourself.
2.10 Back to the top: Excelling We must not follow those who advise us, being men, to think of human things, and, being mortal, of mortal things, but must, so far as we can, make ourselves immortal, and strain every nerve to live in accordance with the best thing in us; for even if it be small in bulk, much more does it in power and worth surpass everything. Aristotle Philosophy begins, then, with the questioning of certainties in the realm of knowledge and the cultivation of a love of wisdom. Philosophy is erotic not just epistemic. Simon Critchley While it is important to analyze effectiveness, we mustn’t lose sight of our overarching aim. For, in pursuing the various permutations of, interstices between and intersections of the foregoing purposes, we ultimately seek overall personal excellence.²⁸ That is the stuff of inspiriting commencement speeches, designed to spur us on to surpass ourselves with reason, knowledge and compassion. All of this we can consider within the manifold construct of excellence, which is the path to human flourishing first systematically described by Aristotle. Having, in this chapter, analyzed purposes of processing knowledge, we can now wean ourselves further (though not completely) from the hoary expression, “learning”, and speak in terms of developing effectiveness, mindware and excellence. Let us now turn our attention to common impediments to our meta-effectiveness. ²⁸With respect to interstices note that two of the noteworthy aims that this chapter had not explicitly addressed are literary experience and knowing one’s way around. With respect to the former, there is the intrinsic enjoyment of reading high caliber material, or more generally, processing it (Bereiter & Scardamalia, 2012). There is also the implicit understanding (alluded to above) that can arise from processing thousands of documents. The other high level objective is “knowing one’s way around” knowledge, thinking and domains (Perkins, 1995). This is a way in which those of us who try to use the term “intelligence” only in the strict sense (Stanovich, 2009) can interpret the notion that processing (with) knowledge makes one “more intelligent”. These learning objectives can be accommodated in the framework of this chapter. With respect to the intersections, for practical purposes, it is important not to get bogged down by analysis. As I mentioned above, the categories overlap. One could argue, for example, that the better part of developing a skill is to master normative statements. The practical objective here is to improve the language used to set learning objectives, not to develop a perfect theory.
3. Challenges to meta-effectiveness Excessive preoccupation with the practical aspects of investigation, without any thought for the basic problems is fruitless. Hans Selye Before we turn to solutions, we need to understand the obstacles that knowledge workers face in developing themselves with knowledge. In this chapter, I analyze three sets of challenges: • Information technology, • Our circumstances, • Our psychological challenges. Let me start by providing an example of the high-stakes cognitive challenges to which scores of knowledge workers are accustomed. In 1998, after having been an employee of Tundra Semiconductor Corp. since its founding, I became the first employee of a high-tech startup, Abatis Systems Corp.¹ We built products to enable telecom carriers to switch their focus from leasing bandwidth to enabling Internet services. To do this, we needed to enable them to guarantee quality of service for different protocols (e.g., Voice-over-IP). We developed a promising new type of Internet router with network-management software that would allow end-to-end management of Internet services. Our systems would enable their users, for examples, to conduct video conferences or audio-calls over Internet without disruptive variations in network performance. Abatis hired me as a Senior Member of Technical staff. I knew next to nothing about Internet standards. However, having successfully transitioned from Psychology to Artificial Intelligence, to being a professor of Military Psychology, to being a technical writer at Tundra Semiconductor (something I initially knew nothing about), I had proven that I could quickly learn the ropes and that I thrived in startups. I was soon assigned to element management software teams. That type of software monitors and controls individual routers. That meant that I needed to sufficiently comprehend all the routing functionality that could be managed by our users. This in turn required that I sufficiently comprehend the core, and growing, Internet protocol suite, and dozens of related standards. Fortunately, relevant documentation was available on-line. The first few months in that high-pressure job were grueling. Fortunately, the VP R&D (Paul Terry) and Software Manager (John Siu) hired scrupulously. I had the opportunity to learn with brilliant experts in networking technology². The following episode epitomizes the mental atmosphere. In 1998 meetings, Dr. Renwei Li, an ambulant theorem-prover, described his new ¹http://connectedplanetonline.com/mag/telecom_channel_changer/ ²Contributing to this incredible learning experience, both Abatis and Tundra were led by brilliant executives, such as Jim Roche & Rick O’Connor
at Tundra; and Paul Terry, Adam Lorant and John Seminerio at Abatis. Three of them were named in Canada’s “Top 40 under 40” by Report on Business Magazine (2001). These two startups were part of a family of companies co-founded by Sir Terry Mathews and Newbridge Networks Corporation. Tundra’s market capitalized soared to over $1 billion; Abatis was acquired for about $1.3 billion, the largest take over of a privately held Canadian company.
Challenges to meta-effectiveness
51
algorithms aimed at our core technical challenges—to classify IP data packets at wire speed. The meetings were one-against-many intellectual ping-pong matches. The team smashed questions at Li who instantly responded. It was a mentally demanding and exhilarating experience. After several months of effectant processing, I was in my own element, writing technical specifications, developing software and at various times leading teams of developers. I experienced what people in R&D routinely experience: being thrown into high-pressure situations where they must rapidly learn and produce. I discovered a couple of problems at Abatis that are particularly relevant to the problem of this book. First, despite the fact that we were equipped with top-end computer monitors, many of us chose to print reams of documents and read them from paper. I began to wonder why we preferred print. When I left Abatis in 2001, I set out to discover how to make it preferable to read and learn with technology. I have been on that journey ever since. While information technology has come a long way since then, many of the requirements I identified early on have yet to be widely addressed. However, one can kludge together useful workflows, as I will show in Part 3. Second, I was struck that most technical staff were not very familiar with cognitive science. Cognitive scientists had evidently not done a good enough job of exporting their knowledge. This situation seemed analogous to building bridges without knowing Newtonian mechanics. Or had cognitive science nothing useful to offer to the brightest minds? To be sure, cognitive science is still mainly a factual discipline. Most cognitive science research is conducted on students. That population is much more variable than knowledge workers (e.g., in terms of intelligence and thinking dispositions). Still, I felt that the potential of cognitive science was left untapped. When psychologically-oriented workshops and training are given in the workplace, they usually deal with social, affective, and communication issues rather than (classical) cognition. When cognition is presented in industry, the information is often superficial and uninformative for knowledge workers, people for whom learning is an intrinsic part of work; sometimes it is simply false. Often it is mislabeled as “brain training”. For example, the workshops sometimes make such superficial recommendations as to take frequent breaks, study actively, listen to Baroque music, eat fish and dark produce and consider oneself to be intellectually gifted (Cullen, 2010; Jones, 2005; Small, 2008). Their vague recommendations to “study actively” could be substantiated with concrete indications about how to use technology to overcome specific challenges to “active study”. However, their recommendations typically skirt core information-processing and technology challenges altogether. Carl Bereiter summed up current “thinking strategy” offerings: The teaching of thinking strategies, although motivated by contemporary research, still relies mostly on stepwise procedures and slogans and could as easily have been designed 50 years ago. (2002, p. 349) The road to thinking skills and creativity is lined with quacks. (2002, p. 413) If we are to help professionals productively develop themselves with knowledge, firstly, we had better understand their challenges in relation to high-caliber findings and theories about cognition. Our information and advice should reflect the fact that professional knowledge workers are different. For example, they are of above-average intelligence. They are highly educated. Reading
Challenges to meta-effectiveness
52
is a significant part of their work. Many of them enjoy learning and are effectant. They tend to use technology to research and learn. This chapter deals with their challenges. Secondly, we had better extend and apply cognitive science to their ends. Thirdly, we ought to provide specific guidance for using information technology. Parts 2 and 3 do some of this.
3.1 Information technology: Lack of support for cognitive productivity This section considers three major respects in which information technology fails to adequately support the cognitive productivity needs of knowledge workers. • Information processing tools are designed for surfing, not for delving. • There is no systematic support for productive practice and instilling concepts. • Syncing and accessing information across IT devices is still a problem.
3.1.1 Tools designed for surfing, not delving We haven’t really improved education with technology Steve Jobs (quoted by Bill Gates) In chapter 1, I summarized Nicolas Carr’s influential concerns about information technology. In this section, I discuss additional respects in which technology makes it difficult for us to process knowledge productively. This analysis is designed to help you • • • • • •
understand your meta-effectiveness requirements, discover new ways of working with and around technology, realize that some of the problems you have with technology are due to flaws in technology, discover, even if you are geek, that your systems are suboptimal in ways you had not realized, develop a productive, “can do” attitude towards technology, request more meta-effective OS and applications from manufacturers.
I mean to reinforce a pro-active attitude opposed to the defeatism of those who lament the supposed effects of technology upon us. To assess and design technology for reading we should compare it with the main competitor, books! Two of the most striking features of learning with books are (1) the speed with which readers can navigate them (in principle); (2) the ease with which readers can mark them up. When I need to devour an important and complex book, I often spend a few minutes setting it up. I apply colorcoded, plastic Post-it flags™ to the table of contents, each chapter and the index the glossary. (E.g,. green flags pages with important concepts.) I write the chapter number and a bit of its title on the flag. This small investment in time enables me to quickly navigate the book. However, neither books nor stationary are ideally designed. It’s hard to write on plastic Post-it® flags. Most books
Challenges to meta-effectiveness
53
do not “lay flat”. So, one often has to use a book holder, such as the Book Gem³®. Margins seem to be getting smaller, which makes annotating harder. Publishers almost never apply bleeds (to facilitate navigation). So, information technology is not the only thing holding back our cognitive productivity.
3.1.2 Inadequate support for annotation We believe that it’s technology married with the humanities that yields us the result that makes our heart sing. Steve Jobs We encourage “active reading” in our children and students. But let’s get our concepts and terms straight so that we can solve the correct problems. What we actually mean, or ought to mean, is to encourage productive information-processing in them and in ourselves. By “productive”, I don’t just mean efficient, I mean thinking that produces effective mindware, understanding, knowledge and other products.⁴ Information-processing includes not only reading, but also processing other sources of information: videos, podcasts, audiobooks, screencasts, traditional paper books, conversations, meetings, etc. Unfortunately, as I proposed in a white paper to Steve Jobs in 2010, current IT encourages passive processing of data, which is not conducive to education and cognitive productivity. It encourages passive processing of data. The expressions “web browsing” and “web surfing” both adequately reflect, and may even have influenced, the design of information-processing software. Web browsers are designed for skimming. Observe yourself and others using a web browser. You’ll see that users spend little time expressing and recording their goals, assessments, thoughts, etc. One would think that technology would actually make it easy to do that. But in many respects they do not. Web browsers are not much better than television for active learning. The mouse and keyboard have become glorified remote controls. Users often fail to produce information when they should. When users write in web browsers, it tends to be for communication as opposed to elaboration. As I mentioned in the introduction, it is not sufficient for the big IT developers to prize humanities and the arts. The design of an ecosystem of IT products needs to be informed by cognitive science with the objective of helping people improve themselves. There have been improvements since early 2010; however, important steps still need to be taken. They are not technically challenging. To assess or design information-processing software, we must not only ask “What do users want to do with information?” but “What should they be able to do with it?” Jobs claimed that “customers don’t know what they want until we’ve shown them” (Isaacson, 2011). To know what to show them, one must study their requirements. Sharon Bratt introduced the concept of pedagogical utility: the extent to which software facilitates instructional design, teaching and learning (Bratt, 2007, 2009). She also developed an instrument to systematically measure pedagogical utility. While this instrument was specifically ³http://www.bookgem.com ⁴Schwartz, Bransford and Sears (2005) characterized the trade-offs between efficiency and adaptability in innovation. The trade-offs apply to the
development of personal effectiveness as well. The authors draw our attention to the fact that one of the major benefits of learning is to prepare one for future learning, as opposed to directly facilitating current or future performance.
Challenges to meta-effectiveness
54
designed for web-based learning, her framework can be adapted to measure all kinds of learning environments. I propose the concept of cognitive potency which is a subset of pedagogical utility.⁵ It is the extent to which software makes it easy for users to comprehend and utilize information, or more generally to develop themselves with it. Knowledge workers have a need for cognitively potent tools.
3.1.3 The need to annotate entire resources There are many studies of expert reading⁶ that are pertinent to the foregoing questions. For example, we know that top performance is predicated on goal setting and on monitoring progress towards goals. Therefore, approaching a document, one should be able to set and record one’s objectives. One could then more readily assess how well software has helped one set objectives. As a side effect, this might help us become increasingly sophisticated judges of the potency of information. Our most precious resource is our time—our cognitively productive time, viz. our attention. It’s easy to waste time reading pointless documents. Annotation systems can help make us more discriminating and hence more productive. Software should therefore also make it easy to assess documents. Software developers should resist the temptation of simply providing us with a 5-star rating of documents. Information can be assessed in multiple dimensions. Not all documents call for the same dimensions. Consider the following reasons why a document might be particularly “good”: • • • • • •
It is well argued, It has good examples, It is pertinent to a certain project, It has potent concepts, Its prose is eloquent, Its bibliography is stunning.
We can become more productive information processors by practicing making such discriminations. I’m not saying that one should assess documents along all these dimensions. It suffices to tag a document according to how it stands out the most. Chapter 11, describes the process of assessing documents in more detail. One might argue that making mental notes ought to suffice. However, we have not evolved to cope with the massive amounts of information we now process; but well-designed technology that allow us to make and record elaborative judgments can help. Annotations would also make the documents easier to find. “What was that document I read that was so well argued?” is not a question that Apple’s Spotlight® or Google® can (currently) answer. We don’t need to wait for large software-manufacturers to solve such difficult Artificial Intelligence problems. More elaborate, thoughtful user-friendly rating systems can and should be designed. ⁵The concept of potency is specified in “P: Gauge its potency” in chapter 11. See also Why and how to use potent cognitive-tools”. ⁶See Pressley & Afflerbach (1995) and Pressley & Gaskins (2006) for reviews.
Challenges to meta-effectiveness
55
3.1.4 The need for synchronized annotation services When we process a document productively, we don’t just see words on a page, we categorize, draw inferences, make predictions, assess and so on. We notice what is new, surprising or potentially useful. As we process information, we often make a mental or physical note of the following: • • • • • • • • • • • • •
The problem or objective of the document, important information, key terms and concepts, principles, references, hypotheses, methodology, facts and findings, things we disagree with, things that are new and surprising, something funny, examples, counter-arguments.
With paper documents, some of us highlight, underline, add margin notes (sometimes with shorthand), attach stickies, and entire documents of our own. Software ought to enable us not merely to highlight with a palette of colors but to tag snips of text, audio, and video in this way. This would help us to focus our attention on the snips of information that matter most. And let us be clear: of the exabytes of information on the net, the information that matters most to you when you have a problem to solve is normally the information you have already read and that has stood out in some way. To be more precise: it is the content you have read and about which you have made a note. It is unrealistic to expect us to remember enough of what we’ve read without relying on such annotations.⁷ Fine-grained tagging (as opposed to tagging entire documents) would also help us summarize, understand, assess, and remember what we read. It would help us to build new documents, new knowledge and new products from information. For example, having read a paper, within a few clicks you could list its important concepts. You could find the bits that you disagree with. Most e-reader software does not adequately enable us to do this. To be sure, some applications, such as Preview, Apple’s PDF reader, do allow us to make basic annotations, such as to: • highlight text (some even allow us to choose colors)⁸, • insert margin notes, ⁷There is an important trend in cognitive science concerning “the extended mind” that is relevant to this issue. The idea is that computers, documents and other artifacts extend our minds (Clark & Chalmers, 1998). For these extensions to be mind-like, we need to be able to index them. Certainly, our minds have sophisticated indexing mechanisms (Sloman, 1978). ⁸Amazon’s Kindle® for Mac as of 2013-04-18 still does not yet support multi-color highlights. See my longish post “What’s Wrong with the Kindle App: A Knowledge Delver’s Perspective”. Adobe Reader’s color highlighting function is tedious to use.
Challenges to meta-effectiveness
56
• insert graphics, • insert bookmarks. One can co-opt highlighting features to tag text, but that is a tedious hack. We would have to manually keep track of the meaning of colors (e.g., green is for terms, yellow for important). We couldn’t easily find all of the annotations of a particular (virtual) tag, within, let alone across documents. There are better ways to co-opt existing software for these purposes, some of which I will describe in Part 3. (At that point, I will show how a Mac PDF reader, Skim, stands out as a more useful reference application.) But annotation should be handled at the level of operating and ecosystem.⁹ If annotation services were provided in the operating system vendor’s development kits, we could easily enable and use them. There is virtually no annotation support in web browsers, yet the most common format for expository information is HTML! On a Mac, the major PDF apps have annotation functions (e.g., Preview, Skim and PDFPen). But to annotate a web page or an email message, one is left to his own devices. One should be able to create and add notes to web pages, PDF files, emails and more using the same consistent interface. The same keyboard shortcuts should be available for adding notes in any application. All of one’s notes and snips should be stored on one’s desktop computer, optionally synced across devices, and available for searching and browsing. That means there needs to be a type of application not currently provided (by Apple): an annotation browser. In order to annotate text, one has to make use of third-party software. Thus, as it stands, major OS vendors like Apple are effectively delegating annotation services to third parties. The delegation model hinders our cognitive productivity and endangers our privacy. Users who want to annotate are left to their own devices—to discover, assess, subscribe to and manage annotation services; they must deal with complexity and inconsistency. For example, the popular web application service, Diigo, allows users to highlight and annotate web pages. Web annotation services store our annotation data on remote servers. These services sometimes become temporarily inaccessible.¹⁰ Consider that our web annotations may contain some of our most important intellectual property (or the property of our employers). Should we trust small players to securely store information about the websites we visit? What if a competitor or a hostile foreign government purchased the web app company? What if the web annotation company goes out of business, as many of them have? Or what if the hacked contents are posted somewhere on the Internet? What if you want to access your annotations without Internet access? Users have complained of Diigo’s privacy rules, defaulting to public. Privacy-conscious users need to frequently check their web clippings to ensure that they are not publicly visible. I suspect many of them don’t realize that their date- and name-stamped clippings are public. Some employers do not want confidential data stored in the United States due to privacy laws, regulations or policies. These are just some of the drawbacks of web annotation services. The delegation model has led to a proliferation of third-party applications (web applications, mobile applications and desktop applications) and services. Even for cognitive productivity experts, ⁹By ecosystem, I mean across devices. When you change your tag colors or perform annotations, the changes should propagate across devices, as I mentioned in Beaudoin (2010a). ¹⁰This can be verified by looking at the history of http://en.wikipedia.org/wiki/Web_annotation. Consider that Google itself ventured into this space. Imagine the inconvenience of losing months or years worth of notes.
Challenges to meta-effectiveness
57
it is difficult to keep track of the various products. Moreover, the cognitive load of using diverse applications is staggering. Each application has its own user-interface and stores its data on its own servers or in an opaque database. Most such applications only cater to a subset of data formats: ebooks, PDFs, web pages, QuickTime, Microsoft® Word. For example, Diigo® does not currently process PDFs on desktops or multimedia. Diigo’s main competitor, Delicious®, has only recently added highlighting. Notes we create are sequestered. They can’t normally be searched, browsed or edited outside of applications designed to process the entire document.¹¹ That means (again) they each have their idiosyncratic user interface, each with its own limitations. Some delegates, such as EverNote®,¹² store your data in their opaque databases which prevent you from using convenient file system tools (such as the Finder or Windows Explorer) to peruse them. That is a huge penalty. The applications become the custodians of your data. The more data you store in them, the more dependent you become. They effectively control network access to your own data! If they decide to increase their fees in the future, or if they get purchased by a rival OS vendor, or if your own budget suddenly shrinks, you might find yourself in a very uncomfortable position. Fortunately, file management apps like Ironic Software’s Yep, BareBones Software’s Yojimbo and C-Command Software’s EagleFiler, and Mekentosj Papers do not have this problem.¹³ These apps store your data in regular files and folders on your local storage media. The apps provide ways of organizing and accessing your data. You can also access the data without these apps. The text editors provided by these applications are quite limited. Most of them do not even allow one to write one’s notes with styled text (underlining, italics).¹⁴ Nor do they support outlining. Those that support graphical annotation each have their own, limited tool kit. One’s annotations are only accessible through these applications rather than via an Annotation Management System. One must learn each application’s note-editing user interfaces (many of which are very inconvenient)¹⁵. As I argue in Part 3, outliners are very powerful thinking tools. We want our children and students to be active learners. Should we not expect technology to make it even easier for them and us to be productive learners? We need powerful tools for formulating and expressing our thoughts about information. That means: outlining and diagramming software. Cognitive scientists have documented benefits of such active processing. Take the “generation effect”, for example. We tend to better remember information that we have generated ourselves (our thoughts and impressions) than external information we experience.¹⁶ This suggests we should ¹¹A note added to a Microsoft Word® 2008 for Mac document, for example, can be accessed by LibreOffice. If you want to quickly view, outside of an app like Word or LibreOffice, a list of all your notes attached to a Word document, and edit them, you’re out of luck (unless you’re willing to write software). Skim.app has an option to store notes in a separate file. But there is not yet an application to edit those notes in that separate window. ¹²I realize that the tools I am describing here are very popular with many of their users. I am not suggesting that they should not be used. I use some of them myself. I am drawing your attention to some of their drawbacks so that you can design and appreciate work-arounds. ¹³The powerful DEVONthink application has an option to index files outside of its priority database. Unlike EverNote, DEVONthink does not tie its users to an online storage service. ¹⁴Here are some examples of Mac applications whose annotations are just plain text: Skim, Preview, Mendeley. ¹⁵Contrast Skim’s fairly sophisticated annotation capabilities with the one provided by Mendeley, which amongst other problems, does not allow you to view more than one PDF file at a time. ¹⁶Slamecka & Graf (1978) described it thus “Most of us have probably encountered the informally expressed sentiment that there is an especial advantage to learning by doing, or that some kind of active or effortful involvement of the person in the learning process is more beneficial than merely passive reception of the same information.” They provided a stunning demonstration of this phenomenon. A meta-analysis of papers published from 1966 to 2006 involving over 17,000 subjects showed the phenomenon to be relatively robust though not without exception (Bertsch, Pesta, Wiscott, & McDaniel, 2007).
Challenges to meta-effectiveness
58
reformulate information that matters to us. This in turn calls for powerful editing tools. One could counter-argue that these tools already exist. They are called word processors and outliners. All we have to do is associate a rich-text document with any resource that we wish to annotate. This suggestion is a step in the right direction. However, it reveals yet another way in which modern operating systems fail to meet our cognitive needs. The flaw requires that we properly understand annotation. Most of us who think of annotation at all tend to think in terms of notes and highlights. Above, I added global and inner tags to this list. In 2002, I proposed that annotation be conceived generally as linking information to information.¹⁷ Typically, one links existing source, such as an entire web page, or a snippet of text, to new information, such as a textual note or diagram. Sometimes, however, we need to link two existing information items together. For example, you might want to link a web page to an email message that describes it. Linking could be done automatically. For example, when watching a video, you should be able to rapidly view all the documents you have created that reference it. It is important to be able to annotate all kinds of resources, not just documents: Audiobooks, podcasts, screencasts, videos, live webinars and audio conferences, and more. Yet most of the apps we use to process this information are rudimentary. The trend towards minimalism (taken to new levels in Apple’s QuickTime® Player 10) has perhaps gone too far. It would be very useful, while listening to a podcast, for example, to be able to add textual, graphical or even voice annotations¹⁸ that would sync across all of one’s devices. We should be able to annotate portions of images and videos. One should be able to create summaries of the video, simply by indicating the parts that matter to us. One should later be able to peruse one’s text annotations, touch or click them, and have the software bring us back to the exact part of the podcast that is of interest. Or to play a summary, containing the tagged parts (as a function of the selected tag.) Such linking capability is essential for multimedia, because we rarely have the luxury of listening to the same audio file twice in its entirety. When we do want to revisit a media file, it is very important to be able to jump precisely to the location that contains information that we previously identified. There is a general principle of retro-relevance: Out of all the information in the world, the most relevant is what you have already processed. This of course assumes that you are somewhat discriminating in your choice of information. So it needs to be easily accessible and well indexed. This applies across media. With this linking capability, it would be easier for users to annotate resources using the word processing and graphical tools of their choice (e.g., OmniGroup’s OmniOutliner and OmniGraffle.) As it stands, every user needs to invent their own system—or be trained on one—to link web pages (and other resources) to their own files about them. But with proper operating system support, one could easily create and access an OmniOutliner document for any resource. For example, from a web page, PDF file or email document, with a single click of the mouse or a keyboard shortcut, you ¹⁷Beaudoin (2002). See also Beaudoin & Winne (2009, 2010). Technically, this requires a language to express resource locations (including URLs) and indexing information with them (e.g., starting and ending character positions at each end of the link.) Links must be as robust as possible, such that they can be resolved even if the source and end documents change after the annotation has been forged. For example, the source or target document might be moved from one folder to another, or its text might be updated. In R&D settings, I led the software development of three personal learning environments that supported such annotation (2002-2010). One of our prototype applications in 2004 even supported fine-grained annotating of video, images and HTML. ¹⁸Of course, these voice annotations should be converted to text automatically.
Challenges to meta-effectiveness
59
could access a new (or existing) meta-document. In this meta-document, you could express your thoughts about the document. You could plan your reading objectives, describe the document, assess the document, etc. You should also be able to access any fine-grained annotation of the document (e.g., highlights) from within the context of the meta-document. Annotation is nearly as important to cognitive productivity as “copy” and “paste” is to general productivity. Part 3 describes a system that addresses these requirements.
3.1.5 Where’s the productive practice app? On the basis of research on expertise, cognitive skill acquisition, memory and educational psychology, I argue in Part 3 that to develop oneself, it helps to engage in regular productive practice. Reading, and rereading to the point of temporarily comprehending complex information is usually not sufficient to guarantee long lasting understanding. Normally, one should practice using, exploring,¹⁹ and utilizing the information. Thus, as one processes information, one would determine what part of the information, if any, one wishes to master. Beyond tagging information, users should be able to easily add knowledge gems to an application that will then train them with it. The training generally consists of posing and answering questions about and with the information, over an expanding period of time. The applications that come closest to being useful for productive practice are flashcard apps. A traditional (paper) flashcard typically contains a question on one side and an answer on the other. As they study a textbook chapter, some students create questions they need to be able to answer in order to master the material. They create as many questions as they feel are required to achieve the level of mastery they seek. These flashcards form a deck. Students then practice answering the questions. Some feel that flash-card learning is necessarily rote learning. However, as we will see in chapters 13 and 14, the principles underlying its usefulness can be leveraged in a highly effective selfdevelopment practice (productive practice). With productive practice one can predict and control one’s learning outcomes. The process of formulating questions itself helps one to understand the material and to determine the relative importance of the ideas it conveys. Developing answers is obviously quite useful. And practicing over time helps students discover what they don’t know and to bridge their knowledge gaps. We will explore other benefits of this type of practice below. There is a dazzling array of flashcard applications to choose from. In 2012, a website²⁰ cataloged 167 flashcard applications, just for iOS devices. Users can purchase flashcard applications for just about any subject. There are online²¹, desktop and smartphone apps such as for users to create and share flashcard decks. Nearly two years after my suggestions that Apple should support practicing throughout their software ecosystem, they added it in a limited form to iBooks® and iBooks Author®. Unfortunately, the former does not support adding flashcards. This means that learners are limited to the flashcards provided by content developers. All users should be able to add flashcards to whatever content they are trying to learn from. ¹⁹Compare the geographical metaphor of intelligence as “knowing your way around” (Perkins, 1995) that I alluded to at the end of the previous chapter and discussed in chapter 13. ²⁰http://www.flashcardapps.info/filter/sounds/leitner/ ²¹http://studyblue.com
Challenges to meta-effectiveness
60
Flashcard applications currently suffer from a number of serious limitations. The most significant problem is the same as the one that plagues annotation software: The delegation model. In order to qualify as a productive practice system—instead of merely being a flashcard app—software must be reasonably integrated with all of the applications and content one wishes to master. It should not matter whether you are reading a PDF file, ebook, or webpage; listening to an audiobook, podcast, lecture, or conference call; viewing a screen cast; participating in a webinar; or simply seeking to capitalize on your own insights. Nor should it matter whether you are interacting with a smartphone, a tablet or a computer. In either case, you should be able to use the same workflow, and a similar interface, to annotate incoming information and to extract knowledge gems. These knowledge gems can then be added to your productive-practice database, so that you can practice with them at a later time. You should be able to link flashcards—in Part 3, I call them challenges—to the resources containing answers. So, when practicing, if the current card pertains to a resource, one click or gesture should bring you to that resource in the application of your choice (e.g., for PDF files, the PDF reader of your choice). Ditto for all the other formats. Otherwise, practice software will not be truly productive in the efficiency sense. Another problem with most flashcards apps is that they are designed by technology enthusiasts without consideration for cognitive science. Conversely cognitive scientists themselves have yet to deploy productive practice systems in authentic settings from which they could collect large amounts of data to advance and test their theories. There are too many other issues with flashcard software to mention here. If you are like most knowledge workers, your need to become ever more effective is so pressing that you can’t afford to waste potential learning time while commuting. Perhaps you listen to podcasts or audiobooks on the road. But beyond processing incoming information, we must also spend time practicing with knowledge gems (given that practice is more potent than listening). Therefore, like annotation systems, productive practice systems need to leverage software and data across all their devices, mobile and desktop. Your learning data²² must be synchronized, such that you can seamlessly switch from one device to another without losing a beat. You would practice on your smartphone on the go. Switch to your tablet or computer at will. Add questions on one device, and use them on another. Let the application convert your questions to audio form so that you can practice eyes-free. This is not yet supported but with cloud storage services (and peer-to-peer equivalents, such as BitTorrent Sync®) it is just a matter of time before such mobile cognitiveproductivity apps are available.²³
3.1.6 Where’s the glossary manager and instiller? To take on new jobs, knowledge workers need a healthy appetite for new vocabulary and acronym soup. It’s not just a matter of learning new words for existing concepts. Terminology maps to a complex web of concepts and relations. You would think that an application specifically designed for this purpose would exist and ship with every computer, tablet and smartphone. But knowledge workers are left to their own devices. ²²I.e., information about your practicing. ²³My colleagues and I at CogZest and CogSci Apps Corp. have developed a prototype mobile productive practice app that I’ve been using for
several years.
Challenges to meta-effectiveness
61
You must be your own terminology curator. Glossaries are scattered across many documents. As you read technical documents, somehow you must learn your way around the corpus of documents. Perhaps you keep your own glossary. Search engines make it easy to look up a term’s definition. They can even help you find the technical term you are looking for. But even when the information is well organized and presented, the principle of productive practice applies: Reading is not a sufficient or efficient way to master information. One must productively practice information to master it. And master we must. For one can’t normally excuse oneself from a meeting to look-up the meaning of our interlocutors’ terms, or to find the acronym that describes that “What-ch’you Mc-Call it” one one wants to refer to. This is not a matter of image management. One can’t expect to know everything. No system will be perfect. Nor can we always anticipate what we will need to know when. Nevertheless, knowledge workers must try to stay on top of pertinent knowledge. According to what I call the two-strike principle, it may be acceptable to fail to produce a knowledge gem the first time one is called upon to use it. However, if I am “tested” again, I ought to succeed. In the major leagues of knowledge work, two strikes is not acceptable. One needs a reliable software-based system to master knowledge gems. Here is how it could work. Suppose you need the term “Thunderbolt interface” in a meeting with a client, but can’t produce it. After the meeting, you would capture this knowledge gem. Assuming you use the software regularly, the software would ensure that you master it, along with other knowledge gems. The next time you need the new word for “Light Peak”, “Thunderbolt” will come to mind. If the next time, you fail because you have not added it to your system (or you do not have a system), then you have struck out. Of course, it’s even better to be pro-active and capture knowledge gems before you need to use them in the first place. So, wouldn’t it be better if we had tools that were designed to help us not merely find and experience information, but master it? When you open a PDF document, your PDF reader could in principle detect which terms and concepts are new to you, based on your prior reading. When you look up a word in a dictionary, or in any resource for that matter, you should be able to add it to your productive practice app’s database with minimal effort. The app would then take care of training you, so long as you regularly use it. There is more to mastering gems than remembering them, as we saw in the previous chapter. Often, it is not even necessary to explicitly remember the information. It’s more important to perceive bids than to recall that they are referred to as bids, for example. We need productive practice technology that helps us produce different types of mindware, including monitors that detect when knowledge is applicable. See “Master concepts and vocabulary”, below.
3.1.7 Drawbacks of smartphones and tablets Smartphones and tablets have many benefits. Reading on a tablet is often a more comfortable experience than reading on a laptop monitor or a large LED display. However, mobile technology has also introduced major headaches. As of this writing, syncing files across devices with different operating systems (e.g., iOS and OS X) is still time consuming and error prone. It is difficult to ensure that information created or accessed on one device is available on the others. However, because syncing technology is changing so rapidly, I have chosen not to address it in detail in this
Challenges to meta-effectiveness
62
book. Accessing files on tablets and smartphones is slow compared to computers. Unlike computer operating systems, iOS does not expose a device-wide file system. Technology for delving on smartphones and tablets is not as cognitively potent as what is available for computers. Small screens make it difficult to read and annotate.
3.1.8 Conclusion So, in some respects we are held back by technology. However, I do not take a defeatist stance. Technology is not hardwiring our brains. The potential to optimize our learning is greater than ever before. We need to use technology and information productively to become more effective. To do this, however, we must come to grips with other challenges to our meta-effectiveness: our circumstances and psychological fallibilities.
3.2 Challenging circumstances Let’s consider three ways in which our circumstances interfere with our meta-effectiveness: the demands on our time, the accessibility and presentation of information, and lack of cognitive productivity training.
3.2.1 Demands on our time [Nelson Mandela] actually said sometimes in the later years that he missed prison. I would say, ‘how could you think that?’ but he would say that at least in prison he had time to think. He was so overwhelmed by the world, he found it very difficult to have quiet time to just think and contemplate things. Zelda la Grange²⁴ We professional knowledge workers often find ourselves under potentially stultifying time pressures. Management and customers are demanding. Knowledge work can be very competitive. We are promoted and assigned new responsibilities. Some of us have family who demand our time and attention. At home, we are the IT experts, repeatedly called upon to troubleshoot and educate. Nowadays, each of our techno-dependents has multiple devices to manage. Contrary to popular belief, young people—even university students—are not normally particularly adept at using technology for learning.²⁵ So the IT burden on knowledge workers at home is significant. It’s even heavier when the family relies upon a poorly designed operating system and software. Technology interrupts and distracts us. Email, social networking services, text messaging, audiomessaging, cell phones, land phones, etc. In addition, we sometimes have to deal with multiple, poorly designed websites. Many web pages are littered with advertisements. The IT burden, interruptions and distractions do not merely consume time, they fatigue us in ways that many socio-recreational activities do not. Again, these problems are not “rewiring our ²⁴http://www.optimamagazine.co.uk/read/leisure/books/1162-memories-of-mandela ²⁵See DiLullo, McGee & Kriebel (2011), Jones (2012), and Littlejohn, Beetham & McGill (2012).
Challenges to meta-effectiveness
63
brains”. We can address them by using the best technology available, helping our IT dependents to become independent, and using productive information processing strategies. For many knowledge workers, ironically, the major obstacle to their professional development is their work. Their employers or circumstances keep them so narrowly focused on short-term deliverables that they feel they don’t have enough time left over to improve and prepare themselves for the future. That is an exasperating experience for effectant people. It can lead to frustration, depression and turnover. It can also lead to obsolescence. To develop their full potential, knowledge workers need to ensure that they create sufficient opportunities to meaningfully and efficiently research and develop.
3.2.2 Sequestered and ill-presented information Ideally, a knowledge worker has easy access to affordable, abundant, high quality, relevant knowledge (chapter 12, spells out quality criteria). The open access²⁶ movement aims to provide unrestricted access via the Internet to peer-reviewed scholarly journal articles. (Why shouldn’t tax payers and donors have access to the research they fund?) Alas, most scholarly information is not available without charge. Universities and some other organizations pick up the tab for their lucky members. Those who do not have access to scholarly research are at a significant disadvantage. They might not even know that the relevant information exists. (Google Scholar, for example, has not indexed every document.) Search engines have improved dramatically over the years, but researching is still time consuming and fatiguing. We must often wade through several distractors before finding what we are looking for. Nicolas Carr raises problems with search engines. “Though much less efficient than searching the Web, old-fashioned library research probably served to widen scholars’ horizons.” He cites James Evans “By drawing researchers through unrelated articles, print browsing and perusal may have facilitated broader comparisons and led researchers into the past.”²⁷ On the one hand, Carr claims that the Internet distracts us with tangential information; on the other he assumes that it doesn’t. Carr can’t have it both ways. While his claims are thought provoking, on reflection they don’t hold water. Scholars have long complained that there is too much information to process (Selye, 1964). Even in the 1980’s researchers had access to CD-ROMs and catalogs to access all kinds of information. There were plenty of seductive distractors. Students and researchers could easily write to each other to ask for more reprints than they could possibly read.²⁸ Undeniably, we can access information at faster speeds than ever before. We knowledge workers now routinely encounter quantities of documents that we would previously been unable to discover. Finding relevant information is essential to knowledge work. It’s a tough technical and personal challenge. While we need to be very careful in our searches, we also need ever better technology to help us find what might be most useful. To solve the problems raised by Carr we must reframe them as cognitive-productivity problems. ²⁶http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0020961 ²⁷Evans (2008). ²⁸When I was an undergraduate in the 1980’s, the University of Ottawa gave students access to postcards to request papers from authors. All I
needed to do was write the citation, my name and the author’s contact information. The University paid for postage. I would receive a steady stream of pertinent research through the mail.
Challenges to meta-effectiveness
64
We must determine to productively regulate how we search and how we utilize search results. We must frequently dedicate time not only to researching, but to becoming more efficient and effective researchers. In particular, we must find ways to obtain and assess documents that yield the best documents for the least amount of fatigue. Our ancestors, hunter gatherers, faced the same type of problem. Finding the right balance can itself be tricky. Having found information, we must process it. Unfortunately, even professionally published articles are not typically designed primarily with the intent of facilitating comprehension and minimizing cognitive load. This has nothing to do with the Internet. For example, the two-column format is one of many unfortunate holdovers of paper. This format makes reading and annotating PDF documents particularly difficult.²⁹ Moreover, many publishers deliberately employ obfuscation techniques in order to minimize copyright violations. They make it difficult to copy and paste information. This decreases the efficiency with which we might annotate and ultimately master the knowledge. The solution is not to revert to scrolls. It is to improve access to information, publishing practices, software and our own meta-effectiveness.
3.2.3 Cognitive productivity training No matter what one’s problem is, provided that it’s hard enough, one always gains from learning better ways to learn. Marvin Minsky Developing knowledge and oneself requires ongoing attention not only to learning particular content, but to developing meta-effectiveness—better ways to process information and to develop. I do not believe most knowledge workers would take issue with this. Yet there is not much training available for it. Why might this be? I was once Assistant Professor of Military Psychology and Leadership at the Royal Military College of Canada. The applied courses we taught, and the textbooks we used, focused mainly on social psychology: motivation, leadership, communication, group dynamics, etc. Perhaps because of Aristotle’s long standing influence in separating cognition from affect (mentioned in chapter 1), there are not many cognitive scientists teaching Organizational Psychology and Motivation and Performance Enhancement (I was an anomaly). Fifteen years later, when I did a competitive analysis for my business’s training services, I found that most psychological training offered in organizations still does not deal adequately with principles of cognitive science. Most psychology students neither start nor end their university degrees with a passion for computer programming. Some of them, however, discover and fall for in love with cognitive science and its core discipline, Artificial Intelligence. Yet, to address personal development, we need to consider the intersection of psychology and information technology: cognitive science. The mind is too complicated to be understood without thinking about it as an information processing system. Bereiter, as I mentioned above, dares say that much cognitive skills training for adults is “quackery”. In order to avoid this charge, one must properly spell out and address significant ²⁹For example, in the PDF reader, Skim.app, annotations in the Notes pane are listed in order that they appear on the Y axis of the page, ignoring the column. So, in the Notes pane, text highlighted in the first paragraph of the right column appears before text highlighted below it in the left column.
Challenges to meta-effectiveness
65
challenges while aiming for conceptual and empirical rigor. Keith Stanovich and his colleagues, for example, are developing a systematic framework of targets for rationality training (Stanovich, 2011; Stanovich, West, & Toplak, 2011). One might suspect that knowledge workers wouldn’t be particularly interested in cognitive skills training. This would follow from the assumption that their perceived self-efficacy is high. (And possibly from negative experience with general training programs.) Human resources managers can circumvent these ostensible attitudinal issues by targeting meta-effectiveness. Professional knowledge workers appreciate the value of time and the importance of productivity. Productivity literature and software are some of the most popular products on the web and in app stores these days. If you can spare a moment, before researching on the Internet, sketch a taxonomy of the productivity solutions out there. What problems ought productivity solutions address? Which ones do they tend to address? Having done that, examine your favorite app store and web search engine to refine your taxonomy. You might find amongst rafts of general productivity solutions, piecemeal, fragmented attempts to address cognitive productivity. The number of solutions aimed at clusters of problems have, however, been multiplying. To optimize cognitive productivity one must leverage a number of applications and software services. For example, • • • • • • • • • • • • • • • •
Services to tidy up web pages and manage reading lists, such as Readability³⁰. Outliners, such as OmniGroup’s OmniOutliner Pro. Graphic organizers, such as OmniGroup’s OmniGraffle. GTD® apps such as OmniGroup’s OmniFocus. Launchers, such as Object Development’s LaunchBar³¹. Sophisticated text editors, such as BareBones Software®’s BBEdit®. Basic Text Editors, such as Brett Terpstra’s nvALT³² based on Notational Velocity. File tagging software, such as Ironic Software’s Leap. File management, search and indexing software, such as DEVONtechnology®’s powerful DEVONthink®. Bibliographic management software such as Mekentosj’s Papers. Time tracking applications such as John Nye’s Tracktime. PDF readers such as Smile®’s PDFPen and the open source, Skim. Text expansion utilities such as Smile®’s TextExpander®. There are no pure productive practice apps yet, however Anki can be used for the purpose. Text to speech apps, such as Nuance®’s Dragon Dictate®. Applications, such as Calibre, to convert (when permitted) ebooks to other formats, such as PDF and e-Pub.
³⁰http://www.readability.com ³¹http://www.obdev.at/products/launchbar/index.html ³²For a feature comparison of text editors on iOS see http://brettterpstra.com/ios-text-editors/.
Challenges to meta-effectiveness
66
Each one of these categories of tools can contribute to one’s cognitive productivity. How to use them to optimize one’s cognitive productivity requires thought, however. For example, you can use TrackTime to help reconstruct how you have spent your time. But to use it to analyze how much time you spend on individual projects and activities requires that you develop a framework for classifying them. You’ll also need to find ways of using a spreadsheet or other software to analyze the data. Part 3 provides tips for most of those applications. You may also find it useful to hire a cognitive productivity trainer who understands meta-effectiveness. When I started using the expression cognitive productivity in 2009, I was surprised to find that it had hardly been used. Where it appeared, its meaning was different. Take for example one of the most popular productivity systems “Get Things Done®”, produced by David Allen, mentioned in chapter 1. Allen refers to GTD® as an action management system for knowledge workers. It deals with all kinds of actions, such as managing e-mail (a word that occurs over 110 times in his book), or cleaning and running errands. It does not address or specify the particularities of knowledge work. While many of his principles can be applied to information processing, it does not systematically address the core challenges of knowledge work, some of which I’ve attempted to describe in this chapter. And what about formal education? K-12 students don’t get training in cognitive productivity. Not all children are taught by teachers who are highly computer literate. Teachers who have productivity skills don’t necessarily pass them on to their students. By the time they reach university, students are assumed to know how to learn, and by implication, how to learn with technology. Some university students are required to take courses that utilize study strategy textbooks. Most books on how to read and study are targeted at sophomores and “developmental” students, meaning students who find it particularly difficult to learn, to read or to write. These books tend to focus on comprehension and on critical thinking, reading, and writing. Hadwin and Tevaarwerk (2011) reviewed 53 college-level study skills textbooks. They found that the books are all remarkably similar in content and suggestions, they do not tend to promote the development of strategy libraries, they tend not to cite pertinent empirical research (such as that reviewed by Dunlosky et al., 2013, on effective learning techniques), and they do not tend to provide much help for learning with technology. My impression in reviewing several of those books is that the students who can understand them probably don’t need the information. And those who can’t are not likely to benefit from them. However, there are some useful ones in the lot. Combined with appropriate instruction the results would be better. In designing productivity resources for knowledge workers we should not repeat the mistakes that were made in designing study strategy books for college students. Bereiter (1995) argued that the problem that most students face is not one of lacking strategies, it is one of values and character ethics. And that is not something that can be taught. One needs effectance, a predisposition, a propensity, to improve one’s cognitive productivity in order to benefit from strategy instruction. If one is in fact motivated to learn new strategies and one is taught helpful strategies, then strategy instruction can work. To design “helpful strategies” one must analyze requirements. Mortimer Adler’s How to Read a Book, aimed at the general public, is worth singling out (Adler & Van Doren, 1972). His book tackles the problem of how to read for understanding and lasting benefit.
Challenges to meta-effectiveness
67
It is dated, but fundamental aspects of how to read have not changed. More serious limitations, with respect to our objectives are—in line with the fact that the first edition of that book was published in 1940—that (a) it is not informed by cognitive science; (b) it is not addressed to knowledge workers (in the general sense used in this book); (c) it does not include guidance on how to use information technology.³³ The current book can be read as a modern version of Adler’s. We can’t fault organizations in the knowledge industry for failing to provide cognitive productivity training if the topic has not yet been suitably addressed by cognitive scientists.
3.3 Psychological challenges Evolution seems to have designed our minds to not be easily self-reprogrammable Kevin Kelly³⁴ It is informative to consider the psychological impediments to our meta-effectiveness. This is not to wave a white flag of surrender but to design strategies that overcome our weaknesses. In the following sections, I propose that: • • • • • •
cognitive science is not as widely disseminated as it should be, we are subject to illusions of helpfulness, comprehension, recall and rationality, our knowledge, skills and dispositions are not necessarily optimal for learning, we must nourish our effectance and our perceived self-efficacy, cognitive aging cannot be avoided altogether but there are ways to age gracefully, we need to understand and counter our distractibility.
To overcome these problems, we need to understand them. Then we need to improve our understanding of our minds and learning. And finally we must develop, apply, monitor and adjust our learning strategies.
3.3.1 Cognitive science in the realm of knowledge work Just as having a good understanding of biological sciences could help athletes (even if only through their coaches), having access to the best and most relevant knowledge about the mind, meta-effectiveness and knowledge itself could in principle help people work with knowledge. Knowledge workers face many problems. There are, in particular, problems with the production and dissemination of cognitive science itself. • The constitutive problems of cognitive science are very difficult. • Cognitive science is a young science. • Cognitive scientists have not done a sufficiently good job of translating their research for adults aiming to develop themselves. ³³The last two of these criticisms were mentioned as areas for future research on study strategies by Mulcahy-Ernt & Caverly (2009). ³⁴http://kk.org/thetechnium/2015/04/why-i-dont-worry-about-a-super-ai/
Challenges to meta-effectiveness
68
• Disseminated cognitive psychology tends to emphasize problems and limitations: illusions, biases, and pathology. • While cognitive psychologists understand information processing, relatively few psychologists have taken up the task of modelling high-level psychological processes using advanced AI programming techniques.³⁵ • The culture of academic empirical psychology is one of high-standard scientific rigor. Empirical psychologists are pioneers of empirical and statistical research methods. The down-side of their virtues is that cognitive psychologists are reluctant to generalize, make predictions, and discuss how their work can be applied in the real world, including knowledge work.³⁶ • In contrast, pop-psychology (virtual) bookshelves overflow with the work of authors unbridled by an appreciation for conceptual rigor, nor by an understanding of research methods or cognitive science. • Only a small minority of cognitive scientists focus on applications, fewer still on cognitive productivity. • Research on advanced education tends to focus on undergraduates, people with difficulties learning, and paper and pen learning.³⁷ • Technology is a moving target. By the time one has studied users interacting with it, it has changed.³⁸ The foregoing ought not to be taken as a blanket characterization or a criticism of cognitive science. Instead, it is meant to explain some of the factors that limit the accessibility of knowledge that knowledge workers can call upon for their trade.
3.3.2 Illusions of meta-effectiveness In order to use information to become more effective, we need to accurately assess how well we have processed it and what we will be able to do with it. This assessment determines what is left to do in order to benefit from the information. An extra investment in a pertinent resource can make the initial investment in it worthwhile. Normally, given a knowledge resource, the first order of business is to determine whether it is relevant to our current or future needs. That’s a function of its caliber and our objectives. We normally must make a judgment of the effort required to process the information. Unconsciously or not, we gauge its potency: i.e., how deeply we may be affected by it. If the resource is worthwhile, we undertake to comprehend it further. Finally, we must set up the conditions to capitalize on the knowledge. Take this book for example. Before reading it, at best you may believe that it is promising, meaning that it might address some of your needs with pertinent, potent information. As you read it, ³⁵Compare Sloman’s (1993a) argument that Artificial Intelligence is the core discipline of cognitive science. ³⁶This is not to deny that cognitive science has been applied to knowledge worker productivity. Hoffman & Militello (2012) provided an extensive
scholarly treatment of research in cognitive task analysis. Their book is replete with examples of applications of cognitive science. ³⁷The editors of a recent handbook on study strategies conclude their article by noting that research on study strategies has given much attention to “developmental students” (students with learning challenges). They call for study of e-learning at school, the workplace and at home (Mulcahy-Ernt & Caverly, 2009, p. 192). The current book provides a new framework for such research and practice. ³⁸Compare chapter 15 of Hoffman & Militello (2012).
Challenges to meta-effectiveness
69
I hope you will conclude that the information is of use. It accurately describes problems you have. It brings sound scientific concepts, and strategies based on cognitive science, to bear on the problems. To come to this conclusion, you must have first understood the book. Finally, to develop beneficial mindware from this book you need to practice applying it. For example, you need to get in the habit of setting learning objectives for documents you choose to read. If you have studied cognitive psychology, you know that humans are subject to a large variety of illusions. Illusions, errors and biases are perhaps the most extensively researched phenomena in cognitive psychology. There are several circumstances under which many people tend to make errors in reasoning, judgment and decision making. Our judgments of learning, as they are called in educational psychology, are conjectural, meaning there is some possibility of error. Each judgment we make about a knowledge resource (summarized above) may be false. Therefore, one of your prized expository books might not adequately address your needs. Perhaps some of the ideas it conveys are wrong. Maybe you think you understand its ideas but you don’t. Perhaps you’ve fully understood the book, in the short run, but will forget it, in the long run. Or maybe you will be able to remember it for years, in the sense that you can describe it to friends, but perhaps the knowledge will remain inert. In the subsections that follow I explain these concerns. I argue that we all at times run the risk of misjudging the relevance of information. I propose that we are subject to a number of illusions of competence: i.e., we believe we comprehend information that we don’t; we believe we will remember information longer than we will; and we believe that we will be able to apply information more than we are likely to. Whether we explicitly believe these propositions is not the issue: we act as if they were true. If we knew better, we might select other information, or process it differently or engage in productive practice. While some of the distinctions I draw are subtly different from the scientific literature, they have for the most part been well documented. 3.3.2.1 Illusions of helpfulness of information Selecting the right resource to process is critical to the development of effectiveness. Unfortunately, it’s easy to be seduced by flawed information. If you put a document in the field of view of literate people, they can no more stop themselves from reading it than they can prevent themselves from parsing a grammatical sentence spoken to them. If they are effectant then their tendency to read is even stronger. As people age, experience distress, or miss out on sleep, they have even greater difficulty to inhibit and control autonomous tendencies (such as reading).³⁹ Knowledge workers therefore chronically face a challenge, namely to assess the relevance of the information before them, and to decide whether, when and how to process it. After processing information, we often conclude (or should conclude) that it was not worth the attention we gave it: We conclude that we experienced an “illusion of helpfulness of information”. To minimize opportunity cost, we must habitually assess the relevance of information we process, and respond accordingly. For, when we spend too much time skimming, or when we attend to low grade information, we miss out on important material or the opportunity to develop effectiveness from relevant, potent information. ³⁹For explanations of the distinctions between autonomous and controlled processes, see for example Norman & Shallice (1986) and Stanovich (2009).
Challenges to meta-effectiveness
70
The opposite problem is to discount relevant and potent information. As our vignettes illustrate, this is a significant problem. For example, selecting relationship books that contain misguided advice, and fail to provide the right advice, can break a marriage. In contrast, selecting high caliber resources, such as Gottman’s, can restore or preserve the vitality of relationships. In The Mating Mind, Geoffrey Miller described what was arguably the most severe and widespread failure of scientists to adequately assess the caliber and potential of a theory. In 1871, Charles Darwin published The Descent of Man, and Selection in Relation to Sex. For a century, biologists systematically criticized and rejected Darwin’s theory of sexual selection. Since the 1990s, Darwin’s masterpiece has been the theoretical source of considerable research. Clearly, one needs a cogent and salient understanding of what it means for information to be helpful. In Chapter 11, I propose a new tetradic schema for assessing knowledge resources in terms of their caliber, utility, potency and appeal (or “CUPA”). 3.3.2.2 Illusions of comprehension One can hardly become effective with a resource unless one comprehends it. But do we always accurately gauge our comprehension? Suppose that you have volunteered to give a mini-course on information processing to an important audience. In 20 minutes, you must present the content of the current section of this book to them. Of course, you would process this section carefully. You would seek to comprehend the problem it addresses, its thesis and its argument. You would assess the relevance of the problem and the adequacy of the solution. You would monitor your comprehension. If you failed to understand a passage, you would take corrective measures. Many readers (even at the college level where students are supposed to be very careful) overestimate the extent to which they have understood the information they have processed. An experiment by Graesser et al. (2003) illustrates this point. They had 67 college students (at Ole Miss, Rhodes College and the University of Memphis) read textbook chapters on Newtonian physics for three to four hours. The students were tested one week later. Surprisingly, students fared no better on the test than students in a control group who had not read the textbook. Interestingly, students who spent a similar amount of time working with a computerized tutor performed much better. Results like these, when taken together, suggest that “normal reading” is rarely enough for complex material. Learning difficult material from text is difficult. Every scholar has seen his work misinterpreted by his peers. Scardamalia & Bereiter (1991) selected a sample of 20 papers from Behavioral and Brain Sciences (BBS), a top tier journal that features open, peer commentary. The average number of misreadings per author was three.⁴⁰ Most frequently, authors felt their readers had overlooked or misread the target text. This is particularly ⁴⁰While BBS is a cross-disciplinary journal, since the publication of Scardamalia & Bereiter (1991), technology has made it very easy for interested researchers to study this problem with a much broader sample of papers across disciplines. Assessing and classifying scholars’ reading errors would make for an interesting education thesis research problem. I had no difficulty collecting examples of purportedly egregious misreading. Martin & Irvine (1985), for example, wrote “It is difficult to know how to take the criticism by Robert Bud when there appears to have been such a fundamental misreading of our work as to lead him to categorize high-energy physics as ‘strategic research’.” Such indictments are not uncommon. Lather (1999), in the context of synthetically reviewing bodies of scholarly literature in education, emphasizes the inherent subjectivity of synthetic review and suggests that we “let go of the idea of the “correct reading” and, instead, see that every reading is a misreading, a partial, situated and perspectival reading.” (p. 4)
Challenges to meta-effectiveness
71
striking because BBS is a very prestigious journal. Each commentary is itself subject to a review!⁴¹ Bereiter (2002a) went so far as to suggest that such reading is not much better than sophomore reading. If misunderstanding is common in settings where experts are supposed to take the time to carefully read (and their interpretations are being judged publicly, meaning that their reputations are on the line), presumably more casual reading is even more at risk. To make sense of comprehension failure, we need an expanded concept of reading, or better, information processing. The ease and fluency with which we can read text can lead us to falsely assume that we have properly understood it. Obviously, we cannot and need not master every document we process. So we need to calibrate our judgments of comprehension. 3.3.2.3 Illusions of (future) recall Suppose that we’re in the midst of another deep recession. Last week your boss gave you seven difficult papers to read in preparation for a new project. You’ve dutifully read the lot, one paper a day. Now, she is standing by your desk. She asks you to brief her on these papers in a few minutes, before she heads off to a related meeting. You don’t have the documents with you. You need to judge how much of the relevant information you would be able to remember. How accurate do you think your judgment of your own memory⁴² would be? To help you answer this question, consider several documents you have read in the last week (e.g., this book). It is difficult to measure the accuracy of memory for prose because of several potentially confounding factors. For example, we need to distinguish comprehension from memory. (You can’t forget information that you have not properly understood). Cognitive psychologists have taken great precautions to isolate contributors to memory performance and prediction. They have studied memory in simplified, laboratory experiments in order to extrapolate to more ecologically valid situations. This has allowed them to make better sense of experimental results concerning reading complex texts where confounding factors play a greater role (e.g., attention and motivation). In these pure tests, people’s ability to predict their memory performance is often quite good. Let me paraphrase a common type of experimental report involving a frequently used paradigm for studying memory, “paired associates” learning. Suppose I were to randomly select 50 words from the dictionary and pair each one of them with a different randomly selected word. When you are given the first word of a pair (the cue word), your task is to remember the second word in the pair (the target word). Each word appears in only one list. I would train you by presenting each pair as follows: first, I present the cue word, then I ask you to think of what the associated (target) word is. Then I present the target. We would then repeat this for the next pair in the list of 50 pairs. (I would randomly select pairs.) We would cycle over this list of pairs until your memory performance reached a satisfactory standard. After this training phase, we would switch to the experimental test phase. Here we would cycle through the lists again, presenting you with the cue; but instead of ⁴¹Moreover, some misunderstandings are very widespread and go undetected for years, as researchers parrot authors considered as authorities. Consider, for example, that many respected authors, such as Daniel Goleman in his book Emotional Intelligence: Why It Can Matter More than IQ (Goleman, 2006), refer to Antonio Damasio’s book Descartes’ Error as showing that emotions are necessary for intelligence. Hundreds of authors, including swaths of highly influential ones, have endorsed Damasio’s view. Yet Aaron Sloman has recently shown that Damasio’s core argument is logically flawed (Sloman, 2008c). (See “Damasio’s Error” in that file.) Will this alter the beliefs of those who adhere to Damasio’s theory? Meanwhile, as illustrated in the commentary section of Sloman (2003) and the lack of uptake of this subtle information-processing concept, emotion researchers have difficulty “accommodating” to the fact that perturbance is central to human (tertiary) emotions. ⁴²I am using “memory judgment” here to mean: judgment of how well one will remember information on a future test. This is known variously as a type of “judgment of learning”. It also is often referred to as metamemory.
Challenges to meta-effectiveness
72
asking you to recall the target, we would ask you, on a scale from 0 to 100% certainty, how likely you are to be able to produce the cue’s target correctly. In other words, we would elicit a judgment of learning from you. In this condition, your memory judgment would actually be quite good. It would even be good if we were to change the question and ask you about the likelihood that you will remember the target next week, given the cue. It turns out, however, that there are number of subtle ways in which experimental psychologists can skew your memory judgments, making you overconfident or under confident at their whim. For example, the information in front of you when you make a memory judgment affects your memory judgment. If, instead of asking you for a memory judgment (“given this cue, how confident are you that you will be able to remember the target word next week?”), you were asked this question with both the cue and the target in front of you, you would be overconfident. In the foregoing (overconfidence) case, you wouldn’t have to test your own memory to make your prediction. So, you would base your prediction on other factors. Cognitive psychologists have been trying to isolate those factors through rigorous experimentation. Their research suggests important facts about how mind and memory work—facts that are relevant to our day-to-day learning. Asher Koriat and Robert Bjork, for example, proposed and tested the hypothesis that we have a built-in “foresight bias”. (It is analogous to, but different from, the well-known hindsight bias.) In other words, once we (even superficially) know something we tend to assume that others around us have that same knowledge (and that we ourselves will continue to have it). Unfortunately, that is often not the case. Koriat & Bjork (2005) contextualized their theory by summarizing a paper by Newton (1990): Newton (1990) asked participants (tappers) to tap out the rhythm of a familiar song to listeners and to predict the likelihood that the listeners would successfully identify the song. Although tappers’ predictions averaged 50%, the actual success rate of listeners averaged less than 3%. This result, as well as other results reviewed by Pronin, Puccio, and Ross (2002), demonstrates the difficulty that people have in discounting their privileged experience. This bias affects memory judgments in subtle but significant ways. The next couple of paragraphs exemplify this point. I include the description to give you a taste of how rigorously cognitive psychologists answer the kinds of questions that concern us here. Koriat and Bjork performed paired associate experiments to see if foresight bias played a role in memory judgment errors. They created three sets of word pairs: 1: pairs of unrelated words; 2: pairs of words that were a posteriori related; 3: pairs of words that were related a priori. These sets of words were constructed based on prior research. 1. The first set contained randomly paired words. 2. For each pair of words in the second set, the first word (the cue) is associated in meaning to the second word (the target); but people cannot explicitly predict the target if they are (independently) only presented with the cue. This property holds for the words “nurse” and “wife”. When (independent) subjects are presented with the word “nurse”, they can more
73
Challenges to meta-effectiveness
rapidly read the word “wife”. However, if (independent) subjects instead are asked to name a word related to “nurse”, they are not at all likely to proffer the word “wife”. The title of a paper names these concepts: “Associative facilitation without expectancy in a lexical decision task” (Fischler, 1977). The terms are semantically associated in one’s mind, but people cannot explicitly predict one from the other. 3. For each pair of words in the third set, the cue word was actually a good predictor of the target word, meaning that independent subjects tend to name the target word when presented with the cue word. The cue word “nurse” could have as such a target “doctor” or “patient”. Such words, are said to have high associative facilitation and high predictive facilitation. The experiment consisted of a study phase and test phase. During the study phase, the researchers presented the pairs of words to each subject on a computer screen, side by side, for 4 seconds, one pair of words at a time. After a pair was presented, they elicited memory judgments (as probability of later recall on a scale from 0% to 100%) from the subject. During the test phase, researchers presented the cue word to the same subjects; subjects were asked to recall the target word. The results are represented in the table below. Subjects were quite correctly confident with respect to highly related words. What is interesting for the purposes of this chapter is that subjects did not have significantly lower expected memory judgments for words that were merely a posteriori associated to the targets, but not predictive. In other words, subjects over-estimated their ability to recall targets that were semantically but not predictively related to cue words. It is also worth noting that even with unrelated words, subjects were overconfident. (As implied by the text preceding my description of this experiment, had subjects made memory judgments when presented with the cue but not the target, they would have performed much better.) The magnitude of this effect might not seem substantial; however, we have to keep in mind that we are dealing with ratios, which have a floor and a ceiling. Moreover, this is a very controlled and pure test. In real life (e.g., reading) the effects seem to be compounded (compare Newton’s study mentioned above). Table 3.1. Mean predicted recall (judgments of learning or memory judgments) of targets that are unrelated, a posteriori related, and a priori related to their respective cues Condition Unrelated A posteriori A priori
Predicted Recall 22.2% 66.0% 71.5%
Actual Recall 8.3% 46.9% 67.4%
Significant difference? Yes Yes No
So, what does this type of laboratory result tell us about real-world information processing? The upshot is that when we have information in front of us, we tend to over-estimate the extent to which we will be able to remember it. If when we are tested, we are presented with an information item that is highly related to another, then we will easily complete the pattern. However, in that case, we have not really learned much. Research with more ecologically valid (i.e., real-world) materials (e.g., prose, as opposed to paired associates) extending these laboratory results suggests that when people have information they are
Challenges to meta-effectiveness
74
reading in front of them, they are overconfident in judging how likely they are to remember what they read. But if you take away the material before asking them, then people will be more realistic. Furthermore, if you ask them after an interval how well they can remember the material, they become more realistic. This may be because they test their own memories and realize that they are coming up rather blank.⁴³ Based on such considerations, I suggest that whereas knowledge workers spend a lot of their time processing knowledge resources, they might not sufficiently ask themselves how well they will be able to remember and use the knowledge. As a result, they may not adopt strategies that ensure that they can commit a certain, albeit extremely small, percentage to memory (or, as I shall argue, to become effective users of this information.) Knowing that and why we are subject to the illusion of future remembering may help us to adjust what we read, when we read and how we read it. One way to remind ourselves of this is by quizzing ourselves (over time) on what we’ve read—a strategy explored in Part 3. To work around this, we can choose to read less and to strategically focus some effort on the most potent material that is in fact worth remembering. We shouldn’t aim to remember everything or even very much of what we process (for that requires too much time and effort). But it’s sometimes useful to commit small bits (“knowledge gems”) to memory (including so called “long-term working memory”). To return to the opening example, assuming (a) that you take the time to poll your own memory, (b) that you don’t have the papers in front of you, (c) you haven’t just read the papers, and (d) that you are predicting your immediate performance, your memory judgment is likely to be reasonably good. That is, you will expect to have forgotten quite a bit of information. However, if you had to make a memory judgment with the information fresh in your mind, then you are likely to be overconfident. This book is meant to help you improve your assessments of what you can remember, to select what to remember, and to choose strategies that will in fact produce the recall levels you expect. 3.3.2.4 Illusions of rationality: transfer reframed The primary gauge of expertise is transfer performance. […] How to train to promote transfer is the most fundamental challenge of education. Mark A. McDaniel The mechanisms producing emotions are also the mechanisms required for great flexibility in a complex environment. Aaron Sloman How many of the following statements apply to you? • I don’t plan to be in two different cities at the same time. • If I know how to avoid an important problem, I will. • If I know how to solve an important problem that I am trying to solve, I will apply that knowledge. ⁴³People tend to be less confident of their ability to remember a piece of information if (a) they have practiced (Finn & Metcalfe, 2011); or (b) they are given more time to make the judgment (Metcalfe & Finn, 2008). In the latter case, they have time to test themselves.
Challenges to meta-effectiveness
75
If you feel that these claims apply to you, then like most people you probably see yourself as a fairly rational person or at least someone who is not plainly irrational. By “rational” I mean someone who earnestly tries to develop beliefs and knowledge that accord with reality and who sets and pursues goals that will optimize the value he or she expects to obtain.⁴⁴ Being rational does not make one omniscient or omnipotent. We can still make errors. Rationality calls for using means, systems and procedures that tend to optimize our outcomes; this does not guarantee success. One of the bases for rationality is the principle of non-contradiction. We ought not to believe statements that are incompatible. Unfortunately, it’s impossible to maintain a completely consistent belief system. (Have you ever accidentally double-booked yourself?) It can take one years to detect inconsistencies that in hindsight seem obvious (as many discover as they ponder divorce). We sometimes make decisions without having or taking time to predict whether our behaviors will lead to consequences that are unduly inconsistent with our values. And even when we do our best, our minds don’t always comply. We sometimes know, or at least strongly suspect, we will regret a decision. When we comprehend and agree with knowledge, we are prone to assume that we will think and act rationally in accordance with it. When this judgment is unwarranted and false, we have experienced what I call an “illusion of rationality”.⁴⁵ The vignettes presented in chapter 1 exemplify this. In reading about investment, Chrissy felt she would be able to use Pat McKeough’s advice to make better investment decisions. Ralph read Gottman’s book in order to improve his marriage. He “knew” how to respond to bids for connection but he never learned to detect them. Janet agreed with Eric Ries that tight build-measure-learn loops would help to prevent disasters, but she never implemented the ideas. Each in their own way knew something and agreed with it at one level, but failed to apply and master it. To promote rationality, one’s knowledge acquisition procedures need to cause one to tend to act in accordance with the “acquired” knowledge. To believe—or act as if one believes—one will effortlessly tend to apply assimilated knowledge is an illusion of rationality. The problem of rationality is one of the most important ones addressed in this book. It can be used to make sense of and address the “problems of transfer” alluded to above. Consider some of the most frequently cited studies in cognitive psychology, conducted by Gick and Holyoak (Gick & Holyoak, 1980, 1983). They asked Introduction to Psychology students to solve the following problem. Suppose you are a doctor faced with a patient who has a malignant tumor in his stomach. It is impossible to operate on the patient, but unless the tumor is destroyed the patient will die. There is a kind of ray that can be used to destroy the tumor. If the rays reach the tumor all at once at a sufficiently high intensity, the tumor will be destroyed. Unfortunately, at this intensity the healthy tissue that the rays pass through on the way to the tumor will also be destroyed. At lower intensities the rays are harmless to healthy tissue, but they will not affect the tumor either. What type of procedure might be used ⁴⁴For a helpful theory of rational thinking and deciding, see Baron (2008). ⁴⁵This also entails an illusion of understanding, meaning that one implicitly believes that one has lasting understanding and not merely short-term
comprehension of information. Kahneman also uses the expression “illusion of understanding” but his examples deal with patently false knowledge (Kahneman, 2011). The cases I consider in this book involve information in high caliber knowledge resources that is either true, or worthy of being considered as true. (I disagree with Kahneman’s position (p. 200) that understanding entails being able to predict, and that knowledge is true justified belief. Knowledge is, for the most part, conjectural.)
Challenges to meta-effectiveness
76
to destroy the tumor with the rays, and at the same time avoid destroying the healthy tissue? Only 8% of students got the right answer. But before I give you an answer, consider the following short story, called “The General”, which other subjects read and memorized a few minutes before trying to solve the tumor problem. A small country was ruled from a strong fortress by a dictator. The fortress was situated in the middle of the country, surrounded by farms and villages. Many roads led to the fortress through the countryside. A rebel general vowed to capture the fortress. The general knew that an attack by his entire army would capture the fortress. He gathered his army at the head of one of the roads, ready to launch a full-scale direct attack. However, the general then learned that the dictator had planted mines on each of the roads. The mines were set so that small bodies of men could pass over them safely, since the dictator needed to move his troops and workers to and from the fortress. However, any large force would also destroy many neighboring villages. It therefore seemed impossible to capture the fortress. However, the general devised a simple plan. He divided his army into small groups and dispatched each group to the head of a different road. When all was ready he gave the signal and each group marched down a different road. Each group continued down its road to the fortress so that the entire army arrived together at the fortress at the same time. In this way, the general captured the fortress and overthrew the dictator. As you may have noticed, these two problems are analogous. Both have a similar structure in antecedent conditions (starting point) and goal specification (desired result). Both problems can be solved with a strategy of divide-and-conquer in parallel: The doctor could simultaneously administer multiple rays of low-intensity, each from a slightly different vantage point, but converging on the tumor with additive intensity. This would zap the tumor without harming the surrounding tissue. Thirty percent of the students who read “The General” before trying to solve the tumor problem correctly solved it. Comparing that result with the 8% success rate in the control condition suggests that the analogy helped many students. But still, even when exposed to “The General”, most students failed to solve the tumor problem. Cognitive scientists have sought to discover why some students “got it” and others didn’t. That’s important research, because if we find it difficult to apply information we read only a few minutes ago, what benefits can we expect months and years later? If there are better ways to learn, we need to know them. Gick and Holyoak’s research suggests that those who didn’t solve the tumor problem did not notice the analogy. (Alas, what it means to notice an analogy is a critical but challenging question.) Perhaps they did not (productively) engage in what Salomon & Perkins (1987) refer to as “backward-reaching transfer”, which involves asking oneself “What do I know that might be relevant to the current situation?” In a weak sense, they were behaving irrationally. They knew something that they could use to solve the problem; but they didn’t apply that knowledge. A key finding is that a group of students who also received the Tumor story but were given a vague hint, to try to
Challenges to meta-effectiveness
77
use “The General” story in solving the Ray problem,⁴⁶ performed significantly better. About 75% of them produced the divide and conquer in parallel solution! One reason we sometimes fail to benefit from what we have learned is that we don’t always meaningfully abstract the structure of the problem and and its solution. In other words, it’s not that we fail to apply what we know, but that we don’t learn properly in the first place. We often fail to analyze (and name) the type of problem and solution we are facing. I referred to the Ray and General problems as calling for a “divide-and-conquer in parallel strategy”, which is a type of divide-and-conquer strategy. If you’ve already dealt with many similar problems in abstract terms, you might quickly apply such a label yourself. Otherwise, it might take some time and effort to detect and label the pattern. When you perform this kind of cognitive task in preparing for future problem solving, you are engaging in “forward-reaching transfer” (Perkins & Solomon, 1987). Here, you are trying to construct personal knowledge that you can apply (“transfer”) to future similar cases. Forward-reaching transfer is something we strive for with all kinds of information that we learn. (It requires a rational processing mindset.) To deeply process this information, I recommend you identify a book or document that you had carefully read that might have helped you with a recent problem but didn’t. Why did you fail to use the knowledge? The backward- and forward-reaching characterization of transfer are examples of the structurematching approach. Structure matching goes like this: discover the structure of a prior problem (i.e., the structure of both the initial and goal conditions, and the mapping between them); discover a solution and express it in abstract terms (e.g., divide-and-conquer in parallel strategy). Then notice, in the future, when a given situation matches the prior problem’s structure; when it does, consider applying the prior solution in its abstract form. Structure matching calls for some heavy thinking up front, and sophisticated pattern-matching at “run time”. Alas, this approach does not capture all failures to apply what we know. A problem with this rather schematic characterization of transfer is that it under-emphasizes some of the most important mindware we develop as we become more effective. That is a web of fine-grained mental mechanisms, many of which are perceptual. The perception is not so much of the external environment as it is of the mind. Often, the reason we fail to apply knowledge is that we fail to detect that it is pertinent. It’s as if all we need is a hint, like the students who were prompted to consider the story, “The General”. But we must provide the hint ourselves! Asking ourselves “what relevant prior knowledge can I bring to bear on this problem?” won’t necessarily be enough (though it may help). Something has to happen to our internal perception in between forward-reaching and backward-reaching processing. To understand transfer failures (and the breakdowns in rationality they entail), we need to refer to a blue-print of the mind, one that starts to make sense of successes and failures of learning. This will enable us to pinpoint some of the mental mechanisms that fail to develop in cases where we systematically fail to apply the knowledge we “acquired.” We also need a relational concept of understanding. For there is more to “transfer” and knowledge-based rationality than applying concepts and skills. We will turn our attention to these matters in Part 2. Chapter 14 describes ways of practicing that increase the likelihood that we will apply what we know. ⁴⁶The hint was no more specific than that. Subjects were also told that “The General” was not necessary to solve the problem. They were allowed to reread the “The General” story at will. Gick & Holyoak (1980, p. 320).
Challenges to meta-effectiveness
78
3.3.3 Cognitive miserliness and its antagonists I’m sorry Darling you are disappointed at the sale of the Book [The Arms and the Covenant]. I’m sure it’s the price—The sort of people who want to hear that the Government is all wrong are not the rich ones —The Tories don’t want to be made to think. Clementine Churchill to Winston Churchill Even with the ideas and tips in this book, using knowledge to become a more effective person requires a lot of effort. Meta-effectiveness makes personal development easier, not easy. One must fight the temptation to passively process information. Rather, we must actively seek the best information, process it carefully, think about and with it, and practice it, whether deliberately or implicitly. This requires thinking dispositions that most people lack, propensities that are at odds with what Keith Stanovich refers to as “cognitive miserliness”, a concept he introduces as follows: Consider the following problem, taken from the work of Hector Levesque and studied by my research group. Try to answer before reading on: Jack is looking at Anne but Anne is looking at George. Jack is married but George is not. Is a married person looking at an unmarried person?
A) Yes B) No C) Cannot be determined
Answer A, B, or C before you look ahead. (Stanovich, 2009) http://www.keithstanovich.com/Site/Books.html
I’ve tucked the answer to this question and a brief explanation of the data in this footnote⁴⁷ to prevent you from accidentally reading it. While solving this problem does require a certain level of fluid intelligence, IQ does not explain the fact that 80% of participants get this wrong. They were all ⁴⁷Most people, at first blush, don’t see a way of proving “yes” or “no”. A way to answer this question is to consider that Anne may be married or unmarried. Most people then seem to suppose that because Anne’s marital status is not given, the problem cannot be solved. And so they answer “C”. This is the easy way out. For one could instead wonder and logically investigate the implications of Anne being married. If she is married, then a married person (Anne) is looking at an unmarried person (George). One could then continue to wonder what follows in the case where she is not married. One may then conclude that a married person (Jack) is looking at an unmarried person (this time, Anne). To solve this problem in this way requires that one consider hypothetical possibilities, store them in memory, draw inferences, and reason about them. That’s cognitively demanding. In Part 3, I give an example of how the concept of cognitive miserliness can be learned with productive practice.
Challenges to meta-effectiveness
79
smart enough to answer the question correctly; but the cognitive misers amongst them eye-balled the problem and then quickly selected an incorrect answer. This shows that smart people shouldn’t expect to be able to coast through life based on their high IQ. Fluid intelligence is of little use if one can’t be bothered to think.⁴⁸ What does this mean for developing personal effectiveness from knowledge? Think back to Janet of Water Flop. Her high IQ will certainly help her to solve problems when she applies herself. She might even tend to process reams of information. However, she also tends to shoot from the hip. She does not seek out the best knowledge resources nor does she effortfully try to apply them in her day-to-day problem-solving. Many of her former classmates who invest more effort in developing themselves, even those with lower IQs, have long since become more effective than she is. As a result, their teams also perform better and they are more often consulted for their expertise. On the one hand, it is helpful to keep in mind the dangerous allure of cognitive miserliness; but on the other, one ought not to depend too heavily on negative self-talk and duty (to avoid cognitive miserliness). Besides, conservation of mental resources is important. It is difficult to nurture the dispositions required to do the necessary demanding, sometimes dry, cognitive work if we cannot even name and describe them. We need positive language to express the affective underpinnings of our cognitive pursuits. Effectance, perceived self-efficacy and thinking disposition are helpful concepts for our pursuit of knowledge-based excellence. 3.3.3.1 Effectance as a propensity to develop competence Helen [Keller] did not come by her knowledge easily. Everything she did was so difficult that most people would have given up early in the learning process. But she worked furiously at mastering all she encountered. Merlin Donald In chapter 1, I introduced White’s concept of effectance: the motivation to develop competence. However, I use the term in a subtly different way than is normally used, namely as the often tacit propensity to develop competence. The key difference is that this propensity does not necessarily involve explicit (let alone conscious) motivation for competence. I also emphasize the role of objective knowledge in adult effectance. This new concept of effectance is more subtle and more powerful. It is based on an architectural concept of motivation, which comes from “designer-based” Artificial Intelligence. White always qualified the term “effectance” with “motivation” or “urge”⁴⁹, as in “effectance motivation”. In so doing, he vitiated his own neologism. I suspect this is why it is not in common currency. There is no use for the term so qualified; one might as well use the phrase “competence motivation”.⁵⁰ Moreover, as I argue below, the compound use of “effectance” betrays a folkpsychological notion of motivation which, though it is for all intents and purposes the only one ⁴⁸Many factors can affect the answer one gives to this question; one must therefore not read too much into a wrong result on one question. Stanovich (2011) calls for the creation of psychometrically valid tests to measure one’s “rationality quotient (RQ)” (p. 246). ⁴⁹For example, “In infants and young children it seems to me sensible to conceive of effectance motivation as undifferentiated.” White (1959, p. 323). ⁵⁰That is in fact how White defined effectance. However, to be fair, White emphasized the ability to affect the environment, which is slightly different from competence. Such hair-splitting distinctions cannot be expected to have moved the pens of White’s contemporaries (but cf. Allport, 1961, p. 251). Moreover, “effectiveness” and “efficacy” would do just as well.
Challenges to meta-effectiveness
80
used in psychology, needs to be superceded with an architectural, designer concept (cf. Part 2). So, I use the term “effectance” by itself and in adjectival form, “effectant”. Concepts similar to effectance have shown up many times in cognitive science—but without an adequate label.⁵¹ Carl Bereiter and Maria Scardamalia provided an insightful analysis of the processes of expertise (Bereiter & Scardamalia, 1993). They did not merely focus on differences between experts and novices. They sought to explain how people acquire and lose expertise. They drew attention to critical similarities between “expert-like novices” and experts. At every level of competence, there are some people who have more fluid expertise than others. Fluid expertise is a major component of meta-effectiveness. It is distinct from effectance. As people gain crystallized expertise, they become more efficient. Fluid experts reinvest the temporal gains of these efficiencies in learning and progressive problem-solving. Thus, they further develop their expertise. Bereiter & Scardamalia stressed the importance of the underlying propensity to develop. However, they did not name their motivational concept. The concept of effectance, as I have adapted it, designates the underlying motivational processes. We can attribute effectance to people using the intentional stance and explain it with the designer stance (i.e., an architecture-based theory of motivation). Effectance is the propensity to develop competence. One need not be an expert to be effectant. But without effectance one cannot indefinitely sustain the development of expertise. The motivational processes underlying what Bereiter and Scardamalia called “reinvestment” deserves its own name (effectance) and further characterization. Gopnik, in her paper “Explanation as orgasm”, reinvented White’s evolutionary explanation for effectance (Gopnik, 1998) without referencing White or using the term “effectance”. She posited a theory drive, “a motivational system that impels us to interpret new evidence in terms of existing theories and change our theories in the light of new evidence”. (p. 101) This is not identical to White’s effectance. For parsimony, rather than introduce a brand new concept, I extend White’s concept of effectance to accommodate Gopnik’s data and other manifestations of effectance described in this book. I want the meta-effectiveness framework to avoid the fate of instinct theory (e.g., William McDougall’s proliferating list of instincts⁵²). There is also a vast literature on thinking dispositions that is relevant to effectance. Stanovich and his colleagues have developed a taxonomy of rationality involving thinking dispositions (Stanovich, 2011; Stanovich, et al, 2011; Toplak, West, & Stanovich, 2012). The “need for cognition” is particularly relevant to effectance as are various factors related to curiosity (Aubé, 2005). The concept of motivation for increased competence shows up in the psychology literature in different forms. The term “effectance”, however, is only rarely used. Moreover, it has never previously been used with the specific meaning I develop here, i.e., one that is grounded in designerbased cognitive science research (Beaudoin, 1994; Sloman, 2010c). In fact, the general concept of motivation described here is not widely known in the psychology literature or elsewhere. Yet it’s impossible to understand and promote the pursuit of excellence without such a concept. It has become customary in attempts to account for competence-motivation (effectance) to ⁵¹Bruner (1966, chapter 6, “The Will to Learn”), who also refers to White, comes closer than White and Bereiter to the concept of effectance described here. ⁵²Bernard (1924) painstakingly identified over 14,000 alleged instincts in the social science literature. The doctrine of instincts is not particularly parsimonious.
Challenges to meta-effectiveness
81
invoke Csikszentmihalyi’s concept of “flow”.⁵³ “The concept describes a particular kind of experience that is so engrossing and enjoyable that it becomes autotelic, that is, worth doing for its own sake even though it may have no consequence outside itself.” (Csikszentmihalyi, 1999, p. 824.) Csikszentmihalyi provides as an example of the state of flow a composer’s account of writing music You are in an ecstatic state to such a point that you feel as though you almost don’t exist. I’ve experienced this time and time again. My hand seems devoid of myself, and I have nothing to do with what is happening. I just sit there watching in a state of awe and wonderment. And the music just flows out by itself. (Csikszentmihalyi, 1975, p. 44) Work becomes as Stuart Brown concisely describes play⁵⁴ (Brown & Vaughan, 2009): A “Goldi Locks” state of peak performance, wherein one addresses a difficult but not insurmountable challenge and feels a sense of timelessness and selflessness, as if the activity was done for its own sake.⁵⁵ Alas, the fact that a concept is commonly invoked to explain a phenomenon does not entail that it is productive. While, as I described elsewhere (Beaudoin, 2014b), I do not doubt that most knowledge workers can relate to and enjoy the experience of flow, the theory of flow betrays a nearly universal yet false assumption in colloquial and scientific accounts of behavior. It is essentially hedonism, that people do things because they enjoy either the feelings the behavior elicits or some other aspect of the state of performance.⁵⁶ More generally, that we are driven by the “law of effect” (reinforcement and punishment). I don’t believe knowledge workers are seeking a fix (“flow”).⁵⁷ Moreover, even if flow had the powerful motivational effects that Csikszentmihalyi claims, it would not be of great use to promote it—pleasure seeking tends to take care of itself. Thus, the explanation of effectance is not flow and the concept of flow has very little explanatory power. The explanation for effectance, like the drive to mate, is instead evolutionary, as White alluded to with respect to children’s play. That is to say that the motivation for competence in humans throughout our evolutionary history provided a reproductive advantage.⁵⁸ ⁵³For example, Bereiter and Scardamalia appeal to flow in their explanation of “fluid expertise”. ⁵⁴The psychological properties of play are described in a concise, informal book (Stuart Brown & Vaughan, 2009). For a development perspective
on play see Pellegrini (2013). For ways to use play (and implicitly, flow) to decrease sleep-onset latency, see Beaudoin (2013, 2014a). The latter paper applies the theory of mind described in Part 2. ⁵⁵Compare Campbell’s (2008) interview of Stuart Brown. ⁵⁶Gilbert Ryle criticizes the notion that people do things for the feelings those things give them (Ryle, 1949). Csikszentmihalyi implicitly applies Ryle’s argument (without referencing Ryle) when he states “Being happy would be a distraction, an interruption of the flow” (Csikszentmihalyi, 1999, p. 825). Csikszentmihalyi acknowledges that flow is not the only motive for behaviors that elicit flow. However, he assumes that flow is an intrinsic motive, without distinguishing between two dimensions of intrinsic motivation: internal vs. external to the agent, and derivative vs. intrinsic value goals. Compare the discussion of functional autonomy in Beaudoin (1994) and Allport (1937, 1961). See also Sloman (2009b). ⁵⁷Hedonism is still alive and kicking in cognitive science: “Higher cognition in its many forms—what it means to think like a human—is simply the chasing of the pleasures and the avoidance of the pains that are supplied by this eclectic group of cognitive, but of course ultimately neurobiological, emotions.” (Hurley et al., 2011) I believe this is false. Moreover, it is incompatible with the architectural view of motivation. See Erber & Erber (2001) and chapter 5 of Frijda (2007) for related arguments against hedonism. ⁵⁸The thoughtful theory of humor recently proposed by Hurley, Dennett and Adams also implicitly hinges on the concept of effectance (Hurley et al., 2011). (In particular, see their sixth chapter). They expatiate about evolutionary bases for thinking dispositions comprising fluid rationality (Stanovich, 2011). However, they do not use the terms effectance, fluid rationality or thinking disposition. (They do not refer to White’s work or that of Stanovich.) I believe their theory would have been easier to communicate with the concept of effectance and thinking dispositions. They further fail to draw necessary distinctions between motivational and emotional states, distinctions which become apparent when one explores the architectural basis for motivation, as described below. I make some of these distinctions in chapter 5 and 6. See also Sloman (2003) and discussions of the work of Andrew Ortony below.
Challenges to meta-effectiveness
82
However, here lies a rarely noted subtlety. Effectance ought not to be understood as a single, top-level drive, goal or motivator.⁵⁹ Nor do our inclinations towards behaviors that increase our competence necessarily involve explicit and conscious representations of competence (e.g., goals to become more competent). Instead, it is reasonable to assume that people have mechanisms that lead them to produce goals the pursuit of which will or may directly or as a side-effect improve their competence.⁶⁰ One normally delves a paper to better understand it and to use it for building knowledge or solving a problem. One does not necessarily engage in this behavior for the explicit or otherwise unconsciously operating motive of improving one’s competence. Yet delving can develop one’s effectiveness and so it reflects our implicit effectance.⁶¹ More generally, the human mind can generate top-level goals as a reflex without deriving them from means-ends analysis, planning or other deliberate processes. I call these “reactive, intrinsic motives”.⁶² A motive whose pursuit improves one’s effectiveness is not necessarily seen, felt or otherwise represented in the mind as a means towards effectiveness. Thus, effectant motives are not simply aimed at flow.⁶³ Nor are they normally aimed (even unconsciously) at improving effectiveness. For one to be effectant is to have mechanisms that produce top-level goals (i.e., goals that are treated as good in themselves) the pursuit of which leads (or tends to lead) to the development of competence. Effectant people implicitly inherently value competence. Having dealt with this special case, I acknowledge that people can become more effectant by becoming conscious of their effectance and by valuing effectance. From a practical perspective, effectance, while natural, can and ought to be nurtured. It can lead to the intrinsic benefits cataloged in the previous chapter. They, in turn, tend to lead to the external consequences and reward of competence that do not need to be explicitly cataloged here.⁶⁴ Effectance, considered this way, calls for a characterization of motivation in terms of the architecture of the human mind. That is to seek the explanation of effectance not in terms of its consequences (rewards or pleasure) or functions but in terms of the mechanisms that give rise to our ascriptions of it. The designer-based concept of effectance can lead us to inquire into the information processing substrate of motivation. In addition to any overarching, explicit drive for effectiveness, there are myriad mechanisms that generate all kinds of motives to behave in ways ⁵⁹However, a person can, of course, form explicit goals to increase competence. And this can lead them (unconsciously) to create motive generators that when acted upon increase their competence. ⁶⁰This is a special case of the argument for architecture-based motivation (Sloman, 2009b). Evolution cannot guarantee that a motive generator will necessarily create motives that provide a selective advantage. Motive generators evolve because they tend to produce an advantage often enough, which might be very rarely. “The main point [of architecture-based motivation] is that the individual concerned has no information about [the benefit provided by this type of motive], not even implicit information (unless the individual is a biologist who starts asking ‘Why do I have these motives?’)” (Sloman, 2013b). ⁶¹“Implicit” does not mean “unconscious”. ⁶²In Part 2, I refer to deliberation processes as management processes. There, we will see that “reactive motives” stem from asynchronous motive generators —reactive mechanisms. ⁶³White’s paper could also be criticized for emphasizing the feeling of competence, though the emphasis there is not as strong. It can also be criticized, along with much literature on affect, for characterizing affect as a matter of feelings. The architectural model described below does not emphasize (or deny a role for) feelings of competence or flow, and yet it does not depend on the rather unparsimonious assumption of the pursuit of pleasure. See also chapter 6 of Beaudoin (1994); Sloman (1987, 2009b). ⁶⁴Peter Brems (personal communication, February 21, 2015) distinguished two types of effectance: propensity to increase a specific competence and propensity to become better at improving oneself (such as by mastering new learning strategies). He suggested we call the latter “meta-effectance”. Understandably, however, most readers are resistant to neologisms and even more to recursive concepts. Moreover, the concept of architecture-based motivation blurs the distinction between competence and motivation. I would be content were the terms “effectance” and “meta-effectiveness” to enter common parlance. So, in this book I stick to these overarching terms.
Challenges to meta-effectiveness
83
which increase effectiveness, even though the agent is not explicitly or even unconsciously seeking to become more effective. The architectural basis of motivation and other affective states is briefly described in chapter 5. Thus, while the concept of flow is of some value, the concept of effectance is of greater theoretical and practical significance with respect to motivation for competence. Its theoretical advantages are implicit in my criticism of the concept of flow. Having a term for this important construct (effectance) may promote both our understanding of meta-effectiveness and the practical development of effectance. To summarize the admittedly complex and uncommon ideas presented in this section: • Humans are capable of generating top-level motives that are not derived from other motives. These “reactive motives” do not necessarily serve any other motive, drive, or purpose. They are not necessarily driven by implicit considerations of reward, punishment, pain, pleasure or “flow”. They may have intrinsic value. • Effectance refers to a person’s propensity to develop effectiveness. • The concept of effectance applies both to cases where an agent engages in behaviors (a) for the deliberate goal of becoming more effective; (b) that viewed from the intentional stance promote effectiveness but that (i) were not explicitly spawned in pursuit of effectiveness and (ii) do not explicitly code for the pursuit of effectiveness (or its consequences). • Effectance is not necessarily explicit motivation for competence; however, it is motivation that tends to develop one’s competence. • Effectance is thus the motivational underpinning of meta-effectiveness. Deliberately nurturing one’s effectance may help improve one’s effectiveness. 3.3.3.2 Perceived self-efficacy One must strike a balance between arrogance and underconfidence. Douglas Kennedy Effectance is predicated on perceived self-efficacy. Believing one inherently is unable to succeed in a domain has been shown to affect performance in a wide variety of areas: work performance, academic performance, health, etc. (Bandura, 1997). Perceived self-efficacy is one of the most researched phenomena in psychology. It ought not to be confused with self-esteem, self-concept or “locus of control”. If a person believes she is inherently incompetent in one area (such as mathematics), it will directly affect that area without necessarily affecting another (e.g., writing). Consider, for an ironic example, the psychologist who sees herself as quite competent in helping children improve their perceived self-efficacy yet who sees herself as being inept with computers. She does not realize it, but her assumption that she is “simply not a computer person” makes it difficult for her to (want to) keep abreast of the literature. I deliberately chose the example of perceived competence with technology because I believe it is one of the most wide-spread self-limiting attitudes people contend with, even young knowledge workers. By failing to become more proficient with technology, highly intelligent people also limit their meta-effectiveness.
Challenges to meta-effectiveness
84
The mechanisms by which perceived self-efficacy affect performance are easy to comprehend and compelling. Wood & Bandura (1991) report that perceived self-efficacy in a domain affects: 1. the activities and environments we choose (people tend to avoid activities at which they expect to perform miserably); 2. the outcomes that we imagine (before and after our choices) will come from these activities (people who feel inept in a particular domain tend to expect to perform poorly when they perform in those domains); 3. our thought patterns while we conduct these activities (e.g., low perceived self-efficacy leads to self doubts, which can interfere with problem solving, e.g., by consuming limited working memory capacity); 4. the stress we feel during problem solving (low self-efficacy tends to lead to unpleasant stress, which itself can interfere with problem solving); 5. our perseverance in these activities (people tend to quit earlier if they feel ineffective). I’ve listed these items because I believe it is important not to merely accept psychological generalizations at face value or based on correlational data. One needs to understand and find plausible the internal mechanism that underlies the link between psychological causes and psychological effects. Clearly, to see oneself as non-technical can become a self-fulfilling prophesy. The opposite attitude from negative perceived self-efficacy is also problematic. It is to see oneself as being sufficiently skilled that one need not seek to surpass oneself. This is how experts get themselves into ruts. There are always better ways of using technology to learn, particularly given the bewildering and increasing number of tools available. Humility helps. As Karl Popper put it: “While differing widely in the little bits we know, or rather guess, in our infinite ignorance we are all equal.” But humility ought to be combined with cognitive zest. While it is natural to expect professional knowledge workers to perceive themselves as being effective, they do face special challenges. First, they are smart people surrounded by smart people. People tend to compare themselves with the persons around them. When comparing themselves with their peers, knowledge workers on average will find that 50% are more competent and more effectant than they are. When knowledge workers begin new jobs, they may find themselves with people who have already learned the ropes. When I started at Abatis, for example, I was new to embedded systems programming and computer networking protocols whereas the employees hired next were experts in them. That created immense initial pressure on me. Second, effectance is a drive to surpass oneself, which means taking on tough assignments and setting high standards. That increases the risk of failure. While this part of the book is not meant to propose solutions, I will note the following. Surrounding oneself with people who are more knowledgeable than oneself and setting high standards are keys to meta-effectiveness. However, one mustn’t allow contexts (at home and at work) to undermine one’s perceived self-efficacy. (It’s better to divorce, for example, than to live with a highly critical spouse even if one starts with high perceived self-efficacy.) One must always nurture effectance and perceived self-efficacy. Self-critical statements about areas of competence that should be nurtured, such as “I am not very good at …” may constructively be followed by “yet” or “as I plan to be, by doing …” This can help one determine whether one should become more
Challenges to meta-effectiveness
85
competent in the target area, and if so, how and when. One must appreciate that the development of expertise is always a gradual process. It is also useful to have studied the lives of great people whose enthusiasm propelled them forward through tremendous challenges. I, myself, have been inspired by biographies of those to whose memory this book is dedicated. Winston Churchill’s enthusiasm in the face of difficulties will long be remembered.⁶⁵
3.3.4 Cognitive aging Mourir cela n’est rien Mourir la belle affaire Mais vieillir… ô vieillir! Jacques Brel Enter a taboo reality that I briefly described in the previous chapter: Cognitive aging is a welldocumented corollary of biological aging (Craik & Salthouse, 2008). Fluid intelligence peeks in one’s early twenties. Focusing in the light of distraction becomes increasingly difficult. Our ability to process information in working memory tends to decline as we age. So does our ability to retrieve information from long-term memory. These declines are not merely consequences of brain injuries, mini-strokes or “Alzheimer’s”. While there are individual differences—some due to lifestyle (e.g., sleep, nutrition, medication, smoking, exercise), some due to genetics—and one can also improve in many respects, no one who survives is immune to the drags of brain aging. One of the most significant things you can do to counter-balance cognitive aging is to seek to become more effective in domains that matter to you. Regularly deploying a high-caliber, potent meta-effectiveness framework is critical to this.
3.3.5 Distractibility and the mind’s design In this chapter, I have distinguished between technical, environmental and psychological challenges to meta-effectiveness. Many of our challenges, such as our susceptibility to distraction, fit in multiple categories. It’s mind boggling to consider the variety of external channels competing for our attention: news sites, blogs and social media, email, instant messaging, traditional telephony. Information is pushed at us. We pull and poll for information. Information sources are often specifically designed to distract us. The purposes of information providers often conflict with ours. We do not consume information, but time slices of our attention are consumed by information. Dr. Terry Wahls on TED⁶⁶ remarked that despite the wide availability of food, we are starving ourselves by failing to eat foods our bodies need to reconstitute themselves. Something similar can be said about information. We live in an era of abundant, high quality knowledge. Yet we struggle to benefit from this knowledge. In The Information Diet, Clay Johnson argued that many of us are feasting on all too readily available junk information. While, as I argued in chapter 1, the digestion metaphor needs to be replaced by the information-processing metaphor, Johnson has a point. The seductiveness of information not worth processing is a major problem for knowledge workers. Time ⁶⁵This is well portrayed by Jenkins (2002). ⁶⁶http://tedxtalks.ted.com/video/TEDxIowaCity-Dr-Terry-Wahls-Min
Challenges to meta-effectiveness
86
spent processing junk is time not spent processing high caliber information. It is time not spent mastering the knowledge required to improve our skills, relationships, projects, health, finances and ultimately our effectiveness. The last few years has seen the introduction of various tools to help keep us focused—but they have not gone far enough. In an article for SharpBrains in 2010 before the long anticipated announcement of Apple’s tablet, I called for Apple to introduce an ecosystem wide “mute” function (Beaudoin, 2010a). Pressing this button would silence notifications across all of our devices so that the user could focus on his cognitively demanding work. Apple has since then introduced a ‘Do Not Disturb’ feature in iOS that is a step in the right direction. There are web browser extensions such as BetaFish’s AdBlock⁶⁷ that enable one to disable or remove ads from web pages. There are tools such as Readability⁶⁸ that remove distractors from web pages. There are also tools, such as Bumblebee Systems’ WasteNoTime⁶⁹ that enable users to monitor time spent on websites and even block access to them. I recommend using all kinds of tools like these. However, knowing that there are distractors out there and having technology to keep focused is no guarantee of remaining focused. It is useful to have an understanding of the psychology of distraction. One also needs to have information processing workflows and habits that are up to the difficult meta-task of staying on task. The rest of this book is pertinent to these two aims. Part 2 describes an architecture of the human-mind in relation to which distraction may be understood. Part 2 breaks down the major components that underlie human attention. Distractibility is not a design-flaw of evolution. To over-simplify: it is a consequence of the mind being designed to balance the requirement of (a) responding to new opportunities and threats and (b) managing previously activated motivators. Distractibility cannot be understood from a purely cognitive perspective. It critically involves motive processing. Part 3 describes work-flows to help you take control of your information processing. This is important, because as we age, we tend to become more easily distractible. Our attention—our minds’ time—is our most precious resource. In particular, chapter 11 provides you with a tetradic schema for assessing knowledge resources to help you more productively decide what to process, when to do so, and to what ends. ⁶⁷http://safariadblock.com ⁶⁸http://Readability.com ⁶⁹http://www.bumblebeesystems.com/wastenotime/
II Cognitive science The advent of Artificial Intelligence is the single most important development in the history of psychology. D. Alan Allport
4. Introduction to Part 2 [P]eople working in computer science and software engineering have, over several decades, unintentionally provided new concepts and tools for thinking about, modelling and explaining some of the kinds of phenomena cited by objectors to Darwinism. The key idea is that all organisms use information: living things are informed control systems — they use information in making control decisions, i.e. selecting between alternative possible actions, internal or internal. This idea is not new. But it is often ignored by people who ask how matter can produce or influence mind without asking how mind can influence matter and its motion, which it clearly does, e.g. as I type these words. Aaron Sloman The purpose of this part of the book is to equip knowledge workers, who may or may not be familiar with cognitive science, with concepts and findings that will help them understand and enhance their mental lives. It is helpful, for example, to understand that your mind comprises countless monitors and that you need to develop the right monitors in order to apply what you learn. Some caveats are in order. Our level of knowledge in cognitive science is unlike that of mechanical physics, for example, where authors can easily draw upon science to inform practice. Some cognitive scientists will disagree with the particular information I have chosen to present. However, I believe most of them will at least find the key ideas to be germane to the purposes of this book. They will also, I believe, find something new here. This part of the book is arranged as follows. • Chapter 5 lists and describes some of the major components of your mind and their relations (a “mental architecture”). It proposes a three-layer architecture, with working memory systems (including long-term working memory), internal motivators, a management layer, a metamanagement layer, and other components. • Chapter 6 describes a way of thinking about learning in terms of mental development. • Chapter 7 describes important findings about deliberate practice. It relates findings from the expertise literature and the test-enhanced learning literature. These findings, which I interpret in the light of the previous chapters, are the main basis for the concept of productive practice that develop in Part 3. In Part 3, I propose that productive practice as a means for developing mindware and expertise.
5. Your mind and its wares (the mind’s design) There are three Things extremely hard, Steel, a Diamond, and to know one’s self. Benjamin Franklin We may define the mind of any organism as the sum of the enduring conditions of its purposive activities. And, in order to mark our recognition of the fact that these conditions are not a mere aggregation, but form rather an organized system of which each part is functionally related to the rest in definite fashion, we may usefully speak of the “structure” of the mind. William McDougall Have you ever wondered about the structure of your mind? If you’ve taken an Introduction to Psychology course, then you’re familiar with Sigmund Freud’s architecture of mind (comprised of the id, ego and superego). If you have read cognitive psychology texts, then you might think of the mind as having processing systems (e.g., for executive functions and language) and memory stores (e.g., sensory memory, working memory, and long-term memory.) You may have read about emotional systems, for example those proposed by Jaak Panksepp. If you’re a neuroscience buff, you know about various brain regions and the latest conjectures about their functions. However, even if you’re a psychology graduate, you might not have put these big systems together in a coherent framework to understand yourself and your learning. I’ve found it useful to have a big picture of the architecture of the mind. Such an architecture describes the major components of the mind, their functions and how they relate to each other. This is something you can use to interpret your own cognition, learning, emotions and motivation. You can apply it to others, too. This chapter presents such an architecture. I have already alluded to its concepts. Of course, reading this chapter, or any other psychology text, will not actually help you understand and improve yourself, unless you work with the knowledge and try to understand and improve yourself with it. It is not, however, self-evident that understanding mental architecture is worthwhile. Nay-sayers might point to the failure of Freud’s project, to the absence of a consensual mental architecture, to disagreements on criteria for assessing architectures, and to the fact that most psychologists do not tend to refer to information-processing architectures. Computer scientists, Artificial Intelligence researchers¹ and software engineers are accustomed to working with and thinking about computational architectures. They know that to model a complex system one must think deeply about its architecture. To rationally select any architecture as a basis for proceeding with a practical or theoretical design, one must study the space of possible ¹For example a textbook on Artificial Intelligence contains 15 index entries under “architecture” (Russel & Norvig, 1995). Alexei Samsonovich (2012) of the Biologically Inspired Cognitive Architectures Society compiled a Comparative Table of Cognitive Architectures.
Your mind and its wares (the mind’s design)
90
designs and their trade-offs (Sloman, 1984, 1998). As one gains expertise in software development, one encounters and studies large numbers of architectures. It’s true that there is currently no dominant architectural model of mind. However, as Alan Newell remarked in Unified Theories of Cognition, cognitive scientists could not propose unified mental architectures until after several smaller scale theories had been developed. Investigating architectures requires integrating information about diverse functions. Given an architecture, AI researchers require years to iteratively build, test, and enhance working systems based on it. This is difficult, time consuming and expensive. Architectural modelling is a relatively new and slow research process². One should not expect a single architecture to dominate a theoretical landscape because several different architectures may be equally valid. A complex system can be modelled in different ways. Architectures can only be assessed with respect to the requirements they address. Getting the requirements right itself is a scientific task.³ Different models may address different requirements. And so, no integrated model of the mind currently permeates modern culture. I don’t think this means we should postpone disseminating architectures. It may mean, instead, that a small number of architectures should be selected that address a broad range of capabilities. The architectures should address a variety of meaningful psychological phenomena, not just the classical aspects of cognition, motivation or emotion. Sloman & Chrisley (2003) explain how architectural modeling can help refine and replace pretheoretical concepts about human consciousness and mental phenomena. In presenting and applying the architecture below, I will emphasize the following key ideas: 1. 2. 3. 4.
The mind is an affective information-processing virtual machine. Affective processing is not restricted to a small set of “emotional” or “motivational” systems. The mind is replete with internal perception mechanisms (“monitors”). The mind, while in many respects modular, is also quite messy; it hosts a huge number of control and communication pathways, many of which implement learned reflexes. 5. Memory functions are diverse. 6. Learning can be understood as mental development, which includes architectural change. We often implicitly refer to mental components. Being explicit about mental architecture can help people refine their understanding of themselves and each other.
5.1 Overview What’s in the brain that ink may character? William Shakespeare ²Cooper (2007) explains how an epistemology of progressive research programs, as described by Lakatos (1980), can be applied to cognitive architectures. ³Compare chapter 2 of Beaudoin (1994). For a good example of the importance of requirements-analysis for the understanding of vision, see Sloman (1989). This paper demonstrates serious limitations in prior work on computer vision (Marr, 1982).
Your mind and its wares (the mind’s design)
91
This chapter presents an extended version of the H-CogAff Architecture, a moderate schematic proposal of human cognition and affect by Aaron Sloman and colleagues (including myself).⁴ (See the figure below.) It is sufficiently broad to have a wide variety of instance models. In other words, it supports a family of theories. It shares features with several other high-level characterizations of mind in Artificial Intelligence. Keith Stanovich’s architecture, like H-CogAff, posits three broad layers of information processing.⁵ Stanovich (2009, 2011) referred to the H-CogAff architecture in presenting his. I start with a high level overview. Then I describe each of the major functional components. ⁴Sloman (2003) presents a series of increasingly complex architectures leading to the one presented here, the H-CogAff architecture (for “HumanCognition and Affect”). That paper compares H-CogAff to several other architectures, including Fodor (1983), the subsumption architecture (Brooks, 1991) and teleo-reactive programs (Nilsson, 1994). Hawes (2011) surveyed cognition and affect frameworks, including the one developed in my Ph.D. thesis (Beaudoin, 1994). In Beaudoin (1994), I compared a precursor to H-CogAff to PRS (Georgeff & Lansky, 1987), blackboard systems, and the contention scheduling model (Norman & Shallice, 1986). Sloman & Chrisley (2003) explain how architectural modeling can help refine and replace pre-theoretical concepts about human consciousness and mental phenomena. ⁵H-CogAff and Stanovich’s architecture differ however from the prevailing, dual-process, theories in cognitive psychology. See Stanovich (2011).
Your mind and its wares (the mind’s design)
92
Figure 5.1 H-CogAff (Human Mental Architecture)
The figure above has four columns. To simplify the diagram, the rich interconnections between components are not displayed.⁶ The perception column represents our abilities to perceive the internal and external environments. The motor column represents our abilities to interact with the physical world (including our own bodies). In humans, there’s usually a lot of mental activity between perception and action. This is captured by the central columns. The top of the second column represents several information processing capabilities: long-term memory, working memory, longterm working memory, internal motivators and alarms. Internal motivators implement, generate ⁶Sloman (2003) provides a more detailed, internally connected H-CogAff diagram.
Your mind and its wares (the mind’s design)
93
and activate motivational states. The third column depicts reactive, management and meta-management processes. The base represents reactive processes.⁷ It is tightly coupled to the environment. Reactive mechanisms implement most of our quick responses to the environment (such as when we rapidly steer a vehicle away from another that has suddenly entered one’s driving lane.) They are difficult to inspect. Management processes are more detached from external perception. They implement our abilities to deliberate, i.e., to entertain, compare and assess possible goals. They also implement arbitration and making of decisions amongst competing motivators, plans and schedules. Deliberative processes can be triggered and controlled by our perception of the environment, and our reactions to it. But they can also themselves be controlled by higher level processes, which we call “meta-management”. Meta-management and management together implement what cognitive neuroscientists refer to as our “executive functions” (Shallice & Burgess, 1996).
5.2 Functional characterization Each of the following sections briefly explains a component of the H-CogAff figure, above, including its purposes and links to other components. Delving Tip: This chapter makes frequent reference to the figure above. You might find it helpful to refer to the figure as you are reading, without needing to scroll back. The image can be displayed on a separate device or in a split display of (if you are using a PDF reader like Skim⁸. You can also take a screenshot of the figure and print it.
5.2.1 Reactive mechanisms Everyone who has ever stripped down an internal combustion engine to see how it works, knows that putting all those pieces, littering the driveway, back together into a working machine is a more daunting task. Jack Panksepp One can imagine a simple robot or insect that could be modeled with a subset of the capabilities depicted in the H-CogAff figure, above. Such an agent is driven by perceptual information, which triggers action routines that mechanistically drive it to completion without needing to consider and select between alternative representations of possible future states of the world. The first six chapters of Nilsson’s Artificial Intelligence textbook explore such reactive systems (Nilsson, 1998). They show, as the history of AI has, that reactive mechanisms can be very complex and that some of them are capable of learning and sophisticated behavior. ⁷Sloman depicts the base, reactive processes, as only being in the tower containing meta-management and management processes. However, he acknowledges that reactive processes are distributed throughout the architecture. Also, Sloman does not classify alarms as motivators. I propose they can conveniently be modelled as motivators, if we grant that not all motivators are alarms and vice versa. ⁸http://skim-app.sourceforge.net
Your mind and its wares (the mind’s design)
94
Konrad Lorenz and Tinbergen documented natural reactive systems underlying so-called “fixedaction patterns”.⁹ They found that when a brooding graylag goose notices an egg out of its nest, it will approach the egg and start nudging it back towards the nest with its beak, in a stereotypic manner. If the egg rolls away from the bird, the bird will nevertheless continue the motions it would have engaged in had the egg not rolled away. Ethologists and some psychologists thought that fixed-action patterns were rigid and complete; however, it turns out that there are many learned components of these behaviors. Moreover, such reactive behavior is actually sensitive to perceptual feedback. For example, the goose controls the movement of the egg on the lateral plane such that if the egg moves too far left or right, the goose will adjust it en route (Moltz, 1965). Furthermore, the goose’s beak’s lateral motions cease when the egg is removed. As such, it is better to think of fixed action patterns as relatively rigid, stereotypic responses than completely ballistic ones. In humans, reactive mechanisms abound though they are subject to more controls than many other animals. Some people refer to reactive mechanisms as “Type 1” processes.¹⁰ Stanovich (2009) attributes these processes to the “autonomous mind”. These processes tend to execute rapidly and to completion once started. We tend to only be superficially aware of their unfolding. Fluent typing, catching a falling object, smiling and stating one’s name all typically involve a host of Type 1 processes. To learn from knowledge resources involves developing new reactive mechanisms; finetuning the conditions to which they respond; and modulating their internal and external effects. To make further sense of reactive processes, we need to relate them to other parts of the architecture including management and meta-management processes, which some cognitive scientists refer to as “Type 2” processes.
5.2.2 Internal motivators While appraising and having appraised, action readinesses are aroused, and actions prepared and executed. […] Quite generally, there is appraisal, yes, simply because that is how the cognitive system works. It plays the action readiness keyboard. Nico Frijda With the opening quotation of William McDougall, I drew your attention to the fact that human behavior is markedly purposive. This means that our behavior is not merely controlled by the environment, but also by our goals and other internal motivators. In contrast, insects, most software and most other animals do not explicitly generate and manipulate goals. They can parsimoniously be characterized as reactive systems. When you click on a button on your mobile device to get the local weather, for example, your mobile device might execute several hundred internal actions. That causal chain is unlikely to involve the generation of a goal.¹¹ In contrast, when you yourself want to know the weather, you are in a motivated state. Whether or not you pursue this objective ⁹Lorenz & Tinbergen (1938/1961) ¹⁰Type 1 processes are sometimes referred to as “automatic”, part of “System 1”. Type 2 processes are sometimes referred to as being “controlled”,
or part of “System 2” (Kahneman, 2011). Kahneman (2011) characterizes the differences between these two aspects of the mind in detail for a general audience. See also Allport (1989). Kahneman (2011) assumes that reactive systems are inherently rapid. However, this is not always the case (Sloman, 2003; Nilsson, 1994). ¹¹This is not to say that it couldn’t involve generating and considering motives, however. Of course, motivator generation, like everything else in the mind, is reactive (and even reflex-like) in the sense that it is mechanistic. In the traditional System 1 vs. System 2 distinction, one would classify asynchronous motive generation as System 1, and synchronous motive generation (e.g., in means-end planning) as System 2.
Your mind and its wares (the mind’s design)
95
may depend on your priorities, the means of ascertaining the weather, and the costs and benefits involved. The box labeled “Internal motivators” in the H-CogAff figure, above reflects some of our abilities to generate and activate motivators.
Figure 5.2 Internal Motivators
An “internal motivator” is an information-processing control state that disposes one to create goals and assessments of particular objects as being liked or disliked, events as being desirable or undesirable, and behaviors as being right or wrong. Internal motivators underlie our attitudes, our goals, our standards and our concerns.¹² Internal motivators are “generative”, meaning that they can also lead us to have new goals, new desires, and new standards. For example, if you generate and execute a goal to go to check the weather forecast, you may generate a sub-goal to get your smartphone. Some motivators are goals, but not all motivators are goals. Some are attitudes and norms. When we speak of a wish, want, urge, desire or intention, we’re also implicitly referring to motivators. I often use the term “goal” generically, to include wishes, wants, urges, and desires, whether or not the motivator is an intention (an adopted goal); however where there is potential for ambiguity, I use the term “motive” for the more general case. Some people find the expression “motivator”, as defined above,¹³ confusing. This may be because in natural language “motivator” often refers to an external object of motivation (e.g., the chocolate torte one craves) as opposed to the internal state that controls our pursuit or avoidance of the object (roughly, our desire to eat the torte). To prevent confusion, I recently began speaking in terms of “internal motivators”. For brevity, I use the expressions “internal motivator” and “motivator” interchangeably. “Internal” is meant to draw your attention to the internal control state; it is not meant to imply that the motivator’s referent is internal. For example, the desire to throw a torte at someone else is a motivator even though its referent is external to oneself. Internal motivators introduce another level of indirection between perception and action. Having a motivator (such as the goal to check the weather) does not guarantee that one will act on it—one might reject or postpone the goal. ¹²Frijda (1986, 2009; Frijda & Parrot, 2011) presents a theory of motivation and emotion in which the concept of concern plays a vital role. His concept, like our concepts of motivator and motive generators, admits of a distinction between goals and the control states from which goals are derived. ¹³See also Sloman (1987) and chapter 3 of Beaudoin (1994).
Your mind and its wares (the mind’s design)
96
Motivators involve evaluators, with respect to which one can assess referents (options, states, objects, events, or situations) as desirable or undesirable, right or wrong and appealing or unappealing. The same state of affairs may be evaluated along multiple dimensions. A referent may be assessed highly in one respect and less so in another. (For example, a potential companion might be judged to be attractive and intelligent but too capricious.) Affective evaluations may be qualitative (as opposed to quantitative), such as judging that a potential companion is narcissistic. Preferences are not necessarily transitive or stable¹⁴. An example of the former is to prefer cake to pudding, pudding to ice cream, and ice cream to cake. This suggests that there is not a common currency in the mind used to assess affective referents. The representations and mechanisms of evaluation are varied and complex. Cognitive scientists have yet to adopt the concept of motivator, typically writing in terms of goals instead. This is partly because they don’t tend to concern themselves with motivation (as information processing.) I believe it is just a matter of time, however, before these concepts receive wide-spread attention (or are re-invented), as behavior outside the laboratory cannot fully be understood without them, nor can affective phenomena.
5.2.3 Management processes (Deliberative processes) Often when a person generates a motivator, she will deliberate about it. For example, Janet of Waterflop was bored in a morning meeting. She felt an urge to go for a walk at lunch. However, she suppressed the urge to check the weather forecast on her smartphone. We might say that she generated a goal to walk, and a subgoal to check the weather, but postponed them both after quick deliberation.¹⁵ There is a huge variety of ways to respond when a motive surfaces, such as to: • assess the motive (e.g., its importance or urgency), • develop possible plans to satisfy it (ways of achieving the goals), • develop possible schedules for it (when, or under what conditions the motive could be pursued), • determine and assess the consequences of candidate plans and schedules (e.g., side effects of a plan, or costs of delaying pursuit of the goal), • determine what motivators would follow from adopting the motive,¹⁶ • decide whether, when and how to pursue the motive, using more or less of the previously generated information. Deliberate decision making is a hallmark of rationality. Deliberation and decision making make use of different mechanisms from those available to the purely reactive layer of mind. They are slower. They involve imagination, counter-factual reasoning (i.e., thinking of possibilities), predicting, and storing information for potentially extended periods of time. They also involve ¹⁴Tversky (1969) demonstrates intransitive preferences. This is discussed in Beaudoin (1994) in the context of a motive-processing architecture compatible with H-CogAff. ¹⁵It’s reasonable to assume that if she had this experience several times, she would develop a new meta-management rule not to consider checking her smartphone for weather in meetings. The surfacing of the motivator to check her smartphone would quickly trigger this rule, without need for further deliberation, unless this motivator were very insistent. This description uses terms that are explained below. ¹⁶This is a special form of consequentialist reasoning. Compare Baron (2008, pp. 473-475).
Your mind and its wares (the mind’s design)
97
integrating diverse information, including (often quite vague) information from long-term memory that requires reasoning. These are the capabilities that are tapped by tests of “fluid intelligence”. They are functions of the “algorithmic mind”, as Stanovich puts it. Deliberation is normally more mentally taxing than purely reactive processing. Deliberation can be more or less elaborate and time consuming depending on the circumstances—such as the stakes and time available—and one’s thinking dispositions. Cognitive misers skip out on deliberation. Chrissy quickly decided to ignore Pat McKeough’s advice to sell her ExtantCo stock whereas Jack carefully weighed the options. Alas, the quality of one’s decision is not always proportional to the time one spent reasoning. Sometimes quick, rule-based management processing is better than extensive deliberation.¹⁷ An important aspect of intelligence is to develop (or “compile”) new reactive processes from extensive, elaborate management processing. This is to ensure that our quick judgments be optimally beneficial. As we will see in the section on long-term working memory, this compilation also involves gradually developing elaborate, optimized long-term memory structures and processes. So we can expect that with experience, Jack was able to handle his investments with less effort. As briefly described below, management processes are time-sensitive and interruptible.¹⁸ Management processes may be interrupted “bottom-up” by external distractors, “laterally” from all kinds of activated mental content, and “top-down” by meta-management processes. Humans are able to pursue goals that require careful deliberation in spite of potential distractions. They can normally resume their deliberation and actions despite their limited working memory, sometimes by using the environment as external memory. While our introspection makes these features (and various other capabilities described here) appear natural and easy, they are significant evolutionary accomplishments. As people age, resisting distraction and resuming suspended mental activities becomes more challenging.
5.2.4 Motive generators Since the early days of Artificial Intelligence, researchers have been designing machines that represent goals and solve problems.¹⁹ For instance, Herbert Simon and colleagues created GPS (“General Problem Solver”) in the late 1950’s. This program would be given a goal, such as to resolve the Towers of Hanoi problem²⁰, a description of logical operations that are useful for solving the problem, and a description of the current state of the world. It could engage in means-ends-analysis to determine the steps required to convert the current state of the world into the desired state. While this program would not impress many people today, it was at the time revolutionary. It was a clear demonstration that machines could be designed with goals and planning capabilities. The limitations ¹⁷Gigerenzer, Todd & The ABC Research Group (1999) present and explain many examples of reliable but simple decision-making rules (heuristics). See also Payne, Bettman & Johnson (1993). For formal ways of reasoning about decision quality as a function of time, see research on “anytime algorithms” (e.g., Horvitz, Breese, & Henrion, 1988; Russell & Wefald, 1991; Beaudoin, 1994). ¹⁸The dynamic, unpredictable, multi-motive requirements of autonomous agency have a significant impact on the architecture. See Hawes (2011) for recent work addressing these requirements. See Beaudoin (1994) for a more detailed discussion of the requirements. Donald (2001) explores how different species meet these requirements to different extents with different levels of consciousness. ¹⁹See chapter 12 of Boden (1977) for a review of early problem-solving programs. Boden (1978) contains a landmark analysis of purposiveness in humans and machines. ²⁰http://www.ccscjournal.willmitchell.info/Vol12-96/No2a/Daniel%20W%20Palmer.pdf
Your mind and its wares (the mind’s design)
98
of this program (discussed in Boden, 1978) were also, as is the rule in science, its strength: to motivate researchers to criticize the model and create ever more sophisticated artifacts. AI has been making steady progress ever since. However, a major limitation remains in most IT systems today. The goals they process (if any) are normally given to them, directly or indirectly, by their designers or their users. GPS needs to be told to solve a particular problem. Having been told to do so, GPS will generate a collection of subgoals to reach the end state. Each subgoal, itself, may lead to the generation of a number of other goals. Humans also often engage in means-ends analysis and other goal management processes. Given a goal, such as to develop understanding from reading a good book, we develop plans that themselves involve goals (e.g., to make a note of the most potent concepts in a paper), and so on. Humans, however, are capable of generating new purposes that they implicitly treat as valuable in themselves. This is not to say that the new goals are not caused, or that they are not in some abstract sense derived. However, some goals, once generated through management processes, are treated as needing to be pursued in themselves. A person may lose sight of why he or she is pursuing them. These goals acquire what Gordon Allport referred to as “functional autonomy” (Allport, 1937, 1961). A man becomes a step-parent. If he perceives that the child is in danger, he is intrinsically motivated to protect the child, not for some ulterior motive. Some motivators may become more important to the individual than what triggered them in the first place. This applies not only to goals, but attitudes and standards. Functionally autonomous motivators may be helpful or harmful to the agent. An important and common form of irrationality is to subscribe to moral or ethical rules that are counterproductive to oneself and others. However, a motivator may be functionally autonomous yet still be adequately managed by the agent. When a motivator becomes functionally autonomous, its host mind tends to generate new internal motivators for it. This means that minds contain “motive generators”. These are mechanisms that monitor the internal and external environment for opportunities to satisfy the motive and for threats to the motive. When opportunities are detected, they generate a new motive, to seize the opportunity. When a threat is detected, they generate a new motive to prevent, minimize or otherwise counteract the threat. For instance, when Jack detects that one of his stocks is plunging, he (automatically) generates a motive to prevent the loss, which he may of course override. The human mind’s propensity to generate motivators is one of the most important facts about human motivation. The mind contains innumerable motive generators. Most of them are mechanisms that operate asynchronously to management processing, meaning in parallel with and without interfering with, management processing. Internal motivators are themselves motive generators. Their actions are reflex-like, and so it is reasonable to consider them as “System 1” or “autonomous”. The H-CogAff figure, above, therefore locates them in the same box as reactive processes, but sets them out because of their distinctive roles. In the next chapter, I will propose that one of the most important ways to learn from knowledge resources is to generate new motive generators. To develop lasting benefits from knowledge, then, is partly a matter of developing motive generators. While the mind has a natural propensity to generate motivators, harnessing this propensity requires effort. Ways of developing motive generators are explored in Part 3.
Your mind and its wares (the mind’s design)
99
5.2.5 Meta-management The common reference to “executive function” by psychologists and brain scientists seems to conflate aspects of the deliberative layer and aspects of the meta-management layer. That they are different is shown by the existence of AI systems with sophisticated planning and problem solving and plan-execution capabilities without meta-management (reflective) capabilities. A symptom would be a planner that doesn’t notice an obvious type of redundancy in the plan it produces, or subtle looping behavior. Aaron Sloman The society of mind is a wonderful metaphor, and I will use it with gusto when explaining the emotions. But the theory can be taken too far if it outlaws any system in the brain charged with giving the reins or the floor to one of the agents at a time. Stephen Pinker Our management processes may be responsible for our highest accomplishments, however they are not the pinnacle of our cognition nor are they self-sufficient. Keith Stanovich and his colleagues have documented how smart people do dumb things. One of the main sources of this, he claims, is that smart people don’t necessarily apply themselves sufficiently. (Another one is that they have not developed their minds sufficiently, which is what this book aims to address.) Having the mental abilities to solve a problem is one thing. Using these abilities is yet another. We saw in Chapter 1 that Stanovich describes cognitive miserliness as tending to cut thinking short of that which is necessary to solve a problem. Recall that most subjects who were asked the truth value of “A married person is looking at an unmarried one” answered that it couldn’t be determined. This was mainly due to them failing to think the problem all the way through. Stanovich explains such errors by appealing to faulty “reflective processes”. Sloman and I use the expression “meta-management” rather than “reflective mind”, for reasons described below. Meta-management supervises and controls management processes. Meta-management can detect flaws in reasoning. It can direct the management layer to keep trying to solve a problem, to try an alternative strategy, to postpone thinking about the current goal²¹, to speed up, etc. This implies that the mind contains a collection of monitoring services that respond to internal states (e.g., errors in reasoning or a gap between a proposed solution and requirements) by generating motivators about management (e.g., to continue reasoning, or to bridge the gap). To carry problem solving to successful completion, one’s meta-management layer must function properly. In following chapters, we will frequently return to the observation that a major difference between “good” learners and “poor” learners is that the former are better at detecting flaws in their reasoning and understanding; they also respond to these flaws with more effective thoughts and cognitive goals. Part three proposes several techniques to support the critical meta-management function of identifying and tracking “knowledge flaws”. (Errors of cognitive omission are referred to as “knowledge gaps”.) While meta-management is weaker in some persons than others, it is weak in everyone. For example, meta-management cannot directly dismantle or suppress asynchronous motive generators. ²¹Scheduling thinking is referred to as “deliberation scheduling”, a form of meta-management.
Your mind and its wares (the mind’s design)
100
You can try to stop drinking fluids. But even the most skilled meditator can’t stop herself from being distracted by the desire to drink and thoughts of fluids. One can’t simply will oneself to stop thinking about a recently departed lover one highly valued. (Compare the discussion of perturbance, below.) Meta-management must have appeared quite recently in phylogenesis. It is further weakened with prolonged exertion, hunger, and consumption of alcohol. Cognitive miserliness refers to the tendency to cut thinking short. “Cognitive parsimony” is a more neutral expression. Cognitive parsimony in itself is often called for. The problem at hand may not be sufficiently important for excessive reasoning. Reasoning on one problem detracts from reasoning about other problems. Reasoning is also tiring. There may also be important temporal constraints to deal with. (Compare the discussion of productive laziness, below.) If you’re playing chess on a timer, your deliberation strategies had better be time-sensitive. AI researchers, Thomas Dean and M. Boddy explored “anytime algorithms”, i.e., algorithms whose solutions improve as a function of time available. (Chess software deploys such algorithms.) One would often want the management layer to use this type of algorithm such that it can be interrupted by meta-layer processes and return a response whose quality is a function of time spent reasoning. AI researchers introduced the concept of “utility of computation” to refer to the expected value and costs of information processing. More generally, to behave rationally, humans must not merely tend to select appropriate behavioral plans, they need to select appropriate reasoning strategies. Some AI researchers would say that intelligent agents ought to optimize their “utility of computation”. The meta-layer proposed by H-CogAff is responsible for deliberate selection amongst reasoning strategies. Given that management processes are slow and limited, it is essential that they should be used properly. So the small decisions we make about what to think about and how to think are of great importance. This is argued persuasively in Do the Right Thing (Russell & Wefald, 1991). One’s meta-management processes can become better at detecting and responding to various problematic states into which one’s management processes fall.²² Some of these are emotional states of “perturbance” (also known as “tertiary emotions”), where insistent thoughts and motives distract one’s attention. When perturbances interfere with the pursuit of important and urgent goals, it might be beneficial to reduce them. That is sometimes more easily said than done. This is partly because management processes also are controlled by reactive motive generators that don’t have access to many other mental resources and are incapable of reasoning.²³ Before the advent of AI, many people believed, and some people outside AI still believe, that the ability to reflect on one’s thinking is purely human. AI systems capable of meta-level reasoning are proof to the contrary.²⁴ Even systems that would not be classified as AI systems, such as UNIXlike modern operating systems, have plenty of self-monitoring capabilities. Sloman and I prefer to ²²Meta-management processes implement some of the functions referred to by increasingly popular mindfulness-based psychotherapies, including acceptance and commitment therapy (ACT), as belonging to “the observing self” (Hayes, Strosahl, & Wilson, 2011). ²³Management conundrums and a priori reasons to limit the control of attention are discussed in detail in chapter 4 of Beaudoin (1994). Sloman and I have documented many other purposes of meta-management—e.g., Sloman (2003)—some of which are discussed below. See Kruglanski et al. (2012) for an attempt to quantitatively formalize the “forces” that determine the cognitive effort a person expends on a given task. Although their analysis omits important contributors to the decision to engage in a cognitive activity (e.g., the distinction between insistence and importance of motivators, Beaudoin, 1994), it is worth pondering. Pessoa (2013) presents a neuroscientific theory of the interactions between attention and motivation that leverages Kruglanski et al.’s analysis. ²⁴See Russell & Wefald (1991) for an important survey of the first 20 years of this AI research. Cox (2005) describes how AI research on meta-level reasoning is pertinent to psychology. See also Beaudoin (1994).
Your mind and its wares (the mind’s design)
101
speak in terms of “meta-management” than “reflection”, partly because there is much more to metamanagement than self-monitoring. Moreover, self-monitoring is widespread throughout the mind (as are feedback mechanisms in the brain²⁵); many low-level, non-deliberative functions involve self-monitoring (e.g., in the control of posture and eye-movement).
5.2.6 Interrupt filters and perturbance (tertiary emotions) We are several sections into this presentation of mental architecture and I still have yet to talk about its most frequently discussed components: working memory and long-term memory. These functions, in fact all of cognition, serve the organism’s purposes. However, to pursue our high-level sketch of the mind, we need to know more about how the organism processes its motivators. This in turn requires that we characterize motivators in more detail. I specified several types of motivators in a section above. This section briefly describes how low-level motivational processes interact with higher, management processes. Perhaps due to people’s tacit understanding, and the scientific imperative to simplify, the concept of goal is usually taken as universally and sufficiently understood. However, like many other concepts from folk psychology, the concept is actually much richer than we normally acknowledge. Thus, many of the capabilities required of machinery to process goals are not normally recognized or discussed even in cognitive science. Our understanding of goals needs to be enriched for a more rigorous understanding of human psychology. The H-CogAff architecture invites us to consider the richness of goals.²⁶ The first move is to consider goals more generally as motives. A motive involves a mental representation of a state of affairs towards which an agent has a motivational attitude, such as to bring the state about, prevent it, make it true faster, or slower, keep it true, and so on. Motives can, of course, be more or less abstract. They may be dormant or more or less active. When motives become (sufficiently) active, they may be subject to management processing. I mentioned above that evaluating goals in terms of importance (value to the organism) and urgency (roughly, importance as a function of time and conditions) are management functions. However, management processes are inherently serial. They use limited capacity buffers (including working memory, discussed below). They also use multiple mental resources that are distributed across brain structures (some of which we can think of as long-term memory).²⁷ Management processes are often required to execute critical tasks that if interrupted could have drastic consequences (e.g., fetching a baby who is dangerously close to a precipice). For this and related reasons²⁸, natural selection favored organisms that optimized access to management processes. Here lies an interesting problem, discovered by Aaron Sloman (Sloman, 1985a; Sloman & Croucher, 1981), for AI researchers who want to design human-like goal processing systems. It ²⁵For a strong argument about the importance of feedback in the brain, see Hawkins & Blakeslee (2005). ²⁶See chapter 3 of Beaudoin (1994). ²⁷Retrieving (or activating) information from (or in) long-term memory requires time and mental energy effort. The concept of access to physically
distributed information in the brain in relation to consciousness has particularly been developed by Baars (1988) and Shanahan (2010, 2012). Donald (2001) makes an interesting argument that deliberate memory access, which he calls auto-cuing, was a critical achievement in the evolution of distinctively human consciousness, and the creation and usage of what we would call World 3. ²⁸Engineering reasons for limited capacity of attention are discussed in detail in chapter 4 of Beaudoin (1994).
Your mind and its wares (the mind’s design)
102
is a problem that in a sense must have been “solved”²⁹ through evolution—random variation and natural selection. The problem is germane to understanding the human mind. It is advantageous for an organism with a management layer not to allow its management processes to be diverted when the cost of disruption exceeds the benefits of disruption.³⁰ However, the (mental and physical) resources necessary to accurately assess motives are only available to management processes, not to reactive processes. How is the agent to know whether it should be distracted without first distracting management processes? Like most cases of problem solving, the real insight comes from discovering and understanding the problems. Once the problem is discovered, researchers can posit a space of possible designs to address it. Sloman’s solution takes an architectural form, meaning that it can only be understood by considering several aspects of the architecture and their relations. He posits that motives vary in insistence. The insistence of a motive is its heuristically determined propensity to disrupt management process.³¹ There are many ways to design systems that satisfy this requirement. We could assume that insistence is a simple numerical measure within certain bounds (e.g., 0 to 1). However, much more sophisticated designs are also possible. Sloman also appeals to variable threshold “interrupt filters”.³² These are mechanisms that determine whether motives should be able to distract management based on insistence information. The permeability of the filter, and thus the interruptibility of management processes, varies as a function of factors such as busyness. For example, if a baby is approaching a ditch but not in imminent danger, its caregiver may be able to consider some other problems while heading towards the baby. This description does not explain in detail how the challenges of designing insistence and “filters” are resolved. What it does is to analyze the functionality and to draw attention to what needs to be designed. Cognitive scientists need to explore heuristics for determining insistence of motivators in ways that approximate importance and urgency without requiring management processes to do the work at the time of motivator activation. The H-CogAff architecture provides a high-level account for the dominant type of emotion experienced by humans (and future robots). These are perturbances, also known as “tertiary emotions”.³³ Reactive processes may trigger insistent motivators that penetrate the interrupt filter. These motivators then tend to distract management processes. Because management processes are not fully controlled by meta-management, the meta-management layer cannot necessarily prevent this distraction. Consider the grief experienced by (hypothetical) Chrissy for several months after her multi-million dollar fortune evaporated. The motivator to do something about her financial loss ²⁹Of course, evolution does not purposively design organisms. But it is convenient in AI to consider evolution to be a designer (Dawkins, 1996). See also Sloman (2005). ³⁰See chapter 2 of Beaudoin (1994) and Russell & Wefald (1991) for a characterization of the requirements of autonomous agency implicit in this discussion. ³¹These are “System 1”, “Type 1” , or “autonomous mind” processes, depending on your terminology. Pessoa (2013) argues against this distinction in favour of a continuous notion. The tenets of H-CogAff are compatible with many of Pessoa’s arguments about the intertwined nature of cognition and affect. It will be important to attempt to reconcile in detail H-CogAff with the findings reported by Pessoa. This will require distinguishing the traditional System 1/System 2 distinctions from the messier (and more realistic) distinctions of the H-CogAff architecture. ³²Models of attention that use the notion of filter have long since gone out of fashion in cognitive psychology. Admittedly, the term “filter”, with its physical connotations, is not particularly inviting. In defense of our use of the term, it is important to realize that Artificial Intelligence has progressed since Broadbent (1958) introduced the term “filter” in cognitive psychology; and so has the cognitive psychology of attention (compare, Lachter, Forster & Ruthruff, 2004). Moreover, the space of possible attention protection mechanisms has not been fully explored. See Beaudoin (1994) for a detailed analysis of insistence, that relates it to other properties of motivators. ³³Primary and secondary emotions are discussed below. For a discussion of perturbances, see Sloman (1987, 2003) and Beaudoin (1994). One can accept the notion of perturbance without conceding that so called primary and secondary “emotions” are emotions.
Your mind and its wares (the mind’s design)
103
kept coming back to her even though she could do nothing about it. The motivator was repeatedly activated with sufficient insistence to “resurface”. This is allowed by H-CogAff, because motive activators are reactive processes not fully controlled by higher layers. The management layers cannot directly neutralize internal motivators. For a more detailed explanation of how this theory accounts for grief, see Wright, Sloman & Beaudoin (1996) and Beaudoin (2011a, 2014b). The management layers’ lack of control of mental processing serves several purposes, including to induce learning. For example, Chrissy, experiencing grief, engaged her deliberative resources to learn from her mistake. In this scenario, she ultimately resolved thereafter to capitalize on potent, high caliber knowledge resources; to this end, she enacted a plan. That, of course, is what this book is about.
5.2.7 Alarm systems and emotions What it means to say that a mechanism evaluates a state or event as good or bad will be slightly different in different contexts. This cannot be defined simply in terms of the form of output of some classification mechanism, whether it is binary, symbolic, or a value on a continuous scale. Rather, what makes it an evaluation is the functional role of the classification within the larger architecture. Aaron Sloman I have already noted that management processes are relatively slow. Yet sometimes, when faced with an urgent problem or opportunity, an agent needs to rapidly reorganize its behavior. This often can partially be achieved by triggering an insistent motivator, such as to protect a toddler from an oncoming Rottweiler. However, the surfacing of even an insistent motivator is not always sufficient to rapidly reorganize behavior. Autonomous agency, through evolution, learning or AI design, calls for “alarm systems” to predispose an agent to response patterns that are appropriate to the urgent contingencies. Alarms, like other motivators, can be triggered in various parts of the architecture (from external perception, to perception of management and meta-management processes, and even records of reactive processes)³⁴. However, they have more global effects than other reactive layer components, which is why I set them apart in the H-CogAff figure, above. Here is a sampling of the diverse types of reactions that alarm systems can generate in humans: • minute changes in posture, pupil dilation, heart rate, etc. (e.g., 20-year old bachelor(ette) faced with attractive person of opposite sex), • a behavior sequence (e.g., deflecting a projectile that is heading towards an infant). • freezing (e.g., when suddenly faced with an armed enemy, or receiving the news that one’s child has had a serious car accident), • alteration of tone of voice, facial expression and verbalization (e.g., when she was frustrated, Ralph’s wife tended to criticize him, with a curled upper lip—a predictor of impending divorce according to John Gottman). ³⁴The internal state of reactive processes is typically considered to be encapsulated and private. However, their operations may leave traces in various buffers that may be accessible, if not globally, at least to some other mechanisms. (Compare the notion of scope (or visibility) in computer science.) This follows from the fact that just about any skilled performance can be improved with deliberate, conscious attention.
Your mind and its wares (the mind’s design)
104
You may recognize some of these responses as signs of emotion. You would be right if scientists could agree on what “emotions” are. More than any other set of terms in the English language, affective terms (chiefly related to “emotions”, “moods”, “attitudes” and “desire”) have caused researchers to tread theoretical water. Researchers have such different views of emotion that they study very different phenomena. That makes it difficult to compare their theories.³⁵ Constructing a powerful understanding of affective states requires a multi-pronged approach.³⁶ Conceptual analysis and empirical literature reviews are essential to get a sense of the space of the phenomena to explain³⁷: their variety and structure.³⁸ One also needs to proceed like an engineer: to study the requirements of the system and the space of possible designs. This is the multi-pronged approach we have taken with the H-CogAff theory. The H-CogAff theory distinguishes three broad classes of emotion, each of which involves alarms systems to different extents.³⁹ The systems are distinguished by their sources and main internal effects. Primary emotions involve the readying of external action systems through diverse physiological responses.⁴⁰ Psychobiologist Jack Panksepp has described seven such systems⁴¹, including: seeking, rage, fear, lust, care, panic/grief, and play. He has attempted to specify these systems with respect to the available neuropsychological and behavioral evidence. When the seeking system is aroused, for example, the agent is manifoldly primed for all kinds of approach behavior. For a general readership, Gottman & DeClaire (2001) described Panksepp’s seven systems in terms of “emotional command centres”. By mastering this theory, one can comprehend affective reactions in terms of these systems. One can conceive bids for connection as an underlying (primary) emotional system at work. For example, as Ralph got to know his new girlfriend, Sally, he found that she sought fulfillment through caring for family and friends. Her priorities were strongly influenced by her “caring” emotion system. When she excitedly proposed hosting her nieces for the week-end, he saw this system at work. In contrast, his dominant system was his seeking system. They learned to spend time together by creatively designing activities that engaged both systems, such as exotic family trips. ³⁵Wechsler (1925) laments the fact that there is no consensus in the meaning of the term emotion amongst emotion researchers. Using a survey, Izard (2010) shows that this is still the case. Dixon (2012) provides some historical context on the different meanings of the term emotion. Read & Sloman (1993) offer helpful suggestions for dealing with the semantic welter. As footnoted in the section on “Alarm systems and emotions” in chapter 5, I introduced the term “perturbance” in the early 1990s to avoid disputes about the “real” meaning of the term “emotion”. ³⁶See Sloman (1993a) for a comparison of these different approaches. Cognitive science is by definition interdisciplinary. ³⁷Ortony, Clore & Foss (1987) analyzed the English affective lexicon. Ortony et al. (1988) provides an overview of semantic structure of emotion-like states. See also chapter 3 of Beaudoin (1994). Compelling theories of moods are hard to come by; but see Thayer (2001). ³⁸Chapter 12 discusses conceptual analysis in more detail. I echo Sloman’s (2010a) call for conceptual analysis to go beyond analyzing meanings that people use (logical geography) to analyzing the space of possible concepts (logical topography). ³⁹One could side-step terminological issues alluded to above, which would lead some to question whether Panksepp’s systems are emotional systems or motivational one, and label primary emotions as motivational systems. (Compare Frijda & Parrot, 2012.) Until recently, I avoided using the term “emotion” in this way myself, and I am not attached to the label. (I introduced the term “perturbance” in the early 1990s to avoid disputes about the “real” meaning of the term “emotion”.) See also Chiew & Baver (2011) on the problem of distinguishing between emotion and motivation. Our triadic account of emotion should not lead one to assume that all human emotional experience is a blend of different primary emotions (“psychological primitives”), a view aptly criticized by Ortony & Turner (1990). We speak in terms of primary emotions, not basic emotions. Note that Panksepp’s theory has evolved since Ortony & Turner’s paper; however, it is still not immune to all of their criticisms. ⁴⁰This is not to deny that primary emotions in humans also affect higher levels of the architecture. It is instead to suggest that before the higher order systems evolved, the primary emotional systems were in place. Similarly, the higher emotions can also involve the lower systems in humans. In humans, all three systems may be active simultaneously. ⁴¹See Panksepp (1998) and Panksepp & Biven (2012). For a summary of this theory, see McGovern (2007).
Your mind and its wares (the mind’s design)
105
To anticipate the third part of this book, I note that it would not have been possible for Ralph and Sally to learn to classify bids just by “reading” the book. Over a period of months, they worked through the numerous exercises in the book. They made a point of explicitly using the theoretical language in their conversations. This “knowledge building”, as Carl Bereiter and Marlene Scardamalia refer to it, in itself, also deepened their relationship. Understanding primary emotions is important. More generally, it is useful to understand and adopt a coherent, design-based way of talking and thinking about other aspects of human psychology. That includes secondary emotions, tertiary emotions and the other aspects of the H-CogAff architecture. This is important not only for mastering psychological books, but for research and personal development in all areas. One can better develop if one understands oneself with high caliber theoretical language that transcends folk psychology. It’s easier to control and fix a machine if you have a good theory about how it works. For that matter, it’s easier to fix your understanding of the machine if it is explicit.
Secondary emotions occur when management processes detect important contingencies through deliberation (planning, scheduling, imagining, predicting, etc.) and deliberate action control. This typically leads to the generation of motivators of approach or avoidance, with specific content, and a cascade of effects, including replacing the contents of management processes (including working memory), and possibly triggering primary emotions. For example, if a hungry primate suddenly realized that it could obtain a desired fruit by throwing a rock at it, it might generate an intense motivator to do so. This management-triggered motivator might trigger a number of internal and external changes, e.g., increased heart rate and pupil dilation. Recall that the H-CogAff theory stipulates that meta-management processes can control management processes. However, meta-management does not have full control over management processes. Internal motivators can penetrate interrupt filters, for instance, causing management processes to deal with them. Other external and internal reactions can also divert the management layer. A tertiary emotion—a perturbance—occurs when a very insistent motivator is generated and activated over a period of time.⁴² The motivator in this case is so insistent that it tends to distract attention (management processing). Even if the meta-management layer tries to keep the management layer on track, the insistent motivator will tend to surface. The concept of perturbance, like insistence, is counter-factual. That is, to be in a state of perturbance is to have difficulty suppressing a motivator if you should try to suppress it. However, one might not want to suppress it. Someone with a strong rage system and without rage compunction (e.g., with anti-social traits) might not want to suppress the manifestation of their anger. This would still qualify as a tertiary emotion, provided the agent in question has a meta-management layer. Consider the following example of tertiary emotions. Soon after his wife left him, Ralph hosted a dinner party for Sally and other important guests. That afternoon, he needed to focus on preparing ⁴²Note that the human mind also supports non-affective perturbance-like states, i.e., where the management later is disrupted by cognitive content, such as ear worms (Beaudoin, 2014b).
Your mind and its wares (the mind’s design)
106
dinner (the analog of obtaining fruit in a prior example). Adopting the designer stance, we say that his meta-management layer needed to keep his management layer on the “food preparation” task (motivator). This, in itself, need not involve a tertiary emotion. However, that day, he could not stop thinking that his wife’s unbridled temper (her “rage system”) was the major source of their marital breakdown. She should, he then felt, have done a better job of managing her emotions. Ironically, at that point, he was angry in the tertiary sense: his meta-management could not suppress those motives and affective thoughts. These mental distractions interfered with his dinner preparations. A tertiary emotion is not just a perturbance, however. It involves an alarm reaction generated at any level of the mental architecture. Here, we are not dealing with an isolated motivator. The alarming motivator causes a variety of effects throughout the organism, including those that are characteristic of primary emotions described by Panksepp.⁴³ There is a dispositional element to the concept of perturbance. One may generate a very insistent motivator that does not manage to distract attention (i.e., divert management processing). There might be other competing motivators at play. There may be factors that temporarily raise the threshold of the filter, thereby attenuating the perturbance. For example, a child might temporarily be relieved of his fear of being alone in the house by playing an attention grabbing video game. Only animals or robots that have management and meta-management capabilities can experience perturbances. If the primate described above does not have meta-management capabilities, it would not experience tertiary emotions. Some brain anomalies can prevent humans from experiencing tertiary emotions. Because normal humans have a meta-management layer, most of their emotions are tertiary. Experiencing romantic love, feeling angry, and feeling afraid, for example, all typically involve a loss of control over management processes due to insistent thoughts and motives. (They do not necessarily reflect a loss of control of behavior, however.) The H-CogAff explanation of tertiary emotions is parsimonious. It does not postulate that tertiary emotions (loss of control of attention) themselves have a function. They were not selected for in phylogenetic evolution. Perturbances are emergent properties of a system that is designed to satisfy requirements of autonomous agency.⁴⁴ Parsimony is a strong point of the H-CogAff theory of emotion compared to all its competitors—they assume that emotion has a function. Moreover, the H-CogAff theory may make it easier for people to understand their own experience. For it explains a variety of emotion types with respect to a simple but general architecture. Further, the H-CogAff theory implies that it is not possible to build an intelligent agent that meets the requirements of autonomous agency without this agent being subject to tertiary emotions. Star Trek’s Spock and Data characters are impossible, not simply because emotions are required to meet the requirements of autonomous agency, but because susceptibility to perturbance is a necessary bi-product of autonomous agency. The other theories of emotion do not have this fascinating implication.⁴⁵ ⁴³Sloman claims that tertiary emotions may or may not coincide with primary or secondary emotions. Strictly speaking, I reserve the word “tertiary emotions” for cases in which an alarm system has been sufficiently triggered. So, all tertiary emotions are perturbances but not all perturbances are tertiary emotions. I could just as comfortably have used the label “motivation” instead of emotion for primary and secondary emotions. However, this non-sectarian, tripartite taxonomy seems more conducive to a progressive research programme. ⁴⁴The rich concept of autonomous agency is explained in Beaudoin (1994). See also Donald (2001). ⁴⁵The communicative theory of emotion (Oatley, 1992; Oatley & Johnson-Laird, 1987, 2011) is similar to ours in many respects. It is also derived from Simon (1967). But there are some significant differences. It takes a functionalist stance with respect to all emotions (i.e., that emotions serve a purpose). It does not draw a distinction between primary, secondary and tertiary emotions.
Your mind and its wares (the mind’s design)
107
There are psychotherapeutic systems predicated on the assumption that our ability to directly control the generation of mental content (thoughts, images, feelings, urges, etc.) is very limited: acceptance and commitment therapy (ACT) and metacognitive therapy (MCT).⁴⁶ The information processing and architectural concepts described here could provide additional theoretical tools for these therapies. Conversely, whereas detailed therapeutic applications of H-CogAff have not yet been derived, there are potent ACT and MCT techniques that indirectly tend to limit perturbance. (See the conclusion of this book and Chapter 6 of Beaudoin, 1994.)
5.2.8 Long-term memory abilities “You can know the name of that bird in all the languages of the world, but when you’re finished, you’ll know absolutely nothing whatever about the bird. You’ll only know about humans in different places, and what they call the bird. So let’s look at the bird and see what it’s doing—that’s what counts.” (I learned very early the difference between knowing the name of something and knowing something.) Richard Feynman quoting his father (according to Bruce Alberts) It is not in constructiveness that constructive imagination is peculiar, but in the range and play of its activity, and in the determination of its points of emphasis Sir Frederick Bartlett The box labeled “long-term memory” in the H-CogAff figure, above alludes to our abilities to store all kinds of information for use over long periods of time. This information needs first to have been adequately processed in working memory, described below. The expression “long-term memory” may evoke an image of a library or store of very discrete records that once “retrieved” need only be read by management processes—but that image is misleading. Perceiving external events involves interpreting those events in the light of previously constructed interpretations. Only interpretations, or the basis for re-interpreting are stored. Raw perceptual data is never stored for more than a fraction of a second. Moreover, we perceive and store far less information about ongoing events and a scene than we are prone to assume. Minsky (1986) refers to the impression that you see most of the information before you in a busy scene, such as a typical living room or office, as the “imminence illusion”. Remembering (in the case of semantic and episodic information, as described below) is mainly a reasoning process that involves re-interpreting previously stored information in the light of one’s current transient state and previous interpretations. It relies on coherence and “sense making”, which is economical.⁴⁷ For example, if Chrissy were asked “when did you stop working for NewCo”, she would not directly retrieve the number “2001”, unless this information was accessed frequently (in a long-term working memory state). She might reason as follows. “I was laid off when the ‘dot com’ bubble burst. That was in 2000. Within about 6 months from that, my stocks were worth less ⁴⁶See Hayes, Strosahl & Wilson (2011) for a scholarly exposition of ACT. Harris (2007) is a self-help treatment of ACT. See Wells (2005, 2008) on metacognitive therapy. ⁴⁷One way in which this is economical is that it reduces storage costs. From a few bits of information, one can in principle draw an infinite number of implications. The AI programming language, Pop-11, uses dynamic lists, which are a useful reference model for this concept. http://www.cs.bham.ac.uk/research/projects/poplog/doc/popprimer.dir/node155.html .
Your mind and its wares (the mind’s design)
108
than $15! Our suitor, ExtantCo, started laying people off at that point… My number didn’t come up until the fifth round of layoffs, but it was before Christmas. So, 2001.” Not all remembering requires as many steps; however, all conscious remembering involves some steps, and some reconstruction. Because the events were significant, she recalled them often. That helped her to instill the key bits of this story (as recall typically does). (The effects of repeated recall on memory are discussed in Chapter 7 and leveraged in Chapter 13.) We durably encode different types of information. In order to reason and communicate about long-term memory processes (or principles), we need a classification scheme. This raises some questions. • Should the taxonomy reflect brain structures? • Should it reflect empirical phenomena, such as how long the information lasts or how the processes are affected by aging? • Should it be organized by empirical research paradigm and methodology? • Should it mainly reflect design-based considerations? • Should it strive to match folk psychology (in some language)? • Should we aim to converge on classification schemes that integrate each level of analysis? If so, when should this integration be attempted? These are questions that may help the reader make sense of the literature on long-term memory. Writing 30 years after Tulving published a list of 25 types of memory, Roedigger, Marsh and Lee (2002) remarked that the list could be substantially augmented. Tulving rejoined in jest with a paper for Roediger’s festschrift, “Are there 256 different kinds of memory?” To say that there are different types of memory does not entail that there are different memory systems. There’s no consensus on memory classification. Some basic distinctions may nevertheless serve us well if we use them cautiously. • Procedural memory. Somehow, the mind stores information about how to execute behaviors and plans. This information varies by level of abstraction, from the very tightly “compiled” routines for well-learned motor behavior (e.g., walking, driving, and typing) to high-level plans (e.g., how to investigate a particular kind of exception in a computer program.) Some of this information is only accessible to reactive mechanisms that execute the actions (i.e., they are encapsulated), whereas some of it is available to management processes. Modifying and enhancing behavior (e.g., after brain damage or for the development of skilled performance) requires re-representing procedural information. • Episodic memory. Somehow, the mind stores information about previous events that were experienced or later interpreted as having been experienced. For example, Ralph recalls many of his interactions with his spouse. Gottman observed that as a relationship sours, people tend to forget good events and focus on the bad ones. Events that originally were interpreted as having turned out well are remembered as having turned out poorly. We’d expect Ralph to have the same pattern.
Your mind and its wares (the mind’s design)
109
• Semantic (or declarative or factual) memory. We somehow store information on the basis of which we make factual statements and judgments. Generalizations, principles, aphorisms, designs, existential statements, and various descriptions fall under this general umbrella. Your memory of the content of this book would be classified as semantic. The list could go on to include all kinds of objects of memory: names, faces, autobiography, various states of goals (intentions, progress, actions one has tried out, partial results, etc.). Some of these objects are implicit in the bullet list. Whatever the implications of this growing list for the difficulty of conducting and interpreting empirical research in psychology, it seems very likely that the human mind and its underlying layers evolved (phylogenetically and ontogenetically) many different ways of encoding, accessing, and storing information. Software engineers are well aware that there are major trade-offs in designing memory stores. They have developed huge software libraries—methods and algorithms for manipulating stored information. They select amongst them according to their priorities: e.g., speed of access, speed of modifying, permanence, interference, scalability. Rather than search for and analyze memory systems or processes, Canadian psychologists Aimée Surprenant and Ian Neath propose it is more productive to study memory principles.⁴⁸ Chapter 14 presents a new memory technique that leverages the following three principles. (1) All memory is cue-driven; (2) cues are subject to overload and (3) all memories are reconstructed. The first principle implies that whenever one recalls an item from memory, this recall is triggered by something. The trigger is called a “cue” whose object is called the “target”. For example, the Canadian national anthem starts with “O Canada, our home and native land.” Canadians have heard the anthem so many times that the cue expression “O Canada”, particularly if sung properly, activates the target memory “our home and native land”. According to the second principle (cue overload), however, cues that are frequently paired with many different targets become overloaded. This means they become less effective at activating their targets. The third principle, implicit at the beginning of this section, states that remembering, like many other cognitive processes, is “constructive”. A cue may trigger the activation of a target memory, but normally the target information needs to be interpreted. When several cues are present at recall, any one of them may bias one’s interpretation. Other principles of memory are described in the section on long-term working memory and leveraged in chapters 13 and 14.
5.2.9 (Short-term) working memory One of the most obvious features of human mentation is that we can only hold a limited amount of information in our “awareness”. One can’t delve a demanding text while delving a demanding podcast. When one executes novel or demanding tasks (such as mental arithmetic), one needs to hold information in a state that allows management processes to operate on it. Cognitive psychologists have settled on the term “working memory” to designate whatever underlies this state. Management processes operate on working memory content. For example, as you read this sentence, you generate information in working memory, which is replaced as you read-on. ⁴⁸Neath & Surprenant (2005); Surprenant & Neath (2009).
Your mind and its wares (the mind’s design)
110
Cognitive psychologists have discovered many interesting facts about working memory. However, they have not come to a consensus about how best to model it. For example, some believe that there are special working memory stores for information retrieved from long-term memory, provided by perception, or produced as results from other computations. Others believe that if information is sufficiently active then other things being equal, it can be operated upon in the same general long-term memory store.⁴⁹ The ST-WM box in the H-CogAff figure, above is agnostic. Having said that, the model proposed by Ericsson and Kintsh is particularly germane to the objectives of this book and is therefore presented below. Moreover, thinking of information as being in a “working state”, if not necessarily in specific working memory buffers, is consistent with most models. It can (but need not) be interpreted as “retrieving information from LTM and storing it in a working memory buffer” or simply “activating it for management processing”. To the extent that we think of working memory as one or more stores, they are re-usable stores. Other boxes in the H-CogAff figure, above are also merely functional. Meta-management and management processes might use the same underlying virtual or physical resources. What matters for this book are the functions, not the detailed design and certainly not their mapping to brain structures. The important facts about working memory states are as follows: • Information in working memory can be more rapidly accessed and utilized than information in long-term memory. • Information in a working state tends to decay rapidly. • There are temporal and processing costs to putting information into a working state and maintaining this state. • Only a limited amount of information can be maintained in a working state at a time. • People with more ST-WM capacity tend to perform better on tests of fluid intelligence and natural problem solving tasks—if they apply themselves.⁵⁰ • A near consensus exists that general ST-WM capacity cannot be increased, however some researchers are currently challenging this assumption.⁵¹ • Mechanisms to protect ST-WM from distraction are critical to cognitive performance. (They include “interrupt filters” and other inhibitory and excitatory mechanisms.) They are an important research target because the potential for enhancing them with cognitive training may be significant. • As discussed in Chapter 7, ease of access to long-term information, in particular domains, can be increased with deliberate practice.
5.2.10 Long-term working memory Perhaps if we should meet Shakespeare, we should not be conscious of any steep inferiority; no: but of a great equality— only that he possessed a strange skill of using, ⁴⁹See Howes (2007) for an overview of human memory. ⁵⁰Unsworth & Engle (2005). ⁵¹See an article by Robert J. Sternberg’s article http://www.pnas.org/content/105/19/6791.full . There is considerable R&D aimed at increasing
working memory capacity with software but more research is required.
Your mind and its wares (the mind’s design)
111
of classifying, his facts, which we lacked. For, notwithstanding our utter incapacity to produce any thing like Hamlet and Othello, see the perfect reception this wit, and immense knowledge of life, and liquid eloquence find in us all. Ralph Waldo Emerson⁵² Major differences between experts and non-experts include (1) the vast amount of knowledge that experts master and (2) how experts encode, store and retrieve this information. In order to account for the particular features of expertise, K. Anders Ericsson proposed a theory of skilled memory (Ericsson & Staszewski, 1989) and the concept of “long-term working memory” (Ericsson & Kintsch, 1995). The latter represents the intersection of long-term memory and working memory. Let’s consider an example that motivates the theory behind long-term working memory—the theory of expert memory. A master chess player who briefly examines a number of chess boards (of real games in progress) can quickly memorize their configurations.⁵³ For each board, he implicitly tracks the content of each square on the board, whether or not he is explicitly asked by the researcher to do so. He can use this information to play several games simultaneously. Moreover, the speed at which the expert accesses information about a chess board approximates the speed at which working memory content is accessed.⁵⁴ Yet the amount of information they process as if it were in working memory is significantly greater than the capacity of short-term working memory. An expert chess player, playing blind-folded chess, was found to be able to make good moves within 2 seconds. The average amount of time it took him to report the content of a particular square was 2 seconds in this condition, without blindfold it was 1 second. Such blindfolded empirical results are noteworthy as they rule out the possibility that chess experts are merely storing and retrieving entire visual patterns. To explain such phenomena, Ericsson proposed a number of key principles of expert memory. 1. According to the meaningful encoding principle, when information, such as a chess board configuration, is presented, it is stored in a meaningful fashion. The new information is associated with meaningful features of previously stored information. For example, a chess expert may view the chess configuration in front of him in terms of one or more familiar arrangements.⁵⁵ 2. According to the retrieval structure principle, when information is stored in long-term memory, it is done in terms of a coherent organization of information that facilitates later recall. This structure includes cues that can be used to trigger the recall of the entire stored information. (Recall the cue-driven principle of memory mentioned above. The retrieval structure principle states that expert minds optimally work with cues.) For example, expert ⁵²http://www.online-literature.com/emerson/609/ ⁵³Cooke, Atlas, Lane & Berger (1993) found that some chess experts could do this with up to 9 chess games simultaneously (cf. Ericsson & Kintsh,
1995). It is frequently reported, even on occasion by cognitive psychologists, that experts are not better than novices at recalling random configurations of items (e.g., in chess). However, this oversimplification has been challenged by Gobet & Simon (1996a). ⁵⁴Cognitive psychologists can assess speed of information access with millisecond precision (Sternberg, 1969, 1975). ⁵⁵These arrangements are known as “chunks” of information, and the encoding as “chunking”. A pattern of chunks is called a “template” (Gobet & Charness, 2006; Gobet & Simon, 1996b). Chunking in itself does not explain expert memory. Another example of meaningful encoding is the expert memorization of large numbers. Mnemonists encode newly presented large numbers in terms of numbers that they have previously mastered, such as historical dates or running times
Your mind and its wares (the mind’s design)
112
chess players store information both in terms of a grid and in terms of associations between pieces on the grid.⁵⁶ 3. According to the memory speed up principle, with adequate practice the speed of encoding and retrieval of information will approach those of ST-WM. Chess players do not become expert overnight. They must at first quite effortfully encode and retrieve information. However, as they practice effortfully, the processes will become easier and faster. Hence expertise is in reach of those who can work for it, which is not to say that individual differences in IQ intelligence are irrelevant, but that they can be dwarfed by practice should the more IQ intelligent individuals satisfy themselves with a lower plateau. Chess, as a domain, has several properties that make it easy to research. These principles are not restricted to chess, however. The concept of long-term memory has also been used to explain more common cognitive skills, such as reading, writing and arithmetic. See Ericsson et al. (2006) for a broad sampling of papers on expertise. However, a chess master’s level of expertise is not a perfect model for all expertise. Knowledge workers develop a broad range of skills and knowledge across much broader domains than chess. So, we ought not to expect to have equal proficiency in all domains. Moreover, one can engage in deliberate practice to improve without aiming to achieve maximum velocity of encoding and retrieval of information. An effectant person may seek to improve a skill without aiming for expertise as it is technically defined in cognitive psychology. Some memory gains are often better than none. It is beyond the scope of this book to present detailed computational models of mental processes. However, it is important to note the template theory of expert memory, for which there is a detailed model developed by Fernand Gobet and his colleagues (Gobet, 2001; Gobet et al., 2001). The model is called CREST (Chunk Hierarchy and REtrieval STructure.) One of the features that makes Gobet’s theory attractive is that, like the view developed here, it emphasizes the intricate link between perception and other cognitive processes, particularly with respect to “chunking”⁵⁷. Gobet criticizes the concept of long-term working memory for being too vague (Gobet, 2000). He finds that Ericsson’s concept of retrieval structure conflates several different types of memory structures and is inconsistent with some critical data. He provides a more specific interpretation of retrieval structures. Gobet’s criticism emanates from a computer modelling approach, which is critical to cognitive science. Computer modelling allows one to discover conceptual problems to which less formal approaches may leave one blind. My reason for using Ericsson’s theory in this book is that it is simpler, of broader scope and satisfactory for a large part of the audience of this book. The theory gets many readers in the right “ball park”.⁵⁸ ⁵⁶For a detailed example of chess retrieval structures, see Gobet & Simon (1996b). ⁵⁷http://psychclassics.yorku.ca/Miller/ ⁵⁸Whether or not one subscribes to Erickson’s theory of long-term working memory, the insufficiency of the concept of working memory definitely
warrants a theoretical response. I have long found the concept of working memory to be an unfortunate “thought stopper” to which people appeal. Donald (2001) corroborates my position. Donald’s expansive notion of consciousness is worthy of careful consideration. He provides a strong argument for the theoretical importance of mental architecture, which is consonant with the H-CogAff framework.
Your mind and its wares (the mind’s design)
113
5.3 Microcognition: Monitors, parallelism and mental reflexes [System 1] continuously monitors what is going on outside and inside the mind, and continuously generates assessments of various aspects of the situation without specific intention and with little or no effort Daniel Kahneman One is not one but many. Very many. I am not referring to “multiple personalities” a concept that has diverted people’s attention away from the fascinating and deep reality investigated by scientists. Nor am I referring to metaphysical dualism, or the tripartite model of Freud. Instead, I mean that a person’s mind is comprised of mechanisms. Not two or three, nor hundreds or thousands, but millions. Most of them operate in parallel with each other and with management processes.⁵⁹ They may be dormant, highly active or somewhere in between. There are mechanisms at different scales, e.g., the big boxes of H-CogAff, and at different layers. This is true both of the mind and its underlying wetware, the brain. The brain is composed of many parts at many scales: lobes, areas, nuclei, pathways, neurons, etc. Neurons used to be considered the only cells of interest for information processing; however, glial cells are now known to be computationally important.⁶⁰ Neurons used to be considered as information processing units, but as I mentioned in Chapter 1, it is increasingly apparent that they are themselves complex computers.⁶¹ The brain is a mind-bogglingly parallel mind-producing machine. The big perceptual modalities, such as “vision”, are not one, either. They are many. Vision involves a huge number of little monitors, each doing its own little thing. Each having one or more outputs. They do not all culminate in high-level descriptors in some big buffer or space, such as “consciousness”. The outputs of fine-grained visual monitors are manifold. Some propagate backwards, some to various other systems, bypassing attentional processes. ⁵⁹In a theory that coheres with the one presented here, some of these mechanisms are called “resources” (Conlin, Gupta, & Hammer, 2010; Hammer, Elby, & Scherr, 2005; Wagner, 2006). This theory is relevant to the problems of “transfer” noted in Part 1 and addressed throughout this book. Their “knowledge resources” are mindware, whereas in this book the term “knowledge resource” is reserved for objective (exosomatic) conceptual artifacts as described in the following chapter. ⁶⁰Fields (2008a) is an accessible review of the possible role of long ignored glial cells. Glial cells regulate myelination, which affects transmission speed of neurones and therefore synchrony of neural firing. Learning does not only require forming new neural connections, but also adjusting the speed of transmission of neurones. Compare Campbell (2010) and Fields (2008b). ⁶¹Campbell (2008); Grant (2003, 2007).
Your mind and its wares (the mind’s design)
114
Figure 5.3 Sloman’s depiction of Jerry Fodor’s modular architecture as sunflower-like (From (Sloman, 2003)). We disagree with Fodor on this.
Aaron Sloman aptly contrasted the modular view of the mind with the labyrinthine view⁶², using a sunflower metaphor for the former.⁶³. The modular view sees the mind as several big modules (petals) that process and transmit information to a central administrator (the circular head) responsible for issuing output commands. The labyrinthine view is much more complicated. It assumes that fine-grained visual processors have a greater variety of outputs that need not go through a central administrator. For example, low-level visual processors participate in posture control systems. Lee & Lishman (1975) had participants stand in front of a wall that was, unbeknownst to the participants, suspended and mobile. When the wall moved away from them, they reflexively leaned forward; when it moved towards them, they leaned backwards. Yet they did not “see” the room as moving. Thanks to their vestibular system, they interpreted the optical flow as indicating they were leaning backwards and forward, respectively; it adjusted their posture accordingly. It makes evolutionary sense to have perceptual mechanisms that can quickly take action as soon as they have enough information and not to bother management processes unnecessarily. There is no need to suppose a final output of perception. Of course, people learn to rapidly perceive all kinds of internal and external events, states and processes. Most of the underlying monitors develop naturally, requiring no or little particular effort. The monitors to detect that someone has stepped on your unprotected toes are a case in point. Yet, even in many simple cases there is usually learning. For example, Lee & Lisham (1975) proposed that vestibular monitoring of muscular signals was learned and used visual information. Congenitally blind people have more difficulty with gymnastics and other balancing activities than the blind who lost their sight later in life. Their vestibular system did not receive the same early-life visual input. It is rarely noted that our ability to detect patterns means that our minds develop new finegrained monitors, let alone explained how our minds construct monitors. Nor do people tend to ponder how to develop monitors. Yet developing monitors is a critical aspect of mental development. Some of our monitors operate through polling, driven by management processes—sometimes quite deliberately. For example, in ⁶²http://www.cs.bham.ac.uk/research/projects/cogaff/Aaron.Sloman_vision.design.html ⁶³Sloman (1989, 2003).
Your mind and its wares (the mind’s design)
115
order to master the concept of bids after his first relationship, Ralph intentionally looked for bids from his partner and others. Over 15-minute periods for several weeks, he would count bids and classify them. He would also engage in retrospective analyses of the day. Eventually, he learned to automatically perceive them. Ralph’s practice may sound over-the-top. However, as discussed in Chapter 7, deliberate practice is a hallmark of expertise. It delivers results that simply reading, discussing and thinking about a book cannot. Some monitors are active most of the time, whereas others are active only in specific circumstances.⁶⁴ Normally, you will notice that your name was spoken even if you are engaged in another conversation or task. Some monitor is constantly on the look out for this. You don’t need to ask yourself consciously “has someone spoken my name?” though you might ask this question if you don’t trust your first impression—monitors can misfire! Of course, you might not “consciously” notice that your name was spoken. If you are very busy or engrossed in some task, then your management filters will protect you from distraction. (Compare the discussion of insistence, above.) Our monitors don’t just detect occurrences, they can trigger a cascade of events. We saw that visual processing can trigger unconscious postural adjustments. At a conference, if someone extends his hand to introduce himself to you, you might respond “automatically”, even if you are so preoccupied with your upcoming presentation that you will not remember meeting him. If you suffer from an obsession with germs, however, your response might be more deliberate. The latter case involves at least two motivators, one to respond to the greeting, the other to protect yourself from infection. The former activates a motor procedure (that underlies the handshaking response), while the latter may cause you to create an intention to wash your hands at the first opportunity. Part 3 describes how we assess and delve knowledge resources. It also makes suggestions to improve these processes. As you delve those chapters, consider the large number of monitors you have developed to process material and that operate, in parallel, more or less effortlessly. In a contribution to the canon of educational psychology, Verbal Protocols of Reading, Michael Pressley and Peter Afflerbach analyze the mental processes involved in reading. The book clearly shows the importance of monitoring in this representative cognitive skill. Their section on monitoring (pp. 62-75) contains 65 entries on monitoring, many of which are compound⁶⁵. Consider the following Java code, for example: Typical Java
1 2
for (int i = 0; i < r.length; i++) r[i].f();
A Java programmer would quickly classify this as a loop that applies a function to every element of an array. The code clearly matches an over-learned pattern. If you are not a programmer, your experience will be very different. However, the same programmer looking at the following code may be momentarily confused.
⁶⁴Compare Sloman’s (1978) discussion of general-purpose vs. special-purpose monitors. ⁶⁵The term “monitor” occurs at least 125 times in that book. See also Winne & Hadwin (1998) to get a sense of the importance of monitoring in
higher level, studying processes.
Your mind and its wares (the mind’s design)
116
Atypical Java
1 2 3 4 5
try { for (int i = 0; i < r.length; i++) r[i].f(); } catch (ArrayOutOfBoundsException e) { }
Something there is awry. He would rapidly detect that he is dealing with a loop. But there is something strangely unfamiliar about it, which might trigger a monitor for novelty. The code, while functional, violates the rule that one should only use exceptions for exceptional circumstances. An experienced programmer has monitors for this too, and they would fire.⁶⁶
Similarly, you probably quickly detected that something is awry with the formatting of this paragraph. You might not have a low-level monitor that quickly flags right-aligned paragraphs. This paragraph would then have triggered a general purpose detector for “something is different”. This information then activated other monitors. Those latter monitors may return results that are now sufficiently insistent to catch your attention. If you encounter more of these cases, your mind may establish an increasingly fine-tuned “right-aligned error” monitor.
People tend to think of perception as being outwardly focused. When they talk about internal perception, they tend to use expressions like “self-perception” that denote a very general, amorphous kind of perception. Yet, per Minsky (1986), the mind is composed of many small parts. Internal perception involves small mechanisms observing and responding to the states and events of other mechanisms. The observed states may be extremely specific, general or global. Most of what we “perceive” are mental (“virtual machine”) states and events—not that we are aware of this. This is not an empirical fact (minds cannot be observed from the outside). It’s an a priori fact that follows from an analysis of our capabilities and designs to realize them. While the distinction between low-level reactive processes and management processes is convenient, it’s important to note that meta-management processes in particular are also reactive; but they react mainly to the management layer. They asynchronously monitor management processes for contingencies that warrant re-directing cognitive activities. This may trigger the meta-management layer itself to redirect thinking and information gathering. Here are some examples: • While reading, you detect that you do not understand a concept being used by the author. So you decide to analyze the concept yourself and return to the document an hour later to see if it helps you understand the author’s work. • While reading, you detect that the author has contradicted one of your prior beliefs or assumptions about the world or about the text. You postpone determining which view if any is correct until tomorrow. In other words, you engage in deliberation scheduling. ⁶⁶Compare chapter 8 of Bloch (2001) on which this example is based.
Your mind and its wares (the mind’s design)
117
• While deciding which of two potential job candidates to hire, you realize that you failed to check the references of your preferred candidate. You classify this as reflecting a cognitive bias. So you decide to contact the references and ask them the same questions you asked the other candidate. • While debating an issue with a friend, you realize that you have only been considering your side of the argument, and not focusing, instead, on understanding the problem and the space of possible solutions to the problem, including those that disagree with your side. You ask your friend to switch sides and argue for your position while you argue for his, taking care to agree on the most productive framing of the problem first. These last two examples, are examples of thinking dispositions discussed by Keith Stanovich below. It is more difficult to learn to detect problems and opportunities in thinking than in external events. I emphasize the development of fine-grained monitors because of their tremendous theoretical and practical significance. Higher learning is not just about storing facts or developing output procedures. To be sure, pattern recognition and proceduralization are commonly recognized as central to learning. However, the concept of pattern recognition is too abstract to forcefully direct our personal development. The concepts of monitors, inner motivators and motive generators are more concrete. The latter three suggest an imperative: to grow the little but important mental mechanisms that implement pattern recognition and mental reflexes. Also, the theory presented in this chapter emphasizes the internal effects (motive generation, cognitive reflexes, etc.) of pattern recognition. When faced with a desire to master a conceptual artifact, for example, we should seek to develop these little “beasties” within us. Keith Stanovich and his colleagues have argued that fluid rationality is an important type of rationality. That is a multifarious concept that includes thinking dispositions and cognitive styles. This they contrast with “crystallized mindware”, such as knowledge of probability. They provide several examples of fluid rationality from cognitive science: • • • •
Resistance to miserly information processing, resistance to my-side thinking; accurate self evaluation, belief flexibility: actively open-minded thinking, tendency to seek information, enjoy thought, and fully process information (Toplak et al., 2012).
They argue that thinking dispositions can be taught. This assumes of course that one wishes to adopt them. And so one should. For rationality, by their definition, is a major source of personal effectiveness. To implement these dispositions, like other dispositions, requires that one develop monitors. These monitors would detect when one has slipped into overly miserly information processing or into my-side thinking, that one is being closed-minded, or that one ought to seek more information. Having general beliefs about the value of good thinking is not enough. One needs monitors to detect violations of thinking standards. However, merely detecting violations is not sufficient. Recognition must be tied, internally, to a motivational state. In other words, one needs motive generators (which themselves are monitors).
Your mind and its wares (the mind’s design)
118
To affect our performance, they must respond with the right level of insistence and intensity to the patterns they are designed to detect. Adequately setting the insistence of motivators is important, because the mind is pandemonium, with millions of monitors, acting in parallel, many of which might simultaneously be vying for attention by management processes.⁶⁷ Thus, to develop thinking dispositions requires that we develop motive generators that fire with appropriate insistence. For an agent to function properly, management processes must not be overwhelmed with too many distracting motivators and other information items. The minds of people with psychiatric-grade obsessions—e.g., to avoid infections—are off kilter in this respect. They generate overly insistent motivators (e.g., for hand-washing) or their interrupt filters are leaky.⁶⁸ However, if motivators are not sufficiently insistent, they will not be attended to. To illustrate these concepts, let’s return to our vignettes. To have properly understood Pat McKeough’s advice, Chrissy needed insistently to feel that her portfolio had become hugely unbalanced as a result of suddenly holding about $2M in ExtantCo stock. When this motivator surfaced, in her case, she failed to treat it with the appropriate importance, because other motivators (her work assignments, her family responsibilities) competed for her attention. To have implemented Gottman’s precepts, Ralph needed not only to detect his partner’s bids, but to perceive them as important requests to turn towards her. This is akin to the complex difference between knowing you ought to eat and feeling hungry.⁶⁹ To benefit personally from Stanovich’s What Intelligence Tests Miss⁷⁰, and the mounds of cognitive science it synthesizes, one needs, for example, to insistently feel the urge to “consider the opposite” when it is time to do so. Of course, directing one’s attention to the object of an insistent motivator is not the final step, either—not that there is a final step. For instance, having detected a bid as a bid, one needs to respond appropriately. In Chapter 14, we will address the question: how can we develop monitors, including internal motivators? ⁶⁷Pandemonium is the name of a model proposed by Selfridge (1959) that assumed multiple concurrent monitoring processes called “daemons”. It is referenced extensively in the blackboard system literature (Hayes-Roth & Perrault, 1979). ⁶⁸In the final chapter of Beaudoin (1994), I suggested that this theory could be used as a new basis for understanding and treating obsessivecompulsive disorder Beaudoin (1994). Wells & Matthews (1994) expound the same assumptions that architectural concepts are clinically relevant, and that attention, emotion and obsession share common mechanisms. See the chapter 15 and Beaudoin (2014b) for further links between this theory and psychotherapy. ⁶⁹Compare the analysis of pain in Dennett (1978). ⁷⁰http://www.keithstanovich.com/Site/Books.html
6. Adult mental development So many types of change are possible in such complex systems that we can expect to find our ordinary concepts of ‘learning’ and ‘development’ drowning in a sea of more precise architecture-based concepts. Aaron Sloman This chapter provides a conceptual toolkit to understand and ultimately enhance learning— i.e., for meta-effectiveness—that departs from common ways of understanding learning. It frames learning in high-level information-processing terms as mental development. The line between learning and development when it is drawn at all, is not clear cut. This chapter blurs it even more, while clarifying the concept of mental development. I am not, however, an iconoclast, bent, as B. F. Skinner was and some philosophers of cognitive science still are, on replacing people’s “unscientific” ways of thinking about the mind. To understand learning—or physical processes for that matter—requires taking multiple perspectives¹. What are these “common ways” of thinking about learning? Chapter 2 presented some of them— e.g.,. developing skills, norms, attitudes and propensities. There, we could also have delved into associationist concepts of learning—e.g., classical conditioning, operant conditioning and neural network concepts—which have seeped into common parlance. Long-term memory and its various posited types (procedural memory, declarative memory, etc.) are now also widely known. We can think of adult mental development in terms of “mindware development” in order to emphasize information processing concepts—concepts from cognitive science—such as the ones introduced in the previous chapter. Mindware refers to intrapersonal structures, mechanisms, processes, states, representations and events. They also include relations between some of these things. To be sure, many of these states refer to things outside of the individual, including knowledge. Minds of course are ultimately the creators and editors of objective knowledge. It is an odd fact about the English language and cognitive science that, apart from the uncommon term, “mindware”, there is no term that refers to “mental stuff” that one can develop. The term “mind” is widely used, though rarely defined. The mind is an intrapersonal system of interacting virtual machines. In humans, mind is layered on brain. But “mind” is a global concept. To understand learning, we need a term that refers to mental constituents, that is mindware. Scientists are quite comfortable with the concept of software and software programs. Of course, software is better understood than mindware. The term mindware is not an explanatory end-point. It can be used to raise many questions. When one learns something with knowledge, one develops new mindware. The practical aim of this book is to help readers master knowledge gems. In order to master knowledge gems from a theory or other conceptual artifact, one must develop several types of mindware. Some of the most important development targets include the following, which were introduced in the previous chapter (without reference to learning): ¹Chapter 3 of Clark (1989) and Dennett (1987) provide related justifications for having alternative ways of thinking about mental phenomena.
Adult mental development
• • • • • • •
• •
120
monitors, internal motivators (motive generation, activation, insistence assignment), cognitive reflexes, long-term working memory, management procedures (for simulating, searching, planning, evaluating, scheduling, decision making), meta-management capabilities (with a particular emphasis on detecting errors of omission and commission), low-level attentional mechanisms (e.g., new and better means of suppressing, censoring and inhibiting motivators and other mechanisms that might inappropriately distract management processes), representations, including representational re-descriptions, the architecture itself.
Notice that this taxonomy is not expressed in terms of “content”, i.e., the classes of knowledge or skills one should master (e.g., mathematics, accounting, leadership). It concerns, instead, the categories of mindware that were described in the previous chapter. It suggests that mastering a complex knowledge resource involves many types of psychological change. Much of the learning we do is an incidental consequence of problem solving and knowledge building. By systematically doing great work, we develop ourselves. This often requires, as Donald Schön emphasized, that we “reflect-in-action”, i.e., purposively reflect while solving problems. It also requires that we routinely “reflect-on-action”, i.e., engage in offline from problem solving. It requires as well that we process knowledge resources deeply and deliberately, something I refer to as “delving” in Chapter 12. It also requires that we practice. Most of the time, we engage in these learning activities without reflecting explicitly on ourselves. Even Schön’s concept of reflection is not necessarily self-referential. We are usually more concerned with the external problem than our own mentation. However, to pro-actively avoid getting stuck, it is useful to explicitly think about our mental processing. Schön’s framework, as rich and useful as it was, did not extend the ontology of the mind (the set of categories used to think about mental processes.) To improve how we learn, how we reflect on ourselves and others, and our learning objectives, we need better ways of thinking about ourselves. This chapter can be read as an answer to the call of Annette Karmiloff-Smith. She wrote, “A developmental perspective is essential to the analysis of human cognition, because understanding the built-in architecture of the human mind, the constraints on learning, and how knowledge [I would say mindware] changes progressively over time can provide subtle clues to its final representational format in the adult mind” (Karmiloff-Smith, 1995 p. 26). However, I do not accept the notion of a “final representational format in the adult mind”, however. Indeed, this book is concerned with ongoing adult mental development. Not only new representations, but new information formats may continue to evolve. I propose in more detail below that adult development is continuous with child development. To progress mentally, adults need to apply more deliberate (self-directed) effort than children. That involves the process of expertise, as Bereiter & Scardamalia (1993) put it. Regardless of age, new overt abilities result from changes in underlying virtual mechanisms—monitors, representations, long-term working memory, architecture, etc.
Adult mental development
121
6.1 Objective knowledge (World 3), virtual machines (World 2’) and the rest (World 1) The term “knowledge” is used in two very different ways. In one sense, knowledge is an objective, external product. In another, it involves a collection of mental states, dispositions and mechanisms. It’s easy to inadvertently slip from one meaning to the other.² I use the term knowledge mainly in the objective sense; when there is ambiguity, I qualify it with “objective”. This distinction between objective and subjective knowledge was described by Sir Karl Popper’s in his book Objective Knowledge. He developed an ontology partly as a response to the need to understand the relations between the mind and physical world, including the body. According to Popper the world consists of at least three ontologically distinct worlds (domains). [World 1] is the physical world or the world of physical states; [World 2] is the mental world or the world of mental states; and [World 3] is the world of intelligibles, or of ideas in the objective sense; it is the world of possible objects of thought: the world of theories in themselves, and their logical relations; of arguments themselves; and of problem situations in themselves. (p. 154) In World 1, the physical world, we have minerals, gases, animals, plants and all kinds of physical stuff that the resource sector of the economy deals with. In the age of cognitive science, our understanding of the mental world has evolved beyond Popper’s conception of World 2.³ I refer to the new concept as World 2’ (World 2 Prime). It is the world of virtual machines. Mental states are subjective in that they are hosted by a subject and they are the computational substrate of experience. World 2’ includes information processing, mechanisms, components, architecture, and their interactions, whether conscious or unconscious. World 2’ subsumes and goes a long way towards explaining psychological states and processes, including subjective beliefs, desires, impressions, perception and working memory. World 2’ is not complete, of course. A virtual machine must ultimately be implemented in World 1⁴. Mental states have referents that are outside themselves, namely World 3 objects.⁵ World 3 consists of products of World 2’. For humans, this is the world of cultural artifacts. Bereiter (2002a) provides a taxonomy of World 3. Some cultural artifacts are physical, such as a carpenter’s toolkit, furniture, paintings, streets and weapons. Most World 3 items are made by humans. However, there is a special case: naturally occurring objects treated as having cultural ²Slaney & Racine (2011) demonstrate the equivocal use of the term “concept” in psychology. The term normally refers to objective knowledge (World 3). However, it is often interpreted in psychology as a mental representation. While their paper does not refer explicitly to Popper’s cosmology, it does show that it is preferable to use the term concept in the sense of an external object as opposed to a mental one. I would emphasize, however, that to understand and utilize concepts involves mental representations (Sloman, 2011a), a fact that is sometimes lost by those who espouse connectionism with over-exuberance. ³Claude Lamontagne (personal communication, 1989) met with Karl Popper at his home and reported that Popper failed to grasp the implications of Artificial Intelligence for understanding mental phenomena. This, I suspect, was due to Popper’s age and obduracy. ⁴This analysis allows us to attribute World 2’ animal and artificial cognition. That depends on a fuller analysis of virtual machines and is beyond the scope of this book. When I speak of World 3 as being produced by humans, it must be understood that I also allow for anything that has a virtual machine with the right capabilities to produce World 3 items. ⁵The concept of virtual machine enables us to resolve the fundamental problem of accounting for the interaction between physical states and mental states (the so called “mind-body” problem), and the problem of characterizing our relation to objective knowledge.
Adult mental development
122
significance or taken from nature and used as tools. A rock collection or dinosaur bone as decorations or a natural stone patio are examples. Some artifacts are abstract: for example an image of a painting or source code. (That these are real is indicated by the laws of copyright and that they are commercially negotiable material.) Some abstract artifacts, such as designs, patents and theorems, are conceptual, meaning they deal with concepts and employ logic. Crucially, conceptual artifacts can be treated as objective knowledge. They serve purposes, meaning they address problems, constraints, requirements and goals. As such they are potential solutions to problems. One can apply them and reason with them; hence they differ from Plato’s realm of pure ideas. They can be improved or replaced, e.g., to the extent that they don’t adequately address requirements, new requirements are introduced or the importance of some of the requirements changes. Some conceptual artifacts have the further property of including statements about the world, such as to express new possibilities, to rule out possibilities, to show how some things are possible, or to express correlations. Bereiter refers to them as assertive conceptual artifacts. They are theories about the form of the world. We will return to them below. Let me illustrate this concepts with a scenario. Ralph and his girlfriend, Sally, are hiking in a forest. She points out how beautiful the valerian, columbine and lilies are. These flowers are World 1, as are mountains, trees and streams. The couple’s delighted perception of these items is a World 2’ state. The perceptual mechanisms underlying their evaluations of the scene are also World 2’. Sally, all dressed in white, adds⁶ “Valerian is said to help with insomnia. But Columbine root is poisonous.” This statement is a World 3 item; it has a truth value. “Yet the flowers would be beautiful in a bouquet.” Ralph misinterprets this as a “fuzzy bid” (an ambiguous request) for him to make a bouquet for her. The concept of “fuzzy bid” and Gottman’s theory about how bid-response patterns affect the course of a relationship are World 3 items (theory). The interpretation is produced and entertained by a mind, and hence involves World 2’. The interpretation as an intelligible proposition is an assertive conceptual artifact, a World 3 item. It is true or false. In saying that knowledge is objective, Popper does not deny that it may be produced by biased processes. Nor does he deny that it is produced by a subject (a being that is the subject of experience produced by the mind interacting with World 2’ or World 3 items).⁷ Instead, he is saying that knowledge is potentially public, potentially shared stuff that humans produce, work with and communicate. It’s a product from which we may draw inferences that are either valid or invalid. It’s a product that we can alter (e.g., by adding, removing or editing components) and sometimes improve. Like other tools it has objective properties. It can be assessed with respect to problems (how well does it work?). It can sometimes even be instantiated (e.g., a design can be instantiated or implemented). Crucially, knowledge in this sense is not stuff that exists in the head any more than a theory exists on paper. It may be represented and worked with on paper, white boards and virtual machines. When an entity sells a patent (a World 3 item), for example, it is not selling the author’s ability or neurons. Nor is it selling the paper or medium on which the knowledge is conveyed. It is selling the abstract product to a new owner. To say that someone “has knowledge” means that he is capable of working ⁶If a branch falls, that is a world 1 event. Hearing it is a World 2 event. Explaining that the cause of the percept is a World 1 event is a World 2’ process. The theoretical explanation itself is a World 3 object. ⁷It might be less confusing to speak in terms of “World 3 knowledge” as opposed to “objective knowledge”.
Adult mental development
123
with the knowledge. He can answer questions with and about it, he can use it to solve problems and reason. Of course, it also means that there is something in his head that implements these abilities. Between World 1 and World 3 there lie his mind and its mindware. Popper’s distinctions, emphasized by Bereiter, will enable me below to better characterize mindware, mental development, understanding, accommodation, and knowledge work. I conclude this section by applying these concepts to mindware. Knowledge is routinely treated in an objective way in the knowledge economy. Patents for example can be bought and sold. The public sector is increasingly concerned with “knowledge translation”⁸, that is to ensure that high caliber knowledge is expressed, communicated, taught and applied. These players may not be aware of Popper’s three-world ontology; however, they are definitely treating knowledge in the objective sense (World 3).
6.1.1 Mindware as World 2’: Virtual machinery Perkins and Stanovich sometimes use the term “mindware” in ways that blur the distinction between World 2 and World 3. In some cases they are referring to objective knowledge, in other cases to knowledge in the subjective sense. Stanovich (2011) defines mindware as “cognitive rules, strategies, knowledge, and belief systems” (p. 161). Here, mindware is equated with principles, and principles are World 3 items: Although it is true that more intelligent individuals learn more things than the less intelligent, much information relevant to rationality is acquired rather late in life, and the explicit (self-)inculcation of this mindware is very spotty and inconsistent. For example, the tendency to think of alternative explanations for a phenomenon leads to the ability to more accurately infer causal models of events. Such principles are taught very inconsistently (by either explicit or implicit means). (p. 146) Stanovich also refers to memes as mindware. The concept of meme explains the acceptance of conceptual artifacts with reference to Darwinian principles and the concept of replicators (Dawkins, 1976). A meme is an element or bundle of information (e.g., a jingle, a rumor, or an idea) that can be communicated within a culture. It is purportedly analogous to a gene. Stanovich makes good use of this concept to explain why smart people can have irrational, self-destructive beliefs. A gene can be lethal to a host. So can a meme, as long as “it is advantageous to itself ” (Dawkins, 1976, p. 200.) To gain this advantage requires that it can replicate itself. Consider the following uses of “mindware”: The debate should not be about whether religion itself is contaminated mindware but about the specific content that tends to go into the “slots” created by the confluence of cognitive modules that support religion […] The specifics that go into the slots can be more or less rational for the individual. (Stanovich, 2011, p. 170) there is parasitic mindware that mimics the structure of helpful ideas and deceives the host into thinking that the host will derive benefit from them. (Stanovich, 2009, p. 164) ⁸For more information, see the Knowledge Translation Clearinghouse, a Canadian Institute of Health Research (CIHR) repository of knowledge translation resources.
Adult mental development
124
Ideas and content are World 3 items. They are objective and public. So by comparing mindware to them, Stanovich seems to imply that mindware is World 3. Mindware is not the only term that has two “world” interpretations. One can think of a gene as functional information coded in a medium that happens to be physical (DNA or RNA). This is a World 2 (information processing) interpretation. Or one can think of a gene as a World 3 construct (a hypothetical construct meant to explain things). The latter is World 3. It appears that “mindware” and “meme” can both be used this way, though the World 2 interpretation of the former is more common and the World 3 interpretation of the latter is more common.⁹ Similarly, one can think of virtual machines as an abstraction (World 3) or as a particular running virtual machine (Sloman, 2009a) (World 2).¹⁰ Of course, the running virtual machine, if it is created by humans, can be thought of as a World 3 item. However, in as much as it is capable of hosting mental states and processes, it is itself a second world. Normally, however, Stanovich, Perkins and their co-authors use the term “mindware” to refer to mechanisms, data structures, and components within the mind itself. To avoid confusion, and because the term “mindware”, unlike “gene” and “virtual machine”, does not have a crisp, standard interpretation, in this book I restrict myself to the World 2 meaning. Besides, we already have a term for the other meaning, namely objective knowledge. Mindware here refers to the dynamic virtual machinery that implements psychological states, mechanisms, representations and processes. Of course, mindware may represent, create and modify the other realms.¹¹ When, in 2001, I began in earnest to address the psychological challenges involved in so-called “knowledge translation”, I adopted the slogan “from information to knowledge” in my white papers and other communications. After re-acquainting myself with Popper’s ontology, in reading Bereiter (2002a), I discovered flaws in this expression. It assumes a psychological (World 2) interpretation of the term “knowledge”. Yet in knowledge translation settings, knowledge is objective (World ⁹Stanovich appeals to memes partly in order to avoid misinterpretation of the term “mindware”: The reason I think the term meme is useful is that the new and unfamiliar terminology serves a decentering function that makes understanding the concept of contaminated mindware easier. It can help somewhat to dislodge the “beliefs as possessions” metaphor that we see implicit in phrases such as “my belief” and “my idea.” Because the usage “my meme” is less familiar, it does not signal ownership via an act of agency in the same way. (Stanovich, 2009 p. 162).
¹⁰Consider Sloman’s distinction between two concepts of virtual machines (which, incidentally, should not be confused with “virtual reality”): Two concepts of a “virtual machine” (VM), have been developed in the last half century to aid the theoretical understanding of computational systems and to solve engineering design problems. The first, Abstract virtual machine (AVM), refers to an abstract specification of a type of system, e.g. a Turing machine, a Universal Turing machine, the Intel Pentium, or the virtual machine defined by a language or operating system, e.g. the Java VM, the Prolog VM, the SOAR VM, the Linux VM, etc. An AVM is an abstract object that can be studied mathematically, e.g. to show that one AVM can be modelled in another, or to investigate complexity issues. Sloman (2009a) The second concept refers to a running instance of a VM (Running virtual machine, RVM), e.g. the chess RVM against which an individual is playing a game at a certain time, the editor RVM I am using to type these words, the Linux RVM running on my computer, the networked file-system RVM in our department and the internet – a massive RVM composed of many smaller RVMs, instantiating many different AVMs. (Sloman, 2009a).
¹¹Engelbart & Ruilifson (1999) explored a similar concept at the organizational level but that straddled World 2 and World 3: “There’s another part of this that it takes to make people effective. Its [sic] the paradigms they live by, the procedures they follow, the customs they use, the language that they employ, the knowledge[…] All of these are added on top of the basic genetic capabilities. And […] There was no name for all of this, so I just crudely coined the name human system for that. I’m still looking to see if there is any other term for it out there in the world.” The concepts of mindware and objective knowledge are more helpful than the concept of “human system”.
Adult mental development
125
3). The slogan “Turning information into mindware” is better. However, “turning” suggests a transformation, whereas the process is really one of construction and derivation (the information remains after the “conversion” process.) It is therefore more precise to speak in terms of “deriving mindware from knowledge” or “implementing knowledge in individuals”. (This is analogous to implementing a software design in a computer program.) Why bother with Popper’s three worlds? We live in the knowledge age. Economic progress is inextricably linked to the development of knowledge. We face a major challenge of implementing knowledge, not only in products and systems (World 3), but in mindware (World 2). Whether or not one adopts the framework presented here, it’s important to understand the problems and to have a clear and coherent conceptual framework that does not confound one concept with another.
6.2 Understanding understanding I have often called this theory the bucket theory of the mind. [ According to it] our mind is a bucket which is originally empty, or more or less so, and into this bucket material enters through our senses […] and accumulates and becomes digested. […] My thesis is that the bucket theory is utterly naïve and completely mistaken in all its versions, and that unconscious assumptions of it in some form or other still exert a devastating influence. Karl Popper Having distinguished between mindware and objective knowledge, I can now resume the analysis of the concept of understanding begun in Chapter 2. Let’s begin with a “schema activation” exercise.¹² Without looking back at Chapter 2, take a moment to summarize the major requirements of understanding. Then, compare your answer with that chapter. What is missing or awry in your concept of understanding? And in mine?
To understand a conceptual artifact you need to know something about the problem it is meant to solve.¹³ In Chapter 2, I described this in terms of the purpose of the conceptual artifact. For example, Gottman proposed the bid-response concept to characterize how close interpersonal relationships work, what makes them work well and what tears them apart. Those questions are the problems that his bid-response theory addresses. And, as I mentioned above, you normally need to know something about the structure of the theory, models and arguments that explain why the theory is a solution to the problem. Following Sloman’s characterization of cognitive science, we may add that it is often useful to also characterize the space of related problems (that need explanations, e.g., why some couples want to do more activities together than others), space of possible solutions, and the relations between them (Sloman, 1984, 1993a, 1998, 2000b). ¹²“Schema activation” is an educational psychology term for exercises that are meant to prepare the reader to process subsequent material (Bruning, Schraw, & Norby, 2010). “Mindware activation” is a better term. ¹³Bereiter (2002a) distinguishes problem-centered concepts and referent-centered concepts. The former are responses to a problem of understanding or explaining; they need not have any instances. The concept of force is like this. Referent-based concepts (e.g., the concept dog) can be instantiated to refer to particular items (e.g., particular dogs.)
Adult mental development
126
This means that the concept of understanding is very much relational. It has to do with grasping the relations between problems, between parts of a theory, between theories, and amongst those various elements (e.g., from concept to problem). Bereiter posited an additional subtle and critical respect in which understanding is relational: With considerable difficulty, science educators try to get across the idea that gravity is a relationship between objects. Suppose we apply the same idea to understanding—that understanding is a relation between the knower and an object of knowledge. You are not going to dig into the brain and find it, anymore than you are going to find gravity by digging into the earth. But we can learn a lot about the relationship. And, as is not the case with gravity, we can influence the relationship through our actions. (Bereiter, 2002a) His point is that to deeply understand a conceptual artifact is to have a deep relationship with it. Bereiter shows that understanding a conceptual artifact is like understanding a person intimately: Many relational statements hold between the knower and that the object. To illustrate Bereiter’s theory, let’s use the example of understanding Gottman’s concept of bid that different people might have. Their understanding depends on their relationship to the theory. Compare and contrast the following relationships with Gottman’s concept: • A journalist writing a 300-word article on Gottman’s theory, • Someone who wrote a comprehensive exam on the theory but never practiced it systematically in her own life, • Someone like Ralph (the first time) who read The Relationship Cure the first time but didn’t do any of the exercises; and Ralph (the second time) who made the effort to master it, • Someone who did the exercises in The Relationship Cure by himself but didn’t discuss them with others, • An instructor who will teach the concepts, • A psychology assistant who has spent two hours a day for several months watching video recordings of couples interacting and classifying every bid and bid-responses. This potpourri of cases illustrates the following. • There are different purposes, types and degrees of involvement with a theory. • One understanding is not absolutely better than another. Purpose and criteria of understanding vary. • Understanding is sometimes implicit. The person who used the exercises without discussing them with others might be excellent at thinking and acting with the theory but perhaps not very skilled at explaining it. The Ph.D. student might be able to explain the theory (so long as she remembers it) but in practice may fail to notice her partner’s bids or classify his and her responses to bids.
Adult mental development
127
• The domain of problems addressed by Gottman is very amorphous. Not all questions can be answered with it in a black and white fashion. However, there are clearly wrong understandings. For example, assimilating the concept of bid to the concept of question or demand would be wrong. • An indication that one has mastered the concept of bid is that one can tell stories about it and its history, such as how the concept or its importance was first discovered (e.g., Gottman’s “ah-ha” moment) and how it evolved. • If two people both deeply and explicitly understand Gottman’s theory, they can describe situations, behaviors, responses, arguments, successes and failures objectively, using the concepts in Gottman’s theory. They might not immediately agree but with some effort they can normally comprehend each other’s characterization. • These two same people can also use the theory to solve deep and subtle problems. The same general principles hold for understanding all kinds of conceptual artifacts. While of course there must be something about the knower’s mind to support understanding, the concept of understanding does not depend on these innards. To have a deep understanding of a complex theory is to have had a long relationship with it and to have manifold abilities with respect to it. To develop an understanding of a conceptual artifact requires that one has had much experience analyzing problems and solving them using it. Understanding difficult conceptual artifacts requires the development of cognitive skills involving the artifact, as will be discussed in the next chapter (VanLehn, 1996). That is a kind of relationship with the object. One of Bereiter’s main claims in presenting the position I have adapted here is that speaking in terms of mental states is not required to characterize understanding. I propose a weaker position. The concept of understanding is relational in critical respects. I believe it is important to think of understanding, for the most part, in relational terms, i.e., relations between the knower and World 3 objects. I agree that many of the terms of folk psychology will be replaced as cognitive science progresses. However, I also believe that some of the terms will be kept but recast. Many new mentalistic concepts, grounded in virtual machinery theories, will emerge and be adopted in the broader culture. This has already started to happen. New cognitive science concepts relevant to self-understanding and self-improvement will continue to emerge. Understanding for the most part comes from knowledge building and progressive problem solving. Those are the kinds of World 3 activities described by Popper and Bereiter. They involve working with problems and knowledge. However, to derive benefits over and beyond their usual knowledge work, people stand to gain from focusing specifically on learning, including practicing. This justifies my presentation of the concepts and practices of mental development.
6.3 Developing monitors As I previously emphasized, to apply knowledge, whether mainly practical or factual, one needs to detect when it is applicable. Skimming, even delving, a document does not naturally lead to the establishment of monitors. We saw in Chapter 1, that “recognition” figures at the bottom of
Adult mental development
128
the revised Bloom’s Taxonomy. This can be taken as reflecting a lack of acknowledgement of the importance and difficulty of establishing monitors. But it is also suggests the authors understand that the perceptual aspect of competence is fundamental. I suggest that when approaching a knowledge resource that one wants to master, it is useful to ask oneself what one would like to be able to perceive with it. One will not find an explicit indication, in the table of contents or index, of the monitors one ought to acquire. It is not merely that the term “monitor” is not used by the author. Rather, authors don’t tend to think in terms of instilling monitors. That is left to the reader. Books don’t tend to explain how one can establish monitors. A striking amount of text in Gottman & DeClaire (2001, e.g., pp. 55-56) concerns the development of monitors, though the term “monitor” is not used explicitly. Their tips and exercises are meant to help the reader perceive behaviours as bids and bid responses. Their work indicates the importance of developing mindware, in particular monitors (including motive generators). The concept of bid will not improve your relationships unless you can detect instances and perceive behavior in terms of bids and responses.¹⁴ If the book you are reading does not contain exercises for building monitors, you need to determine for yourself what those exercises should be to develop the monitors you need. Other practical books can be read in terms of monitors, too. Using The Lean Startup¹⁵ as an input, you can compile a list of events and states to watch out for. I selected a chapter at random from Ries’s book to illustrate this (Chapter 9, “Batch”). For reasons explained in Ries’s book, a lean principle is that it is more efficient to do work in small batches rather than big batches¹⁶. This holds for something as simple as stuffing and stamping a collection of envelopes as well as more complex work. For example, rather than writing a huge piece of software from start to finish and then submitting this to the market, one should develop software in small increments and submit to the market for feedback. The principle applies to all kinds of tasks in the organization. To apply this principle, one somehow needs to develop monitors that will detect when it is applicable. It is more difficult, but equally important, to determine what monitoring capabilities to evolve from factual documents. Understanding mechanical physics is of no use if we do not apply it when it’s pertinent, such as when one is driving too close to the car in front of them. When called upon to explain why a projectile fell short of its target, the better part of the task is to recognize that mechanical physics might apply. Having read Stanovich’s book¹⁷, one might also detect that a flaw in the projector’s knowledge (a “mindware gap” in Stanovich’s terminology) was at play.¹⁸ Monitors can observe World 1, World 2 and World 3 objects events, processes and affordances¹⁹; ¹⁴As noted earlier, not all concepts have referents. For example, the concepts of force and mass don’t. So, the more general perceptual requirement not only to classify or recognize instances but to detect when knowledge is applicable. This involves detecting the functional properties (e.g., affordances) of the world around one. ¹⁵http://theleanstartup.com/book ¹⁶This is “single-piece flow” vs. “large batch flow”. The concept does not fully apply to learning. It is often more effective to learn pieces of information separately (as well as together.) This is because by practicing information together the learner benefits from priming effects which makes performing easier during practice than during play. As Mark Guadagnoli argued, we should practice to improve, not to win (Guadagnoli, 2009). For a systematic approach to integrated learning and instructional design, see van Merriënboer (1997). ¹⁷http://www.keithstanovich.com/Site/Books.html ¹⁸This is an example of meta-semantic competence, i.e., knowledge about someone else’s knowledge. Meta-semantic competence also covers knowledge about one’s own understanding (Sloman, 2008a). ¹⁹Including detecting epistemic affordances (Sloman, 2008a).
Adult mental development
129
they can also detect relations between them.²⁰ We will see in the next chapter that an important kind of monitor is one that detects flaws in your understanding. As Sloman pointed out, the fact that a human mind can grow new monitors, and other kinds of mechanisms, means that the human mental architecture can also develop in adulthood (Sloman, 1993b, 2002).²¹ This is a fascinating topic that, to my knowledge, has not yet been explored systematically and explicitly in cognitive science.²²
6.4 Developing motivators Whereas most other species depend upon their built-in demons to do their mental work for them, we can build our own demons. Merlin Donald There would be little point in developing monitors if they could not trigger internal motivational states and cognitive reflexes. Monitoring is the trigger for reasoning, deliberation and problem solving which themselves can lead to mental development. For example, by detecting a flaw in one’s knowledge one can then work to bridge it. This in turn can lead one to establish new mindware to detect the applicability conditions of the repaired knowledge. A fascinating and theoretically vexing property of the human mind is that it naturally produces motivators²³, those World 2’ mechanisms that guide us to action. A woman “falls in love” with a man, and suddenly she is flush with inclinations to call him, be with him, and do things with him. His concerns become her concerns. You take up a new programming language, and you now notice books and articles on the subject. At cocktail parties you gravitate towards other aficionados. These felicitous tendencies often come for free, without much deliberate control. Yet one cannot simply will an effective motivational state into existence. In reading Gottman during his first marriage, Ralph discovered the importance of bids. Ralph decided that he ought to, and indeed wanted, to respond to her bids for connection—very practical. But here is where a folk psychology analysis breaks down. Wanting to respond to bids does not guarantee that one will want to respond to individual bids when they occur! Explicitly assigning a high value to something does not guarantee that one’s mind will establish and configure the motivational states that implement value. In his first failed marriage, Ralph never even learned to reliably recognize bids. And when he did, he was more concerned with his other priorities. Even when Ralph diligently worked through Gottman’s book the second time around, it required a lot of work to detect bids and respond to them appropriately. Many motive generators needed to be established. A tendency to respond to bids for connection requires, in the end, that one learn to produce or activate motivators that are sufficiently insistent. But insistence assignment is not directly under control of management processes. One way to indirectly control insistence is to regularly assign ²⁰Monitors underlie what Bereiter and Scardamalia refer to as “impressionistic knowledge”, which is not to say that the concept explains that type of (World 2’) knowledge (Bereiter, 2002a; Bereiter & Scardamalia, 1993). ²¹Wright, Sloman & Beaudoin (1996) claim that architecture dominates mechanism. The detailed mechanisms make only marginal differences as long as they support the design features required (Sloman, 1993b). ²²Mounds of empirical evidence on child cognitive development could be explored in these information processing terms. Below, I review a theory by Speelman and Kirsner that uses this conceptualization. See, in particular, Speelman & Kirsner (2005 pp. 215-227). ²³I remind the reader that by motivators I always mean internal motivators unless I qualify it with “external”.
Adult mental development
130
sufficient importance and urgency to the external motivator. As described in Chapter 14, this can be achieved through deliberate practice—e.g., repeatedly imagining scenarios, reviewing past events, and reasoning counterfactually about events and their importance. To respond to bids requires that one can detect them. The global motivational state indirectly referred to by “a tendency to respond to bids” must in fact be composed of a large number of monitors. The global state has manifold perceptual “tentacles” each specialized to its own patterns. For many behaviors correspond to bids. The deliberate practice that regulates insistence assignment (imagining, reviewing) also establishes these monitors. It is of course not normally enough to merely imagine and even reflect upon situations. One needs to engage with environments (simulations, role playing or reality) that implement key aspects of the real world. Those environments provide feedback and enable us to learn. Maharishi Yogi apparently used to say “Whatever we put our attention on will grow stronger in our life”. Experts focus on that which will move them forward. By focusing on objects, such as bids, we will learn their insides and outs. By reasoning about how our responses to a real previous bid may have had some consequence or other (good, bad or neutral), our minds learn to appreciate the value of the object. Some quantitatively-inclined folk believe that value (“just like anything else that is meaningful”) can, and in science ought, always be reduced to a number or vector. Such an assumption is not required or justified. There is no common currency of value in the mind. Value is established for the most part by discovering (postulating) relations between things. By reflecting upon the qualitative and quantitative effects of pursuing various courses of action one starts (unconsciously) to establish value. This leads to the establishment and configuring of the monitoring and insistence properties of motivators. Minds are very much concerned with processing motivational items. More generally, minds are motive processing machines. Ideally, we learn to better compute the specification, importance, urgency, rationale, insistence and intensity of motivators. Let’s consider the first four of these attributes of motivators. • The specifications²⁴ of motivators, meaning the representation of the states of affairs or conditions that they aim to affect, can develop. For example, as one masters Gottman’s theory one learns to distinguish fuzzy bids from clear bids. As one delves into the cognitive science literature on thinking biases, one might become sensitive to several different cognitive states and conditions.²⁵ • Here are a couple of examples of learning to assess the importance of particular types of external motivators. A lean startup expert might detect that a task is being done in big batches, as Ries described. This may trigger a motivational state. (More precisely, the perceptual state may also be a motivational one.)²⁶ This expert might have a sense of what the cost (side effects) of the batch process will be. In other words, he computes the motivator’s importance, a judgment which will change as more information is processed. After making a decision, ²⁴This includes the control aspects and the factual aspects of motivational states. See chapter 3 of Beaudoin (1994). ²⁵Keith Stanovich and his colleagues have organized these findings (Stanovich, 2011; Stanovich et al., 2011; Toplak et al., 2012) in a taxonomy of
thinking dispositions. One of their ultimate goals is to help people develop rational thinking dispositions. A person who masters this knowledge gem might have a rich collection of monitors to detect these various conditions and trigger motivational states to respond to them. ²⁶See chapter 6 of Beaudoin (1994). For a recent, simpler application of this concept, see chapter 6, Section C of Hurley et al. (2011).
131
Adult mental development
someone with rational thinking dispositions might realize that he ignored counter-arguments (“the opposite”). The importance of this motivator instance (“bad reasoning detected”) should be proportional to the importance of the decision. • One also learns to compute the urgency profile of external motivators. Time is of the essence in motivation, but not all motivators have the same temporal properties. • When one makes a decision, one often associates the rationale (or external motivators) to the decision, whether consciously or not.²⁷ One can become better at articulating and keeping track of this information. For example, an expert learns to keep track of rationales, to improve his decisions, to train people, and to justify his decisions to stake-holders and authorities. • One must ultimately learn to calibrate the intensity of motivators. Intensity is the strength of the disposition to act upon the motivator. It is separate from importance. Given mounting evidence that insulin resistance contributes significantly to cognitive decline and even dementia, one might assign negative importance to sweets, and yet still be strongly inclined to eat them. Clearly, meta-effectiveness is not merely a matter of comprehension, it’s being able to control these rich information-processing states of motivators.²⁸
Figure 6.1 summarizes these motivational aspects of mindware development.
Figure 6.1 Developing Motivators ²⁷The H-CogAff theory can illuminate cognitive dissonance phenomena. Motivators that are the rationale of many decisions may gain importance, intensity and insistence, even though, logically they should not. In other words commitment, and particularly social commitment, can influence many facets of motivators. ²⁸Chapters 3 and 4 of Beaudoin (1994) provide additional explanations of motivational states and processes.
Adult mental development
132
6.5 Developing long-term working memory The old adage “It’s not what you know, but who you know” doesn’t apply to expertise. Who you know is mostly relevant to the extent that they might help you know more.²⁹ Perhaps the most significant classically cognitive feature of a person who is developing expertise is building up one’s long-term memory including especially one’s long-term working memory.³⁰ What matters, for expert performance, is how much relevant information a person can access, how well he can encode situations with it, and how quickly he can retrieve information. (That involves the application of the meaningful encoding, retrieval structure and speed-up principles discussed in the previous chapter.) Working memory processes are key bottlenecks and enablers of fluid intelligence and hence problem solving. However, domain-specific long-term working memory enables experts to overcome the genetic limitations in their working memory (processes and capacity) and outperform others in their domain.³¹ It is therefore central to the aims of this book to understand how to develop long-term working memory from knowledge resources. This is particularly pertinent firstly because the role of memory and knowledge are often downplayed in discussions of education.³² There are those who believe that expertise is the result of superior intelligence and experience. Others believe it is a matter of being “highly skilled”. Secondly, even amongst those who know that knowledge is critical to expertise, not everyone appreciates the importance of long-term working memory and the activities and mental processes that develop it. While describing an app that instills long-term working memory, I am sometimes asked “Why bother to perfect one’s memory given that information is readily available on the Internet? Internauts can quickly find and interpret whatever information they need.” Some propose in this context that it will soon be possible to explore the web using “thought-controlled computing” (cf. InteraXon³³). This proposal can be read as an extreme form of increasingly popular “just-in-time (JIT) learning”. ²⁹This is evident in Thomas H. Davenport’s studies of high performance knowledge workers. These people work hard to maintain their knowledge. For this, “[T]hey rely most on their contacts in personal networks.” They tend to invest in mutually beneficial personal networks and share knowledge with others. “High-performing knowledge workers don’t just demand information from others, but also actively offer information and opportunities to those in their networks” (Davenport, 2005). See also Weststar (2009) on informal learning in IT workers. ³⁰Bereiter and Scardamalia have drawn our attention to the difference between fluid expertise and the cognitive states and processes that underlie the expert’s current superior performance, which they referred to as crystallized expertise. The concept of long-term working memory was proposed after they published this distinction. Today, one might say that long-term working memory underlies crystallized expertise. Thus, meta-effectiveness (effectance and fluid expertise) involves developing long-term working memory. The equally important distinction between fluid and crystallized rationality (Stanovich, 2011; Toplak et al., 2012) parallels and complements the distinction between fluid and crystallized expertise. ³¹There is also evidence and strong theoretical reason to suppose that a more general capacity to store information in and retrieve information from long-term memory is of importance to progressive problem solving and fluid intelligence (Mogle, Lovett, Stawski, & Sliwinski, 2008; Unsworth & Engle, 2007). That research and the work on long-term working memory are grounds for the emphasis placed in this book on long-term memory. Ackerman, Beier & Boyle (2005) have also argued that variance in working memory does not fully accountant for measures of fluid intelligence. See also Donald (2001). ³²Bereiter (2002a) discusses how the North American K-12 educational system downplays the mastery of knowledge. Many freshmen in knowledgeintense disciplines experience a shock at just how much factual knowledge they are expected to master. My emphasis on familiarity with and mastery of knowledge does not entail rote knowledge. Meaningful encoding is not rote. Retrieval practice, however, is necessary for expertise. The goal, as noted earlier is one of manifold understanding. Albert Einstein is an excellent example of the importance of deeply processing large amounts of helpful information. After graduating from high school, Einstein became a patent examiner. Wikipedia (2014) reports “Much of [Einstein’s] work at the patent office related to questions about transmission of electric signals and electrical-mechanical synchronization of time, two technical problems that show up conspicuously in the thought experiments that eventually led Einstein to his radical conclusions about the nature of light and the fundamental connection between space and time”. In order to assess a patent, one must extensively research previously filed patents. This leads one to develop an intricately interconnected mental model of the literature, problems, concepts and solutions. ³³http://interaxon.ca
Adult mental development
133
Alas, JIT learning proposals often confound being in possession of World 3 knowledge resources³⁴ and having mindware. Possessing a document does not guarantee that one will be able to rapidly interpret it. In many situations it is unlikely that one will have time to interpret the document. Familiarity with knowledge, while important, is often not enough³⁵. Knowledge workers must make many knowledge judgments on the fly, in meetings or while solving problems. A growing nexus of information needs to be readily accessible and usable as mindware.³⁶ Of course, domain experts are not expected to know everything. Some of their value comes from their ability to quickly locate (World 3) information. However, they are not just librarians. They can rapidly interpret situations with their knowledge. This sense-making ability relies on a huge, intricate, rapidly accessible web of interpretative mindware. To make sense of documents that they retrieve from the web (and thereby improve their understanding and mindware), they need to have built up long-term working memory. Expertise in domain reading is a prime example of long-term working memory. Experts master a huge technical vocabulary. The Oxford Dictionary of Accounting has over 3,600 entries³⁷. Experts can rapidly and meaningfully encode what they read, and the situations they face, in terms of elaborate retrieval structures involving the knowledge. They can quickly learn the meaning of a new term because they understand (or can quickly understand) the terms it refers to. They can utilize the concepts and terms in communication and in problem solving. While experts are capable of qualitatively different behavior from novices due to their long-term working memory, the transition of information from vanilla long-term memory state to a long-term working memory is a gradual one. This important concept can be illustrated with reference to Ralph who, being a regular reader and user of Pat McKeough’s investment information and related publications, has become very conversant with a small network of financial concepts. While he is not expert, some of this information is in long-term working memory or well on its way. For example, Ralph decided to purchase Pat’s U.S. “pick of the year”. He needed to decide in which of his four accounts to effect the transaction: His Tax Free Savings Account (TFSA), his Canadian dollar Registered Retirement Savings Plan (RRSP), his Canadian dollar cash account, or his American dollar cash account. He used some of his prior decisions to guide him. Being Canadian, Ralph had decided that he should hold his dividend-paying U.S. stocks outside his TFSA to avoid the IRS’s 15% withholding tax. In order to avoid foreign currency transactions, he had decided he should hold his U.S. stocks either inside his U.S. cash account or his RRSP. His RRSP is not a U.S. dollar RRSP, but it is now configured with TD’s auto wash. However, aggressive stocks, whether U.S. or Canadian, he holds outside his RRSP for better tax treatment of the gains and losses. Because Pat’s U.S. pick of the year happened to be an aggressive, non-dividend stock, Ralph decided it was best to purchase it in his U.S. cash account. If your Canadian financial knowledge is limited, odds are that the previous paragraph was difficult to read. Some terms are left undefined (e.g., “TD”, which is a bank.) Even if you “know the ³⁴However, it would be useful to have the functionality suggested by this person. This is now an obvious candidate to replace speech-to-text technology. Search results could be returned on the user’s glasses, given technology such as Google Glasses. ³⁵A more extensive assessment of JIT learning would relate it to the concept of learning in depth, (Egan, 2009). ³⁶Compare Pogue (2013) and my comments on it in Scientific American Online posted as lucb-cogzest. ³⁷I don’t know of a study that measures technical vocabulary in economics. However, the research groundwork is being laid for this (Kwary, 2011).
Adult mental development
134
meaning” of all the words, in the sense that you could define and use them to a certain extent, you might find the paragraph difficult to read. There are many hidden assumptions in this paragraph. For example, interpreting the phrase “for better tax treatment of the gains and losses” requires knowing the following: • • • •
Aggressive stocks are more volatile and hence may have greater capital gains and losses. Capital gains are taxed favorably compared to regular income (but not in RRSPs). Capital losses can offset capital gains (but not in RRSPs). Money withdrawn from an RRSP is treated as regular income.
Yet somehow, you need to use the text base to create a model of the situation that it describes. Given enough time, you would be able to acquire information to do this and hence understand Ralph’s decision. In contrast, someone like Ralph can read this paragraph quite fluently by relying on key information in long-term working memory. He might make an external note of the accounts involved (TFSA, US$, CDN$, RRSP[CDN$ a/w]). Each one of these accounts has known financial properties represented in LTWM that enable him to interpret and assess the text on the fly (e.g., to assess the previous “of course[…]” conjunct). Still, it is unlikely that information related to “wash trades” would be instilled in this person’s long-term working memory (or Ralph’s) because trade washing is a fairly obscure, derivative concept that brokerage firms don’t like to talk about and “auto washing” is a new service. A stock broker, in contrast, would have information for all key concepts in long-term working memory, including auto-washing, and not require jotting. She might read the paragraph more quickly—though she would not engage in “speed reading”—than he would yet have very good recall.³⁸ Each financial term would activate a complex retrieval structure of hers. For reasons explained in the previous chapter, she would be able to function better if interrupted than he would. Moreover, she would detect potential implicit problems and opportunities that are beyond Ralph (and this author). To develop long-term working memory in a knowledge-intensive domain, it is necessary to delve knowledge resources and extensively practice with knowledge. Delving, as described in Chapter 12, involves carefully processing knowledge resources (typically documents).³⁹ For example, effectant chess players read records of chess masters’ games. They try to predict the master’s next move. When their predictions are falsified, they try to determine the specific advantage of the master’s move. They refine their representations. Expert readers generally make predictions and frequently analyze their predictive failures in ³⁸Under some conditions, intermediates will remember more information than experts—“the intermediate effect” (Caillies & Denhière, 1999). See Ericsson, Patel & Kintsch (2000) for an account of this that is coherent with the concept of long-term working memory (p. 588). Experts will remember the gist given their meaningful encoding, retrieval cues and retrieval structures. I suspect that, in some of the studies, age partly accounts for the fact that the experts forgot details, given the correlation between age and expertise. Also, some studies claim to be about experts but are actually studying novices, as noted by Ericsson et al. (2000) and illustrated in Blair (2013) & Thompson, Blair Chen & Henrey (2013). ³⁹An excellent heuristic predictor of chess expertise is the number of chess books owned by the player (Charness, Tuffiash, Krampe, Reingold, & Vasyukova, 2005).
Adult mental development
135
order to improve their understanding, and implicitly their mindware⁴⁰. They detect surprising text and question themselves on it. For example, some readers will have been surprised by the expression “effectant” above. They may have paused or at least slowed down and thought to themselves “Why did he use that unusual expression?”, “What does it mean?” They might quickly assess its caliber and usefulness. If it appears useful, they might try to use it themselves in the future (partly to assess it). They might add it, or one of its related forms, to their productive practice database. They might relate it to several concepts, such as effectance, cognitive miserliness and crystallized expertise. The speed-up principle entails that the speed and ease of encoding into long-term memory and retrieval increase gradually with practice. Public performance experts (e.g., artistic performers, athletes, gamers, board game players, card players, etc.) engage in deliberate practice. White collar workers don’t tend to engage in as much deliberate practice. This is partly because of the greater range of problems and knowledge they must master. It is also because they have not been offered credible technology and workflows that would enable them to practice productively. The following chapter deals with practice in more detail, as do Chapter 13 and Chapter 14.
6.6 Developing representational machinery In Einstein’s thought there arose an intimate relationship between time values and the physical events themselves. Thus the role of time within the structure of physics was fundamentally altered. Max Wertheimer Some of the most striking changes in the mind as well as in its products (chiefly, objective knowledge) are representational.⁴¹ Mental development fundamentally involves developing new ways of representing problems, relations, situations and other kinds of information. More generally, mental development involves representing representations. Most representations are unconscious and non-verbal. They refer to internal information (within the mind⁴²) and are “based on” internal states and events. Representational changes are more or less global. They vary in the number of items that they connect. Some changes are discontinuous, others are continuous or nearly so. They may involve the addition, deletion and re-interpretation (or re-description) of representations. Accounting for mental development, viewed this way, is one of the most difficult challenges of ⁴⁰Hurley, Dennett & Adams (2011) made an interesting argument that humans are designed to find disconfirmations pleasurable (humorous) because this tendency will make them actively seek out contradictions. This in turn can improve one’s understanding. Active seeking of coherence and truth is a rational thinking disposition (Stanovich, 2011). So, fluid rationality makes one better able to appreciate humour, which runs counter to the popular belief that to be rational is to be stern. Accordingly, we can expect fluid experts to be more likely to find humor in their domain than novices or crystalized experts. One can extrapolate from chapter 5 of Fisher (2005) that fluid rationality was selected for in evolution not merely because it made survival more likely but because it made the host more likely to be chosen as a suitor by a potential mating partner. ⁴¹“Representation”, like “knowledge”, has two interpretations: mental (World 2’) and objective (World 3). I will use “objective” as a qualifier when necessary. I have already alluded to the wide contentious literature on representations. See Stich (1992) for some cautions on this term. One does not need to have a general theory of representation in order to use the concept. ⁴²Perhaps the simplest and most powerful way to teach the recursive concept of representation is to have students build simple formal neural networks of vision on hex graph paper. This is how Claude Lamontagne teaches the course on perception that earned him university, provincial and national teaching awards. See Beaudoin (1990) and Lamontagne (1987) for examples of this. Formal neural network theory was put forth by McCulloch & Pitts (1943). To get an expanded sense of the concept of representation, see Part 1 of Kintsch (1998) and Sloman (1975, 2011a). A key problem in dealing with the concept of representation is maintaining the distinction between World 3 representations (notations, or conceptual artifacts) and World 2’ representations, as I alluded to in the previous footnote.
Adult mental development
136
cognitive science. This is evident in the debates and controversies regarding representations in cognitive science. Max Wertheimer documents that the (objective) insights to which his friend and fellow chamber musician, Albert Einstein, arrived in formulating the theory of special relativity were the result of seven years of specifying and attempting to resolve problems of understanding. It could be argued that this intense journey was in significant respects a continuation of earlier developmental processes shared, for the most part, by ordinary mortals. To unpack this idea, which underlies much research in cognitive science, is beyond the scope of this book. Aaron Sloman’s work, once again, provides illuminating examples. He has been characterizing toddlers as “theorem provers” (Sloman, 2012b). He conjectures that “human mathematical abilities are a natural extension of abilities produced by biological evolution that are not yet properly understood, and have barely been noticed by psychologists and neuroscientists.” (Sloman, 2012a) Sloman’s project follows on, amongst others, key ideas developed by Alan Turing on “morphogenesis” and meta-computation (Turing, 1937) but not previously examined in this light. Sloman conjectures, in particular, that evolution “discovered” meta-computation long before Turing did (Sloman, 2012a). Meta-computation is computation about computation. Subsets of this are referred to by psychologists as “meta-cognition”. Sloman’s recently coined expression “meta-semantic competence” (Sloman, 2008a) is perhaps more appropriate as it draws attention to the key priority of knowledge about meaning-bearing information. If Sloman’s research programme succeeds, it will have shown amongst other things, an important continuity between the development of a child and the development of expert knowledge workers like Einstein. Thus the importance of understanding mental development. This approach generalizes Annette Karmiloff-Smith’s argument that to deeply understand cognitive phenomena one needs to consider cognitive development. This does not, as is sometimes supposed, lead to a nativist position according to which the mind unfolds in a fixed, genetically determined manner. The architecture of individual minds, and their underlying brains, are the result not only of their genetic code, but also of virtual machine processes and interactions with the environment. Karmiloff-Smith has argued that what appears to be the same competence in different individuals is often the result of different mental architectures. “While some macrostructures of the brain, like the overall six-layer structure of cortex, may be under general genetic constraints, most of the microcircuitry of the brain turns out to be the result of complex multilevel interactions over developmental time.” (Karmiloff-Smith, 2012) In Beyond Modularity, she proposed that the modularity of an adult brain, to the extent that it exists, is the result of an epigenetically labile process of modularization. Well-endowed brains, such as Mozart’s and Einstein’s, require exceptional training, practice and other experience to develop unique mental architecture.⁴³ This development is not just a matter of “neuroplasticity” (i.e., biological morphogenesis) but mental growth. In this view, frequent fine-grained psychological (i.e., virtual machine) demands on the brain lead to brain development.⁴⁴ My objective here is not to delve into neuroscience or nature-nurture debates. It is, instead, ⁴³Mozart was not merely favored by the roll of the genetic dice. He received world class instruction, and was pushed to the limit of his potential by his entourage and his own effectance (Gardner, 1998; Hayes, 2002a). ⁴⁴Many reductionists downplay downward causality. However, the existence of virtual machines, and computer network “stacks” such as TCP/IP, are veritable proof of the concept of downward causality. Moreover, virtual machines provide a model to make this type of causality (and several other interesting types) understandable (Sloman, 2002a; 2011b).
Adult mental development
137
to underscore the importance of “taking development seriously” as Karmiloff-Smith put it. This is partly why, in the meta-effectiveness framework, I often speak of “mental development” rather than merely “learning”.⁴⁵ While Karmiloff-Smith emphasized child development, understanding cognition requires considering development across the life span. People who engage in progressive problem solving, knowledge building and intense learning activities become qualitatively different from others. Their mental changes, over and beyond normal learning, are analogous to the kinds of development that children undergo. Developing new representations—whether in self-directed, taught (“self-regulated”) or happenstance fashion—is often mentally taxing and usually exploits previously acquired cognitive skills and representations. To be sure, it relies upon relatively normal learning, the type we see illustrated in learning curves (where performance on a type of task improves at a decelerating rate until it reaches a plateau). However, it also crucially depends on mental development that sometimes yields non-monotonic (sometimes “U” shaped) learning curves where performance on a task improves and then degrades before picking up again. I proceed to leverage the component theory of skill acquisition (Speelman & Kirsner, 2005) and the theory of “representational re-description” (Karmiloff-Smith, 1995). As far as I know, these complementary theoretical responses to modularity assumptions have not yet previously been combined. Yet doing so yields powerful results.
6.6.1 Growth of component processes I insist only that if we are interested in personality, we must go beyond the elementaristic realm and reach into the more morphogenic realm. Gordon Allport Speelman and Kirsner put forth a theory to explain five key principles of skill acquisition. Their theory emphasizes practice while characterizing some of what happens “under the hood” as people of any age master tasks. A key concept of theirs is that what looks like a single skill can often better be understood in terms of multiple component skills. Those skills, in turn, can be understood in terms of underlying mental resources. I will not delve into their mental resource theory. Suffice it to say that it is inspired by Marvin Minky’s Society of Mind. Like the H-CogAff theory, it supposes that the mind is comprised of a large number of fine-grained mechanisms. Their principles and explanations are relevant to the problem of mental development. Next I quote and describe their principles: • “Practice leads to faster performance.” In “normal learning” (i.e., outside of deliberate improvement as described in this book), component skills improve in a decelerating fashion while performance tends towards an asymptote. • “Practice leads to efficiencies in knowledge access.” (I would say practice leads to more efficient and effective access to long-term memory and other resources.) • “Practice leads to less demand on working memory.” (This is partly due to long-term working memory principles.) ⁴⁵Compare the discussion of learning vs. development in Boden (1988, pp. 187-224).
Adult mental development
138
• “As expertise increases, fewer mental resources are required to perform the particular learned task, enabling the development of a hierarchy of skills.” (I would replace “fewer mental resources” with fewer “large-scale resources”. Experts may actually use more resources in performing a task, but, as the authors posit, the mechanisms involved will be more finegrained and optimized⁴⁶.) • “Mastery in a domain involves the application of an array of component processes, with varying degrees of specificity to tasks and contexts, that are recruited in a manner that allows for consistent performance under stereotypical situations and flexible performance under unusual circumstances.” Mastering a skill is implemented by multiple “component processes” each of which, at any given time, is at a certain stage of development. Each process is responsive to certain inputs, though these inputs can vary in scope (specialization and generalization). The principles are factual in the sense that they seek to characterize real phenomena. They are also closely tied to empirical, measured phenomena. They are nonetheless couched in theoretical terminology (e.g., “working memory”, “mental resources”, and “component processes”.) Consider a child who is learning to read. First, it must be able to understand speech to a certain level, which Speelman and Kirsner refer to as a “fluency threshold”. Then, it learns to recognize letters. Meanwhile, its speech comprehension continues to improve. When it can recognize letters to a fluency threshold (a certain average response time), it is taught to recognize entire written words. Meanwhile, speech comprehension and letter recognition improve. When its ability to recognize words improves beyond a fluency threshold (another average response time), training moves on to text comprehension. Meanwhile the previously established “component skills” continue to improve. All the while, the child is developing a hierarchy of mental representations and mechanisms to interpret the external notations. In addition to Speelman and Kirsner’s component process theory, a theory of mind⁴⁷ proposed by Bernard J. Baars (Baars, 1988) helps to make sense of normal learning.⁴⁸ The theory posits the existence of a “global workspace”⁴⁹ that is engaged when tasks require coordination of multiple mental subsystems (e.g., vision, speech processing, motor execution and deliberative processes.) The global workspace is used for serial processing. As one masters a task, communication through this workspace during the task’s execution becomes increasingly efficient and coherent (Shanahan, 2010). One becomes progressively less aware of the details of the task. The foregoing two theories are manifoldly pertinent to the concerns of this book. Throughout this book, I emphasize practice and the mental mechanisms that develop with practice. My analyses share many of the assumptions of those theories, as qualified above. Like Karmiloff-Smith’s theory, presented below, the component theory of cognitive skill acquisition emphasizes mastery. The concept of mastery is key to theories of expertise. But Speelman, Kirsner and Karmiloff-Smith ⁴⁶Compare Speelman & Kirsner (2005, p. 218). ⁴⁷Baars and his colleagues developed this theory to explain conscious mentation and their concept of consciousness. Baars (1997) presents an
accessible, concise version of this theory. ⁴⁸There are several alternative information-processing explanations of normal learning. The argument in this chapter is not critically dependent on the particular account. The main point is to have a reference model to understand that behavioral competence is made possible through underlying mental development. ⁴⁹Blackboard systems are a predecessor of this concept in Artificial Intelligence (Hayes-Roth & Perrault, 1979; Beaudoin, 1994, Ch. 2).
Adult mental development
139
provide new insights into the role of mastery. The notion of fluency threshold is an important qualifier to the learning curve concept. On the one hand, it represents a line that some people treat as the upper limit of their developmental potential. On the other hand, once overcome it enables people to develop new component skills. However, the notion of “minimum performance time” needs to be generalized as it is not always the right metric to assess cognitive skills. The concept of “skills” itself is only limited a limited conceptual component of mastery (cf. Chapter 2). I attempt to overcome these limits in this book.
6.6.2 Taking child and adult development seriously I now think [Annette Karmiloff-Smith’s Beyond Modularity] should be essential reading for researchers in AI and Robotics as well as cognitive and developmental psychologists, educationalists, and philosophers interested in epistemology, philosophy of mathematics and nature-nurture issues. Aaron Sloman The meta-effectiveness framework is designed to help practitioners develop mentally by exploiting potent knowledge resources. Developing mental representations is an important component of this. Thus, in this section I present concepts and vocabulary for thinking about the growth of mental representations. I claim that mental representations vary and develop in manifold ways and orientations, beyond, for example, the following traditional binary distinctions: conscious vs. unconscious, implicit vs. explicit, verbal vs. pictorial and procedural vs. declarative. Moreover, effective representational change requires mastery (and hence work). Such change is also a requirement for developing further mastery. I claim that mental development is a two-way street. I was not merely paying lip service to Annette Karmiloff-Smith’s contention that in order to understand adult development one must pay heed to child development. Developmental considerations apply across the lifespan, well into the realm of expertise. To support my claim, I present research on child development. But first I present examples from everyday adult life. The informal and research examples are meant to help you question and adapt your understanding of mental representations and their development. 6.6.2.1 Some phenomena that highlight mental representations This section describes some phenomena, likely to be familiar to all my readers, that cannot properly be understood without reference to the concept of mental representation. In keeping with a long tradition in psychology (started by William James), the first two concern errors. The neglected sweater. One evening, as my partner and I were preparing to leave for an orchestral concert, I walked into my den and stared blankly at the clothes in my closet. I hesitated for a moment, pivoted emptied-handed, grabbed the hat off the coat hanger behind me and headed back down stairs. Then I suddenly remembered that I had intended to get a sweater out of the closet. Apparently, when I entered my den the first time, “I” had momentarily “forgotten” my intention. That is to say that in some part of my mind a “procedure” had been initiated with the intention (and thus inner motivator) of retrieving my sweater; but this information transiently became inaccessible to at least some of my management processes.
Adult mental development
140
There are several different ways in which this example—and the ones that follow—could be interpreted and modeled . In The Psychopathology of Everyday Life, Sigmund Freud devised elaborate explanations for a plethora of thought-provoking errors. He interpreted errors similar to the ones described above as being the result of hidden motives.⁵⁰ Did “I” mean to thwart my adorable partner by making her late to a concert that I, too, was looking forward to? As Aaron Sloman put it in conversation with me in 1991, a problem with Freud’s explanation is parsimony. It attributes too much intelligence to humans. It does not make sufficient place for real error.⁵¹ To Socrates, wisdom is knowing the limitations of one’s knowledge. For AI and biology students, it is knowing that there are always trade-offs in the design of sophisticated, autonomous systems, whether such design be the blind work of nature or the deliberate work of engineers. Intelligent systems, inevitably, will be prone to error, perturbance (tertiary emotions) and perhaps even mirth (Hurley et al., 2011; Sloman, 1987). Thus, the interesting factual question to ask about my sweater neglect is not, as Freud would have us ask, “Why did I forget?” Nor is it, “Who did the forgetting?”—the intentional stance, described above, gets us close enough: I forgot. But, “How did I ‘forget’”? Perhaps minds contain a processpurpose index⁵², as Sloman (1978) suggested. “The function of a process-purpose index is to store […] information about the reasons for various actions.” There may be various ways in which the information in that index can become out of date. Could it simply be that its entries have levels of activation that need to be maintained? How can process-purpose indexes be maintained? The best laid schemes of mice and men go oft astray. While authoring this book in Scrivener recently, I wanted to sort some paragraphs. So, I selected the paragraphs. In the menu bar, I clicked on the Edit menu, moved the cursor to the “Sort Paragraphs” menu item and waited for “myself” to find this menu item. Approximately a second later “I” realized the cursor was on the menu item I was looking for. I’ve had similar experiences with menu items in graphical user interfaces ever since I first started using computers in 1987. So, I can’t blame the errors on my age. There are many “sub-personal” explanations we could devise for this phenomenon. It doesn’t seem to be an error with my processmotivator index (should I have one). For I hadn’t lost track of my purpose. Had I selected the menu item ballistically without registering at any level the text “Sort Paragraphs”? Not likely given that I use monitors of different sizes. Beyond “once more from the top”. Quick. Think of two of your favorite songs or poems that are reasonably but not excessively complex. Write down the words of the first one from the top (no cheating). Then write down the words of the second from the second stanza (or the fifth line in). Now, hide your notes, and try writing the first one backwards (line by line). Odds are you found those memory tasks increasingly difficult. The information is there in your mind somewhere, but “you” cannot easily access it. With the right cue, such as the initial chord, you would get off to a good start on the first task. But starting from scratch in the middle of a song ⁵⁰See in particular, chapter 7, “Forgetting of impressions and resolutions”. For example, “I have collected the cases of neglect through forgetting which I have observed in myself, and endeavoured to explain them. I have found that they could invariably be traced to some interference of unknown and unadmitted motives - or, as may be said, they were due to a counter-will.” (Freud, 1914). ⁵¹For a more detailed account of how slips can arise as a side effect of complex action-selection mechanisms, see Cooper & Shallice (2000, 2006), Heckhausen & Beckmann (1990), Norman (1981) and Norman & Shallice (1986). Some of these models also parsimoniously account for errors that arise from brain damage. ⁵²In light of the H-CogAff architecture, presented in the previous chapter, today we would call this a “process-motivator index”.
Adult mental development
141
without text is difficult for most of us, even for well rehearsed songs. If you were cued with the line just before the middle, you would have a better shot at remembering the middle line. Working backwards is the toughest challenge. Choirs typically sing with the text before them, which makes it easier to execute the director’s instruction to start from a particular line.⁵³ Some expert singers, however, commit musical text to long-term working memory⁵⁴. They establish various performance cues⁵⁵ that enable them to quickly start from a random location in a piece without reference to text (Chaffin, Logan, & Begosh, 2008). Two prototypical serial memory access tasks at which educated adults excel involve the alphabet and number systems, each one of which have been studied extensively by cognitive psychologists. In an elegant paper, Klahr, Chase and Lovelace (1983) speculated that the alphabet is mentally represented as a list of six lists: [[A→B→C→D→E→F→G] → [H→I→J→K] → [L→M→N→O→P] → [Q→R→S→T] → [U→V]→ [W→X→Y→Z]]. Each sub-list is a “chunk” of information that can be held simultaneously active in short-term memory. If a subject is probed with a given letter, they can only directly access the chunk in which the letter representation is embedded. Once the chunk is activated, they work forward from its first letter, using a serial, selfterminating search algorithm. In the two experiments on access to the alphabet, they reported that “A model using only a single parameter for the time required to access the next element at either the chunk or letter level accounts for about 50% of the variance in RTs [response times] for our two experiments. A two-parameter model accounts for over 80% of the variance in previously published studies of covert and overt alphabet recitation.” (p. 462) They thus found evidence for hierarchical memory structures similar to the ones alluded to in the previous chapter on long-term working memory⁵⁶. Their explanation did not make use of all the concepts of long-term working memory, however—that theory had not been invented at the time of their study. The augmented explanation will not surprise you, because the alphabet and number systems are highly practiced, meaning that literate people are expert in these domains. So, people can more rapidly recall the letter following the first item in a segment from the alphabet song. For example, if you were asked to name the letter in the alphabet after “J” you would likely first access the entire “chunk” starting with “H”. You would tend to more rapidly access “J” given “I” than “K” given “J”, because “J” comes before “K” in the same sublist ([H→I→J→K]). However, you’d be even slower at accessing “L” given “K” because you’d need to activate an entire new chunk ([L→M→N→O→P]). Klahr et al. speculate that the structure of the Alphabet song is dictated by the structure of memory as opposed to the other way around. The theory of long-term working memory predicts that expert singers can rapidly commit songs to long-term working memory in a meaningful, hierarchical fashion; they serially access any part as an offset from a rapidly-accessed node in memory. This entails content-addressable memory. This is similar to the offset phenomenon ⁵³Choir singers have the added task of locating the initial note to sing. A choir’s accompanist may provide a cue chord, from which skilled members of each section (bass, tenor, alto, soprano) locate their starting note. Frequently, the accompanist will provide individual notes to the different sections. To work from internal cues is of course more demanding. Thanks to Mark Beaty of both La Candela and the Vancouver Island Symphony Orchestra for information about choral practice. ⁵⁴Lehmann et al. (1997) note that some conductors perform long operas without using a score. They did not report whether such conductors can randomly access parts of the score. ⁵⁵Chaffin (2011) and Chaffin et al. (2008) reported that expert musicians use various types of performance cues to facilitate recall. This is consistent with long-term working memory theory. ⁵⁶The alphabet song itself has been conjectured to reflect these properties.
Adult mental development
142
observed in professional singers and musicians (Chaffin, 2011). Theories that account for serial access performance, such as long-term working memory, cannot completely account for the other phenomena in this section (e.g., slips of the tongue). This points to the complexity of mental representations and mechanisms that process them. 6.6.2.2 Representational redescription (RR) This section describes a theory of mental development. While the research was proposed to account for findings of childhood mental development, I propose that its core ideas area also pertinent to adult mental development. Child development data are clearer cut and somewhat less variable, in that within a culture most children acquire similar capabilities, though not always through the same mental or neurological routes. Based on a series of studies, Karmiloff-Smith developed a theory of the development of finegrained mental representations. (Compare microcognition in Chapter 5). I have already alluded to her claim that the modularity of adult brains and minds is the result of labile modularization processes. She rejected the notion of her former colleague, Jean Piaget, that cognitive development proceeds in discrete global stages. She proposed instead that there are manifold “microdomains” of cognition, to which distinct, specific mental abilities pertain (e.g., counting.) A child may be in one phase for one microdomain, a more advanced phase for another, and a less advanced one for yet another. The various sub-skills modeled by Speelman and Kirsner (mentioned in the previous section) could be considered as micro-domains.⁵⁷ I will describe her theory abstractly and then provide several examples. This subtle theory is useful for understanding critical differences between humans and other animals, and between developing and stagnant minds. Karmiloff-Smith posited a developmental pattern in which a child (a) masters a behavior in a microdomain, then (b) becomes able to treat the procedural representations underlying this mastery as objects, then (c) becomes conscious⁵⁸ of the latter representations, and then finally (d) develops linguistic representations of this information. In each phase, a new representation of prior information is created. Phases (b) to (d) involve a process of describing the previously developed representations in a new way, a process Karmiloff-Smith calls “representational redescription” (RR). Her theory does not assume that children zip through these stages. In particular, she emphasizes that a period of behavioral mastery is a pre-requisite for redescription. These four levels of representation are depicted in Figure 6.2. ⁵⁷See chapter 2 of Karmiloff-Smith (1995) for an application of her theory to the development of language which can be compared with the accounts of language development in Speelman & Kirsner (2005). ⁵⁸The concept “conscious” is notoriously difficult to specify, and I use it sparingly. This adjectival term, however, is more useful than “consciousness”. Baar’s theory of cognition can be used to interpret the term conscious here. A representation is conscious if it can be used by disparate mental mechanisms sharing a global communication channel (Baars, 1988; Shanahan, 2010, 2012). The vast majority of introspection, or “self-monitoring”, is unconscious. The mind contains innumerable mechanisms that monitor, control and communicate with other mechanisms.
143
Adult mental development
Figure 6.2 Microdevelopment. (Beyond Modularity)
Microdevelopment thus involves rendering increasingly explicit, malleable and useful information that has previously been constructed. A key point is that the RR process occurs spontaneously as children develop competence, provided they have an adequate genetic makeup and opportunities. Thus children progress if and only if they evolve their representations. In order for adults to continue to benefit from this trajectory, they need to continue to challenge themselves⁵⁹. Experimentally teasing out these four levels of representation requires ingenuity—fortunately, Karmiloff-Smith has that in abundance, as illustrated by the following examples. Karmiloff-Smith notes that children as young as four learn to correctly produce in speech the adjective “the” as a definite article and the possessive adjective “my”. However, at that age they are not able to explain their choice of words. The information they use to select and comprehend these adjectives is only implicit. An older child maintains implicit representations; but, in contrast can explain why it chose a certain word. Karmiloff-Smith gives an example of a 10-year old child who was given two pens, an eraser, an earring and the child’s own watch. The experimenter hid the watch and then asked the child “What did I do?” This conversation followed: Child: You hid the watch. Exp: Why did you say “the watch”? Child: Well . . . “my watch” because it belongs to me, but I said, “you hid the watch” because there are no other watches there. If you’d put yours out, I would have had to say “you hid my watch”, because it could have been confusing, but this way it’s better for me to say “you hid the watch “ so someone doesn’t think yours was there too. This child has “meta-linguistic knowledge” of possessive adjectives, by which Karmiloff-Smith means the child has explicit, conscious and verbal access to information she used to generate the sentence.⁶⁰ In other words, this child has the full spectrum of representations from I (implicit) to ⁵⁹Bereiter & Scardamalia (1993) have described how effectant people improve themselves through progressive problem solving. Effectant people tend to work on difficult problems. They attempt to represent these problems in new ways (i.e., to “accommodate”), often at higher levels of abstraction. They develop new concepts and methods (new objective knowledge). ⁶⁰Karmiloff-Smith actually wrote “This is an eloquent example of how children can produce elaborate verbal statements once they have access to that part of their linguistic knowledge.” (p. 50). She uses “knowledge” here in the subjective sense, i.e., as mindware.
Adult mental development
144
E3 (explicit, level 3) pertaining to this utterance. However, when the child answered “You hid the watch”, she did not call upon the meta-information; instead, she simply used implicit, procedural representations. I information is compact and rapidly accessible information used for “on-line” verbal processing. The key distinction here is that the I information is in the mind but it is not information to [higher levels of] the mind; E (explicit) information is information to the mind. However, such data are not the only basis for Karmiloff-Smith’s four-format model. For the data also fit a more parsimonious two-format explanation that simply distinguishes between implicit information and explicit information, with all explicit information being potentially conscious and potentially verbal. Karmiloff-Smith studied the development of children’s representations of open- vs. closedclassed words in an attempt to ascertain whether she could find evidence for the intermediateforms of representation between I and E3. Linguists define two word classes: open and closed. Open-class words are morphologically malleable base terms. One can add morphemes to them (appending or prefixing). A morpheme is a meaningful word unit that cannot be further subdivided while preserving meaning. For example the suffix, “-ing” in English is a morpheme that conveys action meaning. To divide it further brings us into the realm of phonemes, which don’t carry (much) meaning.⁶¹ English verbs and nouns are open-class. “Love” begets “loving”, “loveable”, “unlovable”. Closed-class words, such as English prepositions (e.g., “at”, “to”), determiners (e.g., “a”, “the”), conjunctions (e.g., “or”, “and”), and pronouns (e.g., “he”, “she”), cannot be morphed. Whether a syntactic category is open- or closed- class varies across languages, so children must learn this (as opposed to relying on innate representations). Children aged three can segment the acoustic stream of language into discrete word boundaries. They are able to use open- and closed-class words appropriately, meaning that they must have some mental representations of this information. However, it is not until reaching the age of six (at the earliest) that children display explicit awareness of these categories. When asked whether an openclass word, such as “table”, is a word, young children will say yes. But when asked whether “the”, or another closed-class word, is a word, they will say no. Accordingly, when asked to count words in a sentence, they will not count closed class words. But again these data do not speak to the precise level of representation accessible to children, as they are consistent with a more parsimonious two-tier model (implicit/explicit). Karmiloff-Smith devised a study in which children aged three through seven underwent two tasks. In the “off-line” task, children were asked specifically whether certain terms (such as “table” and “the”) are words. In the partially “on-line” task, children were told a story. The narrator paused on open- or closed-class words. The child was asked to repeat “the last word”, “the last sentence” or “the last thing” they had just been told in the story. They were not told what was meant by “word”, “sentence” or “thing”. The offline task requires E2/E3 awareness. One needs to know what counts as a term out of context of processing a normal sentence. In contrast, the partially online task only requires E1 and E2 representations. In accordance with her predictions, Karmiloff-Smith found children aged three and four failed ⁶¹Traditionally, morphemes are seen as the the smallest meaningful unit of language. However, Sereno (2013) presented research demonstrating that smaller sound structures, including phonemes, are stochastically reliable cues to meaning that are exploited in the processing of speech (Sereno & Jongman, 1990). For example, in English, verbs tend to use front vowels whereas nouns tend to use back vowels (Sereno, 1994, cited by Sereno, 2013) listeners are sensitive to this distinction. Onomatopoeia and phonesthemes are more obvious examples of semantically predictive sound structures.
Adult mental development
145
both tasks. Children aged around five succeeded with the partially on-line task but failed the offline task. Children aged around seven succeeded at both tasks. This corroborated her hypothesis that younger children only have I representations of word-class (indicated by their ability to comprehend and use these classes of words appropriately in speech), whereas children around five years old have E1 representations, and children around seven have the full spectrum (I to E3). This illustrates Karmiloff-Smith’s theory that as children gain mastery with a domain, they spontaneously (and unconsciously) redescribe implicit information into an explicit format that is usable for certain cognitive purposes. Then they redescribe the information again in a manner that allows conscious access. Under appropriate conditions, which may include formal education, they rerepresent them further still in a public language. A key concept of her theory is that behavioral competence is required in order for the redescription process to happen. To create an explicit representation one needs an implicit one to begin with. To have conscious (E2) representations one needs E1 representations. And the linguistic representations (E3) need to be grounded in lower-level representations. Behavioral mastery is not, however, a qualitative requirement for progress; it is a matter of degree of competence (Karmiloff-Smith & Clark, 1993). This is in line with Speelman and Kirner’s concept of fluency threshold in cognitive skill acquisition. Karmiloff-Smith & Clark (1993) qualified this theory. They noted that the epithet, “spontaneous”, must not be taken literally to mean “causeless”. At the time at which she developed her thesis, child psychologists were overly-focused on the role of negative feedback (corrections). She means to emphasize that much representational change is the result of endogenous, “success-based” processes without negating effects of negative feedback. Furthermore, she and Clark view explicitness as a continuum “being tied to the wider exploitability of information [that] names a continuous property and not a simple dichotomy between implicit and explicit knowledge.” (p. 578.) They are referring to “connectionist” and dynamic models, which are outside of the scope of this book. In contrast, however, I believe we must not discard the notion of non-continuous representational change.⁶² There are indeed many discontinuities in representational development.⁶³ In addition, not all linguistic representations are created or connected alike. “In our view, something other than linguistic labeling underpins RR. In other words, a crucial distinction must be drawn between linguistic representations in their communicative function, in which linguistic labels may be relatively shallow in their cognitive underpinnings, and linguistic re-representations which lead to ever-increasing relations between different aspects of the cognitive system.” (Karmiloff-Smith & Clark, 1993, p. 576). Each one of these distinctions is relevant to the problems of mental (or mindware) development as will be seen below. One could apply the RR model to any area of expertise. For example, someone who enjoys and regularly attends classical music concerts but has no formal training in music would be chalk⁶²See chapter 8 of Karmiloff-Smith (1995) and Elman (1996). ⁶³Even within a single modality there are many layers of processing and representation. In the case of vision, for example, the detection of lines and
edges requires several layers of grouping. Unfortunately, these discontinuities (and multitudinous layers) are often lost in computer models, including some connectionist models, as an artifact of the mathematical tools used by scientists. Claude Lamontagne formalized the Gestalt notion of grouping (using a principle of adjacency). He and I applied it to the evolution of fine-grained perceptual processes from the retina upwards (Beaudoin, 1990; Lamontagne, 1987). In our theory, each layer of grouping is an implicit rerepresentation of a lower-layer. Strictly speaking, informational layering is discrete, not continuous, though it allows the system to construct fine-grained representations that are treated as if they were continuous. For a much higher-order use of the principle of adjacency, in the formation of dream representations, see Seligman & Yellen (1987).
Adult mental development
146
full of implicit musical representations. Should this person be able to play classical music by ear, without instruction in music theory, she would have an even larger number of I representations, now complemented by E1 and E2 representations. With training in music theory, she would gain a huge collection of E3 representations and the underlying representations would also multiply and be enriched. If you are trained in music theory, you might find it to be a particularly fruitful domain in relation to which to explore this theory⁶⁴. The RR theory represents an important schematization for the practical and theoretical purposes of this book. In the next chapter, for example, I will interpret the well-known benefits of selfexplanation in representational terms. Her theory provides concepts and terms we can use to think about internal representations. To be sure, it abstracts from important complexity. However, we can still use the theory to help us realize that, and express how, there is more to representational development than it accounts for. For it is not a complete theory of cognition, nor should we expect one any time soon. For example, it seems likely that in addition to re-description, mental mechanisms are capable of “peering into” representations that were previously opaque. That might in fact be a mechanism of representational description. Elsewhere in this chapter I describe other types of representational development that go beyond RR, including architectural development.⁶⁵
6.6.3 RR in reverse: The problem of instilling mindware The bottom-up processes of cognitive development described in the RR theory capture only a subset of the data on mental development. The major challenge this book addresses can be expressed as effecting a top-down process that takes objective knowledge—expressed in various notations and comprehended with “E3” representations—as input to generate and organize lowerlevel representations: that is RR in reverse. ⁶⁴Kentgen, Allen, Kose, & Fong (2011) briefly applies Karmiloff-Smith’s theory to learning to play the piano. ⁶⁵See Sloman (2011c) for other pointers to architectural development. Karmiloff-Smith & Clark, (1993) refer to very simple forms of architectural
development.
147
Adult mental development
Figure 6.3 The Mental Development Challenge
To understand the bi-directionality of cognitive development, it may help to consider, as plainly as possible, three fundamental concepts proposed by Jean Piaget: assimilation, accommodation and reflective abstraction. I will start with some epistemological caveats. Piaget proposed these concepts to explain development. As MacNamara (1976) argued, their explanatory value is limited. However, cognitive science, with AI as its core discipline, requires a wide array of types of scientific contributions. As Max Wertheimer argued, the better part of problem solving is to characterize the problem. In AI, this means describing the requirements of the system one is trying to understand (Sloman, 2005). By systematically refining requirements and building prototype designs and implementing them in software, and analyzing our results, cognitive scientists move forward. If you get the requirements drastically wrong your specific design will be irrelevant to the task. Piaget’s three key concepts are requirements—capabilities to be explained. To deny them is to ignore fundamental human capabilities. These three concepts are also important tools for knowledge workers themselves to understand their own, ongoing cognitive development. Assimilation and accommodation are two processes by which we adapt to the environment. When one is faced with an external object or event, including a knowledge resource, one may adapt to it partially by assimilating the external information and partly by accommodating it. Ultimately, of course, perception, unlike digestion, is always a matter of actively processing information as opposed to absorbing external material. However, this processing can involve more or less significant mental change. If we pigeon-hole a document’s ideas into the concepts we already use, we would say that we are merely assimilating it. That might be the right thing to do if our concepts are adequate. However, if our concepts are significantly different from the ones conveyed by the document, then we have missed something. For example, while reading about the technical concept of inner motivator described in the previous chapter, one might simply assimilate it to the folk psychological concept of motivator or need. This force-fitting would lose many of the nuances of the inner-motivator construct. Chapter 13 lists a number of questions that we need to be able to answer in order to
Adult mental development
148
master the concept of inner motivator. The assimilator would not be able to answer many of those questions, unless he has already mastered the H-CogAff theory. In contrast, if we are sensitive to the ways in which new information is different from what we already know, then we have a shot at accommodating to the information. Strictly speaking, accommodation is a psychological (World 2’) process. It is the psychological source of changes in understanding, where understanding is, as described at the beginning of this chapter, a relationship between oneself and a World 3 knowledge object. To accommodate involves realizing that the knowledge that we process is significantly different from the knowledge that we already understand. In order to accommodate, we need to apprehend the new objective knowledge and adjust internally to it. The concept of inner motivator, for example, refers to the concept of motive generator. It is also related to the concepts of insistence, importance and intensity. To accommodate to the concept of inner motivator we need to become able to use the concept in ways that reflect these objective relations. Thinking, however, is about World 3; it does not happen in World 3. So, to accommodate, something in ourselves must change such that we acquire the dispositions (and hence mechanisms) to operate with and on this knowledge. These are the kinds of changes referred to in Chapter 2, and more technically in this (second) part of the book. Whether or not we use the terms assimilation or accommodation explicitly, it is important to determine whether the knowledge resource we are processing contains material that is significantly new (i.e., potentially to be accommodated or not). If it does, then one needs to determine whether it is worth accommodating to the information. Chapter 11 describes four ways of assessing knowledge resources such that we can rationally make this decision: the objective caliber of the knowledge resource, its usefulness, its potency and its appeal. Potency is a measure of the accommodation required to understand the information. Productive readers are able, at least tacitly, to apply these criteria. Piaget’s concept of reflective abstraction (in my opinion)⁶⁶, as it applies to adults, involves generating new internal representations and building new knowledge as we try to solve problems of understanding. This is not best described as a “spontaneous” event or change, though its results may occur spontaneously. We do not have direct knowledge of, let alone direct control over, how our mind generates the internal structures that enable knowledge to be built. Reflective abstraction requires that we deliberately set ourselves the task of reading, learning and solving problems of understanding. Creating new internal representations, building new objective knowledge, even making discoveries, does not necessarily lead to permanent mental change. A proof is that we can learn a lot by re-reading papers, emails, books and course materials one authored years ago—and sometimes more recently! People use dictation and other note-taking devices to capture solutions to problems before they forget them. As I mentioned above, I try to reserve the word “understanding” for conditions that endure a considerable period of time. In that sense, to produce an idea that one comprehends is not necessarily to understand it. In contrast, a child who learns its native grammar will be (nearly) ⁶⁶My primary goal isn’t to characterize Piaget but to help resolve the problems this book is concerned with. As MacNamara shows, the concept of reflective abstraction is particularly underspecified. In fairness, Piaget admitted as much: “However, a notion such as reflective abstraction would not have value unless one substituted a detailed model for such vague formulations. We are not there yet.” (Piaget, 1977, p. 113, quoted in translation mine.) In this context, I am trying to describe human capabilities so that we can enhance them.
Adult mental development
149
permanently changed. There is a saying that “people never forget to ride a bicycle”.⁶⁷ Thus, whether we have created fresh new knowledge or are trying to accommodate to someone else’s conceptual artifact, we face the challenge of doing RR, reflective abstraction, and all manner of mental development in reverse. As I alluded to above and we shall see below, knowledge is potent to the extent that mastering it would require significant mental development. To master knowledge is to get to the point that we can use it to assimilate new situations, problems and information. Thus, the process of mastering knowledge is one of “depotentiating” it. This in turn requires that we develop mentally. English and French have inherited from Latin the verb “to instill”. In both languages it is occasionally used, metaphorically, to mean to gradually acquire knowledge or a sentiment. We must beware of the tacit image this term may conjure of a source bucket pouring knowledge into a target head. Instead we should imagine the gradual development of information-processing mechanisms. To instill mindware, in the meta-effectiveness framework, means to engage in activities that produce the mindware that enables us to master the knowledge in question. Following Bereiter, I agree that this involves progressive problem solving and knowledge building. However, those activities are often not sufficient to create top-down enduring changes in monitors, inner motivators, and mental representations, particularly given the problems described in Chapter 3. I will argue in the remainder of this book that it is often helpful and sometimes necessary to engage in productive practice, which is a form of deliberate practice. Let us therefore turn our attention to the concept of deliberate practice. ⁶⁷For an extensive analysis of the longevity of subjective knowledge, see Ackerman (2008).
7. Deliberate practice: A source of effectiveness Memory is fundamental to all cognition. We saw in chapter 6 that as people develop expertise, they develop long-term working memory. That is, within their domain of expertise, they learn to encode experience in a structured and meaningful fashion that facilitates later retrieval. (You will remember the meaningful encoding, retrieval structure and speed-up principles.) The chess master, for instance, rapidly perceives a chess board in arrangements of chunks known as templates. You view this page not as a collection of squiggles but as characters, words, phrases, sentences, and paragraphs from which, if I have done my job right, you construct meaning. If you had to refer to a dictionary for every second word, you would not stand a chance of understanding this book. Every domain of expertise is like this: it heavily depends on memory. Thus, to improve cognitive performance one stands to gain from finding better ways to encode, store and retrieve information. One would expect effective teachers and trainers, also, to emphasize memory processes. There is no domain of expertise to which this analysis does not apply. Yet, many North American educators view the construction of memory in a bad light. According to some, the ultimate job of K-12 teachers is to teach thinking skills, creativity or self-expression. Memorization is often viewed as “rote learning”—parroting information without comprehending it. Yet in developing expertise one must master vast amounts of knowledge. Hence many students experience the shock of a lifetime in their first university semester: Undergraduate courses often require memorizing volumes of information! Many consider this to be an aberration, an expression of the bucket theory of knowledge. Some think this practice is useful because it teaches students how to learn. There are those, however, who (rightly) believe that becoming conversant with vast amounts of knowledge is critical to education. Thinking of memory in information-processing terms, they tend to emphasize encoding over retrieval. After all, in order to be able to remember something, one first must represent and comprehend it. What’s the point of remembering a formula, such as F=ma, for instance, if one doesn’t comprehend it? So, students are taught to use concept maps and various other encoding and elaborative strategies. Textbooks are increasingly made learner friendly (or so publishers claim). They contain plenty of schema activators, diagrams, sidebars, highlights, bold font and examples. The problem is that many people underestimate the importance of a critical fact about memory. Retrieving, or inferring, information from memory is a potent, and usually necessary, mechanism for being able to retrieve and use it again in the future.¹ More generally, in order to master knowledge in ¹Authors contributing to the literature on test-enhanced learning and the so called “test-effect” routinely speak in terms of information “retrieval” and “recall”. This ought not to be taken to mean that using information is merely a matter of retrieving it explicitly. Often, an exact copy of information need not even be stored in the first place. One can store a more general procedure for producing, or inferring, a new instance of the information. For example, while looking for your keys, you might infer that they are in your rain coat, because (a) they are not in their usual place and (b) it was raining yesterday. Compare my reference to dynamic lists in the Poplog Pop-11 AI programming language above. In natural learning situations it is often impossible to distinguish between an inference and simple recall of information.
Deliberate practice: A source of effectiveness
151
a domain one must productively practice using it. Publishers capitalize on this fact by selling study guides separately from textbooks. Practice figures prominently, though disjointedly, in several bodies of literature. We began to discuss some of the cognitive effects of practice in the previous chapter. The current chapter reviews additional literature on the subject with a view to designing forms of practice that are suitable to the specific needs of effectant, knowledge-driven adults.
7.1 Practice enhances factual learning and memory Jack and Chrissy, the former colleagues whose fortunes dramatically diverged after the acquisition of NewCo, were lovers and classmates at university. As students, they both used an effective and simple set of learning strategies, test-enhanced learning. While reading a textbook, and transcribing their lectures notes, they would each make a note of important information they felt they needed to master, i.e., “knowledge gems”. For each knowledge gem they would create a list of practice questions.² Then several times during the semester, they would practice answering these questions— together and apart. They knew better than to cram. They would space the questions out over a period of time. With this technique, they could not only predict their grades but also determine them. Jack was intrinsically interested in knowledge, but not the particular courses he took with Chrissy. Chrissy, in contrast, was extrinsically motivated to get good grades. As a result, Chrissy wound up with slightly better grades than Jack did. But, being an effectant person possessing a potent learning tool, Jack also routinely used test-enhanced learning as a way to master information for his hobbies and future profession. After graduating, Chrissy ditched test-enhanced learning. Being extrinsically motivated, she didn’t see the need to exert herself with practice. And she had no decent software to facilitate her practice. When she did feel the need to master a document, she reverted to the easier (and popular) strategy of re-reading passages she’d marked as knowledge gems. Jack, in contrast, knew that if he wanted to retain and utilize information, he had better practice using it. So he found various ways to practice. He hangs out with people who enjoy talking about his work and hobbies (so called “communities of practice”). He participates in online discussions. He seeks new ways to extend and use his knowledge. While using his knowledge he is effectively testing himself. When ExtantCo acquired NewCo, Jack realized that he had a decision to make. Sell or hold? That was a particularly big decision. Naturally, he turned to his source of investment knowledge, the newsletters of Pat McKeough. He asked himself, “What would Pat do?” He didn’t need to re-read Pat’s letters to figure it out. He had already applied Pat’s ideas. As you will recall from chapter 1, Jack sold his shares for a fortune. Chrissy, in contrast, was not very steeped in the principles of investing conveyed by Pat. She would simply use his specific recommendations to purchase and sell stocks on his buy/sell lists. But when it came time to make decisions about stocks outside Pat’s lists, she was vulnerable because ²Some students put such questions on paper “flash cards”, i.e., a card on which the question figures on one side, and the answer (and/or a reference to the answer) figures on the other.
Deliberate practice: A source of effectiveness
152
the knowledge hadn’t really sunk in. As you will recall from chapter 1, Chrissy lost her shirt on ExtantCo. It is tempting to think that with the ubiquity of information technology, one doesn’t need to learn deeply, all one has to do is know how to search. However, in order to use information one must understand it. That requires applying it, whether in deliberate practice or for real.
7.1.1 Practicing slows forgetting Most people intuitively know that recalling information can slow forgetting. For example, if you read a sentence or a paragraph and are asked a question about it a few minutes later, you are much more likely to remember that information the next day, and possibly for several days. Here are some questions that you could use to help you remember key ideas presented in the previous chapter. • • • •
What is the difference between objective knowledge and mindware? Does the concept of mindware matter to understanding yourself and learning. Why (not)? How does understanding differ from mental development? Why is developing long-term working memory relevant to expertise?
To the extent that you utilize what you remember about the previous chapter in answering these questions, you are likely to strengthen your memory of that information. More precisely, if you are asked a similar question at a later date, you are more likely to be able to answer this question than if you hadn’t attempted it earlier. Successful testing (normally) has a beneficial effect on recall of tested material. This is known as “the testing effect” or test-enhanced learning. This includes any attempt that one makes to recall information from one’s own memory. Productive practice, discussed below, is a productive form of deliberate practice that exploits test-enhanced learning. Spitzer (1939) presents one of the first, and largest, compelling studies on test-enhanced learning. Each of his 3,600 student subjects studied a 600-word text and were tested on it two or three times, at various intervals. (The intervals were: less than a day, 1 day, 7 days, 14 days, 28 days and 63 days.³) Spitzer found several stunning testing effects. If the first test was done early enough, then results on a second test were much better. For example, the average score of students who took the first test within a day of reading the text and the second test 21 days later was c. 11.5 out of 19 on the second. In contrast, students who took their first test on day 21 scored, on average, c. 6.3 out of 19 on the second. The former retained almost twice as much information than the latter! A similar difference was found on day 63. Moreover, the rates of forgetting of students who had an early test were much lower. Testing effects are some of the most well established findings in cognitive psychology. There are, however, multiple factors and processes that could account for them. Did students all use the same strategies to interpret, store and retrieve the information? Which strategies work best? Cognitive psychologists have worked diligently to isolate different factors. However, as shown by Delaney et al. (2010), their work is not complete. Still, there is enough data to provisionally adopt explanations and make recommendations. ³Students were not given feedback on their tests and they were not given an opportunity to review the text. Subsequent studies have compared testing with reviewing, and investigated the role of feedback.
Deliberate practice: A source of effectiveness
153
7.1.2 Practicing trumps reviewing Spitzer didn’t compare taking tests with other learning activities, such as reviewing the material, transcribing it, experiencing it in a different modality, and so on. Most people assume that learning happens mainly as one is experiencing external information (e.g., while reading, reviewing or listening to it). Thus, reviewing material is the most common technique used by students. Re-reading material is a lot easier and sometimes faster than answering questions about it. Strictly speaking, encoding material is normally necessary to be able to retrieve it later.⁴ But is reviewing better, equal or worse than practicing? That is a critical question that many cognitive psychologists, including Jeffrey Karpicke and Henry Roediger III, have addressed recently. Karpicke & Roediger (2008) had English-speaking college students learn a list of 40 Swahili words. All students went through an initial treatment in which (a) they read the pairs of words (e.g., hariri = silk) and (b) were then tested on the same list of words. Review and tests were conducted on a computer screen. In review trials, students first read the Swahili word and then read the corresponding English word. In the test (i.e., practice) trial, students viewed the Swahili word and tried to remember the corresponding English word (but were not given feedback.) Each student then went through one of four learning treatments. Each treatment consisted of 8 cycles of alternating study and test trials. In the first cycle of each treatment, students reviewed each of the 40 pairs, performed a distractor task, and then practiced each pair. In the subsequent 7 cycles, the number of review and practice trials varied as follows. • In the Review Lazily (R.L) and Practice Lazily (P.L) condition, if the student remembered the target word correctly, it would be removed from the list of pairs to review and the list of word pairs to practice. • In the Review Zealously (R.Z) and Practice Lazily (P.L) condition, if the student remembered the target word correctly, it was removed only from the list of word pairs to practice, not from the list of words to review. • In the Review Lazily (R.L) and Practice Zealously (P.Z) condition, if the student remembered the target word correctly, the pair was removed from the list of word pairs to review; it was nevertheless kept in the list of words to practice. • In the Review Zealously (R.Z) and Practice Zealously (P.Z) condition, students reviewed and practiced all word pairs, regardless of whether they correctly recalled the target word. Each student was then asked to assess how well he or she would do on the same test one week later. A week later, students returned to the lab for a final test. The performance of students across the four conditions did not differ on the learning day. Regardless of the treatment condition, they all learned all 40 words during the learning phase, and they did so at the same rate (their learning curves were indistinguishable). Before reading further, please take a moment to predict the relative performance of each of the four groups on the final test. In particular, would reviewing zestfully but practicing lazily lead to ⁴As mentioned earlier, often you need not be exposed to specific information in order to remember it, you can infer it from more general knowledge. If you have a perfect work attendance record, then if you were asked where you were on the first Monday of October 5 years ago, you may “recall” that you were at the office. That recollection is an inference.
Deliberate practice: A source of effectiveness
154
better results than reviewing lazily and practicing zealously? Does reviewing zealously confer an advantage at all?
Figure 7.1 Percentage of terms correctly recalled on the final test 1 week after learning. Differences greater than 4% are statistically significant. Repeated practice improved long-term recall by over 150%
According to common sense, students in the Review Zealously & Practice Lazily condition should perform better than the students in the Review Lazily & Practice Lazily conditions. That is, reviewing frequently should be better than reviewing infrequently. Yet Karpicke & Roediger found that reviewing zealously (i.e., viewing the material an additional 80 times) made no difference to final test performance. Students who practiced zealously, whether they reviewed zealously or not, answered 80% of the Swahili-English translation questions correctly. Students who practiced lazily (whether they reviewed zealously or not) remembered only around 35% of the words! Repeated study, once students could recall the material, had no effect on the delayed recall task. One way to interpret these results is to conclude that reading a document is good practice for reading the document again in the future. Practicing recalling information is good practice for remembering the information in the future. Re-reading a document is, of course, often a way to refresh one’s memory if one will be tested on it before the information fades again. Unfortunately, the exams of life often come without warning. Given that we often need to remember information in order to use it, this suggests, more generally, that recall practice is more effective than reviewing for becoming skilled at using information. Several studies have supported the same conclusion: if you want to remember something, it is better to practice remembering it than to review it a similar number of times. This is necessarily a simplification, but it is a useful one. While (as will be shown below) people usually do not anticipate the results of the Karpicke and Roediger study, the findings should not be surprising. Consider, for example, a song that you have frequently heard. Suppose it’s the Beatles’ A Hard Day’s Night. Take a moment to write down the lyrics to this song. Many people can’t complete this type of exercise, despite having heard the song many times. (As we saw in chapter 6, it’s much easier if you are cued with a word or phrase where you stumble.) If, however, you had sung this song several times unaided, you would have a much better chance of remembering it. This is not to say that practice guarantees recall. Paul McCartney, for example, stumbled on A Hard Day’s Night while a Beatle.⁵ He admits to forgetting lyrics (Gardner, 2010). ⁵The slip happens c. 1:50 in Lennon & McCartney (1961). See also: Gardner (2010).
Deliberate practice: A source of effectiveness
155
7.1.3 Why practicing works: Explanations of test-enhanced learning Given that distributed recall testing is one of the most potent ways to enhance learning, it is important to understand how and why it works. This may help end users select methods of learning that tap into the powerful mechanisms underlying testing effects. The effects of testing memory on learning are manifold and more or less indirect.⁶ This section deals with the more direct effects. Below, I introduce productive practice, which combines distributed recall practice and other forms of practice. There, and in Part III, I will describe important indirect effects of productive practice. I propose the heuristic relevance-signalling hypothesis as an explanation of the effects of distributed recall practice. This is an extension and adaptation of Anderson’s (1990) idea that memory is rational and adaptive.⁷ This is a “why” as opposed to a “how” (mechanistic) explanation. To understand this hypothesis, we need to do a bit of reverse-engineering of the problems (the requirements) that memory faces. That is, we need to adopt a designer stance⁸. Marcus (2012) noted: “Computer memory is much better than human memory because early computer scientists discovered a trick that evolution never did: organizing information by assigning every memory to a master map in which each bit of information to be stored is assigned a uniquely identifiable location in the computer’s memory vaults.” (p. 39) In contrast, given an arbitrary piece of information, such as the name of a historical figure, one cannot simply tell oneself, “I shall remember this information”. Even applying a mnemonic is often not very effective unless one also practices with it (Fritz et al., 2007). I don’t think the trick Marcus alluded to would have been selected for if mutants had produced it, given humanoid adaptive requirements. The problem has to do with the fact that the human mind processes far more information than it ultimately needs or can use. One of the mind’s main tasks is to figure out what information might be pertinent in the future and how it may be used in the future. One of the implicit tricks that evolved in humans is not to make the accessibility of information an all-or-none matter, but a matter of degree. The mind must prioritize some stored information items over others. It makes high-priority items easier to use, in given contexts. Evolution implicitly faced a major challenge: how can the mind it builds determine what information ought to be made easiest to remember? One solution would be to offload this relevance function to high-level management processes (distributed across cortices). But that wouldn’t work. High-level (cortical) decisions can too easily go awry: one might tend to try to “commit to memory” information that is irrelevant to Darwinian considerations. Conversely, such high-level control over memory would not be sufficient, it would ignore important items to remember. Further, and perhaps most importantly, it would be a poor use of high-level mental powers to engage them in deciding what to remember. Thus, the mind requires simple heuristics to quickly and repeatedly determine what information ought to have mnemonic priority. Intelligent organisms with a simple mnemonic priority heuristic ⁶Roediger & Karpicke (2006) distinguish between direct and indirect (or mediated) effects of recall testing. I prefer to think in terms of degrees of indirection. Few mental “effects”, if any, can aptly be described as being direct. For example, heuristic relevance signalling, described below, is a relatively direct effect of testing memory. An indirect effect is to provide feedback on one’s level of comprehension and memory. This can help one tailor learning activities to repair one’s knowledge gaps. ⁷See especially chapters 1 and 2 of Anderson (1990). ⁸http://www.cs.bham.ac.uk/research/projects/cogaff/misc/design-based-approach.html
Deliberate practice: A source of effectiveness
156
had an adaptive edge. The evolutionary trick is to prioritize information that higher-level (management) processes attempt to retrieve from memory. The more frequently a piece of information is requested, the more likely it is to be of importance. Therefore, mental indexing mechanisms must make that information more accessible. Conversely, information that is never requested is less likely to be relevant. Because attentional capacity (working memory) is limited, very little information is actually requested from memory compared to the information that is taken in perceptually. Attempts to retrieve information are much more reliable signals of relevance (or priority) than desires or instructions to be able to retrieve it in the future. If one only practices recalling material over a few days, the unconscious indexing mechanisms conclude that the material is not of lasting pertinence. These indexing mechanisms seem to be quite active during sleep. Multiple attempts to recall information over time (i.e., distributed recall, or spacing of recall) are a strong indicator to memory-indexing mechanisms to increase the future accessibility of the information. ASIDE Prioritizing memory is not the only aspect of cognition over which management processes have limited control. I described the heuristic relevance-signalling hypothesis in a paper that presented a new technique for cognitively facilitating sleep onset (Beaudoin, 2013). There, I noted that the brain faces a similar challenge with regard to sleep onset, i.e., to heuristically determine, based on indirect higher-level (cortical) activity, when it is time to enter the first stage of sleep. I proffered the somnolent mentation hypothesis which is to sleep-onset signalling what the heuristic relevancesignalling hypothesis is to memory: the human brain does not directly respond to the command to fall asleep.[…] lower levels of the brain critically must be able to trigger sleep on the basis of cortical states. Otherwise, the agent might experience something akin to an episode of [narcolepsy.] According to the incoherent mentation hypothesis, prolonged incoherent mentation is, intrinsically, such a trigger. (Beaudoin, 2013, pp. 13-14). Perhaps the most striking example of lack of control of mental processes is evident in tertiary emotions, described in chapter 5. Witness the phenomenology of romantic love and grief. Following Darwin, it is normally assumed that emotions serve a purpose. However, tertiary emotions (perturbances) are not in themselves adaptive or maladaptive, they are simply byproducts of an adaptive mental architecture. As Sloman (1993b) put it: “I have tried to show elsewhere (Sloman & Croucher 1981, Sloman 1987, Beaudoin and Sloman 1993) that certain kinds of resource-limited systems can get into states that have properties closely related to familiar aspects of certain emotional states, namely those in which there is a partial loss of control of our own thought processes. Such capabilities would not be the product of specific mechanisms for producing those states, but would be emergent properties of sophisticated resource-limited control systems, just as saltiness emerges when chlorine and sodium combine, and ‘thrashing’ can emerge in an overloaded computer operating system. Our vocabulary for describing such emergent global states will improve with increased understanding of the underlying mechanisms.” Another striking example of limitation is working memory capacity (Section 4.3 of Beaudoin, 1994; Beaudoin, 2011c).
Having proposed that natural selection favored organisms that display the testing effect, the
Deliberate practice: A source of effectiveness
157
question remains: how is test-enhanced learning implemented? A popular explanation of test-enhanced learning is couched in terms of “transfer-appropriate processing”. Simply put, in order to be able to utilize knowledge in the future that one is learning today, one should engage in learning activities that are similar to one’s future performance requirements. If you read an article and want to be able to utilize it in the future without having to refer to it, then you need to practice recalling and using its core ideas without reference to the paper. That is, you need to engage in distributed recall practice. Re-reading and recognition tests (e.g., many multiple choice and true-false questions) are so different from the future requirements of applying knowledge that they do not adequately support future offline application. Of course, a pianist who practices playing a song without reference to sheet music will be able to perform that song more independently than one who practiced with the aid of sheet music. While the transferappropriate processing explanation is useful for guiding practice, it does not, however, provide deep insights into underlying memory mechanisms.⁹ Another popular explanation of testing effects is the “desirable difficulties” hypothesis. The idea here is that effortful learning is more beneficial than easy (lazy) learning, provided that the extra labour is desirable. More specifically in this context, difficult but successful retrieval potentiates memory more than easier successful retrievals. Benjamin, Bjork, & Schwartz (1998) corroborated earlier demonstrations that attempts to retrieve recently presented information are not as potent as attempts to retrieve less recently presented information. More generally, as noted earlier, for optimal potentiation of memory, recall attempts should be sufficiently but not excessively delayed (the lag effect.) As I mentioned in chapter 5, cued recall is easier than free recall. The more cues one provides on a memory test, the easier the retrieval attempt is. Thus, other things being equal, practicing with fewer cues (free recall) is a desirable difficulty. Appealing to difficulty of retrieval may have practical benefits (e.g., in the design of productive practice). However, it is no better an explanation of the data than are appeals to transfer appropriate processing.¹⁰ The question is when and why (some) subjective difficulty potentiates memory? Roediger & Karpicke (2006, p. 198) and others proposed that effortful retrieval is useful when it promotes “deep processing” and the construction of multiple “retrieval routes” (both of which occur at recall).¹¹ This suggestion is not logically dependent on the concept of desirable difficulties. It also has the virtue of potentially being mechanistic, provided one can flesh-out the concept of route and mechanisms of route formation. Shana Carpenter proposed a more mechanistic explanation of testing effects known as the elaborate retrieval hypothesis. She assumes that memory is a network of interconnected nodes (Carpenter & DeLosh, 2006; Carpenter, 2009). Activation spreads from one node to nodes it is connected to. The activation spreads less as a function of distance from the original node. Carpenter ⁹Delaney et al. (2010, pp. 119-121) provide a cogent critique of transfer-appropriate processing explanations. See also Carpenter & DeLosh (2006). ¹⁰For example, ““difficulty” is an unsatisfactory concept because it involves and possibly confuses different levels of analysis: phenomenological
feelings of difficulty, the mental effort required to perform the task satisfactorily, and the resulting performance level” (Craik & McDowd, 1987, p. 478). Furthermore, like the transfer appropriate hypothesis, the desirable difficulties hypothesis does not easily lend itself to mechanistic interpretation. Pyc & Rawson (2009), however, show that the ACT-R architecture (Anderson & Lebiere, 1998) can provide some mechanistic underpinning to the desirable difficulties hypothesis. ¹¹One can interpret this statement in terms of the context passively providing retrieval cues (through some form of associationist conditioning) and the agent activating different internal paths. Delaney et al. (2010) argue against retrieval route interpretations of spacing effects. However, they favorably describe the elaborative retrieval hypothesis (also considered by Delaney) which assumes passive spreading of activation. The mediator shift hypothesis, considered below, is of the more active variety of explanations.
Deliberate practice: A source of effectiveness
158
argues that more difficult tasks (e.g., recall tests vs. recognition tests vs. re-reading) lead subjects to elaboratively encode information. This process causes connections to be formed between memory nodes. Suppose, for example, that you are presented with vocabulary words and definitions. Your task is to learn the definition of each word. “Propitiate” figures in this list, which is defined as “to win or regain the favor of someone by doing something that pleases them”. Later, you are asked to produce the definition of propitiate. Suppose the definition does not immediately come to your mind. You would then do some mental work to search your memory for the target. You might notice that the cue (“propitiate”) is a cognate of “propitious”. And propitiate sounds like “ingrate”, which reminds you of “ingratiate”, which in turn is nearly a synonym of ingratiate. These two internal cues might activate the full definition. You could then create a mnemonic, picturing a person finding a propitious moment to catapult gifts to ingratiate himself to an ungrateful giant (an ingrate). Through this process, propitiate in your mind becomes connected with several nodes such as propitious, ingrate, ingratiate, giant and catapult. These nodes themselves become interconnected. (For example, propitiate → person → catapult → giant → ingrate → ingratiate; propitiate → propitious → ingratiate; propitious → please → giant → ingrate.) To remember the meaning of propitiate, you now have several retrieval routes. These laboriously constructed routes pay off on future tests, because through a passive process of spreading activation, the cue can then indirectly activate the target. More generally, Carpenter’s research illustrates that testing memory is more than just a test. It is an occasion to actively enhance the connections between items in memory, i.e., to learn. This can be taken further. Learning to recall information (be it a fact, a concept, a procedure, or any other type of information), is a cognitive skill. By this I do not merely mean that memorizing is a skill. I mean more generally that when one learns to remember particular information, the ability to recall that information in the future is actually in itself a micro-skill. In learning a name from a book, for example, you are actually acquiring a procedure to recall that name. This further blurs the distinction between procedural mindware and declarative mindware, and between the types of learning outcomes described in chapter 2. The implication is that the literature on cognitive skill acquisition, discussed in the next section, is relevant to test-enhanced learning. Interestingly, Carpenter has argued that such elaborative structures are more likely to be formed at recall than at study time. This is not to say, however, that such laborious processing can’t be done at study time. In fact, Mary Pyc and her colleagues have found ample evidence that testing affects how one restudies (Pyc & Rawson, 2012). It stands to reason that after being tested on an item, particularly a difficult one, one will process it differently in the future. I would not be surprised to learn that you found the foregoing “propitiate” example to be rather implausible. How likely is it that one can reason all the way from the cue to the target so elaborately after one trial? Not very. It’s more reasonable to suppose that the elaboration is performed over a number of test-study iterations. Pyc speaks in terms of retrieval strategies. That is, many participants actively develop strategies to retrieve information from memory. Given the task of recalling a target for a cue, participants seek “mediators”, that is internal retrieval cues that they can use to reach the target information. Testing encourages them to shift and improve their strategies for particular cue-target associations. This she calls the “mediator shift hypothesis”. For instance, if relying on the term “propitious” doesn’t work
Deliberate practice: A source of effectiveness
159
very well, the participant can seek a better mediator. You may have noticed that the concept of mediator is similar to Ericsson’s concept of retrieval structure presented in chapter 4. Mediators are a kind of retrieval structure. Ericsson proposed the concept of retrieval structure in the literature on expertise and deliberate practice. The test-enhanced learning literature and the expertise literature are not often combined. However, they deal with very similar mental processes. The concept of productive practice, which is the cornerstone of the meta-effectiveness framework, brings them together. I briefly summarize the key ideas of deliberate practice below.
7.2 Developing cognitive skills with practice learning theories may be the only scientifically adequate theories of expert problem solving. Kurt VanLehn One of the main purposes of processing expository material is to become a more effective problem solver. But how can we develop cognitive skills from a knowledge resource? Good students implicitly know the answer to this question. Cognitive scientists have explicitly answered it. In 1993, I had the pleasure of meeting Kurt VanLehn, one of the world’s top experts on cognitive skill acquisition and problem solving. He was an invited speaker at an AI conference hosted at my alma mater, The University of Birmingham (England). The paper he presented had a profound influence on my understanding of learning—and on this book. VanLehn and his colleagues gather verbal protocols from students as they learn to solve problems in knowledge-rich domains. (These domains are contrasted with knowledge-lean problem-solving domains such as puzzles.¹²) Interpreting these “think aloud” protocols is very demanding. With a computer program called Cascade, VanLehn painstakingly modelled the cognitive processes underlying such learning.
7.2.1 Three phases of cognitive skill acquisition Of course, not all students address problems in the same way. Nor do they reach the same level of mastery. By understanding the differences between the way good learners and poor learners attempt to develop cognitive skills—i.e., by analyzing their differences in meta-effectiveness—we can better understand how to use knowledge to become more effective. VanLehn published a landmark review of the literature on cognitive skill acquisition, the core ideas of which are still relevant. He describes students as acquiring their cognitive skills (such as college physics) in three phases (VanLehn, 1996; Fitts, 1964). • The early phase involves boning up on objective knowledge (World 3) without focusing on applying it: attending lectures, reading, learning definitions, concepts, theory, equations, principles, etc. ¹²The Tower of Hanoi and the Cannibals and Missionaries problems are examples of knowledge-lean domains. These types of problems can be solved on the mere basis of a few instructions. This book, of course, is concerned with knowledge-rich domains. See VanLehn (1989) and Zimmerman (2000) on this distinction.
Deliberate practice: A source of effectiveness
160
• The intermediate phase is when the largest mindware changes happen (World 2). This is when students try their hand at solving textbook problems—they make mistakes and correct them. They practice with a variety of problems. Their accuracy and speed improves throughout this phase. This is also when the best students shine by developing a deep understanding. • Having covered a sufficient variety of cases, in the late phase students can focus on solving problems faster and reliably. Here, they achieve perfect accuracy without searching for new ways of solving problems. Of course, these three phases are an idealization. For example, effectant students will revisit the text, do additional reading, and continue to innovate beyond the late phase. (Compare the description of expertise in the following section.) The analysis in this section is confined to the intermediate phase. In Section 12.1 on effective delving, the initial phase is discussed. According to VanLehn, the most ubiquitous finding concerning the intermediate phase of cognitive skill acquisition is that both good and poor students prefer to learn with examples, such as the ones they find in textbooks.¹³ They differ, however, in how they use examples. Good students do more self-explanation as they work with examples. They pay careful attention to what they don’t understand. Poor students casually read examples. They tend to be overly optimistic about how well they will learn from them. They assume they understand things that they don’t. However, poor students actually spend more time examining examples during problem solving than others. They focus on what they take to be the most applicable example, then they try to apply the structure of the example to the problem they are trying to solve. That is, they engage in analogical reasoning. This may be a surprise to many who assume thinking by analogy is normally a powerful approach. Good students will study an example before they need to use it—not for the purpose of copying it, but for the purpose of understanding its underlying structure and rationale (VanLehn et al., 1993). (Compare the discussion of understanding in chapter 1.) If they get stuck while working on a problem, they might go back to the example to test and enhance their understanding. Then use the new understanding to work out their own solution to the current problem; they won’t try to directly copy or apply the example. Based on this research, one would expect these differences to show up in all kinds of knowledgeintense domains. For example, one might expect poor software developers, when faced with a problem for which they have not previously formed a solution, to be likely to use an Internet search engine to find example code earlier than a good programmer would. There are, after all, many examples of code available on the Internet for popular programming languages. When the good programmer seeks out code examples, one would expect him to use them for insight. The poor programmer would be more likely to use a copy-paste strategy. This can lead to all kinds of bugs ¹³Examples are not merely useful in formal education, they are potent for experts as well. John Gottman related a striking example of this: I had read probably fifty books on nonlinear dynamics and chaos before I read James Murray’s book Mathematical Biology. The other books left the process of modeling mystical and somewhat romantic. I had no real idea how to apply these methods to my data. Fortunately I was in a book club that selected James’ book, and once I started reading it, my eyes were opened. It contained example after example with real problems and real data. I began to see that it was possible to demystify these marvelous methods and bring them into my own laboratory. (Gottman et al., 2002, p. xv). It’s no coincidence that Gottman’s own books (and this one) are loaded with examples. Thus, other things being equal, prefer knowledge resources with many examples; or create your own examples. See also the discussion of examples in conceptual analysis, in chapter 12 and “Structure concept instillers”.
Deliberate practice: A source of effectiveness
161
and does not promote learning.¹⁴ I imagine that the Internet is posing challenges in other disciplines as well (e.g., the use of case law). Even with textbook learning, however, poor students are at high risk of selecting the wrong example, particularly when multiple principles are being taught. Textbooks themselves sometimes lack information required for solving problems they present. Cognitive scientists have also researched how students utilize and benefit from feedback. Here again there are crucial differences between good students and poor students. Good students don’t require as much immediate external feedback on their performance because they tend to monitor their performance and look for flaws in their mental models. Again, this normally means detecting knowledge gaps. When good students reach an impasse, i.e., a problem in their attempt to solve a problem, they pause and take heed. Though subtle, this is apparent in careful observation of students problem solving.¹⁵ Poor students seek a quick fix, which might mean assimilating false information (e.g., replacing a line of code that does not compile with a quick and dirty fix that does not adequately address the code’s requirements.) Good students, in contrast, are on the lookout for specific, informative feedback and tend to process it deeply, even if the feedback is delayed. The students that develop expertise don’t stop practicing at the end of the intermediate phase: they progress into the third phase. Having understood the principles and solved a number of problems for each principle, they seek perfection. They improve their speed and avoid accidental errors (as opposed to errors based on deeply flawed knowledge, which have already been debugged in the intermediate phase.) Ideally, in this stage they apply the learned knowledge to a wide collection of cases such that they can learn to recognize every potential case where the principle applies. Authentic experts remain on the look-out for red-herrings, cases that don’t fall into the pigeonholes of their existing knowledge. When they encounter an odd-duck, they will try to extend their understanding, search the literature for clues, talk to their peers, build new theories, and test them out in case they’re wrong. In Piagetian terms, they know when to assimilate and when to accommodate. This can be summarized in terms I introduced previously. Students differ in their thinking dispositions. Poor students are cognitive misers. Their meta-management processing is weak. Strong students tackle desirable difficulties head-on. Good students have monitors and motive generators that propel their understanding. They are meta-effective, meaning they are both effectant (motivated to learn) and skilled at learning (they use their fluid expertise to build upon and extend their crystalized expertise). The development of expertise requires that learners transcend this three-phase model of skill acquisition, as we shall see next.
7.2.2 Beyond Ericsson’s theory of expertise The position advocated […] by Ericsson and Charness (1995) should be called “absurd environmentalism.” Douglas K. Detterman, Lynne T. Gabriel & Joanne M. Ruthsatz ¹⁴This is not to say that copying and pasting code is always inappropriate. A large amount of code in running systems has been copy-pasted (Chatterji et al., 2010). This discussion is about the use of examples in learning. See also Jeff Atwood’s view on the matter. ¹⁵See for example VanLehn (1999). For a fascinating example of 4- to 9-year old children’s monitoring of knowledge gaps in problem solving, see Karmiloff-Smith & Inhelder (1975). Note the frequent reports in that paper of children pausing when faced with unexpected results.
Deliberate practice: A source of effectiveness
162
Ericsson and Charness help us to understand some hitherto less-appreciated aspects of training, but in belittling the role of individual differences in interest, motivation, and, above all, relevant computational powers or “intelligences,” they undercut the power of their case. Howard Gardner It is implausible to claim that deliberate practice is sufficient to overcome genetic variation. Ackerman’s (2013) response to Ericsonn’s framework is as trenchant as the others in the opening quotations: “It is patent nonsense that every healthy child or adult need simply engage in extensive, deliberate and motivated practice to attain expert performance.” Ackerman points out that no one has yet demonstrated that expert performance in complex intellectual domains has been achieved by anyone without reasonably high levels of talent¹⁶ and related education. The experts studied in the expertise literature are sufficiently intelligent that one ought not minimize the impact of IQ on expertise. Ackerman also provides several compelling methodological, logical and statistical arguments that destroy the grounds given by Ericsson for concluding that individual differences apart from deliberate practice are irrelevant to the acquisition of expertise. Here, Ackerman points to small sample sizes, restriction in range (population bias), faulty measurement of traits, and a misreading of the literature on individual differences. Hambrick et al. (2014) have reanalyzed two of the major papers on which Ericsson’s framework is based (in music and chess). Whereas Ericsson claims that most of the variance in performance amongst experts themselves can be accounted for by deliberate practice, Hambrick et al. discovered that only 30% to 34% of this variance could be so explained. They also found much more variance in amount of prior deliberate practice than acknowledged in Ericsson’s framework: Some chess masters only had 832 hours of deliberate practice whereas others had 24,284. Moreover, at each level of music skill reported by Sloboda, Davidson & Howe (1996), some students did 20% less practice than the average whereas some did 4 times more practice. Therefore, deliberate practice is far from sufficient to account for the development of expertise. What other factors are at work? There are important critical periods. For example, there seems to be a critical period for the acquisition of absolute pitch: age 8 (Deutsch et al., 2006).¹⁷ Hambrick et al. report that the effect of starting age on expertise is not mediated by the number of hours of practice. Many cognitive abilities decrease as we age, including our ability to learn. Season of birth may also have an effect (Gobet & Chassy, 2007). The 10,000 hour rule fails to account for these and other factors. Hambrick et al. reviewed a collection of studies that together demonstrate intelligence is both a significant and an important contributor to expertise. For example, people who scored at the 99.9 percentile on an IQ-correlated measure were 3.6 times more likely to have obtained a Ph.D. than people at the 99.1 percentile. They were also much more likely to have achieved other forms of intellectual success (e.g., registered a patent). Eight out of eight prodigies had working memory scores in the 99 percentile. Clearly, basic mental mechanisms have important and significant effects on what one can achieve with practice. ¹⁶Many proponents of the deliberate practice theory of expertise equate talent with genetic endowment. Ackerman is careful to point out that talent is a dynamic, interactive expression of genetics and the environment. ¹⁷As Sacks (2007) points out, absolute pitch is not necessary for musical excellence. It is undoubtedly a boon, however.
Deliberate practice: A source of effectiveness
163
Gardner (1995) points to the important role of motivation, and implicitly to effectance, in the development of expertise. He claims that one simply can’t “deny that a major variable operating on most individuals is their own success at a particular pursuit.” Of course, perceived self-efficacy can be manipulated in various ways. And effectiveness can improve with practice. But the drive for effectiveness affects (and indeed ought to affect) not only the level of effort one applies, but also the object of one’s effort. Ericsson’s framework also suffers from not providing detailed insights into underlying mechanisms. He has proffered long-term working memory, meaningful encoding and retrieval structures. As I mentioned earlier, these concepts are very high-level. They’re useful enough for non-specialists. I use them myself as communication devices for a general readership. However, performance is the operative word in Ericsson’s texts. His theory does not delve much more deeply into the expert mind than behaviorists do in their subjects. Further, a theory of expertise ought to sketch not only traditional cognitive mechanisms (of memory, etc.) but also affective mechanisms. I am not merely referring to how motivation is typically conceived in psychology, i.e., as “the direction of effort, the intensity of effort, and the duration or persistence of effort expenditures” (Ackerman, 2013). I am referring here to the use and development of motive generators, valenced perception, and so forth. (Compare chapters 5, 6.)¹⁸ For what develops in expertise is not just the operation of “dry” memory and cognition, but the affective processing of information. There is a significant problem with Ericsson’s framework that also applies to much of the literature on expertise. Research on expertise has focused mainly on performance that can easily be measured. Those are domains where the “rules of the game” can easily be defined: chess, music, medicine and sports. A contrasting facet of knowledge work is that in business the rules of the game change. A highly accomplished CEO once explained to me that the most important question a new CEO needs to answer when taking over the leadership of a company in distress is: “What game are you playing?” Sometimes the rules are given. Sometimes one can change them. Either way, one needs to know the rules. As British rock band Queen put it, “It’s so easy, when you know the rules.” Just before Apple entered the smartphone market, a RIM (currently known as Blackberry) officer was asked how they planned to respond to Apple’s anticipated entry in the smartphone market, given their inexorable propensity to change the rules of the game. Clearly, RIM failed to anticipate and adapt to the new game foist upon them.¹⁹ Knowledge workers must frequently anticipate, grasp and adapt to changes in technology, knowledge, and others’ expectations. Yet chess, the drosophila of expertise research, and the other narrow models of expertise, lead one to focus on narrow, skilled performance and knowledge. Expertise as normally framed in this research is itself only a small contributor to overall excellence. We need to invoke the superordinate concepts of meta-effectiveness and cognitive productivity to understand and foster knowledge-intense excellence. ¹⁸See Beaudoin (1994) for a description of affective mechanisms in autonomous agency. See Beaudoin (2014b) for high-level suggestions of their involvement in expertise. For very high-level affective aspects of expertise, see sections 3-7 of Ackerman (2013), Bereiter & Scardamalia (1993) and Ackerman (1996). ¹⁹Consider this further example of a preliminary 1994 Caribbean Cup game. In order to advance to the final Barbados deliberately scored a goal on itself because a little known tournament rule awarded a two-goal victory to a team that won in overtime. Barbados had changed the rule, and won the game. See Ingle & Murray (2002).
Deliberate practice: A source of effectiveness
164
In sum, along with the critics of Ericsson, I accept that everyone is partly constrained by the intertwinement of genetics, personal history, personality, conflicting commitments and circumstance. Further, I grant that deliberate practice may account for less than a third of the variance in expert performance. I also grant there is very little, if anything, individuals can do to improve their fluid intelligence. For example, deliberate practice itself has not been shown to improve fluid intelligence. To improve ourselves we must focus on what can change. Meta-effectiveness is the top-level, relatively malleable contributor to excellence, particularly in knowledge-intense domains.²⁰ Two sources of meta-effectiveness, deliberate practice and reflective practice, are clearly within the purview of many.
7.3 Reflective practice and deliberate performance Expounding on the art and science of using knowledge to become profoundly effective assumes that objective knowledge can reliably guide problem solving. This assumption is challenged in Donald A. Schön’s seminal book, The Reflective Practitioner: How Professionals Think in Action. Schön draws attention to a wide gulf between objective knowledge and professional practice (e.g., architecture, medicine, engineering, psychotherapy and education). So, let’s deal with Schön’s chasm. Schön refers to the standard, scientific, paradigm for applying knowledge as technical rationality. The paradigm works as follows. Objective knowledge is produced by scientists. The pinnacle of knowledge, pure science provides clear laws, principles, generalizations, concepts and a mass of observations. From this are derived the applied sciences. Accordingly, we have a distinction between biology and medicine, mechanical physics and mechanical engineering, empirical psychology and clinical psychology, and so forth. Lower in the hierarchy of knowledge (and prestige) we have the worlds in which practitioners operate. Architects, engineers, psychotherapists have the relatively easy task of applying knowledge they humbly receive from above. Schön attacks this paradigm on many fronts. Science can rarely be directly applied to real world situations. Professional practitioners are called upon to solve problems not in the simple, idealized situations scientists deal with, but in an inherently complex, uncertain, unstable world. The “swampy lowlands” of practice present unique situations. Further, science is supposed to be value-neutral; yet to solve practical problems one must address conflicting values.²¹ For instance, a medical doctor whose time is very limited must balance his proactive efforts to educate his clients to prevent disease with efforts to respond to concerns they express. More generally, practitioners must specify (or frame) and prioritize problems.²² Practitioners develop and rely upon not only technical skills, but an amorphous body of tacit, impressionistic knowledge. This is subjective (World 2) knowledge rather than objective knowledge. Schön also emphasizes that professional practice itself is an important source of knowledge ²⁰I grant however that it is malleable to different degrees in different people. Some people seem to have very little inclination to improve their effectiveness. The malleability of meta-effectiveness—i.e., the overarching set of personal contributors to expertise—poses a significant challenge to research on expertise. It is analogous (at a higher level of abstraction) to the difficulty that the ongoing remodeling of the brain itself poses to neuroscience in general and in particular connectomics, the study of comprehensive maps of connections within a nervous system (Campbell, 2013). ²¹I would add that most people graduate from university without an explicit grounding in the different forms of value (cf. chapter 11.) ²²Max Wertheimer’s seminal book, Productive Thinking, is relevant here again. He emphasized the importance of specifying the problems and requirements. Contra Schön, problem framing is also a major skill that good students learn in STEM disciplines.
Deliberate practice: A source of effectiveness
165
for practitioners. Effective professionals reflect on their work: their problems, clients, procedures, processes, etc. Through trial and error, they “muddle through”, solving problems and learning along the way. They perform experiments to test hypotheses, to produce desired outcomes, and to explore. They reflect in their practice and on their practice. Those who do not operate and reflect in this manner stagnate. Schön draws our attention to many important dispositions and processes that must be considered in designing a meta-effectiveness framework. Schön’s framework makes plenty of room for learning on the job; but it has little to say about deliberate practice. It can, however, inform our design of a new deliberate practice framework. Schön’s work focuses on the limitations of objective knowledge and the importance of tacit knowledge. It provides no explicit guidance on how to master objective knowledge to the point of valid and reliable application.²³ Schön’s theory correctly states that professionals generate and improve (objective) knowledge. In other words, they engage in knowledge building. Schön’s framework fails to address the fact that professionals often do not apply knowledge that they themselves create. Despite generation effects, the outputs of reflection can be just as sterile as academic outputs. To use self-generated knowledge effectively and reliably, one should apply a similar process to it as other knowledge. In particular, the epistemic output of reflection may be an input to deliberate practice. Fadde & Klein (2010) provide a framework for the development of tacit knowledge and intuitive expertise, based on the concept of deliberate performance. Their framework combines and extends Schön’s and Ericsson’s. Deliberate practice is offline practice—exclusively or mainly for the sake of self-development as opposed to external production. The title of Mark Guadagnoli’s book, Practice to Play, Play to Win captures this distinction. It also illustrates that deliberate practice is not normally applied to knowledge work. Deliberate performance is a mode of operating on the job. One designs and applies formative exercises without unduly compromising current performance. The exercises have similar requirements to deliberate practice. For example, they involve varied repetition with feedback. However, normally the learner should obtain feedback on his performance without engaging someone else. In contrast, the types of deliberate practice examined by Ericsson often involve a separate trainer who provides feedback. Deliberate performance calls for extensive self-monitoring and analysis. However, technology can in principle help. Moreover, coaches can be involved. Fadde and Klein describe the following types of deliberate performance: • • • •
estimating, experimenting, analyzing successes, near failures, and failures while building causal models, and combining the above into reflective explanations.
²³It is also noteworthy that the amount of factual and practical knowledge available to practitioners today has increased exponentially since the publication of The Reflective Practitioner. For instance, medical doctors now have extensive guidelines to deal with diabetes (American Diabetes Association, 2012). This facilitates action by handling a much greater number of contingencies. It does not eliminate the need for the kinds of activities Schön emphasized; however, it does suggest an increasingly important problem for practitioners is to deploy this vast arsenal of objective knowledge. (This, again, is the practitioner’s version of the so called “knowledge translation” problem.)
Deliberate practice: A source of effectiveness
166
Thus, while the main objective of deliberate performance is to build tacit and implicit expertise, it also involves considerable knowledge building. Deliberate performance is expected to decrease occupational stress if the agent treats sub-standard performance as a learning experience. The agent anticipates better future performance. This is a mechanism of perceived self-efficacy. Consider an engineer seeking to improve his writing skills with outlining software (such as OmniOutliner²⁴ or UV Outliner²⁵). He might systematically estimate the time it will take to write each section of a document, timing himself as he writes. He could create documentation quality metrics, using them to assess his work while attending to feedback from others. He could attempt to analyze the impact of working with an outliner: What were the costs and benefits of the approach? Was it worth it? Why? As with reflective practice, the objective knowledge gleaned from deliberate performance can be used as an input to productive practice.
7.4 Enter productive practice The bodies of cognitive science literature reviewed here point to the considerable potential of deliberate practice. Self-testing can directly and indirectly improve memory. Practicing is essential to the acquisition of cognitive skills. It has beneficial cognitive effects. Deliberate practice potentiates cognitive performance at school and in the realm of traditional, public expertise. Deliberate performance and reflective practice have not been the subject of extensive research in cognitive science. Yet clearly they can contribute to adult mental development.²⁶ The vast majority of research on deliberate practice is conducted in academic and public performance domains. Knowledge workers do not systematically test themselves or deliberately practice the information they read. This led Fadde & Klein to claim that “it seems impractical to expect professionals and businesspeople to practice in addition to doing their jobs.” Appearances, however, can be deceptive. There was a time when it would have seemed impractical to expect people to jog or to dictate instructions to a computer. Eventually, deliberate practice may be adopted by large numbers of excellence-seekers, if supporting conditions are established. Knowledge workers already spend very large amounts of time processing information. However, they don’t yet understand that failing to practice limits the potency of their cognitive efforts. We are now presented with the exciting possibility of combining and exploiting, for the first time, the bodies of knowledge presented so far in this book. The opportunity and the challenge are to design ways of interacting with knowledge that enable effectant adults to develop themselves with cognitive science and technology. Given that the knowledge economy is very competitive, and deliberate practice is a potent source of competency, and potentially success, it is just a matter of time before deliberate practice is recognized as such. However, we must not fall into the same trap as popular books like Outliers and The Talent Code. As we saw, some of their generalizations do not even hold in traditional domains of expertise. ²⁴http://www.omnigroup.com/omnioutliner ²⁵http://uvoutliner.com ²⁶The question is not whether these forms of practice work. More interesting questions are: How can the practice be supported by technology?
How does the architecture of the mind change with practice? What detailed changes to mental mechanisms can be fostered with practice?
Deliberate practice: A source of effectiveness
167
Deliberate practice principles cannot optimally be applied directly to professional and personal development, domains where they have hardly been studied by cognitive psychologists. In order for knowledge workers to systematically engage in deliberate practice, the challenges referred to above needs to be addressed. In particular: • A new form of deliberate practice must be developed that addresses the challenges, circumstances and concerns of knowledge workers laid out in chapter 3. I call this form productive practice: effective and efficient offline practicing with knowledge gems for manifold psychological purposes. • Excellent technology must be developed to support productive practice. It must make it very easy for users to identify, tag and capture knowledge gems in the resources they are processing. It must also make it easy for them to acquire, develop, tailor and execute their practice exercises. It must provide excellent coaching services. It must make it extremely easy for users to navigate between knowledge resources and practice information. This technology must optimize training time (including time spent designing practice) and mental development, for example by enabling users to practice eyes-free and “on the go”. • Productive practice systems must exploit cognitive science. They should also feed information back into cognitive science. With technology, we can potentially collect enormous amounts of scientific data about learning that can be used to test and extend theories about learning. (Compare Winne, 2006, 2014; Winne & Baker, 2013.) • Information about cognitive productivity must be disseminated to help people understand why and how to engage in productive practice. Productive practice will spur new markets and new forms of media. Books are often sold by word of mouth. But readers cease to talk about books that they have forgotten. Readers talk about books that have transformed them. Traditional content (ebooks, audiobooks, etc.) is not nearly as transformative as productive practice content. Thus traditional content that is not accompanied by or integrated with productive practice kits is normally not memetic. As productive practice catches on, authors and publishers will want to ensure that their content is augmented with productive practice kits. The closest media to these kits are textbook study guides.²⁷ Competitive pressure on knowledge workers to change how they interact with high quality information will concomitantly evolve. Effectant people will respond to this opportunity not primarily for competitive reasons, but because of their implicit drive for self-actualization. Many will come to understand that by allocating time to productive practice they can decrease their overall information-processing time while becoming more effective. Productive practice, however, is not the only tool required in one’s cognitive productivity toolkit. Let’s turn our attention to this toolkit. ²⁷Before and after the announcement of the iPad, I called for Apple’s new tablet to include a productive practice system. I made the same suggestion in a 2010 white paper on cognitive productivity for Steve Jobs. In Jan 2011, Apple released a version of iBooks Author that enables authors to add flashcards to books. However, iBooks and existing flashcard software lack most of the power and convenience of the productive practice system described in Chapters 13 and 14.
III Solutions THE READING OF THIS book would be a disappointing experience for anyone who expects easy instruction in the art of loving. Eric Fromm Industry need not wish, and he that lives upon hopes will die fasting. There are no gains without pains[…] Benjamin Franklin
8. Introduction to Part 3 The goal is that you’re able to keep the good parts and not descend into insanity. That the pain of the experiment will end up making life better in the end. And that your spouse will forgive you. A. J. Jacobs Let’s summarize where we’ve been and where we are going in the rest of this book. I argued in Part 1 that the Knowledge Age brings a number of opportunities and challenges. There is potentially helpful knowledge we can exploit for many different purposes: to develop skills, propensities, habits, attitudes, solve problems, develop knowledge and other products. Ultimately, we process knowledge to become more effective. But have potentially helpful books on building good relationships improved marriages as much as they could? Have great management books like The Lean Startup¹ always sufficiently transformed their readers? Has reading about the psychology of rational thought made us sufficiently rational? More generally, have the knowledge resources we have processed improved our effectiveness as much as they should have? I answer “no”. It is not that authors have failed us. The knowledge they conveyed is there to be reaped. In Part 1, I listed many challenges to our meta-effectiveness. The first is that not all of our concepts, terms and tools have yet been adequately adapted to the demands of the Knowledge Age. For example, the concept of “meta-effectiveness” itself had not, before this book, been articulated and emphasized as an imperative for knowledge workers. Meta-effectiveness includes our ability to systematically exploit any knowledge resource to develop effective products and, most importantly, to develop ourselves such that we use the knowledge “at run time”. The information consumption metaphor needs more broadly to be replaced with the information processing metaphor. We work with knowledge but we don’t necessarily have an explicit understanding of the work we do with knowledge and how we can improve it. I noted that we have plenty of tools with which to access and superficially process information. However, we need better tools to deeply process resources: to delve and master them. We need tools to systematically annotate resources in a fine-grained manner and across IT devices. We need tools to help us master technical concepts. Mastery requires practice and so we, knowledge workers, need tools to help us productively practice knowledge gems. I also listed various circumstantial challenges to our meta-effectiveness: e.g., demands on our time, ill-presented information, and lack of cognitive productivity training. As chapter 3 demonstrated at length, however, the problems we face are not merely external to us. We are subject to various illusions of meta-effectiveness. There are illusions of comprehension, illusions of future recall, and illusions of rationality. It is all too easy to assume that we will be able to apply the information we have, even carefully, processed. We give in to the so-called “transfer problem”. And so we don’t apply the means to develop the competence that would justify the time ¹http://theleanstartup.com/book
Introduction to Part 3
170
we have spent processing resources. By nature, humans are somewhat miserly: they must judiciously use their limited resources. However, cognitive miserliness can get us into trouble. Thus we need to bolster our effectance (our motivation to improve our effectiveness), protect our perceived cognitive self-efficacy, and engage in productive laziness. While the mind is a virtual machine, its brain is a real biological organ that, like any other, ineluctably ages as the clock ticks. So, meta-effectiveness calls for the brain to be nurtured through best lifestyle practices (nutrition, stress management, exercise, and so on). We can also exploit the fact that some capabilities can improve as we age. Those capabilities are the result of the exercise of fluid expertise and effectiveness—the skills and motivation at the heart of our meta-effectiveness. Distractibility is a requirement of the human mind that countless agents in the attention economy attempt to exploit. Meta-effectiveness calls for our most precious resources, our management and meta-management processes, to be focused on developing the right stuff at the right time. Rather than cower at the sight of these formidable challenges, we must, forewarned and forearmed, pro-actively address them. For, as Aristotle said of the intellect over 2000 years ago, we may say of our meta-effectiveness “even if it be small in bulk, much more does it in power and worth surpass everything.” Part 2 provided information-processing concepts to help readers think about the mind with cognitive science. I described a high-level architecture of the mind. I proposed thinking of adult “learning” as mental development. Research shows that deliberate practice is a potent tool to help people use knowledge to become more effective. I propose that it is a potent instrument not only for people to improve at performance arts, but for meta-effectiveness in general. In the following chapters, I will explain how one can use information technology and cognitive science to process and master knowledge. I am confident that even tech and cognitive science experts will discover potent new meta-effectiveness concepts, strategies and techniques. You don’t need to be an expert, however, to benefit from this book. You do need effectance; to be reading this book, you qualify. • Chapter 9 presents a framework to help you learn your way around your knowledge resources, your R&D projects and your R&D activities. It distinguishes three levels of informationprocessing, i.e., of your R&D: surfing, delving and developing. The objects of development are solutions, products, understanding and yourself. • Chapter 10 will help you inspect knowledge resources. • Chapter 11 will help you systematically assess knowledge resources. This will help you determine what to do with the information: toss it, file it for future reference, delve it for problem-solving and knowledge building, or develop yourself with it. • Chapter 12 describes useful ways of delving knowledge resources with information technology. It describes annotation concepts and tools. It shows you how to qualitatively tag entire resources and snips of resources. It shows you how to quickly create and access meta-docs (documents about the knowledge you process). It even helps you delve eBooks, audio and other media. • Chapter 13 describes the overall process of productive practice: how to capture knowledge gems, design instillers and practice with them. It shows you how to co-opt flashcard software for this purpose until apps that are specifically designed to fit the workflows described in this book are available.
Introduction to Part 3
171
• Chapter 14 shows you how to design and use instillers for specific types of material and learning objectives. There are tips for mastering concepts, mastering collections and developing monitors, propensities and developing attitudes. In other words, this chapter is meant to help you exploit knowledge in your pursuit of personal effectiveness. By the end of reading this part of the book, you will, I believe, have at your disposal an enhanced conceptual toolkit to address the formidable challenges and wondrous opportunities of the Knowledge Age: One that will enable you to exploit knowledge to become more effective.² ²Part 3 delves into its subject in considerable detail. Keshav (2013) is an extremely succinct, but useful, treatment of how to read a paper. Thanks to Stefane Fermigier for mentioning that paper.
9. Learn your way around your R&D The growth of widely accessible information can lead to a decline in the overall level of our knowledge because access to knowledge requires a set of highly developed skills in appropriating knowledge. Gernot Böhme Processing enormous amounts of information is challenging, even for knowledge workers. We sometimes inadvertently switch from one task to another—from processing email to delving a complex document, for example. We sometimes treat a distractor as if it was a knowledge gem. We sometimes overlook knowledge gems. So, we turn to productivity systems and tools that are supposed to help us focus and accomplish tasks. However, we meet with limited success if our tools are designed for transient efficiency rather than developing effectiveness. That is the contrast between using knowledge merely to “get immediate things done” and to use it to become a more effective person. David Perkins proposed a geographical metaphor for understanding intelligence that is germane to this chapter (Perkins, 1995). Reflective intelligence, to Perkins, involves knowing one’s way around, and being able to navigate within, realms of thinking: thinking dispositions (dispositional realms), thinking challenges (challenge realms), techniques for thinking (tool realms), resources that support thinking (technical realms), thinking situations (situational realms) and contexts of thinking (contextual realms). Each of these increasingly specific realms has its action, belief and conceptual systems. If you are merely acquainted with a big city, for example, you probably don’t really know how to get around there. You will need to call upon maps, information technology and people. Without knowing your way around linear algebra (a technical realm), you would struggle to model cognition with neural networks. Part 1 and 2 of this book were designed to help you “learn your way around” your purposes, challenges, dispositions and mind. Meta-effectiveness involves knowing one’s way around the realms of one’s research and development.¹ More specifically: • To know one’s way around the different levels of knowledge-processing: surfing, delving and developing. This pertains to important things we do with knowledge resources: we inspect, assess and delve them. By developing and applying knowledge, we develop ourselves. • To know one’s way around one’s knowledge resources and the meta-information one generates in processing them. This concerns generating navigating, accessing and utilizing meta-information. Ultimately, our effectiveness hinges not on exploiting entire knowledge resources but the information we construct about them. ¹Perkins described realm theory as a theory of intelligence. I prefer to think of it as a theory of components of meta-effectiveness. I agree with Keith Stanovich that it is better to stick to the technical definition of intelligence widely but not universally accepted in psychology (i.e., as that measured by I.Q. tests). Perkins himself in introducing realm theory wrote “The whats to be found—the important realms of thinking—are just those that correspond to major mysteries that arise as we attempt to act effectively in the world.” The term “effective” shows up many times in his book. Clearly, effectiveness is a major aspect of what Perkins had in mind.
Learn your way around your R&D
173
• To know one’s way around one’s R&D projects and activities: capturing, classifying and organizing our knowledge work in projects and tasks. We also need to understand the different types of activities we engage in with knowledge. The following chapters focus on several key activities outlined above: to inspect, assess, delve and practice knowledge.
9.1 Learn your way around levels of processing We process knowledge resources in many different ways. To manage one’s processing, it helps to have a simple schema for classifying information-processing activities. This section provides an overall framework that will be elaborated throughout this part of the book. As you acquaint yourself with a knowledge resource, you begin to form an opinion about what to do with it. You have a number of choices. You might postpone processing it. You might immediately trash it (reject it.) You might take immediate action (which is to accept the resource.) You might file it for future reference or action. You might skim it. Or you might process it more elaborately. If you do process it further, you normally ought to have an objective in mind, something you want to accomplish with this knowledge resource now or in the future. Compare the opening figure of chapter 2 that illustrates the different destinies of information, from being rejected on contact to being mastered through reflective performance or productive practice. Thus, I propose a way of classifying our processing of knowledge resources in terms of how “deeply” we process it. These qualitative distinctions are sometimes hard to apply; however, it helps to note the following categories and think in terms of them. I distinguish between surface processing (surfing), delving and developing. These activities differ with respect to the degree of involvement with the knowledge conveyed by the resource and the understanding they engender. Each is important in its own right.
174
Learn your way around your R&D
Figure 9.1 Levels of Information Processing
Surface processing (“surfing information”). Surface processing refers to the activities we perform with knowledge resources that deal only with its surface aspects and that are not likely to redesign our minds, change our understanding in a lasting and pervasive way, or to enable us to significantly change how we develop new products. Here, we do not engage in extensive, elaboration of the information. This includes activities such as the following. • • • • • •
Browsing physical or digital bookstores, catalogs, etc. Surfing the web. Shopping for resources, acquiring them and purchasing them. Scrutinizing, sampling and pigeon-holing. Preliminary assessments, e.g., how promising they are with respect to their utility and potency. Making preliminary decisions about whether, when and how to use them (i.e., planning, scheduling and goal setting). • Scanning (e.g., with OCR), photocopying, storing, filing, tagging, handling and transporting them. • Searching for information (e.g., using a search engine). • Inspecting: Skimming documents, superficial reading, processing quickly and mass “reading”; attending presentations, seminars, workshops, meetings, etc.; watching educational TV, screencasts, videos, etc.; listening to podcasts, radio, audio books and presentations. Notice that for simplicity I’ve included “inspecting” under the rubric of surfing even though it does involve processing some of the resource’s innards. Inspection is still a superficial engagement
Learn your way around your R&D
175
with the knowledge conveyed by the resource. Think of it as the deepest level of surfing. You might prefer to extract it from surf, which is fine so long as you don’t confound it with delving. The following chapter deals briefly with inspection. How well you surf determines the extent to which you will capitalize on knowledge. For surfing ought to be, in particular, a gatekeeper to deeper processing. It’s also about controlling whether, when and how you will further process information. This is a function of what I called “meta-management” in chapter 5 and what psychologists include in executive functions. Surfing is a recursive concept, in that it also applies to how one controls one’s surfing. Spending too much time at the surface of information relative to deeper processing stunts one’s effectiveness. To be sure, some material, such as routine journalism, is of so little value that, for most of us, it does not invite elaborate processing— though even there we need to be careful lest we be unduly influenced. Surface processing does not mean processing in a slapdash or mindless fashion. One can carefully, systematically and zestfully surf information. But you don’t need to comprehend something deeply to classify it. The distinction between surfing and delving has to do with depth of engagement with the knowledge conveyed by a resource. Delving, described next, involves being engaged with the innards of knowledge and comprehending it. You can think of a knowledge resource as a door onto knowledge. Surfing means you are operating at the perimeter of a garden of knowledge, peeking through fences and gates. You can do this carefully (or not). You might diligently browse the web, shop, and then assess resulting resources. That still counts as surfing. When you delve, you walk through the gates of the resource and start to dig into the knowledge it conveys. The distinction between surfing and delving also has to do with the kinds of changes it entails in the knower. Although this is not a precise concept, interacting deeply means that one is engaging more of one’s mental processes (e.g., imagining, considering alternative possible meanings, etc.). In a sense, with delving one disengages from the resource as an artifact and engages with the knowledge referenced by it. Nicolas Carr’s book, The Shallows assumes that Internet users are shallow processors. Yet technology does not condemn us to be superficial information processors. With the right concepts and meta-management strategies, it is routinely possible to delve knowledge gems. Delving.² I use the term “delving” to refer to a wide variety of ways in which we elaborately process knowledge resources. It’s a generalization of the concepts of active reading, critical reading, constructively responsive reading, and studying applied to all kinds of knowledge resources (not just documents) and various forms of technology. Knowledge workers don’t tend to think of themselves ²Here follows a detailed explanation of my selection of the term “delving” and my development of the concept. The concept and term help us make sense of the most interesting mental work we do with knowledge resources. The term delving is sometimes used for this concept, however the use is relatively rare and circumscribed. Having a term for this concept wasn’t as important before it became clear we had entered the Knowledge Age. This is another example of the thesis of “Updating how we think about knowledge and ourselves”, above. Before settling on the term, “delve”, I considered several neologisms, none of which satisfied me. The term “read” is related and also short (a good thing for a frequently used word) but it only applies to what we do with documents–one doesn’t read lectures, presentations, podcasts and the like. Active reading is no better. The expression “process” is too general because the most superficial inspection also counts as processing (cf. speed reading). In their landmark book on reading, before the prevalence of multimedia over the Internet Protocol, Pressley & Afflerbach (1995) proposed the expression “constructively responsive reading”. They loaded the definiens into the definiendum–the expression is too long and insufficiently general. “Active processing” is also too abstract and wordy. I considered reviving the obsolete word “to dep”. It’s more parsimonious to proffer a precising definition than a neologism (Copi & Cohen, 2005.) But this word is clearly archaic. The term “study” has the closest meanings. However, it has too many established usages that depart from delving. Academic studying is sufficiently different from professional delving. It is too difficult to change the meaning of such an established term. Also, knowledge workers don’t tend to think of themselves as studying. By using the term “delve”, I can stipulate certain criteria.
Learn your way around your R&D
176
as “studying”; but if pressed they would acknowledge that this is something they do. Delving, like reading, is not just one thing. Pressley & Afflerbach (1995) documented that we read paper documents for many purposes and in many ways. Information technology has significantly added to the complexity of delving. I will provide examples of delving to give you a flavor of this variety and set the stage for an analysis of the requirements of delving tools.³ You could delve in order to solve a problem, build a product, or learn something useful. We often don’t need to delve an entire resource to benefit from it. You might want to “ransack” it for some particular facts, concepts or principles. You might interleave superficial processing with delving. When you delve a resource, you normally have some goals in mind. You’re typically vigilant for unexpected surprises, or knowledge gems. When you delve, you aim to comprehend some or all of it; otherwise, you’re just surfing at best. Here’s a sample of things that qualify as delving. • Determine your objectives with respect to the resource. What do you want to get out of it? • Analyze the resource. This could involve highlighting or tagging parts of the resource. Get a sense of the structure of resource itself and of the knowledge that it conveys. • Try to comprehend, make sense of and ultimately understand the knowledge conveyed by the resource. The resource is really just a lens on a conceptual artifact or some abstract artifact that could have been described differently in a different resource. (For example, the theory of relativity has been documented in countless ways.) All kinds of activities that generate understanding are involved here. For example, to get a sense of the problem and goals that the knowledge addresses, how the knowledge accomplishes this end, what other ways there are to resolve the said problems, and how well the knowledge addresses the problem. (see Understanding understanding below.) Compare the knowledge with other resources. Draw inferences about the knowledge. Explain the knowledge to yourself and others. Monitor your understanding and make notes of things you don’t understand. Try to bridge your knowledge gaps (e.g., by thinking, consulting other resources). • Assess the knowledge. Assess the knowledge according to standard metrics. Detect inconsistencies, gaps, flaws, etc. (See chapter 11). • Apply the knowledge. Run some thought experiments with the knowledge. Try using the knowledge on test cases to get the hang of the knowledge. Do some of the problem exercises. Implement small prototypes. Run simulations. For example, after reading The Lean Startup⁴, Delve is short. Many of its Oxford English Dictionary definitions are apposite. According to the Oxford English Dictionary, it can be used transitively with a direct or indirect object and intransitively (Simpson, 2013). The transitive use of the term seems to be rare, but the Oxford English Dictionary does not list it as its first definition of delve (one of its 10 examples of this first definition is “Time was when our countrymen united every employment; they delved the soil, they wove the fleece.”). I couldn’t find a similar transitive use in the Corpus of Contemporary American English (COCA). It is of course common to speak figuratively of delving into abstract items. The term does not have an established technical meaning. It conveys the notions of working with materials, of digging into materials. “Delving” aptly conveys the ideas of active processing, elaborate processing and depth of processing from cognitive psychology upon which I draw. It complements surfing. The Oxford English Dictionary also offers this definition: “To obtain by digging; to dig up or out of (the ground)”. This enables us to talk about delving knowledge gems. Finally, sometimes people do speak of delving into a document. The concept of delving is not tied to a particular theory of reading or learning. It can be used within the general context of theories of self-regulated learning. By having a term for this concept, I believe it will become easier for knowledge workers to think and communicate about this important class of mental work. It may even help them favor delving over surfing, when it is appropriate to do so. ³This is important because, as the history of AI has shown, people tend to assume that mental processes are much simpler than they actually are. To enhance these processes we need to acknowledge their sophistication. ⁴http://theleanstartup.com/book
Learn your way around your R&D
177
you might try to run a micro-project in a lean fashion. While reading Gottman’s books, you might start looking out for bids in your partner and at work. • Dream and meditate. Hans Selye provided sage advice “I use my reading largely as a scaffolding for meditation on research and the planning of experiments. It is very difficult to sit down and tell yourself, “Now think about an interesting experiment,” but ideas come automatically as you mix someone else’s thoughts with your own by reading his text at leisure.” (1964, p. 331) This is an important counterbalance to advice I provide about rigorous analytical reading. • Give reign to your passion. Pressley and Afflerbach observed that experts tend to be passionate about their reading (p. 101).⁵ Passion shows up in many ways. For example, effectant people read more documents they find challenging. It’s also evident in how effectant experts respond to text. Effectant experts often vigorously criticize documents. They can be passionately excited about it in ways that bewilder others. One needs cognitive zest—enthusiasm and selfconfidence about cognition—in order to plough through texts that really stretch our minds. • Extract knowledge gems. One of the most important components of delving is to identify and extract information with which you can develop yourself, products and solutions. Before setting aside a knowledge resource indefinitely, you need to determine whether it contains knowledge gems worth extracting for future development. For example, while reading Gottman’s books you might realize that the concepts of bid and harsh startups are gems that you ought to extract. Or while reading The Lean Startup⁶, you might want to extract a slew of concepts, such as his reasons that explain why startups fail, his definition of startup, the “Build-Measure-Learn feedback loop”, “Andon cord” and so on. Extracting a knowledge gem for learning is to ensure that you will master it to the desired extent, by the date you want to master it, and for the period you want to master it. This requires of course that you have a system that provides these guarantees. Most people don’t. One also needs to be systematic about how one extracts knowledge gems and what one does with them. See chapter 10, Delve, below. If the knowledge you are processing is deep, complex and relatively new to you, then you won’t fully understand it just by delving it. Sure, you may have done some experimenting and application. But to really master a knowledge gem you need to use it deliberately and systematically. In other words, you need to enter development mode.⁷ Developing. There are three main types of things you might want to develop on the basis of knowledge resources: products, solutions and yourself. The first two are external (World 3), so I will treat them together. ⁵With reference to Pressley and Afflerbach’s research, I spoke in terms of experts’ tendencies; but don’t get me wrong. It’s a fallacy to claim that people who want to enhance their expertise ought to ape experts. For one thing, there is not enough data about expert’s knowledge work to be sure what they do, how they do it, and how often they do it. Even if the data existed, it would (for the most part) only be correlational. Correlation does not imply causation. We must instead base our recommendations on logical and mechanistic considerations, i.e., on understanding. Clearly, one can’t develop one’s expertise in a knowledge-based domain without processing knowledge generated by others. There are a lot of smart people writing a lot of important things; even if the documents are not directly on one’s topic they may be very germane. But, of course, there is such a thing as too much reading. And as I argue in this book, it’s not how much you read but what and how you read. If you spend much more time surfing as opposed to delving or developing, you won’t benefit much from your reading. A challenge we face is to balance these three types of activities. ⁶http://theleanstartup.com/book ⁷This is not to say that development mode guarantees understanding. Your goal might just be transient use of information. Or you might drop the gem before mastery.
Learn your way around your R&D
178
Developing products and solutions. This amounts to using knowledge for external purposes. You might engage in progressive problem solving, develop new knowledge, train and teach others, or use your knowledge offline from the knowledge resource itself in other ways. Amongst products you might create with knowledge there are, as mentioned above, physical goods and abstract goods, including knowledge. Of course there are all kinds of solutions you might use knowledge for. For example, you might want to use the ideas of the The Lean Startup⁸ book to design a new software development process for your company (that’s an abstract good). Or you might want to use them to tweak existing processes. You can use a manufacturing idea to improve the design of a knife. There’s no end to the examples. Often, you can create external products without needing to develop personally. It might be that you need to comprehend the information just long enough to solve a one-off, transient problem. For example, you might be a Mac user and yet agree to help your friend fix a problem that is specific to Windows 8. You might keep a bookmark or two around, but you don’t need to have that information available to you after today. Developing effectiveness. In contrast, you sometimes need to learn something to the point of being able to do it quite well, on demand, without brushing up—i.e., without relying on search engines and documents. Or, you may be allowed to re-access knowledge resources, but be required to re-interpret them very quickly and accurately. That calls for real mental development. For example, you might want to apply lean startup methods to future projects. And so you want to master the concepts, to be able to think with them, and to have the right affective and cognitive reflexes that implement the knowledge. You might want to be sure that when you detect a process problem, you immediately consider pulling an andon cord (a lean startup concept). And per genchi genbutsu, when there is a problematic situation, you want to consider going to see it for yourself. Such effectiveness cannot be developed through delving. Moreover, bookmarking resources will not suffice. As I shall argue below, it is very useful to be able to access information more rapidly than even excellent bookmarking affords. Developing effectiveness calls for considerable mental development. The process you use to derive personal effectiveness from information needs to set up mindware in yourself: new monitors, new motive generators, new conceptual understanding, new management procedures, new cognitive reflexes, modified filters, etc. There is no magical solution to the complicated problem of taking knowledge as input and producing the desired effectiveness as output. There are very many different forms and mechanisms of learning. “I don’t think anyone has a comprehensive theory of the mechanisms and processes involved [in various forms of learning], including for example learning to recognize fugues, learning to enjoy fugues, learning to play fugues, and learning to compose fugues” (Sloman, Personal communication, 2012). We should not expect the same set of techniques to work for all of them. However, there are some general principles, such as to engage in productive practice and knowledge building with the knowledge. Here follow three other diagrams to illustrate this thesis. The next one illustrates how at each stage of processing information we are in effect funneling information through increasingly elaborate mental activity. Our information processing “velocity” decreases as a function of the type of processing. We can surf vast amounts of information. But we can only delve a small portion of ⁸http://theleanstartup.com/book
Learn your way around your R&D
179
knowledge. And we can only develop (knowledge, solutions, and ourselves) with an even smaller portion of it. As information traverses these metaphorical funnels, the value we assign to it and that it consequently acquires, increases dramatically. This diagram illustrates that we must resist the urge to overly rely on digitally stored information and surface processing if we are to optimally develop. The diagram does not illustrate every possible path, the most common one being to redirect information into the bit bucket (i.e., ignore it).
Figure 9.2 The Information to Effectiveness Funnel
We can use this tripartite framework—of surfing, delving and developing—to describe “the
180
Learn your way around your R&D
shallows” problem presented by Nicolas Carr. Let’s contrast three hypothetical people (or phases in a person’s life), in terms of the amount of time and effort they spend in each category. (See the figure above). The superficial processor, depicted at the top of the figure, lives in the shallows (presumably, what Nicholas Carr had in mind). There he spends most of his time surfing the net, searching, skimming, browsing, etc. He prefers easy resources that won’t extend his understanding though they may add to the repository of superficial knowledge with which he is vaguely familiar. He spends only a small portion of his time productively processing knowledge. Whether the resource he is processing is easy or difficult, he doesn’t tend to set goals for himself, monitor his comprehension, or demonstrate effectance. He rarely notices, let alone extracts, real knowledge gems.
Figure 9.3 Shallow vs. Deep Processing
Contrast this surfer with a deep processor. This person also spends time surfing (we all need to). He avoids superficial and irrelevant information. He looks for knowledge gems. In contrast with the surfer, he spends over a third of his time delving—elaborately processing potent, useful information. He sets processing goals. He analyzes resources carefully. He sees through information and engages with the knowledge to which it refers. He gets excited about its potential uses. He explores them. He runs mental simulations with the knowledge. He questions the resource. He wonders “what if we looked at this differently?” He might highlight, annotate, tag, summarize, etc. Of course, he winds up with more knowledge and competence. He eats the surfer’s lunch, outperforming him on knowledgeintensive tasks. Unfortunately, he skimps on development. This might be because he doesn’t have a system for development or he is simply not motivated to develop. As a result, his memories fade more than they should, and he ultimately does not get as much mileage out of knowledge as the next fellow. Figuratively speaking, he gets a B. When a colleague or client wants some help on a different problem, they prefer to talk to the productive processor. The productive processor spends as much time delving as his B colleague. However, he systematically extracts knowledge gems from excellent resources and then develops products and
Learn your way around your R&D
181
becomes effective with the knowledge. When he puts down a good resource, it stays on his mind. He uses the information in conversation. He tries thinking in terms of the information. He has a system to ensure that when the information may be of use in the future, he will be able to call it up with little or no recourse to the original resource itself. To use Bereiter’s (2002) metaphor of understanding, presented in Section 6.3, the productive processor develops a relationship with knowledge. Does the productive person spend more time processing information than the deep processor or the superficial one? Not necessarily. Productive people work smarter, not necessarily longer. It’s the effectiveness of processing that matters. Judging by the amount of shallow information on the Internet and the advertising dollars that support it, there is a lot of superficial processing going on. We know from cognitive psychology that re-reading is not a very effective way to develop one’s mind. The productive processor can cut back on both superficial processing and delving, and yet develop more knowledge and mindware. Admittedly, surfing, delving and developing are not precise concepts. Nor is having a framework a guarantee of success. Nevertheless, I believe being in the habit of classifying your activities in this way can help you distinguish them. It makes it easier to measure and control the quality of your information processing. The Hawthorn effect kicks in, i.e., you perform better because you feel you are being observed —if only by yourself, by your own meta-management processes. For, as Marvin Minsky argued, the mind is not a monolithic entity, but composed of a multitude of components—far more than Freud discussed. That’s one of the reasons why self-observation can work in the same way as being observed by someone else.⁹ How many of us have a clear sense of how much time we spend surfing vs. delving? How many of us systematically deliberately practice? With these concepts in your toolkit, you can get a better sense of how much time you spend in each activity, which can help nudge you towards more effective processing.
9.2 Learn your way around your meta-information As you process knowledge resources you generate and access information about them, i.e., metainformation. Managing all of this information can be challenging.¹⁰ Knowledge workers process thousands of resources a year, each with its own cloud of meta-information. So, it is helpful (a) to have a good understanding of the types of meta-information you generate and access; and (b) to use workflows and software that enable you to productively process the meta-information. In this section, I describe meta-information, the meta-access problem and how it can be resolved. As an illustration, let’s look at some meta-information you might generate with respect to The Lean Startup¹¹. I’ve italicized types of meta-information. ⁹I.e., the core of “social facilitation” is necessarily a cognitive phenomenon. Moreover, the auto-Hawthorn effect is not restricted to cognitive productivity. For example, recording your weight every day can motivate you to lose weight. ¹⁰This section addresses the research information management aspect of personal information management (PIM), meaning the individual’s management of research information they generate and acquire. In my experience, knowledge workers rarely discuss their PIM strategies. Oh & Belkin (2014) and Oh (2012) illustrate how simplistic people’s information management strategies can be. Hardof-Jaffe & Nachmias (2013) found interesting developmental differences amongst high school and university students’ PIM. Yamamoto (2013) is a large compendium of papers on user interface issues related to information management, of which Al-Omar & Cox is most germane to this book, concluding “If judged by the ability to re-find information[,] scholars’ research-related PICs [Personal Information Collections] were in many respects a failure.” However, it one cannot generalize from their sample. One would expect PIM skills and propensities to develop as a function of one’s meta-effectiveness. Kljun, Mariani & Dix (2013) present over 100 prototypes PIM tools and discuss their adoption. ¹¹http://theleanstartup.com/book
Learn your way around your R&D
182
Suppose the title of the book catches your eye and seems pertinent, so you decide to inspect it. The intention itself is meta-information. You form an idea of the problem it addresses, its thesis, its structure and so on, i.e., you pigeon-hole the book. That’s all meta-information. Odds are that you have a specific motive for inspecting this book, i.e., some prior problems or opportunities you’ve detected. If the book seems pertinent to a project of yours, you might record it in an action management system such as OmniFocus™; or you might tag¹² it with “!Consider buying” and the project name to which it applies; or you might just make a mental note of the project(s) to which it pertains. Your decisions about whether to read this book will depend partly on this project information and your preliminary assessment of how promising the book seems. You might then read the book over a week or two while unconsciously keeping track of your processing state, e.g., what parts you have and haven’t read. Your e-reader software might help you a bit by remembering the page you are on as you read. Meanwhile, you implicitly assess the book, making decisions about whether to keep reading it, when to read it, how deeply to read it, how to use it, and so on. Those are your intentions about the resource. They are guided by your assessments of the document. (We will discuss ratings of utility and potency in chapter 10). New tasks may delay your reading and cause you to forget even this simple information. It’s useful to systematically annotate our readings, i.e., to take notes in the margins, to highlight or underline text, to draw diagrams, make calculations and so on. Sometimes you need to take more detailed notes than is convenient or even possible to do in a reader app. You might want to write an extensive outline, draw a diagram, do some calculations in a spreadsheet, write a small program, or send some emails about the book. You’re better off doing this work in a special purpose program, such as OmniOutliner Pro, OmniGraffle, or LibreOffice. The resulting documents are meta-docs (meta-documents)—documents that you compose about a focal resource. chapter 12 Delve, discusses this kind of meta-info and provides tips for managing it. The Lean Startup¹³ exhorts you to change how you run entire startup projects. Before deciding whether to follow this, or another book’s momentous advice, you would reflect and might seek others’ opinions. You might find a useful book review on goodreads.com ¹⁴or elsewhere that helps you decide whether to adopt its concepts and practices and how to do so. In my case, Ian Hand, Managing Director, VentureLabs (Simon Fraser University) recommended the book to me in a meeting about my startup projects. Since then, I have encountered many other documents and videos that describe ideas from the book. You will also encounter references to this document in other resources and contexts. Related documents, episodic memories, meeting notes, web pages, and videos are also meta-information. If the book is sufficiently useful, you may also develop your own knowledge resources that utilize its ideas. For example, you might write a new white paper for your company about how it should run startup projects. More meta-information. Finally, you might identify within this book, or amongst your reflections about it, a few knowledge gems that you wish to extract and master through productive practice or deliberate performance. Each gem is a potent bit of knowledge for which you may construct one or more ¹²See “Tag entire resources” below. ¹³http://theleanstartup.com/book ¹⁴http://goodreads.com
Learn your way around your R&D
183
challenges. By practicing with those challenges, you will master the gems. For example, from The Lean Startup¹⁵, you might want to leverage the concepts of validated learning, build-measurelearn, innovation accounting, andon cord, genchi genbutsu, five whys, and split-testing of value propositions. You would create challenges that, through productive practice, would design your mind. As a result, you would not merely be able to regurgitate definitions for these terms; you would eventually perceive your projects through the conceptual lenses of these knowledge gems. Chapters 13 and 14 describe how to do this. Thus, even for a single resource one can generate, and then need to access, a large amount of meta-data. Figure 9.4 is meant to help you create order from the welter of meta-information around potent resources.
Figure 9.4 A Focal Resource and its Meta-Information
The focal resource (e.g., The Lean Startup¹⁶ book) lies at the center. Around it, you have several belts of meta-information: internal states, external documents and other products in relation to the focal resource. The motivational information, depicted in pink, consists of your personal state, goals, problems, projects, intentions, plans and assessments related to the resource. The elaborations, in ¹⁵http://theleanstartup.com/book ¹⁶http://theleanstartup.com/book
Learn your way around your R&D
184
green, consist of your thoughts about the content, structure and implications of the information. The red circles represent other resources that refer to the document or are otherwise related to it. The distance from the center symbolizes the relative amount of time that might typically elapse between you encountering the focal resource and you developing or finding the meta-information. For example, we normally develop plans about a document before outlining it. If we are to create meta-docs at all, we normally do this before creating end-products from the knowledge. The foregoing figure only reveals part of the complexity you must manage, because, of course, you process hundreds or thousands of resources a year. The categories and dimensions proposed above are not mutually exclusive nor is all the information explicit. A document (and indeed a single sentence) might be both elaborative and motivational. Moreover, you won’t explicitly think about, let alone record, all of the meta-information you generate. Some of it might remain in your head.
9.2.1 Appreciate the meta-access problem This raises a challenge to every knowledge worker’s meta-effectiveness: How do you quickly and reliably access meta-information for a given knowledge resource? I propose that within two seconds flat, you should be able to access meta-information of a resource: e.g., the summary you’ve written of a book; screenshots you’ve taken from a video; minutes of a meeting; instillers you’ve created from a workshop; the terms that you’ve indicated are an important part of a PDF document. You should be able to do this 80 percent of the time without reaching for the mouse or losing your train of thought by engaging in an elaborate search. I doubt most people have a sufficiently dependable solution to the meta-access problem. This is partly because software vendors have not aptly addressed this problem yet. More generally it’s because the problem had not been sufficiently well articulated—it’s hard to solve a problem that is not recognized as such. So, most people rely on a hodgepodge of bookmark managers, reading list managers, search engines, launchers, and schemes for organizing folders that just do not scale to the challenge of managing a large, growing, manifold collection of knowledge resources. For example, most bookmark managers and reading list apps only deal with web pages. These apps do not provide us with a way of seeing what meta-documents pertain to a focal resource. So, many are not as effective at navigating their meta-information as need to be. Many do not even have a system for annotating the dozens of resources they process per week. The meta-access problem prevents people from fully capitalizing on knowledge resources and software. Thus, it interferes with meta-effectiveness. There may be no perfect solution to the metaaccess problem. But as we shall see, some ways of doing things are better than others. The metaaccess problem is a form of a more general problem, which is organizing and accessing resources.
9.2.2 Address the meta-access problem Here are some tips for solving meta-access problems. First, you will not be able to quickly navigate information if you use the mouse or trackpad too frequently. Avoiding these devices also delays fatigue. Most computer programs have keyboard shortcuts. Make a point of learning one or two new commonly used keyboard shortcuts per week
Learn your way around your R&D
185
until you hardly use the mouse. Learn to touch-type. Second, consider using an adaptive “launcher” program such as ObDev’s LaunchBar¹⁷ (Mac) or ConceptWorld’s RecentX (Windows)¹⁸. A launcher is a program that enables you to access almost any resource that you are likely to use on your computer with very few keystrokes (normally 3 to 5). This also includes web pages. Launchers index your managed folders, i.e., the folders that contain most of the files that you typically want to access. With a configurable keyboard shortcut (such as Control-A), you launch the launcher. This brings the launcher unobtrusively to the foreground. Then you type in the name of the file you want to access. As you do so, a list of candidate files is presented to you. Navigating with arrow keys if necessary, you select the target file by hitting the ENTER/RETURN key. An adaptable launcher learns what resource matches this name. It will quickly learn abbreviations, e.g., that when you type “Pln14” , you want to access the file named “Plan for 2014.oo3”, nested deeply in one of your folders. “Pln14” becomes an abbreviation for that file. There is plenty of free information about launchers—they have more features than I’ve described. Third, to fully benefit from a launcher app and to address the meta-access problems, consider the following organizational tips. • Organize your files into a sensible folder hierarchy. Make believe that you will soon need to teach someone how to navigate your folders and how folders ought to be organized. Follow the golden rule: do unto them as you would like them to do unto you. • Give your files meaningful names and adopt conventions that facilitate accessing files. For hierarchies of folders, adopt naming conventions that enable you to rapidly find items in any subfolder. • Make a point of learning your way around your folder hierarchy. Per Perkins’ geographical metaphor: A good cook knows where each item needed for cooking is. The same goes for your files. This means you need to dedicate time to study your own hierarchy, utilize it, think about it, and tweak it. • Start small, with your current or upcoming projects. • Revisit these steps when you have a lull and before starting a new project. • Read blogs that provide high caliber organizational tips. • If in doubt, ask other users or hire a productivity consultant. Fourth, develop a system that enables you to rapidly file documents. There is software—such as Noodlesoft’s Hazel, Apple’s Automator and Ironic Software’s Leap¹⁹—that can help you with this task. David Sparks’s e-book, Paperless²⁰, provides clear explanations and screencasts about this. It’s very difficult to manually manage a meaningful, deep, purely hierarchical system for all your files. Often, you come across files that you’d like to extract but that don’t fit neatly into a hierarchical folder system. If you simply throw these files into a catch-all folder and rely on search tools to access them later, you’re likely to spend a lot of time sifting through search results—and often giving up. This is where tagging software, like Ironic Software’s Yep and Leap, comes in. With this software, ¹⁷http://www.obdev.at/products/launchbar/index.html ¹⁸http://www.conceptworld.com/RecentX/ ¹⁹http://www.ironicsoftware.com/leap/ ²⁰http://macsparky.com/paperless/
Learn your way around your R&D
186
you simply drop new files into a “managed folder”. The software will file it for you according to some convention (e.g., year, month, date). The software also enables you to apply one or more tags to the file. By applying multiple tags to a file, it is as if the file was in two different folders at the same time. (This is known as a heterarchy as opposed to a strict, tree-like, hierarchy.) Later, you can find the files by searching for any combination of tag, file name and content information, and other information. There are other types of applications that have similar capabilities. DevonThink is a prime example for the Mac.²¹ DevonThink allows you to sync content to iOS devices while maintaining file tags—an important consideration for iOS users. Here’s an example of where this comes in handy. Suppose you are writing a review of The Lean Startup²². You might have a folder for the book review. And you might have PDF files stored in a reference management application such as Mekentosj Papers2™. Papers2 stores its files in its own directory, which is separate from your project directory. You might also collect pertinent bookmarks in several web browsers. You might have mail messages on the subject. You don’t want to store copies of each of these meta-documents in your “Lean Startup Book Review” project folder. You want to be able to reuse the information in the context of different projects. So you need a logical system to relate disparate files. Because you are dealing with so many files, you need software to implement this system. Here are the most common ways of addressing the issues. 1. Using aliases, shortcuts or symbolic links. In your project folder, you might create aliases to the meta files. Aliases are robust in that when the target file changes, they tend to remain valid. (They do have their limits.) Unless you write custom software, or new technology emerges, this requires quite a bit of mousing and maintenance. 2. Naming files systematically and relying on a good desktop search engine such as Apple’s Spotlight or DevonAgent.²³ (Whether you use this technique or not, you need to be an expert user of your desktop search engine.) Example meta-data you would include in the title are: author, date, and project ID. There are several problems with this approach. (a) It’s very difficult to be systematic about how we name our files. (b) It’s often difficult to view file names properly (File managers work best with short names.) (c) So you won’t be able to add as much meta-data as you might want, and you’ll need to use abbreviations. (d) Often we do not have control over file names. For example, email messages are typically named by subject and PDF managers like Mekentosj Papers and Mendeley support only a limited number of naming schemes. (e) It requires guts, skill or time to batch rename thousands of files; this can pose problems when you want to upgrade your naming conventions or change embedded meta-information. Nevertheless, this solution works well for some meta-docs, as described in chapter 12. 3. Writing meta-information to the file’s comment field.²⁴ Effectively, this allows you to tag documents. You can prefix your tags with one of thousands of special characters at your ²¹For PDF files you could also use Mekentosj’s Papers. ²²http://theleanstartup.com/book ²³Seth Brown of Dr. Bunsen Blog suggests relying on naming files. See also Sparks (2012). Sparks uses a software application, Hazel, to move files
to their places. ²⁴On Mac OS X, this is known as the Spotlight comment field.
Learn your way around your R&D
187
disposal. For example, you might use the degree symbol. So if the ID of your “Lean Startup Book Review” project is Ries2011, you could easily tag all kinds of files pertaining to this project with Ries2011. So, if you want to find web pages that are pertinent to this project, you could search for bookmarks that contain “˚Ries2011” in their comments. Project names, and other meta-data, change, so when you change your meta-information, you may have some problems. As with file naming described above, without scripting or software support, it’s not practical to change such meta-information. 4. OS X Mavericks® and Windows 8, you can tag files²⁵. With a tagging system you could tag each file that pertains to your project with the tag Ries2011. Thereafter, you can use a tag browser (such as Leap, DevonThink or Finder on a OS X) to list documents with that tag. You could even define a more sophisticated search, e.g., to list all files of a given type (e.g., Mail) tagged with that project. To tag a file in OS X, you can use the Finder’s “Tags” menu item. This brings up a window. You can also apply a tag to a file while you are saving it. There are other applications that enable you to work with tags, such as Ironic Software’s Leap²⁶. Later you can retrieve files with the tag you applied, using the Finder (with Spotlight) or an OpenMeta app (such as Leap). To tag effectively, you need to explicitly consider your requirements. The section “Tag entire resources” describes the main types of tags I recommend you apply to focal resources: ID, topic, project, action, caliber, utility. Each category has its own syntax. Some of these tags are also pertinent to ancillary resources. I have found this tagging system that I have invented to significantly improve the usefulness of tagging. In this section, I only discuss ID’s. It is sometimes useful to tag a focal resource with an ID and to tag ancillary files with a reference to that ID. For example, it’s useful to convert web pages worth delving to PDF and read them in a powerful PDF reader such as Skim. In the process, you can copy and paste the web page’s URL into the PDF subject field. The URL is unique therefore it serves as an ID. Should you land on the web page again, you can quickly retrieve its PDF equivalent.²⁷ You could then apply this URL as a tag to any meta-resource you create. That way, you can quickly access the summary or concept map about it, for example. Not all resources have a URL. For books, you could use an ISBN or a citation key. A “citation key” is a document ID that is unique with respect to your own reference database but not globally unique. In LaTeX²⁸’s bibliography management system, BibTeX, they are called “cite keys”. For email messages, you can use the message ID. Many Mac OS X applications enable you to obtain a unique link address to resources they manage. For example, OS X’s mail has a “copy message URL” command that returns an ID for the current email message. OmniFocus and Papers, amongst others, have a similar command. Unfortunately, this is not yet provided globally by the operating system. One can work around this by generating one’s own unique ID’s. ²⁵Versions of this book published prior to December 2013 were written in terms of OpenMeta tags. OpenMeta is tagging technology maintained by Ironic Software. Apple released OS X 10.9 (Mavericks) on October 22, 2013 with file tagging capabilities. If you are a gmail user, you may already be familiar with tags because gmail allows you to label messages. (Tags are labels.). ²⁶http://www.ironicsoftware.com/leap/ ²⁷Currently, this requires software, but one could easily automate the process such that within one’s web browser, one would get a visual indication that a PDF file has been generated for the current page. One could also automate finding the target PDF with a script that can be activated with a launcher program. This enables accessing the target file quickly—satisfying the 2-second rule. ²⁸http://en.wikipedia.org/wiki/Latex
Learn your way around your R&D
188
When generating and accessing a unique ID is too inconvenient, it is often sufficient to use a nearly unique ID. One must simply keep in mind, at tag application time, whether one to identify a single (focal) resource or a collection of them. There are many trade-offs amongst the strategies and software options described above and no silver bullet to fire on the meta-access problem. I use a combination of strategies, myself and am designing software to make this a lot easier. By developing an understanding of the space of meta-access problems and solutions, you will be able to select and construct strategies that work for you.
9.3 Learn your way around your R&D projects and activities Knowledge work is no different than any other activity in that you are most productive when you are goal-directed²⁹ and when you use a reliable and valid system for getting things done. But knowledge work is also special. Its input consists of knowledge resources. At its core are demanding mental processes. At its best, it strives to efficiently produce excellent products (e.g., knowledge, documents, software, services), solutions and personal development. Thinking systematically about knowledge work and meta-effectiveness poses some challenges. Many people aren’t familiar with the concept even if they are themselves knowledge workers. The activities and processes involved in knowledge work—such as reading, writing, experimenting, debating, debiasing, designing, explaining, and practicing—are abstract, elusive and recursive. People who are not professional knowledge workers engage in them; knowledge workers also engage in routine work. The “same” knowledge work process can lead to multiple outcomes. For example, by writing one and the same document, you can solve a problem, propose new knowledge and develop new understanding. During knowledge work, it is sometimes difficult to know specifically what problems one is addressing, what goals one is pursuing and what constraints one is dealing with, not to mention the relative importance and urgency of these sometimes conflicting requirements. One can nevertheless benefit from trying to understand and improve one’s knowledge work. In this respect, knowledge work is no different from other types of expertise. If you are deliberately trying to develop yourself, knowledge and other products, you need to use a task management system that is up to this task. In this section, I briefly describe some of the requirements and challenges of task management so that you can select or devise a system that works for you. I emphasize advanced challenges related to information complexity and learning. Many of today’s R&D teams use “agile” (“lean”) processes and standards according to which they execute their projects.³⁰ They implement these processes with software, such as: • Project management software, like OmniPlan and Microsoft Project®. ²⁹Many knowledge workers are acquainted with principles of goal setting (Latham & Locke, 1991), i.e., that people are more productive when they pursue goals that are clear, specific and reasonably difficult. In Beaudoin (1994), I explored several other relevant attributes and processes of goals. In chapter 3 above, I pointed out that perceived self-efficacy is also important to cognitive performance. In this chapter, I explore several other aspects of goal processing that are pertinent to knowledge work. ³⁰The examples below are all applicable to software development. However, much of the software and principles are applicable to other kinds of knowledge work.
Learn your way around your R&D
189
• Issue-tracking systems (such as Bugzilla³¹) to help manage individual issues (problems or tasks) of finer granularity. • Revision control systems (such as Git³² and SVN³³) to provide concurrent access to multiple revisions of files and documents. There are even suites of collaboration tools teams can use, such as GitHub³⁴. For product developers, there are also team work methodologies (such as Scrum and other forms of Agile development). However, even on agile team projects, employees are typically left to their own devices for managing their individual tasks. Employers don’t necessarily provide training on “time management”, let alone on how to organize one’s learning. Some organizations provide learning management systems; but usually knowledge workers need to access important resources outside of these systems (the Internet). Organizing their work is one of the biggest challenges knowledge workers face. The next few sections are designed to help you integrate your learning with your task management systems. There are several personal management systems to choose from, which vary in rigor, detail and difficulty. One that is gaining popularity is David Allen’s Getting Things Done (GTD, cf. chapter 1). In this book, I do not mean to promote GTD or other systems. I am mainly conveying requirements and illustrating ways of achieving them. I use GTD as an example system here for several reasons, such as that it is becoming popular with knowledge workers, there are some good software implementations of it, it is fairly systematic. OmniGroup’s OmniFocus provides outstanding support for GTD. (I depend on it.) The GTD framework itself is not sufficient for managing knowledge work. I propose some adaptations. Some people are quite content to use to-do lists to manage their work. Some of us want richer task management software. In selecting a rich action-management software, consider that it should help you to • establish a clear representation of your areas of responsibility, your projects and your tasks; • express your projects and their tasks in a hierarchical fashion, similar to the way in which folders can have subfolders, and outliner documents can have rows that can recursively have subrows; • rapidly assign tasks to projects; • quickly specify and find your next actions for a given project; • assign attributes to tasks that are relevant to you³⁵, for example you might want to tag tasks according to the type of processing they entail (e.g., repairing a flaw vs. inspecting a document); • focus on a project (i.e., remove the clutter of other projects); • organize your priorities in relative terms, not necessarily quantitatively; ³¹http://www.bugzilla.org ³²http://git-scm.com ³³http://subversion.tigris.org ³⁴http://github.com ³⁵OmniFocus 2 does not support this. For example, it does not have a tagging system. However, there are kludges, such as adopting textual tagging
conventions. For example, one can enclose expressions to be treated as tags within square brackets, like this: [important].
Learn your way around your R&D
• • • •
190
express dependencies between tasks; temporarily suspend tasks; express start times and deadlines; filter your lists of projects and tasks according to multiple criteria.
In addition the system should • be easy to use. You should in particular find it easy to determine where a task is (to be) noted. For example, given a document you want to read, you should know whether to include this in your GTD app or as a note somewhere; • be flexible, such that you can classify items in different ways. For example, sometimes it makes sense to include a “to read” item on a to-do list, and another time in your GTD app; • Allow you to access any project from anywhere on your desktop within two seconds³⁶; • not take more time to manage than it saves you; • be scalable (e.g., to handle large numbers of projects, tasks and contexts); • handle your mobility needs (e.g., sync to iOS devices); • give you some peace of mind that nothing is falling through the cracks; OmniFocus meets most of these criteria. It enables you to make an inventory of the areas of responsibility in your life and to express projects within each of these areas. Within projects, you can enter and manage individual tasks. It provides a widget to quickly enter (“capture”) new tasks as they come to your mind, from within any application. The widget enables you to quickly assign a project and a context (a kind of hierarchical tag) to a task, as well as other parameters. OmniFocus has advanced features for reviewing and managing your projects. Its Perspectives feature helps you focus on particular projects. You can launch uncluttered windows that are focused directly on a given project. As you check tasks off the project list, they can disappear from your view. ³⁶This can be accomplished with OmniFocus 1 and an AppleScript described in this post.
191
Learn your way around your R&D
Figure 9.5 Sample Areas of Responsibility
Learn your way around your R&D
192
It takes a few weeks to get the hang of OmniFocus.³⁷ It requires that you be willing to learn and practice new ways of thinking about your work. So don’t try this when an important major deadline is hanging over your head. You don’t need to follow GTD principles religiously to benefit from this application. But if you want to benefit deeply from this tool, you do need to know the concepts and to use it systematically. With OmniFocus, you create an entire project to reflect your goal of learning something. For example, you might create a project to learn a certain programming language. This project could have all kinds of child tasks. You can also assign learning tasks as components of a given project which is not primarily about learning. A learning task can be a means to an end (compare the section Classify your R&D tasks below.)
9.3.1 Identify your projects To manage R&D tasks, one relies on other software as well. For example, while working on a knowledge-intensive project one typically generates many project-related documents (e.g., drafts, project specification, ancillary diagrams and spreadsheets). Task management systems (such as OmniFocus) are not repositories for resources pertinent to projects. If you use a task manager, consider organizing your project folders in a manner that reflects your task manager’s project hierarchy, or is at least coherent with it. For example, if you are writing a review of this book, you might create a folder (e.g., named “Cognitive Productivity review”) to contain resources pertinent to your review. It might include an outline, figures, various ancillary documents as well as the draft itself. You would also have a project in OmniFocus named “Cognitive Productivity book review”. In addition to the documents you create for a project, you will also often obtain or locate pertinent third-party documents. Since you might use these documents in different projects, you might not want to include them in your project folders. So, you might want to tag them with the name or the ID of the project. For example, while reviewing this book, you might find some other reviews of my book on the web, as well as related books. You could tag them all with the ID of your “Cognitive Productivity book review” project. You could also use the tagging system to tag files that are on your computer but in different folders. (See Section “Address the meta-access problem”.) If you are working collaboratively, you probably need to refer to your projects in other software, such as a project planning system or an issue (ticket) processing system (e.g., Bugzilla). In this case, you’ll probably refer to issue numbers in your tasks, and you’ll want a standard way to do this. If you bill for your time, or simply like to know where your time goes, then you might also apply project names or IDs to job events in your time-tracking software. So you might face a situation where you need to organize and refer to your projects in several different contexts and for different purposes: (a) personal task management; (b) organizing your own documents; (c) referring to, storing or tagging third-party documents; (d) using collaborative software such as problem-tracking system; (e) time-tracking; (f) billing. To deal with this complexity, you need a way of organizing and referring to your projects that is coherent, simple and effective. Here are some tricks for coherently representing project information across different activities and files. ³⁷http://www.omnigroup.com/blog/omnifocus-2-for-mac-available-now
Learn your way around your R&D
193
First, select a particular repository as containing the canonical representation of your project. For example, you might choose an OmniFocus project or a folder as being authoritative. Second, leverage the containment hierarchy of this repository. Think of it as having a qualified path structure similar to file systems (compare the concept of path name³⁸ in file systems). You’ll apply this path in Step 4. Third, without seeking perfection, and to the extent that it is under your control, organize your other repositories so that they reflect this project structure. For example, organize your project folders on your file system to reflect your GTD software’s project folders. That way, the time you spend navigating in one repository may help you subsequently navigate in the other repository. Fourth, assign globally unique project IDs to your projects. Associate those IDs across repositories (e.g., in OmniFocus and on your file system). For example, you could include the project ID in the name of the folders that deal with the project (in OmniFocus, your file system, and elsewhere). Or you could tag the folder with that ID. You could additionally use that ID as a launcher abbreviation to open the folder. (You could even have a collection of launcher abbreviations that have a different prefix or suffix and the same ID; there could be one unique abbreviation for each type of resource: OmniFocus, mail, etc.) Fifth, define a TextExpander abbreviation which expands the project ID to the full project name. These global IDs will allow you to access different project resources. So, if you are in OmniFocus and can’t recall where you’ve stored your project folder, you can use spotlight to find the relevant folder. Consider, when appropriate, sculpting IDs in a way that reflects the hierarchical structure of your projects. If a project with ID Foo has a child with Barbaz, you might select FooBar as its ID. For example, I have project ID’s for chapters of this book. The code for this chapter is “/cz/bk/Part 3/31 ˚bk31”. The first part of that, “CogZest/Book/Part 3/Chapter 1”, is the path. (That’s similar to the path in a web address or file system.) The second part is a unique ID ˚bk31. This reflects the structure of my OmniFocus project and my file system. I have defined a TextExpander abbreviation, bk31, which expands to the entire tag. When I encounter a new PDF file, web page or other resource that is pertinent to this chapter, I tag it with “/cz/bk/Part 3/31 ˚bk31”. Using TextExpander, I only need to type in bk31 to get that tag. In Ironic Software’s Leap, I can search for files pertinent to “/cz/bk” which means this book, and further narrow them down to part 3, “/cz/bk/Part 3” and then to this chapter. This system works relatively well despite the fact that Mavericks Tags do not provide special support for hierarchical tags (neither does OpenMeta.) Leap incrementally filters tags as one types. It functions implicitly. This is explained in chapter 12 below. This project information can be re-used with activity monitoring software and a spreadsheet. When I switch tasks, I create a new entry in my time-tracking spreadsheet.³⁹ One cell captures the project as a string, another captures the activity, and several others capture other parameters. Again, I use my TextExpander shortcuts to express the project in a hierarchical way with a unique ID. That way, I can track how much time each chapter and section has cost. I also classify these events with the type of task in this project (e.g., writing as opposed to reading), as described in the next section. This ³⁸http://en.wikipedia.org/wiki/Path_name ³⁹I’ve tried several time tracking applications but none of them is sufficiently powerful and flexible for me. By keeping information in a spreadsheet,
one has rapid access to all spreadsheet functions, without needing to depend on the time tracking app’s typically limited export capabilities. You can add columns and tabs to suit your needs.
Learn your way around your R&D
194
technology makes activity-tracking quite easy and rapid. It would be even better if GTD software were directly integrated with time-tracking software. I’m not suggesting you should always track your time in detail. But occasionally doing so can help you get a more realistic sense of how you are spending your time. Without time-tracking tools, people are inaccurate time trackers.⁴⁰ In sum, I believe that in order for researchers and developers to make optimal sense of their work, which is often very fluid, they need to have a coherent way of referring to and thinking about their projects and activities. I’ve argued that hierarchical task management systems like OmniFocus can be used in combination with other software to develop, apply and maintain such an understanding.
9.3.2 Classify your R&D tasks Users of GTD software (like OmniFocus) can assign a “context” to their tasks. The context is meant to be the setting in which an activity can be performed. To narrow the search of relevant actions to perform now, users can filter their tasks by context. So if they are at their office, only their office tasks would appear. When with a particular person, they can refer to the actions whose context is that person. And so on for all kinds of contexts, e.g., the garage, a particular conference, a grocery store, the cottage, home office, whatever. The original concept of context suggests that GTD was developed as a general task management as opposed to one specifically tailored for knowledge-intensive work. This is not a criticism of GTD— universality is one of its strengths. Many of the examples in the original GTD book refer to handson tasks like filing physical papers and cleaning one’s garage. So, as a bit of web searching reveals, many GTD application users, particularly mobile workers whose virtual offices are always available to them, have struggled with the concept of context. My solution is to “overload” the context field in OmniFocus.⁴¹ Where it is relevant, it refers to the traditional GTD context. But it now can also refer to type of activity. Many examples of GTD contexts on the net are of that nature. What was lacking was a systematic way of organizing their knowledge-intensive activities. The two following figures illustrate a taxonomy of knowledge tasks. There’s an R&D “context” which includes surfing, processing and developing. The processing “context” is for activities in which one processes knowledge resources; e.g., reading, listening, watching and attending (seminar, workshop, podcast, TED, screencast, book, paper, article, etc.) Each “context” has sub-contexts. For instance, the “Process” context has three sub-contexts, corresponding to levels of processing information: inspecting, delving and capturing gems. Rather than merely specify that you aim to “read” a certain document, you can specify that you aim to inspect or delve the document, or extract knowledge gems from it, master it or develop products with it. ⁴⁰See for example Robinson, Martin, Glorieux, & Minnen (2011). ⁴¹This use of the term “overload” comes from software engineering. It is to use the same information structure for different purposes depending
on context. It would be possible for GTD software developers to enable users to classify activities without using the “context” field. Furthermore, they could guess the type of activity from the verb that the user writes in the task description. For example, if I write “Outline chapter 3” as my task description, then OmniFocus could detect that this is an outline task and classify it accordingly, while allowing the user to override its potentially erroneous attempts.
Learn your way around your R&D
Figure 9.6 Example Development Activities as OmniFocus Contexts
195
Learn your way around your R&D
196
Figure 9.7 R&D activities (surf, process, develop) as OmniFocus contexts
The develop context reflects an assumption that it is useful to distinguish between developing solutions, products, and mindware. The products context reflects an assumption that knowledge (e.g., an idea) is a product, as are abstract artifacts such as documents, code, diagrams, etc. They are all things that can be created, developed, edited, criticized, shared. Some of them can be bought and sold. (See “Objective knowledge (World 3), virtual machines (World 2’) and the rest (World 1)” above and Bereiter, 2002a). One noteworthy context in this example is to repair flaws (including bridging knowledge gaps).⁴² ⁴²I use the symbol “!?” as a shorthand for “Repair knowledge flaw”.
197
Learn your way around your R&D
Recall that in Part 2 (chapter 7) we saw that good learners are more likely to detect and repair their knowledge flaws. You can use OmniFocus to flag tasks as being to “Repair flaw”. OmniFocus’s context input field uses an intelligent autocomplete algorithm that makes specifying contexts very rapid even without a text expansion utility such as TextExpander. So, it only takes a second to designate a task as a repair task (or one of the other “contexts”). Figure 9.8 illustrates using OmniFocus to create a task to repair a knowledge gap pertaining to a reading list. It raises the question “How can I optimally manage my reading (or more generally, information processing) lists?” Notice the context “!? Repair flaw/knowledge gap” matches an R&D context listed in the previous figure. After adding this task, when you select the “!? Repair flaw/knowledge gap” context, you see all the tasks pertaining to “!? Repair flaw/knowledge gap”, including this one. In terms of the analysis presented in chapter 6, understanding is not knowledge or a product, but a relation; the hierarchy of contexts does not reflect this very well. When you create tasks in OmniFocus, you don’t normally see the nesting of contexts (though you can).
Figure 9.8 OmniFocus Task Capture
Once you have set the activity types of your tasks, you can filter tasks by type. For example, you can quickly find your “repair knowledge flaw” task across projects or regarding a specific project. This can help you make optimal use of thinking time while commuting, exercising or running errands. I’m not suggesting that people should always drive their cognition with GTD. One can imagine a voice-driven application that speaks the next cognitive task. You could ask it, “What problem is there to think about on my book review project?” It would return the next knowledge gap to fix in your book review project. This is an example of mobile cognitive-productivity. When it’s your scheduled study time, or you are simply in the mood to read (or more generally to process) knowledge resources, you can turn to your tasks whose contexts is “delve”, optionally as filtered by project. GTD apps are not a panacea for cognitive productivity. OmniFocus, for instance, is not a readinglist manager. Moreover, it is currently not feasible to use a single application, even OmniFocus, to manage all of your electronically recorded tasks. You can use multiple applications to track your cognitive tasks, and OmniFocus is one of them. For example, sometimes it suffices to add tasks as margin notes in PDF files. A taxonomy of cognitive activities can also be used in time-tracking. In the previous section, I mentioned that you can apply hierarchical project descriptors (such as OmniFocus allows you to define) to job events in a spreadsheet.⁴³ You can also apply task information. You can store ⁴³If one forgets to complete or add entries, one can use activity monitoring software, such as John Nye’s TrackTime, to reconstruct one’s activities. This software graphically presents the applications that were in the foreground and also keeps track of which files were active or URLs were visited.
Learn your way around your R&D
198
textual representations of commonly executed tasks as text-expansion snippets (in software like TextExpander). You can then apply these descriptors to task events. For example, you could map the abbreviation “>out” to “> /R&D/write/outline”. Then, whenever you switch to an outlining task, you can create a new job event containing this descriptor. For example, when outlining this chapter, I used the following event descriptor: /cz/bk/Part 3/31 ˚bk31 > /R&D/write/outline This allows you to quantify time spent on different types of cognitive tasks. This can allow you to review how you have spent your time. It can give you a better sense of how much time you spend reading email, using social media, surfing, inspecting and delving, for example. Unfortunately, commercial time analysis software is not yet available for this, so you need to develop your own spreadsheets. Admittedly, relatively few people are willing to do this with current technology. However, given that time is the denominator of productivity, and that people are very inaccurate estimators of their own past cognitive time (Robinson, Martin, Glorieux & Minnen, 2011), there is a need for more sophisticated time-tracking tools than are currently available. I suppose in the future there will be ubiquitous time-tracking systems to make this rather effortless. Until then, even occasionally sampling and analyzing your time is likely to give you these benefits. My objective is not to convince you that you should adopt the particular taxonomy of R&D tasks described here. Everyone’s cognitive activities and needs are different. Moreover, one’s taxonomies change over time. What matters is to develop an understanding of your R&D. This is a way of abiding by the general principle that in order to develop effectiveness in a domain, one needs to think about and enhance all of its major constitutive activities.⁴⁴ The concepts and technology described here can help you to do that. ⁴⁴The value of this, too, is consistent with the general principle that to improve productivity one needs to measure and analyze core business activities (Gerber, 1995).
10. Inspect Researchers spend a great deal of time reading research papers. However, this skill is rarely taught, leading to much wasted effort. Srinivasan Keshav Two of the most critical things we do with knowledge resources come so naturally to us that we don’t give them much thought. We inspect them and we assess them. From the moment we become aware of any resource, whether it be in World 1 or 3, we start to form an impression about it, we classify it, we become sensitive to its affordances, and we try to decide what to do next with it. As we deepen our relationship with a resource, we may adjust our assessments and our judgments of confidence in our assessments—sometimes gradually, sometimes radically. There is no universal, optimal process for reading in particular or processing knowledge resources in general. It is nonetheless important, even for experts, to revisit their information processing concepts, particularly if they aim to be even more productive. “Inspection” refers to the first part of the work we do with a knowledge resource as we become familiar with its innards. This might involve skimming it, sampling it here and there, or even processing it straight through a first time. This can be done both systematically and opportunistically. The concepts and outcomes of inspection determine, amongst other things, whether, how, and for what purposes one will pursue the resource further. Here is a sample of noteworthy types of inspectional determinations you might make, many of which are of the pigeon-holing, or classification, variety. • The type of relevant knowledge conveyed by the resource. Does it convey useful practical information (how to accomplish something), factual information, or normative information (an opinion about what is right or wrong)? Each one of those categories is very complex. The factual category is the most complex one; even many scientists misunderstand what factual knowledge is about. (See chapter 2¹ of Sloman, 1978.) A resource may contribute to more than one type of knowledge. • The genre of the resource: e.g., a literature review or experimental report. • The particular fields and topic areas to which it pertains. • The problems or questions addressed by the resource. Knowledge resources are products. Products must address problems and needs. The concept of problem is fundamental to the growth of knowledge. Getting at the target problem(s) is sometimes itself problematic. Some resources are topic-centered rather than problem-centered. • The projects, problems and other motivators of yours that this resource might address or be relevant to. Here you answer, “Why would I bother processing this resource?” Perhaps the resource addresses limitations or flaws in your knowledge or understanding. How would this ¹http://www.cs.bham.ac.uk/research/projects/cogaff/crp/chap2.html
Inspect
• • • • •
200 resource make you more effective? The resource might address a problem already noted in a project or objective of yours (per the above bullet), or it might reveal new flaws in your knowledge, which you may or may not respond to by setting a project to repair the flaws. The section on potency, below, expands upon this. The following chapter breaks this down into assessments of usefulness and potency. The section on usefulness, in the next chapter, expands upon this. How promising you judge the resource to be with respect to the previous two bullets. This is the usefulness. How appealing it seems to be, with respect to your attitudes and preferences. The semantic content of the resource. In other words, the thesis, the major proposition, concepts and the argument. Chapter 12, expands on this. The structure, complexity and length of the resource. The expected actions and effort required to achieve desired outcomes from this resource.
Expert readers can quickly and reliably characterize a resource along these lines. Often, the judgments at inspection are final: we reject the resource. Or we may decide that at least some parts of it may be worth delving. We refine our judgments as we process and reflect upon the resource. We might discover months later (e.g., while problem solving) that we mis-categorized the resource in some way or that it proved to be much more valuable than we expected. Sometimes, we don’t even realize that a book has impacted us. Aaron Sloman told me in the early 1990s that many who read Marvin Minsky’s, The Society of Mind, conclude the ideas are of little consequence; they then re-invent the ideas or have deep insights based on them! I’ve caught myself leveraging it more than once. The rest of this chapter deals with assessments that can be made during or (mainly) after inspection. From the moment you first access a resource, you may want to start tagging it with project information and preliminary assessments. You can revise your tags as you delve into the resource. After discussing evaluation and other concepts that are useful for tagging productively, I further explain tagging in chapter 12. The following chapter discusses our evaluation of knowledge resources. It extends and deepens the evaluations alluded to above. I treat evaluation in its own chapter because we start to assess a resource as soon as we become acquainted with it. The bulk of our evaluations occur while we delve the resource and try to develop with it. If we become able to reliably assess knowledge resources at inspection, it is because we have spent thousands of hours carefully assessing resources—i.e., while delving them and developing with them. That is why this chapter on inspection is so short: we learn how to inspect by evaluating, delving and developing. The latter each have their own chapters in this book.
11. Assess Hence, in regard to reading, it is a very important thing to be able to refrain. Skill in doing so consists in not taking into one’s hands any book merely because at the time it happens to be extensively read; such as political or religious pamphlets, novels, poetry, and the like, which make a noise, and may even attain to several editions in the first and last year of their existence. Consider, rather, that the man who writes for fools is always sure of a large audience. […] You can never read bad literature too little, nor good literature too much. Bad books are intellectual poison; they destroy the mind. Because people always read what is new instead of the best of all ages, writers remain in the narrow circle of the ideas which happen to prevail in their time; and so the period sinks deeper and deeper into its own mire. Arthur Schopenhauer ¹ Why bother to take time out to think about the process of evaluating knowledge resources? Given the number of knowledge resources to choose from, in order judiciously to decide what knowledge resource to process, you need to be a connoisseur of information. You need to be able to reliably and validly² gauge whether, when and how to process a document. The first item in the foregoing list stands for the gatekeeper function of inspection: to protect one’s most precious resources, i.e., one’s attention and one’s time, from unworthy distraction. The second of three principles that Mike Markkula ³ wrote to set the direction of Apple Computers in 1977 expresses this well: “Focus. In order to do a good job of those things that we decide to do, we must eliminate all of the unimportant opportunities.” (Isaacson, 2011) In order to apply this principle one needs to develop an ability to detect and select worthy opportunities and to reject the rest. Markkula did not specify what it means for an opportunity to be “unimportant” and how can we determine it. The term “important” points to myriad concepts. This chapter aims to make sense of them. As an undergraduate, I observed myself and others saying of some documents, presentations, courses and people, that they were “interesting”. The expression “interesting” could be a starting point for a discussion. It gravelled me, however, that it was often the end point as well. People would move on to the next thing. I’d imagine, in Monty Pythonesque fashion, the audience chitchatting on their way out from a presentation by Einstein on relativity. “That was interesting.” “Yes, quite nice.” “What do you make of the haircut in the third row?” Supposing one does detect that a contribution to knowledge is interesting, what makes it so? What can make it so? Whenever one develops authentic expertise, one becomes a better assessor, at least in the expert domain. Imagine a great architect walking through a particular building for the first time. He doesn’t merely conclude that the building is “good” or “bad”, or rate it on a scale of 1 to 5. He can make detailed observations and relate them to manifold aspects of the building. After watching two drives ¹http://www.egs.edu/library/arthur-schopenhauer/articles/the-essays-of-arthur-schopenhauer-religion-a-dialogue-etc/on-books-and-reading/ ²Reliability and validity are important concepts in psychological measurement. See for example Anastasi(1988). ³http://en.wikipedia.org/wiki/Mike_Markkula
Assess
202
of a football game, an expert football coach will know more about both teams than an armchair quarterback would after the entire game, even with the benefit of rewinding the digital video recording. Within a few minutes of attending an orchestra’s performance, a professional classical musician will detect myriad strengths and weaknesses that I cannot perceive, let alone describe. The ability to make expert judgments does not appear over night. Like other competencies, it develops through years of laborious experience. It is not a single “skill”, but a collection of complex abilities implemented in mindware. Experts build up untold monitors that detect signs of promise or problems in documents. They increasingly rely on these rapid monitors. Expert quarterbacks, musicians, and architects do not (normally) serially run through checklists to assess. Their monitors assess in parallel. Effectant minds convert serial processes into parallel ones—a process that requires effort. Moreover, as I alluded to in chapter 2, using quotes from Mark Guadagnoli about Tiger Woods, experts often go back to the basics, to rework something that is already functioning quite well, to make it even better. This goes for our ability to assess, too. We certainly don’t use templates and check-lists on a routine basis, but I believe that it doesn’t hurt to revisit how we assess and characterize. At their best, experts diligently and zestfully assess. In contrast, when someone slips into autopilot, he slides off the rails of expertise onto the track of reductive expertise. Ask any good clinical psychologist: assessment is hard work. An authentic expert remains on the look-out for surprises, in our case knowledge gems. I recently heard a famous Québec actor, interviewed on Tout le monde en parle (c. April 2012) complain about the manner in which some film directors assess actors: having them play some banal part that can’t possibly bring out the actor’s strengths and weaknesses. They hire the wrong actor for the job and the film suffers as a result. I assume that anyone who has made, or experienced the sequelae of, a wrong hire knows what I am getting at. Using sub-optimal knowledge resources stunts our development. We assess and characterize resources not just to determine whether to use them, but in order to use them better. By coherently articulating the strengths and weaknesses of documents and other resources, rather than relying on impressionistic responses, we can better understand them, zero in on the gems they contain, and utilize them in our own development. I suspect that being more reflective about assessments encourages our minds to develop new monitors for other aspects of knowledge processing, such as detecting knowledge gaps, the depth of our processing, the type of resource we are dealing with, and so on. My claim does not rely on the prospect of far transfer. It is merely that if we consciously monitor for such meta-information as part of assessing and characterizing knowledge resources, we will develop mechanisms to do it automatically, which means new mechanisms will grow inside our minds. There is a possible counter-argument to consider. Carl Bereiter argues that people don’t need to be taught critical reasoning skills because they are already quite good at it. To be sure, the population to which this book is addressed, i.e., knowledge workers, is already quite capable of criticizing knowledge resources. This chapter, however, is not an attempt to directly teach skills or processes (nor is this book). It specifies concepts that you can use, adapt or even replace in your assessment, selection and use of resources. It assumes that intuitive abilities, including impressionistic knowledge about knowledge resources, benefit from occasional analysis, reflection and elaboration on the processes themselves.
203
Assess
Of course, the assessment of knowledge resources is critical to the development of objective knowledge and other products. Many of our knowledge building goals come from detecting strong knowledge to build upon and flawed knowledge to correct. Assessment is integral to the workings of effectance; it makes knowledge building fun. Understanding assessment itself poses fascinating theoretical problems, e.g., to consider the mechanisms underlying assessment and how one might profitably assess knowledge resources. The mindware involved in assessing knowledge resources are at the intersection of classical cognition and affect, and cannot be understood without considering affect itself. Look at a dozen or more book reviews and you can tell that there are many ways to assess resources. They vary as a function of the kind of resource, the kind of knowledge and other factors. As a schema activation exercise, I recommend that, before reading on, you construct your own schema for evaluating resources. Then revise it in the light of reading several reviews on a site like goodreads.com⁴ (for reviews from the general public) or a journal with open peer commentary⁵, such as Behavioral and Brain Sciences⁶ (for scholarly reviews). Finally, rework your schema.
11.1 About Assessment So you see, Gentlemen, while pleading before you that Reading is an Art— that its best purpose is not to accumulate Knowledge but to produce, to educate, such-and-such a man—that ‘tis folly to bite off more than you can assimilate—and that with it, as with every other art, the difficulty and the discipline lie in selecting out of vast material, what is fit, fine, applicable—I have the great Francis Bacon himself towering behind my shoulder for patron. Sir Arthur Quiller-Couch Before I provide a tetradic schema for characterizing knowledge resources, let’s consider some general issues about assessment. The first thing to note is that making sense of how we can and ought to assess knowledge resources is a very difficult problem. Schemes for rating resources vary in complexity. At one extreme, services like Facebook enable you to “like” a page. That is a binary rating. (Reddit lets users “upvote” and “downvote” posts.) Several applications for managing resources—such as Mekentosj Papers, Apple’s iBooks version 2.1.1, and Ironic Software Yep version 3— enable readers to rate resources with a 5-star system. This is not altogether useless; but these systems probably won’t make you more of a connoisseur nor provide you with much help in managing your reading (or knowledge processing), unless you routinely back the ratings up with a more detailed assessment. At another extreme, you could in principle rate a book along dozens of attributes. No one has time for such detailed explicit judgments. With respect to assessment, virtue lies, as Aristotle put it of activities in general, “in a mean between the two extremes of excess and defect”. As an example of a middle ground, as of the time of this writing, Wikipedia invites its readers to rate articles along the following ordinal scales: trustworthiness of sources, ⁴http://goodreads.com ⁵http://en.wikipedia.org/wiki/Open_peer_commentary ⁶http://https://journals.cambridge.org/action/displayJournal?jid=BBS
Assess
204
objectivity, completeness and comprehensibility. For each scale, and each value of each scale, Wikipedia provides a description, to help you use their criteria. Pre- and post-graduate texts on scholarly reading provide a slightly more extensive template than Wikipedia (Wallace & Wray, 2006); but they are quite focused on basic student concerns (passing exams and writing papers and theses) as opposed to the more demanding and varied reading tasks of lifelong learners. We can’t simply import the foregoing criteria for assessing resources in general. There are various types of knowledge resources—e.g., empirical studies, various types of design (experiments, surveys), software designs, standards (e.g., RFCs), processes (e.g., ISO), patents, law, editorials. Each has its own special criteria and conventions in addition to general criteria. How you ought to assess a resource depends on the type of resource, and other factors, such as your project, urgency, busyness, and so on. The importance and meaning of completeness, for example, is not the same for judging an encyclopedia article as it is for a literature review or an article in Science magazine. In contrast to Ph.D. theses, scientific reports, editorials or autobiographies, encyclopedia articles ought not to include original content and must be written from a neutral point of view. Still, you may well make a mental note of whether the resource you are processing is original and neutral. Thus, this chapter is not meant to provide a universal evaluation scheme but to propose concepts you may use and adapt to suit, or at least activate, your needs. While using numbers to assess and compare resources is sometimes useful, assessments can not be reduced to a scalar, nor to a collection of scalars.⁷ Normally, a huge part of your judgment is qualitative. You might find for example that an argument is significantly flawed or does not address your concerns, and reject the resource immediately on that basis. No rating is required or ultimately specified. This echoes debates in cognitive science between those who think that the mind in general and assessment in particular can be understood in terms of number crunching (e.g., dynamic systems, neural networks, etc.) and those who believe that mental processes operate on structures and in qualitative spaces with multiple discontinuities.⁸ I’ve mentioned this in passing but it is worth making the point explicitly that often assessments remain, or ought to remain, provisional. They start even before inspection, which is what leads us to inspect a resource in the first place, and may be changed during inspection, during and after delving. Thin-slicing is to make rapid initial judgments. (Cognitive scientists speak in terms of heuristic judgments.) The better we are at judging how promising a resource is, the better we are at selecting and pursuing the most “worthy” resources. It is by pursuing the most worthy resources that we have the best shot at capitalizing on the knowledge work of others. This pursuit is critical whether you are in a competitive environment or simply want to optimize your R&D. Prospecting is therefore not just a trope; it’s a core skill in R&D. Routinely comparing our initial assessments with our later assessments can help us improve our ability to detect knowledge gems. This in turn requires that we calibrate our confidence in our judgments. The ability to detect promisingness is part of what Bereiter calls “impressionistic knowledge”. It also falls under the rubric of “attitudes” (Ortony, Clore, & Collins, 1988). To name this is not to ⁷See chapter 6 of Beaudoin (1994). ⁸For example, “We should not presuppose a sharp, clear, boundary between non-biological and biological structures and processes: seeking
dichotomies and precise definitions often obstructs science. The alternative to a dichotomy is not necessarily a continuum, with fuzzy transitions: it is also possible to have a very large collection of discontinuities, big and small, including branching and merging discontinuities.” (Sloman, 2012a p. 53). Egan (1997) defends binary thinking in education.
Assess
205
explain it, however. When we assess documents, we are in fact making value judgments about them. But what does that mean? Ortony, Clore and Collins presented a compelling taxonomy of valenced reactions that helps answer this question. They proposed that there are three major types of value: standards, goals and attitudes. Standards concern what is right and wrong. Here, we assess consequences of behavior in terms of their praiseworthiness. Goals concern what we want to achieve, prevent or otherwise control. Here, we assess actions in terms of their desirability (below, I write in terms of “usefulness”). Attitudes concern our likes and dislikes. Here we assess things in terms of their appeal. Our valenced reactions to knowledge resources vary along these three attribute clusters. For example, we might find an author’s production of a theory to be praiseworthy because it is more general and parsimonious than its competitors. We might find a document to be desirable because it criticizes an algorithm we have been relying upon—we need to know if it is unreliable. We might find a video to be appealing because it makes original use of concepts that we find fascinating. Sometimes attitudinal judgments (like vs. dislike) are sufficient to rule in or rule out a document, other times not. Ortony and colleagues argued that the cognitive structure of our emotional responses could also be understood within their framework. (These “emotional responses”⁹ can be leveraged in our R&D, e.g., to help us remember what we read; but I will defer this to future publications.) There is not a sharp distinction between assessment and description. “[T]he familiar contrast of “normative or evaluative” as opposed to the factual is in need, like so many dichotomies, of elimination” (Austin, 1956, p. 148). The same characterization of a knowledge resource may be interpreted in an evaluative or factual manner. For example, to say that a hypothesis is inconsistent with a theory may simply be a matter of fact, or it may in addition be a criticism of one or the other conceptual artifact. This is another reason why we cannot be expected to hold our judgment of a document until we have fully understood it: judging and understanding are indistinguishable. So one should not read too much into my treating assessment, delving and practice in separate chapters, nor on the title of this chapter being Assess rather than Characterize. There are some subtle complications in evaluating information. One is: are we responding to the knowledge resource or to the knowledge pointed to by the resource? And to what ought we respond? For knowledge workers, these are not rarefied issues. Above, I drew an analogy between circumambulating vs. gardening and inspecting vs. delving. Similarly, we need to separate our assessments of the surface and the essence of a resource. Suppose an author does a poor job of describing or using a theory. Ideally, our judgment of the theory will not be impacted by the document about it. It is not always clear whether an author is extending a theory (producing new knowledge) or whether what he is describing is implicit in the theory itself: in either case, we are dealing not just with a knowledge resource but knowledge itself. A theory might have multiple possible yet inconsistent interpretations or extensions. This issue is not limited to science, but to any type of knowledge. Jurists and their readers face the same conundrum. Another complication is that the criteria we use in our assessment in every day life, as in R&D, are not all orthogonal. The tetradic schema in the meta-effectiveness framework includes caliber,¹⁰ usefulness, potency and appeal. Each of these clusters of attributes has some connection to the ⁹I use scare quotes here because, as shown in chapter 7, there are several distinct concepts designated by the term “emotion”. Their concept does not fit perfectly with the technical concepts described in that chapter. ¹⁰One could say that high caliber documents are praiseworthy, in Ortony’s terminology.
206
Assess
others. The contributors to potency and caliber can make a document more useful, for example. Still, separating these attributes will inform our judgments as a function of our purposes. Moreover, documents can also vary independently between these attributes. For example, a document might be of high caliber, but useless and impotent for one person, while only being slightly useful but quite potent for another person. One might find a document to be appealing because it is “fascinating”, yet this fascination might be a response to its potency. A document might even propose a seriously flawed theory or design and yet be very useful — we can learn from the errors of others. (Tastelessly designed software has been known to inspire great work. Practical books are filled with rules about what not to do.)¹¹ Even to the extent that the clusters of evaluative attributes I propose are somewhat logically independent, keep in mind that we are not fully aware of our own evaluations. Thus, one can think one is accepting or rejecting a document on one basis (e.g., its caliber) while one is actually responding to a different attribute (e.g., appeal).
11.2 CUPA: Caliber, utility, potency and appeal Whether it’s Beaujolais or business principles, orcas or orchids, seeking out the best books ever written on your subjects holds the keys to your kingdoms. Steve Leveen Wouldn’t it be wonderful if, whenever you were faced with a knowledge resource, you could touch a button to receive a complete and accurate assessment of the objective and subjective value of the resource from a superbly intelligent machine (SIM)? It would characterize its caliber according to objective standards (or norms). You would learn whether it was true, coherent, well described, and so on. The objective worth of the document would not be enough for you to fully gauge its value. There are plenty of great documents that are of no use to you. So SIM would then give you a subjective assessment, i.e., one based on SIM’s intimate understanding of you. It would tell you about the resource’s usefulness, i.e., how processing it might advance or hinder your particular projects, goals, concerns, preoccupations, and predilections — in short, your motivators. As we saw earlier, something can be very useful but not particular appealing. Think of Buckley®’s, an awful-tasting cough syrup whose manufacturer claims is very effective. Other things being equal, why not choose the more appealing document and ask SIM for an input on that? For completeness, the SIM would provide you with potency descriptors indicating the manner or “extent” to which the knowledge resource might change—hopefully, improve—your understanding of the world, your skills, your habits, and so forth. That would tell you how you yourself could develop as a result of processing this document. By fleshing out this thought experiment we can get a better understanding of our assessment of knowledge resources. This is the kind of thought experiment that could drive AI research on the subject of this chapter. There is the issue of naming clusters of criteria. It’s a useful exercise in conceptual analysis to try to explore and attempt to select the most apposite terms for the concepts one is developing. I’ve chosen to draw from the English lexicon some terms that have a suitable connotation and ¹¹For example, Johnson (2007), Bloch, (2001), and Mullet & Sano (1995).
207
Assess
denotation (chiefly, “caliber”, “usefulness”, “potency” and “appeal”). I have assigned to them some technical meanings that depart from established usage. You might prefer to substitute different terms for these four concepts. If you need help remembering these criteria, think of it as a good “cup’a” (“cup of”) criteria. CUPA is its acronym. “Cup’a” is an analogical mnemonic.¹²,¹³
11.3 C: Gauge its caliber Doubt is cast on the value of a theory and its concepts if the theory does not enhance our practical abilities, if it doesn’t explain a variety of observed facts better than alternative theories, if all its predictions are very vague, if it never generates new research questions that lead to new discoveries of things that need to be explained, if its implications are restricted to very rare situations, and if it cannot be used in making predictions, or selecting courses of action to achieve practical goals, or in designing and steadily improving useful kinds of machinery, In such cases, the concepts implicitly defined by the theory will be limited to reference within the hypothetical world postulated by the theory. Concepts like “angel” and “fairy” are examples of such referentially unsuccessful concepts, though they be used to present myths of various sorts, providing entertainment and, in some cases, social coercion. Aaron Sloman As you assess a knowledge resource against objective standards, norms or criteria—which may be qualitative or quantitative—you are determining its caliber. Caliber, like usefulness, potency and appeal is not a single criterion but a cluster of assessments. Not all types of knowledge and resources are assessed according to the same standards. For example, the objective criteria we use to assess a taxonomy are not the same as those we use to assess an experimental report. But some criteria are quite general (e.g., parsimony). In this section, I will describe three sets of objective standards: • Rhetorical standards. • General epistemic criteria. • Criteria for assessing theories to explain known possibilities. The first two sets are fairly general, while the third set applies more specifically to explanations. I could have elaborated on other kinds of knowledge instead of explanations (e.g., software designs, ontologies, practical documents, normative knowledge). I chose explanations because they are an important type of knowledge that we all frequently process and yet whose explicit criteria are often mischaracterized, even in scientific circles. You might want to supplement my analysis with standards for another type of knowledge that you tend to develop or utilize (e.g., in the legal profession). ¹²This mnemonic, if practiced accordingly, qualifies as reconstructible cue mnemonic (RD cue mnemonic) as described in chapter 14. ¹³Gardner (1999) proposes a related taxonomy in K-12 education that focuses on the true, the beautiful and the good.
208
Assess
11.3.1 Rhetorical and rational compellingness Now, Mr. Gladstone is fond of employing the phraseology of which we speak in those parts of his works which require the utmost perspicuity and precision of which human language is capable; and in this way he deludes first himself, and then his readers. The foundations of his theory, which ought to be buttresses of adamant, are made out of the flimsy materials which are fit only for perorations. This fault is one which no subsequent care or industry can correct. Thomas Babington Macaulay In addition to considerations of factual truth, we assess knowledge resources according to the way knowledge is presented, constructed and argued. In a narrow sense, rhetoric concerns the artifices deployed by authors and speakers to persuade their readers and listeners. More broadly, however, rhetoric blends into rationality and logic. As there are too many rhetorical criteria to consider here, here I will focus on some of the most important ones. Contributions to knowledge normally address a set of requirements, some of which constitute an epistemic problem. When processing a resource we detect whether it clearly specifies the content of a problem and its boundaries. We also seek to understand the scope of the resource: how broad is the problem? And what would count as a solution? In this book, for example, I have chosen to address a very broad set of requirements. There is no simple principle to determine whether the scope of a problem (big or small, one problem or another) is right. Judging the scope of the problem helps us assess the usefulness of an author’s contribution (as described below). Some students on entering graduate programs find it difficult to identify a suitable research problem for their dissertation. Having studied for years with topic-centered textbooks, they often fail to understand the importance of problems. This can persist, awkwardly, into the thesis writing phase. They lack the knowledge to identify a promising problem. And they might not realize that disciplines have constitutional problems that may never be completely solved but still must be pursued. Authors themselves aren’t always clear that they are or ought to be addressing particular problems and requirements. And so readers may fail to appreciate that the growth of knowledge is predicated on the identification and targeting of problems. Discovering problems can itself be a significant contribution to knowledge. The same principle is true for practical knowledge, daily life and marketing as well. If a corporation’s products address the wrong problem they are unlikely to advance its business. Indeed, the most significant advances in science are sometimes based on problem discovery, or what some call “problematization”. Sandberg and Alvesson analyzed 52 then-recent articles dealing with organization studies and found that, alas, most of them merely dealt with spotting gaps in the literature. Very few focused on discovering new problems.¹⁴ “Gap-spotting questions are unlikely to lead to significant theories because they do not question the assumptions which underlie existing literature in any substantive ways.” (p. 25). The “future research” section of a document can be its most interesting contribution. It can also betray a researcher’s weakness: starting and ending with problems the researching of which is not likely to significantly contribute to the growth of knowledge. ¹⁴Sandberg & Alvesson (2011).
209
Assess
Of course, we use our understanding of the problem meant to be addressed by a resource to gauge how adequately it actually addresses the problems. One way to do this is to ask whether the methodology used in the underlying research is adequate to address the problem. Here there are often mismatches. A document might start with a grandiose problem statement but pursue it with unsuitable means. Alluring titles can lead us to examine papers whose objectives or methodologies disappoint. A vast amount of academic empirical research in the social sciences contains criticism of the methods employed by previous researchers. Often, the choice of problem is at least equally to blame. We also, of course, assess resources against other criteria, such as compellingness of the thesis, clarity of the claims, rigor, coherence, validity of the argument, falsifiability (for empirical science), objectivity, completeness, comprehensibility, logic and research methods. My purpose is not to try to instruct you on how to assess documents according to these criteria, nor to tell you to do a better job of it. I instead provide you with a framework to help you make sense of the criteria so that you can determine for yourself how to more productively assess knowledge. To be sure, those who are not sufficiently well grounded in logic, probability theory, empirical research methods and argumentation are particularly vulnerable to being hoodwinked or to misinterpret information. However, “having” formal skills and knowledge (i.e., the external mindware) to assess arguments is no guarantee that one will apply them. One has to have and apply the inner mindware—informally, the dispositions, inclinations and propensities. That is what Keith Stanovich and his colleagues refer to as “fluid rationality”. So we all need to keep our critical reasoning skills sharp and primed. It is essential to periodically read about empirical research methods, rationality, skepticism and cognitive biases. Fortunately, there’s no shortage of high caliber resources on these subjects (e.g., Anastasi, 1988; Goldacre, 2014; Lakatos, 1980; Pigliucci, 2010; Popper, 1983; Shaughnessy, Zechmeister & Zechmeister, 2009; Stanovich, 2010; and various Internet resources¹⁵). While being aware of the pitfalls of epistemic paralysis, we must be highly sensitive, as we process information, to reasons why its assumptions or claims may be false, its inferences invalid, and so forth. Thinking of the opposite often pays off.
11.3.2 General epistemic criteria one of the most difficult concepts to convince students—indeed to most people including many practicing professionals—is the profound limitations of personal experience and knowledge gained from practical settings. Robert E. Haskell There are many criteria in relation to which a knowledge resource may objectively distinguish itself. The importance of each criterion varies depending on the circumstances. And so we may ask of a knowledge resource: ¹⁵For example, Rationally Speaking Podcast, Skeptic.com, Sense about Science, and Ben Goldacre’s extensive Bad Science web site. The Fallacy Files contains an extensive taxonomy of fallacies. Your Logical Fallacy Is is a succinct infographic of logical fallacies. The CRAP test is a memorable filter for a few aspects of caliber: currency, reliability, authority, point of view. It is a helpful stepping stone for novices but needs to be augmented, which this chapter does. For example, while gauging authority can help you filter reams of information out, as Ben Goldacre put it “The academic literature is a buyer-beware environment” (2014, p. 146). Sometimes accuracy and purpose are thrown in, yielding “CRAAPP”. I consider purpose more generally in the section on usefulness, below. Assessing knowledge resources is an endlessly complex component of meta-effectiveness, worthy of daily reflective practice.
210
Assess
• How original is it? What contribution(s) to knowledge does it make? Even a literature review ought to be original in a substantive respect, but an encyclopedia article need not. An original contribution might add positively to existing knowledge, i.e., extend knowledge, or it might provide a criticism, or point to a weakness or show a way forward. Lakatos (1980) demonstrates the complexity of the growth of knowledge and criticism. Starbuck used his influential paper on “Knowledge-intensive firms: learning to survive in strange environments” to answer the question “What makes a paper influential and frequently cited?”.¹⁶ He argued that courses on research methods emphasize approaches that are unlikely to uncover interesting new phenomena. They do not foster the questioning of important hidden assumptions. Contrast this with his discovery that senior partners in knowledgeintensive firms are valued not so much because of their technical expertise but because of their social skills and their long-term relations with clients. “At the firms I visited, social skills and social capital were both rarer and more precious than technical expertise.” (p. 1399).
• •
•
•
There are at least as many ways in which knowledge might be “surprising”, “interesting”, or “original” as there are kinds of contributions to objective knowledge. But it is difficult to characterize objectively what counts as surprising and interesting in general. To understand these concepts, we need to consider the psychological concepts of potency and appeal (see below). To experience significant surprise at a paper or other resource, one must possess knowledge and have deep expectations. How well informed was the author? Are his claims based on appropriate uses of prior research? Did he overlook, in a consequential manner, relevant research? And so on. How conceptually rich is the resource? It might explore the “logical geography” of existing concepts. That is to describe and explain how concepts that are already in use work. Or, it might explore “logical topography”. That is to explore the space of possible concepts including new distinctions that are not currently captured by everyday language or scientific accounts.¹⁷ The most potent resources extend our ontology in ways that “cut nature at its joint”. In the following chapter, I demonstrate some of the richness in Keith Stanovich’s book¹⁸. How ambitious is the resource and more generally the research program in which the resource is inscribed? A resource might seek, promise or portend to address, elucidate or resolve a major, deep or broad problem. It may get extra points if it is promising where great minds have failed before. It might also be ambitious in trying to shake highly connected nodes in the network of knowledge. This relates to the concept of scope mentioned in the previous section. Does the resource reveal underlying concepts, structures, mechanisms or principles that explain or make sense of surface level phenomena; or that disrupt prior understanding? Has the author imparted a particularly perspicacious insight? If the resource conveys practical knowledge, is this knowledge based on underlying mechanisms or surface level considerations? The next section on explanation deals with this issue more specifically. Documents that clearly convey and rely upon mechanisms are more likely to be potent (as described above) because they can better generate understanding.
¹⁶Starbuck (2010). ¹⁷See Sloman (2010a). ¹⁸http://www.keithstanovich.com/Site/Books.html
211
Assess
11.3.3 Assessing explanatory theories “Have the courage to use your own understanding,” is therefore the motto of the enlightenment. Immanuel Kant We are often faced with alternative explanations for the same phenomena (known possibilities). How might one assess their respective strengths and weaknesses? To answer this in a general way, one could select sets of explanatory theories that deal with related phenomena. For example, for each of the following one could identify rival explanations and contrast them: the sketch of mind presented in chapter 5, Gottman’s explanation of what makes marriages fail, Ries’s explanation of why startups thrive or fail, an explanation of why Skype keeps dropping calls. Consider them or other explanatory theories recently as you contrast your answer to this question with the answer provided below. We, knowledge workers, sometimes fool ourselves into believing that in the final analysis, a theory’s ability to quantitatively predict phenomena is what matters. Here are some criteria we use to select between theories or to criticize and improve them, most of which are adapted from Sloman (1978)¹⁹. • Is the theory sufficiently definite? Ideally, it is clear what counts as a phenomenon to be explained by the theory vs. what it does not cover. The broad theory of the mind sketched in chapter 5 is meant to account for purposive behavior but is not meant to specify fine details of execution (e.g., how speech is produced, how visual perception happens). • How general is the theory? The more general the better. Thayer’s (2001) two-dimensional theory of moods accounts for all kinds of mood states in a compelling way. A theory that could also account for all kinds of other affective states and processes, such as perturbances, attitudes and motive processing would be preferable all other things being equal. • Is the theory sufficiently parsimonious? How could it be altered to make it simpler and better? This criterion is can be taken too far. “I think that when we are speculating about very complicated adaptive systems, such as the human brain and social systems, we should especially beware of oversimplification—I call such oversimplification ‘Ockham’s lobotomy’.” (Good, 1971 p. 375). • Is it mechanistic and mechanistically plausible? Many theories fall short of being good explanations because they do not reveal the hidden mechanisms that make the observed phenomena plausible. Yet the essence of an explanation of possibilities is to shed light on internal mechanisms, which must be plausible once understood. Ideally, one could construct a computer simulation of the theory to reproduce or simulate the observed behavior. Behaviorists fail to explain motivated behavior in this way—they abstemiously avoid speculating about underlying mechanisms and stick to observable input/output constructs. • Does the theory account for fine-grained structure? Freud’s psychoanalytic theory provided coarse mechanisms for some motivated behavior, but failed to address how people could even in ordinary circumstances generate goals, plan for them, schedule them and pursue ¹⁹http://www.cs.bham.ac.uk/research/projects/cogaff/crp/
Assess
•
•
•
•
212 them. AI researchers in contrast have tried to explain mentation in fine detail. Thayer’s twodimensional theory of moods is general and has heuristic power; but it does not account for the fine-structure of the cognitive content of affective states. Does the theory enable one to rigorously derive or deduce predictions, applications and solutions to practical problems? Psychodynamic theories, for example the ones proposed by Sigmund Freud, are useless for making predictions and their applications are lengthy and impractical. Failure to make predictions is not necessarily fatal to a framework if it can adequately be enhanced by more specific theories and meet other criteria listed here. Alas, that cannot be said for psychodynamic theories which as a whole are anathema to science in general and cognitive science in particular.²⁰ While one must distinguish between practical and factual aims of science, a theory’s practical use enhances its overall caliber. For example, cognitive-behavioral theories such as Gottman’s have greater merit than psychodynamic theories partly because they are instruments to solve clients’ practical problems. They can be used in comparatively short periods of time and at lower costs (e.g., with group courses and literature). Does the theory have heuristic power? As Lakatos explained, a scientific theory should enable and facilitate research and applications: to ask new research questions, apply the theory to diverse problems, perform various types of research, etc. More generally, the theory should enable and participate in the development of a progressive research program. This is a difficult criterion to apply. It takes time to tell that the research generated by a theory is a dead-end. Just as genes and memes are not necessarily beneficial to their hosts, popular research programs do not necessarily advance knowledge—science has its fads too. So one must try to distinguish heuristic power from allure and other criteria. This criterion is not often discussed explicitly. Is the theory improvable? Improvement can sometimes be accomplished by modifying its components, adding components or changing the interpretation of terms. It should preferably be possible to embed a theory within other theories that account for related phenomena or finer-grained phenomena. Software developers have explored a large variety of ways in which software can be extended and embedded that are pertinent (with some adaptation) to theories in cognitive science. Thayer’s theory of moods for instance could be preserved within a broader theory of affective processes. Freud’s theories are less extensible. Bereiter (2002a) warns us against theoretical tropes—such as “Children’s minds are not empty vessels waiting to be filled”—that staunch thought, discourse and knowledge building. “An improvable conceptual artifact, by contrast, is likely to strike us as interesting, at least somewhat unsettling to our existing beliefs, and as raising questions and having implications beyond those that are immediately apparent.” (p. 42) Valuing the criterion of improvability of knowledge, i.e., adopting a designer stance towards it, disposes us to criticize and improve knowledge. A theory may serve its users well for a period of time while undergoing a series of improvements until it reaches its apotheosis. It may then need to be superseded by another (Bielaczyc et al., 2011). To make sense of the concept of improvement, one needs to systematically characterize the ways in which theories can change. Paul Thagard (2012, Part IV) describes
²⁰Psychodynamic theory and psychoanalysis illustrate how far awry one can travel when unbridled by considerations of caliber. Alas, these ways of not thinking are still utilized in by some medical doctors. And so the “past of a delusion” (McCulloch, 1953) needlessly extends human suffering.
213
Assess
nine types of conceptual change, the most radical of which are (a) reoganizing hierarchies by shifting a concept from one branch of a tree of concepts to another (“branch jumping”), e.g., tuberculosis an infectious disease or emotions as perturbance; (b) tree switching, i.e., changing the principle in accordance to which concepts in a domain are organized. As an example of tree switching, Thagard mentions the reclassification of diseases in terms of causes instead of symptoms. Another example is classifying emotions in information processing terms, as we did in Part 2, rather than in phenomenological or behavioral terms, as is more common. • A theory might be weak in its current form but promising through its potential for extension. The foregoing criteria should be addressed as open-ended “how” questions. For instance “how can the theory be improved?” as opposed to simply “Can it be improved?” The criteria taken together are very exacting and serve multiple purposes. They enable us to recognize the specific strengths and weaknesses of a theory, to select amongst competing theories or to realize that none of the competing theories is of adequate caliber. They can help cure us of a false sense of understanding, instill epistemic humility and motivate and direct scientific inquiry. The criteria demonstrate that there is more to assessing a theory than to gauge whether it is empirically falsifiable²¹ or can account for more data than its rivals. Nor is assessment merely a matter of subjecting theory to empirical test. Researchers may sometimes save themselves the trouble of testing a weak theory and instead pursue the more pertinent task of criticizing, developing and replacing theories. This is a point that, to my knowledge, does not get sufficient attention in courses and textbooks on research methods.
11.4 U: Gauge its usefulness The acquisition of knowledge has an obvious evolutionary purpose: to create the expectations that guide the organism’s behavior. […] the arms race of anticipationgeneration has created an unremitting pressure on us to become virtuoso expecters. Mathew Hurley, Daniel Dennett & Reginald Adams While the most useful knowledge resources are of high caliber, at least in some respect, a resource might be of the highest caliber and yet be irrelevant to your concerns. Alternatively, it might be useful despite being flawed. For example, it might contain a knowledge gem; or you might learn by criticizing it. So, we must consider the usefulness of a resource to our purposes. The usefulness of a knowledge resource is how processing it might help or hinder you in the pursuit of your projects, goals, concerns, preoccupations, predilections and fulfillment of your roles and areas of responsibilities—in short, how it pertains to your motivators. When you determine that a resource is sufficiently pertinent to one of your projects, it becomes part of a small and privileged portion of information you might process further. An excellent way to filter and assess resources, and focus your attention, is thus with respect to their usefulness. ²¹Falsifiability, a criterion so well articulated by Karl Popper, is very important. Ch. 1 of Lakatos (1980) demonstrates that the concept is much more complex than is normally assumed. For instance, in progressive research programmes theories with heuristic power can be protected from falsification in ways that naive falsificationism does not allow.
Assess
214
Usefulness is an umbrella concept. There are many ways in which a resource might be useful and there are many features that might attenuate usefulness. Urgency and importance are significant aspects of usefulness. These notions themselves are complex. Our tacit abilities to assess importance and urgency are much more sophisticated than one might think.²² In the simplest case, urgency is just a matter of how much time you have left before you can take advantage of an opportunity or avoid a threat. In the more general case, urgency refers to the costs and benefits of processing the resource as a function of the time course of processing it.²³ This is not necessarily a monotonic function. The importance of processing a resource is related to its criticality (i.e., whether this resource will suffice or a replacement can easily be obtained) and its sufficiency (i.e., how much the resource will contribute to satisfying the goals with respect to which it is deemed useful). Our judgments must, and I am sure usually do, factor in uncertainty in one way or another. When we avert our gazes from the information parade purposefully designed to seduce curious minds, we may least tacitly factor in the opportunity costs of processing a resource. Time and opportunity costs are dimensions of usefulness. Given this general conception of usefulness, you might be wondering why I did not use the term “utility”. Utility theory formalizes and utilizes concepts of uncertainty and value. Baron (2008) explains how utility theories describe human behavior, specify norms for human behavior, and even prescribe behavior. The utility framework is indeed helpful for a deep understanding of assessment. (It is also critical to the theme of rationality that runs throughout the current book.) However, one can accept it for normative purposes without using it for practical purposes. For example, there may be all kinds of normatively valid heuristics. In my opinion, judging usefulness ought not mainly or even normally be a matter of assigning a scalar rating. When dealing with large numbers of documents, many of which are “irrelevant”, the most important move is not to rate, but to classify them with respect to the particular project to which they might be relevant. In order to classify documents with respect to projects, you need to have a fine-grained idea of your current and upcoming projects. GTD software, such as OmniFocus, is designed to help you keep track of these projects. I suggest that becoming skilled at matching documents to projects can enhance one’s cognitive productivity. Beyond matching a candidate document to a project, of course, we (ought to) detect why it is relevant, i.e., to have a sense of what aspect of the information serves or might serve what potential goals. What specific benefits will arise from processing this document? How difficult will it be to understand and derive value from the resource? How much deliberate practice will it require? This process involves accumulating qualitative descriptors that can be leveraged in decision making. For example, when considering Gottman’s books, you might say to yourself “To adopt Gottman’s theory will probably cost me dozens of hours, spread over the next six months; but it will likely improve my relationships with my spouse, family, friends and co-workers. In particular, mastering all the principles, instead of just a few, might make the difference between a happy and an unhappy marriage. But I will need to invest in productive practice.” These questions are not just quantitative, but qualitative. Utility theory underemphasizes these qualitative processes. Nevertheless, the utility concepts are useful background knowledge. There are some boundary conditions to consider. Usefulness, is not just about how a resource ²²This sophistication is analyzed in chapter 3 of Beaudoin (1994). See also Kagan (1972). ²³I mention generalized urgency, a concept specified in Beaudoin (1994), because the concepts of importance and urgency, which are components
of desirability, as typically described fall far short of our actual abilities to assess opportunities (and threats). See also Russell(1995).
215
Assess
might hinder or advance adopted goals, it applies to motivators that are not (yet) represented as goals or that may have been dormant. Often, particularly for effectant people, through serendipity (or bad luck), documents will trigger new top level goals and other motivators. They may reveal problems and opportunities. They may ultimately lead us to reconsider our roles. Here we can say that documents have “forward-reaching usefulness” meaning that they are pertinent to future goals, should they be adopted. Their usefulness is contingent upon the adoption of the goals that they have triggered. More generally, usefulness is always contingent, but sometimes contingent on the production or adoption of goals and projects. Sometimes, we get distracted and fail to postpone reading resources whose usefulness is contingent upon goals that are or ought to be suspended. Conversely, we can fail to notice that a document is pertinent to our goals or well-being. In this tetradic framework, to the extent that processing a resource leads to the production and pursuit of motivators that will enhance our understanding, we say that the documents are potent. The next section expounds the progressive concept of potency.
11.5 P: Gauge its potency The deepest advances in science are those that extend our ontology substantively Aaron Sloman E = mc² Albert Einstein To a first approximation, the potency of a resource is the extent to which understanding it would lead you to a better understanding of the world. Yet most resources, at best, merely present additional facts. News articles tend to fall into the latter category. They’re easy to read because they don’t stretch your understanding. In the terms of Jean Piaget, you can easily assimilate their information without needing to accommodate, i.e., to undergo deep (and potential painful) mental changes. Potent resources, in contrast, convey “mind-bending” knowledge. They may contain facts that contradict your explicit or deeply tacit beliefs (in facts or theories.)²⁴ Because propositions are interconnected and have dependencies, often when one belief changes many other beliefs need to be re-examined. The more interconnected the challenged belief, the more potent it is. However, it is difficult to work out the implications of a changed belief on the other beliefs that originally depended on them. Potent resources may provide you with the opportunity to acquire new concepts that you had not previously encountered or mastered, or to change the concepts you use. Some of these documents are conceptually rich: If they had (or have) a glossary, you would likely find terms in there that you had not yet fully grasped. Or they would organize these concepts in ways that are significantly new for you. Potent resources do not merely convey new terminology, they convey new meanings—for new or existing terms or combinations thereof. Newton changed the meaning of force, mass and acceleration. Einstein altered the concepts of energy, space and time. ²⁴That is to say that even empirical facts may be revised. Samuel Arbesman put forth the provocative idea that facts have a half-life (Pigliucci & Galef, 2013).
Assess
216
It can take a long-time for one to master new concepts. Mastering concepts involves more than being able to explain or use them when prompted. It requires that one be disposed to think and know with them (See “To develop implicit understanding” in chapter 2). They enable us to reframe problems, as Schön emphasized²⁵ in The Reflective Practitioner. Because potency is defined in terms of understanding, to understand potency one needs to understand understanding itself. I described the concept of understanding in chapter 2 in relation to Bereiter’s concept of understanding. I would add that understanding involves the ability to interpret surface features, events and processes in terms of invisible, underlying mechanisms. There is an “under” in understanding. Potent concepts do not tend to refer to surface features of the world, nor do they necessarily have referents (i.e., “instances”). To use Bereiter’s terminology, they are problem-based concepts, meaning they are concepts that have been proposed to solve problems of understanding the world or deep practical problems. Newton proposed the concepts of weight, mass and acceleration not to describe the world but to explain it, i.e., to resolve problems of understanding the physical properties he had observed and found puzzling. Thus, the mark of potent concepts is that you can use them in solving conceptual and practical problems. To paraphrase my software engineering mentor, Stephen Leach: “Knowing which concepts are really potent effortlessly guides you to the right answer.”²⁶ This requires that your monitors detect that the concepts apply to the problematic situation. Very potent knowledge resources can change your relationships to many concepts you tacitly or explicitly use. Given a new theory, it can take a long time, as well, for you to realize that you need to modify, abandon or restrict the conditions under which to apply previously acquired ideas. For example, before mastering Keith Stanovich’s conceptions of intelligence and rationality, you might have indiscriminately applied the labels “stupid”, “short-sighted”, “foolish” and “irrational” to various people. It might take months or even years to revisit your judgment of a particular person whom you had characterized as “stupid” to determine that he was smart but intellectually lazy. Or more precisely, that his reflective mind did not exercise sufficient control over his algorithmic mind. You might drop some terms (e.g., “foolish”) because neither do they neatly map to a concept that is explicitly specified in the new theory nor can they be mapped to an appropriate combination of concepts in the theory. Conceptually rich theories have a greater potential to be potent because they propose many intertwined concepts. Understanding with one concept requires understanding many concepts. This is another reason why understanding can take time to develop. For example, to understand the lean startup methodology of innovation, a product manager needs to understand the intertwined concepts of minimum viable product, validated learning, build-measure-learn, innovation accounting, genchi genbutsu, andon cord and many others. This theory would significantly alter how he and his team develop products. Their day-to-day work would not be the same. They would be opting for product development cycles that could be 1/10 their prior length. They would need to learn to view product modifications as hypotheses that need to be tested rather than as sure-fire enhancements. The product manager would begin to think and function like an empirical scientist. He would need to quickly alter his course should the tests fail. He would learn to detect mistakes immediately that in ²⁵Schön (1982). ²⁶Leach (2012).
Assess
217
the past he might have overlooked until they had snow-balled. And he would see them as potentially indicating errors in a systemic process that should be altered immediately with a pull of an “andon cord”. He would adopt the rule “Be tolerant of mistakes the first time. Never allow the same mistake to be made twice.” Thus, one potent book proposes many deep changes in understanding. Potency is a subjective notion in that it is relative to what the person (subject) already understands. A book will not be equally potent for all readers. On the one hand, a book will not be as potent for a reader who already masters most or all of its concepts as it will be for a reader who masters fewer of them. A manager who already works for a company that has an agile culture and processes won’t benefit as much from Ries’s book as one who doesn’t. On the other hand, a book might (currently) be relatively impotent for a reader who lacks too much of the requisite background knowledge. In the case of The Lean Startup²⁷, I suppose one should be acquainted with startup projects and businesses to understand and be able to apply its theory. However, there is no simple rule— quantitative or not—to which I can appeal to describe potency as a function of prior mastery; nor is one necessary. One cannot adequately reduce potency to a scalar, e.g., a 0- to 5-star rating. Potency, like understanding itself, is a relational concept. It involves relations between the knower and the knowledge resources, and between knowledge resources themselves. For example, a document might be potent because it sheds some new light on an old theory, or extends, limits or tweaks a theory. Potent documents can take us out of our “comfort zone”—they require effort to understand. While I believe it is useful to think about documents through the conceptual lenses of potency, I recognize that ultimately potency judgments are always intuitive and incomplete. Even after reading a document, one cannot necessarily adequately characterize its potency.
11.5.1 Potency as the potential for mental development In the previous section, I provisionally specified potency as a resource’s potential to impact one’s understanding. I now want to generalize the concept of potency even further to capture the panoply of mental effects that processing and mastering a resource might have. I described understanding in a very broad manner in chapter 2 and Section 6.3, to capture manifold statable and implicit understanding. Potency of a resource, in the most general sense, is the extent to which mastering it can alter one’s mind. Or, from a teleological perspective, potency is the depth and breadth of the mental development required to be able to systematically think with and in terms of the knowledge. For example, to master the lean startup methodology requires that one be willing to expose one’s product ideas to the market in very small increments. This introduces the risk that the market (and perhaps stakeholders) will reject the product before one has had a chance to fully implement the founders’ vision. Ries proposes that successful entrepreneurs strike the golden mean. “[They] do not give up at the first sign of trouble, nor do they persevere the plane right into the ground. Instead, they possess a unique combination of perseverance and flexibility.” The lean mentality calls for the entrepreneur to know when to “pivot”, i.e., to change courses by way of a new conjecture about how the product could be developed aptly to provide value for customers. Lean perseverance and flexibility are not just matters of knowledge and understanding. They entail propensities, attitudes, ²⁷http://theleanstartup.com/book
Assess
218
habits and abilities to regulate one’s behavior in accordance with standards. To thrive in an agile environment, one must come to feel at ease and even to enjoy this more dynamic system. How is one supposed to respond to a potent knowledge resource, such as The Lean Startup, such that one develops one’s mind to attain the target state? How does one become one’s mindware developer? This calls for a system to help one enjoy things one ought to enjoy. Psychological development can be facilitated by configuring the external environment with agreements, processes and norms. For example, to potential shareholders and employees one can explain the “accountability framework” in which the startup operates and ensure that they agree to it as a precondition of participating. Further, an individual can set up productive practice routines for developing themselves, as we will see in chapter 13 and chapter 14. As we saw in Part 2, folk psychology concepts have their uses but also significant limitations. Ultimately, personal development is a matter of developing mindware and objective understanding. A knowledge resource is potent to the extent that mastering it would lead to significant mental changes, such as new low-level connections, motive generators, motivators, cognitive reflexes, filters, inhibitors, insistence assignments, management procedures, long-term working memory structures, meta-management controls, and so on (see chapter 5). Potency and usefulness are different. Usefulness is mainly about World 3, i.e., how a knowledge resource might help one to solve one’s problems — creating, criticizing, improving, and delivering products. Potency in contrast is concerned with World 2’, your mind, and its relation to the rest of the world. Normally, the more potent a knowledge resource is, the more work it requires. In contrast, other things being equal, a document has higher utility the less work it requires. Potency and usefulness differ in further respects. Potent documents require more time and effort to master: Like investing, accommodation pits long-term utility against the short-term good. Further, a document might be very potent without being useful to the reader. Moreover, one may develop substantially but for the worse in response to a knowledge resource. Consider the toxic mental impact that Hitler’s speeches and book had on many who accepted the ideas despite their negative utility and low caliber. Consider also the perpetually “contaminated mindware” (in Stanovich’s sense of the term) of pseudoscience and dogmatic religions that vitiate credulous minds of all ages. Some knowledge resources, if adopted, can even stunt mental development. Crystalized inhibitors are beliefs, strategies and rules that are detrimental to rational thinking (Stanovich, 2011; Toplak, West & Stanovich, 2012). Crystalized facilitators improve thinking, decision making and further mental development. Meta-effectiveness resources, such as this book, are supposed to help develop crystalized facilitators—i.e., to be potentiators of potency. So, in deciding which knowledge resources to pursue one needs to separate its usefulness from its potency. It is not always easy or possible to determine to which specific projects some knowledge will apply. And yet, if the knowledge is particularly potent it may be worth pursuing. This is a principle for funding theoretical research in universities. Some children evince curiosity that cannot easily be explained on an instrumental basis. This tendency, if nurtured, can persevere in adults. This notion
219
Assess
is captured by the propensity concept of effectance presented in chapter 3.²⁸ Often we engage in activities that cause us to learn, even if we are not even unconsciously aiming to learn, or to become more effective or to stretch our minds. Many researchers in AI and other disciplines (though not all) assume that learning must be related to reward in some way, e.g. through positive or negative reinforcement. I think that is false: some forms of learning occur simply because the opportunity to learn arises and the information-processing architecture produced by biological evolution simply reacts to many opportunities to learn, or to do things that could produce learning because the mechanisms that achieve that have proved their worth in previous generations, without the animals concerned knowing that they are using those mechanisms nor why they are using them. (Sloman, 2009b) Still, effectance can and should be nurtured. Because of the effort required to master (or “accommodate” to) them, potent resources are less likely to be memetic. Compare, for example, three different views about the progress of science: Kuhn (1962/1996), Popper (1959) and Lakatos (1980). Kuhn’s simplistic concept of paradigm shifts is most widely known, but arguably of lowest potency (and caliber). Popper’s theory that science proceeds through conjecture and refutation is simple to understand in its naive form, but more complex and subtle than Kuhn’s in its sophisticated forms. Coincidentally or not, falsificationism is widely known, even paradigmatic,²⁹ in scientific and scholarly circles. However, fewer people outside scientific circles know Popper than Khun’s theory. Lakatos’s treatise on scientific progress is the richest of the lot. It’s core ideas are augmented with a large array of powerful concepts and propositions. It is the most difficult to understand but (in my opinion) it is of the highest caliber. For most, its utility is diminished by its potency, because of the effort required to understand it. Thus, it’s the least known of the lot, even in scientific circles. Potency and memetic potential are inversely related. By separating potency from usefulness, I do not mean to suggest that utility is irrelevant to knowledge building or practice. For example, most of those who are not knowledge workers, in the narrow sense, do not require a sophisticated concept of the progress of knowledge—the middle ground, Popper’s, will do. I am simply explaining how the concepts differ and relate to each other, while suggesting that utilitarian considerations can stunt mental development. The concept of potency as potential for mental development ought to be of interest not only to ourselves as learners, but to cognitive scientists, educational psychologists and educators. It is up to cognitive science to catalog and explain the changes in mental architecture, structures and representations (including affective ones) involved in knowledge-based mental development. What mental changes does a person undergo when he masters knowledge resources? To answer ²⁸Chapter 4 of DiSessa (2001) provides a very personal description of knowledge building and effectance motivation. DiSessa does not use those terms; she speaks, instead, of “the regime of competence” and “committed learning”. “I got stuck only when I had a strong motivation to continue in a certain line, so strong that it felt like the right thing to work on despite ordinarily unpleasant feelings of frustration. Some unpleasant feelings are the happy consequence of motivation, and motivation cannot be the pure pursuit of pleasure or feeling good.” (p. 84.) ²⁹This is not to say that scientists operate in a falsificationist mode, but that they tend to realize that (a) knowledge is conjectural and (b) it is important to try to criticize and refute theories. That knowledge does not necessarily lead to falsificationist practices. Some would call the result “lip service”, or a “transfer problem”. Others would say that their practice, while being far from homogeneous, simply corroborates Lakatos’s view of the growth of scientific knowledge.
220
Assess
this question requires considering a wide variety of knowledge resources: with different kinds of knowledge (factual, practical, normative and each of their subdivisions), caliber and potency. As a practical problem, cognitive science must investigate the trade-offs amongst the different activities for affecting mental changes. Instructional designers could revisit their learning materials (and methodologies) to consider the concept of potency and the underlying mindware students need to develop to achieve their learning objectives.
11.6 A: Gauge its appeal and analyze your intuitions I have as usual failed to leave enough time in which to say why what I have said is interesting. J. L. Austin A culinary expert can describe foods in terms of basic sensory qualities (bitterness, saltiness, sourness, sweetness and umami) and higher-order ones—e.g., fruity, gamy, harsh, lush, nectarous, piquant, sharp³⁰. To characterize our assessments of knowledge resources we need to understand how appeal differs from other types of assessment. We may find documents “interesting” for other reasons than those listed under the rubrics of caliber, utility and potency. Conversely, a document might meet high objective standards, be very useful and even be potent and yet not be very appealing. We can like things that are not good for us. We also need to know something about appeal judgments in themselves. This in turn can inform our choice of resources. This can also help us understand aspects of the mind that we might want to change through practice. Compare the section “Develop attitudes” in the final chapter on productive practice. Impressions of appeal are affective feelings–tacit evaluations that are not necessarily accompanied by explicit judgments. In the affective taxonomy of Ortony, Clore and Collins (1998), they would be classified as attitudinal evaluations. These authors classify such evaluations as “emotions” (generically, “likes” and “dislikes”), however they wouldn’t necessarily be classified that way in the three-level taxonomy of emotion concepts presented in chapter 5. A valenced reaction is often part of an emotion, but it’s not sufficient for an emotion. Impressions of appeal have various causes, effects and purposes. The mechanisms that determine these feelings are poorly understood. These feelings often contribute to our decisions of what objects to choose and how to use them. They can be affected by, and affect, judgments of caliber, potency and utility. One might genuinely like something (i.e., find it appealing) because it serves a need and then continue to like it for that reason. One might like something because it is of a very high standard. But liking something does not necessarily imply one finds it of high caliber, though it might bias us to judge it as high caliber. We may resist objects that appeal to us and we may elect to process resources that are not inherently appealing, because we (perhaps tacitly) judge them to be useful or of high caliber. Here are some contributors to impressions of appeal. Surprising. It is no surprise that interesting papers tend to be surprising. But what makes information surprising? The concept involves events, conditions and states: objective and subjective ³⁰http://www.world-food-and-wine.com/describing-food
Assess
221
elements. Being surprised is a classically cognitive state that does not necessarily entail value (affective) judgments nor does it necessarily entail a loss of control of attention.³¹ It is not itself an emotion. People like pleasant surprises. Some like to be shocked. (Some enjoy thrillers and horror movies.) While scholarly papers are supposed to contain significant novel information, few of them are surprising. Try this exercise. Randomly choose five papers at random that you’ve read in the last decade. How many of them contained information that you found surprising at first? Find some surprising ones. In what respect were they surprising? More generally, what makes a claim, concept, inference or finding surprising? To find a document surprising, one needs a fair amount of prior knowledge—knowledge contradicted by the document. Like chance, surprise favors the prepared mind. But of course if one fully masters the conveyed knowledge, one is less likely to find it surprising. Inference and prediction are normally involved in surprise. One can fail to notice that something is surprising. The well-documented confirmation biases can distort reading, making one fail to realize that information is surprising or of great value.³² Somehow, one needs to develop mechanisms to detect whether the information contradicts one’s beliefs. A trick for this, while reading, is to activate related, potentially incompatible, theories and to compare the information to them. While reading this chapter, for example, you might consider theories of logic, research methods, critical thinking, and affect. What would you expect them to say about assessing knowledge resources? How is this chapter different from what those authors would have said? Or from your taxonomy of assessment? Did you have or know of a taxonomy of assessment before reading this one? The human ability to predict has been the subject of recent attention in cognitive science (e.g., Clark, 2013; Hawkins & Blakeslee, 2005; Hurley et al., 2011). According to Hawkins, the cortex is a predictive, hierarchical, memory system where feedback is predominant. In a computational theory of the perception of visual motion, Lamontagne (1973) posited a stunning predictive neural mechanism. Our eyes normally move in saccades. However, when we track moving objects, our eyes move in a smooth fashion. The mechanisms underlying smooth eye pursuit are continuously predicting the position of the pursued object. Based on his theory, Lamontagne predicted an entire new class of illusions in which smooth eye pursuit would be triggered, in the absence of motion, by a predictive mechanism.³³ Hurley and colleagues argue that humans have an innate tendency to anticipate and to verify their inferences. Our ancestors who had a tendency to debug their theories had a selective advantage. This Hurley et al. (2011) claim is at the heart of mirth: we draw inferences in the implicit hope of obtaining the pleasure of a surprise. Mirth is pleasant in order to reinforce rational thinking. This may also be a function of pretend play in children even as young as two years old (Dias & Harris, 1988; Pellegrini, 2013). While many predictions happen automatically, as mentioned above, we also often fail to draw relevant inferences. Not everyone gets the joke. We double-book ourselves. Some readers are more likely to think through what they read than others. Cognitive miserliness is also parsimony of pleasure. ³¹This is explained in Ortony, Clore and Foss’s conceptual analysis of affective terms. ³²Digdon (2015) presents a detailed and very insightful case study that shows how confirmation bias, and related logical errors, can adversely
impact reading and historical research. ³³The discovery of this class of illusions demonstrates the power of the design stance. Arguably, the illusion would not have been discovered by empirical methods (Lamontagne, 1976).
222
Assess
Fascinating. What scientist does not want his or her contributions to knowledge to be regarded as fascinating, in a non-derisive sense? Consider the meaning and connotation of the following terms related to “fascinating” according to Apple’s British thesaurus.³⁴ engrossing, captivating, absorbing, interesting, enchanting, beguiling, bewitching, enthralling, enrapturing, entrancing, spellbinding, transfixing, riveting, mesmerizing, hypnotizing, engaging, compelling, compulsive, gripping, thrilling; alluring, tempting, enticing, irresistible, tantalizing, seductive; charming, attractive, intriguing, delightful, diverting, entertaining, amusing; informal, unputdownable. ANTONYMS boring; dull. These terms point to different shades of thoroughly affective states. Fascinating is cognitive (in the classical sense) but it also has an affective connotation, meaning that one is drawn into the resource. We can be drawn into such a document whether or not it serves our current commitments. The term “fascinating” is not used to describe seductive news articles. There is something else at play with such documents, which are alluring for different reasons. Aesthetically appealing. Aesthetic appeal is an important collection of mental reactions to objective properties of knowledge resources. It usually involves finding something beautiful. Some authors use the expression to mean all kinds of appeal. Nathalie Sinclair studies aesthetics in mathematics throughout the spectrum of expertise (Sinclair, 2004, 2006). She argues that mathematicians face many choices about which problems to choose and which problem-solving paths to pursue that they cannot make on purely logical grounds. Mathematicians often invoke various aesthetic criteria and concepts of beauty, elegance, appeal and feeling. Similarly, the literature on expert reading, summarized by Pressley and Afflerbach, shows that experts tend to read with passion. They see shades of beauty and ugliness in knowledge resources. Sinclair distinguishes three roles of aesthetic responses in mathematics. The first role is to assess mathematical products. This corresponds to many of the caliber judgments in my taxonomy. The second role is motivational, which is to attract mathematicians to certain fields and problems. The third role is generative, which is to guide them as they are solving problems. She attributes the generative concept to Seymour Papert, a prominent AI researcher: “[Papert] shows that an aesthetic response to a certain configuration is generative, in that it leads the inquirer down a certain path of inquiry, because then she feels that the appealing configuration should reveal some insight or fact.” (Sinclair, 2006, p. 100). There is, however, a dark side to appeal. We are sometimes fooled into believing things because they appeal to how we want them to be or think they should be. That is to say that they appeal to our motives or standards. This is, of course, wishful thinking. In David and Goliath, Malcom Gladwell recently proposed that dyslexia and other problems are often desirable difficulties. Some readers who want underdogs to win, to be uplifted, or simply prefer to see the world through rose-tinted glasses find this book appealing. Their attitudes can inhibit their tendency to monitor for caliber: They might overlook or simply discount the book’s lack of rigor. Gladwell’s Outliers, and Ericsson’s theory of expertise similarly appeal to our desire for all to have an equal opportunity to succeed. ³⁴In chapter 4 Sloman (1978) several techniques of conceptual analysis are described that one could apply to these concepts. I listed in bold font some terms that I feel would be particularly worth analyzing in a treatise on appeal.
223
Assess
However, the appeal of these books can be mitigated when one draws some of their unappealing implications.³⁵ We need not merely be aware of appeal, we also need to keep this dimension in check. A theory’s beauty, elegance or simplicity does not make it better according to other criteria. According to Pigliucci (Pigliucci & Galef, 2012), Smolin (2006)³⁶ claims that over-valuing aesthetic properties of theories in physics, including string theory, has often held the field back. The truth is not always beautiful. (Nature plays dice.) The literature on cognitive biases demonstrates that people often hold irrational views for reasons they ignore. We all need to beware especially to (1) not give authors and theories that we like an easy ride; and (2) avoid dismissing opposing views too easily.³⁷ For example, we might wrongly dismiss a paper from an opposing camp because of an irrelevant methodological flaw; and we might overlook a critical flaw in a paper that aligns with our pet theories. To help ensure that our preferences and vested interests are not unduly biasing us, it helps to deliberately probe our own assumptions and attitudes and also to consider the opposite. The more we try to do this, the easier it comes to us. Or, in H-CogAff terms, as our meta-management processes compel management processes to think along these lines, our brains will grow and fine-tune critics, internal monitors to detect our own biases. I presented the four categories of my taxonomy of assessment, CUPA, in descending order of the ease with which a reader can articulate them. Impressionistic judgments are the most difficult to articulate and justify, yet they may be as powerful and important as the others. The concept of the “unconscious mind” has long been fixed in lay psychology. Cognitive scientists have been exploring highly mathematical models to explain mental capabilities. These models assume large swaths of processing are extremely opaque and inaccessible to higher level functions (e.g., that can control language or mental simulations). They use representations that are not easily translated into natural language. One is only aware of a small aspect of her own mental processing; that awareness is only slightly more extensive than one’s knowledge of others’ minds. The attempt to explicitly think about “unconscious” processes is nevertheless important for the development of mindware and expertise. By making explicit judgments of caliber, usefulness and potency one can train oneself to make better impressionistic judgments. However, this hypothesis needs to be tested.
11.7 CUPA: Helpful information Don’t bite my finger, look where I am pointing Warren S. McCulloch (quoted by Seymour Papert) While I proposed above that a typical 5-star rating system is of limited use for assessing a knowledge resource, I am not opposed to simplification. On the contrary, it is helpful to have a structured, bottom-line label to capture one’s overall assessment of a knowledge resource. “CUPA” ³⁵For instance, Seidenberg (2013) points out that Malcom’s David and Goliath may compound false beliefs and counterproductive attitudes towards dyslexia. He adds: “Does [David and Goliath] promote shallow thinking about dyslexia? Absolutely.” Hatano (1998) expresses fear that the deliberate practice theory of expertise may lead people to blame students and educators for failing to achieve higher levels of performance than should be expected. An unappealing consequence. ³⁶http://leesmolin.com ³⁷Considerations of cognitive dissonance of course apply (Tavris & Aronson, 2008) here.
Assess
224
will not become a readily accepted noun (or adjective for that matter) for this purpose any time soon. Therefore, I’ve chosen to appropriate “helpfulness” (and “helpful”)³⁸, defined in terms of caliber, utility and potency of the information. That is not merely a matter of the information being of an objectively high quality (though caliber does figure in the conception), but whether you can exploit the information (a) to become a more effective person, (b) to improve your (relational) understanding of the world, (c) to solve important problems, or (d) to achieve pertinent purposes. Whether you wish to include appeal in this is entirely up to you. You may object that I have introduced vicious circularity. For helpfulness includes usefulness or utility, which appear to all be synonyms. Except they are not. My suggestion is to isolate the assessment of usefulness from potency, caliber and appeal, and then to consider helpfulness as a pragmatically bent integration of the lot. There are entire bodies of philosophy of knowledge in general and philosophy of science in particular that could be brought to bear on this new pragmatic view of knowledge,³⁹ which are beyond the scope of this book. It is up to you to decide whether this way of assessing knowledge resources is helpful. Has it expanded your understanding? Can you use it to become a more effective selector and processor of information? If not, how would you replace it?
11.8 Other minds: Their recommendations, reviews and commentary One of the most important sources of information about knowledge resources is, of course, other smart, knowledgeable people. My brilliant, late friend, Ralph Greer⁴⁰, would always ask me what book I was currently reading. I acquired his habit, generalizing the query to include other media and asking about the resources that affected them the most (i.e., the most potent ones). It’s a great way to drill past conversational shallows into potentially rich seams of knowledge and intellectual pleasure. This also yields important knowledge about the other person. The flip side, providing very helpful (high CUPA) recommendations, is a brilliant way to serve one’s network. These exchanges also remind ourselves, and others, of the most important things in life—knowledge, learning, that which we’re learning about, and why we aim to learn. Thus, it is essential to associate with other knowledge-oriented people. They can be more useful sources of recommendation than those we haven’t met (e.g., on the Internet), for not only can they provide us with insight into the caliber of knowledge resources, but because they know us personally, they can consider utility, potency and appeal. (That expert you follow on Twitter doesn’t know what you, personally, need to know, what will bend your mind, and what will appeal to you.) Also, your ³⁸Earlier revisions of this document used the term “efficaciousness”, to emphasize the potential contribution of the knowledge resource to one’s effectiveness. ³⁹C. S. Peirce introduced pragmatism but ultimately fixated on a much narrower conception (“pragmaticism”) than concerns us here (Peirce, 1905). The concept of helpfulness is also quite prominent in acceptance and commitment therapy, to which I allude at various points in this book. In this framework, the major criterion for determining which thoughts to process further is helpfulness. ⁴⁰http://cogzest.com/2014/03/ralph-greer-reminiscences-on-a-gentleman-and-reflections-on-cognitive-aging/
Assess
225
personal knowledge of them can help you assess their recommendations.⁴¹ Having said that, one of the greatest things about the era in which we live is that we can tap into recommendations and assessments from a huge number of smart, knowledgeable people we have never met. We mustn’t squander the opportunity. Our own ability and propensity to tap into these seams is integral to our meta-effectiveness. That means seeking out and being attuned to recommendations from trustworthy sources, and using technology wisely. It also means delving reviews. Delving, as we will see in the next chapter, is mindful, strategic and zestful processing of knowledge. By comparing our own assessments (and other meta-information) with reviews, we can discover strengths and weaknesses we failed to detect. Moreover, we can contribute to online discussions. The goal here is not merely to build knowledge nor to learn about the target resource. It is to improve our own abilities and propensities to assess knowledge resources, that key component of our meta-effectiveness. ⁴¹Davenport (2005) emphasizes that high performing knowledge workers also use peers to filter out irrelevant information and receive tailored information. This reduces the amount of information they themselves need to process.
12. Delve The real importance of bad reading lies in the degraded thinking, the conceptual oversimplification of which it is an aspect. Minimal comprehension means not only paying little attention to the words in the message; it means putting little effort into working out the connections between the text and what you already know. Carl Bereiter In order to comprehend, assess and benefit from knowledge resources, one needs to delve them. Unfortunately, web browsers and document readers are for the most part designed for surface processing. As a result, readers hardly make notes on or about electronic documents. They overestimate how much they will be able to retain and utilize. Before the introduction of the iPad, I pointed out that Apple’s then upcoming tablet would need to contend with its major competitor: paper¹. In Scientific American, Jabr (2013) reviewed research suggesting that paper, in certain respects, provides a superior reading experience to tablets. If a document requires delving, readers often still choose to print it. Reading from hard copies, however, does not guarantee that one will apply optimal learning strategies, as research has amply demonstrated. The fact that most people currently read and learn better with paper does not mean that paper is necessarily better overall for delving. I believe that if they apply appropriate learning strategies that leverage the strength of today’s information technology, readers would delve better with IT than paper. This has not yet been demonstrated by empirical researchers. For researchers have not yet pitted paper learning against adequately strategic delving with technology.² Advanced strategies for delving text and multimedia have not previously been adequately tested.³ This chapter provides detailed strategies for delving with technology. Chapters 13 and 14 describe productive ways of practicing with technology. These strategies apply to text and other media—e.g., podcasts, audiobooks, and screencasts. This chapter begins with a description of the characteristics of expert reading. It then provides strategies for delving documents and other media with technology. These strategies involve creating inline and external annotations.
12.1 Effective delving In most law school education, reading is practiced as a means to an end—to produce a description of the substance or procedure of a particular area of the law. Too often, it is only in legal research and writing courses that reading is explicitly addressed. Even ¹Beaudoin (2010a). ²Mueller & Oppenheimer (2014) claimed to have shown that laptop note taking results in shallower processing. On the CogZest blog, I criticized
their paper. ³See Beaudoin & Winne (2009) for a description of an advanced e-learning tool, nStudy, that supports several strategies described in this chapter. nStudy not only supports delving, it is also a tool to research self-regulated learning.
227
Delve
there, it is most often analyzed only in its role as a counterpart to the goal of writing; of producing tangible text. But although reading is not studied on its own terms, it makes up most of the practice of learning the law and much of the practice of law. Kirk W. Junker Cognitive scientists have shined some light on a largely private process, adult reading. This section describes some of the major features of expert reading. These I take to be characteristic of authentically expert delving in general, whether it be with paper documents, or other types of knowledge resources. Pressley & Afflerbach (1995) provide an extensive review of the literature on reading. Researchers have asked students and experts to verbalize their thoughts as they were reading. From these “thinkaloud” protocols, Pressley and Afflerbach identified over 300 types of reading responses and mental processes involed in reading. They demonstrated that, like many other cognitive skills, reading may seem simple but it is sophisticated, complex and opportunistic (see also Pressley & Gaskins, 2006).⁴ • Expert reading of complex material is laborious. Pressley and Afflerbach refer to reading as “constructively responsive”. Contrast this laboriousness with cognitive miserliness. The idea that experts can take in a document through “speed reading” is a myth. For example, it has been shown that “speed readers” overlook crucial words in text (Wagner & Stanovich, 1996). Note that skimming and “speed reading” are not the same. • Experts bring their prior knowledge to bear on what they read. They make predictions. They build up a mental model of the text and its relation to their prior knowledge. They experience surprise. They revise their model when appropriate—they’re open minded. • Experts read with cognitive zest. They are engaged, confident, enthusiastic and motivated. They respond affectively to their reading. They can get excited, angry and upset (compare the discussion of assessment in the previous chapter). Passion and effectance are required to sustain delving. • Expert reading is purposive. Experts typically have a goal in mind that directs and focuses their information processing (the concept of goal setting applies across areas of expertise.) They seek to benefit from the time they invest in reading. • Expert readers tend to seek the gist of a document. • Pressley & Afflerbach report that “Good readers are massively strategic before, during, and after reading” (p. 100, emphasis mine). Experts use strategies to satisfy their main objectives while respecting multiple, often conflicting constraints (e.g., minimizing time and maximizing development of understanding). • Expert reading is opportunistic. Experts, while purposeful, monitor for unexpected knowledge gems. They detect and respond to opportunities. Consequently, there is no simple description of expert reading processes. This poses a challenge for improving already competent reading. At any given point in the reading process there may be many promising avenues to pursue. It also makes it difficult for cognitive scientists to model reading. ⁴Those studies involved reading paper documents. See Rouet (2006) and Fayol & Rouet (2008) on hypertext processing. See Wolf (2008) for an accessible book on reading.
228
Delve
• Experts are knowledge builders. They seek to understand the knowledge conveyed by the document. They use this knowledge to solve problems and to create new knowledge products. • Strong readers monitor themselves while reading. A weak reader may not even realize that what he is reading is new to him or that he has failed to understand something. In contrast, strong readers, like the strong problem solvers described in chapter 7, detect flaws in their knowledge and generate motivators to repair them. In other words, strong readers are “aware” and they engage their meta-management functions (or “self-regulate”, as some would say). This is just one of many examples of monitoring during reading. Unsurprisingly, many of the characteristics of expert reading pertain to expertise in general. Thus, they also overlap with the “pyramid of success”⁵ proposed by one of the world’s best coaches, John Wooden: E.g., industriousness, enthusiasm, self-control, alertness, intentness, skill and confidence.
12.2 Annotation concepts and tools Nowadays, in American schools, students are taught to read as quickly as possible. This is supposed to save time, but I am afraid it also saves learning. Hans Selye This section briefly describes concepts and software to help you identify, find again, comprehend and utilize gems in the knowledge resources you delve.⁶ First, the concepts. When one processes a document, one starts to create information about it— that is meta-information. It is useful to think of some meta-information as being internal to the document and some of it as being external. For example, bookmarks and web address are typically external to the documents they refer to. There are outer tags (also known as resource tags), which apply to entire documents. Diigo and Pocket allow users to tag URLs. OS X Mavericks has a more general tagging facility. CaseApp’s Tags app allows users to tag web addresses from within a web browser. One can also in principle apply inline (internal) tags to snips of texts, images or audio. I wrote “in principle” because this is not explicitly supported by popular applications. I will show how to use existing apps to approximate inline tags. Similarly, one can take notes about a document within or outside of it. For example, many PDF readers (such as Skim.app) enable you to create notes within a PDF file.⁷ The inner-outer distinction is relevant because currently OS vendors do not provide sufficient support for dealing with annotations; so we must invent work-arounds. A meta-doc (meta-document) is any document one creates to elaborate upon, summarize or critique a document. For instance, any information about this book that you’ve recorded in a document—e.g., a word processor document, images, email messages, outliner documents, or just plain paper—are all meta-docs. An annotation is any comment, highlight or tagging about (or of) a document that one makes, whether it is internal or external to a file. ⁵http://www.coachwooden.com/pyramidpdf.pdf ⁶For scholarly research on annotation see in particular Bélanger (2010), Whittaker (2012), and Wolfe (2000, 2001, 2002). I published the introduction
to an empirical research proposal on meta-documentation and self-testing on the CogZest blog. There, I defined meta-documentation generally to include both in-line annotation and external meta-docs. These concepts are explained below. ⁷Some of them (such as Skim.app) can store these notes outside the file, but they remain functionally internal.
229
Delve
To get better results when delving on a computer, I recommend using software in the following categories that meets the following requirements. • A PDF editor that is capable of (a) quickly highlighting text with a configurable color, (b) applying tags, labels or text to selections or highlights; (c) applying other textual and graphical annotations; (d) batch finds of text and annotations; (e) exporting annotations. I recommend the free open-source Skim PDF reader for Mac users. • An e-book reader that supports as many of the foregoing as possible. Apple’s iBooks 1.0.1 (for OS X 10.9) supports many of these requirements. (Contrast my critique of Amazon’s Kindle apps⁸.) • A launcher application (as described in chapter 9). It should enable you to quickly access 90% of the files you might use at any moment within two seconds, without using a mouse. For Mac users, I recommend Object Development’s LaunchBar⁹; but compare Alfred¹⁰. • An editor for creating meta-docs. There is no shortage of options here. Brett Terpstra’s nvALT¹¹ for OS X, based on Notational Velocity¹², provides a highly efficient (and innovative) way of retrieving meta-docs. It also supports MultiMarkdown and other features. • An outliner for structured, detailed meta-docs. The outliner should enable you to perform batch finds, quickly fold and collapse sections, add columns, add and preview attachments, audio record, edit inline notes, access notes, access sections, and create templates. I recommend OmniOutliner Pro¹³ to Mac users. • A drawing application that supports shared layers, stencils and multiple page drawing. I recommend OmniGraffle¹⁴—it’s a lot cheaper than Adobe® Illustrator®, easier to use, and yet very powerful. This can be used to quickly draw diagrams about what you’re delving. • Software for tagging, managing and associating other meta-data to files and Internet addresses. Basic tagging functionality is now included in Microsoft Windows and OS X. You want – to be able easily to tag files across your storage devices and services (e.g., Dropbox) with the same tag set, while avoiding proprietary or opaque database (this rules out EverNote 3.1.2.), – to be able to tag all files, resources and fine-grained information (e.g., bookmarks/web addresses, mail messages, PDF files, instant messaging messages, ebooks, other resources that are not stored in electronic formats such as your paper books); – your tags to be stored with the files (Mavericks and Windows tagging functionality support this) as opposed to only in a separate, opaque database, – your tags to be accessible programmatically, ⁸http://cogzest.com/2013/11/whats-wrong-with-the-kindle-app-a-knowledge-delvers-perspective/ ⁹http://obdev.at/products/launchbar/index.html ¹⁰http://www.alfredapp.com ¹¹http://brettterpstra.com/projects/nvalt/ ¹²http://notational.net ¹³http://www.omnigroup.com/omnioutliner ¹⁴http://www.omnigroup.com/omnigraffle
230
Delve
– to be able to quickly find tags and to find files by tag, – your tags to be synced across your desktop and mobile devices (tablets, smartphones), preferably without storing your data in a cloud service that charges you a significant and unpredictable¹⁵ fee, – your comments about resources to be treated in the same way as tags (with respect to the foregoing bullets), – to be able optionally to share your tags and comments with configurable groups of people. • Software for managing your tasks and projects. For Mac users, I recommend OmniGroup’s OmniFocus. • A text expansion utility that enables you to quickly expand abbreviations into snippets of text. It should make it easy for you to add, remove, edit and find snippets of plain or formatted text and pictures. It should enable you to sync your abbreviations across all your mobile devices (tablets and smartphones). For Mac users, I recommend Smile’s TextExpander. Windows users should check-out PhraseExpress¹⁶ from Bartels Media GmbH. Other things being equal, it’s a good idea to use software that enables or facilitates syncing across your devices.
12.3 Tag entire resources The secret, which I’ve learned from a few years of doing this, is to tag intelligently, sparingly and consistently. Brett Terpstra¹⁷ If a resource is sufficiently valuable that you might want to use it again in the future then it may be worth tagging externally. It seems that most people who tag websites tend to do so in web applications such as Diigo¹⁸ or Delicious. While these systems facilitate sharing of information between users, they are quite limited in several ways that are typical of web services. For example, one has to be online to access the tag information. Sometimes these services become temporarily unavailable (e.g., in October of 2012). Many such services have folded altogether. One can’t tag one’s locally stored files with them. This leads users to maintain multiple tag taxonomies across multiple applications (The keywords you apply in Mekentosj Papers2 for instance cannot be read by any other application.) In contrast, choosing to tag uniformly with support from the operating system vendor enables your tags to be accessible across your desktop computers. The advent of tags on OS X and Windows might eventually lead people to emigrate from services like Diigo. ¹⁵Consider that the rates that are currently charged for storing cloud data may increase in the future. Also, not only must the data be held by a company you trust, you need to consider the possibility that the company may be acquired by another company that you do not trust (e.g., from a hostile nation). There are alternatives to cloud services, such as the free peer-to-peer service, BitTorrent Sync. ¹⁶http://www.phraseexpress.com ¹⁷http://brettterpstra.com/on-sorting-tagging-and-other-nerdery/ ¹⁸http://diigo.com
231
Delve
Tagging resources has several benefits. As noted above, an advantage of tagging over vanilla folder-filing is that you can apply multiple tags to the same resource. Further, you can apply the same tag to resources that are stored in multiple folders. In other words, tags are folder agnostic and enable you to create a “heterarchy” (as opposed to a strict containment hierarchy). This advantage comes at a cost, however. Tagging today requires a hodgepodge of software beyond what currently ships with major operating systems. Figure 12.1 illustrates an OS X Mavericks “Tag…” window involving a local PDF file (Nathalie Sinclair’s Ph.D. thesis). The window displays (and allows me to edit) the tags that I’ve assigned to this resource: “!delve”¹⁹ (to indicate that I want to delve this resource), “/cz/bk/Part 3/32” (the project to which this document is pertinent—i.e., /CogZest/Book/Part 3/ChapterID=32), “$5” (to indicate that the resource is likely to be very useful to this project), and “+5” (to indicate that it is likely to be quite potent). I describe tagging conventions below.²⁰
Figure 12.1 Mavericks Tag Input Window
Having tagged some resources, it helps to have an application, such as Leap or DEVONthink, that allows you to leverage that information. Leap, for example, acts as an enhanced file system manager. It provides and extends many of the functions of Finder, which is to Apple’s OS X what Windows Explorer® is to Microsoft Windows. For example, it enables you to • view a list of all your tags, • search for resources by tag, • list all of the resources to which a tag or fragment of a tag has been applied (using an autocomplete text input field), • sort resources by tag, • view meta-data for each tagged (or managed) resource (such as date created, date modified), • assign tags (with auto-complete), descriptions and ratings to a file. Tagging can help you capitalize on information. More specifically tagging can help you to (a) find resources you’ve previously viewed; (b) mentally encode and elaborate upon resources; (b) classify ¹⁹I follow the convention of prefixing action items with an exclamation mark, reminiscent of the Procedural Reasoning System language (Georgeff & Lansky, 1986). This allows me to rapidly find all my action items in tag browsers supporting type-ahead (incremental) search (e.g., Ironic Software’s Leap). ²⁰This window also allows you to display recently applied tags, your favorite tags and/or all of your tags. Clicking on a tag in those panes adds it to the Assigned Tags pane.
Delve
232
information; (c) describe the resource (though meta-docs are much more suitable for this purpose); (d) express an abbreviated assessment of the resource; (e) otherwise support your projects, plans and actions. To get these benefits, you need to apply the right technology, concepts, and strategies. It helps to have an overarching system. Public information on tagging practices is not particularly useful at this point. For example, as Markus Strohmaier pointed out, most people (who think about the matter) tend to think of tags in factual rather than purposive terms (Strohmaier, Körner, & Kern, 2010; Strohmaier, 2008; Strohmaier & Christian Korner, 2010). OS vendors are largely to blame for users’ difficulties: tagging productively requires a collection of integrated tools and concepts that are hard for most people to assemble. It’s like trying to maintain a complex budget without the benefit of a spreadsheet application. This explains why many people who experiment with tagging give up on it. So here is some help. Topic tagging. The most obvious and common information conveyed by tags is the resource’s topic—i.e., what the resource is about. It’s not always easy to succinctly specify the topic of a document—particularly if it’s complex—let alone to do so consistently. And both succinctness and consistency are important. So be careful with topic tags. A meta-doc can be used to capture additional information. Intentional tagging. By “intentional tagging” I mean tagging a resource with project or task information (or both). Information is so abundant, and computers enable us to switch so easily between tasks, that we sometimes lose track of why we are reading a document. Moreover, whereas we are goal-directed agents, we are also reactive and opportunistic. Seductive lures abound. Yet, as mentioned above, strong reading is goal-directed. Tagging a resource with some indication of the purpose you have in mind in reading it might help you ensure that you do in fact have a clear purpose, and perhaps to reject or postpone the document if you don’t. Also, it could help you to find the document later because you can search for documents that are tagged with that purpose. An obvious problem with such a scheme is that we sometimes only have vague goals. Intentional tagging can be used to counter this problem, and help us develop a deeper understanding of our purposes, particularly if it is combined with action management software, such as OmniFocus, a hierarchical scheme for representing one’s projects, and software (such as TextExpander) for consistently applying this scheme. As we saw in chapter 9 (Identify your projects), apps like OmniFocus enable you to create a hierarchical structure to capture all of your major areas of responsibilities, projects and tasks. So, you can tag files according to the projects to which they pertain. Let me provide a concrete example for my book project.²¹ ²¹I use scare quotes because in OmniFocus my book is not a project but a folder. You can treat a folder as a project or as a collection of projects.
233
Delve
Figure 12.2 OmniFocus Project View for this Book
For each chapter, I have one or more OmniFocus projects (not shown in the figure). You can think of each action as having a “path” (akin to the path of file on your computer or Internet addresses). When I encounter a resource that is pertinent to a chapter in my book, I tag it with the project information (path and optionally ID). I already provided you with an example of this above, when I tagged Nathalie Sinclair’s Ph.D. thesis with “/cz/bk/Part 3/32”. You’ll notice that my File tag (Finder tag) is not identical to my project tag. This is because, unfortunately, there does not yet exist software to easily extract and apply fully-qualified OmniFocus project or task names.²² When I want to find literature on my computer that is pertinent to this chapter, I can type “/cz/bk/Part 3/32” in Leap. As I type that string, Leap filters out the tag list to display all the tags that match it. So, I could just as easily see all the references that pertain to “/cz/bk/Part 3/” in Leap. You might object that this is a very long, difficult to remember, and easy to mistype tag name to apply to a file. This is where TextExpander ²²It would not be difficult to write software to do this. OmniFocus has an AppleScript dictionary. A reference example for this is MailTags which enables users to tag mail messages by project.
Delve
234
comes in. When I want to refer to a project for the first time, I add it to TextExpander. So, when I want to refer to this project in the future, I merely type in the project abbreviation, hit the expansion key, and TextExpander expands the snip for me. Without an application like TextExpander, this scheme would not be practical (unless the OS provided additional support). Tagging by project is a powerful addition to the more usual way of tagging (by topic). You might object, further, that this requires a lot of memorization and familiarity with one’s projects. My rebuttal is that it is not as difficult as it may seem. And using this information frequently helps one to be intimately familiar with one’s own projects and tasks. Given the advantages of being goal-directed and reflective, this is a helpful state of affairs. Furthermore, project information is not merely useful for tagging files. For example, one could use it to name job events in time-tracking systems, as mentioned in the previous chapter. This and many other suggestions listed here have yet to be put to empirical scrutiny by psychologists. However, they are testable, and I invite such tests. A problem with my system is that it introduces a separation between task representations across applications. If you change the name of a project in OmniFocus, it will not propagate to your tagging system, your prior time sheets, and so on. This situation could be improved with additional software. But ultimately, I believe OS vendors should provide functionality to help manage this information.²³ Fortunately, Leap does enable one to rename a tag, and it applies this new name to all items with that tag. We organize our time not only according to projects, but also according to specific activities and contexts. Thus, you can build a reading list by tagging items with tags denoting the type of task you intend to perform next with them, such as “!read”, “!inspect”, “!post”, “!delve”, “!buy”, “!capture”. An exclamation mark at the beginning of a tag indicates an action. So, when you have only enough time to skim documents, you can refer to your “!inspect” tag. If you want to do some serious reading, you can choose from the “!delve” list. When preparing for a trip or otherwise deciding which books to buy next, you can consult your list of ebooks to buy for the road (“!buy”). You can refer to multiple tags to help you make your decision. The same action tags can be used across different projects. Resource-rating tags. The previous chapter proposed categories for assessing resources: caliber, utility, potency and appeal (CUPA). Whether you use that system or your own, you can systematically rate documents with tags. The key is to be systematic about it. Don’t expect to be able to assess documents easily if you’re not used to doing this. Even being systematic about assessing gustatory taste is difficult, though of course it’s comparatively a lot simpler. OpenMeta provides a 5-star, scalar, rating system. If you use this, be sure to settle on what the ratings mean to you. If you use them sometimes to refer to caliber and other times to usefulness, you won’t be able to infer much from the rating. You can use tags to rate documents. I use “+” to designate the potency of a resource and “$” to designate its utility. A utility rating by itself does not mean much. It depends on the project. Thus, when you apply a utility tag, consider also applying a project tag, to express the project in relation to which it is useful. You might be surprised that I recommend scalar ratings given that I have frequently argued that numeric ratings are of limited value. I believe such ratings are potentially useful, provided they ²³Given the open source LinkBack project, Mac software vendors can enable users to obtain the address of objects they manage and to access those objects. It is reminiscent of the innovative OpenDoc project that Apple launched and canned in the 1990s. See http://x-callback-url.com for iOS call-back information. (Compare Steve Job’s equanimous response to a provocative question about why Apple nixed OpenDoc.
Delve
235
are applied properly and used in the right way. Multiple ratings are better than one (e.g., separating utility from potency). The process of rating a document helps to ensure that we think about resources in terms of the value they might provide to us. Ratings do provide a quick and dirty way of filtering reading lists. This is important given the amount of information we process. They can help us gauge how much of our time is dedicated to low or high quality resources. Still, one should keep in mind the limitations of numeric ratings that I described in the previous chapter. Such ratings need to be backed by qualitative judgments. For as implied by this section’s opening quotation, the problem with bad reading is that it reflects and promotes poor thinking. Meta-docs and inner annotations enable us to capture some of our more elaborate judgments. Rating processing. You might find it useful to occasionally rate the depth of your information processing. You can use this information to ensure that you spend enough of your time delving, as opposed to merely surfing and inspecting resources. You could apply this information to resources as tags. Or you could include this information in your time-tracking. Final caveats. As of this writing, Mavericks Finder tags are not normally managed by iOS devices. For example, suppose you tag a PDF file on your Mac with the label “Productivity”. You use Mekentosj Papers, bibliographical database software, to sync and read this file on your iPad. In Mekentosj Papers on your iPad, you’d like to access all files tagged “Productivity”. Alas, Mekentosj Papers does not yet expose this information. If you apply a Mavericks tag to a file, when choosing synchronization services, you need to ensure that they at least preserve Apple’s extended file attributes. DropBox®, iCloud® and DEVONthink all preserve this information. The same applies to syncing Windows Explorer tags on mobile devices. A final noteworthy problem is the lack of proper support for hierarchical tagging. Hierarchical tags, e.g., “/cz/bk” need to be managed manually. You cannot change a node in the hierarchy in one fell swoop (but this can be automated through scripts). I’m not suggesting that you apply multiple tags to every document you process, or even most. Merely making a mental judgment of some of the information (e.g., the document’s potency and usefulness) is often adequate (e.g., for meta-management).
12.4 Tag snips of text and images A distinguishing feature of effectiveness in a domain is mastering a huge number of its concepts. To be an effective knowledge worker one must be an expert delver. This involves wielding a huge number of schemas. Effectiveness is not conferred upon us by the master of a graduation ceremony or professional accreditation authorities. It comes from years of carefully processing documents and explicitly classifying information. This section is designed to improve your meta-effectiveness by showing you how to use technology to explicitly categorize relevant snips and knowledge gems. Until recently, tools to implement these techniques, and study their use, were not widely available.²⁴ Many people, when reading paper documents, highlight text. However, simply highlighting text, even effortfully, will not build understanding—and how could it? When I was an undergraduate ²⁴See Beaudoin & Winne (2009) for a description of a platform to support and research delving.
Delve
236
student in the 1980s, I devised a simple scheme for classifying what I read (necessarily, on paper) with different color highlights: • Yellow highlights for significant claims made by the author, • Pink highlights for ancillary claims, • Green highlights for important technical terms (nowadays, in textbooks they are often presented in bold), • Blue highlights were wild-cards. I supplemented this scheme with an extensive short-hand annotation system. For example, tilde (∼) in the margin indicated that I disagreed with the author. Double tilde (∼∼) meant that the author was criticizing someone, relating a counter-argument, or offering a rebuttal. The advent of HTML and PDF threw my system—and surely that of millions of others—for a loop. As I mentioned in chapter 3, due to the lack of annotation capabilities in web browsers and PDF apps, I was frustrated to find that my colleagues and I had to revert to printing. After receiving a golden handshake from troubled Redback Networks Canada in 2001, I resolved to understand and address the requirements knowledge workers face when trying to learn with technology. This called for an annotation system integrated with a productive practice system. I have been designing and developing solutions to these problems for more than a decade. I’ve also been practicing what we preached. Meanwhile, PDF and web technology has improved. Modern PDFs readers enable users to highlight noteworthy snips of text. Alas, that’s not much better than what you can do with paper. Unless you have a multi-colored scheme (or tags) for categorizing what you read, highlights don’t provide much value. Fortunately, some PDF readers and e-book readers do allow you to change the color of highlights. (Skim supports multi-colored highlights.) So you could in principle implement a scheme such as the one I summarized above. However, these applications still don’t allow you to change your highlight color as fast as you can a physical marker. Moreover, it’s difficult to keep track of more than 4-5 color assignments. Experts mentally apply a much larger number of categories to text than that. Even experts are being held back by technology. Many still prefer to print important documents than to read them. Others use technology grudgingly for random access to resources and other secondary benefits. I will show you how you can use existing software to make them much more suitable to delving. I hope this will give you a richly expanded view of how to capitalize on documents and that you will start demanding more of software vendors. This discussion of delving focuses mainly on PDF. The reason is that you can convert most documents (web pages, emails, ebooks, etc.) to PDF. For example, the Calibre application²⁵ enables users to convert ebooks to PDF. You’re better off reading documents in one powerful application (e.g., Skim) than switching between applications.²⁶ As of the time of this writing, Skim is a much more powerful delving tool than Amazon’s Kindle application. Moreover, it’s free, open source and extensible. Apple’s iBooks 1.0.1 on OS X supports more of the system described here than Kindle ²⁵http://calibre-ebook.com ²⁶PDF is not inherently better than e-Pub. e-Pub has more potential. However, e-Pub apps have not yet caught up to Skim.app. A reader could be
developed to support the functionality described below for multiple formats (PDF, e-Pub, etc.)
Delve
237
(1.10.1.)²⁷ For example, iBooks also has a batch find; it also supports pasting information from the book into a separate meta-doc. As you delve text, you constructively respond to it. You analyze it, you elaborate it and you criticize it. You can record some of these judgments by applying fine-grained qualitative tags. Elaborations can be captured by annotations in the document itself (inner-annotations) or outside the document (meta-docs). Here follows a tripartite taxonomy of tags one might apply to content one is delving. The classes are somewhat arbitrary. You may organize your tags as you wish. By consistently using a collection of tags you will later be able to extract the information from the document much more easily than you otherwise could. In effect, you will be able to construct multiple views on a document. So, you will be better able to use its information for your own purposes. There are other benefits of tagging mentioned below. First, there are the formal categories. These are the simplest ones. For example, a scientific researcher can quickly orient himself to the structure of a paper with its introduction, methods, results, discussion and conclusion. Lawyers, engineers and others have their own formal schemas. Delvers also pick out more subtle rhetorical structures (the components of arguments), references, cross-references and other information. These structures help them perceive and interpret the following types of semantic and evaluative information. Second are the descriptive factual categories. They are used to classify the text and to describe what the author is saying and why he is saying it. Amongst them are classifications of propositions (i.e., ideas expressed in sentences or propositions within sentences). They include • The goal of the document and problems addressed by its author(s), • Major claims that are part of the thesis of the document or that you consider as being a major “take away” from the document, • Ancillary claims that support the major claim. They might be warrants for an argument, for example, • Amongst the last two, there are empirical findings (or results of studies, e.g., a mean), principles, rules, generalizations, axioms, laws, equations, etc., • Hypotheses and conjectures. The same proposition can function as a claim or a hypothesis, depending on the author’s attitude towards it. • Examples. Terms and concepts are a special kind of factual knowledge. Potent new concepts are often the most important type of information we can extract from a document. In other words, they can have the most significant and lasting impact on our understanding and problem solving abilities (our effectiveness). (In Gauge its potency, I discussed the importance of acquiring potent new concepts. I expand on this in the Section, Master concepts and vocabulary, below.) A document can be analyzed into its major concepts. And so, we are especially attuned to detecting them, their importance, and ²⁷PDF is not without its own problems. For example, some scholarly publishers, intentionally or not, output PDF whose text when selected contains no spaces. Optical character recognition software is still not perfect. Thus documents converted to PDF cannot always easily be searched and manipulated. Effectant readers should demand more of publishers, and favor DRM-free publishers such as O’Reilly, even if it means paying more in the short run.
Delve
238
their relations to each other. This information can then be utilized in concept maps²⁸. It is useful when tagging text to have separate tags for terms and concepts. In contrast to terms, concepts can be conveyed by long snips of text, including semantic criteria and conceptual distinctions. By having separate tags for each category, you can quickly extract the document’s key terms. Finally there are assessments of information and our motives (wants, wishes, goals, intentions) to act upon them. For example: • Information that we don’t understand. Our knowledge gaps and flaws. We saw in chapter 7, that the most successful students are those who detect and respond to their knowledge gaps. By applying a “knowledge gap” tag to a resource while processing it, with a few keyboard and mouse gestures (within 1-3 seconds) you can obtain a list of all of the information that you don’t understand about the resource. In other words, you can filter your tags to focus specifically on your knowledge gaps. • That information is new or surprising. • That information is appealing (e.g., beautiful, etc.) When you review a paper, you might not recall the particular bit that was beautiful in it. But you might recall that there was something beautiful in there. Using a tag filter, you can quickly access the part of the document that previously elicited an affective response from you. • That information is potent (i.e., information that when understood may lead to the greatest mental development.) • That information is humorous or playfully expressed. That a paper makes a point in a humorous fashion is often memorable. However, as anyone who has had trouble recounting a joke, that doesn’t guarantee that you can remember what the exact point was. On paper, you could add a little smiley or “lol” in the margin. With our annotation system, you can flag it as funny. When you later return to the paper you can zero-in specifically on the part you found funny by searching for the “funny tag”. There are many ways in which something might be funny or playful. Unless you are specifically analyzing humor or play, then it suffices to have one tag to capture related concepts. The same goes for all tags: you need to pitch them at the right level of abstraction, so that you can apply them quickly and search efficiently for them. • The author’s criticisms (along the lines of the above) of others’ conceptual artifacts. (Firstorder criticism). • Your criticisms and “red flags” about information that seems weak, implausible, illogical, false, or flawed in some other way. Or the author noting criticisms that others may have of his own claims and arguments. (Second-order criticism) • The author’s rebuttals. (Third-order). • Action items. You can tag text as a “to do” in cases where you want to follow up on it in some manner, e.g., to reread, to email someone about it, or to use in a project or document. You could add a document to OmniFocus and then use Skim to find those action items. You can also create temporary tags for special purposes. For example, if a paper presents a list of techniques for inducing sleep, you can tag each snip that describes a sleep-induction technique. Then, by applying a filter, you can extract all of the sleep-induction techniques from the paper. ²⁸http://en.wikipedia.org/wiki/Concept_maps
Delve
239
This tagging system provides you with the ability to view a document from multiple perspectives, i.e., to filter it according to different tags. The concept of viewing information from different perspectives has not been sufficiently exploited outside of computer science (e.g., Model-View-Controller architectures²⁹, separation of concerns³⁰ and XML³¹ in relation to CSS).³² For example, having tagged a document’s text, you can later quickly reconstitute its argument. Reverse-engineering arguments is one of the most important ways of comprehending documents (Wiley & Voss, 1999). The kind of tagging I propose is different from what you would find in the subject index of a book. The latter is a detailed list of topics covered by the book, whereas tags can be used to indicate that information matches a general schema. I’m not suggesting that you should completely tag each snip of text that meets the criteria of your taxonomy. It is sufficient to tag information that is noteworthy and that you may want to refer back within a few minutes, weeks, or years. You will implicitly call upon your evaluation schema. Here’s how to tag text with Skim, a popular and free PDF reader for OS X. Skim does not yet explicitly support tagging, so my method exploits its other annotation features.³³ A Skim window has three panes: • A Contents pane on the left, • A PDF pane in the middle (which shows the PDF file you are reading), • A Notes pane on the right. This displays each snip you’ve highlighted—one line per highlight. It also displays inner notes. Highlighting text with Skim is easy. Just select the text and issue the Highlight Text command with your keyboard or mouse. Skim will apply the (configurable) default highlight color to the snip in the PDF pane. You can easily change the color of a snip (e.g., by using the OS X Color Palette). Skim creates an editable note object for each snip you’ve highlighted. These notes get displayed in the Notes pane. In order to make a highlight function as a tag, prefix the note’s snip with a tag. For example, if you encounter a key term, you can highlight it and prefix the annotation with “Term: “. This can literally be done within a couple of seconds. You can apply the same technique with all of the tags you commonly use, such as the tags I listed above, for example, “Goal of the paper:”, “Finding:”, “Hypothesis:”, “Criticism:”, “I disagree:”, “I don’t understand:” (“?:”), “!re-read:”. Skim’s Note pane has a filter that enables you to list only notes that match your typing. So if you want to find all the snips you don’t understand in the document, just enter “I don’t understand:” in that field.³⁴ When you double-click on an entry in the Note pane, Skim scrolls the PDF pane to display the note (and snip) associated with the entry. So you can quickly focus on the information you’re interested in. ²⁹http://en.wikipedia.org/wiki/Model-view-controller ³⁰http://en.wikipedia.org/wiki/Separation_of_concerns ³¹http://en.wikipedia.org/wiki/XML ³²I expect it to be possible in the future for readers to request a specific perspective on any arbitrary document. For example, one will be able
to specify the number of paragraphs to view and the document will automatically be summarized to that specification. One will be able to ask for a resource’s argument, major claims, hypotheses, concepts, references, and/or counter-arguments, etc. Some of the meta-information required to serve these perspectives will be embedded a priori in documents by their authors and publishers. Some of it will be obtained via crowd-sourcing of annotations. Some of it will come from automatic text summarization. This will facilitate, but not replace, delving. ³³Hopefully, after the publication of this book, PDF and ebook readers will properly support the features I’ve described. ³⁴I use the shorter, form “„?:”
240
Delve
Some PDF files come with a table of contents; but some do not. If you’re reading a very important but difficult document, you can easily create your own table of contents by selecting headers and marking them as such.³⁵ (You can export this to your meta-doc.) Then, you can filter the Notes pane to reveal the table of contents. If you’re writing a critique of a document, at some point you will want to focus on all the items that you disagree with. In an instant, you can get a list of every passage in the book that you disagree with, if you have tagged them, by filtering the Note pane appropriately. (I use the tilde symbol, ∼, as a tag for this.) Of course, no one wants to type in long snips such as “Goal of the paper:” while reading. That would interfere with comprehension and consume too much time. Besides, it’s error prone. Your tagfiltering won’t be effective unless (at least the first few characters of) each category’s tag is spelled consistently. It would not be difficult for Skim developers to add a popup menu for tag filtering. Meanwhile, abbreviation expanders, such as TextExpander, solve this problem for us. All you need to do is define an abbreviation in TextExpander for each tag you commonly use. That way, you can quickly add tags to snips, and you can quickly filter your snips by tag in the Notes pane. Experts in every discipline have countless schemas. TextExpander users often have hundreds of abbreviations. One can easily manage upwards of 30 tags. You can even have multiple abbreviations for each tag. When I said you can apply a tag to a highlight within a couple of seconds, I assumed the use of text expansion software. This opens the door for you to create a hierarchical tag system. For example, you can create tags for major claims like this: • • • • •
Major proposition:Argument: Major proposition:Claim: Major proposition:Thesis: Major proposition:Problem: Major proposition:Goal:
This way, you can search for major propositions in a document, and then narrow them down. You can apply this organizational principle to all of your tags. I prefer to prefix my tags with a unique string, “„”, to prevent matching false positives.³⁶ You can apply tags not only to the author’s snips of text, you can also tag your own notes. For example, if you want to write a summary, critique, or action item about the document, you can create a new anchored note. Anchored notes have a title field. This field is nearly useless, in my opinion, unless you use it to insert a tag. So, you can title your note with a tag, and then enter the content of your note in the note’s text area. Later, within two seconds, and without even using the mouse or touchpad, you can obtain a list of your criticisms, actions, summaries, or any other type of note you’ve added to the document. The section “Write meta-docs” below provides tips on using PDF annotations when creating meta-docs—documents about knowledge resources—to help you get more out of your delving. ³⁵Some PDF tools enable you to create a table of contents yourself. For example, with Smile Software’s PDFPenPro®, you simply select a header in the contents pane (on the right) of the PDF file and issue a keyboard command. PDFPenPro then adds a table of contents entry (on the left) that is linked to that header. ³⁶An advantage of using „ as a delimiter is that it is readily available on the iOS keyboard, without needing to use the shift key. Typing two commas in a row is fast.
Delve
241
Tagging has many benefits. It might not only help you better apply your existing schemas, but also create and refine them. It helps you become reflective and strategic about what and how you delve. It also facilitates text summarization. You can use tags to extract the elements of a text and your responses to it. With improvements in technology, it would be possible to build links from any part of a document to any other resource. And you would need to experiment with the practice yourself to make up your own mind. I’m not suggesting you do all of your elaborative thinking about and annotating of a document with a PDF reader. Notes in PDF readers are too closely tied to the text. Their text editors are limited: they don’t support outlining or rich formatting. When you are developing an argument about a document, you need to alternate between focusing on your argument and focusing on the text. You need to write meta-docs about the document. Working on a meta-doc frees you from the clutter of the original resource and makes it easier to focus on your ideas about it. When you need to return to the original document, you can quickly do so using a launcher app. Unfortunately, Skim annotations are not currently usable outside of OS X. However, there are several mobile PDF readers that can be used in a similar, if not equally sophisticated, way as described above. Given the competition in this market and current rate of improvement, it is reasonable to expect functionality described above to be implemented on mobile platforms.
12.5 Write meta-docs As I mentioned previously, a meta-doc is a document you compose about a knowledge resource to help you understand, assess and utilize the knowledge conveyed by the resource. I suspect the reason why most people skip out on creating meta-docs is that they don’t have a system to quickly create, access and use them. If you’re processing many knowledge resources, you need to be able to create a new meta-doc within a couple of seconds, whether the target is a presentation, a previous phone call, an e-book, a paper book, a screencast, a podcast, an audio book, or anything else. You also need to be able to retrieve the meta-doc very quickly. This is explained below. As indicated in chapter 9 (Learn your way around your meta-information), it’s useful to be able to create all kinds of metadocs: spreadsheets, diagrams, outlines, plain text documents, audio recordings and even one’s own program. Like the knowledge resources to which they are responses, meta-docs can also be created and stored on mobile devices. When you want to take notes about a resource, rather than starting from an empty meta-doc, consider starting from a meta-doc template. A meta-doc template has fields or sections for types of information you might want to express about a resource, such as your objectives and assessments and its problem, major claims, argument, methodology and results. First, I justify the use of meta-doc. Then, I provide tips for creating and using meta-doc templates. While you can write meta-docs without using templates, using templates has several benefits. Like any other template, meta-doc templates enable you to structure your documents in a consistent way and to proceed systematically—i.e., they support developing and applying one’s best practices. They encourage delving (as opposed to surfing) knowledge resources. They help remind you of questions to asks about resources. They help you create and sustain a propensity to ask the right questions about resources. This benefit will transpire even when you don’t write meta-documents—
Delve
242
which is the majority of cases—because it creates the right thinking habits. Also, as technology evolves, templates will help you automate certain tasks. For example, it may soon be possible for apps to exchange information between the target resource and meta-docs. Apps would be able to extract the table of contents, glossary and all kinds of other information from (compliant) knowledge resources. This would enable you to write your notes within an outline of the paper, without having to manually copy them to the outline. So, if you have a comment about Section 6.3 of a document, you could write your comment in that section of its meta-doc. You’ll be able to collapse that section and use other outlining functions, as described below. Given that delving is a major component of knowledge work, you may find it useful periodically to revisit your templates, just as expert golfers like Tiger Woods often revisits aspects of their swing. Using a meta-doc template will help you orient yourself when revisiting your meta-docs in the future. I’m not suggesting that you ought to compose meta-docs for every resource you delve. Often inline annotation or resource-level tagging is sufficient. Nor am I suggesting that one’s notes should always be very extensive. However, if (for example) you listen to several podcasts without taking a single note about them, you need to ask yourself whether they were worth listening to. Will this way of processing those resources give you the lasting benefits you seek or ought to seek? It baffles me when I attend a high caliber seminar or an important meeting and I look around and no one is taking notes. If it’s not worth capturing information about an event, was the event worth attending? I know the information will be forgotten. (Compare the meta-cognitive illusions described in chapter 3.) Some people think that taking notes distracts from comprehension. This depends on one’s skills as a note taker, which in turn are a function of experience and effectance. I will first describe an elaborate meta-doc template. Most of the time you would use only few of its elements. You’d use different ones at different times. This template is written for people who deal with all kinds of knowledge. It can easily be adapted for specific disciplines and for academic delving.³⁷ More important than the specific template I describe are the principles for developing and using them. These principles will survive over generations of technology. For elaborate meta-docs, I recommend using a special-purpose outlining app rather than a general-purpose word processor or plain text editor. Above, I listed a few requirements of outliners and recommended OmniOutliner Pro for Mac and iPad users. Windows users should consider Microsoft OneNote®³⁸ and UV Outliner³⁹. Many children are introduced to outlining in school; but most adults today aren’t yet familiar with outlining technology. If you’re one of the latter, I recommend you try one out for a few weeks. They can improve your cognitive productivity. There are plenty of plain text editors⁴⁰ to choose from for outlining purposes. For Mac users, nvAlt⁴¹ makes it very easy to create and access meta-docs. ³⁷For example McKinney (2005) and Wallace & Wray (2011). ³⁸See the LifeHacker article on “Five Best Outlining Tools”. ³⁹http://uvoutliner.com ⁴⁰http://brettterpstra.com/ios-text-editors/ ⁴¹http://brettterpstra.com/project/nvalt/
243
Delve
12.5.1 An elaborate meta-doc template This section describes the kinds of information you might include in an elaborate meta-doc template. Each paragraph could be captured in a separate section of the outline. You will notice that this expands upon the information that you glean at the inspection stage of information processing (see chapter 10, Inspect).⁴² It’s useful to include a column in one’s meta-doc template to store index information (page number, %, or minutes.) You can later use this information to look up the original information. For example, earlier I referred to a comment on aesthetic judgments of science in episode #73 of the Rationally Speaking Podcast. That reference was about 52 minutes into the podcast. I made a note of the aesthetics’ comment, including its index. Thereafter, I can quickly access the relevant frame. (Compare the following screenshot.) A good meta-documentation tool would automatically record this index for you. A good podcast tool would support all the annotation features described earlier. Each anchored comment would be a hyperlink to the frame in question. This is particularly useful for multimedia and audio resources, which are difficult to search.⁴³
Figure 12.3 A Portion of a Meta-doc (Note the index column)
Figure 12.4 is a screenshot of an empty meta-doc in OmniOutliner. Template sections are in blue (or green for assessments). Text written in the instance document⁴⁴ itself is in black. That enables you to easily distinguish the meta-doc content from boiler plate headings. ⁴²Electronic versions of this meta-doc template are available as “extras” on Leanpub.com and at CogZest. ⁴³Phil Winne, colleagues and I developed such a tool (gStudy) at Simon Fraser University. It allowed users to link from any bit of text, frame or
portion of an image to any other. Each anchored annotation was a hyperlink to its target. Beaudoin (2010a, 2010b) implores OS vendors to support such annotation and linking capabilities. ⁴⁴An “instance document” is a document that implements a template. Instance documents are derived from templates. Compare van der Vlist (2002).
244
Delve
Figure 12.4 An Example Empty Meta-doc (Triangles indicate there are subsections)
I recommend you use a convention to make it easier to find information you include in metadocs. That is to prefix some of your meta-information with tags, such as „ (two commas). For example, where you record your criticisms, you could use „∼:, where you record your action items, you could use „!, where you note terms, you could use „Term: . That way, you can use the batch find tool to find all your criticisms, action items, terms and whatever other information is in your tag taxonomy. You can re-use the taxonomy and TextExpander shortcuts you developed for annotating PDF files⁴⁵ as described in Section “Tag snips of text and images” above. Next up is a description of the meta-doc sections. Generic meta-information. This section contains whatever highly generic information about the resource you want to include. This might include its URL, title, name, and year. If you are using a citation manager (such as Mekentosj’s Papers or Mendeley), you can copy the entire citation into this field. If the book or its table of contents is available online, you can capture the URL here. With this information in place, you can later execute a Spotlight search (in OS X) to quickly find this meta-doc. For example, given the URL you can quickly find the meta-doc. In a following section, I will show you how to name meta-docs for even more rapid retrieval of meta-docs. Scratch pad. This section is for free-form comments when you can’t be bothered to use the structured sections. For most resources, this may be sufficient. To facilitate future retrieval, you can still tag information. For example, if you want to make it easier to find your criticisms, you can prefix them with your criticism tag (e.g., „∼:.) Project. Delving a big and complex resource is a project—but then so is delving a smaller one. This section is for recording project information. Your rationale for processing the resource ⁴⁵Again, you can use TextExpander to record this information. However, if your meta-doc software happens to support inner tags, then you don’t need to manage meta-information this way. No Mac outliner does yet. Thus, the situation is the same for PDF readers as it is for outlining applications: they don’t support inner tagging yet. The system I have developed allows you to get around this limitation.
Delve
245
summarizes the projects,⁴⁶ problems, goals or questions that make this resource worthwhile in the first place. Whether you record it or not, articulating this consciously may help you determine whether, when and how to process the resource. This may help you capitalize on the document. Expert delving is purposive. Most practical books on reading for adults are aimed at studying for exams or writing academic papers. There’s a much larger variety of non-academic R&D purposes. One can’t fully predict what will be in a resource, and so one needs to look for unforeseen knowledge gems worth capturing—i.e., new problems, concepts, claims, etc. One needs to strike a balance between opportunistic ransacking and premeditated delving. You can also make a note of your action items (to-do lists) and plan for processing this resource. For instance, you might decide to read chapters 3-6 and skip the rest. If you are interrupted and come back to the resource weeks later, this information can serve as a record about what you wanted to do next with this information, particularly if you order your action items in a consistent way (e.g., with the next thing to do being the first item in the to-do list). Sounds like overkill? Consider that a major reason that interruptions are so costly is re-orientation. Like other meta-information, this information can be splattered throughout the document if tagged properly. For example, if you tag action items with “„!”, then you can use a batch-find command to quickly list all of your action items. You might also want to make an explicit note of any pre-conditions or post-actions.⁴⁷ These are things you want to do before and after delving the document, respectively. Recording post-actions is consistent with the GTD framework, wherein it’s important to organize information in a way that helps you decide what to do next. This project information is also (and even more) useful in other project templates (e.g., for writing your own primary resources.) Of course, you could alternatively include project information in your action-management system (such as GTD). But then you need to resolve the meta-access problem, to quickly be able to locate the OmniFocus project for this particular resource. The bottom line. It’s often useful to make a mental or electronic note about the take home message for a document. This could be a summary. The “rule of threes” is sometimes cited as “the single most important rhetorical device” (Morgan, 2009) in public speaking and writing: keep your lists to threes if possible. Nature cannot always be cut in three parts. Hence authors routinely exceed this number. Paul Terry, an Abatis co-founder and CEO of Polar Systems, explained that this rule applies to processing reading materials. Board members must make critical governance decisions based on long and complex documents. Hence “simplify in threes” is advice David R. Beatty gives to his board certification students to help them deal with board materials (Terry, 2011). This helps the reader to accommodate to the information, a concept I described in RR in reverse: The problem of instilling mindware. Such a summary might also include the upshot or implication of a document. In other words, what you want to do with the resource. Outline. For very serious delving, it’s sometimes useful to copy the entire table of contents into the meta-doc and use this as a basis for outlining the document. This way, you can insert your comments directly in the sections to which they apply. This also provides a very convenient way to learn the structure of the resource. Good readers are sensitive to the structure of a document and can use the outline to help understand the gist of it. Outliners allow you to expand and collapse entire ⁴⁶As mentioned in the section “Tag entire resources” above, this information could instead be captured in the tag applied to the entire document. ⁴⁷I’ve borrowed these terms from early AI literature on planning.
Delve
246
sections of an outline. OmniOutliner even allows you to hoist a row in the outline. This enables you to focus specifically on one section. This supports a well-known disposition of high performers: they focus. But first, you need to get the table of contents into the document. Fortunately, it’s often easy to obtain the entire table of contents of a resource, either on the Internet or from the resource itself. Usually, this requires a bit of massaging of the text, to remove tabs and spurious spaces. This is best done with a powerful text editor—such as Bare Bones Software’s free TextWrangler or their BBEdit—that supports regular expression pattern matching. This may seem daunting at first, but there are plenty of resources on the Internet describing how to do this.⁴⁸ You do not need to be a professional programmer to use regular expressions. If you can’t easily obtain a table of contents for your resource but you have access to a PDF version, then you can easily construct the table of contents yourself using a powerful PDF reader such as Smile’s PDFPen Pro. Skim does not currently contain a table of contents editor; however, you can use the tagging procedure described in “Tag snips of text and images” to create a table of contents that can easily be exported to a meta-doc. You may wish to reserve a section of your meta-doc for an analysis of the resource, including the following types of descriptions: • Free form comments. This could include comments inspired by the following bullets or whatever comments you want to record about the resource. • A more or less detailed summary. • The problem and objectives addressed by the author and whether or why they matter. However, you might not be interested in the author’s problem yet still find it worth ransacking for your own purposes (discussed above). You’re likely not reading this paper for a review or term paper, after all. • The argument. One of the best ways to understand and assess a document is to analyze its arguments. This is where one identifies the major claims (or thesis) and their grounds (premises or evidence). One might also make a note of the warrants, i.e., the rationale that ties the grounds to the thesis, and the backing for the entire argument. Bereiter (2002a) claims that experts don’t need training on how to think critically about the work of others (chapter 10). They’re quite good at criticizing other people’s work. However, I believe there’s usually room for improvement. It is important to activate one’s fallacy monitors. Moreover, today’s mainstream information processing technology is poorly designed for extracting and analyzing arguments. This is another reason why some people prefer to print important documents than to read them on a machine. In a previous section I provided tips on how to extract argument information from a PDF file. If you tag claims, grounds, counter-arguments, and the like, you can quickly filter the PDF for these components. This can help you summarize the argument in the meta-doc. • Ancillary claims. You might find it worth noting claims that are not central to the argument but useful for your own projects. ⁴⁸See Goyvaerts & Levithan (2012).
Delve
247
• Methodology, results and figures. It’s often useful to summarize methodologies if they are particularly complex or innovative. Some readers find it so difficult to manipulate tables and figures presented electronically that they either print them out or, worse, skip the analytical step that was once natural to them. Many reader apps, like Skim, have markup tools that enable you to graphically annotate figures. (Alas, as of the time of this writing, the iBooks and Kindle do not even have an embedded graphics tool.) Even in PDF readers, graphic tools are very limited compared to powerful diagramming, charting, and visualization applications such as OmniGraffle, which is available on OS X and iOS devices. So, it’s often preferable to insert the document’s tables or figures directly into an OmniGraffle diagram. If the license allows you to do it, just take a screenshot of the page and import it into the visualization app. Once you get the hang of it, you’ll find it only takes a few seconds.⁴⁹ You can then manipulate and annotate the information to your heart’s content. In the next section, I describe how you can keep such meta-docs organized for rapid future access. If there’s ambiguity in the data or methodology you can often email the author directly to resolve it. It’s sometimes useful to fire up a spreadsheet application to ensure you comprehend the paper’s results. • Glossary and concepts. It bears repeating that some of the most useful information to harvest from a knowledge resource are its terms and concepts. If you attend a seminar in a field with which you are not familiar, you might get the impression they are speaking a different language. Concepts help you see the world in new ways. Yet many people don’t have a system for reliably mastering new terminology and concepts. We saw in a previous section that important terms can be tagged in Skim for future harvesting. It’s sometimes useful to make a note of the most useful concepts in the meta-doc, particularly if you are reading a paper document or using a DRM-laced e-reader such as Amazon’s Kindle. You can have one row for each noteworthy term. You can then add your comments in child rows. (A child row is a row that is below its parent and indented from its parent.) For example, you might copy or paraphrase the definition and provide your own examples. But what if several months later you want to refresh your memory about the meaning of the term and the brilliant examples and comments you made about the concept? If you’re a voracious reader, you might be hard-pressed to remember the resource that defined this term. If you were to search for the term on your hard disk, there might be many false positives to sift through. Thus, you need a way to find annotated instances of terms. This points to a major gap in mainstream productivity software. Apple’s OS X has a dictionary application. But it does not provide an interface for commenting on terms or adding new entries. Sure, you can use an Internet search engine and sift through its results. But that will not provide you with information you’ve stored locally. One solution is to prefix the term with a term tag, as described in the previous section. For instance, searching for “„Term: Immanence Illusion” on my machine would lead me to find my comments on this term. Another solution (that can be combined with the previous one) is to maintain your own file or database of technical terms. This way, your semantic information ⁴⁹ObDev’s LaunchBar, for example, has an action called “Copy Screenshot of Selected Area” that can be accessed in four keystrokes (c. a second). Once you have issued this command, you just draw a rectangle over the area of the window you want you to capture. Then you access your OmniGraffle meta-doc (again, within a few key strokes) and paste the screenshot in your OmniGraffle meta-doc.
Delve
248
is not locked up in particular papers. The next section contains a template to help you analyze potent concepts in documents. Section Master concepts and vocabulary above, shows you how to use a productive-practice system to keep track of, and master, conceptual knowledge gems. When you encounter such gems, consider adding them to your productive-practice knowledge-base. • Finally, you might want to keep track of noteworthy references. You can dump them in a meta-doc and flag references as “!get” or “!read” for later action. However, I also recommend using a citation manager, such as Mekentosj Papers2 (available on both Windows and Mac). Even if you don’t publish papers yourself, it’s a very handy application for storing knowledge resources and information about them. Moreover, it allows you to sync your PDFs to iOS devices. Unfortunately, it does not support Mavericks tags and it is not scriptable. The previous sections of this template are designed for factual information, to help you comprehend, analyze and apply knowledge resources. You can enter evaluative comments in metadocs as well. As I argued previously (in agreement with John L. Austin), there is not always a sharp distinction between description and evaluation. For example, to describe an argument as inconsistent is both factual and evaluative. If you include critical remarks in the previous section, you could tag them with a “criticism tag (I use “„∼:”). That way, when you want to review your criticisms of a resource, you can use your outliner’s “batch find” feature to list them all in one go. The assessment section is where you assess the resource. Here, you can make a note of the various strengths and weaknesses of the resource. As I argued in the previous chapter, whether you use my categories or different ones, the main thing is to have a systematic way to assess documents. You could reserve one row for each major category of assessment: Caliber, usefulness, potency and appeal. These categories are explained in detail in chapter 11, so I won’t elaborate further, except to say that “usefulness” is where you can express how to use this document for your projects. This can be compared to your initial reason for reading the document which is captured in the rationale rubric above. Strong readers are reflective, meaning that they assess their own processing of information, whether they do so consciously or not. When they encounter a flaw in their knowledge worth repairing, they tend to generate a motive to repair it. If you are using the right tool, such as Skim, you can tag information you don’t understand. If you’re processing a DRM-laced resource, such as a Kindle book, it’s probably best to record knowledge gaps in the meta-doc so that you can more easily process it later. In order to capitalize on useful new information, it’s useful to flag it as such. But beware: fluent readers are subject to several illusions, such as the illusion of comprehension or familiarity, as described in chapter 3. (The “I knew it all along syndrome.”) Related information is captured in the project plan section above, where you keep track of where you’ve left off and what you’d like to do next with the information. So, you could also keep this knowledge gap information in the project section. Some reflective information is also captured in the sections on potency and appeal. Comparative analysis. This is where you capture whatever you find most noteworthy regarding the relations between this knowledge resource and others. If the author has written several documents on the same subject, how does this document stand out? Is this an original contribution? What would the authors of some of the most pertinently related documents say about this one? How
249
Delve
would they object? What would they like? How can this document help make sense of phenomena or ideas that others have found vexing? Are there pertinent critiques of this document? What would you make of them? Some or all of this information can be captured under the rubric of assessment, above. You might also refer to your literature review outlines covering this and related documents. Having processed a resource, you may have the impression that you will remember the gems of knowledge it contains—assuming there are some— and that you will be able to apply them when they are relevant. As we saw in Chapter 3, those are illusions of remembering and illusions of competence. In order to improve the likelihood that you can apply knowledge gems—i.e., benefit from them—you need a system. That means, you need to spend some time thinking about this system. Applying the system will come at a cost—there is no free lunch. For some benefits, it might be as simple as organizing information in repositories such that you are likely to remember it when you need it. That is a technique promoted in the GTD system. (GTD focuses on offloading cognition.) However, if you actually want to master the information—to use it in the future—GTD practices are not enough. For very useful knowledge, ask yourself: how can I ensure that I can use this information when I need it? If you have an answer to this question, you could sketch the plan out in the meta-document, because this document contains all kinds of other pertinent information. Or you could develop and record your plan in your action management software (e.g., OmniFocus). For example, having read John Gottman’s Seven Principles once through, you might decide to do exercises from every chapter with your partner on a weekly basis. You could record your progress in your meta-doc. That is a big investment, but then a big investment is needed to obtain the kinds of benefits that readers hope to derive from Gottman’s book. To develop a plan to use practical information, it helps to ask oneself: under what situations may I need this information? Many a good book provides some obvious clues. For example, as noted above, according to Gottman, when your partner makes a bid, you need to turn towards them, rather than away or against them. This means, you need to become skilled at detecting your partner’s bids, and also detecting your own urges to respond. In other words, you need to develop new mindware: all kinds of new monitors including motive generators. Developing monitors is critical to developing the tendency to apply knowledge. They are what makes you notice problems and opportunities— based on knowledge you’ve acquired—and dispose you to respond to them. This goes for most of Gottman’s principles. As I argued in chapter 3, this is a key to being a rational agent, one who acts in accordance with the best knowledge he believes in. Some educationalists would say the person “transfers” what he knows. (I prefer to think in terms of being transformed by what one knows.) It’s in this “post-action” phase that you create a plan for developing the mindware that will enable you to apply the knowledge you’ve delved. Developing effectiveness doesn’t just come from delving or reviewing. It requires practice. To ensure that you will utilize and master knowledge gems, add them to your productive practice database.
12.6 A template for conceptual understanding Mental boxes work because things come in clusters that fit the boxes. Stephen Pinker
Delve
250
To assess, comprehend, master and improve high caliber concepts from a knowledge resource, it is often useful (and sometimes necessary) to apply techniques of conceptual analysis. Analyzing a concept involves asking and answering several questions about it. Unfortunately, this is not something that is typically taught to university students. Aaron’s Sloman’s free book, The Computer Revolution in Philosophy, contains a chapter with detailed instructions on how to do conceptual analysis.⁵⁰ This section builds on his and related work. In this section, I propose a simple template for analyzing scientifically relevant concepts. It contains sets of questions that you can ask about a concept. Not all concepts call for the same questions. You may need to modify these questions depending on whether you are analyzing a state, condition, event, entity, process or other type of concept. Given its pertinence to this book, I use the concept of mindware, as developed by Keith Stanovich, as an example. I use this example because it is pertinent to this book. Some of my answers are derived from Perkins (1995). My answers to the template questions are necessarily concise and incomplete. Some of my answers use concepts that are not in Stanovich or Perkins’ books (e.g., CUPA criteria of knowledge resources, productive practice, objective vs. personal mindware). A. What is the purpose of the concept? To understand a problem-based concept, one needs to know the problems that it is meant to solve. Example. The concept of mindware was proposed by David Perkins as a metaphor to help account for the fact that intelligence (in the broad sense) is learnable. Given two people of equal intelligence (in the sense of IQ), one of them might make much better decisions. Stanovich rejects the broad interpretation of intelligence and is content to identify intelligence with what current IQ tests measure. Rationality then is not the same as intelligence. Stanovich’s concept of dysrationalia, being IQ intelligent but unable or unwilling to behave rationally, poses a problem of understanding— Why do some intelligent people tend to behave irrationally? The concept of mindware—especially the mindware of rational thought—is a critical component of Stanovich’s extensive answer to this question. B. Semantic questions To understand a concept explicitly, one must characterize it. This sometimes involves proposing an intensional, or classical, definition—i.e., with a genus and species. Some concepts, such as game, humor, pain, and consciousness, resist classical definitions. This is not a condemnation of conceptual analysis. On the contrary, discovering and explaining why concepts resist classical definitions or are otherwise difficult to characterize can improve one’s understanding of the world. One can propose a collection of conditions with no one condition being considered the superclass or genus. One might ask, what is its superclass (or genus)? What are its siblings (i.e., its superclasses’ other children)? How does it differ from its siblings? What are its subclasses (or subtypes)? What is its structure? What are its properties, attributes or dimensions of variation? (In object-oriented programming, a software programming paradigm, the latter can be modeled as instance variables.) Some concepts are best characterized in terms of the roles that they play within an architecture, system or theory, which blurs the boundary between this section and the next one. Example. Objective mindware resists classical definition. Perkins takes it to be a metaphor. ⁵⁰See also Austin (1956), Beaudoin (1994), Ortony et al (1987) and White (1975).
Delve
251
Perkins sees mindware as the mind’s software. An important type of mindware is the mindware of rational thought. According to Stanovich, the latter is not a component of crystallized intelligence (Gc) as measured by IQ tests. That is because Gc tests are deliberately designed to tap into general knowledge that does not require specific education, whereas the mindware of rational thought requires specialized training in “probabilistic reasoning, causal reasoning, and scientific reasoning”. However, I presume Gc taps other mindware. Stanovich distinguishes between mindware that is accessed by the reflective mind—beliefs, goals and general knowledge—and knowledge that is accessed by the algorithmic mind—strategies and productive systems. Mindware may be more or less persistent, costly, refutable, falsifiable, disposable, egotistical. Perkins’ concept of mindware is broader than the one expressed in Stanovich’s book⁵¹. It includes dispositions and attitudes. While thinking dispositions are a major component of Stanovich’s theory of rationality, he treats them separately from mindware. However, he doesn’t explicitly state that mindware excludes thinking dispositions. Neither author (in the works cited in this book) explicitly consider all kinds of other attitudes (apart from thinking dispositions) as mindware (e.g., one’s attitude towards one spouse). C. Etiological questions Scientific and engineering concepts serve to help us understand the structure and functioning of the world. This can in turn help us understand how to control the world. Etiological questions are not directly questions about the meaning of a concept. To answer them requires empirical work. Discussions of the meaning of a term often go sideways because people confuse meaning with etiology. For example, one might confuse the question of the meaning of the term mood with the question, what causes moods to fluctuate. The value of scientific and technical concepts normally hinges on how well one can formulate and answer etiological questions with them. Similarly, our grasp of a concept can sometimes be measured by the accuracy and power of our etiological thinking with the concept. C.1 Inputs. For a state or process concept: what can cause it? What can facilitate, accelerate, influence or impede its occurrence, rate or development? Example. Mindware is acquired through learning. Some of the mindware of rational thought requires formal education and reading, and deliberate problem solving (e.g., probability theory). Mindware can be transmitted through cultural and “memetic” processes. Mindware can be acquired even if it is flawed and even if its application is harmful to its host. Intelligent people (in the IQ sense) are more prone to flawed complex mindware (less intelligent people can’t understand very intricate ideas, so they are immune to them). Flawed mindware with evaluation-disabling properties tends to linger. C.2 Outputs. What are the effects of instances of the concepts (or states to which the concept can affirmatively be applied)? How do variations in the presence or properties of the concept affect other parts of the system of interest? How can the concept be used for making predictions? Example. Dysrational thinking and behavior can be caused by: contaminated mindware (e.g., dogmatic religious beliefs), mindware gaps (e.g., not knowing that in order to remember complex material one normally needs to practice with it), the reflective mind failing to consult cogent mindware (such as the “consider the opposite” strategy) and the reflective mind failing to exercise ⁵¹http://www.keithstanovich.com/Site/Books.html
Delve
252
control over the algorithmic mind. The solution to some problems requires specific mindware (e.g., probability theory). If we know that a person lacks this knowledge, we can predict that he will fail to solve problems that require it. C.3 Control. Based on the foregoing analysis, what principles and rules may be recommended to influence and control the events, states and processes to which the concept applies? What practical considerations arise? Example. One should: • develop an understanding of formal knowledge as objective mindware and learn to think in terms of mindware and other concepts in Stanovich’s book⁵², • assess objective mindware in terms of caliber, utility, potency and appeal before deciding to install it (as personal mindware), • assess the potential benefits and harm of mindware before installing it, • beware of mindware that may restrict one’s goals, • develop the subjective mindware of rational thought. D. Examples. What are some good examples of the concept? Why are they good examples? What makes for a poor example of the concept? Philosophers and scientists often work with imaginary examples (“thought experiments”) to analyze and develop concepts. For example, Einstein imagined himself chasing a beam of light⁵³. Software engineers have systematic ways of analyzing requirements with examples; e.g., use-case analysis.⁵⁴ Example. “Consider the alternative hypothesis” is a good example of mindware because it is simple and illustrates the role of strategies and dispositions. This rule states that when one is presented with an explanation of an event, one should not necessarily accept it as given; one should treat it as a tentative hypothesis, knowing that it might be false, and consider alternatives. One might comprehend this rule, be able to describe it, to apply it if prompted, and yet still not tend to apply it. Why? Because one lacks the mental machinery (a) to detect conditions under which it is relevant and (b) to generate motives to apply the rule. In other words, one has not adequately developed inner motivators (monitors and motive generators). However, even when such a motive is generated, it ought not necessarily interrupt or control one’s thinking. The CogAff architecture explains how that may be. This example also points to a weakness in the concept of mindware, mentioned next. E. Assessment. Assess and criticize the concept, e.g., with respect to its caliber, utility, potency and appeal (see chapter 11). For example, with respect to caliber, does the concept adequately help us solve the problems it was meant to solve? Or other problems? Does it do so better than its rivals? How can the concept be improved? Some concepts are initially useful not so much because they serve to explain a phenomenon, but because they serve to direct our attention towards the need for further explanation. The Ancient Greek philosopher, Democritus’s concept of atom was such a starting point. It didn’t explain much, ⁵²http://www.keithstanovich.com/Site/Books.html ⁵³http://www.pitt.edu/~jdnorton/Goodies/Chasing_the_light/ ⁵⁴See Cockburn (2001).
Delve
253
but it opened the door to asking all kinds of productive questions. The heuristic power of a concept can only be determined in hindsight; that can take a long time. This particular concept was not extensively used in the West for 2000 years⁵⁵. The concepts of mindware and motive generator are also early spurs for inquiry. Whether scientists will be able to gradually enhance the concept is for history to determine and our readers to assess. To a certain extent, it already has. Keith Stanovich has refined Perkins’ notion of mindware, and I’ve tried to refine it further. Example: I am in near complete agreement with the content of Stanovich’s book⁵⁶. So, I will focus on a subtle reservation of mine that I touched upon in “Mindware as World 2’: Virtual machinery”. The analogy between software and mindware needs to be interpreted carefully. Software can be viewed in two ways: (1) as objective code (i.e., as source code, as code that is compiled and stored on a physical medium, or as an abstract artifact); (2) as running software in a machine. Similarly, mindware can be interpreted as objective knowledge (e.g., a theory or conceptual artifact) or as the subjective state of a person or mind that “has” the knowledge. Most of Stanovich’s and Perkins’ references to mindware can be interpreted in the former, objective, sense. However, they sometimes switch between the two different meanings.⁵⁷ To the extent that these authors use the subjective and objective interpretations interchangeably, they assume a correspondence view of understanding. As Bereiter (2002a) and connectionists have argued, to assume that knowledge and understanding are in the mind is problematic. To make progress, we need to develop ways of speaking separately about the mind (its mechanisms and contents) and the objective world (the natural world, and products including knowledge). I prefer to reserve the word “mindware” for information implemented in the mind (mental stuff), and “knowledge” for abstract information that transcends the mind (e.g., theories, rules, beliefs). Sticking to this distinction is not easy—you may even be able to find it violated in this very book. F. Other questions There are all kinds of other questions you may ask of a concept, such as the history of a concept and the various questions mentioned by Sloman that are not covered here (Sloman, 1978, 1993a).⁵⁸ Of course, not all of these questions need to be answered for every concept in a resource. One needs to develop one’s intuitions about which concepts are worth pursuing. I hope you will find this template as useful as I have over the years for making sense of knowledge resources. You can use it in all kinds of contexts. Progressive discourse, reading and inquiry require that we understand each other’s problems and concepts. You might find it useful to imagine the following diagram when the need to analyze a concept arises. ⁵⁵http://www.nobeliefs.com/atom.htm ⁵⁶http://www.keithstanovich.com/Site/Books.html ⁵⁷For example, Stanovich (2009, p. 40) wrote that each level in his tripartite model—i.e., the reflective, algorithmic and autonomous minds—could
access knowledge structures. This is a view of knowledge as World 2’ information stored in the mind. He also referred to probability theory as mindware. This is World 3 (abstract knowledge, a conceptual artifact). ⁵⁸In particular, one can analyze (a) the space of possible “problem situations”, as Popper (1979) calls them, or “niche spaces”, as Sloman calls them, (b) the space of possible designs and (c) the relations amongst them (Sloman, 2000b).
254
Delve
Figure 12.5 A Template for Understanding Concepts
Because conceptual analysis sometimes receives a bad rap at the hands of people who confound it with essentialism,⁵⁹ I will offer the following qualifications. The expression “conceptual analysis” is short-hand for all kinds of critical investigation of concepts. It does not suppose that concepts are immutable. For example, Albert Einstein analyzed the concepts of space and time and proposed a theory that changed our understanding of these and many other concepts. This shows that conceptual analysis ought not merely be restricted to colloquial concepts: it can also be performed with scientific and technical concepts, though with different questions. One can consider all kinds of empirical and engineering data when analyzing concepts. There is no simple, clear-cut demarcation between science, engineering, conceptual analysis and everyday thought. Many cognitive processes are involved in all those activities. Scientists in particular ought not eschew conceptual analysis. It can determine that some concepts and questions are in serious need of repair or rejection. One develops skills in conceptual analysis in the same way as one develops other skills—by engaging in the practice, documenting one’s results (e.g., in meta-docs) and reflecting on the process. This can involve elaborating meta-docs, discussing with colleagues and engaging in productive practice (see next chapter). ⁵⁹See Ch. 3 of Stanovich (2010) for a description of essentialism.
Delve
255
12.7 Quickly create and access meta-docs Why bother to develop meta-docs unless you can quickly find them when you need them? Chapter 9 presents this meta-access problem and some solutions to it. In this section, I will illustrate one of those techniques and describe a new one involving nvALT. You need to be able to quickly create a meta-doc in the first place. Some applications, like OmniOutliner Pro, provide a menu item to create new documents from a template⁶⁰. In OmniOutliner Pro, there is one item for each OmniOutliner template that is stored in the appropriate location. You can then assign a keyboard shortcut to this item. On a Mac, you do this in your System Preferences. If you are using a different application or a different OS, you can use a keyboard accelerator or a launcher (like LaunchBar). In this manner, you can literally create a meta-doc within one second. Then you can name the file to facilitate future access. You can often easily extract the author, year, name and title of the resource you are reading. A citation manager, such as Mekentosj Papers or Mendeley, provides this information. You can save your meta-doc with that information in its filename. You can also systematically pre-pend the filename with a special character, such as “[” that is not likely to be used in other filenames, to indicate that it is a meta-doc. For example, you might have a meta-doc called “[Minsky, Marvin. The Emotion Machine—Commonsense Thinking, Artificial Intelligence, and the Future of the Human Mind. Simon & Schuster, 2006.” You would store this document in any folder that is indexed by your launcher (e.g., LaunchBar) or that you will search for later (e.g., using Spotlight, on a Mac). To retrieve the meta-doc later, you just need to search for documents containing the special character followed by some of the meta-information you can recall about it (e.g., the title). The special character helps weed out false positives. If you are annotating a resource for which you are not likely to remember these meta-data, you can use the other solutions described in chapter 9, such as including the resource’s unique identifier (e.g., URL) in the document or in its meta-data. It’s a good idea to precede the URL with a special character, so that you can find the meta-doc specifically rather than other files that may happen to contain that URL but that are not its meta-doc. With such conventions, you don’t need to carefully file your meta-documents in some deeply nested folder. (You can even let software file this for you.)⁶¹ They don’t have to be stored in the same folder as the resources they annotate—that’s important, because some of the resources won’t even be on your computer. If this sounds too complicated, there’s a simple powerful trick for developing and accessing metadocs. It uses the lightweight, open-source Notational Velocity⁶² project. Notational Velocity features a text input field for naming, creating and searching documents. It’s title field is an incremental search⁶³ field: as you type, it lists matching documents. The search examines not only the title but the contents of your notes. When you select a note, its contents appear in the contents field, which supports all kinds of formats and markups. Notational Velocity automatically saves your notes for you as you type—there is no save operation. The search feature is so effective that you don’t need LaunchBar or Spotlight to access documents. This application shines when you want to write simple (plain text or RTF) meta-docs and don’t need the additional features of OmniOutliner. ⁶⁰http://www.omnigroup.com/documentation/OmniOutliner3/Templates.html ⁶¹For example, Leap and Yep file information for you by year/month/date. Sparks (2012) explains how to use Hazel to file documents automatically. ⁶²http://notational.net ⁶³http://en.wikipedia.org/wiki/Incremental_search
256
Delve
Figure 12.6 Schematic of Notational Velocity User Interface
(Figure 12.6 is from http://notational.net⁶⁴.) There is a fork of this project—nvALT⁶⁵ by Brett Terpstra—that packs some additional useful features. Here’s how you can use it to create and access meta-docs. Suppose you are in a web browser on your Mac and you want to annotate a web page. You just copy its web address (URL), open nvALT, paste the web page in the Search/Title field, hit Enter, and start annotating. Terpstra has even provided a web-browser extension⁶⁶ to automate this. When you return to this web page and want to view your annotations, just paste in the URL (or some part of it) in the Search/Title field, and your meta-doc will be listed in the Filtered Note list. With nvALT, you can even use tags to categorize your meta-docs. This is a very efficient way to document web pages. It does not depend on any particular browser and it supports Markdown. You can use nvALT to annotate any kind of resource. You can enter any identifying information as part of the note title (e.g., author, year, title), and you’re done. Want to write some notes about an email outside of your mail client? Just name it after the subject, sender or both. (Using a special meta-doc identifier such as “[” will help restrict the search here too.) Want to write some notes about an information-rich meeting you’re attending? Call it “meeting minutes” with the date and your done.⁶⁷ If you want some of the benefits of the meta-doc template described above, just add its elements to a text expansion snippet and provide a handy abbreviation for it. For example, you might use “mt” for “meta-doc template”, to give you a template like this:
nvALT Meta-doc Template Scratch pad: Rationale: Commentary: ⁶⁴http://notational.net ⁶⁵http://brettterpstra.com/project/nvalt/ ⁶⁶http://elasticthreads.tumblr.com/post/8212672178/nvit-chrome-and-safari-extensions-for-nvalt ⁶⁷You can even define a snippet in your abbreviation expander for this. For example, in TextExpander, the following snip can be used to name
documents. It expands the year, month and date automatically: [Meeting Minutes %Y-%m-%d
257
Delve
Summary: ANALYSIS: Problem addressed: Argument: Ancillary claims: Methodology: Glossary: Assessment (Caliber, Utility, Potency, Appeal) :
You can use such templates in any text editor. My objective is not to recommend particular applications or templates but to describe requirements and workflows to address them. This will help you decide what tools you need and help you to use them to become more meta-effective.
12.8 Delve ebooks, audio and other media The principles described in this book apply to all kinds of information formats. This section shows how delving principles can be used with audio, ebooks, multimedia and more.
12.8.1 Delve audio on the go There is an abundance of high-caliber audio-recorded information—podcasts, audio books, etc. It is also easy to convert documents to audio files⁶⁸ and transfer them to your mobile device for mobile delving. Listening is more efficient than reading: you can process information while also getting things done: housework, commuting, exercising, waiting, running errands, etc. Audio resources have their drawbacks, of course. For example, you can’t yet search for content (words or concepts) in them. If you merely aim to skim the information or are involved in a potentially dangerous activity like driving, all you have to do is listen. But if you wish to derive lasting benefit from the information—to extend your understanding and develop effectiveness—then you need to delve. Alas, it’s all too easy to assume that we will be able to remember and use potent information without additional effort. Some audio players allow you to listen to podcasts at a slower or faster rate than normal. Higher speeds are not suitable for delving. Speed reading is no more effective than speed listening. To delve audio normally requires that you interact frequently with the audio-player: e.g., pause to think; rewind if you haven’t understood something important; etc. One approach is to listen to the content once through quickly; the second time more carefully if it’s worth the effort. Or to listen quickly until you get to the knowledge gems. You might need to pause to take some notes. Or, you can use mnemonics to postpone note-taking. If you process the content by thinking and talking about it, you can avoid note-taking altogether. ⁶⁸Some applications, such as ConvenienceWare’s GhostReader convert PDF files and other documents to MP3s and store them in iTunes or elsewhere. I recommend putting the document through a text editor that supports regular expressions and batch find/replacement to strip out text you don’t want to hear. I use BareBones Software’s BBEdit on a Mac.
Delve
258
But it’s a bit riskier. If there is nothing worth taking notes about, was listening time well spent? You should be on the look-out for knowledge gems as you listen. The annotation taxonomy presented above is also useful for processing audio content. You can seek useful or potent concepts, claims, problems, criticisms, references, etc. You can also discover gaps and flaws in your knowledge and understanding that you may wish to fix. Here are some tips for taking notes while listening to content on a mobile device. • Use a very efficient means of capturing your notes, such as a dictation device that converts speech to text (e.g., Apple iPhones with Siri® support). In this case you would dictate your notes. If your smartphone can’t convert speech-to-text for you, you can still use speech-to-text at transcription time (e.g., with Nuance’s Dragon Dictate or Apple’s Siri). Alternatively, you could hire a transcription service. It’s usually a good idea to have pen and paper handy as well while ambulant—touch screen typing is slow and error-prone for this use case. See my blog⁶⁹ for more information. • Use a GTD application (e.g., OmniFocus) on your mobile device to capture some of your knowledge gaps and actions. OmniFocus even has an audio-recorder to facilitate capture. You’ll first want to create a project for this document, such that your next actions will by default be inserted in this project (see Identify your projects). • If possible, use an audio player that efficiently supports annotation and bookmarking (e.g., DockMarket LLC’s Bookmark iPhone app.) Add a bookmark for every annotation to which you might want to return. • Include the playback time in your annotations so that you can later return to the precise location in the audio resource. These last two tips are important because audio resources are much more difficult to navigate than documents. So, it’s important to index knowledge gems that you want to capture. • Take advantage of the audio control buttons (play, pause, rewind, etc.) to revisit important bits you didn’t fully understand. • Apply the principles of expert delving and meta-doc categories listed above, while conceding that listening “on the go” limits what you can do with the material. Identifying knowledge gems is of no use unless you have systems and habits to capitalize on them. The content ultimately needs to be treated as any other meta-doc. This might just be a question of naming your note file according to the right conventions, storing it in the right folder, tagging it, or including some meta-information in the document that will enable you to find the meta-doc very quickly when you need it. Or you can copy your document into a normal meta-document on your computer. Some outliners, such as OmniOutliner, enable you to store (and record) audio (and other formats) within them. It’s sometimes useful to copy your knowledge gaps and actions from your notes into action-management app (which you might not have done while listening). You’ll want to set the “context” appropriately to facilitate future processing (e.g., as a “knowledge gap”) Some of these notes will lead you to record actions in other projects. For example, you might find a concept you want to use in a product you are designing or a solution you are developing for a client. You’ll want to put some of the gems in your productive practice knowledge base. ⁶⁹http://cogzest.com/2012/03/some-benefits-of-dictation-for-cognitive-productivity/
Delve
259
12.8.2 Delve e-books Many of the examples I used above involved delving PDF content. There are also millions of documents available in formats other than PDF. Some of them in principle can only be read using the book seller’s e-book reader. I criticized popular e-book readers above (e.g., Amazon’s Kindle and Barnes & Noble Nook®) because they inadequately support delving. Manufacturers’ obsessions with digital rights management limits your ability to master the resources you’ve licensed. It also tends to lock you into their software and stores. If you hire a contractor to do numerical analyses for you, you wouldn’t accept that he deliver his spreadsheet in a file format that required a special program, one that interfered with you using the information. No. You’d expect the spreadsheet to be convertible to one of the major spreadsheet formats so that you can freely choose which app to process the file with. It should be the same for ebooks. The ebooks sold by O’Reilly and Leanpub are DRM-free. Their books are available in the major formats (e.g., PDF and e-pub). You can read them with the software of your choice. There are applications, such as Calibre⁷⁰, that enable you to convert between popular e-book formats. This means, for example, that you can convert most ebooks to PDF or e-pub. You should first verify the e-book software provider’s license agreement as well as the publisher’s agreement. Cory Doctorow asked Amazon to clarify whether users are allowed to convert Kindle books to devices they own—Amazon may limit the number of times you can download “your” Kindle ebooks—but Amazon didn’t answer⁷¹. You may also need to check with law, case law or jurisprudence in your country—it might over-rule vendors’ attempts to interfere with your right to delve content you’ve licensed.⁷² E-book technology and formats are always changing. So, I will not get into the details of how to use these applications. I will merely recommend that you consider Calibre as an e-book manager. This is a Swiss Army knife of an e-book converter. The user interface is not particularly intuitive. However, it does a good job of converting ebooks to PDF. And it allows you to manage all kinds of news content. Once you have converted your content to PDF, you may need to get it onto a tablet. Tablets of course have many reading advantages over computers. They do introduce reading problems, as well, however: 1. synchronizing your content (including annotations) between devices can be complicated. 2. data entry in tablets is slower than with personal computers, which affects annotation. 3. they have small monitors (see the section “Computer monitors and other hardware”, below). Mobile technology is changing so fast and there are so many options that I will stick mainly to principles here, while providing some illustrative examples. Some of the factors in selecting sync technologies are: ⁷⁰http://calibre-ebook.com ⁷¹http://boingboing.net/2009/06/22/some-kindle-books-ha.html ⁷²In the U.S., consumers need to consult the Digital Millennium Copyright Act and periodic exemptions by the Library of Congress. Consider this
2012 interview of Cory Doctorow: Is publishing looking at a DRM-free future?.
260
Delve
• Cost. If your computer is your sync service, syncing is virtually free for you. OmniGroup has recently released OmniPresence⁷³, which enables you to sync any document type that supports OS X Versions on their server or via your own server. If you use a storage service like DropBox, iCloud, Google Drive®, Amazon Kindle, Microsoft SkyDrive® or EverNote there may be charges per megabyte of storage; with Kindle there are additional charges for data transmission⁷⁴.⁷⁵ Consider using a personal bibliographic management app such as Mekentosj Papers (Springer Science+Business Media) that supports syncing between devices.⁷⁶ • Speed. Syncing with a computer may be faster than with an Internet service such as DropBox. Slow devices on the path between your devices and Internet storage provider limit your transfer rates. • Transparency and ease of use. It’s normally better for your content and annotations to be synced automatically. Otherwise, you run the risk of forgetting to issue a sync command, which may leave you with stale data. And who needs the headache of manually syncing? Still, you’ll need to configure each application on your tablet with the sync solution of your choice. • Security. You need to choose a sync service provider that you trust whose data are stored in a country that respects users rights to privacy. (In some cases, there are legal limitations on where people can store information.) Also, it’s often preferable for one application’s data to be sequestered from another, otherwise, through malice or error, an application may compromise your information. Some mobile operating systems are more secure than others. • Search. The sync client on the mobile platform should have good search facilities. If you have hundreds or thousands of files, it’s important to be able to find them quickly. You need to be able to find files by author. You should also be able to do a full text search of your PDFs. This is where Dropbox and Bit Torrent Sync currently disappoint. Mekentosj Papers 3 for iOS allows you to search all your PDFs. It provides an abstraction layer over Dropbox. It’s worth taking the time to make an informed decision about storage and (other things being equal) sticking with one’s preference. Many tablet e-readers (e.g., Kindle and iBooks) provide annotation features (notes, highlights, etc.). However, they are very limited compared to what you can accomplish with a general-purpose computer. For example, with the Kindle for iOS (version 3.2): • • • • • •
the note fields are very small, only one note field can be open at a time, one can’t annotate figures, there are no inline tags, there is no undo for note creation, dictation is limited,
⁷³https://www.omnigroup.com/omnipresence/ ⁷⁴http://boingboing.net/2009/08/28/david-byrne-kindle-d.html ⁷⁵For storing large amounts of data, consider Copy. Attractive alternatives to cloud services include the peer-to-peer service, BitTorrent Sync, and
Transporter’s personal storage access solutions. ⁷⁶There’s a wikipedia page, “Comparing reference management software”.
Delve
261
• outlining is not supported, • the desktop version of Kindle app does not allow one to search, access, manipulate and extract one’s annotations very flexibly (compared with Skim). See my blog post on “What’s wrong with the Kindle app: A knowledge delver’s perspective”⁷⁷. So, you may be better off making your annotations outside of Kindle, using a separate capture device such as an additional tablet or iPod®, or a voice recorder. You can in principle tag your notes using conventions and follow up on them later (see the section “Write meta-docs”). To be fair, there are also benefits to delving with Kindle app. If you use the Kindle app, your notes can be synced to Amazon’s server along with some of the text that you’ve highlighted. Unfortunately, some publishers limit the number of characters highlighted from a book that you can sync to the web. So, you can’t rely on this method for many of the delving purposes described above. For example, you can’t use Kindle’s services to extract all of a book’s snips you’ve tagged as major claims. Still, this enables you to extract the notes that you’ve written. Apple’s iBooks 1.0.1 is better in this and several other respects. For example, you can copy annotated text directly from the app to your meta-doc. Fortunately, the quality of PDF readers for tablets is improving. If you convert your ebooks to PDF, you can delve them on your tablet. For example, for the iPad, there is iAnnotate, GoodReader, PDF Expert, and PDF Pen⁷⁸. In conclusion, tablets pose special annotation problems as contrasted with reading with a general-purpose computer. To get many of the benefits of mobile reading on a tablet, without their inconveniences, one is often better off using a notebook computer with a solid-state drive and a high resolution monitor.
12.8.3 Delve other media on your computer The common media players on desktop computers (e.g., QuickTime on the Mac) still do not provide much by way of annotation facilities, and so once again you are left to your own devices. However, you can annotate screencasts, webinars, and flash presentations using the annotations principles I described above. As usual, it’s important to include meta-data in your annotation files that enable you to easily navigate between the content (if it is still available) and your meta-doc. And it’s useful to include the index (time) for some of your comments. If you are viewing a media resource (e.g., a TED talk) outside your meta-doc application (e.g., OmniOutliner), you will need to switch back to the media application in order to pause it so that you can take your notes. Then you will need to go back to the media application to resume playing. This is a bit tedious. So, if your media application has keyboard shortcuts for play/pause (the space-bar, is used for this in many Mac applications), use them to speed up the process. If you have desktop access to an audio resource that you want to delve, you can use Scrivener⁷⁹ as a meta-doc application. It has a split-pane window. You can view the media controls and status in one pane while annotating in the other pane. You don’t have to switch windows or panes in order to ⁷⁷http://cogzest.com/2013/11/whats-wrong-with-the-kindle-app-a-knowledge-delvers-perspective/ ⁷⁸http://news.lib.uchicago.edu/blog/2011/10/17/pdf-tools-on-an-ipad-for-scholars/%20 ⁷⁹http://www.literatureandlatte.com/scrivener.php
262
Delve
play and pause the media resource. See this screencast on transcription⁸⁰ for a clear demonstration of this. Apart from this use case, it’s usually better to use an outliner for meta-documentation, for reasons mentioned above, and also because audio and video streams quickly: you don’t want to have to pause it too often. Outliners facilitate capturing the hierarchical structure of the content you are delving like no other type of application can.
Figure 12.7 Example Transcription in Scrivener
It is often useful to take screenshots of screencasts and presentations as you are viewing them, i.e., to capture the key slides that you need for future reference. On a Mac, you can use a keyboard command for this (command-shift-3), which enables you to draw a rectangle around the slide. These screenshots are then stored by the OS in a folder. If you are using OmniOutliner, you can then quickly import the image as you are writing, with three or four keystrokes. Or, as mentioned above, you can use LaunchBar’s “Copy Screenshot of Selected Area” command.
12.8.4 Productive laziness (cognitive parsimony) Some people feel that trying to build effectiveness from listening and viewing is too effortful—they like the pleasure of just listening. This is where the concept of productive laziness kicks in. ⁸⁰http://www.literatureandlatte.com/videos/Transcription.mov
263
Delve
Productive laziness often means applying previously acquired knowledge about the problem or some general class of problems. So it requires learning: the ability to form new concepts and to acquire and store new knowledge for future application. Sometimes it involves creating a new form of representation, as has happened often in the history of science and mathematics. Laziness motivates a desire for generality — finding one solution for a wide range of cases can save the effort of generating new solutions. This is one of the major motivations for all kinds of scientific research. It can also lead to errors of over-generalisation, prejudice, and the like. A more complete survey would discuss the differences between avoiding mental work (saving computational resources) and avoiding physical work. (Sloman, 1988⁸¹). Cognitive work in the short run can stave off pain and work in the long run, by allowing one to capitalize on the knowledge one has gained and the mindware one has developed from delving. Moreover, one can be productively lazy by capturing information selectively and processing it efficiently.
12.9 Computer monitors and other hardware It is very important to configure one’s physical environment properly to optimize one’s productivity and enjoyment, while minimizing the physical and cognitive drawbacks of working long hours at a desk. I recommend routinely noting, researching and resolving problems in your physical work setup. The United States Department of Labor: Occupational Safety and Health Administration issues guidelines for working at computer workstations (“Computer workstation evaluation checklist” 2013). It is also useful to consult with an ergonomist, colleagues and other professionals (e.g., physiotherapists). There are studies on the impact of monitor size on productivity (Robertson et al., 2005). Because there is an interaction between monitor size and task, one cannot simply say that a particular range of monitor sizes is optimal for productivity. In particular, there is no research that examines the impact of monitor size specifically on delving productivity or meta-effectiveness. I will simply state that it’s important to ensure that your computer monitor is large enough but not too large. It is useful to be able to see, side-by-side, the resource you are annotating and your meta-doc. In my experience, this calls for a high-quality monitor that is at least 24 inches (on the diagonal) but not more than 27 inches. Using multiple monitors is a popular solution but it poses challenges. It’s important to be able to adjust the height, position and angle of your monitor. This should enable you to lower your monitor slightly during the day. (People shrink slightly as the day wears on. ) It should also enable you to shift your position at your desk and adjust to new task demands, such as working with a tablet or paper. If, like mine, your monitor is not adjustable but conforms to the VESA standard, consider purchasing an adjustable VESA-compliant monitor stand. I use a NeoFlex® Widescreen Lift Stand. Its promo reads “five-inch (12.7 cm) height adjustment, tilt, pan and portrait-to-landscape rotation for optimum viewing. Embedded with patented motion technology, ⁸¹http://www.cs.bham.ac.uk/research/projects/poplog/doc/popteach/aithemes
Delve
264
this stand enables light-touch adjustments without any knobs or levers.” (Ergotron, 2013). Thanks to this, it’s a cinch to raise and lower my (otherwise heavy) 27-inch Apple LED display–using two fingers does the trick! The stand is on wheels, so I can easily move my monitor around on my desk. I place my notebook computer to the left of my monitor on a WorkEZ® Light highly adjustable monitor stand from Uncaged Ergonomics®⁸² (WorkEZ Light, 2013). This extremely versatile, light and inexpensive stand is a spectacular piece of engineering. It leaves plenty of desk space underneath it for writing and storage. Jacques Brel, Victor Hugo and Sir Winston Churchill wrote untethered from chairs. I, too, like to write standing up. That way, I can easily move around and even pace. That helps me solve problems. I sit for about 20 minutes every couple of hours⁸³. This calls for an ergonomic arrangement to rapidly and effortlessly switch between sitting and standing positions. Before I perfected this system, it used to take me over a minute to switch between standing and sitting positions. Through successive improvements I gradually brought the time under 8 seconds! To save you some time, I will share my results. My monitor, mouse and keyboard are on a bar table (my “computer desk”) that is 101 cm tall. I use a drafter’s chair. To work sitting down, I merely need to push the monitor down several inches, pull the chair towards me, and sit-down. To work standing up again, I just need to stand, push the chair behind me and raise the monitor. Whether I am standing or sitting, the top of my monitor is approximately at eye-level. When I type, whether standing or sitting, my forearms are as they should be: nearly parallel to the floor. At a right angle from my computer desk, I have a general-purpose work desk (my “side desk”). I have raised this desk on cement blocks such that it is at the same height as my computer desk (101 cm). There’s no need to raise or lower my desks while I’m working. My side desk is 160 cm wide by 80 cm deep. This gives me plenty of room to read books, write and draw. There are two double-shelf book cases on my side desk for frequently accessed resources. I sometimes use a computer copyholder for tablet and paper reading. I fold down my laptop and place the copyholder on it. (When I have an external display connected to my Macbook Pro, it keeps running even when the laptop is folded down.) It too is at just the right height for me. There are photographs and a 12-second video clip of my workstation setup in a CogZest blog post⁸⁴. For enhanced productivity, consider using solid state drives (SSD) as opposed to traditional hard ⁸²https://www.uncagedergonomics.com/workez-light/ ⁸³It’s disheartening to reflect upon the fact that we manage to persuade children, through our culture, that spending hours a day sitting down at
a desk is natural. The amount of time people spend sitting down is associated with increased adiposity, the metabolic syndrome, abnormal glucose regulation, and mortality (Beavers et al., 2010; Yates et al., 2012). A study of 71,000 women found that those who reported sitting more than 10 hours a day had a greater risk of heart attack, stroke and other cardiovascular events (Chomistek et al., 2013). Exercise did not attenuate this risk except for the most active women. (That is, a small amount of exercise does not compensate for spending a whole day sitting.) These findings are correlational not causal. The data are, in particular, linked to body-mass index. Moreover, there is no indication that simply standing provides the same benefits as more strenuous physical exercise, such as walking and jogging. Chronic low-grade inflammation is a significant risk factor for several diseases. It is not yet clear that exercise can attenuate such inflammation; however, it is an important research target (Beavers et al., 2010). The Just Stand web site, which seems to be commercially backed, is worth consulting if only to induce readers to investigate the possibility that working standing up is healthier and more conducive to productivity. While on the one hand my scientific mindset and lack of rigorous data have restrained me from outright advocating working standing up, consider, on the other hand, that there is no data or persuasive argument to the effect that (other things being equal) working sitting down is conducive to productivity, health or well being. ⁸⁴http://cogzest.com/2013/08/productivity-tips-for-working-standing-up-using-workez-light-adjustable-monitor-stand-from-uncagedergonomics-and-other-components/
Delve
265
disks.⁸⁵ SSD drives are much faster and they’re more reliable. Purchase a drive that is large enough that you won’t waste your precious time juggling your precious content across different drives or even worry about having enough space. You don’t want to have to ask yourself “on what drive should I store this file?” You can periodically reclaim disk space by deleting duplicate files⁸⁶. On OS X, I use the free OmniDiskSweeper⁸⁷ utility when I feel the urge to reclaim disk space. ⁸⁵There are products that contain both a SSD and a traditional hard disk. Apple’s Fusion Drive is a case in point. ⁸⁶http://www.araxis.com/find-duplicate-files/ ⁸⁷http://www.omnigroup.com/products/omnidisksweeper/
13. Productive practice: A master maker Practice can lead to mental processes and structures that are vastly different [from] those that we might develop after only a little experience in a domain. And yet, the power of practice is often overlooked. Craig P. Speelman & Kim Kirsner Well-intentioned people read books that provide the best information available on subjects of great importance to them: e.g., the principles of harmonious interpersonal relationships, rationality, agile project management, sleep, financial investing, nutrition, effective software development, and many other factual, practical and normative subjects. Alas, many of those readers continue to live in disharmony, make poor decisions, manage projects the old-fashioned way, resort to unnecessary medication, mismanage their finances, eat poorly and so on. What have they learned? They can perhaps recognize the ideas when they are explicitly referenced. They even might sometimes think about the ideas in the resource. However, they often cannot freely recall the major claims, arguments, and so forth. For many, when they can recall information, it qualifies, at best, as bookish or conversational knowledge. They may think about what they’ve read; but they don’t tend to perceive, think, assess, feel, and act with that knowledge. In short, whether they have superficially read or even delved a resource, they have not mastered the gems in the knowledge they’ve processed. We are all them at some point. To distinguish ourselves, we need to systematically apply a reliable system for developing personal effectiveness from excellent information. In this chapter, I describe a system to help people master and apply knowledge gems. This system involves productive practice. Productive practice is a form of deliberate practice and test-enhanced learning designed to help you develop effectiveness from knowledge resources. The system system works best with software.
13.1 Productive practice in a nutshell Suppose you are preparing to be examined on your recall and understanding of factual information in a subject like history, psychology or biology. You are likely to be successful if you take a productive practice approach that goes like this. While reading and forming an understanding of the material—i.e., delving— you prepare a list of potential exam questions. By expressing these questions you form a judgment about the requirements and objectives of your study: i.e., what you need to know. You formulate an answer to each question. You periodically practice answering the questions until and beyond the point at which you are confident that you have achieved your target mastery, whether it be a C+ an A+ or
Productive practice: A master maker
267
anything in between. Productive practice is not a linear process. Accordingly, you interleave delving and practice. You revisit your questions and answers. This is the essence of productive practice: a wonderfully simple process for mastering knowledge. I didn’t, however, write this book to help you pass exams. I assume you were or could have been a very good student (and you may even have used test-enhanced learning yourself). However, you’re now more concerned about life’s exams than professors’. Real-world learning is more difficult than academic learning. Even if you merely wanted to learn factual knowledge as students typically do, you probably don’t yet know how to incorporate test-enhanced learning into your daily learning. Nor do I expect you to know, yet, how to use test-enhanced learning to achieve the various types of effectiveness that this book is meant to help you achieve: lasting statable and tacit understanding, skills, attitudes, habits, propensities and so on. The rest of this chapter is meant to teach you how to use productive practice to help you develop. Here’s a break-down of the major steps involved in productive practice. Step 1: Identify knowledge gems. Most of the information in a knowledge resource, such as a book or video, you want to keep in mind just long enough to be able to accomplish whatever task brought you to it in the first place. Occasionally, you encounter potent new information that you want to be able to think with or about without needing to search for it externally. Those are knowledge gems. A knowledge gem is information that you want to be able to use unaided, in context. If you’re in a meeting, or having an important conversation, or need to solve a problem quickly, you normally don’t have time to brush up on knowledge. Some knowledge you just have to be able to work with. You need to be able to work with some of the information in your long-term working memory, as if you had just consulted your notes (i.e., as if it was in long-term working memory); or you need other mindware, such as motive generators, to have been “programmed” by the information. Step 2: Extract knowledge gems. Once you have identified a knowledge gem, you need to add it to a system that will allow you to master it. If you’re familiar with the “getting things done” (GTD) methodology of David Allen, you are familiar with capturing information in a system. The GTD workflow, however, does not actually deal specifically with developing oneself, and so it lacks the concept of capturing knowledge gems. One way to capture a gem is to create a mindware instiller for it. So, while delving, you could tag or highlight the text with a tag that means “To instill”. Or, with action-management software like OmniFocus you might create an action to create an instiller. Or you might interrupt your delving to create an instiller on the spot. I’ve coined the verb “to instillerize” to denote the act of creating an instiller for a knowledge gem. An instiller is a data structure designed (by yourself or someone else) to help you master a knowledge gem. An instiller contains one or more challenges. A challenge specifies an activity with the information. It usually consists of a question and answer pair. The activity could be to draw a diagram, imagine a particular scenario, solve a problem, recall some information, etc. It represents questions that you want to be able to answer unaided. It elicits an activity designed to build your mindware and help you (and software) assess your mastery of a knowledge gem. Challenges can be designed to develop your understanding, rather than merely help you memorize information. By practicing with challenges you might even create new (World 3) knowledge. For example, you might create some historically new knowledge or at least some new knowledge resources based on
Productive practice: A master maker
268
productive practice. Step 3: Design instillers. In order for productive practice to actually instill mindware, you need to configure instillers accordingly. This involves creating challenges and possibly associating other information with the instillers. For instance, you might want to include a link from the instiller back to the source of the knowledge, such as the paragraph in a document describing a concept that you want to master. The section “Design instillers”, below, describes several types of information to associate with instillers. It also explains how to automate the process. The whole point of productive practice is that it should have a negative time footprint: i.e., it should save you time, not cost you time, while enabling real mental development. You will likely create most of your own instillers. Eventually, I expect content developers and publishers to provide instillers for their content, so that readers don’t merely skim or delve the content, but actually master it. Apple has already adopted a primitive version of my suggestion to them by adding flashcard capabilities to their iBooks software.¹ I will comment on flashcard software in a section below. If you have a coach or staff to help you master information, they could create instillers for you. Imagine a politician or other speaker preparing for question period with instillers. By capturing gems and developing instillers, you will gradually build up a database of software to develop the mindware that matters to you. Step 4: Practice with mindware instillers. As we discussed above, public performers, such as athletes and musicians, practice on a regular basis—usually every day. Given that productive practice is deliberate practice, it also requires that you practice regularly—several times a week. But because knowledge work is different from public performance, you don’t need to practice nearly as much as, say, a musician. But you do need to practice a certain amount. General rules of deliberate practice apply to productive practice. That means, you need to plan your practice, set difficult goals, focus while practicing, do your best, monitor your performance, obtain feedback, adjust your goals as a function of your performance, and so on. You need a system (processes) for your practice. I mentioned in chapter 7 that to achieve world class performance often requires having a teacher or coach. Most knowledge workers don’t have access to a human teacher or coach. However, fortunately, as we will see in the following chapter, many of the benefits of having a coach can be derived from using a system and software. While there is a logical order to these four phases, this workflow is flexible. For example, you can extract knowledge gems on the first or a subsequent pass through a knowledge resource. You can tag information as a knowledge gem and at a later date generate instillers from them. Even once an instiller is ready to be practiced, you can postpone it. You can decide when to start practicing. You can even expire an instiller when it’s no longer relevant. While practicing, you will occasionally want to revisit the knowledge resource for further clarification. Productive practice is useful not only for instilling knowledge you glean from others, but knowledge that you yourself have built. And so, you may reconfigure, update and relink instillers. You might revisit the original resource after capturing some of its gems in order to clarify them or ¹In an article anticipating the announcement of the iPad in January 2010, I called for Apple’s anticipated ebook reader to contain support for this (Beaudoin, 2010a). Apple introduced a limited version of this feature in January 2012. You will see that my concept of productive practice takes this much further. For example, one’s instillers should not be trapped in a book. Productive practice software should have access to instillers from multiple knowledge resources of any type, so that users can easily practice with material gleaned from any app (e.g., from a web page, email, PDF file, ebook, podcast, audiobook or movie).
Productive practice: A master maker
269
to extract more of its gems. Productive practice, as described here, leverages principles of cognitive science. Productive practice is not a silver bullet. It is more likely to enable you to effectively use knowledge, and may do so more efficiently, than re-reading or even discussing materials. However, it takes time and effort to perform the steps listed above. Fortunately, that time can often be obtained by cutting down on less cognitively productive activities.
13.2 An example: Learning the motive generator concept Given that we often learn best by example, this section provides an example of how you might apply productive practice to learning the concept of motive generators, described in chapter 5, Your mind and its wares. In the Delving phase, you would read the chapter. On first or subsequent reading, you would tag noteworthy passages according to your annotation schema. (See “Tag snips of text and images” above.) You might even tag text as “knowledge gem”, if it is worth mastering. You wouldn’t make this judgment lightly. Like a real gem, knowledge gems need to be bought—with time and mental effort. So you need to have a sense that it will provide value for you. You would quickly review the chapter and your annotations. You would create instillers for items tagged as knowledge gem, or other items worth mastering. Let’s suppose that you were struck by the idea, presented in Part 2, that the mind contains motive generators and long-term working memory. You want to ensure that you master these ideas and can use them to interpret and shape your own experience in the future. So you create some instillers for them. You might create one instiller per gem. Your instiller for motive generator might contain challenges similar to the following ones. These example challenges only deal with developing understanding, as opposed to developing attitudes or habits. In subsequent sections, I will explain how to develop in other ways. Challenge: “What is a motive generator?”. Answer: “A motive generator is a mental mechanism, a control state, that generates motivators (e.g., wishes, wants, desires, aversions). It monitors (or observes) information in the person’s mind.” Challenge: “What problem of understanding is the concept of motive generator meant to solve? Why was the concept proposed?” Answer: “While we may take mental states for granted in every day life, cognitive science seeks to explain and characterize them. Goals, desires, and other motivators, do not come out of nowhere. In proposing the concept of motive generator, Sloman acknowledges that there must be mechanisms that generate these control states. He gives them a name. The concept is also a step towards understanding how motives differ (e.g., in intensity and insistence). It is not in itself an explanation. It is a tool for formulating questions and answers about motive processing.” Challenge: “Suppose you are at the airport with some time on your hands. You haven’t had lunch. You have some work to do, and you notice a single, attractive acquaintance that you would like to get to know. Explain what goes on in your mind in terms of motive generators.” Answer: unspecified.
Productive practice: A master maker
270
Challenge: “What high-level concept does Beaudoin use to characterize learning and the development of expertise.” Answer: “Mental development. The development of motive generators, cognitive reflexes, long-term working memory, and other new components. Changes in mental architecture and connections amongst mental components.” Challenge: “Why does Beaudoin claim that a lot of learning, or mental development, involves establishing motive generators.” Answer: See the description of motive generators in Ch. 5 and chapter 6. Challenge: “What is the main type of mechanism proposed by Beaudoin and Sloman as triggers of wishes, wants, repulsions and similar motivational states.” Answer: motive generators. Challenge: “Compare and contrast the concept of motive generator with folk psychology concepts such as needs, wishes, wants, attention and emotion.” Answer: unspecified. Challenge: “Explain and illustrate why mastering principles, concepts and techniques specified in self-help books, such as John Gottman’s concept of bids, requires developing new motive generators. “ Answer: See “Developing motivators” and Illusions of rationality: transfer reframed above. You will notice a few things with these examples. 1. The majority of the questions call for elaborative answers, but one of them calls for a term (“motive generator”). The mix of short answer to elaborative answer will vary. Some authors (e.g., Nist-Olejnik & Holschuh, 2013) disapprove of short, “what” questions as if they call for merely rote learning. Yet mastering the same gem often calls for multiple questions (see list item 5), some of which are “what” questions. As described in Developing long-term working memory, the development of expertise involves programming long-term working memory with much factual knowledge, much of which is quite simple. 2. Some of the answers are left blank. Often, they can be left blank in the instiller indefinitely. Sometimes, you will merely want to put in a few key words. This is useful when you want to trigger reflection, imagination or some other mental process. You can nevertheless provide a hint, mnemonic, bibliographical reference or hyperlink. According to VanLehn, this is also an advantage of self-explanation. If you’re explaining something to someone else, you need to spell it all out. But when you’re practicing by yourself, you can cut to the chase. That saves time. 3. There is often no single right answer. Some questions have multiple answers and some answers are better than others. This is another case where you might want to leave the answer blank. As you practice, you should try out different answers. This is a form of “elaboration” and it helps you develop your understanding of the concept. 4. It’s often desirable to have more than one challenge for a concept. Some of the important types of questions for a concept are of the following forms: • • • •
Provide examples of the concept, list related concepts, distinguish the concept from a related concept, explain the problem of understanding that the concept was meant to address,
Productive practice: A master maker
271
• read (or listen to) this example, • clozes, (i.e., clozes), i.e., where you are provided with a sentence that calls for the word, but where the blank is presented you are given a definition of the term, and you are called upon to educe the word, • simulations: Imagine a situation where this concept applies, what should you do? These types of questions can usually easily be generated. During practice, these challenges should be interleaved with challenges on different topics. Otherwise, practice will be too easy. That is, you’d be benefitting from priming that won’t be available when you really need it. Some of the challenges are easy to address immediately after delving. However, we tend to forget knowledge that we do not instill through repeated practice. Productive practice software helps you practice challenges over a period of time, until they become self-sustaining or your suspend them. The remainder of this book contains several other examples and tips for designing instillers.
13.3 Co-opt flashcard software Productivity software normally enables you to create, edit, transmit, obtain, compute and store information items. Word processors, spreadsheet apps, diagramming apps, email apps, password managers, citation managers, web browsers… they all operate on external products, which, following Karl Popper in Part 2, I referred to as items in “World 3”. Productive practice software, in contrast, will be designed to help you develop your mindware. For busy people, productive practice is not practical without software. It requires too much book keeping to do manually. Unfortunately, the state of the productive practice software market is similar to that of the delving software market: There is no application that fully meets the requirements set out in this book. So, why am I writing about it? Mainly because one can cobble together workflows that leverage the most suitable existing applications. I explain how to do that. But it’s also to prepare you for the advent of cognitive productivity software. I am also providing another nudge to software community to address this huge opportunity. In chapter 3, I mentioned that the closest thing you will find to a productive practice app is flashcard software. I also mentioned some of their major limitations—they are not closely integrated with the operating system and various applications and they are not designed with a particularly deep thorough or creative application of cognitive science. Nor are they particularly attuned to productivity requirements. While the vast majority of this software is toy-ware, I will show you in this section how Damien Elmes’s Anki Desktop flashcard program can be used for productive practice purposes. I will gloss over some of Anki’s complexities and limitations while referring you to its well-written user manual, freely available on the developer’s website². This section starts by using Anki’s terminology. Then it shows how to map that onto our new, productive practice way of thinking. The rest of the book then sticks to the new terminology. A typical flashcard app has two modes: editing and “reviewing”. The former is for creating and updating collections of “cards” called “decks”. Each card has a front and a back. Users write the ²http://ankisrs.net/docs/dev/manual.html
Productive practice: A master maker
272
question on the front and the answer on the back. They might even leave the answer blank. Users create as many questions as they want for the material they are learning. In “review” mode, the software presents cards one at a time to the user. The software chooses the card to present according to some rules (called a “scheduling algorithm”). For example, the question might be “Compare and contrast prurience and lasciviousness.” The user tries to answer the question, typically in their heads, but possibly indicating the answer with the software. Then the user issues a command to view the answer. At this point, the user indicates whether they answered the question correctly or not. If the user fails the question, the card will be presented again soon (e.g., in the next 5 or 10 card presentations), otherwise it will presented later. The program is supposed to schedule the next presentation of the card in such a way as to minimize the number of trials (repetitions) of each card and maximize the user’s ability to correctly respond to the question. In other words, the software developers try to minimize the user’s failure rate³ and to minimize the number of presentations required to correctly answer each card. However, Anki is more sophisticated than most flashcard apps. In Anki, users create notes in a deck. Anki notes come in various types. Each type specifies the fields contained in notes of that type. There is a basic built-in note type, but users can add their own. When users create a note, they assign a type to it. They can change their minds later and apply a different type to an Anki note. The note acquires the fields that are defined by the user-selected type. For example, one can define a Concept type to include ‘term name’, ‘term definition’, ‘usage examples’, and ‘synonyms’ fields. When a user creates a new Concept note (an instance of that type), those fields become available for editing⁴ in the instance. Users can create one or more flashcards for each note. When creating a flashcard for a note, users (implicitly or explicitly) select a flashcard template.⁵ The combination of template and note determines what information is presented on the front and back of flashcards that use that template. For example, a “Define-Term” template for a Concept note might specify that the front of the card should contain “Define: {{term name}}” and the back should contain “{{term definition}}”. The double curly brackets are used to reference (by name) a field from another. When the user practices with this flashcard, Anki substitutes the value of the note’s “term name” field with the value of the expression {{term name}}. For example, if the note’s “term name” field was “prurience” and the “term definition” field was “Having or encouraging an excessive interest in sexual matters”, the front of this card would like: Define: prurience and the back would be Having or encouraging an excessive interest in sexual matters. The Anki template mechanisms are flexible. Some conditional statements are permitted. You could, for example, choose to prefix a mnemonic field with the term “mnemonic:” only if there was ³Note, however, that many cognitive psychologists believe that failure is important to learning and that “retrieval difficulty” is desirable. For example, “Experts learn more from their mistakes than from what they get right.” (Hoffman et al., 2009) ⁴Anki does not support type extension, which means that you can’t define a type to be a subclass of another type or to inherit the fields associated with a more abstract type. If you want to re-use field specifications across types, you are on your own. ⁵In Anki 2, users can still control which templates are used for cards, but it’s handled differently.
Productive practice: A master maker
273
in fact a mnemonic. Flashcards can include arbitrary HTML for styling. They can have multiple fields on either side. A “cloze” mechanism is also provided. This enables you to easily create “fill in the blanks” type questions. A rudimentary hint mechanism is also provided. As in other flashcard applications, practicing with Anki consists of being presented with questions from a deck and attempting to answer them. Anki chooses which card to present to you according to an algorithm and various parameters you may set. For example, you can mix new cards with previously practiced cards, or show new cards before or after practicing the old ones. Unlike most other flashcard applications, however, with Anki you don’t merely indicate whether you were able to answer the question correctly. Instead, you rate how challenging you found the question: very hard, hard, easy, very easy.⁶ Anki associates an inter-card delay with each rating. For example: “soon”, 1 day, 4 days and 9 days, respectively. Between presentations of cards, delays increase systematically (unless you answer “very hard”). This is known as an “expanding schedule”⁷ which cognitive psychologists frequently contrast with a equally spaced schedules. The rationale for expanding retrieval schedules is that the more you master a card, the less frequently you need to practice it. In principle, many more schedules could be implemented than have been published in the test-enhanced learning literature or that flashcard developers have implemented. Anki also provides flexible ways of aggregating and searching for notes and cards. Cards keep a reference to the notes. Cards are also aggregated in decks. Decks can contain decks. When you study, you study with a particular deck. You can configure practice options for each deck. You can also apply tags to your note. You can create filtered decks: they will contain all the cards that match your search criteria. Filtered decks are similar to smart folders in OS X. Later in this chapter, I will show you ways of using many of the features described above. If you use flashcard software, consider using software that most closely matches requirements of productive practice, such as Anki. Anki reminds me of Calibre, the e-book management application I mentioned in chapter 12: powerful software in an awkward cross-platform user interface. (Anki 2’s user interface is an improvement, however.) You could alternatively try a simpler application with a more attractive user interface such as MentalCase⁸ by The Mental Faculty⁹. Or search for flashcard apps by feature on the Flashcard Apps¹⁰ website. The flashcard metaphor, however, is outdated. It is based on paper flashcards. Physical cards only have two sides. As we just saw, sophisticated flashcard software has a more complex ontology that includes what they call notes, types, templates, flashcards, fields, etc. It can be difficult to understand flashcard software using the old flashcard terminology. For example, the term “notes” does not convey the meaning of these data structures very well. Moreover, the terminology is disconnected from cognitive science. Hence, I propose ways to map flashcard concepts to productive practice concepts: • Anki 2 “notes” approximate instillers (in Anki 1, they were called “facts”), ⁶The Anki 1.2.8 rating button labels are “Again”, “Hard”, “Good” and “Easy”. They mix two concepts: time of next repetition and difficulty of the card. Anki 2 has fewer rating options. ⁷In the test-enhanced learning literature, the expression “expanding retrieval schedules” is used. However, productive practice is not merely concerned with recall. ⁸http://www.mentalcaseapp.com ⁹http://www.mentalfaculty.com/mentalfaculty/Home.html ¹⁰http://www.flashcardapps.info
Productive practice: A master maker
274
• • • •
flashcards in Anki approximate challenges. “decks” in Anki approximate kits, “ratings” in Anki 2 approximate mastery ratings, Anki calls “studying” that which I, and many other cognitive scientists, call “practice”. We use the term “study” to refer to delving. Many cognitive scientists use the term “recall practice” to refer to answering questions. (I prefer to speak in terms of “productive practice” because not all practice is merely recall. Moreover, even remembering involves a collection of active mental processes beyond mere “retrieval.”) • There is no notion of knowledge gem or mnemonic in Anki 2, • “review” has a different meaning in cognitive psychology than it does in the flashcard world. In cognitive psychology reviewing means rereading or reprocessing, as opposed to practicing. It’s an important distinction. I will therefore use the terms as they are used in psychology. To get more out of flashcard applications, users should implement productive practice workflows with them. I expect the concept of productive practice to catch on and for new productive practice applications to be developed free of the archaic flashcard terminology. Presumably, the new software will be able to import flashcards from applications like Anki, so that users can upgrade to the new system. In keeping with this, I will use productive practice terminology in this book rather than flashcard terminology. To implement productive practice, flashcard software needs to provide the following functions amongst others: • • • • • • • • • • • •
Enable users to group challenges into hierarchical collections. Enable users to tag challenges with built-in and custom tags. Enable users to collect challenges related to the same gem in one instiller. Provide multiple fields per instiller. For example, there should be fields to include notes, references, URLs, action items, etc. Provide useful built-in instiller types: for concepts (and vocabulary), facts, people, procedures, etc. Enable users to define custom types of instillers. Enable users to generate one or more challenges per instiller according to the challenge’s type and templates. Enable users to search instillers and modify multiple instillers at once. Fields should support rich text, media and outlining (cf. OmniOutliner). Enable users to link to local files. Support syncing between its mobile and computer applications. Be scriptable to enable users and third party developers to automate functions, such as accessing, creating, updating and deleting instillers.
Other software requirements are specified below.
Productive practice: A master maker
275
13.4 Capture and instillerize While delving, you occasionally encounter knowledge gems by which you would like to be deeply transformed. These are items to “capture” in your mindware development system. You might want to go the trouble of “instillerizing” them, that is, create instillers for them in your productive practice database. Chapter 2 described the kinds of transformations you might seek—new skills, attitudes propensities and so on. Some transformations require more effort than others. Chapter 14 describes how the type of transformation you are seeking ought to affect how you design instillers and how you practice with them. For now, it suffices to note that as you capture a gem, you should already be thinking in terms of the kinds of transformation that you want. So, as you delve a document, keep an eye out for any information that you want to be able to use, apply, or think with in the future. What gems you should capture depends largely on your projects (goals, deadlines, etc.), the information’s utility and its potency. An inspirational resource might also trigger new goals and new projects; alternatively, it may cause you to give up projects and goals. You also need to consider the cost of mastering a gem—some information, such as lists, is particularly challenging. There are different approaches to capturing knowledge gems. One approach is to apply an inner tag. For example, if you are reading in Skim, you could reserve a particular tag, such as “!capture” for information to be captured. You might even add some of the information that will figure in the instiller, such as a particular question or comment. Then you would add an item in your task manager (e.g., OmniFocus) to complete capturing the knowledge gems in the document. The task should contain information to identify the document. You could add a unique ID to the document and copy that to the task. When you’re ready to execute this capture task in the context of OmniFocus, you can copy the tag from OmniFocus and use it to find the PDF file—that should take less than a couple of seconds. At this point, you would create instillers for the information tagged as to be captured. Another approach is to capture individual gems to be instilled in your meta-doc or task manager. You would start a project (or file) for the document. Then, while delving, you’d add gems to it. Keeping this information in a meta-doc may be helpful for review purposes. As usual, the key is to be able to quickly navigate between related information. (See the meta-doc figure, above.) Another approach is to create instiller stubs in your productive practice application as you are delving a resource. Here, it’s a good idea to apply a tag (e.g., “stub”) to indicate that the instiller needs to be expanded. You’d exclude stub challenges from the filtered kits you practice with. I expect software in the future to make it easy for users to instillerize knowledge gems as they are delving. Users will be able to select a snippet of a knowledge resources (e.g., text) and issue an “Instillerize” command. This will create an instiller in the productive practice app. Robust links between the knowledge gem and the instiller will automatically be maintained, making it easy for you to navigate between the instiller and the instillerized gem. The delving app will provide an indication in the margins not only of whether you have instillerized the content, but whether you have instilled it (more generally, your degree of mastery of the knowledge gem). Imagining such futuristic capabilities might help you overcome cognitive productivity limitations of current software. In the next few sections, you’ll learn how to edit instillers.
Productive practice: A master maker
276
13.5 Design Instillers The core assumption of deliberate practice […] is that expert performance is acquired gradually and that effective improvement of performance requires the opportunity to find suitable training tasks that the performer can master sequentially — typically the design of training tasks and monitoring of the attained performance is done by a teacher or a coach. K. Anders Ericsson As with deliberate practice in general, productive practice hinges on designing effective and efficient training tasks. For productive practice, this boils down to selecting knowledge gems to master (the capture phase) and designing instillers to develop the desired level of mastery. Alas, learners tend to spend too much time surfing and delving, and not enough time designing their practice and practicing. You shouldn’t assemble instillers willy nilly and expect to effectively develop mindware. To design instillers well, you need to think like a great coach.¹¹ As I mentioned in Part 2, Coach John Wooden shaped training sessions very carefully. Becoming an excellent coach—an excellent instiller designer—itself requires a reflective mindset. You will find yourself reworking particular instillers, just as a coach gradually improves drills, and becoming generally better at designing them. Like other great coaches, you will derive new insights about how the mind works, and you will use those insights to improve your training. This book and other cognitive science resources can assist you in this process.
13.5.1 Instiller types and challenge templates Instillers are based on knowledge gems. They can be designed to instill mindware necessary to achieve desired learning outcomes (improvements). As such, it helps to know what kind of improvement you are seeking. In the psychology literature, deliberate practice tends to be framed in terms of behavioral improvement, as the opening quotation of this section illustrates. Test-enhanced learning tends to focus on mastering formal knowledge. In developing the concept of productive practice, I’ve cast a wider net. Instillers can be designed for all kinds of psychological outcomes. In chapter 2, I put forth some high level learning outcomes: • To master objective knowledge (e.g., knowing what, declaratively knowing how, knowing with, understanding), • to develop skills and abilities, • to master norms, • to develop affective attitudes, • to develop propensities (habits, self-regulation, active motives). ¹¹Various mental and behavioral therapies (e.g., speech and language therapy, cognitive behavioral therapy and rational emotive therapy) also involve practice. Productive practice concepts, techniques and information technology could usefully be added to the toolkits psychotherapists provide to their clients. Compare chapter 15.
Productive practice: A master maker
277
I also emphasized adult mental development, effectiveness and meta-effectiveness. These are not outcomes that can be obtained simply from delving. Many of them also require knowledge building and practice. One needs to design instillers to build the underlying mindware for these outcomes. These outcomes, however, do not necessarily map neatly onto your instiller typology (i.e., the types of instillers that you will define). Software is not yet at the point where you can simply tell it the kind of transformation you want and have it construct instillers for you. Still, it is useful when designing and using instillers to know the outcome you seek. When you create an instiller, you need to assign a type to it. The type determines the instiller’s structure (fields) and the challenges you can generate for it. Productive practice apps do not yet come with useful built-in instiller types. So you need to develop your own instiller types and challenge templates. This chapter provides you with some guidance. Start by building a general purpose template. Then construct specialized templates for concepts & terminology, procedures, names of people, etc. As I mentioned above, an instiller template consists mainly of a collection of user-defined fields and challenge templates. A generic template might contain the following fields: • A collection of question-and-answer pair fields. In Anki, each of these pairs must be specified as separate fields (e.g., “Question 1”, “Answer 1”, “Question 2”, “Answer 2”, etc.). • A mnemonic. It’s often useful to construct mnemonics to help instill mindware. Sometimes, the main thing one needs to learn is the mnemonic which acts as a cue to activate previously a stored memory. Thus, the challenge question is sometimes meant to elicit the mnemonic as an answer. For example, if you wish to be able to list (recall) the names of the great lakes, a challenge’s question might be “What is the mnemonic for the great lakes?” Its answer would be “HOMES”. Within a few trials, you’ll be good for life. In this field, you could also store the RD cue mnemonic, as described below. • Hints. Mnemonics, when properly designed, understood and recalled, are very powerful hints. They activate the target mindware. The hint specified in this field, in contrast, is a vague cue that still requires thought, at recall time, to activate the target mindware. Productive practice software should enable you to store many more than one hint ; otherwise, the hint might give away the answer, which defeats the purpose of practicing. It’s normally better to design mnemonics and use them to train yourself to give yourself hints when you need them than to passively rely on receiving hints. However, for bootstrapping purposes, being provided externally with a hint may be useful. (This could also help you learn to provide the particular hint to yourself.) • References. To facilitate revising the instiller, insert references to its knowledge resources, e.g., one or more web addresses, bibliographic references, BibTeX¹² “cite keys”, bibliographic software citations (e.g., Mekentosj Papers 2 “magic citations”), or linkbacks¹³. • Gem notes. This is a field for comments you might have about the knowledge gem. • Instillment notes. This is a field to keep track of what you have learned about learning the particular instiller. Whether you record the information or not, as a self-coach, it’s useful to ¹²http://en.wikipedia.org/wiki/BibTeX ¹³http://www.linkbackproject.org/
Productive practice: A master maker
278
articulate what is difficult about a particular instiller, why you have improved it in a particular way, and so forth. This will help you become better at designing instillers in the future. You could later search for and review all your instillers that contain instillment notes. This field can also explain the design of the instiller. • Tasks. This field is to record things to do with respect to the instiller or knowledge gem. For instance, you could record a gap in your understanding of the knowledge. Unfortunately, no current practice software is integrated with task management applications (such as OmniFocus.) • Project. It may be helpful to include an identifier of the project to which this instiller applies. Section “Address the meta-access problem” provided some tips on identifying and referring to projects that you can use here. This would allow you, later, to apply all the instillers that are pertinent to a particular project, such as a presentation or area of research. The instiller tag field can also be used for this purpose. I am not suggesting that you should include all of this information for every instiller you design. The previous bullet list describes types of information that are useful to include in instillers. As productive practice software improves, it will become even easier to add the information. Most of the fields are left blank in a template, because a template is not an actual instiller; it is the basis for constructing particular instillers. For example, you wouldn’t specify the question in a template, you would leave it blank. However, you can include stems, as I will illustrate below with respect to concepts instillers and conditional fields. The fields in the foregoing template apply to most instillers. Anki’s instiller (“note”) types do not support the object-oriented concept of inheritance. That means that to every new type of instiller you implement, you will need to manually add these generic fields. And whenever you edit generic fields, you will need to edit all the instiller types that implement them. Fortunately, one does not need many types of instillers. Anki does, however, provide some useful universal attributes, including custom tags, a binary “marked” flag, a binary suspended flag, and several statistical attributes. It is useful to tag instillers with custom information. This will enable you to find them more easily later, to cluster related instillers, to create “smart collections”, and to design practice sessions. For example, you could tag all the instillers related to Gottman’s books with “Gottman”. As you are designing instillers, you could create a smart folder of instillers tagged with “Gottman”. A smart folder is a collection of items that match certain search criteria. It’s sometimes necessary to focus on particular content. For example, you might wish to prepare for a presentation, an exam or a new job. If your productive practice app does not provide an explicit feature to learn content by a certain deadline, you need to use other features to obtain this effect. Smart folders are one way to do this. In Anki, creating a smart instiller-folder (which they call “filtered decks”) is easy. Once you select the folder, you can simply begin practicing with its items. You can focus even more specifically by editing the smart-folder criteria or creating nested smart-folders. However, as described in the section on practicing, however, you ought normally to practice with a mixture of instillers based on various knowledge gems.
Productive practice: A master maker
279
13.5.2 Grow your understanding Concepts are not simple things which you either grasp or don’t grasp, or which can be completely conveyed by an explicit definition or axiomatic characterisation. For instance, as work of Piaget has shown so clearly, and Wittgenstein less clearly, very many of our familiar concepts, like ‘number’, ‘more’, ‘cause’, ‘moral’ and language’, are very complex structures of which different fragments may be grasped at different times. Aaron Sloman I do not fully agree with Piotr Wozniak, the author of Anki’s predecessor, SuperMemo, who, like many others, claims that distributed practice will only help you if you already fully understand the material to be practiced. This illustrates a difference between distributed recall practice and productive practice. Productive practice is not merely designed to potentiate memory. It is also designed to develop one’s understanding, which often requires that one be challenged multiple times with the same or similar questions. As discussed in “Understanding understanding”, the concept of understanding is complex. Understanding is not an all-or-none matter: as alluded to in the opening quotation, understanding nontrivial concepts can take time. In chapter 2, I drew a distinction between transient comprehension of a knowledge resource and enduring understanding. In attempting to use knowledge, flaws in our understanding, or in the knowledge itself, may reveal themselves. This detection happens not only while delving, but also while practicing. Once these flaws are detected they can be repaired. By practicing with knowledge over time one comes to understand it better. (Therefore, if we followed Wozniak’s prescription, we might never practice, let alone utilize, certain concepts!) Consequently, it is acceptable to design instillers for knowledge that one has not yet fully understood. Designing the instiller itself may help one better understand and apply knowledge. Moreover, productive practice is not a sequential process of designing and then practicing specific instillers. One switches back and forth between the two. One evolves instillers as one’s understanding develops. Formal students have the benefit of teachers and textbook authors who design practice questions meant to progressively develop their understanding. How are you as a practitioner supposed to develop your understanding of material that you have not yet fully grasped? Re-reading is inefficient and often ineffective. This is one of the areas where Bereiter’s concept of knowledge building overlaps with productive practice. The idea is to design instillers that challenge you to explain and use concepts that you have not yet sufficiently understood. When you practice these instillers, you give them your best shot, knowing that you may fail. You can then mark the challenge as something to think about during “problem-solving time”, so as not to break the flow of your practice. In other words, don’t necessarily shy away from difficult challenges, though you can only afford to deal with a limited number of them at a time. By having this challenge within your practice set, your memory of the knowledge may remain sufficiently active (“primed”) in your mind that you attempt to think with the knowledge when it is applicable. And also, when you have a free moment you might be more likely to think about it (particularly if you keep track of knowledge gaps you wish to bridge.) This is a bootstrapping strategy. Creating effective questions is not always easy, even when one has mastered the material. One useful trick is to create challenges that ask you to consider whether the knowledge in question
Productive practice: A master maker
280
applied to any recent event, problem, knowledge resource or task. And if it was applicable, you could ask how to use the knowledge in this case, how well it works, why it works (or fails to work), how you can improve it, what knowledge is related, and so forth. This potent tactic is discussed further in the section “Grow monitors through review and reflection” below. It addresses, head-on, problems of “transfer” of learning. Another heuristic is to break the knowledge down into smaller chunks and to practice with them. A knowledge resource often contains several concepts. Rather than merely creating one or two challenges for an entire resource to be mastered, it may be worth creating instillers for each of several concepts. Given that collections are particularly difficult to master, I’ve devoted an entire section to them. One may be tempted to assume that productive practice is an alternative to techniques that promote understanding. However, one can design challenges to promote understanding. For example, it is widely accepted that drawing concept maps is a useful way to develop comprehension of many types of material. (Concept maps are diagrams that depict the important ideas in a knowledge resource and the relations between them. ) One can set as one’s challenge to draw a concept map, or other kind of meaningful diagram.¹⁴ In other words, practicing elaborative challenges can lead to deep comprehension. To practice productively means to systematically and regularly practice in a way that produces manifold mindware, understanding and (sometimes) new knowledge. While it’s important to directly foster deep and varied thinking about and with knowledge through productive practice, it is often also useful to engage in what may appear to be rote learning. It it is often useful to memorize definitions, equations, rules, etc., so that they may be consulted mentally without recourse to external information stores. For, as described in Part 2, building up long-term working memory is critical to the development of expertise and may be enhanced through deliberate practice. Once you reach a critical threshold, the information becomes autonomous and can more readily be used in progressive problem solving. You can then think more frequently with it and about it. This bootstrapping strategy can assist in building new knowledge, particularly if you ask the right questions outside of productive practice. (Compare Reflective practice and deliberate performance.) For example, by practicing the concept of bid, you can think more often in terms of bids. You will start to notice boundary conditions. You might find the need to explore, derive or apply other concepts to complement bids, such as concern (see Internal Motivators). Sometimes it is useful to learn information by heart temporarily in order to build up other mental structures. In the following sections we gradually elaborate an example of this, regarding harsh startups. The example culminates in the section Avoid harsh startups with your new mindware.
13.5.3 Divide and conquer As we saw in chapter 12, while delving, you analyze a knowledge resource in multiple ways. For example, you might tag and extract major claims, concepts, etc. You might outline sections, etc. Capturing gems is another type of analysis. For each gem, you might create one or more instillers. And for each instiller, you might create more than one challenge. There are several advantages to creating multiple challenges for the same gem. This can help you better recognize the information, ¹⁴Karpicke & Blunt (2011a) make a similar point.
Productive practice: A master maker
281
interpret it, and utilize it when it is applicable. By phrasing the same question in multiple ways you become less dependent on the particular wording to trigger the mindware. Keep in mind that a major property of the mind is its ability to form precise associations. As we will see, one needs not merely to establish a mental path to information, one needs to develop monitors to ensure that the various conditions to which a knowledge gem applies will actually trigger the appropriate mindware. I will provide examples of this in the section on learning vocabulary. This section focuses instead on creating multiple instillers for a knowledge resource. I will use Keith Stanovich’s book, What intelligence tests miss¹⁵, as an example. You will recall that I summarized the thesis of Stanovich’s book in chapter 1. For many readers, this book is very potent —it can radically improve their understanding of human behavior. It’s also of high caliber: It explains why “smart” people sometimes do stupid things in a way that makes deep, architectural, mechanistic and conceptual sense. It is well grounded in empirical and theoretical cognitive science research. It’s also potentially very useful. For example, it can serve as a compelling motivator to adopt mindware and dispositions that can help the reader make better decisions in every day life. (Remember the fly-by-night investor.) Such a book is well worth not merely reading (and hence likely forgetting) but practicing productively. Now, you could create a simple instiller with one challenge, such as “What is the thesis of Stanovich book, What Intelligence Tests Miss¹⁶”. This is a good starting point, because it requires that you invoke the major claims, concept and argument of this book. However, this question would be of limited use. With global questions, it is difficult to assess one’s answers and to detect objective knowledge gaps. Also, it is difficult to derive monitors from single very abstract questions. It is often easier to master several simple short challenges than to master one difficult challenge. That is to say that constructing several challenges out of a greater challenge, can facilitate mastering the composite challenge. While the main idea of Stanovich’s book is simple (“some smart people tend to do stupid things because they are overly cognitively-miserly”), it contains many deep concepts, findings and claims that might require considerable effort to master, particularly if you’re not very familiar with the literature it cites. So, amongst all the potentially new information in the resource, you need to decide what you would like to instill in yourself. Here are a few questions you can use to master Stanovich’s book to a certain degree (they leave out several gems, e.g., the role of memes). They are annotated with tips to help illustrate how to design instillers. I’ve listed some hints you can use. Only call on the hints if you need them (at practice time). Consider creating a challenge to instill the hint so that you can reconstruct the hint yourself. Most of the answers to these challenges are in “A template for conceptual understanding”, above, but to fully comprehend you will need to read the original book. • What is the thesis of Stanovich’s 2009 book? Hint, the book’s title is What Intelligence Tests Miss¹⁷. The title is in the hint because it’s a powerful cue. You might be tempted to create a mnemonic to break the thesis down (as noted above). However, because this book is particularly coherent, you may be able to get the gist of it without mnemonics. For a terse summary, see “Keith Stanovich (2009). What Intelligence Tests Miss” above. ¹⁵http://www.keithstanovich.com/Site/Books.html ¹⁶http://www.keithstanovich.com/Site/Books.html ¹⁷http://www.keithstanovich.com/Site/Books.html
Productive practice: A master maker
282
• Explain cognitive miserliness by describing an experimental demonstration. Hint: What percentage give the wrong answer? This refers to the experiment, described in the section, “Cognitive miserliness and its antagonists”, that asks the subject whether a married person is looking at an unmarried person. This challenge is useful because it enables you to learn with an excellent concrete example. It is acceptable to use mnemonics to remember the first part of the answer, because it’s arbitrary. But you must not use mnemonics to remember the second part of the answer, because you need to understand it. By practicing answering this challenge you will deepen your understanding of cognitive miserliness. You may wish to reflect on why it’s an excellent example. One of the reasons is that it has affective import. One doesn’t want to be amongst the 80% who fail the test. Another is that it shows just how easy it is to fall into a cognitive miserliness trap. If, when you think of cognitive miserliness, you think of this example, you will be well on your way to mastering it. • How can type 2 reasoning errors be avoided, according to Keith Stanovich? • With what concept does Stanovich distinguish intelligence? • With what concept does Stanovich distinguish rationality? • Give examples of thinking dispositions. This question is actually quite difficult, because it calls for a list. So you should use a mnemonic. I provide an answer and explanation to this question in “Consider the opposite, above”. • What concepts of Keith Stanovich’s book on intelligence were new to you? • How does Keith Stanovich’s architecture of the mind explain individual differences¹⁸ in intelligence and rationality? An answer: There are few continuous individual differences in the autonomous mind. The algorithmic mind is responsible for individual differences in fluid intelligence. The reflective mind accounts for many of the differences in rational thinking dispositions (amongst other things.) This is a useful challenge because it shows how Stanovich explains some of the key individual differences involved in smart people doing dumb things: their algorithmic mind is capable, but their reflective mind lacks certain things. This also is a good launch pad for your own reflections on the architecture of the mind. It can help you to think in terms of the major components of the mind. (Compare chapter 5 on mental architecture.) • What politician does Stanovich use as an example of a cognitive miser? Why is George Bush a good example of cognitive miserliness? These two challenges should be built into one instiller because otherwise one will cue the answer to the other. When answering the first challenge, for efficiency, you need only mention George Bush. The sibling challenge will handle elaboration. The example of Bush may be useful to you if he tends to come up in conversation. This will afford opportunities for you to use your knowledge, whether you verbalize it or not. By practicing this question a little bit, the gem will become autonomous because thinking of Bush will make you think of cognitive miserliness. (This is a bootstrapping strategy.) • How does Beaudoin’s concept of mindware differ from that of Stanovich? This question is important because the evolving concept of mindware is critical to Stanovich’s thesis, to this ¹⁸http://en.wikipedia.org/wiki/Individual_differences
Productive practice: A master maker
283
book and to the future of cognitive science. Without invoking mindware (and architecture of the mind), one can’t adequately explain cognitive miserliness. These questions help to illustrate that challenges may call for zestful thinking dispositions that themselves are opposite to cognitive miserliness, which Keith Stanovich identifies as the source of much irrationality. Still, we will see that in addition to questions that require such elaborative answers, simpler ones that promote cognitive reflexes can support reflective action, progressive problem solving, etc.
13.5.4 RD cue mnemonic system: From free recall to cued recall This section describes a mnemonic system that you can use to design instillers for words, names, phrases and other expressions. It implements the principles mentioned above. Before describing this system, I explain the ideas that inspired me to design it. At its best, a mnemonic system is an information encoding and processing system designed to facilitate recall of information. It productively leverages key principles of memory.¹⁹ Alas, some popular mnemonic systems, particularly those that rely heavily on imagery, are of limited use.²⁰ ²¹ In this section, I introduce a new mnemonic system, the RD cue system, that leverages several key principles of memory described in chapter 5: (1) All memory is cue-driven; (2) cues are subject to overload; and (3) all memories are reconstructed. The first principle implies that whenever one recalls an item from memory, this recall is triggered by something. The first two principles are also basic, associative properties of animal memory²², including human memory. It occurred to me several years ago that if I could find a way to enable people to systematically convert free-recall tasks into cued-recall tasks, then I could provide them with several memory benefits. Let me first explain the difference between the two types of recall. An example of a free recall task is to try to remember an erstwhile friend’s surname without additional information. Being ¹⁹Amongst many books on memory, I recommend two short ones: Surprenant & Neath (2009) and Einstein & McDaniel (2004). The first one is written for readers who are familiar with cognitive science, whereas the second one is written for general readers. For an abbreviated version of the first book, see Neath & Surprenant (2005). See also Part 2 of this book. ²⁰A problem with visual mnemonics is the amount of training required to efficiently and effectively create them. Trying to generate them can consume time and waste mental resources. Morris et al. (2005) demonstrated the superiority of two other memorization strategies over visual mnemonics for remembering names. They argued that attempting to come up with visual mnemonics distracted participants. That is, it distracted participants from encoding other information. (It is difficult to focus on a conversation with a person you’re being introduced to if you’re trying to connect his name to a vivid image.) A journalist turned mnemonist documented the extensive effort required to achieve a high level of visual mnemonic skills. He concluded that he could still not adequately apply these techniques in his every day life (Foer, 2011). However, visual mnemonics can be of some use if you have sufficient time to generate them and you develop the skill gradually over years. You need to decide whether the gem is worth the effort. ²¹To provide some context for the RD cue mnemonic, consider that Worthen & Hunt (2008) emphasize four major processes in mnemonics: retrieval cues, organization, mental imagery and elaboration. For some recursive fun, I’ll use this tetrad to illustrate it. The acronym “ROME” as an organizational mnemonic to memorize this list of abstract terms. An example of mental imagery for this list is to imagine a Roman (“ROME”) orator organizing pool queues (they stand for “retrieval cues”) to form a portrait (stands for “mental imagery”) of a woman giving birth (stands for “elaborate” via the concept of labor). This mnemonic could further be elaborated by noting that some Roman orators used such mnemonic systems (Post, 1932). With practice this mental “image” could be a retrieval cue for the acronym; the acronym could then be a retrieval cue for the list. This then is retrieval cue chaining. One would want to elaborate the targets of these cues. The ROME mnemonic illustrates that mnemonics themselves normally require practice. The fact that the scene in ROME is not clearly visualizable also shows that there is more to imagination than meets the eye. Imagination is fragmentary (Beaudoin, 2013). Compare Sloman’s afterthoughts on analogical representations (Sloman, 1975). Higbee (1988) is a more accessible treatment of mnemonics than Worthen & Hunt (2008). ²²See for example the work of McGill University Psychology professor Andrew Baker on learning correlations (Baker & Mercier, 1982; Baker, Berbrier, & Vallee-Tourangeau, 1989; Baker, Murphy, & Mehta, 2003)
Productive practice: A master maker
284
told their first name or their spouse’s name would make this a cued-recall task: the first name is a cue for the surname. Thus, in cued-recall, the participant is provided with a cue, a bit of information that was previously associated with the target. (I review what makes for a useful association below.) The likelihood of remembering a target is much higher with a good cue than no cue. The speed of accessing a target with a cue is also much faster. It approaches the speed of short-term working memory. Furthermore, as I alluded to in chapter 2, free recall performance declines significantly with aging, but cued-recall is comparatively robust. As you would expect, the superiority of cued-recall reflects the cue-driven principle and the cue-overload principle. Free recall tasks, then, are also cued recall tasks; it’s just that the cues are implicit. The RD cue system is also based on a mode of memory retrieval proposed by Norman & Bobrow (1979). They suppose that the target of a memory may be specified by a more or less specific description. A description is a way of viewing the target from a particular collection of perspectives. A description can act as a cue for the target memory. They propose the following: the effectiveness of […] descriptions for retrieval is determined by two properties, discriminability and constructability. Discriminability is the ability of a description to discriminate among all possible records in memory at the time of retrieval. Constructability is the likelihood that an appropriate description will be constructed at the time retrieval is desired. (p. 107). The R&D cue system requires that one create cues that are reconstructible (“R”) and discrimative (“D”). Look at it this way. If, based on much experience, stimulus C is highly predictive of a stimulus T, but not very predictive of other stimuli, then C discriminates T. C is not overloaded, hence C is a good cue for T. This suggests that a good mnemonic for any given target memory would be a cue that only ever appears before the target. By this reasoning, other things being equal, a stimulus that has already been frequently encountered before (such as a common dictionary word) is not likely to be as good a cue for some target as a new one that one will reliably associate with the target. If, at test time, one wishes to recall a target for which there is a discriminating cue, all one needs to do is remember the cue. Of course, the cue will only be effective if one can remember it! Hence, as Norman and Bobrow pointed out, to be useful, a cue must not only have high discriminability for its target, it must be reconstructible.²³ Many mnemonics fail because they are not reconstructable or not discriminative. One of the aims of this chapter is to dispel the myth that question-and-answer practicing is simply distributed recall practice or rote practice. Productive practice is a type of test-enhanced learning. Practice recalling an item (whether one does it deliberately or incidentally) is normally required to develop an ability to recall that item over long intervals. The ability to recall information on demand is an important aspect of expertise. For example, there are articles, judgments and definitions that a trial lawyer needs to be able to cite instantly. As described in chapter 5, experts have long-term working memory for key information. In order for productive practice to be a useful addition to a practitioner’s cognitive toolkit, it needs to optimize memorization–that is, to minimize ²³Compare the mediator effectiveness hypothesis (Pyc & Rawson, 2010), discussed below.
Productive practice: A master maker
285
practicing time and to maximize probability of accurate recall. You can use productive practice software to bootstrap such memorization, i.e., to make the information easily available. Once you can recall something somewhat easily thanks to productive practice, you will tend²⁴ to be able to refer to this information when you need it (it will be available). By repeatedly accessing the content from memory outside of deliberate practice contexts, this information will become even easier to access. Thus, productive practice can establish a healthy positive feedback loop. Indeed, a goal of productive practice is to make mindware sustain itself without the need for further deliberate practice. If you adopt productive practice, you will design many instillers that are specifically meant to facilitate recall of information. Some of this information you want to be able to use at “run time” to solve real-world problems. Some of this information is there only to bootstrap the development of other mindware. And so, instillers have challenges that are directly pertinent to knowledge gems and ancillary challenges as well. In light of this background, the RD cue mnemonic system goes like this. If you want to remember a target expression, then construct a unique cue expression that you will be able to remember—i.e., a “reconstructible” cue, to use Norman and Bobrow’s expression. Then associate this expression with the target expression. Because the cue is an expression that is reconstructible and that discriminates between memory contents, I call this a reconstructible, discriminative cue expression mnemonic, or RD cue mnemonic for short. Using a reconstructible cue will make the cue easier to remember. It will be very rare for a cue to perfectly meet the criteria of reconstructability and discriminability. The criteria are matters of degree–not all-or-none. For example, if you are familiar with cognitive psychology, you could use the term redintegration as a RD cue for “RD” itself. Both words are memory constructs with related meanings; they both start with the letter “r”, closely followed by “d”. Redintegration is when a small part of a memory can remind a person of a much larger fragment of the memory. In contemporary memory research, redintegration is defined as the use of long-term memory to facilitate recall. An example may help here. Suppose that you want to memorize the name of a new person. You could create a RD cue for that person.²⁵ To be easily reconstructible, the cue should describe features of the person that may readily come to mind such that merely thinking about the person will remind you of this description. It should also be discriminative, such that it does not activate memories of several other people (otherwise, the cue will be overloaded and hence less effective). Suppose that the target person is your boss’s former accountant and you know that he is tall. Your boss’s name is Frank and the target person’s name is John Doe. The RD cue for “John” could thus be “Frank’s tall former accountant, John.” Even though you know that John is Frank’s former accountant in that you would answer “yes” to the question “Is John Frank’s former accountant?”, because not all measures of memory correlate²⁶, you might not be able to remember the RD cue for John—unless you practice. The cue is discriminative, because you don’t know other people who fit that description. So the trick here is to create a challenge to instill the RD cue. The instiller might include the following challenges: • Question: What is the RD cue for John Doe? Answer: Frank’s tall former accountant. ²⁴The “tend” qualifier is important. Remembering involves a multitude of stochastic processes and is always relative (Roediger, 2008). If someone claims their mnemonic system easily guarantees recall they’re being deceitful or are informed. ²⁵In contrast, most mnemonic systems rely on relatively arbitrary information. … ²⁶Roediger (2008).
Productive practice: A master maker
286
• Question: Frank’s tall former accountant. Answer: John Doe. You can help make the RD cue self-sustaining by getting in the habit of thinking of John Doe as “Frank’s tall former accountant.” While instilling a RD cue takes a few trials, it has several benefits. For example, creating RD cues does not require a fertile imagination because you can normally easily discover a unique “signature” for the target.²⁷ Constructing a RD cue also contributes to its effectiveness–it is an elaborative mental process. Also, unlike many visual mnemonics, interpreting RD cues does not require elaborate interpretation at run time: it relies on having constructed a cognitive reflex. You can think of an established RD cue as a “path” to a memory. After training, recalling the RD cue’s target becomes a matter of remembering the (inherently memorable) path, i.e., the RD cue. If you are familiar with mnemonic systems, you might realize that the RD cue system is similar to the keyword method²⁸. However it is more general, it does not rely on concrete imagery, it allows cues and targets to be lengthy expressions as opposed to single words, and it is aimed at developing all kinds of memories, not just mastering vocabulary. These distinctive properties are particularly useful when you need to learn lists and even trees (hierarchies), because RD cues can be chained and can contain RD cues.²⁹ I present several examples of this system below. The next chapter shows how such a RD cue can be used to instill content into long-term working memory, and how that can be used to help learn new skills, such as avoiding harsh-startups with your partner.
13.5.5 Instiller design rules Here are some generic rules you should keep in mind when designing instillers. • To streamline the construction of instillers, try to automate data entry. For example, copy/paste content from the knowledge resource to the instiller, or use other capture features provided by your software. Use the syntax provided by the practice software for one field to refer to another. Become familiar with challenge templates and leverage them. • Ensure that your instillers help you develop manifold types of mindware in relation to different types of knowledge. Do not focus exclusively on memorization, but do not disdainfully exclude memorization either—it has its place. Ensure that some instillers help you develop understanding of problem-centered knowledge in a manner that favors elaboration and thinking (e.g., use why and how questions that call for explanations). Don’t forget that you can use instillers to implement condition-action rules (as described in the next section). • Choose your battles. Just as you need to judiciously select knowledge gems to master, choose your challenges wisely. As you get the hang of productive practice, you will get a sense of the cost of different types of instillers. You will find that lists are amongst the most expensive gems to master—particularly if you don’t use the tips provided here. ²⁷The RD cue system can also be referred to as the signature-cue mnemonic system. ²⁸http://www.jstor.org/discover/10.2307/1001598?uid=2129&uid=2&uid=70&uid=4&sid=21102289869241 ²⁹This is a good example of the advantage that cognitive science confers to development of mnemonic systems. Given that Artificial Intelligence
(computational modeling of real or possible minds) is a discipline at the core of cognitive science, a cognitive science mindset helps one to think in terms of data structures and algorithms which is helpful for developing mnemonics.
Productive practice: A master maker
287
• Watch out for obscurantism in knowledge resource. Names do matter. You don’t need to regurgitate content, as some students feel compelled to do. So, ditch the source’s abstruse terminology. For example, the expression “negative suggestion effects of testing”, found in the literature, could be renamed “undesirable suggestion effects of testing” because “negative” can also mean “taking away”. The misleading term “transfer” has, by its name, caused more ink to be spilt than perhaps any other in education. Swaths of literature in social science and humanities are difficult to comprehend because of conceptual confusion. When delving, focus on understanding the problem that authors are addressing, or ought to be addressing, and try to carve up conceptual space in the most meaningful manner. In other words, engage in conceptual analysis and knowledge building. Then design instillers to master the comprehension you have built. – When possible, construct answers that contain words and concepts that you already master. For example, if you are learning the meaning of the word “demotic” and are not familiar with the term “idiomatic”, don’t use “idiomatic speech” as the answer to a challenge whose question is “What is the meaning of the term demotic?” even if that is the dictionary definition you encounter. In this case, you might try “ordinary, colloquial speech” instead. Alternatively, create other challenges to help you master the constituent concepts of the question, and learn them first. In this example, you would create an instiller for the term “idiomatic”. (This is an example of divide-and-conquer). • Don’t express challenges as multiple choice or true or false. They tend to carry undesirable suggestion effects (Roediger & Karpicke, 2006). Also, they tend to merely rely on recognition rather than recall and reconstruction. • Continue to tweak your instillers as needed. Here are some conditions under which to revisit instillers: you find yourself failing a challenge many times; you fail to apply the knowledge when you need it; your understanding of the gem has changed; or you have learned new instiller design principles. • Revisit sections of this book with a view to improving your understanding of relevant concepts. I expect instiller design will become easier in the future if the practice catches on, new cognitive science is brought to bear on it and software manufacturers develop productive-practice software that meets the requirements described in this book.
13.6 Practice with these general considerations in mind Everything in life worth achieving requires practice. In fact, life itself is nothing more than one long practice session, an endless effort of refining our motions. When the proper mechanics of practicing are understood, the task of learning something new becomes a stress-free experience of joy and calmness, a process which settles all areas in your life
Productive practice: A master maker
288
and promotes proper perspective on all of life’s difficulties. Thomas Sterner For practice to be rewarding, it needs to be effective and efficient. On the surface, the practice phase of productive practice is very simple. All you need to do is: Receive the challenge; try to execute the challenge (which is normally to simply answer the question); and rate your mastery. However, practicing itself is a complex skill. The following subsections provide some general tips for this. Subsequent sections will help you design, and practice with, particular kinds of instillers. If you have engaged in deliberate practice before (e.g., to learn an instrument or play a sport), or if you remember what you read in the chapter on deliberate practice above, you know that one must practice several days a week to benefit from it. You also know that it is difficult to keep up with practice. Practice is effortful and there are many competing demands on your time. It is partly in recognition of the need to convince my readers of the relevance of productive practice that I wrote this entire book and the following chapter, which summarizes its benefits, in particular. I also refer you to general practice books and inspirational books³⁰ to help you motivate yourself to develop effectiveness, knowing that practice (in one form or another) is key to self-improvement, and that one can design one’s productive practice routines to optimize this process.
13.6.1 Set your practice time Only you can decide when, how long, and how often to practice. However, the principles of psychology (ultimately, the mechanisms of your mind) dictate how effective the practice you choose to engage in will be. The more mindware you choose to develop, the longer you will need to practice. Every knowledge gem has a particular theoretical minimal cost, i.e., theoretical minimal amount of practice required to master, assuming that you have optimally designed your instiller and that you practice optimally with it. How much time a day to spend practicing should partly be a function of the content that you wish to master and how much time per day you are willing to spend on developing effectiveness. It’s more difficult to gauge the former than the latter. So, your time constraints will throttle the number of instillers you practice with. Practicing 15-20 minutes a day, a few days a week, may suffice. If you find that you have too much content to practice with, e.g., because you are failing too many challenges, you may need to prune or reschedule some instillers. As with performance practice (e.g., playing musical instruments), you will occasionally need some longer practice sessions to work through difficult challenges. This sometimes calls for focusing on particular challenges and overlearning the gems. Also, your practicing needs will vary according to the amount and purposes of your delving (the main input to productive practice). You also need to allocate time to plan, schedule and design instillers and practice sessions. Remember that coach Wooden’s success had a lot to do with the time he and his assistant coaches spent planning practices. Some of this planning can take place as part of your weekly review. To make time for productive practice you need to prioritize it with respect to processing knowledge resources (surfing, inspecting and delving content.) Effectant people love information ³⁰For example, Guadagnoli (2009), Robbins (1991), Sterner (2006). Many of Robbins’ principles can themselves be instilled through productive practice.
Productive practice: A master maker
289
and knowledge. Thus, ironically, they are susceptible to spending too much time reading. We are drawn to news, for obvious evolutionary reasons. In chapter 9, I suggested that you monitor your information processing time to ensure that you spend enough time delving relative to the time you spend surfing. Why? Because you get more bang for your temporal buck out of delving. You get an even bigger bang by practicing. For many, the best way to make time for productive practice is to cut down on surfing and delving time.
13.6.2 Respond to challenges Those texts that mention the distributed practice effect often devote a paragraph or less to the topic […] and offer widely divergent suggestions – many incorrect – about how long the lag between study sessions ought to be […]. The present studies begin to fill in the gaps that have maintained this unsatisfactory state of affairs and suggest the need for research that applies distributed practice principles within classrooms and embeds them within educational technologies. Nicholas Cepeda et al. When presented with a challenge, you need to try to respond to it as well as you can and then rate your mastery. If it’s a question, that means trying to answer it. You can design challenges to elicit all kinds of cognitive processes. Some challenges merely call for you to recall information that you’ve delved or created. Others call for you to solve problems, imagine possibilities, simulate procedures, construct examples, remember recent events to see if they match a given rule, experience feelings, etc. What if you cannot rapidly respond to the challenge? Here, you have a choice. You can take a few minutes to elaborately construct the best response. Exactly how you ought to think at this point depends on the challenge and what you know. If the problem is one of remembering, then you could try to recall explicit information and representations that might cue the target. For example, if you are trying to remember the name of an important algorithm, try thinking of key terms it involves or the source of the information. If you had previously established a RD cue, then try to remember it. In other words, leverage the principle that all (some would say most) memory is cuedriven. Pyc (2010) explains the effectiveness of test-enhanced learning by postulating that people naturally search for mediators (internal cues) that might trigger the target memory. You can train yourself to do this. Think of yourself as constructing a mental path to the target or searching for the path you previously constructed. (Compare the concept of retrieval structure.) Recall practice is partly a matter of developing a navigational skill. This fits with David Perkins’ view of intelligence as “knowing your way around” (Perkins, 1995), in this case, knowing your way around your own mind. For efficiency, it’s often necessary to postpone editing difficult challenges until after the practice session is over. You can do this by marking the challenge for future review, possibly tagging it as difficult, and also setting your mastery rating accordingly. Postpone viewing (or listening to) the answer when it is on the tip of your tongue. Through spreading activation of mindware structures, the target information may be primed sufficiently for you to correctly respond later. Here you could use Anki’s ability to “bury” a challenge, i.e.,
Productive practice: A master maker
290
to postpone it until after the app is next launched. Or reschedule it. Future software may provide more flexible scheduling options. Often, through incubation (or after sleeping) the answer will come to you. By postponing experiencing the answer, you might have the experience of producing an appropriate answer yourself. This potentiates the memory more than receiving the answer. This may also shave a few seconds off your practice time. However, you should not use this strategy too often for the same challenge, lest the challenge become a cue for whatever dead-end mnemonic cues (mediators) you are relying on. Please refer to the other tips I provide in this chapter for dealing with difficult gems and instillers. As I alluded to in the section on co-opting flashcard software above, a proper productive practice app, unlike most flashcard systems, must not merely allow you to indicate whether or not you know the proper response to a challenge. It should allow you to grade how difficult you found the challenge, e.g., • • • •
Very hard (Try again soon) Hard (Try again in 2 days) Easy (Try again in 12 days) Very easy (Try again in 16 days)
The app should calculate the optimal time at which to next present the challenge. Selecting an option has two effects: rating mastery and scheduling the challenge. How you answer these questions will substantially affect how much time and overall effort you will expend on a challenge and how well you will master it over time. This raises the question: what is the optimal algorithm for scheduling challenges? How ought software decide when to present challenges? When should you override the software’s recommendations? Cognitive psychologists have examined a variety of challenge-spacing schedules and related factors. For example, they have compared equal spacing schedules with expanding schedules. Under equal spacing schedules, after a challenge is presented, software waits a fixed period of time before presenting that same challenge again. With expanding schedules, the interval between successive presentations (and successful answer) of a challenge increases (that’s a logarithmic function). The results are unclear. For example, in a review of the literature on distributed recall practice for verbal tasks, Cepeda et al. (2006) wrote: The effects of nonconstant (i.e., expanding or contracting) learning schedules on retention are still poorly understood. Expanding study intervals rarely seem to produce much harm for recall after long delays, but there is insufficient data to say whether they help. This has not stopped some software developers from assuming that expanding study intervals work better than fixed intervals. For example, Wozniak & Gorzelanczyk (1994) and SuperMemo World (n.d.) offered a “universal formula” designed to space repetitions at an interval that will produce 95% retention, based on Bahrick and Phelps’s (1987) proposal that the ideal spacing interval is the longest ISI before items are forgotten. ³¹ ³¹Bahrick and Phelps’s proposal is the logical implication of the widely held “desirability difficulties” assumption mentioned elsewhere in this
book.
Productive practice: A master maker
291
Here is the conclusion from a recent related study: [T]here was no evidence that a particular relative spacing schedule (expanding, equal, or contracting) was inherently superior to another. Although expanding schedules afforded a pattern of increasing retrieval difficulty across repeated tests, this did not translate into gains in long-term retention. Repeated spaced retrieval had powerful effects on retention, but the relative schedule of repeated tests had no discernible impact. Karpicke & Bauernschmidt (2011) In an attempt to make sense of the literature on spacing schedules, Roediger, III concludes: [T]esting should not be done under massed or even closely spaced fashion; if the literature is clear on any point, it is that repeated testing under conditions in which retrieval is easy leads to poor long-term retention. […] But what about the mechanism for spacing of retrieval? Our data reviewed above suggests that the critical ingredient is encouraging fairly difficult retrieval, especially on an initial test. Beyond that point it probably does not matter whether students test themselves using expanding or equal interval conditions. What matters is repeated space retrieval with feedback if an error is made. Roediger & Karpicke (2006) Thus, while the cognitive psychology literature has clearly demonstrated that distributed recall practice is effective, it does not yet provide us with precise schedules, let alone more general algorithms, for spacing challenges. It does, however, suggest some principles, including the notion of “desirable difficulties”. In other words, in order for a challenge to be effective, it must be somewhat difficult. Cramming does not produce long-term gains as reliably as spaced practice. Trying to recall information without waiting for some forgetting (as in cramming) is too easy. Why? Because the mind stores recently used information in more readily accessible states. In order to signal to lower mental layers that information needs to be accessible on a longer term basis, one needs to wait. Waiting makes it appear more difficult. As you retrieve information often enough over a period of time, the mind gradually stores it in a place or state that is both enduring and rapidly accessible (long-term working memory). Alternatively, the storage location may not shift, but the mind update its indexes.³² Also, inter-challenge intervals of greater than one day are superior to smaller intervals. (See “Why practicing works: Explanations of test-enhanced learning”.) Another interesting principle, discovered by Roediger and his colleagues, is that the desirable difficulty effect is not homogenous across trials. It appears that that the first time you try to recall information, such as a cue and its target, should be sufficiently long after your first processing of it. This would help explain why some research has found that expanding retrieval schedules are not as effective as fixed spacing schedule: the initial retrieval interval³³ in expanding retrieval tests are too small, making the first retrieval too easy. Still, the fact remains that the psychology literature has not yet studied a sufficient number of schedules to provide us with as much guidance as we’d like to have (Rawson & Dunlosky, ³²Compare the discussion of indexes in Sloman (1978). ³³That is the period of time between being presented with a cue-target pair and being asked to produce the target given the cue.
Productive practice: A master maker
292
2011). Most studies are performed in artificial laboratory conditions under short-term schedules³⁴. The few more realistic studies are performed with students learning classroom material. None of the studies directly address the concerns of researchers (as opposed to students) themselves being involved in productive practice. Also, although researchers have examined simple schedules, they have not yet examined complex algorithms that factor-in types and features of instillers, number of challenges pending, assessment of difficulty of material, presence of RD cues, sibling challenges, users overriding recommendations, variable session length, non-monotonic schedules, user created and edited instillers, algorithms that adapt to the user’s response patterns, individual differences, mobile productive practice, etc. Moreover, they have focused on declarative memory and have not systematically considered different types of mental enhancements. It is unlikely that the simple schedules examined empirically thus far, which ignore the factors I listed in the previous paragraph, will prove to be optimal. Nor is it likely that the schedules implemented by software such as Anki are optimal, for the same reasons. It is now possible to create productive practice applications that collect data from millions of consenting users to compare different scheduling algorithms. Software providers could vary all kinds of parameters and measure the effects on mastery ratings, response time, and other dependent variables. I.e., they can use the principles of lean product development to optimise software. In the future, software itself will learn from large populations of users to program users so that they in turn can learn better. It is reasonable to expect that productive practice apps, like other AI software (e.g., speech processing), will become increasingly and surprisingly better. In research and development, as in other types of learning, many of the major gains happen early on. So, we may soon witness major progress in both cognitive science and its applications to productive practice. A reasonable approach is to practice in a manner that is consistent with principles that make sense of the current data. In real-world settings, it is necessary to utilize somewhat expanding schedules because there is simply too much material for equal spacing. Normally, productive practice apps ought to schedule challenges so that you are quite likely to respond correctly, but also stand a chance of getting it wrong. Failing a challenge sometimes even improves long-term retention or mastery,³⁵ provided that there is corrective feedback (Roediger & Karpicke, 2010). As mentioned above, you may wish to mark a failed challenge for review if you feel that some reflection will help you do better later (or perhaps improve the instiller).³⁶ Scheduling a challenge too far in the future is not helpful for that challenge, but may be required in order to deal with more pressing instillers. Sometimes, you need to override the app’s choices and manually schedule a challenge. This is to be expected because current scheduling algorithms are not optimal and you may have some relevant information that the software cannot accommodate. ³⁴Cepeda et al. (2006) reviewed over 400 research papers on distributed verbal recall practice. Only a dozen examined intervals of a day or more, and fewer than 10 of them examined intervals greater than a week. Cepeda et al. (2008) examined a greater number of schedules than any prior published study. However, even this outstanding study only considered a small fraction of possible learning schedules. Optimal learning schedules cannot easily be determined without software that pools large amounts of data. (See Mozer et al, 2009, for a model). ³⁵Bjork et al. (1968) cited in Roediger & Karpicke (2010). ³⁶Compare the discussion of reflective practice (Schön, 1982) and deliberate performance (Fadde & Klein, 2010) in chapter 7.
Productive practice: A master maker
293
13.6.3 Be efficient and effective Odds are, you don’t have time for an additional regular activity. Productive practice is only feasible if it can deliver cognitive benefits with a “negative time footprint”, meaning that it saves one time overall. This section provides you with tips for practicing in the most productive manner, one which efficiently generates and sustains the mindware you wish to develop. Track your time with software. Periodically record how much time you spend processing material. (This can be done with the help of a spreadsheet and software that tracks how you spend your time on a computer, such as Timing.app³⁷.) Attend to the helpfulness (caliber, utility and potency) of the information you process and the depth of your processing. Simply measuring performance is often enough to improve it. You may find yourself reading less news for instance. You might also detect what distracts you and learn to stay away from it. Let productive practice replace some of your information processing. Productive practice can actually save you time if it prompts you to cut down on processing suboptimal information. Consider productive practice as an alternative to re-reading. Set your practice goals, plan and mode. Before practicing, decide what you want to achieve, how you will achieve it and what mode you will practice in. In execution mode, you aim to get through challenges quickly, marking challenges that you’d like to attend to later (e.g., because they are difficult or they have triggered some thoughts).³⁸ In elaboration mode, you allow one or more challenges to trigger trains of thought to solve problems, build new knowledge and extend understanding. Either way, you want to benefit from the surprising creativity boosts that practice provides— because it exposes you to a variety of meaningful, familiar and mind-expanding content. If you don’t indulge in creative thought on the spot, you can quickly record an action item about the challenge that triggered the thought. Sometimes you will miss practicing sessions. As a result, a large number of challenges will be due. You can the engage in a catch-up session or postpone specific challenges. Focus. Be as focused as possible while practicing. Put distractions outside of your mind. Thomas Sterner’s short book, The Practicing Mind: Bringing Discipline and Focus Into Your Life, which I mentioned above makes a good case for this. Consider how focused a concert pianist is while deliberately practicing a piece of music for an upcoming performance. Execution mode is particularly focused. But even while reflecting and elaborating, you need to stay on track, otherwise you will fall behind on your practicing. Consider limiting the number of challenges you allow to trigger reflection. Mark others for future consideration (e.g., tag them with “!edit”.) Assess instillers and suspend or reschedule them. You will likely create more instillers than you can master. As your priorities change, you will need to drop those that are no longer sufficiently pertinent to practice. In Anki, you can suspend challenges instead of deleting them. You can revisit postponed challenges periodically. Conversely, if your productivity app supports it, you may add expiry dates to contingent instillers. Practice with zest. Many of the characteristics of expert delving, listed in chapter 12, apply to expert productive practice, including cognitive zest: enthusiasm and perceived self-efficacy. We have every reason to believe that productive practice will contribute to your perceived self-efficacy ³⁷http://timingapp.com ³⁸This means that your productive practice software should enable you to mark challenges. Anki has this feature.
Productive practice: A master maker
as you gain mastery—let it.
294
14. Practice productively We all share the same origin, our mothers; all of us come in at the same door. But each of us—experiments of the depths—strives toward his own destiny. Hermann Hesse Whereas the previous chapter introduced productive practice and provided general considerations for designing instillers and practicing with them, this chapter explains how to master particular types of material and achieve specific types of mastery with productive practice. It provides suggestions to • address the problem of behaving rationally, in accordance with what you have learned (i.e., to apply knowledge and transfer skills), • develop monitors, including motive generators, • master concepts and vocabulary, • learn collections, • develop propensities to apply rules, and • develop attitudes.
14.1 Aim for effectiveness with knowledge: Rationality and transfer In chapter 3, I reframed the problem of transfer¹ as a problem of rationality. In the final analysis, to be rational requires that we act in accordance with our beliefs, knowledge, principles, and goals. But these are objective (“World 3”) and sometimes conflicting abstracta. Formal knowledge is not strictly speaking “in” our minds, it is stuff that our mind thinks about and with. It is one thing to subscribe to a principle, for example, but it is another thing to detect that the principle applies in the current situation and to be inclined to follow it. Productive practice is meant to help us to apply, or at least consider applying pertinent knowledge. One of the most important objectives of productive practice is to develop mindware that enables and predisposes one to utilize practiced knowledge. Whether the knowledge is factual, practical or normative, in order for it to be useful in practice, one needs to produce the right mindware for it. This mindware ideally should provide a propensity to use the information. For didactic purposes, I have dedicated a section below specifically to the problem of developing propensities. However, keep in mind that developing propensities is part and parcel of mastering all kinds of knowledge and developing all kinds of mindware. The concept of propensity is most evidently relevant in relation to rules—adopted rules are rules we want to have a propensity to ¹I do not mean to imply that there is just one problem of transfer. There are several. Moreover, the term “transfer” has several different meanings.
Practice productively
296
follow. However, the concept of propensity is relevant to other types of knowledge, even learning vocabulary and concepts—one wants to tend to think in terms of the concepts one has adopted. One also wants to be able to detect that multiple concepts, theories, rules, and so on are appropriate to a given situation such that, if time permits, one can deliberate before selecting the most applicable one. As Bereiter (1995) pointed out, the tendency to transfer is ultimately a matter of character, a virtue. Like most other virtues to develop this virtue requires that we deliberately try to apply it, until it becomes second nature. To this end, when delving and practicing, it helps to keep in mind the ultimate goal to be rational, and to realize that this requires applying what we know.
14.2 Grow monitors through review and reflection Behaving in accordance with knowledge starts with perception. Whether tacitly, consciously or somewhere in between, something about the situation must prompt us to detect that our knowledge is applicable. We say that mindware is inert if it remains dormant while events that are pertinent to it unfold. Here are some examples. • A developer reads Stanovich’s, What Intelligence Tests Miss. Several months later, she concludes that her boss has a low IQ, because he tends to make bad decisions when faced with complex problems. However, based on her prior experience with him, it is clear that he is smart in the IQ sense. The developer failed to detect that Stanovich’s theory applied to her boss, who would be better described as a “cognitive miser.” • Ralph’s wife was upset once because he messed up the kitchen just before her friends arrived. She told him “You really don’t care what my friends think of us do you?” In fact, his wife had read Gottman’s book and in principle knew that this kind of response is counterproductive. It may trigger a harsh-startup and poison their relationship. She failed to detect the incipient conditions of a harsh-startup. So she did not generate an intense motive to avoid the harshstartup. • One Saturday morning, Ralph’s wife asked him to join her in folding the laundry so they could chat. He told her that he first needed to read his email, but didn’t return on time. Ralph’s response was problematic because he habitually failed to recognize his wife’s bids for connection. Yet, he too had read Gottman’s books. In each of these cases, someone failed to apply pertinent knowledge that they once comprehended. The agent might even be able to remember and apply the pertinent knowledge if cued appropriately; but he or she failed to recognize the relevance of the knowledge gem. We all on occasion fail to apply what we, in some sense, know. How then can we develop the necessary perceptual foundation for rationality conceived as tending to apply relevant knowledge? To answer this question, let’s roughly characterize what happens when we do master knowledge to the point of being disposed to apply it when it is pertinent. I’ve frequently mentioned above that this involves establishing asynchronous monitors— motive generators and cognitive reflexes. Normally, when a healthy mind detects the relevance of
Practice productively
297
knowledge to a prior situation, it generates a signal to develop a monitor to detect that situation— that monitor may already exist or it may need to be created. We usually need to receive feedback (from others, the environment, software or ourselves) to discover that some knowledge applied to a more or less recent event. If we process this feedback appropriately, to the extent of understanding it, and are also motivated to improve, odds are that the monitor will develop—at least a bit more. Monitors do not normally develop to perfection in one-trial. At first, they are slow to respond (if they respond at all). However, if feedback is provided, the monitor will become increasingly adept at detecting conditions under which it should “fire”. The time it takes to detect the relevance of knowledge to a situation—the relevance detection latency—will decrease. It is not just one component of the mind (the monitor) that learns. The rest of the mind must adapt to it. To illustrate this, consider some recent, relevant experience of mine. Having used several earlier models of the iPhone, I purchased an iPhone® 5 soon after it was released. It boasts a USB and power connector dubbed Lightning®² which is only distinguishable (to me) by virtue of its smaller dock end. Because I only have one Lightning connector, I need to remember to remove it from my computer (at which I usually work late into the evening) and bring it to my bedside at night. I didn’t use to need to move my iPhone’s USB cable, as I had several around my home—they are incompatible, however, with the iPhone 5. As one would predict, it has taken me several evenings to remember to bring the cable before heading to bed. The time between me leaving my computer and remembering to go back for the connector decreased as one would expect, i.e., following an exponential learning curve (Heathcote, Brown, & Mewhort, 2000). One could characterize this prosaically from a purely behavioral (black-box) or (traditional) cognitive perspective. But that would miss the establishment of motive generators. Every time I forget that cable, I feel frustrated and annoyed at myself. (I concurrently find the fact amusing and fascinating. Hence conflicting emotions can coexist.) I realize that I should have brought it with me before heading downstairs. I imagine the cable where it is, and I wish that I had brought it with me. Imagining, reviewing, feeling frustrated and wishing, in this case, have a point: they cause me to develop a monitor to detect future cases in which it is time (or the situation) to bring the Lightning connector down with me (i.e., to generate a motivator to bring the cable down). Every time I realize that I have forgotten the cable, my mind is giving itself feedback. The motivational state generated by this situation (to get or bring the cable), like other motives, is dispositional. It does not directly cause me to bring the cable down; i.e., it does not trigger a behavioral reflex. I could override the motive. The motivator does, however, trigger internal reflexes. This feedback (somehow) increases the priority of developing this motive generator. The motive generator develops unconsciously during sleeping and waking hours. It becomes increasingly apt at generating an insistent motive just at the right time. To explain how motive generators develop in detail is an important task for broad cognitive science. In general one can trigger the development of motive generators by reviewing past experience, or by imagining possible situations in which one ought to respond with a particular motive. Thus recall, imagination and review not only potentiate declarative memory. But more generally they can lead to the establishment and development of motive generators. We will leverage this in productive practice. This, then, is yet another reason to distinguish productive practice from distributed recall ²http://en.wikipedia.org/wiki/Lightning_(connector)
Practice productively
298
practice. The activities involved in productive practice are not just about recalling information. And they don’t merely potentiate memory. They can develop manifold mindware. This proposal can be used to answer the question: How can one learn to apply knowledge one has delved? One needs to train oneself to be on the lookout for conditions under which the knowledge might apply. One also needs to receive feedback—from another, software, or oneself—that the knowledge was applicable so that one’s mind can develop monitors to detect those conditions as they arise. One approach is to coach oneself through productive practice. As you delve new knowledge that you want not only to understand but reliably apply, you create instillers with challenges to review your recent past with respect to the knowledge. You would ask yourself questions like: • When was this knowledge recently applicable? • Could any problem or situation I experienced today or earlier this week have been better addressed had I used this knowledge? Or, more generally, • How could I have used such knowledge, and to what effect? You would frame these questions with respect to specific knowledge gems rather than in purely abstract terms. Repeatedly and elaborately reflecting upon one’s diverse experience in this manner will lead to the unconscious development of monitors to detect these situations. Such reflection is more effective than re-reading the original text or using reminders. It is one of the keys to resolving the so-called problem of “transfer”. As mentioned above, it takes time. One cannot simply command one’s mind to grow motive generators any more than one can effectively command oneself, “Be able recall the answer to a question for as long as I live!” In time, the motive generators may become active mindware that look out for situations to which they apply. They may eventually become akin to the bits of mental machinery, “demons” in AI terms, that listen asynchronously and effortlessly for your name when it is spoken or for your baby’s cry at night while you are sleeping. In the sections below, I will describe ways to develop monitors for bids, harsh startups, and other conditions, through productive practice.
14.3 Master concepts and vocabulary If we want to change our lives and shape our destiny, we need to consciously select the words we’re going to use, and we need to constantly strive to expand our level of choice. Anthony Robbins So concepts and symbols are tools for generating possibilities or questions for investigation. They have greater generative power than theories. The scientist who usefully extends the language of science, unlike one who simply proposes a new theory using existing concepts and symbols, extends the hypothesis-forming powers of the scientists who understand him. In this sense conceptual advances are more profound. So the important differences between modern scientists and those of the distant past include not merely the statements and theories thought to be true or false, but also which statements and theories could be thought of at all. Not only are more answers known now, but more questions are intelligible. The same applies to development of an
Practice productively
299
individual. Aaron Sloman The most fundamental type of mental development involves mastering new concepts, symbols, forms of representation, notations and languages. Concepts and conceptual machinery are the basis for perceiving, thinking, and communicating. Conceptual knowledge trumps empirical knowledge. Empirical statements themselves are expressed with concepts. In “Gauge its potency”, above, I discussed the importance of acquiring potent new concepts. I described potent concepts as concepts that have the potential to significantly change one’s understanding and perception. Researchers need to learn large numbers of vocabulary terms and concepts. Many people frequently take on new jobs, roles or projects that require that they rapidly master a large corpus of vocabulary and concepts. Chung & Nation (2003) found that technical words accounted for 38% of the vocabulary in an anatomy textbook and 16% in an applied linguistic textbook. Such texts are incomprehensible without knowing the vocabulary. The standard recommendations given to normal adults for learning new vocabulary are to read voraciously, to look up new words in a dictionary or glossary, to record them, and to use them repeatedly in conversation or writing.³ When we read a sentence that uses new words whose concepts have recently been explained to us, we may have an illusion of competence, i.e., a belief that we fully understand the concepts and that we can henceforth use them. However, in order to master potent concepts, it is normally necessary to use them, to think with them, and to ask questions with and about them, over a period of time. Students of first or second language courses are sometimes advised to use flashcards. There is, in fact, considerable evidence that deliberate practice is very efficient for such learning (e.g., Nation, 2011). Elgort (2011), for example, describes deliberate practice of vocabulary as “high-return-on-investment vocabulary learning method.” Yet, graduates seem to throw systematic, deliberate practice of vocabulary and concepts out the window. When you encounter a term or concept you’d like to appropriate, you can add it to your productive practice system. For example, suppose that you bump into the concept of “andon cord” for the first time while reading The Lean Startup⁴ by Eric Ries. It’s a very easy concept to grasp. But there’s a difference between grasping a concept and becoming disposed to apply it when it matters. You might create an instiller with a number of challenges. In your regular productive practice sessions, these challenges would be presented to you, spaced out over a long period of time. Spacing has multiple benefits, including to help ensure that you install the mindware on a relatively permanent basis. 1. Question: What is the meaning of the term andon cord? 2. Question: How should you respond to a problem in continuous deployment? 3. Question: In the last week, have you (or your business) encountered a process problem? Did you sweep it under the rug or did you pull the andon cord? 4. Question: Think back to the latest time you should have pulled an andon cord. What would have been an agile response? ³See for example Young & Gibson (1976, pp. 139-140). ⁴http://theleanstartup.com/book
Practice productively
300
These challenges are designed for manifold mental development. Mastering the first challenge will help you to recognize the term when you encounter it. That’s not of much use initially, because you won’t likely encounter the word very often until you enter or create a culture in which the concept is prevalent. The second question will help you think in terms of continuous deployment and to relate this concept to andon cord. An andon cord is what, according to Toyota and The Lean Startup⁵, we are supposed to pull when we encounter such a problem. The third challenge is where it starts to get interesting. To answer this question, you need to engage in some mental review. It may be unpleasant at first, because you are probably not used to examining your past in search for instances of (a) continuous deployment; (b) continuous deployment problems; (c) pulling an andon cord. If you draw a blank, i.e., cannot answer the question the first time, that’s fine. In fact, it might be even better than if you happen to have encountered this issue 5 minutes ago. This challenge acts as an implicit signal to your mind: “Start looking out for each of those items, because you may be called upon in the future to answer such a question again.” This in turn will cause your mind to establish monitors specifically for those items. And that’s what you want: to start seeing the world in terms of situations that may call for an andon cord to be pulled. Concepts are not very useful unless you develop a propensity to see the world with respect to them—when they apply. Fine tuning for false positives will take care of itself. First you have to deal with the false negative problem. The major problem we have when we think we have acquired a concept is that in fact, we’re blind to instances or to valid applications of the concept. The fourth challenge in addition helps you to build knowledge about how to respond to these problems. It will help you develop management procedures for dealing with such situations. It will also help you build motive generators such that when you detect situations that match the andon cord criteria, you will tend to want to pull an andon cord. Ultimately, practicing this deliberately, and in action, may help you develop cognitive reflexes, such that when you detect these situations, you will have a good idea of what strategy to use. These results don’t all stem directly from practice (i.e., running mental simulations offline). Instead, practicing with challenges helps dispose you to reflect in action. So, some of your practice will be in the real world. (Compare the concept of deliberate performance discussed in chapter 7.)
14.3.1 Some basic distinctions When you set out to learn a concept, you’ll want to pigeon-hole it in your mind, if not electronically. Here are some basic distinctions: • Is it technical or colloquial? Both are important. The English language has not only the largest technical lexicon in the world, it also has the largest colloquial lexicon. Natural language has evolved to make myriad important subtle distinctions. Many of us have so much technical reading to do that we have very little time for humanities. Yet delving good fiction and history can help improve our understanding of human nature. It’s an implicit source of tacit knowledge.⁶ ⁵http://theleanstartup.com/book ⁶Oatley (1992) argued that literature is a major potential source of ideas about emotion for broad cognitive science (in particular pp. 123-129).
Cognitive science can also inform literature. The H-CogAff theory of mind inspired and permeates David Lodge’s novel, Thinks….
Practice productively
301
• Regarding colloquial concepts, keep in mind that there are relatively few real synonyms. Most new words pick up, and help one to specify, one or more different conceptual nuances. Consider the following list: bellicose, bumptious, cantankerous, churlish, claptrap , dilatory, emollient, expatiate, flaccid, hubris, imperious, impudent, inveigh, irascible, languid, obsequious, ponderous, portentous, profligate, pugnacious, refulgence, rodomontade, rumbustious, sullen, sunder, surly, truculent, waggish, churlish, cloying, contra mundum , desultory, elegiac, fulsome, indolent, inspiriting, irascible, laconic, languid, lugubrious, pokey, pugnacious, rapier (wit), sullen, surly, sycophant, vitiate. To learn such words it helps to classify them as to whether they are of a person’s character, state or actions, and to distinguish similar words. It also helps to compare them to each other. (Sloman, 1978, provides a collection of other techniques.) You need to encounter a word a large number of times before mastering it. To optimally master a term and its concepts, create an instiller for it. Include the sentence and reference in which you encountered the term. • Sometimes you need to learn a new term for a concept you already have (e.g., from a second language or the knowledge you have built yourself). Other times, you need to learn a new concept. • Some concepts are anonymous. To master new concepts it is sometimes necessary to coin new words, even if only for your own thinking, or to provide new definitions of existing terms. • Many higher order concepts, particularly problem-based ones, do not have referents (Bereiter, 2002). For example, the concept of force, mass and energy do not have referents, yet one ought to detect situations to which the concepts critically apply and to think in Newtonian terms at that point. To understand problem-based concepts, you need to understand their purpose. These distinctions may be useful in creating, editing and practicing with concept instillers.
14.3.2 Structure concept instillers This section describes an elaborate structure for concept instillers. • The term. For example, “Mindware”. • Definitions/Descriptions. This is a list of short descriptions or definitions of the term, or a specification of the concept. Normally, you’ll only have one item in this list. However, it can be useful to have multiple definitions or characterization even of the same meaning. This might promote transfer or generalization (particularly given challenges that present you with a definition). • RD cue. This is a reconstructible mnemonic cue for the term. For example, if you need to learn the term “surly” and you know an imperious, bad-tempered woman named Shirley, then Shirley could become a RD cue for surly. This cue has several good things going for it. Surly and Shirley are synophones (i.e., they sound similar)⁷ and synographs (have similar spelling). Moreover, the concept surly has a meaning associated with your view of the person. Shirley ⁷I include literary consonance, assonance, alliteration and rhyme as types of synophony (Laufer, 1981). Matching starting sounds are more effective cue than matching later sounds. If you can’t remember a name or word, you can try cycling through the phonetic alphabet. In that respect, “Shirley” is not as good a cue as “Sir Lee”.
Practice productively
• •
•
•
302
is a concrete example (rather than an abstraction). Finally, you can also use “surly” as a RD cue for Shirley herself, which might help you remember her name. I call synographs that are synophones and have similar meaning “synophonymns”. Synophonymns make for excellent RD cues. Practice can make a cue discriminative. Near synonyms. These are words you would find in a thesaurus. Example usages. This is a collection of example uses of the term. By using a quote from the resource in which you encountered the term, the example will make more sense to you when you listen to it during audio-practice. It also might help you master the information in which it figures in the original source, by keeping it active. For example, for the term “fissiparous”, this could be “the novel and fissiparous nature of the opposition was even more indicative of how the old pattern had been shattered.”⁸ A good dictionary application will provide examples that you can easily include here. Clozes. This is a list of “fill in the blank” type challenges. Most useful for general as opposed to technical vocabulary. A sentence that serves as a good example typically also can be a useful cloze. In place of the term (e.g., fissiparous) you might write its meaning. Example applications. For problem-based concepts, you could include some example applications. For example, for the concept of mindware, there could be an example of how the concept can be used to explain superior or inferior performance on a particular task.
Of course, a concept instiller should inherit generic fields that apply to most instillers, such as lists of questions and answer pairs (see “Instiller types and challenge templates”.) For many concepts, it is sufficient to fill out two or three fields. While instillers can be used to store and look up information, their main purpose is to help you master knowledge gems through productive practice. You can design challenge templates that are specific to concepts. You would then generate specific challenges from instillers based on these templates.⁹ Here are some example specifications of challenge templates. Keep in mind that you wouldn’t use every template for every instiller. After editing a particular instiller, you generate challenges based on the templates that are most applicable to the instiller. That way, you don’t need to create each challenge manually. As with other types of instillers, you would have templates for each of the pairs of question and answer fields. Those are very simple. The question and answer are taken directly from the respective question and answer fields. In addition, the following challenge templates are useful. • Definition to term. This asks you to provide a definition for the value of the term field. • RD-cue to term. This asks you to provide the term, given the RD cue. This helps you to learn the association between the RD cue and the term. This is important because even the most brilliant mnemonic can easily be forgotten if it is not practiced until self-sustaining mindware for it is developed. ⁸Jenkins (2002, p. 338). ⁹This can be effected with a simple command in Anki 1.2 (“Generate cards”). The second version of Anki automatically generates challenges,
based on your templates, when you save an instiller (“note”).
Practice productively
303
• Definition to RD cue. This helps you learn to think of the RD cue, given a definition or description. (A mnemonic is not helpful if you can’t recall it.) • Give an example. This asks you for an example of the instiller’s term. For example, you might be asked to provide an example of surliness. As mentioned in chapter 7, studying examples is a potent way to build mindware and knowledge. Conjuring up examples of concepts is critical in conceptual analysis and instruction; but many find it challenging. Fortunately, this skill can be developed with practice—a great side effect of productive practice. Applying this skill during practice can also help you become better at writing instillers (and hence programming your own mind).¹⁰. • What is this an example of? This presents an example of a concept and asks one to classify it. Ultimately, one needs to tend to recognize instances of concepts, unprompted. Reviewing experience is another way to learn this, one that may be more conducive to generalization. • Provide a synonym. This would ask you for words related to the key term. The answer might be contained in the “related terms” field. You need not have a template for each common type of question. Another alternative is to use a text expansion utility (such as TextExpander). You could create an entry containing a master list of questions. Then, while designing the instiller, you would expand the list in a question field and delete all but the most pertinent questions. You would generate a challenge based on that question field. When constructing instiller challenges, you don’t necessarily have to include the answer. Practice is an opportunity for deep thinking. If while practicing, an answer doesn’t come to you, the instiller itself can serve as a resource for constructing the answer. You can also mark the challenge for review.
14.3.3 Instill mindware about mindware, for example In the section above, “A template for conceptual understanding”, I illustrated how one might analyze the concept of mindware while delving Keith Stanovich’s book, What Intelligence Tests Miss¹¹. Conceptual analysis can help one comprehend a concept, but it does not necessarily yield lasting understanding,¹² particularly of subtle concepts nested in rich theories. As I’ve often said here, practice helps one become more effective. If one wanted to master the knowledge gem, mindware, one might create instillers with challenges like the following. • What role does the concept of mindware play in Keith Stanovich’s explanation of why smart people sometimes do stupid things? • What are some properties, dimensions of variation and types of mindware? • Give an example of the evaluation-disabling property of some mindware. ¹⁰What is the covert mechanism underlying the development of example-generation skills? Perhaps it lies in one becoming more attuned to instances of concepts that interest one and one’s mind updating some of its indexes soon after encountering said instances. Compare Sloman (1978, chapter 6) on the importance of indexes in the design of intelligent mechanisms. This is a topic worthy of more designer-based cognitive science. ¹¹http://www.keithstanovich.com/Site/Books.html ¹²Karpicke & Blunt (2011b) argued that distributed recall practice is more effective than concept mapping. Moon, Hoffman, Eskridge, & Coffey (2011) responded critically. It’s a moot point, partly because concept mapping can be used as a productive form of practice.
Practice productively
304
• Looking back on the last few days, try to recall an example where an (IQ) intelligent person did something dumb. Explain their behavior in terms of the mindware of rational thought and the tripartite architecture of the mind. • How does the mindware of rational thought differ from other mindware? • How does one normally acquire mindware? • Think of someone other than people mentioned in Stanovich’s book¹³ that seems to lack the thinking dispositions of rational thought. Let that person be your exemplar of dysrationalia. In other words, while learning and using the concept of mindware, use that person as an example. • Describe the concept of mindware to someone by applying it to an example, and then make a note of the problems this explanation has raised for the concept or your understanding of the concept. • What is your main criticism of Stanovich’s concept of mindware? Is the concept salvageable? How? • How does the concept of mindware of rational thought relate to fluid intelligence and crystallized intelligence? Stub answers to these questions are in the section “A template for conceptual understanding” above. I recommend reading Stanovich’s book. Of course, you can master these concepts in other ways. However, if your job or other constraints do not provide you with sufficient opportunities to apply what you delve, productive practice can help you systematically develop the mindware you seek. This section illustrated productive practice of a theoretical concept, one that is used mainly for understanding and explaining cognition and behavior. Let’s turn our attention to the acquisition of a more practical concept.
14.3.4 Develop effective (affective) bid monitors He who knows nothing, loves nothing. He who can do nothing understands nothing. He who understands nothing is worthless. But he who understands also loves, notices, sees … Paracelsus The current section gives another example of deeply mastering a concept with productive practice, i.e., the concept of bids. While the concept of bid also can be used for explanatory purposes, to most readers of Gottman’s popular books the concept is mainly useful for practical purposes. To them, mastering this knowledge gem means being able to use it automatically and effectively in everyday circumstances to which it applies. The concept is useful for them to the extent that it will help them achieve its advertised benefits. For Gottman & DeClaire claim that people who fail in bid-response interactions are susceptible to having more arguments and conflicts, which lead ¹³http://www.keithstanovich.com/Site/Books.html
Practice productively
305
to marital strife, divorce and the failure of all kinds of family and social relationships (2001, p. 6). It also leads these people to be lonelier, dissatisfied and to have unstable careers. In contrast, productively responding to bids can sustain relationships, nourish romance, and help each member manage external stressors. Clearly, applying one’s meta-effectiveness to this knowledge gem can be worthwhile! To master bids requires that one • • • • •
understand the importance of bid-response patterns, recognize and classify bids and bid-responses, perceive bids with the right affect, i.e., affectionately,¹⁴ read the underlying significance of bids, tacitly understand that how one responds to bids will determine the quality and course of the relationship, • react to bids by generating a motivational state of turning towards rather than one of turning away or turning against.¹⁵ Now let’s have a look at how to design instillers to develop bid-monitoring and -processing mindware. Ironically, what makes bids difficult to perceive is also what makes the concept of bid seem so easy to understand: we’ve seen and issued thousands upon thousands of them. Any little request we make of another, from extending a hand to be shaken to asking for a hand in marriage is a bid. Our minds are intricately programmed to respond automatically to thousands of bid types. But, we don’t stop to classify every little request as a bid: to respond, we use reactive mindware (in what Keith Stanovich calls the “autonomous mind”) that is fast and normally opaque to the rest of the mind. Many of us are blind to bids as Gottman wants us to see them, because we haven’t learned to classify behavior at this level. To learn to see bids as bids requires that we develop new bid detectors that will access information that was previously opaque.¹⁶ To learn to recall instances of bids will therefore itself be difficult at first. This is due, in addition, to a major fan effect (cue overload). For the concept of bid applies to so many types of behavior. Fortunately, Gottman provides several techniques. One could convert many of his exercises into instillers; or one could create new ones. The challenges I propose are of the latter variety. They focus on building bid detectors. • Give four examples of bids for connection (try to vary them between sessions). Answer: For example, request to listen, go out together, perform a task, answer a question. • What small bids for connection did you receive from your partner today? (Small bids can’t easily be remembered, so the “today” qualification is important. Practicing to remember small bids is important.) ¹⁴Effective bid monitors are affective bid monitors. They perceive with affect. ¹⁵I could have written more compactly “react to bids by turning towards rather than turning away or against”. However, it is important here to
outgrow behavioral talk and emphasize the internal motivational state one must generate. It is ultimately the motivator (the internal motivational state) that determines how one will respond. It affords many different ways of responding. See also Boden (1972). ¹⁶Compare the discussion of re-representation in Part 2. Young people learn to re-represent previously opaque information in their own minds (Karmiloff-Smith, 1995). Learning to view behavior as bids involves re-representation and is a form of adult mental development.
Practice productively
306
• Try to remember a recent example when your spouse issued one of the following types of bids¹⁷: request to listen, go out together, perform a task or answer a question. Did you turn towards, turn away or turn against your spouse? • What big bids has your partner issued to you recently? • Recall the last time, or a significant time, that you turned away from a bid for connection. (This may help you learn to detect your responses to bids. It may also help you eventually detect the motivators that you generate before you act upon them.) • Recall the last time, or a significant time, that you turned against a bid for connection. (Once you identify an example, it might become an effective tool for thinking about bids.) • How did you respond to bids for connection from your partner or someone else today, in terms of Gottman’s taxonomy? (This too may help you detect your motivators and incipient behaviors.) • What bids for connection did you issue to your partner or others today. And how did your partner respond? (This question is meant to focus your attention on your own bidding and its ramifications.) If you are not in an intimate relationship, your partner is not available, or you simply want to rely more on imagining than remembering, you can rephrase many of these questions. For example: Think of a hypothetical example in which you turned against your partner (or someone else) when she issued a bid for connection. How would your partner feel? Professional athletes often use mental simulation (imagination) in preparation for real competition. Imagined experience can instill mindware. The previous challenges are designed to help you develop bid monitors. By challenging yourself to recall and imagine bids over a period of weeks, you will become increasingly aware of bids, i.e., you will learn to classify behavior as bids. I wouldn’t recommend that you add all these challenges into your bid instillers all at once. Also you (and your software) would need to space them out appropriately. These challenges, however, won’t necessarily help you classify bids with the affect that they require. Yet to be effective, bid monitors must not merely elicit a dry cognitive response, they must be affective. Your perception must itself be a motivational state, i.e., activate or generate sufficiently insistent motives.¹⁸ The following challenges are meant to imbue your perception of bids with the right affect, so that you may be disposed to turn towards your partner instead of away or against. • Think of a recent example in which you turned away from your partner when your partner issued a bid for connection. How did your partner feel? ¹⁹ • Think of a recent example in which you turned towards your partner when your partner issued a bid for connection. How did your partner feel? ¹⁷Ideally, the author would provide a kit that contained dozens of stock examples of bids, and the software would randomly select questions, so that you could build up monitors for each one. ¹⁸The classical view of the mind, which is implicit in much of cognitive science, is that perception yields dry descriptors. Sloman (1989) argued that vision inherently involves affective assessment. Cf. Sartre’s treatise on emotion (Sartre, 1938), Gibson (1979)’s emphasis on the perception of affordances, and Beaudoin (1994) on “valenced perception”. Pessoa (2013, chapter 3) makes a strong case for this, which he calls affective perception. ¹⁹According to Gottman et al. (2002, Ch. 16), marital improvements are most likely to occur when interventions target affect during everyday interactions, conflict resolution and actual conflicts. The cognition and affect framework presented in this book helps to make sense of the intrapersonal aspects of such development.
Practice productively
307
• Think of a recent example in which you turned against your partner (or someone else) when she issued a bid for connection. How did your partner feel? Were you responding to the content of what your partner said, or to your partner’s feelings? What would the outcome have been had you responded differently? • What are the long-term consequences of habitually turning towards bids for connection, such as to […]? (You would fill in the blanks with types of bids that your partner issues that are problematic in your relationship. Gottman’s book provides help for discovering them.) • What are the long-term consequences of habitually turning away from bids for connection? • What are the long-term consequences of habitually turning against bids for connection? • How can you use the principle “It is foolish to sacrifice the long-term good for the short-term good” to help guide your response to a specific type of bid your partner has recently issued. • How will your partner feel if you turn away from bids for connection, such as to spend time together or do chores together, such as to […]? Such challenges, if practiced productively, would not merely help one develop motive generators and insistence assignment rules, they would also help one to influence management processing—e.g., (a) assessments of importance and urgency and (b) selection, scheduling, and planning decisions. See chapter 5. Monitoring bids is not enough to become an effective partner; of course, one also needs to respond to them. I don’t provide example challenges for responding to bids. “Develop propensities to apply rules”, below, illustrates responding to other contingencies below.
14.4 Master collections of information People find it particularly difficult to remember collections of information. In this section, I explain why collections are difficult to remember. I then propose ways of designing instillers and practicing that avoid these problems. This illustrates that meta-effectiveness requires using, rather than fighting, the mind’s information processing principles. Because people find learning lists so difficult, this section goes into quite a bit of detail. Given that people learn best from examples, this section provides a detailed worked example. When a conversation starts off on a very sour note, it tends to end poorly. Gottman & Silver (1999) provide the following important tips for avoiding what they call a “harsh startup”. Complain but don’t criticize or blame. Make statements that start with “I” instead of “You.” Describe what is happening, don’t evaluate or judge. Be clear. Be polite. Be appreciative. Don’t store things up. Memorizing this collection is not useful in itself—this is knowledge that you want to apply, not pass a written exam on. However, we can still use this as an example, as there are plenty of cases
308
Practice productively
where it is useful to remember collections. Moreover, I will argue below that the collection is actually worth committing to memory as a step towards mastering the principles. If you are interested in actually mastering this information, you should first read Gottman’s books. Try memorizing this collection and coming back to this section in a few days. Test yourself at this point.²⁰ Unless you go about this task using the right mnemonics, you are likely to find it surprisingly challenging. Here’s why. Adverse cue effects. As I mentioned previously, one of the most fundamental properties of memory is that it is associative and cue-driven. Mäantylä (1986) showed that subjects could easily learn 500 random words, provided that they construct an appropriate cue for each word. However, the effectiveness of a cue decreases rapidly as a function of the number of targets to which it is associated. This is known as the “fan effect”. When you learn a collection of items, you typically have a cue that is associated with multiple targets. The cue here might be the question, “What 5 steps comprise Gottman and Silver’s principle, ‘Solve your solvable problems’”. This cue is overloaded— it’s associated with 5 targets.
Figure 14.1 Cue Overload
To make matters worse, odds are that when you try to remember these 5 items, you don’t use exactly the same cue. You might think something like, “Gottman’s conflict-resolution principles”, “Gottman’s problem-solving principle”, or other variants. Further, you might focus on one aspect or another of the vague idea of solving one’s solvable problems. So the cue’s effectiveness is reduced even more. Even if you do tend to use the same cue, you might often think of that cue in the absence ²⁰In addition, I also recommend that you try memorizing several unrelated lists. Use the RD cue system in some cases and other mnemonics systems in others. By using Anki, you may get a sense of how long it takes to learn with vs. without the RD cue mnemonic.
Practice productively
309
of thinking about the targets. This decreases the discriminability of the cue even further. For a cue to be effective, it needs to be crisp, clear and highly discriminative (or predictive) of the target. You may have noticed that I’ve been speaking in terms of collections rather than lists. The reason is that whereas we often describe collections as lists, we don’t always treat them that way. When people try to remember a list, they often do not pay careful attention to the order of its elements. Semantically, this makes sense. There is often no inherent reason why one item should come before another in a collection. So, at recall, people try to remember them in any order. By doing this, they fail to leverage the cue-driven principle of memory. If you can find a way to make each item in the list cue the other, you can reduce the fan effect. I will return to the RD cue (reconstructible, discriminative cue expression mnemonic) system below to leverage this principle. Meaningful encoding and elaborate processing. Another reason that lists are difficult to remember is that what makes the elements part of the collection is often fairly abstract. If you can’t overlay meaning on the list such that you are reminded of the items that are part of it— but not of the items that are not part of it—your task is made fairly difficult. This partly explains why the components of “solve your solvable problems” are hard to remember. When you read the steps, they all make sense. But you can probably think of a number of other things you can do to solve problems that are not on the list. So, the list may seem rather arbitrary to you. It’s difficult to remember arbitrary information. Many mnemonics are designed to get you to impose meaning on arbitrary information. For example, you’ve probably heard that to remember someone’s name you should try relating a property of the name to the person. For example, if you meet someone called Smith, you might try picturing him as a blacksmith. This can work, but the mnemonic is still rather arbitrary. You’ll probably find it hard to remember “blacksmith” in relation to the person. If you can remember blacksmith, cuing may (or may not) get you back to Smith (you might erroneously call him Mr. Blacksmith.) This is one reason why these mnemonics are often ineffective. They are more effective if one can find, and practice with,²¹ a very meaningful match. Standard advice for learning lists is to use imagery and the method of loci²². When you learn a list with this technique, you utilize a familiar route within some structured building or other place. Within each place along the way, you imagine an item of the list. At recall you imagine each place along the route, sequentially. Hopefully, you’re reminded of the items on the list along the way. This does not work very well for abstract items such as the steps of solving your solvable problems, unless you can convert each element to a concrete noun. Imagery mnemonics tend to rely on fairly arbitrary information and require more effort to construct than most people can be bothered to apply. While some degree of arbitrariness is acceptable in mnemonics, and practice is almost always required, it’s preferable to construct mnemonics whose components have inherent meaning. Inadequate practice. In order to be able to remember lists, it’s usually necessary to practice recalling them. Most people don’t have a system for doing this. Productive practice can help here. This example is extended below to help you learn to develop propensities to apply complex rules. ²¹This qualification is important. Even highly meaningful mnemonics normally fail unless one practices them. ²²http://en.wikipedia.org/wiki/Method_of_loci
Practice productively
310
14.4.1 Apply the RD cue system My first suggestion for dealing with lists is not to bother unless the payoff exceeds the effort. This is true for mastering any knowledge. But collections are particularly costly to learn. This tip requires that you learn to think of knowledge acquisition as having both value and cost. The cost is your time and mental energy. It’s also the cost of lost opportunities—other gems you can master with less effort. Try converting the list to a collection of independent gems. If you do want to learn the collection, try to prune it first. Then, design an instiller for it that leverages, rather than conflicts with, how your mind generates mindware. The RD-cue list-learning technique, as follows, is cognitively friendly. • Convert the collection to a list. As I mentioned above, unordered collections involve cue overload. When you convert the list to a sequence, you need to order the items in a way that will make remembering the items in the list easy. (More on this below.) If the author’s order is important, you can still violate his order in memorization and reconstruct it at recall time. Just be sure to make the cue-driven principle work for you. You might have to play around with the list a few times as there are several constraints to consider. Try not to practice the list until you are happy with its internal order, however; otherwise you’ll have to fight proactive interference (where previous learning interferes with new learning). • Create a RD cue (a reconstructible phrase or sentence) for the list which you can interpret as relating to the target list. For example, for Gottman and Silver’s list of rules to avoid a harsh startup, you might use the following mnemonic: “I clearly, appreciatively & politely describe (but don’t judge) and complain (but don’t blame)”. This meaningfully summarizes the individual rules. Each significant word in the mnemonic maps to a principle in the list and can function as a cue. This assumes, of course, that you will be able to expand each element in the sentence. Often, early comprehension is enough; if it’s not, create additional challenges within the instiller. Note that this mnemonic re-orders the list elements. This meets the criteria for being a RD cue. (It’s reconstructible and a discriminative cue.) However, it’s not perfectly reconstructible. You might have difficulty remembering the sentence. Hence the following tip. • To ensure that you can remember the RD cue, create a RD cue for it. It too must be a reconstructible cue. For a French Canadian, an example of this is “I cappot, Crust!” “Cappoter” is a colloquial French Canadian verb that means to blow one’s lid. Crust is a memorably shocking near expletive. Given that harsh-startups are about losing one’s lid, this little mnemonic is not arbitrary: it carries some of the meaning that it is meant to bring to mind. It’s also a bit vulgar—a useful mnemonic feature. It derives its cue value from being an acronym of sorts for “I clearly, appreciatively & politely describe (but don’t judge) and complain without storing things up” The letters “d” and “t” may be different, but phonetically they differ only in one facet. If the emphasis on the “st” of “Crust” is insufficient to remind you of “store” then just add “without storing things up” directly to “I cappot”. A mnemonic need only be useful to its user. This one might not work for you. • Devising acrostics, like other cognitive skills, takes a bit of practice. It’s something you do at instiller-design time, not while you’re delving, practicing or applying knowledge. The effort applied at design time offsets the effort required to use it in practice. Because they
311
Practice productively
are reconstructible, once one has practiced them, one just has to read them out. Then one can rely on cuing. In contrast, mnemonics that use visualization are difficult to decode, do not leverage cuing, and are subject to cue-overload. The following diagram puts it all together:
Figure 14.2 RD Cue for Avoiding Harsh-Startups
I call this process cue chaining. It reduces the process of learning a list to learning small, cue expressions. Each expression can be part of the same instiller. So you could create an instiller with the following challenges: 1. What is the RD cue for the components of a harsh “harsh-startup”? Answer: “I cappot, Crust.” Essentially, here, “harsh-startup” is a cue for the target “I cappot, Crust”. 2. What is the target of “I cappot, Crust”? Answer: “I clearly, appreciatively & politely describe (but don’t judge) and complain without storing things up”. 3. What is the target of “I clearly, appreciatively & politely describe (but don’t judge) and complain without storing things up”? The answer is summarized in the previous figure. A principle that is less applicable to this example than it is to many others is to group semantically related items together. In other words, order and group information in a way that is meaningful to you and memorable. Notice that the RD cue mnemonic system can be used not merely for flat lists, but for lists of lists as well. In other words, it can be used recursively. One could, for instance embed this mnemonic within a mnemonic for the broader principle of “Solve your solvable problems”, and the latter could be embedded within a mnemonic for the seven principles that are discussed by Gottman (not listed here). So, it is very general.
Practice productively
312
Of course, a list instiller is useless unless one has the knowledge to understand it (in this case, Gottman’s book) and unless one actually practices with it. When it comes time to practice a list, whether with a chain of RD cues or other techniques, try to obey the following principles. They are not laws, just principles—so use your judgment. • Pay particular attention to learning the first two words of the RD cue. They are the most critical ones. Fortunately, the primacy effect works in your favor here. This is the principle according to which the first and last elements of a sequence are easiest to remember. Keep in mind that your goal is to set up an association between the mental state in which you are searching for the RD cue and the first two words of the RD cue. That mental state is an internal cue.²³ • Fail early. This is an application of the principle of pulling an “Andon cord” discussed earlier. Rather than stumble along trying to remember blurry list items, skip the challenge and come back to it. This may prevent cues from becoming associated with the wrong information. It also makes for quicker practice, and separates instiller design from practice. • If you have not converted the collection to a list and you are relying on some other technique, make a note of the items that you had difficulty recalling. The next time you practice this challenge, try to start by recalling the items that you couldn’t remember. • Perceive and work with the information meaningfully. If you use a mnemonic, be sensitive to why the mnemonic makes sense in relation to the meaning of the target knowledge. If it doesn’t, consider changing the mnemonic to something imbued with meaning. • Apply the more general rules of practice discussed in the previous chapter. For example, make a note of flaws in your understanding and address them in due course. I chose a list of rules rather than other knowledge items in order to set the stage for learning to apply rules. However, I could have used other types of knowledge as examples. Learning lists is more time consuming than learning simpler material. However, leveraging the principles of memory used by the RD cue mnemonic system and practicing optimizes this process. If the software does its job right, and if you use the knowledge outside of practice (which is the whole point of practicing), you should be able to use the information for as long as you need it. Thus, after practicing, when you want to remember the rules for avoiding a harsh startup, having read their book], you just need to remember the first cue (“Harsh startup RD cue”, or whatever your entry point is), and you can practically read the entire list, step by step. Forgetting is natural. “Generalizing” is difficult: You may not be able to recall the information in all situations (e.g., under pressure). But then you probably don’t need to remember it under all circumstances either. If remembering in a particular situation is important to you, then be sure to practice in similar real or imagined situations. The RD cue is just one amongst many learning techniques. My main point is not that you should use this technique. It’s that you should have an effective, systematic way of dealing with lists that accounts for the psychological difficulties presented by list-learning. In the following section, I explain a way in which memorizing lists can be very useful for learning to follow new rules. Reviewing list information represented in long-term memory can enable you ²³This has not been empirically verified yet, but I conjecture that attempting to recreate this mental state consistently would improve learning.
Practice productively
313
to develop a propensity to respond to the situations that the rules are about. Once you develop this propensity, you will often even be able to dispense with your ability to recall the list items themselves. The list will have served an important transient function of “bootstrapping” your mental development.
14.5 Develop propensities to apply rules: Self-regulate with productive practice The most important thing is that you actually care, that you do something to the best of your ability. Sir Jony Ive²⁴ In order to benefit from practical knowledge one normally needs to be able to use it, and to consider using it, when it is applicable. In other words, one must develop a disposition to think, perceive, act and sometimes even feel with the knowledge. As mentioned in chapter 1, a propensity is more than an ability or even a weak disposition. It is an active, agentic disposition. Practical knowledge often boils down to rules or principles. Consider some of your most prized practical sources of knowledge, and you will find rules. They are things to do in certain conditions. In order to master rules, we need to be sensitive to the conditions under which they apply and tend to respond in a manner that is informed by them. This does not necessarily mean that we will apply the rules, for in any given circumstances, multiple, conflicting rules may apply. As with other forms of knowledge, reading does not instill all the required mindware for rules: monitors, retrieval structures, motive generators, cognitive reflexes, insistence assignment, management procedures, long-term working memory, etc. Instilling knowledge-based mindware normally requires practice. Accordingly, this section describes how productive practice can be used to develop propensities— and hence their underlying mindware. I have selected three examples to illustrate a variety of knowledge worth mastering. All three examples have been explained in prior literature. • Consider the opposite. A very general thinking disposition concerned with making judgments under uncertainty. (Keith Stanovich and other authors). • Andon cord: a simple rule for handling potential process problems. (Eric Ries and prior authors.) • Avoid harsh startups. A bundle of rules to help prevent flare-ups and solve problems. (John Gottman.) Instillers can be designed for each of these practical knowledge gems. ²⁴http://www.vanityfair.com/business/2013/11/jony-ive-marc-newson-design-auction
Practice productively
314
14.5.1 Consider the opposite Reflection involves considering at least one alternative and finding evidence that favors one’s own view more than it favors the alternatives or evidence that impugns the alternatives more than one’s view. Joseph Baron Merely describing a bias, at least in an involving domain, has no ameliorative effect. Charles Lord, Mark Lepper & Elisabeth Preston With Keith Stanovich’s theory, rationality, not intelligence, is the best predictor of the quality of judgments and decisions people make. Rationality is a broader concept than intelligence. It includes intelligence (the “algorithmic mind”), knowledge and dispositions to use knowledge (the “reflective mind”). That view accords with humanism. It’s uplifting given that the algorithmic mind’s abilities decrease as one ages. It underscores the importance of striving to be rational.²⁵ Who would not want to possess the beneficial thinking dispositions studied by cognitive psychologists? To be a clear, broad, deep, sound, curious, systematic, rigorous, aware and balanced thinker? To be someone who is open-minded, has a high need for cognition, considers various points of view, engages in counterfactual thinking at just the right time and anticipates potential consequences before making a decision? To tend to detect flaws in one’s knowledge or understanding and generate an appropriate motivator? To be able to make calibrated judgments and decisions within reasonable time constraints. We ought to all want these dispositions. For we would tend to make better decisions and have better outcomes with than without them. Beneficial thinking dispositions enable us to help our selves, families, employers and society. How can one bridge the gap, alluded to by Fischhoff in the opening quotation, between comprehension and active mindware? One can read about thinking dispositions until the cows come home and, like Chrissy, still hold on to a stock that now represents 99% of one’s wealth, thereby completely violating sound asset allocation principles (that call for diversification). Let’s look at how to master a particular strategy, “consider the opposite”, with a view to conveying principles for the acquisition of thinking dispositions in general. The “consider the opposite” strategy is a good example because it is general, it can be used to repair a large number of cognitive biases Lord et al., (1984) and it is easy to apply (once one is cued to apply it). This strategy applies in all kinds of situations where you are called upon to make or withhold a judgment in the face of information and uncertainty. It might be a matter of deciding whether a stock really is a good investment, whether a design or implementation meets its requirements, whether a particular hypothesis is true, a prediction is warranted, and so on. The consider the opposite²⁶ strategy involves explicitly considering (and potentially imagining) that an incompatible hypothesis might be true. For example, Lord et al., (1984) had subjects consider evidence in favor or against positions such as that capital punishment has a deterrent effect on crime or that a confederate is introverted or ²⁵This admittedly contentious claim cannot reasonably be assessed without first understanding the concept of rationality I have in mind. Compare Baron (1994, 2008). ²⁶Perhaps a better name for this strategy is “consider alternatives”. For there might not merely be a pair of binary opposites, but a collection of more or less compatible applicable alternatives. It is ironic that the expression meant to instill broad thinking implicitly reinforces binary thinking. Some might counter, however, that binary thinking has advantages (Egan, 1997).
Practice productively
315
extraverted. All subjects were told this was an important judgment. In this kind of situation, most people will interpret the evidence in accordance with their preconceptions. They will tend to ignore evidence that is incompatible with whatever judgment they heuristically and rapidly make. People who are opposed to capital punishment tend to interpret information (“evidence”) as supporting their view that capital punishment has no deterrent effect. Their views and attitudes will become more polarized in the face of evidence. Furthermore, on the basis of the same information (“evidence”), the same will happen to the views of people initially (and hence subsequently) in favor of capital punishment! Some of the subjects in each experiment, however, were explicitly told to consider the opposite. For example, some of them were told: Ask yourself at each step whether you would have made the same high or low evaluations had exactly the same study produced results on the other side of the issue. This simple instruction had a significant effect on the results. The views of these subjects were hardly affected by the evidence, which they presumably dismissed as being too weak. I relate this experiment mainly to illustrate the principle in action. For to suppose that this experiment demonstrates the general effectiveness of the consider the opposite strategy would be to fail to apply that very strategy. However, the strategy has received additional support and, more importantly, is itself valid on rational (a priori) grounds. The hypothesis that Lord et al. (1984) evoked to explain their data was that a consider-theopposite instruction activates the opposite anchor (I would say mindware). The opposite idea can then also be considered in assessing evidence. It is not reasonable to suppose that the effect on the subjects lasted well beyond the experiment. Lasting mental change does not tend to come that easily. In order for the change to last, one needs to grow monitors to detect situations in which one ought to consider alternatives. These monitors need to trigger cognitive reflexes and motive generators to consider alternatives. And one’s mind must become equipped to respond appropriately to them (with management processes). I believe the crux of the problem for most of us is with the former (detection and motive generation), rather than the latter. This is because we can expect normal learning to be triggered by motive generators: if you frequently generate a motive to think of the opposite, you will gradually find good ways to do so. This proposal is consistent with the consider the opposite strategy results. The subjects in these studies benefitted from very simple instructions. Moreover, normal learning, by definition and per empirical research, happens normally. A problem we face in training ourselves to master heuristics, such as to consider the opposite and to turn towards bids for connection, is that they apply in a very wide variety of cases. One can’t be considering the opposite for every decision. Yet somehow, one must learn to recognize situations that call for the rule. As I argued in the section “Grow monitors through review and reflection”, I believe that monitors, including motive generators, can be developed through regularly reviewing events to which concepts or principles may have been applicable. This can be achieved by designing and utilizing instillers with challenges such as the following: Question: In the decisions under uncertainty that I made today, did I consider the opposite?
Practice productively
316
Of course, the answer cannot be anticipated; so the answer field will be left blank. I propose that such questions can improve one’s: • • • •
ability to recall judgments made on any given day, awareness of judgments as one makes them, tendency to consider specific alternatives, judgments (their calibration) and hence decisions.
If one doesn’t see positive results with the proposed questions, one could change the questions to suit oneself. The effects of training tend to wear off. However, with a productive practice system one can keep instillers active for years, selecting a schedule to suite oneself. This ought to improve the likelihood of generalization and transfer by keeping the mindware active. Once one develops effective monitors for a rule, they become self-sustaining; productive practice is no longer required because one practices them naturally. Of course, this assumes that the conditions of applicability of the rule are sufficiently frequently met. The frequency with which the rule is likely to apply may justify the initial effort required to create and practice with its instiller.
14.5.2 Andon cord [Andon cord] is one of the most important discoveries of the lean manufacturing movement: you cannot trade quality for time. If you are causing (or missing) quality problems now, the resulting defects will slow you down later. Defects cause a lot of rework, low morale, and customer complaints, all of which slow progress and eat away at valuable resources. Eric Ries The many dispositions inherent in a lean startup mentality cannot be acquired simply by reading The Lean Startup. One way to learn is through participation in a culture whose members model the framework on a regular basis and who provide feedback to new employees. But what if your current employer is immutably steeped in old ways and you want to prepare yourself mentally for a lean opportunity? You could read more, participate in discussion groups online, and so forth. But ultimately, you need to practice applying the knowledge to your own life. Productive practice presents a systematic way of doing this. You could develop lean startup mindware by creating instillers for each of the most effective concepts and principles—the knowledge gems—of the book, and then training with them. This section focuses on one specific agile propensity: to apply the andon cord principle. The idea comes from Toyota’s manufacturing. When an employee finds a defect in the assembly line that could affect the quality of the product or safety, he is not merely allowed but supposed to pull an “andon cord”. This causes the leader of the line to investigate the problem. If the problem cannot be resolved, that line grinds to a halt and attention is directed at resolving the issue. They pay a price up front (in time, effort and delayed production), rather than allowing the problem to continue and accumulate greater costs. Edmondson et al. (2004) claim that the employees “embrace a culture of valuing the learning and the pursuit of quality that the Andon Cord signal encourages.”
Practice productively
317
This “fail-fast” principle²⁷ can be applied to any systematic process, well beyond manufacturing. Software²⁸ and hardware can also be designed to fail upon error-detection. In the Section “Master collections: Sets, lists and trees”, above, I gave an example of how this principle can itself be used in rehearsing lists (as opposed to sets): Stop immediately when you cannot recall the next element of the list rather than trying to hop over it onto the next one. In software development, applying the andon cord involves identifying and fixing problems quickly, as they occur. It also involves immediately analyzing the source of the problem: Why did it happen in the first place? Was the process followed? Does a process need fixing? Ries describes how his company (IMVU) requires that the team responsible for the problem address these issues before they can make further changes to the software base. While this lean principle, andon cord, may appear to slow things down, it actually enables projects to move forward because they do not create a shaky welter of software. A generalization of andon cord allows you to suspend a failed process and attend to it at a later point (by adding an entry into your task manager). Andon cord is itself a general learning (development) principle, because it is a signal to stop doing, or more generally to suspend doing something a certain way, and to engage in progressive problem-solving to improve a process. You can improve any task that you perform through applying the andon cord. This principle challenges us to confront the problem of transfer head on. When designing instillers for a principle, you need to reflect on its conditions of applicability. Under what circumstances would you want to pull an andon cord? This reflection in itself will deepen your understanding of the principle. Beneficial transfer is about generalizing appropriately. To truly master the andon cord principle is to be able to apply it to a very wide collection of circumstances. If this can be trained through productive practice, productive practice has made a significant contribution.²⁹ It will have bolstered not only one’s ability to innovate, but also one’s tendency to innovate. Once again, the key to developing a propensity is to develop motive generators: monitors that detect certain kinds of problems and respond to them by generating the appropriate motivator with the right insistence. To develop these monitors, we can set up regular review challenges. Many organizations perform regular reviews; often, however, reviews come late in the process, e.g., after a major release ships. Delaying feedback too much delays its effectiveness. A key to establishing monitors is to review recent events, because episodic memory decays rapidly³⁰. There is no other research on how to design instillers to detect process failures. So we can only speculate at this point, based on a cognitive task analysis. You could create instillers to review process-driven activities you executed at a random time of the given day, asking questions such as: What was the activity? What was its goal? Was the output of this activity of the expected quality? Was there an error? Did you interrupt the activity to fix the process or set a goal to do so? Beyond the andon cord, one could ask: Could the process be improved? Is the process even necessary? You ²⁷http://en.wikipedia.org/wiki/Fail-fast ²⁸http://www.martinfowler.com/ieeeSoftware/failFast.pdf ²⁹There is growing evidence that testing promotes transfer even without special transfer-focused practicing as described here. See Carpenter (2012)
for a review. Perkins (1995) argues that transfer is more likely to be obtained if training is transfer-oriented (provided participants are motivated to apply what they know, of course). ³⁰See Heathcote et al. (2000). See Mastaglio, Jones, Bliss, & Newlon (2007) for military applications.
Practice productively
318
may not be able to accurately answer these questions. However, the questions may cause monitors to develop (further).
14.5.3 Avoid harsh startups with your new mindware One of Gottman and Silver’s main marital problem-solving suggestions is to avoid “harsh startups.” That is a situation in which one member expresses a problem about and to the other in an unpropitious manner and neither party manages to diffuse the situation quickly enough. “Unpropitious” here typically means in a manner that is critical of the other person as opposed to being an aptly expressed complaint about their behavior. If the initiator expresses contempt and the other person criticizes, defends or stonewalls in response then the marriage is in trouble. Mismanaging problems in this manner leads to misery and divorce. Implicit in the foregoing description is that harsh startups are normally a joint responsibility. Thus, you need both to avoid blowing your lid and help your partner handle their own frustrations as well. Gottman and Silver provide a number of helpful exercises to avoid harsh startups. You could use the principles and software described in this book to design instillers for them. This section analyzes the information processing and tasks involved in avoiding harsh startups to help you design instillers for such mindware. As with any engineering task, before we design mindware instillers, we need to consider our requirements: What do we want our mindware to do? What mindware do we need to do it? What monitors, cognitive reflexes, motive generators, management procedures and long-term working memory are required to avoid harsh startups? To respond to a situation in which a member of a couple feels irritated towards the other for some seemingly offensive behavior requires the following. • Detect the signals for a possible harsh startup (monitoring). This means, we need to learn to detect situations that irritate our partners (e.g., feeling over-worked, hungry, etc.) and make her irritated at us. Conversely, we need to learn to detect when we are feeling irritated and when our partners annoy us. Once we detects such situations, we need to trigger the right motivators, as follows. • Suppress inappropriate motivators. Most of our motivators are generated unconsciously (asynchronously to conscious processes)³¹. They exert an influence on us whether or not we are aware of them. Anger is an emotion that is triggered when we feel that someone has done something wrong to us. Whether or not we notice or give way to it, anger tends to include a motive to harm the other party, a perceived license to harm the other, or a lowering of one’s normal standard to protect the other.³² One needs to learn to anticipate, detect and suppress this state.³³ (Notice that this too involves monitoring, but the monitoring is mainly of one’s mental state as opposed to external events.) The following recommendation is also a way to disarm such counterproductive motivators. ³¹See Beaudoin (1994) for a characterization of asynchronous motive processing. ³²This is not an empirical fact, it is an a priori conceptualization of anger (Sloman, 1987; Ortony et al., 1988). ³³I don’t mean to suggest that anger need always be suppressed. It is more accurate to state that anger, like other emotions, needs to be regulated
or controlled. See also Sloman (2001).
Practice productively
319
• Activate an appropriate high-level motivator and recognize it as important.³⁴ The key to avoiding harsh startups in my opinion is to activate an appropriate goal from which you can derive specific goals through means-end planning or other techniques. (The other principles proposed by Gottman and Silver can help here.) It’s up to you to decide the precise specification of the goal should be. A good starting point is a maxim consistent with Aristotle’s ethics: “It is foolish to sacrifice the long-term good for the short-term good”. The long-term good here is having a good relationship with your partner in which you both feel nurtured by each other. • Determine an appropriate objective and set of constraints for the situation, and assign them high intensity. (Recall that the intensity of a motivator is how strongly you pursue it at the moment.) These become part of your motivational set in this situation. These constraints should include the seven rules proposed by Gottman and Silver that I captured with the “I cappot, Crust!” RD cue in the section “Apply the RD cue system”. You should have practiced these rules already such that they will act as constraints on your motive processing and behavior such that you can avoid and defuse conflicts. • Generate and enact a plan to achieve the foregoing motivators. This may include some of the steps that Gottman and Silver propose under solve your solvable problems: e.g., to quickly correct the course of an interaction and to compromise. In order to learn what a harsh startup looks like and how you should respond to it, it helps to commit the essential components of Gottman and Silver’s harsh startup principle to memory. Simply committing the information to memory is not enough. I don’t suggest that you should recite these rules in the heat of a conflict. Nor are you expected to pass a written exam on this subject. By committing the information to memory, however, you will be able on demand to quickly review a recent situation in which you averted or engaged in a harsh startup. You will also be able to use the information to plan predictable conflict situations and practice with imaginary situations. Doing this in the right way will help. This is an example of the idea, discussed above, that experts differ from others not only in their skill, but in their knowledge. They have critical long-term working memory. If you are familiar with computer technology, you can think of this as rapid-access nonvolatile working memory, or information stored on a solid-state drive rather than on the network or a slow disk. That is why I used this principle as an example in the Section “Apply the RD cue mnemonic system” above. This will make it much easier for you to master the following challenges, and consequently generate mindware for marital conflict resolution. Here are some example challenges with which you can construct one or more instillers. • Consider a recent harsh startup: In order of intensity, what motivators drove your interaction? In terms of importance, what high-level motivators should have driven your interaction? (This will help you become aware of the motivators that are at play during interactions. It will also help you to develop an understanding and awareness of the significant difference between intensity, importance and urgency of motivators.) • Consider a recent harsh startup: In order of intensity or importance, what motivators drove your partner’s interaction? What was she feeling at the outset? Was she in an emotional state? ³⁴This principle is in accordance with broad cognitive science, as opposed to behaviorism, as it emphasizes the mental states that control other mental states and behavior. This is an information processing conceptualizaton of the affective principle noted in this section’s opening quotation.
Practice productively
320
(This will help you become more aware of your partner’s mental states. It will help you think of the different dimensions of motivators, such as insistence, intensity, importance, urgency. It will also encourage you to have debriefing sessions with your partner. It may also help you develop your ability to think of mental states, e.g., the distinction between motives, feelings and tertiary emotions.) • Which of the rules for avoiding harsh startups, if any, did you or your partner violate recently? (This challenge is meant to help you notice violations of the rules, and to avoid said violations. It’s best for the challenge not to mention the rules, so that you can practice recalling them. You might run through the following questions in your head: I cappot, I clearly, appreciatively & politely describe (but don’t judge) and complain (but don’t blame). Did I make statements that started with I instead of you? Was I unclear? Was I appreciative? Was I impolite? Did I evaluate or judge rather than describe? Did I blame rather than merely complain? Had I been storing things up?) • How could you have avoided violating these rules? What specific motivators should you have had? (This will help you develop new procedures to avoid violating the rules, and it will help you develop the tendency to apply these new procedures.) • Imagine a situation in which your partner pushes one of your buttons, what motivator should you respond with? (This requires that you document what your buttons are and that you have taught yourself to recall them on demand. ) As you practice these challenges, you will develop expertise and concomitant mindware. For example, many of these challenges call for you to remember a particular explosive situation. This will help you develop monitors for explosive situations. Because you are regularly (through productive practice) being called upon to answer questions about harsh startups, you will start noticing and anticipating them. The better you become at anticipating and noticing them, the more likely you will be to notice them in the future—that’s not a tautology but a reference to a bootstrapping positive feedback loop. This also of course leads one to avoid them. Practicing with these challenges may also help you to develop episodic memory for prior harsh startups, because they call upon you to remember past startups. (This will affect encoding, storage and retrieval of this information.) They will also help you develop long-term working memory for the list of harsh startup conditions (the target of the “I cappot, Crust!” RD cue), because you will frequently need to explicitly recall them. During your initial phase of practicing, you might run through the harsh-startup rules sequentially (using the RD cue mnemonic). If these exercises are well designed, this will lead you to develop monitors for each rule that operate in parallel.³⁵ A noteworthy element of these instillers is that they are designed to engage affective processes. This is consistent with the premise of Gottman’s theory, referred to in this section’s opening quotation. The challenges are also constructive. Apart from the list of principles to memorize, the challenges involve elaborative thinking, imagination and feeling. ³⁵While I refer to rules, I do not assume that they are represented explicitly in the mind. The mental architecture I have in mind is very high-level and could be implemented in various ways. As Bereiter (1991, p. 15) put it, “rules may influence behavior by being consulted, as we consult rule books in games and recipe books in cooking. Such rule consulting is especially important in the early stages of developing a skill that will eventually develop beyond the rules.”
Practice productively
321
14.6 Develop attitudes Generally, it is an artificial and narrow view which conceives of thinking as only an intellectual operation, and separates it entirely from questions of human attitude, feeling, and emotion — “because such topics belong to other chapters of psychology”. Max Wertheimer People are content with the vast majority of their own attitudes—their likes and dislikes. However, because attitudes can profoundly influence behavior and they can conflict with one’s best interests, reflective people sometimes (ought to) want to change some of their attitudes. In this section we will see how productive practice can be used not merely for self-persuasion, but to affect attitudinal changes. Attitude change is a problem that is normally considered to be within the realm of social psychology, and increasingly within cognitive social psychology.³⁶ Here, psychologists are mainly interested in making it easier for one party to change attitudes of another, through advertising, persuasion or other means. If you doubt the possibility of influencing attitudes, pray consider the large amounts of money that organizations spend on advertising. Much of this effort is to change behavior through changing people’s attitudes. They make careful use of attitude change literature. One might think that to change one’s attitudes is easier than to change those of others. But that is not always so. Advertisers use sneaky methods that are not easily applied by and to oneself. Moreover, if you are deliberately trying to change your attitude, odds are that you are dealing with a deep-rooted and relatively intransigent attitude. Advertisers typically address more superficial attitudes. We here, however, are concerned with the problem of changing one’s own attitudes. Open-minded people may find it easy to change their beliefs when faced with good arguments. (However, many beliefs are rooted in affect including wishful thinking.) Even they may nevertheless find it difficult to appreciate all the implications of their new views. Self-disciplined persons are relatively successful at changing their behavior. But for all persons, it is, by definition, very difficult to change deep-rooted attitudes. Why is attitude change sometimes difficult? One difficulty is conceptual: there are many different phenomena to which we apply the label “attitude” — it’s a polymorphic concept. Another problem is that there is no widely held explicit theory of attitudes in particular and affect in general. The traditional separation of cognition and affect (outside of broad cognitive science) has not helped. A good theory of affect should reinterpret, extend and in some cases replace folk concepts with mechanistic ones. In chapter 12, on assessment, I summarized Ortony, Clore and Collin’s cognitive theory of emotion as valenced reactions. They view attitudes as a kind of value, namely dispositional liking or disliking of objects or attributes of objects. Attitudes determine judgments of appeal. Goals determine judgments of desirability. Norms determine judgments of praiseworthiness and approval. More abstractly, however, we can view attitudes as a valenced component of motivators. They are effective to the extent that there are underlying mechanisms that generate motives to act upon them, i.e., motive generators (see chapter 5, “Internal Motivators” and “Motive generators”.) ³⁶For the history of treatment of affect in cognitive science, see Boden (2006, pp. 368-404). This includes a discussion of attitude change and the work of Sloman and myself.
Practice productively
322
Attitudes in this sense involve control states, including motive generators and beliefs. They cause us to select objects, behaviours and future states of the world. These control states not only incline us towards their respective attitudinal objects, they inhibit other motivators. To complicate matters, these objects are not necessarily whole, they may be features. For example, you might like one aspect of a food (its sweetness) but dislike another (its glycemic load). Yet the biggest difficulty in changing deep-rooted attitudes cannot be overcome through further understanding. It is that attitudes involve motive generators that are modular and themselves relatively autonomous.³⁷ They have their own defenses within the society of the mind they inhabit. Try to stop liking sweets, symmetry or your own children and you likely will have a major struggle on your hands. Here are some examples of typical attitudes people might want to change. One might want to like someone else more, e.g., one’s spouse, teen-aged step-children, family of origin, in-laws, colleagues, or other people that one cannot easily avoid. One might even want to like someone less (e.g., a CIA leader’s mistress might want to feel less attracted to her lover; one might want to stop longing for someone who has left or died). In order to bridle one’s inclinations, one might want to dislike objects of temptation: e.g., sweets, fatty foods, cigarettes, caffeine, alcohol. Because some attitudes are so deeply rooted, some of the major techniques for modifying them target the lowest form of learning: associative conditioning. One way to dislike something is to associate it with something you strongly dislike. When I became aware of health problems posed by hydrogenated vegetable oils, I decided to put a very conspicuous note on packages containing them that read “POISON!” This seems to have worked for me. I would then check labels and recoil at the sight of the packages that have them. (I later simply stopped eating products with grains in them.) Psychologists have found that not all stimuli are as easy to mentally associate as others. However, the brains of mammals can easily associate distinctive tastes (foods and liquids but not water itself) with nausea. If you have ever been very nauseous very soon after eating a distinctive food, you know what I mean. The “POISON!” technique is also an example of what Zhang calls “counteractive construal” (Zhang, Huang, & Broniarczyk, 2010). It is to exaggerate the negative side-effects of a choice in order to control the choice behavior. I construe this more generally, in that the target is not merely a specific choice, but an entire (more general) underlying attitude. There are other techniques, including making a public commitment towards abstaining from objects that conflict with one’s goals or standards, or conversely to engage with some that are consistent with our desired preferences. For example, someone who is tempted by speculative investments could talk about the dangers of aggressive investments to his friends, using the example of John Allen Paulos (WorldCom) described by Keith Stanovich (and Paulos himself). One can also leverage the principle that familiarity breeds liking and expose oneself to objects one wishes to like. And to learn more and more about them. Therapists attuned to the relations between cognition and affect have developed several techniques that are aimed at changing behavior via changing not only beliefs but underlying attitudes. A common technique used by cognitive-behavioral psychologists to deal with attitude change is to ³⁷Huang and Bargh (in press) claim that goals are relatively autonomous, which is consistent with my position. They apply the selfish gene analogy to goals. Their theory would be stronger if it was directed at motivators rather than only goals and if it considered more of the attributes and processes involving motivators.
Practice productively
323
provide clients with exercises to modify the beliefs underlying attitudes and standards they’d like to modify. (Leahy, 2003, provides several examples.³⁸) This sometimes involves asking questions that will elicit answers about the desirable aspects of the objects that one wants to like, and the undesirable aspects of objects one wishes to dislike. There need to be several questions that get at manifold aspects of the object in question. For example, Gottman and Silver provide a large number of such questions for partners to redevelop and sustain positive attitudes towards each other. This suggests some very important connections between self-directed learning and psychology, a subject broached in the next chapter. ³⁸That book is designed for psychotherapists. I see no good reason why an effectant person should not equip himself or herself with such high caliber knowledge if they need it. It is not particularly abstruse. It also serves to demystify psychotherapy.
IV Conclusion This “bootstrap” group has the interesting (recursive) assignment of developing tools and techniques to make it more effective at carrying out its assignment. Douglas Engelbart
15. Meta-effectiveness framework and clinical psychology Être figé, c’est une erreur colossale. Il faut arriver, par discipline, que d’avoir des tentations relativement nobles. Et après ça, il est urgent de les succomber. Vous comprenez? Même si c’est dangereux. Même si c’est impossible. Surtout ci c’est impossible.¹ Jacques Brel Latent in the meta-effectiveness framework lies an important idea. I have emphasized mental development, I’ve used examples from relationship books, I’ve referenced cognitive therapy, metacognitive therapy, and acceptance and commitment therapy (ACT²). I have refuted the common implicit assumption that education and self-directed learning are merely concerned with “cognitive” change—affective, cognitive and executive processes are intertwined. The latent idea, then, is that there is a significant overlap between what psychologists often try to achieve with their clients, teachers aim to effect in their students, some authors hope to provide for their readers, and what readers should try to do with (at least some) potent knowledge resources, namely to produce deep mental change. That is, the purpose of psychotherapy, teaching and self-directed learning is normally mindware development. Here are three sets of important connections between psychotherapy and the meta-effectiveness framework (of self-directed learning). 1. Whether one is delving self-help literature or technical information, if one aims to learn, one is engaged in self-help. That is, self-directed learning is a form of self-help, of autopsychotherapy. Being informed of psychotherapeutic ideas that are pertinent to one’s situation is often not sufficient to effect the required change. Psychotherapists, and this book, recognize the difficulty of effecting mental change. Such change requires, or at least can be facilitated by, a deliberately meta-effective approach to learning. 2. In developing a meta-effectiveness framework, the cognitive scientist should draw from clinical psychology. Clinical psychology is a powerful source because it draws from multiple branches of psychology. Psychologists have an opportunity to apply and refine ideas in deep, meaningful and temporally extended interactions with clients (compare the discussion of reflective practice.) 3. Clinical psychology stands to benefit, both theoretically and methodologically, from research on meta-effectiveness, including its information processing models. The rest of this chapter focuses mainly on the latter two ideas, chiefly with respect to ACT. ¹To stagnate is a colossal error. One must, through discipline, come to have only noble temptations. And then it is urgent to succumb to them. You understand? Even if it is dangerous. Even if it is impossible. Especially if it is impossible. ²See Hayes, Strosahl & Wilson (2011) for a scholarly exposition of ACT. Harris (2007) is a more accessible book on ACT. See Wells (2005, 2008) on metacognitive therapy. Wells & Matthews (1994) provides the theoretical basis for metacognitive therapy.
Meta-effectiveness framework and clinical psychology
326
15.1 The pertinence of psychotherapy concepts and methods to meta-effectiveness People tend to assume that learning is monotonic. They read a book, and they want to add new skills, facts, etc., to their “store”. They want to master more knowledge. They might think in terms of “adding new connections” to their brain. But learning actually involves a lot of reorganization of mindware. It also requires dismantling mindware (“unlearning”). For example, in order to master the lean approach to product development, you may need to dismantle an engrained tendency to put out dazzling, feature rich initial products and revisions. A competent clinical psychologist is able to help her clients disassemble unhelpful mindware (though psychologists don’t tend to use the term “mindware” yet). Cognitive therapy, for instance, is ripe with techniques to help people identify, challenge and ultimately drop their unhelpful beliefs (Leahy, 2003, provides an extensive catalog.) Some of these techniques could be leveraged in productive practice to unlearn mindware outside the clinical realm. ACT takes a more subtle approach to unhelpful mindware. ACT (like H-CogAff) recognizes that mentation is generated by heuristic mechanisms, not by an omniscient self. Whereas cognitive therapy focuses on the caliber of beliefs, ACT zeroes in on their pragmatic value: whether or not they are useful. Much of our mental content is neither true nor false. (Austin, 1956, provides many illustrations of this with respect to language. Expressions, such as, “I pronounce you man and wife”, may look like assertions, but they are acts. Pushing a button on a computer could have the same effect.) Even true beliefs can be unhelpful. Treating unhelpful beliefs as necessarily true, wise, pertinent, important, and/or imperative commands is maladaptive. For example, believing that you must carry on your deceased spouse’s mission might deprive you of pursuing a more meaningful, self-actualizing path. ACT has many “cognitive defusion” techniques to help clients distance themselves from their unhelpful thoughts. The operant goal of these techniques is not to make the thoughts go away, but for them to lose some of their potency. One of these techniques is to name the story behind an unhelpful thought. For example, the widower might name his intrusive theme, the “I’m on a mission from the grave” story. Another is to sing an unhelpful thought to the tune of a silly song, such as the Happy Birthday song (Harris, 2007). If you’re trying to apply lean principles to a project, but feel uncomfortable about releasing a partial version of it, you might call that the “It’s gotta be a perfect product” story, and sing it to the tune of Happy Birthday. ACT also recognizes that much of our affective mentation is not particularly helpful. We’ve evolved to survive and procreate in a very different environment from the modern one. As a result, even in a safe environment, we tend to overestimate the magnitude of threats and losses (catastrophizing). ACT promotes accepting unpleasant feelings and situations that cannot be changed. It tries to help clients give up on their futile attempts to control things they can’t control. Several ACT techniques are aimed at making the client aware, or more aware, of unpleasant affect. (There are different levels of awareness of affect.) ACT does not try to assist clients to like these unpleasant feelings and situations, but to be willing to co-exist with them. The ultimate goal is to help clients direct their conduct in productive directions that they value. A common side-effect (one that paradoxically must not be directly sought) of these techniques is that the unpleasant feelings
Meta-effectiveness framework and clinical psychology
327
tend to wane when accepted.³ My point is not that these leading psychotherapies can easily be applied to self-directed learning. Instead, I am pointing out overlaps and potential synergies for future research. Investigating why and how research-based therapy affects mindware and mental architecture might lead to new developments in the underlying theories of this book (e.g., H-CogAff architecture).
15.2 The practical relevance of meta-effectiveness to psychotherapy However many holy words you read, however many you speak, what good will they do you if you do not act on upon them? Gautama Buddha When a psychotherapist works with an educated adult, normally, she should direct him to helpful knowledge resources. This is not to say that psychotherapy is primarily didactic. However, there is significant knowledge behind evidence-based psychotherapy. There is also an important instructional aspect to psychotherapy (Martin & Hiebert, 1985.) Clients normally do need to acquire concepts⁴ and skills. Normally there are accessible knowledge resources that can aid the client with specific content. I also suspect that some chapters written for psychologists themselves can be directly prescribed to their more effectant clients. For instance, chapter 10 of Hayes, Strosahl, & Wilson (2011), on acceptance, seems to be pertinent to, but not too abstruse for, educated ACT clients. But therapists shouldn’t assume that a client will sufficiently benefit from a resource through normal reading. It stands to reason that a client who applies a meta-effectiveness framework, such as the one described in this book, is more likely to deeply understand and apply the knowledge the resource conveys than someone who doesn’t. This suggests that therapists themselves should become familiar with this or other meta-effectiveness frameworks. They could either pass on some of the knowledge to their clients or train them to apply specific learning strategies for the purpose of instilling the required knowledge. For example, many psychotherapists get their clients to conduct homework. However, to my knowledge they do not tend to leverage the extensive literature on test-enhanced learning and deliberate practice. The concept of productive practice, which is explicitly geared towards developing ³Given the allusion to unlearning with which I opened this chapter, I should mention that Hayes, Strosahl & Wilson (2011, chapter 2) claim that there is no such process as unlearning. This, however, is not correct—while memories tend to persist, transience is an established counterbalancing principle of memory (Schacter, 1999, 2002). There are mechanisms that can alter memories, affect their accessibility and decrease affective responses to events—all forms of unlearning. Rather than deny unlearning or suggest it is easy to attain, I’d say that the mechanisms of unlearning need to be better understood. Consider the following anecdote recounted by cognitive scientist Randy Gallistel: “A couple of years ago [Jennifer Groh, a neurophysiologist] was down giving a talk to us. And she came into my office, and the first thing she said was, ‘You know, Randy, we really don’t understand how the brain computes.’ I said, ‘Jennifer, that is God’s own truth!’ (Campbell, 2010). (Nor do physicists know why there is gravity.) We must, however, proceed, with humility, as reflective practitioners of the most helpful psychological knowledge at our disposal. ⁴For example, metaphor is a potent educational technique exploited in psychotherapy. See Linehan (1988, pp. 209-212) for references and examples of metaphor in the dialetical treatment of borderline personality disorder. (Treating borderline personality clients calls for oblique methods because these people typically have low insight into their conditions. Understanding how these methods work might provide insight into normal mental development.) Anthony Robbins is a master of metaphor in inspirational self-help literature. (Compare Robbins 1991). Potent metaphors can lead to deep mental change. The meta-effective self-directed learner is disposed to master highly helpful metaphors he encounters.
Meta-effectiveness framework and clinical psychology
328
mental mechanisms based on this literature, seems quite promising for effecting psychotherapeutic change. For example, instillers could be designed for most of the nearly 90 techniques in Leahy’s Cognitive Therapy Techniques: A Practitioner’s Guide. Instillers could also be designed to convey the concepts of ACT (acceptance, diffusion, etc.) I have amply illustrated in this book how this framework can help one acquire the concepts of bid and harsh-startup. Well-designed productive practice software would minimize the effort required for the client to master the required knowledge while also promoting transfer. Psychotherapy and self-directed learning are well-aligned, for the goal is not merely to comprehend the resource, but to help the client perceive, think, act and feel in accordance with it. The concepts and techniques described here can be applied not merely to third-party knowledge resources, but to the knowledge conveyed directly by the therapist in conversation. If you are on the client side of therapy, consider requesting bibliographical references from your therapist. In consultation with your therapist, consider applying what you have learned here to these resources and to the knowledge conveyed in session.
15.3 H-CogAff (mental architecture) and ACT as complementary I believe there is potential for psychotherapy systems—such as cognitive therapy and ACT—to benefit from a close examination of the meta-effectiveness framework, including information processing architectures, and vice-versa. These clinical approaches in particular leverage general notions from classical cognitive psychology. They do not tend to invoke information processing mechanisms and architectures. These frameworks could be improved with the broad, modern cognitive science described in this book. Consider that Wright, Sloman & Beaudoin (1996) applied the H-CogAff theory of affect to grief. Boden (1996) described our contribution as follows: What they have done is to show—in relatively clear, and potentially testable terms— how this sort of psychological phenomenon [grief] is possible, and how the cognitive, affective, and motivational aspects of the mind are intimately combined in its very structure. That is, they have shown what sort of information-processing system a mind must be, in order that it may evince emotional phenomena of this type. In doing so, they have laid the groundwork for a systematic approach to therapy. And they have whetted our appetites—or at least, mine—for the implementational studies that are already being commenced in their research program. The defining feature of perturbant emotions is their ability to disrupt management processes. In the conclusion of my Ph.D. thesis, I suggested, in general terms, that our theory could advance the understanding of obsessive-compulsive disorder. Perturbant emotions essentially involve obsessions and compulsions. Given that ACT is based on a behavioral framework (relational frame theory)⁵, one might ⁵Cf. Torneke (2010).
Meta-effectiveness framework and clinical psychology
329
find my proposal of melding it with H-CogAff incongruous. However, consider that ACT has already assimilated ideas outside of clinical psychology, such as some of the non-mystical ideas of Buddhism (Hayes, 2002b). H-CogAff has been reconciled with elements of Gibson’s (1979) theory of affordances (Sloman, 1989). (Gibson was no cognitivist!) ACT emphasizes that much mental experience is not under deliberate control. H-CogAff characterizes lack of control of mental processes in an architectural and functional manner (Sloman, 1987; Beaudoin, 1994). H-CogAff sees tertiary emotions (perturbance) as a loss of control of management processing. ACT refers to an “observing self” and implicitly to generators of mental content. However, neither ACT nor relational frame theory specifies a mental architecture.⁶ Thus, there is a large void waiting to be filled in ACT’s theory. The computational architecture (described in chapter 5) or one of its alternatives would complement ACT. There is also a need to explain the information processing mechanisms in phenomena and techniques ACT emphasizes, such as fusion and defusion. H-CogAff has an architecture that might prove helpful. Presumably, defusion affects insistence assignment mechanisms and interrupt filters. By trying to explain motivational and emotional processes exhibited in clinical settings, researchers could extend the H-CogAff framework (or alternative frameworks). This in turn could lead researchers to propose new therapeutic techniques. I recently demonstrated this kind of synergy between clinical psychology and cognitive science in a paper on sleep onset (Beaudoin, 2013). I analyzed sleep onset mentation. I examined existing psychotherapeutic techniques for facilitating sleep onset and found they lacked an information processing basis. I assumed that by understanding why they work when they do, we could improve the techniques. In my foray into sleep psychology, I needed to propose several new concepts (e.g., counter-insomnolent, somnolent and super-somnolent mentation). From my analysis, I derived a new family of somnolent techniques (e.g., the cognitive shuffle and serial diverse imagining). I proposed these techniques could be used in principle to fool parts of the brain that control sleep onset into thinking it is time to initiate (and then deepen) the progression into sleep. (By way of disclosure, CogSci Apps Corp., of which I am a co-founder, is now marketing software, mySleepButton⁷ that exploits the new prediction.) To put it abstractly, clinical phenomena of insomnia provided an impetus to develop theory. From theory, new techniques (i.e., hypotheses) can be developed, applied, evaluated and potentially applied by mental health practitioners. Of course, some of the theory’s predictions might be falsified; and the newly discovered phenomena might be accounted for by an alternative theory. Even so, this synergistic process advances science and practice. If the cognitive shuffle techniques pan out on a large scale, improving the lives of millions of people, we will have a compelling example of the relevance of broad cognitive science to psychotherapy. As a non-clinical example of the potential of theoretical cognitive science, consider Lamontagne’s computational theory of visual motion perception Lamontagne (1973, 1976). His AI simulation correctly predicted a stunning class of visual illusions that probably never would have ⁶The notion of “self” used in relational frame theory is described in Torneke (2010, pp. 101-110). It is clearly non-computational and behaviorist. H-CogAff does not have a technical notion of self. I agree with the conceptual analysis of self presented by Sloman (2011d), namely that self refers to the entire person. If one wished to isolate self as the non-material aspect of a person, it would suffice to use it as “mind”, and for that to refer to the information processing system (virtual machines) layered on the human brain. ⁷http://mySleepButton.com
Meta-effectiveness framework and clinical psychology
330
been discovered without his information processing theory. Metacognitive therapy shares several of ACT’s tenets but is founded on information processing psychology (Wells & Matthews, 1994). It has several similarities with the one I’m advocating. It draws on some of the bodies of literature referenced in the current book. It also emphasizes the obsessive nature of emotion. Wells & Matthews expressed an opinion that is central to my criticism of cognitive therapy “[existing] cognitive theories offer no more than a basic principle for therapy: namely, modifying patient’s belief in dysfunctional appraisals and assumptions and generating replacement knowledge. This approach tends to focus on modifying the content of cognition (declarative knowledge), but it is likely that procedural knowledge is at least as important in dysfunction.” (p. 297). H-CogAff goes well beyond the declarative/procedural distinction. Thus metacognitive therapy might also benefit from the H-CogAff theoretical approach (e.g., its analysis of motivators and concepts of motive processing). It is an important challenge for science, psychotherapy and personal development to better understand both how to dismantle “sticky” motive generators that have functional autonomy and how to assemble new mindware.
16. Delve and instill the knowledge of your choice The relevance of the opening quotation of this book, “Only the ideas that we actually live are of any value”, should now be evident. Potentially useful, high-caliber knowledge too often lies wasted in superficial mindware. It would be difficult to overstate the importance of that which enables and motivates you to instill knowledge: meta-effectiveness. With the right mindware one can intelligently perceive the world, prevent predicaments and solve problems. Having reached the conclusion of this long book, how are you supposed to instill the knowledge expressed in Part 3 to bootstrap your learning? I recommend that you start by choosing a helpful resource—something potent, useful and of high caliber that appeals to you. It may be as broad or narrow in scope as you like. Look at your library for inspiration. For reasons discussed in chapter 12, select a resource that you can access electronically, preferably with a PDF reader or Apple’s iBooks. It’s important to pick a challenging resource the mastery of which will immediately give you significant benefits. The expected yield will motivate you to apply the required effort. Applying your new mindware will be inspiriting. This might motivate you to sharpen your meta-effectiveness “saw” with other resources too. You need not master all aspects of the resource. Focus on its gems. Selecting and mining will help you sharpen your assessment skills and dispositions. While the topics of Part 3 are presented in natural order, you can focus on areas of competence (and hence chapters) in the order of your choice: learning your way around, assessing, delving, or practicing. It’s best to focus on one skill set at a time with one resource. Then repeat with other resources. That way you will get the benefits of spacing. Further, I recommend that before or as you delve into your chosen knowledge resource, you also apply delving techniques to Cognitive Productivity itself. By regularly applying cognitive productivity concepts and techniques, you will get the benefits of practice that are described throughout this book. When world champions rework some of their core competencies, their performance degrades temporarily. Thus, your own information-processing velocity will decrease temporarily as you develop your meta-effectiveness. That is to be expected and accepted. What previous quotation of Marvin Minsky is relevant here? Oh yes, “No matter what one’s problem is, provided that it’s hard enough, one always gains from learning better ways to learn”.
Postscript Since I first marked this book complete in 2014, I have frequently improved it. The revision history is available online¹. The word count of this book is quite high, so I am unlikely to add to it substantially. I have even decided to publish the bulk of its postscript on CogZest², rather than in the book itself. However, I will correct and update this book as necessary. The online postscript³ describes several projects of mine that are derived from this book. These include new research, software development and books. In the online postscript, I also comment about select, recently published books that are pertinent to Cognitive Productivity. In particular, I discuss Cal Newport’s Deep Work. I encourage you to compare and contrast it with Cognitive Productivity. On CogZest.com⁴, I continue to write about cognitive productivity. ¹http://cogzest.com/books/release-notes-for-cognitive-productivity-book-by-luc-p-beaudoin/ ²https://cogzest.com/books/cognitive-productivity-luc-beaudoin/postscript/ ³https://cogzest.com/books/cognitive-productivity-luc-beaudoin/postscript/ ⁴http://CogZest.com
References 60-Minutes-Overtime-Staff. (2013, May 12). Bill Gates on Steve Jobs: We grew up together. 60 Minutes Overtime. Retrieved May 13, 2013, from http://www.cbsnews.com/8301-504803_16257584072-10391709/bill-gates-on-steve-jobs-we-grew-up-together/ Aarts, H., & Dijksterhuis, A. (2000). Habits as knowledge structures: Automaticity in goal-directed behavior. Journal of Personality and Social Psychology, 78(1), 53–63. doi:10.1037//00223514.78.1.53 Ackerman, P. L. (1996). A theory of adult intellectual development: Process, personality, interests, and knowledge. Intelligence, 22(2), 227–257. Ackerman, P. L. (2008). Knowledge and cognitive aging. In F. I. M. Craik & T. A. Salthouse (Eds.), The Handbook of Aging and Cognition. New York, NY: Psychology Press. Ackerman, P. L. (2013). Nonsense, common sense, and science of expert performance: Talent and individual differences, Intelligence doi:10.1016/j.intell.2013.04.009 Ackerman, P. L., Beier, M. E., & Boyle, M. O. (2005). Working memory and intelligence: The same or different constructs? Psychological Bulletin, 131(1), 30–60. doi:10.1037/0033-2909.131.1.30p Adler, M., & Van Doren, C. (1972). How to read a book (2nd ed.). New York, NY: Simon & Schuster. Al-Omar, M., & Cox, A. (2013). Finders, keepers, losers, seekers: A study of academics’ research-related personal information collections. In Human Interface and the Management of Information. Information and Interaction Design (pp. 169-176). New York, NY: Springer. Alberts, B. (2000). Some thoughts of a scientist on inquiry. In J. M. E. van Zee (Ed.), Inquiring into inquiry learning and teaching in science (pp. 3–13). American Association for the Advancement of Science. Allen, D. (2001). Getting things done: The art of stress-free productivity. New York, N.Y.: The Penguin Group. Allen, D. (2008). Making it all work: Winning at the game of work and business of life. Allport, A. (1989). Visual attention. In M. I. Posner (Ed.), Foundations of cognitive science (pp. 631– 682). Cambridge, MA: MIT Press. Allport, D. A. (1980). Patterns and actions. In G. Claxton (Ed.), Cognitive psychology: New directions (pp. 26–64). London, England: Routledge & Kegan Paul. Allport, Gordon W. (1937). The functional autonomy of motives. American Journal of Psychology. 50 141–56. Allport, G. W. (1961). Pattern and growth in personality. Harcourt College Publishers. American Diabetes Association. (2012). Standards of medical care in diabetes (Position statement). Diabetes Care, 35. doi:10.2337/dc12-s011 Anastasi, A. (1988). Psychological testing. New York, NY: Macmillan.
334 Anderson, J. R. (1990). The adaptive character of thought. Mahwah, NJ: Laurence Earlbaum Associates. Anderson, J. R., & Lebiere, C. (1998). The atomic components of thought. Mahwah, NJ: Erlbaum Publishers. Anonymous (2001). The in crowd: the seventh annual selection of the best and brightest among achievers on the rise. (2011, June 26). Report on Business Magazine, 17(11), 62–82. Aristotle. (350 BCE, 2000). Nicomachean Ethics. (W. D. Ross, Trans.). MIT (Internet Classics Archive). Retrieved from http://classics.mit.edu/Aristotle/nicomachaen.html Aubé, M. (2005). Curieuse curiosité… Vie Pédagogique, 135, 46–49. Austin, J. L. (1956). A plea for excuses. Proceedings of the Aristotelian Society, 57. Baars, B. J. (1988). A cognitive theory of consciousness. New York, NY: Cambridge University Press. Baars, B. J. (1997). In the theater of consciousness. Oxford, England: Oxford University Press on Demand. Baker, A. G., & Mercier, P. (1982). Extinction of the context and latent inhibition. Learning and Motivation, 13(4), 391–416. doi:10.1016/0023-9690(82)90001-7 Baker, A. G., Berbrier, M. W., & Vallee-Tourangeau, F. (1989). Judgements of a 2× 2 contingency table: Sequential processing and the learning curve. The Quarterly Journal of Experimental Psychology. B, Comparative and Physiological Psychology, 41(1), 65–97. doi:10.1080/14640748908401184 Baker, A. G., Murphy, R. A., & Mehta, R. (2003). Learned irrelevance and retrospective correlation learning. The Quarterly Journal of Experimental Psychology. B, Comparative and Physiological Psychology, 56(1), 90–101. doi:10.1080/02724990244000197 Bandura, A. (1997). Self-efficacy: The exercise of self-control. Gordonsville, VA: WH Freeman & Co. Chicago Bahrick, H. P., & Phelphs, E. (1987). Retention of Spanish vocabulary over 8 years. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13(2), 344. Barnacle, R. E. S., & Abbott, D. A. (2009). The development and evaluation of a Gottman-Based premarital education program: A pilot study. Journal of Couple & Relationship Therapy, 8(1), 64–82. doi:10.1080/15332690802626734 Baron, Jonathan. (1994). Nonconsequentialist decisions. Behavioral and Brain Sciences, 17(01), 1–10. doi:10.1017/S0140525X0003301X Baron, Joseph. (2008). Thinking and deciding. New York, NY: Cambridge University Press. Bartlett, F. (1938). Remembering. Cambridge, England: Cambridge University Press. Baumeister, R. F., & Twenge, J. M. (2003). The social self. In T. Millon, M. J. Lerner, & I. B. Weiner (Eds.), Handbook of Psychology (Personality and Social Psychology) (Vol. 5, pp. 327–352). Hoboken, NJ: John Wiley & Sons, Inc. Beaudoin, L. P. (1990). A computational investigation of the evolution of vision (unpublished Honours thesis) University of Ottawa, Ottawa, Canada. Beaudoin, L. P. (1994). Goal processing in autonomous agents. (Doctoral dissertation). University of Birmingham, Birmingham UK. Retrieved from (http://www.sfu.ca/∼lpb/tr/Luc.Beaudoin_thesis.pdf Beaudoin, L. P. (2002). gStudy - Release Functional Specification (No. GST-RFS-01) Learning Kit Project, Simon Fraser University, Burnaby, Canada.
335 Beaudoin, L. P. (2010a, Jan. 26). Will the Apple tablet support or hinder users’ cognitive fitness? Retrieved from http://www.sharpbrains.com/blog/2010/01/26. Reprinted with corrections and additional commentary as Beaudoin (2011b). Retrieved from http://cogzest.com/2011/10/stevejobs-and-the-topic-of-cognitive-productivity/ Beaudoin, L. P. (2010b, Feb. 11). Apple iPad thumbs-up: brain fitness [sic] value, and limitations [should have read “cognitive fitness”]. Retrieved from http://www.sharpbrains.com/blog/2010/02/11/. Reprinted with corrections and additional commentary as Beaudoin (2011, Oct. 8) Retrieved from http://cogzest.com/2011/10/steve-jobs-and-the-topic-of-cognitive-productivity/ Beaudoin, L. P. (2011a). Experts’ productive learning from formal knowledge: Motive generators and productive practice. In J. Wyatt (Ed.), A symposium in honour of Aaron Sloman: From animals to robots and back: Reflections on hard problems in the study of cognition. (pp. 139– 172). Birmingham, UK: University of Birmingham. (See Beaudoin, 2014b) Beaudoin, L. P. (2011b). Steve Jobs and cognitive productivity. Retrieved from http://cogzest.com/2011/10/steve-jobs-and-the-topic-of-cognitive-productivity/ Beaudoin, L. P. (2011c). The designer stance towards Shanahan’s dynamic network theory of the ‘conscious condition’. International Journal of Machine Consciousness, 3(2), 313–331. doi:10.1142/S1793843011000716 Beaudoin, L. P. (2013). The possibility of super-somnolent mentation: A new information-processing approach to sleep-onset acceleration and insomnia exemplified by serial diverse imagining. Simon Fraser University Summit repository. Retrieved from http://summit.sfu.ca/item/12143 Beaudoin, L. P. (2014a, July). A design-based approach to sleep-onset and insomnia: super-somnolent mentation, the cognitive shuffle and serial diverse imagining. Paper presented at the 36th Annual Conference of the Cognitive Science Society’s workshop on “Computational Modeling of Cognition-Emotion Interactions: Relevance to Mechanisms of Affective Disorders and Therapeutic Action”, Québec, Canada. Beaudoin, L. P. (2014b). Developing expertise with objective knowledge: Motive generators and productive practice. In J. Wyatt, D. Petters, & Hogg, D. (Eds.), From robots to animals and back: Reflections on hard problems in the study of cognition (pp. 161-190). Berlin, Germany: Springer. Beaudoin, L. P., & Sloman, A. (1993). A study of motive processing and attention. In A. Sloman, D. Hogg, G. Humphreys, D. Partridge, & A. Ramsay (Eds.), (pp. 229–238). Prospects for Artificial Intelligence (Proceedings of AISB–93, Birmingham, England), Amsterdam, Netherlands: IOS Press. Beaudoin, L. P., & Winne, P. (2009, June). nStudy: An Internet tool to support learning, collaboration and researching learning strategies. Paper presented at the Canadian E-learning Conference (CELC-2009). Vancouver, BC. Retrieved from http://www.sfu.ca/∼lpb/tr/2009-Luc_P_BeaudoinPhil_Winne-nStudy.pdf Beaudoin, L. P. & Winne, P. (2010). Principles of psychology for designing software to improve recall performance in older adults. Gereontechnology Conference Issue: Abstracts of the 7th World Conference, The Vancouver Conference Issue. 9(2), 192. Beavers, K. M., Brinkley, T. E., & Nicklas, B. J. (2010). Effect of exercise training on chronic inflammation. Clinica Chimica Acta, 411(11–12), 785–793. doi:10.1016/j.cca.2010.02.069 Bélanger, M.-E. (2010). The annotative practices of graduate students: Tensions & negotiations
336 fostering an epistemic practice (Master’s thesis). University of Toronto, Toronto, ON, Canada. Benjamin, A. S., Bjork, R. A., & Schwartz, B. L. (1998). The mismeasure of memory: when retrieval fluency is misleading as a metamnemonic index. Journal of Experimental Psychology: General, 127(1), 55. Bereiter, C. (1991). Implications of connectionism for thinking about rules. Educational Researcher, 20(3), 10–16. Bereiter, C. (1995). A dispositional view of transfer. In A. McKeough, J. Lupart, & A. Marini (Eds.), Teaching for transfer: Fostering generalization in learning (pp. 21–34). Mahwah, NJ: Laurence Earlbaum Associates. Bereiter, C. (2002a). Education and mind in the knowledge age. Mahwah, NJ: Laurence Earlbaum Associates. Bereiter, C. (2002b). Design research for sustained innovation. Cognitive studies, Bulletin of the Japanese cognitive science society, 9(3), 321–327. Bereiter, C., & Scardamalia, M. (1993). Surpassing ourselves: An inquiry into the nature and implications of expertise. Chicago, IL: Open Court. Bereiter, C., & Scardamalia, M. (2012). Theory building and the pursuit of understanding in history, social studies, and literature. In J. R. Kirby & M. J. Lawson (Eds.), Enhancing the quality of learning: Dispositions, instruction, and learning processes (pp. 160–177). Cambridge, England: Cambridge University Press. Bernard, L. L. (1924). Instinct: A study in social psychology. New York, NY: Holt, Rinehart and Winston. Bertsch, S., Pesta, B. J., Wiscott, R., & McDaniel, M. A. (2007). The generation effect: A meta-analytic review. Memory & cognition, 35(2), 201–210. Bielaczyc, K., Hakkarainen, K., Ritella, G., Seitamaa-Hakkarainen, P., Stahl, G., Scardamalia, M., & Bereiter, C. (2011). Strengthening the conceptual foundations of Knowledge Building theory and pedagogy. In H. Spada, G. Stahl, N. Miyake, & N. Law (Eds.), CSCL2011 Computer Supported Collaborative Learning: Connecting Research to Policy and Practice, Conference Proceedings, Vol. III (pp. 1089-1094). Hong Kong: International Society of the Learning Sciences. Bjork, E. L., & Bjork, R. A. (2003). Intentional forgetting can increase, not decrease, residual influences of to-be-forgotten information. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29(4), 524–531. doi:10.1037/0278-7393.29.4.524 Bjork, R. A., Laberge, D., & Legrand, R. (1968). The modification of short-term memory through instructions to forget. Psychonomic Science, 10(2), 55-56. Blair, M. (2013). Real-time strategy video games: a new “drosophila” for the cognitive sciences. In N. Hedberg (Ed.). Presented at the 2013 Spring Defining Cognitive Science talk series, Burnaby, Canada. Bloch, J. (2001). Effective Java: Programming language guide. Sun Microsystems. Bloom, B. S. (1956). Taxonomy of educational objectives: The classification of educational goals: Handbook I, cognitive domain. (B. S. Bloom, Ed.). New York, NY: David McKay. Bloom, B. S. (1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13(6), 4–16. doi:10.3102/0013189X013006004 Boden, M. A. (1977). Artificial Intelligence and natural man. London, England: MIT Press.
337 Boden, M. A. (1978). Purposive explanation in psychology. Cambridge, MA: Harvard University Press. Boden, M. A. (1988). Computer models of mind. Cambridge, England: Cambridge University Press. Boden, M. A. (1996). Commentary on “Towards a design-based analysis of emotional episodes.” Philosophy, Psychiatry & Psychology, 3(2), 135–136. Boden, M. A. (2004). The creative mind: Myths and mechanisms (2nd ed.). New York, NY: Routledge. Boden, M. A. (2006). Mind as machine: A history of cognitive science (2 volumes). Böhme, G. (2012). Invasive technification: Critical essays in the philosophy of technology (C. Shingleton, Trans.). London: Bloomsbury. Bostic, K. (2013, October 4). Behind-the-scenes details revealed about Steve Jobs’ first iPhone announcement. Retrieved from http://appleinsider.com/articles/13/10/04/behind-the-scenes-details-reveal-steve-jobs-first-iphone-announcement Bratt, S. E. (2007). A framework for assessing the pedagogical utility of learning management systems (pp. 218–225). Presented at the World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education (ELEARN), Quebec, Canada. Bratt, S. E. (2009). Development of an instrument to assess pedagogical utility in e-Learning systems. (Unpublished doctoral dissertation) Simon Fraser University, Burnaby, Canada. Brel, J. (1977). Vieillir. Les Marquises. Broadbent, D. E. (1958). Perception and communication. New York, NY: Oxford University Press. Brooks JR, F. P. (1995). The mythical man-month (anniversary ed.). Boston, MA: Addison-Wesley Longman Publishing Co., Inc. Brooks, R. A. (1991) Intelligence without representation. Artificial Intelligence 47(1-3), 139–159. Brown, Stuart, & Vaughan, C. (2009). Play: How it shapes the brain, opens the imagination, and invigorates the soul. New York, NY: Penguin Books. Bruner, J. S. (1966). Toward a theory of instruction (Vol. 59). Cambridge, MA: Harvard University Press. Bruning, R. H., Schraw, G. J., & Norby, M. M. (2010). Cognitive psychology and instruction (5th Edition). Upper Saddle River, NJ: Prentice Hall. Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013). Power failure: why small sample size undermines the reliability of neuroscience. Nature reviews: Neuroscience, 14(365), 1–12. doi:10.1038/nrn3475 Caillies, S., & Denhière, G. (1999). The intermediate effect: Interaction between prior knowledge and text structure. In H. van Oostendorp & S. Goldman (Eds.), The construction of mental representation during reading (pp. 151–168). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Campbell, G. (2008, December 9). Interview with Seth Grant. Brain Science Podcast, 51. Podcast retrieved from http://brainsciencepodcast.com Campbell, G. (2012, May 10). Exploring Glial Cells with R. Douglas Fields. Brain Science Podcast, 69. Podcast retrieved from http://brainsciencepodcast.com Campbell, G. (2010, February 10). Memory: Challenging current theories with Randy Gallistel. Brain Science Podcast, 66. Podcast retrieved from http://brainsciencepodcast.com Campbell, G. (2013, November 22). Connectome Update with Olaf Sporn. Brain Science Podcast, 103 Podcast retrieved from http://brainsciencepodcast.com
338 Carpenter, S. K. (2009). Cue strength as a moderator of the testing effect: The benefits of elaborative retrieval. Journal of experimental psychology. Learning, memory, and cognition, 35(6), 1563– 1569. doi:10.1037/a0017021 Carpenter, S. K. (2012). Testing enhances the transfer of learning. Current directions in psychological science, 21(5), 279–283. doi:10.1177/0963721412452728 Carpenter, S. K., & DeLosh, E. L. (2006). Impoverished cue support enhances subsequent retention: Support for the elaborative retrieval explanation of the testing effect. Memory & Cognition, 34(2), 268–276. Carr, N. (2011). The shallows: What the Internet is doing to our brains (Kindle Edi.). New York, NY: W. W. Norton & Company. Cavallini, E., Pagnin, A., & Vecchi, T. (2003). Aging and everyday memory: The beneficial effect of memory training. Archives of Gerontology and Geriatrics, 37(3), 241–257. doi:10.1016/S01674943(03)00063-3 Cepeda, N. J., Coburn, N., Rohrer, D., Wixted, J. T., Mozer, M. C., & Pashler, H. (2009). Optimizing distributed practice: Theoretical analysis and practical implications. Experimental Psychology. doi:10.1027/1618-3169.56.4.236 Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132(3), 354–380. doi:10.1037/0033-2909.132.3.354 Cepeda, N. J., Vul, E., Rohrer, D., Wixted, J. T., & Pashler, H. (2008). Spacing effects in learning: A temporal ridgeline of optimal retention. Psychological Science, 19(11), 1095–1102. doi:10.1111/j.14679280.2008.02209.x Chaffin, R. (2011). Thinking about performance: Memory, attention, and practice (pp. 1–11). Paper presented at the International Symposium on Performance Science. Chaffin, R., Logan, T. R., & Begosh, K. T. (2008). Performing from memory. In S. Hallam, I. C. Thaut, & M. Thaut (Eds.), The Oxford handbook of music psychology (pp. 352–363). Oxford, England: Oxford University Press. Changeux, J. P & McGinn, C. (2013). Neuroscience & Philosophy: An Exchange. The New York Review of Books. Retrieved from http://www.nybooks.com/articles/archives/2013/aug/15/neurosciencephilosophy-exchange/ Charette, R. N. (2005, September 1). Why software fails. IEEE Spectrum. Retrieved from http://spectrum.ieee.org/computing/software/why-software-fails/0 Charness, N., Tuffiash, M., Krampe, R., Reingold, E., & Vasyukova, E. (2005). The role of deliberate practice in chess expertise. Applied Cognitive Psychology, 19(2), 151–165. doi:10.1002/acp.1106 Chatterji, D., Massengill, B., Oslin, J., Carver, J. C., & Kraft, N. A. (2011). Measuring the efficacy of code clone information: An empirical study. Paper presented at the Empirical Software Engineering and Measurement Conference (ESEM, IEEE). Chiew, K. S., & Braver, T. S. (2011). Positive affect versus reward: Emotional and motivational influences on cognitive control. Frontiers in Psychology, 2, 1–10. doi:10.3389/fpsyg.2011.00279 Chomistek, A. K., Manson, J. E., Stefanick, M. L. Lu, B., & Sands-Lincoln, M. and Going, S. B. … & others (2013). The relationship of sedentary behavior and physical activity to incident cardiovascular disease: Results from the Women’s Health Initiative. Journal of the American College
339 of Cardiology, 61(23), 2346–2354. Retrieved from http://dx.doi.org/10.1016/j.jacc.2013.03.031 Christian, B. (2012, April 25). The A/B Test: Inside the technology that’s changing the rules of business. Wired. Retrieved from http://www.wired.com/business/2012/04/ff_abtesting/ Chung, T. M., & Nation, P. (2003). Technical vocabulary in specialised texts. Reading in a Foreign Language 15(2), 103–116. Churchill, W. S. (1949). Great contemporaries. London, England: Odhams Press Limited. Clark, A. (1989). Microcognition: Philosophy, cognitive science, and parallel distributed processing. Cambridge, MA: MIT Press. Clark, A. (2013). Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behavioral and Brain Sciences, 36(03), 181–204. Clark, A. (2013, 2nd ed.). Mindware. New York, NY: Oxford University Press. Clark, A., & Chalmers, D. (1998). The extended mind. Analysis, 58(1), 7–19. Cockburn, A. (2001). Writing effective use cases. Boston, MA: Addison-Wesley. Cognitive science. (n.d.). In Wikipedia. Retrieved from http://en.wikipedia.org/ on November 27, 2012. Conley, C. (2007). Peak: How Great companies get their mojo from Maslow San Francisco, CA: JosseyBass. Conlin, L., Gupta, A., & Hammer, D. (2010). Where to find the mind: Identifying the scale of cognitive dynamics. In K. Gomez, L. Lyons, & J. Radinsky (Eds.), Learning in the Disciplines: Proceedings of the 9th International Conference of the Learning Sciences (Vol. 1, pp. 277-284). Chicago, IL: International Society of the Learning Sciences. Cooke, N. J., Atlas, R. S., Lane, D. M., & Berger, R. C. (1993). Role of high-level knowledge in memory for chess positions. American Journal of Psychology, 106(3), 321-351. Cooper, R. P. (2007). The role of falsification in the development of cognitive architectures: Insights from a Lakatosian analysis. Cognitive Science, 31, 509–533. Cooper, R. P., & Shallice, T. (2000). Contention scheduling and the control of control of routine activities. Cognitive Neuropsychology, 17(4), 297–338. doi:10.1080/026432900380427 Cooper, R. P., & Shallice, T. (2006). Hierarchical schemas and goals in the control of sequential behavior. Psychological Review, 113(4), 887–931. doi:10.1037/0033-295X.113.4.887 Copi, I. M., & Cohen, C. (2005). Introduction to Logic (12 ed.). Upper Saddle River, NJ: Pearson. Covey, S. R. (2004). The 7 habits of highly effective people. New York, NY: Free Press. Cox, M. T. (2005). Metacognition in computation: A selected research review. Artificial Intelligence, 169(2), 104–141. Coyle, D. (2009). The talent code: Greatness isn’t born. it’s grown. Here’s how. New York, NY: Bantam. Craik, F. I. M., & Salthouse, T. A. (2008). The handbook of aging and cognition. New York, NY: Psychology Press. Craik, F. I., & McDowd, J. M. (1987). Age differences in recall and recognition. Journal of Experimental Psychology, 13(3), 474–479. Critchley, S. (2011). The book of dead philosophers. London, England: Granta Books. Csikszentmihalyi, M. (1975). Play and intrinsic rewards. Journal of humanistic psychology. 15(3), 41-63.
340 Csikszentmihalyi, M. (1999). If we are so rich, why aren’t we happy? The American Psychologist, 54(10), 821–827. doi:10.1037//0003-066X.54.10.821 Cullen, B. (2010, November 25). Nine tips for more effective study. Retrieved from http://www.standinginspirit.com/nine-tips-for-more-effective-study/ Damasio, A. (1994). Descartes’ error: Emotion, reason and the human brain. New York, NY: Avon Books. Darwin, C. (1871). The Descent of Man, and Selection in Relation to Sex. Davenport, T. (2013, February 18). Jonathan Ive on how Apple names its products. Retrieved from http://tomdavenport.co.uk/post/43400664026/jonathan-ive-on-on-how-apple-names-its-products Davenport, T. H. (2005). Thinking for a living: How to get better performances and results from knowledge workers. Boston, MA: Harvard Business Press. David, P. (2013). Wedding the Gottman and Johnson Approaches into an Integrated Model of Couple Therapy. In Pair Bonding & Repair: Essays on Intimacy & Couple Therapy (Therapeutic models). Retrieved from http://pauldavidphd.com/wp-content/uploads/Davids-Integrated-CoupleTherapy-Model.pdf Dawkins, R. (1976). The selfish gene. Oxford, England: Oxford University Press. Dawkins, R. (1986). The blind watchmaker. New York, NY: W. W. Norton & Company. Delaney, P. F., Verkoeijen, P. P. J. L., & Spirgel, A. (2010). Spacing and testing effects: A deeply critical, lengthy, and at times discursive review of the literature. The Psychology of Learning and Motivation 53 , 63–148. doi:10.1016/S0079-7421(10)53003-2 Dennett, D. (1978). Why you can’t make a machine that feels pain. Synthese, 38(3), 415–456. Dennett, D. C. (1987). The intentional stance. Cambridge, MA: The MIT Press. Dennett, D. C. (1991). Consciousness explained. Boston, MA: Back Bay Books. Dias, M., & Harris, P. L. (1988). The effect of make-believe play on deductive reasoning. British Journal Of Developmental Psychology, 6, 207–221. DiLullo, C., McGee, P., & Kriebel, R. M. (2011). Demystifying the Millennial student: A reassessment in measures of character and engagement in professional education. Anatomical Sciences Education, 4(4), 214–226. doi:10.1002/ase.240 Digdon, N. (2015, July). Use of patient files to identify Watson and Rayner’s Little Albert. Paper presented at The European Society of History of Human Sciences Anger, France. DiSessa, A. A. (2001). Changing minds. Cambridge, MA: The MIT Press. Dixon, T. (2012). “Emotion”: The history of a keyword in crisis. Emotion Review, 4(4), 338–344. doi:10.1177/1754073912445814 Doctorow, C. (2009, June 22). Some Kindle books have secret caps on the number of times you can download them. Retrieved from http://boingboing.net/2009/06/22/some-kindle-books-ha.html Doctorow, C. (2012, April 26). Is publishing looking at a DRM-free future? CBC Podcast. Podcast retrieved from http://www.cbc.ca/books/2012/04/is-publishing-looking-at-a-drm-free-future.html Donald, M. (2001). A mind so rare: The evolution of human consciousness. New York, NY: W. W. Norton & Company. Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14(1), 4–58.
341 doi:10.1177/1529100612453266 Edmondson, A. C., Roberto, M. A., Bohmer, R. M., Ferlins, E. M., & Feldman, L. R. (2004). The recovery window: Organizational learning following ambiguous threats in high-risk organizations. Division of Research, Harvard Business School. Harvard Business School Working Paper. Egan, K. (1997). The educated mind: How cognitive tools shape our understanding. Chicago, IL: University of Chicago Press. Egan, K. (2010). Learning in depth: A simple innovation that can transform schooling. Chicago, IL: University of Chicago Press. Egan, K. & Madej, K. (2009). Learning in depth: students as experts. Education Canada, 49(2), 18–20. Einstein, G. O., & McDaniel, M. A. (2004). Memory fitness: A guide for successful aging. New Haven, CT: Yale University Press. Elgort, I. (2011). Deliberate learning and vocabulary acquisition in a second language. Language Learning, 61(2), 367–413. doi:10.1111/j.1467-9922.2010.00613.x Elman, J. L., Bates, Johnson, M. H., Karmiloff-Smith, A., Parisi, D. & Plunkett, K. (Eds.). (1996). Rethinking innateness: A connectionist perspective on development. Cambridge MA: MIT Press. Emerson, R. W. (1904). Essays and Representative Men (Vol. 1). London, England: George Bell and Sons. Engelbart, D., & English, W. K. (1968). A research center for augmenting human intellect. Proceedings of the AFIPS Fall Joint Computer Conference, 33, 395–410. doi:10.1145/1476589.1476645 Engelbart, D., & Ruilifson, J. (1999). Bootstrapping our collective intelligence. ACM Computing Surveys, 31(4es). doi:10.1145/345966.346040 Erber, R., & Erber, M. W. (2001). Mood and processing: A view from a self-regulation perspective. In L. L. Martin & G. L. Clore, Theories of mood and cognition: A users guidebook (pp. 63–84). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Ergotron. (2013, April 9). Ergotron 33-329-085 33-329-057 Neo-Flex Widescreen Monitor Stand. Retrieved from http://www.ergotron.com/Products/tabid/65/PRDID/242/language/en-US/Default.aspx Ericsson, K. A. (1998). The scientific study of expert levels of performance: General implications for optimal learning and creativity. High Ability Studies, 9(1), 75–100. Ericsson, K. A. (2006). The influence of experience and deliberate practice on the development of superior expert performance. In K. A. Ericsson, N. Charness, P. Feltovich & R. R. Hoffman (Eds.), Cambridge handbook of expertise and expert performance (pp. 685–706). Cambridge, UK: Cambridge University Press. Ericsson, K. A., & Charness, N. (1995). Abilities: evidence for talent or characteristics acquired through engagement in relevant activities. American Psychologist, 50(9), 803-804. Ericsson, K. A., Charness, N., Feltovich, P. J., & Hoffman, R. R. (2006). The Cambridge handbook of expertise and expert performance. Cambridge, England: Cambridge University Press. Ericsson, K. A., & Kintsch, W. (1995). Long-term working memory. Psychological Review, 102(2), 211–245. Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review. 100(3), 363–406. doi:10.1037//0033295X.100.3.363
342 Ericsson, K. A., Patel, V., & Kintsch, W. (2000). How experts’ adaptations to representative task demands account for the expertise effect in memory recall: Comment on Vicente and Wang (1998). Psychological Review, 107(3), 578–592. doi:10.1037//0033-295X.107.3.578 Ericsson, K. A., & Staszewski, J. J. (1989). Skilled memory and expertise: Mechanisms of exceptional performance. In D. Klahr & K. Kotovsky (Eds.), Complex information processing the impact of Herbert A. Simon(pp. 235–267). Hillsdale, NJ: Erlbaum Evans, J. A. (2008). Electronic Publication and the Narrowing of Science and Scholarship. Science, 321 (5887), 395–399. doi:10.1126/science.ll57215 Fadde, P. J., & Klein, G. A. (2010). Deliberate performance: Accelerating expertise in natural settings. Performance Improvement, 49(9), 5–14. doi:10.1002/pfi Fayol, M., & Rouet, J.-F. (2008). Memory processes in text and multimedia comprehension: Some reflections and perspectives. In R. Lowe, J. R. Rouet & W Schnotz (Eds.), Understanding multimedia documents (pp. 267–280). Boston, MA: Springer US. doi:10.1007/978-0-387-733371_14 Fields, R. D. (2008a). White matter in learning, cognition and psychiatric disorders. Trends in neurosciences, 31(7), 361–370. Fields, R. D. (2008b). White matter matters. Scientific American, 298(3), 54–61. Finn, B., & Metcalfe, J. (2008). Judgments of learning are influenced by memory for past test. Journal of Memory and Language, 58(1), 19–34. doi:10.1016/j.jml.2007.03.006 http://pss.sagepub.com/content/22/6/781 Fischler, I. (1977). Associative facilitation without expectancy in a lexical decision task. Journal of Experimental Psychology: Human perception and performance, 3(1), 18–26. Fisher, H. (2005). Why we love: The nature and chemistry of romantic love. (iBooks version). Available from https://itunes.apple.com/ca/book/why-we-love/id569886738 Fitch, W. T. (2012). An instinct to learn. In J. Brockman This will make you smarter: New scientific concepts to improve your thinking (pp. 154-156). New York, NY: Harper Perennial. Fitts, P. M. (1964). Perceptual-motor skill learning. In A. W. Melton (Ed.), Categories of Human Learning (pp. 243–285). New York, NY: Academic Press. Fodor, J. A. (1975). The language of thought. Cambridge, MA: Harvard University Press. Fodor, J. A. (1983). The modularity of mind: An essay on faculty psychology. MIT press. Foer, J. (2011). Moonwalking with Einstein: The art and science of remembering everything. New York, NY: The Penguin Press. Franklin, B. (1758) The way to wealth. Retrieved from http://www.swarthmore.edu/SocSci/bdorsey1/41docs/52-fra.html Franklin, B. (1914). Poor Richard’s Almanack. Waterloo, IA: U. S. C. Publishing. Retrieved from http://ia600200.us.archive.org/. Selections from material first published 1732-1758. Freud, S. (1914). The psychopathology of everyday life, (A. A. Brill, Trans.). London, England: T. Fisher Unwin. (Original work published 1901). Frijda, N. H. (1986). The emotions. Cambridge, England: Cambridge University Press. Frijda, N. H. (2007). The laws of emotion. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Frijda, N. H. (2009). Emotion experience and its varieties. Emotion Review, 1(3), 264–271. doi:10.1177/1754073909103595
343 Frijda, N. H., & Parrott, W. G. (2011). Basic emotions or Ur-emotions? Emotion Review, 3(4), 406–415. doi:10.1177/1754073911410742 Fritz, C. O., Morris, P. E., Acton, M., Voelkel, A. R., & Etkind, R. (2007). Comparing and combining retrieval practice and the keyword mnemonic for foreign vocabulary learning. Applied Cognitive Psychology, 21(4), 499-526. Fromm, E. (2013/1956). The art of loving. Available from http://www.openroadmedia.com/the-artof-loving Funt, B. V. (1980). Problem-solving with diagrammatic representations. Artificial Intelligence, 13, 201–230. Gallimore, R., & Tharp, R. (2004). What a coach can teach a teacher, 1975–2004: Reflections and reanalysis of John Wooden’s teaching practices. Sport Psychologist, 18, 119–137. Gardner, H. E. (2010, June 3). Help! Paul McCartney admits to forgetting his own lyrics. Mail Online. Retrieved from http://www.dailymail.co.uk/tvshowbiz/article-1283542/Paul-McCartneyadmits-forgetting-lyrics.html Gardner, D. (1998). Extraordinary minds: Portraits of 4 exceptional individuals and an examination of our own extraordinariness. New York, NY: Basic Books. Gardner, H. (1999). The disciplined mind. New York: Simon & Schuster. Gawronski, B., Hofmann, W., & Wilbur, C. J. (2006). Are “implicit” attitudes unconscious? Consciousness and Cognition, 15(3), 485–499. doi:10.1016/j.concog.2005.11.007 Georgeff, M. P., & Lansky, A. L. (1986). Procedural knowledge. Proceedings of the IEEE, 74(10), 1383– 1398. Georgeff, M. P., & Lansky, A. L. (1987, July). Reactive reasoning and planning. In Proceedings of AAAI (Vol. 87, pp. 677-682). Gerber, M. E. (1995). The E-myth revisited: Why most small businesses still don’t work and what you can do about yours. New York, N.Y.: HarperBusiness. Gibson, J. J. (1979). The ecological approach to visual perception. Boston, MA: Houghton Mifflin Company. Gick, M. L., & Holyoak, K. J. (1980). Analogical problem solving. Cognitive Psychology, 12, 306–355. Gick, M. L., & Holyoak, K. J. (1983). Schema induction and analogical transfer. Cognitive Psychology, 15(1), 1–38. doi:10.1016/0010-0285(83)90002-6 Gigerenzer, G., Todd, P. M., & The ABC Research Group. (1999). Simple heuristics that make us smart. New York, NY: Oxford University Press, USA. Gladwell, M. (2008). Outliers: The story of success. New York, NY: Little, Brown and Company. Gladwell, M. (2013). David and Goliath. (iBook version). Available from https://itunes.apple.com/us/book/david-and-goliath/id599651578?mt=11 Gobet, F. (2000). Some shortcomings of long‐term working memory. British Journal of Psychology, 91(4), 551–570. Gobet, F. (2001). Is experts’ knowledge modular? In J. D. Moore, & K Stenning (Eds.), Proceedings of the Twenty-Third Annual Conference of the Cognitive Science Society (pp. 336–242). Mahwah, NJ: Erlbaum. Gobet, F., & Charness, N. (2006). Expertise in chess. In K. A. Ericsson, N. Charness, P. J. Feltovich, & R. R. Hoffman (Eds.), The Cambridge handbook of expertise and expert performance (pp. 523–
344 538). Cambridge, England: Cambridge University Press. Gobet, F., & Chassy, P. (2008). Season of birth and chess expertise. Journal of Biosocial Science, 40(2), 313. Gobet, F., & Simon, H. A. (1996a). Recall of rapidly presented random chess positions is a function of skill. Psychonomic Bulletin & Review, 3(2), 159–163. Gobet, F., & Simon, H. A. (1996b). Templates in chess memory: a mechanism for recalling several boards. Cognitive Psychology, 31(1), 1–40. doi:10.1006/cogp.1996.0011 Gobet, F., Lane, P. C. R., Croker, S., Cheng, P. C. H., Jones, G., Oliver, I., & Pine, J. M. (2001). Chunking mechanisms in human learning. Trends In Cognitive Sciences, 5(6), 236–243. Goldacre, B. (2014). I think you’ll find it’s a bit more complicated than that. London: Fourth Estate. Goldberger, P. (2013, October 10). Apple’s Jony Ive and designer Marc Newson on their shared “level of perfection”: “It is actually very sick”. Retrieved from http://www.vanityfair.com/business/2013/11/jony-ive-marc-newson-design-auction Goleman, D. (2006). Emotional Intelligence. New York, NY: Bantam. Good, I. J. (1971). Comments on Herman Rubin’s “Occam’s razor”. In V. P. Godambe & D. A. Sprott (Eds.), Foundations of statistical inference. Toronto, Canada: Holt, Rinehart, and Winston. Goodwin, P. (2010). Why hindsight can damage foresight. Foresight: The International Journal of Applied Forecasting, 17, 5–7. Gopnik, A. (1998). Explanation as orgasm. Minds and Machines, 8(1), 101–118. Gottman, J. M. (2002). A multidimensional approach to couples. In F. W. Kaslow & T. Patterson (Eds.), Comprehensive handbook of psychotherapy: Cognitive behavioral approaches (Vol. 2, pp. 355–372). New York, NY: John Wiley & Sons. Gottman, J. M., & DeClaire, J. (2001). The relationship cure: A five-step guide for building better connections with family, friends, and lovers. New York, N.Y.: Crown Publishers. Gottman, J. M., & Driver, J. L. (2005). Dysfunctional marital conflict and everyday marital interaction. Journal of Divorce & Remarriage, 43(3-4), 63–77. doi:10.1300/J087v43n03_04 Gottman, J. M., & Levenson, R. W. (2002). A two-factor model for predicting when a couple will divorce: exploratory analyses using 14-year longitudinal data. Family Process, 41(1), 83–96. Gottman, J. M., & Silver, N. (1999). The Seven Principles for Making Marriage Work. New York, NY: Three Rivers Press. Gottman, J. M., Murray, J. D., Swanson, C. C., Tyson, R., & Swanson, K. R. (2002). The Mathematics of marriage: dynamic nonlinear models. Cambridge, MA: MIT Press. Goyvaerts, J., & Levithan, S. (2012). Regular Expressions Cookbook. Sebastopol, CA: O’Reilly Media. Graesser, A. C., Jackson, G. T., Mathews, E. C., Mitchell, H. H., Olney, A., Ventura, M., … the Tutoring Research Group. (2003). Why/AutoTutor: A test of learning gains from a physics tutor with natural language dialog autotutor and Why/AutoTutor. In R. Alterman & D. Hirsh (Eds.), Proceedings of the Twenty-Fifth Annual Conference of the Cognitive Science Society (Vol. 1, pp. 1–6). Mahwah, NJ: Laurence Earlbaum Associates. Grant, S. (2003). Systems biology in neuroscience: Bridging genes to cognition. Current Opinion in Neurobiology, 13(5), 577–582. doi:10.1016/j.conb.2003.09.016 Grant, S. G. N. (2007). Toward a molecular catalogue of synapses. Brain Research Reviews, 55(2), 445–449. doi:10.1016/j.brainresrev.2007.05.003
345 Grigorenko, E. L., & Sternberg, R. J. (1998). Dynamic testing. Psychological Bulletin, 124(1), 75–111. doi:10.1037/0033-2909.124.1.75 Guadagnoli, M. A. (2009). Practice to learn, play to win. Cornwall, UK: Ecademy Press. Guadagnoli, M. A., & Lee, T. D. (2004). Challenge point: A framework for conceptualizing the effects of various practice conditions in motor learning. Journal of Motor Behavior, 36(2), 212–224. Hardof-Jaffe, S., & Nachmias, R. (2013, June). Students Personal Information Management. In World Conference on Educational Multimedia, Hypermedia and Telecommunications, 2013(1), pp. 820828). Hadwin, A. F. & Tevaarwerk, K. L. (2011, unpublished). Are we teaching university students to strategically self-regulate their learning? An analysis of contemporary study skills textbooks. (35 pp.). Hambrick, D. Z., Oswald, F. L., Altmann, E. M., Meinz, E. J., Gobet, F., & Campitelli, G. (2014). Deliberate practice: Is that all it takes to become an expert?. Intelligence, 45, 34-45. Hammer, D., Elby, A., & Scherr, R. E. (2005). Resources, framing, and transfer. In J. P. Mestre (Ed.), Transfer of learning from a modern multidisciplinary perspective (pp. 1–26). Greenwich, CT: Information Age Publishing. Harris, R. (2007). The happiness trap. Auckland, New Zealand: Exisle Publishing. Haskell, R. E. (2000). Transfer of learning: Cognition and instruction. San Francisco, CA: Academic Press. Hatano, G. (1998). Might we adopt the learning-related account instead of the talent account? Behavioral and Brain Sciences, 21(03), 416–417. doi:10.1017/S0140525X98331234 Hawes, N. (2011). A survey of motivation frameworks for intelligent systems. Artificial Intelligence, 175(5-6), 1020–1036. Hawkins, J., & Blakeslee, S. (2005). On intelligence. New York, NY: St. Martin’s Griffin. Hayes, J. R. (2002a). Three problems in teaching general skills. In D. J. Levitin (Ed.), Foundations of cognitive psychology: Core readings (pp. 551–564). Cambridge, MA: MIT Press. Hayes, S. C. (2002b). Buddhism and acceptance and commitment therapy. Cognitive and Behavioral Practice, 9(1), 58–65. Hayes, S. C., Strosahl, K. D., & Wilson, K. G. (2011). Acceptance and commitment therapy: The process and practice of mindful change. New York, NY: Guilford Press. Hayes-Roth, B. (1985). A blackboard architecture for control. Artificial Intelligence, 26(3), 251–321. Hayes-Roth, B. (1990). Architectural foundations for real-time performance in intelligent agents. Journal of Real-Time Systems, 2, 99–125. Hayes-Roth, B., & Perrault, F. H.-R. (1979). A cognitive model of planning. Cognitive Science, 3(4), 275–310. doi:10.1207/s15516709cog0304\_1 Heathcote, A., Brown, S., & Mewhort, D. J. K. (2000). The power law repealed: The case for an exponential law of practice. Psychonomic Bulletin & Review, 7(2), 185–207. Heckhausen, H., & Beckmann, J. (1990). Intentional action and action slips. Psychological Review, 97(1), 36-48. Heng, C. S., Tan, B. C., & Wei, K. K. (2003). De-escalation of commitment in software projects: Who matters? what matters? Information & Management, 41(1), 99-110. Herrin, T. C. (2009). The analysis of an integrated model of therapy using structural and Gottman
346 method approaches: A case study. (Unpublished master’s thesis). Utah University, Salt Lake City, UT. Retrieved from http://digitalcommons.usu.edu/etd/368 Hesse, H. (1965). Demian. (M. Roloff & M. Lebeck, Trans.). New York, NY: Harper & Row. (Original work published 1919) Higbee, K. L. (1988). Your memory—How it works and how to improve it. New York, N.Y.: Marlowe & Company. Hill, C. R. (2007). Relationship Rhetoric: Representations of intimacy in contemporary self-help literature. (Unpublished doctoral dissertation.) Simon Fraser University, Burnaby, Canada. Hoffman, R. R. (2005). The psychology of intelligence analysis revisited: An update from developments in cognitive science post-1980. Pensacola, FL: Florida Institute for Human and Machine Cognition. http://www.ihmc.us/groups/rhoffman/ Hoffman, R. R., & Militello, L. G. (2012). Perspectives on Cognitive Task Analysis. New York, NY: Psychology Press. Hoffman, R. R., Feltovich, P. J., Fiore, S. M., Klein, G., & Ziebell, D. (2009). Accelerated learning (?). Intelligent Systems, IEEE, 24(2), 18–22. Hoffman, R. R., Marx, M., Amin, R., & McDermott, P. L. (2010). Measurement for evaluating the learnability and resilience of methods of cognitive work. Theoretical Issues in Ergonomics Science, 11(6), 561–575. doi:10.1080/14639220903386757 Horvitz, E. J., Breese, J. S., & Henrion, M. (1988). Decision theory in expert systems and artificial intelligence. International Journal of Approximate Reasoning, 2(3), 247–302. Howe, M. J., Davidson, J. W., & Sloboda, J. A. (1998). Innate talents: Reality or myth? Behavioral and Brain Sciences, 21(3), 399–407; Discussion 407–442. Howes, M. B. (2007). Human memory. Thousand Oaks, CA: SAGE Publications. Huang, J. Y., & Bargh, J. A. (in press). The selfish goal: Autonomously operating motivational structures as the proximate cause of human judgment and behavior. Behavioral and Brain Sciences. http://journals.cambridge.org/BBSJournal/Call/Huang_preprint Hurley, M. M., Dennett, D. C., & Adams, R. B. (2011). Inside jokes: Using humor to reverse-engineer the mind. Cambridge, MA: MIT Press. Ingle, S., & Murray, S. (2002, March 28). My goal, your goal. The Guardian. Retrieved from http://www.theguardian.com/football/2002/mar/28/theknowledge.sport Intensional definition. (n.d.). In Wikipedia. . Retrieved from http://en.wikipedia.org/ on May 30, 2011. Isaacson, W. (2011). Steve Jobs. New York, NY: Simon & Schuster. Izard, C. E. (2010). The many meanings/aspects of emotion: definitions, functions, activation, and regulation. Emotion Review, 2(4), 363–370. doi:10.1177/1754073910374661 Jabr, F. (2013, November 27). Why the brain prefers paper. Scientific American, 309, 48–53. Retrieved from http://www.nature.com/. Jacobs, A. J. (2010). My life as an experiment. New York, NY: Simon & Schuster. Jagger, M., & Richards, K. (1966). Mother’s Little Helper. On Aftermath [CD]. London, England: London Records. Jenkins, R. (2002). Churchill: A biography. New York, NY: Plume. Jones, C. (2012). The new shape of the student. In R. Huang, Kinshuk & J. M. Spector (Eds.), Reshaping Learning (pp. 91–112). New York, NY: Springer. doi:10.1007/978-3-642-32301-0_4
347 Jones, D. (2005, December 28). Brain science is key to learning, specialist says. The Globe and Mail, p. S28. Toronto. Johnson, J. (2007). GUI Bloopers 2.0: Common user interface design don’ts and dos. San Francisco: Morgan Kaufmann. Junker, K. W. (2008). What is reading in the practices of law? The Journal of Law Society of America, 9(1), 111–162 Kagan, J. (1972). Motives and development. Journal of Personality and Social Psychology, 22(1), 51– 66. Kahneman, D. (2011). Thinking, fast and slow. New York, NY: Farrar, Straus and Giroux. Kant, I. (1917) Answering the question: What is enlightenment? (M. C. Smith, Trans.) (Original work published 1784.) Retrieved from http://www.columbia.edu, September 23, 2014. Kant, I. (2011). Critique of pure reason. (J. M. D. Meiklejohn, Trans.). Seattle, WA: Pacific Publishing Studio. (Original work published 1781.) Karmiloff-Smith, A. (1995). Beyond modularity: A developmental perspective on cognitive science. Cambridge, MA: MIT Press. Karmiloff-Smith, A. (2012). From constructivism to neuroconstructivism: The activity-dependent structuring of the human brain. In E. Marti & C. Rodriguez (Eds.), After Piaget (pp. 1–14). Piscataway, NJ: Transaction Publishers. Karmiloff-Smith, A., & Clark, A. (1993). What’s special about the development of the human mind/brain? Mind & Language, 8, 569–581. Karmiloff-Smith, A., & Inhelder, B. (1975). If you want to get ahead, get a theory. Cognition, 3(3), 195–212. Karpicke, J. D. (2009). Metacognitive control and strategy selection: deciding to practice retrieval during learning. Journal of Experimental Psychology: General, 138(4), 469–486. doi:10.1037/a0017341 Karpicke, J. D., & Bauernschmidt, A. (2011). Spaced retrieval: absolute spacing enhances learning regardless of relative spacing. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(5), 1250–1257. doi:10.1037/a0023436 Karpicke, J. D., & Blunt, J. R. (2011a). Response to Comment on “Retrieval practice produces more learning than elaborative studying with concept mapping.” Science, 334(6055), 772–775. doi:10.1126/science.1204035 Karpicke, J. D., & Blunt, J. R. (2011b). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 331, 772–775. doi:10.1126/science.1199327 Karpicke, J. D., & Roediger, H. L. (2007). Expanding retrieval practice promotes short-term retention, but equally spaced retrieval enhances long-term retention. Journal of Experimental Psychology: Learning, Memory, and Cognition, 33(4), 704–719. doi:10.1037/0278-7393.33.4.704 Karpicke, J. D., & Roediger, H. L. (2008). The critical importance of retrieval for learning. Science, 319, 966–968. doi:10.1126/science.1152408 Keegan, P. (2007, June 21). How David Allen mastered getting things done. Business 2.0 Magazine. Retrieved April 9, 2013, from http://money.cnn.com. Kentgen, L., Allen, R., Kose, G., & Fong, R. (2011). The effects of rerepresentation on future performance. British Journal of Developmental Psychology, 16(4), 505–517. Kern, R., Körner, C., & Strohmaier, M. (2010). Exploring the influence of tagging motivation on
348 tagging behavior. In M. Lalmas, J. Jose, A. Rauber, R. Sebastiani, I. Frommholz (Eds.). Research and Advanced Technology for Digital Libraries, 461–465. Berlin, Germany: Springer BerlinHeidelberg. Keshav, S. (2013, August 2). How to read a paper. Retrieved November 11, 2013, from http://blizzard/cs/uwaterloo.ca. Kintsch, W. (1998). Comprehension. Cambridge, UK: Cambridge University Press. Kipling, R. (1895/1910). If. Retrieved from http://unix.cc.wmich.edu . Kirby, J. R. & Lawson, M. J. (Eds.). (2012). Enhancing the quality of learning: Dispositions, instruction, and learning processes. Cambridge, MA: Cambridge University Press. Klahr, D., Chase, W. G., & Lovelace, E. A. (1983). Structure and process in alphabetic retrieval. Journal of experimental psychology: Learning, Memory, and Cognition, 9(3), 462–477. Koriat, A., & Bjork, R. A. (2005). Illusions of competence in monitoring one’s knowledge during study. Journal of experimental psychology: Learning, Memory, and Cognition, 31(2), 187–194. doi:10.1037/0278-7393.31.2.187 Kruglanski, A. W., Bélanger, J. J., Chen, X., Köpetz, C., Pierro, A., & Mannetti, L. (2012). The energetics of motivated cognition: A force-field analysis. Psychological Review, 119(1), 1–20. doi:10.1037/a0025488 Kljun, M., Mariani, J., & Dix, A. (2013). Transference of PIM research prototype concepts to the mainstream: successes or failures. Interacting with Computers,. doi:10.1093/iwc/iwt059 Kuhl, J. (1992). A theory of self‐regulation: Action versus state orientation, self‐discrimination, and some applications. Applied Psychology, 41(2), 97–129 Kuhn, T. S. (1962/1996). The structure of scientific revolutions. Chicago, IL: University of Chicago Press. Kwary, D. A. (2011). A hybrid method for determining technical vocabulary. System, 39(2), 175–185. doi:10.1016/j.system.2011.04.003 LaBossiere, M. C. (2013, November 25). E-reading & education. The Philosophers’ Magazine Blog. Retrieved November 26, 2013, from http://talkingphilosophy.com. Lachter, J., Forster, K. I., & Ruthruff, E. (2004). Forty-five years after Broadbent (1958): Still no identification without attention. Psychological Review, 111(4), 880–913. doi:10.1037/0033295X.111.4.880 Lakatos, I. (1980). The methodology of scientific research programmes: Philosophical papers. (Vol. 1). Cambridge, UK: Cambridge University Press. Lamontagne, C. (1973). A new experimental paradigm for the investigation of the secondary system of human visual motion perception. Perception, 2, 167–180. Lamontagne, C. (1976). Steps towards a computational theory of visual motion detection: Designing a working system. (Doctoral dissertation). University of Edinburgh, Edinburgh, UK. Lamontagne, C. (1987). Sensorymotor emergence: Proposing a computational “syntax.” In W. Callebaut & R. Pinxten (Eds.), Evolutionary epistemology: A multiparidigm program (pp. 283– 310). Boston, MA: Reidel Publishing. Latham, G. P., & Locke, E. A. (1991). Self-regulation through goal setting. Organizational Behavior and Human Decision Processes, 50(2), 212–247. doi:10.1016/0749-5978(91)90021-K Lather, P. (1999). To be of use: The work of reviewing. Review of Educational Research, 69(1), 2–7.
349 Laufer, B. (1981). A problem in vocabulary learning — Synophones. ELT Journal, 35(3), 294–300. doi:10.1093/elt/XXXV.3.294 Leahy, R. L. (2003). Cognitive therapy techniques: A practitioner’s guide. New York: The Guilford Press. Lee, D. N., & Lishman, J. R. (1975). Visual proprioceptive control of stance. Journal of Human Movement Studies, 1, 87–95. Lehmann, A. C., K. A. Ericsson, N. Charness, P. Feltovich. (1997). Research on expert performance and deliberate practice: implications for the education of amateur musicians and music students. Psychomusicology, 16, 40–58. Lennon, J., & McCartney, P. (1995). A hard day’s night. On Anthology 1 [Audio CD]. London, UK: Apple Records. (Original recording 1964.) Lepage, G. A. (Host), & Brisebois, M. (Director). (2012, November 18). Auteur à temps plein (Douglas Kennedy). [Television series episode segment]. In G.A. Lepage (Producer), Tout le monde en parle. Montréal, QB: Retrieved from http://ici.radio-canada.ca/. Lepage, G. A. (Host), & Brisebois, M. (Director). (2013, November 10). La face cachée de Louis-José Houde. [Television series episode segment] In G.A. Lepage (Producer), Tout le monde en parle. Montréal, QB: Radio-Canada. Retrieved from http://www.radio-canada.ca/. Leveen, S. (2005). The little guide to your well-read life. Delray Beach, FL: Levenger Press. Linehan, M. (1993). Cognitive-behavioral treatment of borderline personality disorder. New York, NY: Guilford Press. Little, J. L., Bjork, E. L., Bjork, R. A., & Angello, G. (2012). Multiple-choice tests exonerated, at least of some charges: Fostering test-induced learning and avoiding test-induced forgetting. Psychological Science, 23(11), 1337–1344. doi:10.1177/0956797612443370 Littlejohn, A., Beetham, H., & McGill, L. (2012). Learning at the digital frontier: A review of digital literacies in theory and practice. Journal of Computer Assisted Learning, 28(6), 547–556. doi:10.1111/j.1365-2729.2011.00474.x Livingstone, D. W. (2010). Lifelong learning in paid and unpaid work: Survey and case study findings. New York, NY: Routledge. Lodge, D. (2011). Thinks… [Kindle version]. Retrieved from Amazon.com. Lord, C. G., Lepper, M. R., & Preston, E. (1984). Considering the opposite: a corrective strategy for social judgment. Journal of Personality and Social Psychology, 47(6), 1231–1243. Mäantylä, T. (1986). Optimizing cue effectiveness: Recall of 500 and 600 incidentally learned words. Journal of Experimental Psychology: Learning, Memory, and Cognition, 12(1), 66–71. doi:10.1037/0278-7393.12.1.66 Macaulay, T. B. (1950). Critical and historical essays. London, UK: Longman, Brown, Green and Longmans. MacNamara, J. (1976). Stomachs assimilate and accommodate, don’t they? Canadian Psychological Review/Psychologie canadienne, 17(3), 167. MacNamara, J. (1978). Another unaccommodating look at Piaget. Canadian Psychological Review/Psychologie canadienne, 19(1), 78–81. Marano, H. E. (1997). Gottman and Gray: The two Johns. Psychology Today, 30(6), 28. Retrieved from http://www.u.arizona.edu/.
350 Marcus, G. (2012). Cognitive Humility. In J. Brockman, This will make you smarter: New scientific concepts to improve your thinking (pp. 39–40). New York, NY: Harper Perennial. Marr, D. (1982). Vision: a computational investigation into the human representation and processing of visual information. San Francisco, CA: W.H. Freeman. Martin, B. R., & Irvine, J. (1985). Evaluating the evaluators: A reply to our critics. Social Studies of Science, 15(3), 558–575. Martin, J., & Hiebert, B. A. (1985). Instructional counseling: A method for counselors. Pittsburgh, PA: University of Pittsburgh Press. Mastaglio, T. W., Jones, P., Bliss, J. P., & Newlon, E. (2007). An integrated theory for after action review. In Proceedings of The Interservice/Industry Training, Simulation & Education Conference (I/ITSEC), 1. Arlington, VA: National Training Systems Association. McBurney, S. (Director). (2013). Steve Jobs: Visionary entrepreneur [Motion picture]. (Available from Silicon Valley Historical Association, Menlo Park, CA.) McCarthy, J. (2008). The well-designed child. Artificial Intelligence, 172(18), 2003–2014. doi:10.1016/j.artint.2008.10.001 McCulloch, W. S. (1953). The past of a delusion. In W. S. McCulloch, Embodiments of Mind (1965). Cambridge, MA: MIT Press. McCulloch, W. S. (1965). A heterarchy of values determined by the topology of nervous nets. In W.S. McCulloch, Embodiments of Mind (1965). Cambridge, MA: MIT Press. McCulloch, W. S. (1965). Embodiments of mind. Cambridge, MA: MIT Press. McCulloch, W. S. & Pitts, W. (1943). A logical calculus of the ideas imminent in neural activity, Bulletin of Mathematical Biophysics, Vol. 5, 115–133. McCulloch, W. S., We, S., & Twain, M. (1964). What’s in the brain that ink may character? In W.S. McCulloch, Embodiments of Mind (1965). Cambridge, MA: MIT Press. McDaniel, M. A. (2007). Transfer: Rediscovering a central concept. In H. L. Roediger, Y. Dudai, & S. M. Fitzpatrick, Science of memory: Concepts. McDaniel, M. A., & Bugg, J. M. (2012). Memory training interventions: What has been forgotten? Journal of Applied Research in Memory and Cognition, 1(1), 58–60. doi:10.1016/j.jarmac.2011.11.002 McDaniel, M. A., Einstein, G., & Jacoby, L. (2008). New considerations in aging and memory: The glass may be half full. In F. Craik & T. Salthouse (Eds.), The handbook of aging and cognition (3rd Ed., pp. 311–372). New York, NY: Psychology Press. McDaniel, M. A., Roediger, H. L., & Mcdermott, K. B. (2007). Generalizing test-enhanced learning from the laboratory to the classroom. Psychonomic Bulletin & Review, 14(2), 200–206. McGovern, K. (2007). Emotion. In B. J. Baars & N. M. Cage (Eds.), Cognition, Brain, and Consciousness. Waltham, MA: Academic Press. McKinney, R. A. (2005). Reading like a lawyer: Time-saving strategies for reading law like an expert. Durham, NC: Carolina Academic Press. Metcalfe, J. (2011). Desirable difficulties and studying in the region of proximal learning. In S. Benjamin (Ed.), Successful remembering and successful forgetting: A festschrift in honor of Robert A. Bjork (pp. 259–276). London, UK: Psychology Press. Metcalfe, J., & Finn, B. (2008). Familiarity and retrieval processes in delayed judgments of learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34(5), 1084–1097.
351 doi:10.1037/a0012580 Miller, G. (2000). The Mating Mind: How sexual selection shaped the evolution of human nature. New York, NY: Random House. Minsky, M. L. (1986). The society of mind. New York, NY: Simon & Schuster. Minsky, M. L. (2006). The emotion machine: Commonsense thinking, artificial intelligence, and the future of the human mind. Hew York, NY: Simon and Schuster. Mogle, J. A., Lovett, B. J., Stawski, R. S., & Sliwinski, M. J. (2008). What’s so special about working memory? An examination of the relationships among working memory, secondary memory, and fluid intelligence. Psychological Science, 19(11), 1071–1077. doi:10.1111/j.1467-9280.2008.02202.x Moltz, H. (1965). Contemporary instinct theory and the fixed action pattern. Psychological Review, 72(1), 27. Moon, B. M., Hoffman, R. R., Eskridge, T. C., & Coffey, J. W. (2011). Skills in applied concept mapping. In B. M. Moon, R. R. Hoffman, J. D. Novak, A. J. Cañas (Eds.). Applied Concept Mapping: Capturing, Analyzing, and Organizing Knowledge. (pp. 23-46.) Boca Raton, FL: CRC Press. Moore, G. (1999). Crossing the chasm. New York, NY: HarperCollins. Morgan, N. (2009). The rule of threes. Retrieved from http://publicwords.com/. Morris, P. E., Fritz, C. O., Jackson, L., Nichol, E., & Roberts, E. (2005). Strategies for learning proper names: Expanding retrieval practice, meaning and imagery. Applied Cognitive Psychology, 19(6), 779–798. doi:10.1002/acp.1115 Mozer, M. C., Pashler, H., Cepeda, N., Lindsey, R., & Vul, E. (2009). Predicting the optimal spacing of study: A multiscale context model of memory. In Y. Bengio, D. Schuurmans, J.D. Lafferty, C. K. I. Williams, & A. Culotta (Eds.), Advances in neural information processing systems, 22 (pp. 1321–1329). La Jolla, CA: Neural Information Processing Systems Foundation. Mueller, P. A., & Oppenheimer, D. M. (2014). The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological Science, 25(6), 1159–1168. http://doi.org/10.1177/0956797614524581 Mulcahy-Ernt, P. I., & Caverly, D. C. (2009). Strategic study-reading. In R. F. Flippo & D. C. Caverly (Eds.), Handbook of college reading and study strategy research (pp. 177–198). Mahwah, NJ: Lawrence Earlbaum Associates. Mullet, K., & Sano, D. (1995). Designing visual interfaces: Communication oriented techniques. Englewood Cliffs NJ: SunSoft Press. Nadel, L. (2007). Consolidation: The demise of the fixed trace. In Y. Dudai, H. L. I. Roediger, E. Tulving, & S. M. Fitzpatrick (Eds.), Science of Memory: Concepts. New York: Oxford University Press. Nation, I. S. P. (2011). Research into practice: Vocabulary. Language Teaching, 44(04), 529-539. Naveh-Benjamin, M., & Old, S. R. (2008). Aging and Memory. In J. H. Byrne (Ed.), Learning and memory: A comprehensive reference (Vol. 2). Waltham, MA: Elsevier. Naveh-Benjamin, M., Craik, F. I. M., & Ben-Shaul, L. (2002). Age-related differences in cued recall: Effects of support at encoding and retrieval. Aging, Neuropsychology, and Cognition, 9(4), 276– 287. doi:10.1076/anec.9.4.276.8773 Neath, I., & Surprenant, A. (2005). Mechanisms of memory. In K. Lamberts & R. L. Goldstone (Eds.). Handbook of cognition, (pp. 221–238). New York: Sage Publishing.
352 Nesbit, J. C., & Adesope, O. O. (2006). Learning with concept and knowledge maps: A meta-analysis. Review of Educational Research, 76(3), 413–448. Nesheim, J. L. (2000). High tech start up, revised and updated: The complete handbook for creating successful new high tech companies. New York, NY: Free Press. Newell, A. (1990). Unified theories of cognition. Cambridge, MA: Harvard University Press. Newport, C. (2016). Deep Work: Rules for Focused Success in a Distracted World. Grand Central Publishing. Newton, L. (1990). Overconfidence in the communication of intent: Heard and unheard melodies. (Unpublished doctoral dissertation.) Stanford University, Stanford, CA. Nilsson, N. J. (1994). Teleo-reactive programs for agent control. Journal of Artificial Intelligence Research, 1(1), 139–158. Nilsson, N. J. (1998). Artificial intelligence. San Francisco: Morgan Kaufmann Publishers. Nist-Olejnik, S., & Holschuh, J. P. (2013). College success strategies. New York, NY: Penguin Academics. Norman, D. A. (1981). Categorization of action slips. Psychological Review, 88(1), 1–15. Norman, D. A, & Bobrow, D. G. (1979). Descriptions: An intermediate stage in memory retrieval. Cognitive Psychology, 11(1), 107–123. doi:10.1016/0010-0285(79)90006-9 Norman, D. A., & Shallice, T. (1986). Attention to action: willed and automatic control of behavior. In R. J. Davidson, G. E. Schwartz, & D. Shapiro, (Eds.) Consciousness and self-regulation. (Vol. 4, pp. 1–18). New York: Plenum. Oatley, K. (1992). Best laid schemes: The psychology of the emotions. Cambridge, UK: Cambridge University Press. Oatley, K., & Johnson-Laird, P. N. (1987). Towards a cognitive theory of emotions. Cognition & Emotion, 1(1), 29–50. doi:10.1080/02699938708408362 Oatley, K., & Johnson-Laird, P. N. (2011). Basic emotions in social relationships, reasoning, and psychological illnesses. Emotion Review, 3(4), 424–433. doi:10.1177/1754073911410738 Oh, K. (2012). What happens once you categorize files into folders? Proceedings of the 75th American Society for Information Science and Technology (ASIS&T) Annual Meeting, USA, 49(1). doi:10.1002/meet.14504901253/full Oh, K., & Belkin, N. (2014). Understanding what personal information items make categorization difficult. Proceedings of the 77th American Society for Information Science and Technology (ASIS&T) Annual Meeting, USA, 51(1). Ordóñez, L. D., Schweitzer, M. E., Galinsky, A. D., & Bazerman, M. H. (2009). Goals gone wild: The systematic side effects of overprescribing goal setting. The Academy of Management Perspectives, 23(1), 6–16. Ortony, A., & Turner, T. J. (1990). What’s basic about basic emotions? Psychological Review, 97(3), 315–331. Ortony, A., Clore, G. L., & Collins, A. (1988). The cognitive structure of emotions. Cambridge, UK Cambridge University Press. Ortony, A., Clore, G. L., & Foss, M. A. (1987). The referential structure of the affective lexicon. Cognitive Science: A Multidisciplinary Journal, 11(3), 341–364.
353 Otani, H., & Stimson, M. J. (1994). A further attempt to demonstrate hypermnesia in recognition. The Psychological Record, 44(1), 25. Panksepp, J. (1998). Affective neuroscience: The foundations of human and animal emotions. New York, NY: Oxford University Press USA. Panksepp, J., & Biven, L. (2012). The archaeology of mind: Neuroevolutionary origins of human emotions. [Kindle edition.] Retrieved from Amazon.com. Papert, S. (1965). Introduction. Embodiments of Mind. Cambridge, MA: MIT Press. Park, S. M. (1994). Reinterpreting Ryle: A nonbehavioristic analysis. Journal of the History of Philosophy, 32(2), 265–290. Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993). The adaptive decision maker. Cambridge, UK: Cambridge University Press. Peirce, C. S. (1905). What pragmatism is. The Monist, 5(2), 161–181. Pellegrini, A. D. (2013). Play. In P. D. Zelazo (Ed.), Oxford Handbook of Developmental Psychology (pp. 276-299). New York: Oxford University Press USA. Perkins, D. N. (1986). Knowledge as design. Hillsdale, NJ: Lawrence Earlbaum Associates. Perkins, D. N (1995). Outsmarting IQ: The emerging science of learnable intelligence. New York, NY: Free Press. Salomon, G., & Perkins, D. N. (1987). Transfer of cognitive skills from programming: When and how?. Journal of Educational Computing Research, 3(2), 149-169. Perlmutter, M. (1979). Age differences in adults’ free recall, cued recall, and recognition. Journal of Gerontology, 34(4), 533–539. doi:10.1093/geronj/34.4.533 Pessoa, L. (2013). The cognitive-emotional brain. Cambridge, MA: MIT Press. Petty, R. E., Wheeler, S. C., & Tormala, Z. L. (2003). Persuasion and attitude change. In I. B. Weiner (Ed.) Handbook of psychology (2nd ed.). New York, NY: John Wiley & Sons. Piaget, J. (1967). Biologie et connaissance. Paris, France: Éditions Gallimard. Piaget, J. (1977). Recherches sur l’abstraction réfléchissante. Paris: Presses Universitaires de France. Pigliucci, M. (2010). Nonsense on stilts: How to tell science from bunk. Chicago, IL: University of Chicago Press. Pigliucci, M., & Galef, J. (2012, November 7). Answers for Aristotle. Rationally Speaking Podcast. Podcast retrieved from http://rationallyspeaking.blogspot.ca/. Pigliucci, M., & Galef, J. (2013, April 24). Samuel Arbesman on the half-life of facts. Rationally Speaking Podcast. Podcast retrieved from http://www.rationallyspeakingpodcast.org/. Pinker, S. (1999). How the mind works. New York, NY: W.W. Norton. Pinker, S. (2010, June 10). Mind over mass media. New York Times. Retrieved from http://www.nytimes.com/. Pogue, D. (2013, July 17). 6 reasons smartphones won’t replace our brains. Scientific American, 309(2). Retrieved from http://www.scientificamerican.com/. Popper, K. R. (1959). The logic of scientific discovery. New York, NY: Basic Books. Popper, K. R. (1979). Objective knowledge. Oxford, UK: Oxford University Press. Popper, K. R. (1983). Realism and the aim of science: From the postscript to the logic of scientific discovery. (W. W. Bartley III, Ed.). Totowa, NJ: Rowman and Littlefield.
354 Post, L. A. (1932). Ancient memory systems. The Classical Weekly, 25(14), 105–110. Power, R. (1979). The organisation of purposeful dialogues. Linguistics, 17, 107–152. Pressley, M., & Gaskins, I. W. (2006). Metacognitively competent reading comprehension is constructively responsive reading: How can such reading be developed in students?. Metacognition and Learning, 1(1), 99-113. Pressley, M., & Afflerbach, P. (1995). Verbal protocols of reading. London, UK: Routledge. Pronin, E., Puccio, P., & Ross, L. (2002). Understanding misunderstanding: Social psychological perspectives. In T. Gilovich, D. Griffin, & D. Kahneman (Eds.), Heuristics and biases (pp. 636– 665). Cambridge, UK: Cambridge University Press. Psotka, J., Massey, L. D., & Mutter, S. A. (Eds.). (1988). Intelligent tutoring systems: Lessons learned. Hillsdale, NJ: Routledge. Pyc, M. A. (2010). Why is retrieval practice beneficial for memory? An evaluation of the mediator shift hypothesis. (Unpublished doctoral dissertation.) Kent State University, Kent, OH. Pyc, M. A., & Rawson, K. A. (2009). Testing the retrieval effort hypothesis: Does greater difficulty correctly recalling information lead to higher levels of memory? Journal of Memory and Language, 60(4), 437–447. doi:10.1016/j.jml.2009.01.004 Pyc, M. A., & Rawson, K. A. (2012). Why is test–restudy practice beneficial for memory? An evaluation of the mediator shift hypothesis. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38(3), 737-746. Pyc, M. A., & Rawson, K. A. (2010). Why testing improves memory: mediator effectiveness hypothesis. Science 330(6002), 335. doi:10.1126/science.1191465 Quiller-Couch, A. T. (1920). On the art of reading. [E-book.] Gutenberg Project. Retrieved from http://www.gutenberg.org/. Rawson, K. A., & Dunlosky, J. (2011). Optimizing schedules of retrieval practice for durable and efficient learning: How much is enough? Journal of Experimental Psychology: General, 140(3), 283–302. doi:10.1037/a0023956 Read, T., & Sloman, A. (1993). The terminological pitfalls of studying emotion. Paper presented at the Workshop on Architectures Underlying Motivation and Emotion - WAUME 93, Birmingham, UK. Retrieved from http://www.cs.bham.ac.uk/, 23 September 2014. Redding, R. E., Herbert, J. D., Formann, E. M. & Guadiano, B. A. (2008). Popular self-help books for anxiety, depression, and trauma: How scientifically grounded and useful are they? Professional Psychology: Research and Practice, 39, (5), 537-545. doi:10.1037/0735-7028.39.5.537 Ries, E. (2011). The lean startup: How today’s entrepreneurs use continuous innovation to create radically successful businesses. New York, NY: Viking Press Rigney, D. (2010). The Matthew effect: how advantage begets further advantage. New York, NY: Columbia University Press Robbins, A. (1991). Awaken the giant within: How to take immediate control of your mental, emotional, physical and financial destiny! Mahwah, NJ: Free Press. Robertson, G., Czerwinski, M., Baudisch, P., Meyers, B., Robbins, D., Smith, G., & Tan, D. (2005). The large-display user experience. Computer Graphics and Applications, IEEE, 25(4), 44–51. Robinson, J. P., Martin, S., Glorieux, I., & Minnen, J. (2011). The overestimated workweek revisited. Monthly Labor Review Online, 134(6), 43–53.
355 Roediger, H. L. (2008). Relativity of remembering: Why the laws of memory vanished. Annual review of psychology, 59, 225–254. doi:10.1146/annurev.psych.57.102904.190139 Roediger, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on Psychological Science, 1(3), 181. Roediger, H. L., & Karpicke, J. D. (2010). Intricacies of spaced retrieval: A resolution. In A. S. Benjamin (Ed.), Successful Remembering and Successful Forgetting: Essays in Honor of Robert A. Bjork. New York: Psychology Press. Roediger, H. L., Marsh, E. J., & Lee, S. C. (2002). Varieties of memory. In D. Medin and H. Pashler (Eds.). Steven’s handbook of experimental psychology, Vol. 2: Memory and cognitive processes, (3rd ed.) (pp. 1–41). New York: John Wiley & Sons. Rouet, J.-F. (2006). The skills of document use: from text comprehension to web-based learning. Mahwah, NJ: Lawrence Erlbaum Associates. Russel, S., & Norvig, P. (1995). Artificial intelligence. Upper Saddle River, NJ: Prentice Hall. Russell, S. (1995). Rationality and intelligence. Artificial Intelligence, 94(1-2), pp. 57–77. Russell, S. J., & Wefald, E. H. (1991). Do the right thing. Cambridge, MA: MIT Press. Ryle, G. (1949). The concept of mind. Chicago, IL: University of Chicago Press. Ryle, G. (1954). Dilemmas. Cambridge University Press, Cambridge, England. Sacks, O. (2007). Musicophilia. Toronto, ON: Alfred A. Knopf. Salomon, G., & Perkins, D. N. (1989). Rocky roads to transfer: Rethinking mechanism of a neglected phenomenon. Educational psychologist, 24(2), 113-142. Samsonovich, A. (2012, June 18). Comparative table of cognitive architectures. Retrieved from http://bicasociety.org/, August 8, 2014. Sandberg, J., & Alvesson, M. (2011). Ways of constructing research questions: gap-spotting or problematization? Organization, 18(1), 23–44. doi:10.1177/1350508410372151 Sartre, J. P. (1938). Esquisse d’une théorie des émotions. Paris, France: Hermann. Satel, S., & Lilienfeld, S. O. (2013). Brainwashed: The seductive appeal of mindless neuroscience. New York, NY: Basic Books. Saul, J. R. (2009). A Fair Country. Toronto, ON: Viking Canada. Scardamalia, M., & Bereiter, C. (1991). Literate expertise. In K. A. Ericsson & J. Smith (Eds.), Towards a general theory of expertise: Prospects and limits (pp. 172–194). Cambridge, UK: Cambridge University Press. Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and technology. In K. Sawyer (Ed.), Cambridge Handbook of the Learning Sciences (pp. 97–118). New York, NY: Cambridge University Press. Schacter, D. L. (1999). The seven sins of memory. Insights from psychology and cognitive neuroscience. The American Psychologist, 54(3), 182–203. Schacter, D. L. (2002). The seven sins of memory: How the mind forgets and remembers. Boston, MA: Houghton Mifflin Harcourt. Schön, D. A. (1983). The Reflective Practitioner. New York, NY: Basic Books. Schön, D. A. (1987). Educating the reflective practitioner. San Francisco, CA: Jossey-Bass. Schonfield, D., & Robertson, B. A. (1966). Memory storage and aging. Canadian Journal of Psychology, 20(2), 228–236.
356 Schrank, F. A. Engels, D. E. (1981). Bibliotherapy as a counseling adjunct: Research findings. Personnel and Guidance Journal 60(3), 143-147. Schwartz, D. L., Bransford, J. D., & Sears, D. (2005). Efficiency and innovation in transfer multidisciplinary perspective. In J. Mestre (Ed.), Transfer of learning: Research and Perspectives (pp. 1–51). Charlotte, NC: Information Age Publishing. Seidenberg, M. S. (2013, October 28). The Gladwell pivot. Retrieved from http://languagelog.ldc.upenn.edu/, November 26, 2013. Selfridge, O. (1959). Pandemonium: A paradigm for learning. London, UK: HM Stationery Office. Seligman, M. E., & Yellen, A. (1987). What is a dream? Behaviour research and therapy, 25(1), 1–24. Selye, H. (1964). From dream to discovery: On being a scientist. New York, NY: McGraw Hill. Sereno, J. (2013, April 7). Form and meaning regulation in language. In S. DiPaola (Director), Defining Cognitive Science Talks, Simon Fraser University, Burnaby, BC. Retrieved from http://www.sfu.ca/. Sereno, J. A., & Jongman, A. (1990). Phonological and form class relations in the lexicon. Journal of Psycholinguistic Research 19(6), 387–402. Sereno, J.A. (1994). Phonosyntactics. In L. Hinton, J. Nichols, and J. Ohala (Eds.), Sound Symbolism (pp.263-275), Cambridge, UK: Cambridge University Press. Shallice, T., & Burgess, P. (1996). The domain of supervisory processes and temporal organization of behaviour. Philosophical Transactions of the Royal Society of London: Series B, Biological Sciences, 351(1346), 1405–1412. doi:10.1098/rstb.1996.0124 Shanahan, M. (2010). Embodiment and the inner life: Cognition and consciousness in the space of possible minds. New York, NY: Oxford University Press USA. Shanahan, M. (2012). The brain’s connective core and its role in animal cognition. Philosophical Transactions of the Royal Society B: Biological Sciences, 367(1603), 2704–2714. doi:10.1098/rstb.2012.0217 SharpBrains. (2013). The digital brain health market 2012-2020: Web-based, mobile and biometricsbased technology to assess, monitor and enhance cognition and brain functioning. (A. Fernandez, Ed.). San Francisco, CA: SharpBrains. Shultz, T. R. (1982). Rules of causal attribution. Monographs of the Society for Research in Child Development, 47(1) 1–51. Shultz, T. R., Fisher, G. W., Pratt, C. C., & Rulf, S. (1986). Selection of causal rules. Child Development, 57(1), 143. doi:10.2307/1130646 Simon, H. A. (1967). Motivational and emotional controls of cognition. Psychological Review, 74(1), 29–39. Simon, H. A. (1971), Designing organizations for an information-rich world. In M. Greenberger (Ed), Computers, Communication, and the Public Interest. Baltimore, MD: The Johns Hopkins Press. Simon, D. A., & Bjork, R. A. (2001). Metacognition in motor learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27(4), 907. Simon, H. A. & Kaplan, C. A. (1989). Foundations of cognitive science. In M. I. Posner (Ed.), Foundations of cognitive science (pp. 1–47). Cambridge, MA: MIT Press. Simpson, J. (Ed.). (2013). Oxford English Dictionary. Oxford, UK: Oxford University Press. Retrieved from [http://www.oed.com].
357 Sinclair, N. (2004). The roles of the aesthetic in mathematical inquiry. Mathematical Thinking and Learning, 6(3), 261–284. doi:10.1207/s15327833mtl0603_1 Sinclair, N. (2006). Mathematics and beauty: aesthetic approaches to teaching children. New York, NY: Teachers College Press. Slamecka, N. J., & Graf, P. (1978). The generation effect: Delineation of a phenomenon. Journal of Experimental Psychology: Human Learning and Memory, 4(6), 592–604. Slaney, K. L., & Racine, T. P. (2011). On the ambiguity of concept use in psychology: Is the concept “concept” a useful concept? Journal of Theoretical and Philosophical Psychology, 31(2), 73–89. doi:10.1037/a0022077 Sloman, A. (1975). Afterthoughts on analogical representations. In B. L. Nash-Webber & R. C. Schank (Eds.), Proceedings of the 1975 Workshop on Theoretical Issues in Natural Language Processing (TINLAP ‘75) (pp. 431–439). Stroudsburg, PA: Association for Computational Linguistics. Sloman, A. (1978). The computer revolution in philosophy: Philosophy, science and models of mind. New York, NY: Harvester Press. Retrieved from http://www.cs.bham.ac.uk Sloman, A. (1979). The primacy of non-communicative language. In M. MacCafferty & K. Gray (Eds.), The analysis of meaning: Informatics 5, Proceedings ASLIB/BCS Conference (pp. 1-18). Retrieved from http://www.cs.bham.ac.uk/. Sloman, A., (1983), Image interpretation: The way ahead. In O. J. Braddick & A. C. Sleigh (Eds), Physical and Biological Processing of Images (pp. 380-401). Berlin, Germany: Springer-Verlag. Sloman, A. (1984). The structure of the space of possible minds. In S. Torrance (Ed.), The mind and the machine: Philosophical aspects of artificial intelligence (pp. 35–42). Chichester: Ellis Horwood. Sloman, A. (1985a). Real-time multiple-motive expert systems. In M. Merry (Ed.), Proceedings of the fifth technical conference of the British Computer Society specialist group on expert systems (pp. 1-13). Cambridge, UK: Cambridge University Press. Sloman, A. (1985c). What enables a machine to understand? In A. Joshi (Ed.), Proceedings of the 9th international joint conference on artificial intelligence (pp. 995–1001). San Francisco, CA: Morgan Kaufmann Publishers. Sloman, A. (1987). Motives, mechanisms, and emotions. Cognition & Emotion, 1(3), 217–233. http://www.cs.bham.ac.uk/research/projects/cogaff/81-95.html#6 Sloman, A. (1988). TEACH AITHEMES: A personal view of artificial intelligence. In M. Sharples, D. Hobb, C. Hutchinson, S. Torrancce, & D. Young, 1989, Computers and thought: A practical introduction to artificial intelligence (preface). Cambridge, MA: MIT Press. Sloman, A. (1989). On designing a visual system. Journal of Experimental and Theoretical Artificial Intelligence, 1(4), 289-337. Sloman, A. (1992). Prolegomena to a theory of communication and affect. In A. Ortony, J Slack, & O. Stock (Eds.) Communication from an artificial intelligence perspective: Theoretical and applied issues (229-260.) Heidelberg, Germany: Springer. Sloman, A. (1993a). Prospects for AI as the general science of intelligence. In A. Sloman, D. Hogg, G. Humphreys, D. Partridge, & A. Ramsay (Eds.). Prospects for Artificial Intelligence, (pp. 1–10). Amsterdam, Netherlands: IOS Press. Sloman, A. (1993b). The mind as a control system. In C. Hookway & D. Peterson (Eds.), Philosophy and the Cognitive Sciences (pp. 69–110). Cambridge, UK: Cambridge University Press.
358 Sloman, A. (1996). Actual possibilities. In L. C. Aiello, J. Doyle, & S. C. Shapiro (Eds.) Proceedings of the fifth international conference on principles of knowledge representation and reasoning (KR’96) (pp. 441–448). Burlington, MA: Morgan Kaufmann. Sloman, A. (1998). The “semantics” of evolution: Trajectories and trade-offs in design space and niche space. In H. Coelho (Ed.), Progress in artificial intelligence - IBERAMIA 98, 6th Ibero-American conference on AI (pp. 27–38). New York, NY: Springer. Sloman, A. (2000a). Models of models of mind. In J. Barnden & M. Lee (Eds.), Proceedings of the AISB’00 symposium on how to design a functioning mind. (pp. 1-13). Hove, UK: Society for the Study of Artificial Intelligence and the Simulation of Behavior. Sloman, A. (2000b). Interacting trajectories in design space and niche space: A philosopher speculates about evolution. In M. Schonauer, K. Deb, G. Rudolph, X. Yao, E. Lutton, J. J. Merelo, & H.-P. Schwefel (Eds.), Parallel Problem Solving from Nature - PPSN VI (pp. 3–16). Berlin, Germany: Springer Berlin-Heidelberg. Sloman, A. (2001). Beyond shallow models of emotion. Cognitive Processing, 2(1), 177-198. Sloman, A. (2002). Architecture-based conceptions of mind. In P. Gärdenfors, J. Wolenski, and K. Kijania-Placek (Eds.), In the scope of logic, methodology, and philosophy of science (Vol. 2), (pp. 403–427). Dordrecht, Netherlands: Kluwer Academic Publishers. Sloman, A. (2002b). Getting meaning off the ground: symbol grounding vs symbol attachment/tethering. Lecture conducted from MIT Media Lab, Cambridge, MA. Retrieved from http://www.cs.bham.ac.uk/. Sloman, A. (2003). How many separately evolved emotional beasties live within us? In R. Trappl, P. Petta, & S. Payr (Eds.), Emotions in humans and artifacts (pp. 35–114). Cambridge, MA: MIT Press. Sloman, A. (2004). Do machines, natural or artificial, really need emotions? Lecture conducted from Birmingham Café Scientifique, Birmingham, UK. Retrieved from http://www.cs.bham.ac.uk/. Sloman, A. (2005). The design-based approach to the study of mind (in humans, other animals, and machines), including the study of behaviour involving mental processes. Aaron Sloman. Retrieved November 17, 2012, from http://cs.bham.ac.uk/research/projects/cogaff/sloman-aiib10.pdf Sloman, A. (2008a/2011). Varieties of meta-cognition in natural and artificial systems. In M. T. Cox and A. Raja (Eds.), Metareasoning: Thinking about thinking (pp. 12-20). Cambridge, MA: MIT Press. Sloman, A. (2008b). What evolved first and develops first in children: Languages for communicating, or languages for thinking. Mind as machine weekend course. Lecture conducted from Oxford University, Oxford, UK. Retrieved from http://www.cs.bham.ac.uk/. Sloman, A. (2009a). What cognitive scientists need to know about virtual machines. In N. A. Taatgen, H. van Rijn, L. Schomaker, and J. Nerbonne (Eds.) Proceedings of the 31st annual conference of the Cognitive Science Society (pp. 1210–1215). Retrieved from http://csjarchive.cogsci.rpi.edu/. Sloman, A. (2009b). Architecture-based motivation vs reward-based motivation. Retrieved from: http://www.cs.bham.ac.uk/. Sloman, A. (2009c). Understanding causation in robots, animals and children: Hume’s way and
359 Kant’s way. Retrieved from http://www.cs.bham.ac.uk/. Sloman, A. (2010a). Two notions contrasted: ‘Logical geography’ and ‘logical topography’. Retrieved from http://www.cs.bham.ac.uk/. Sloman, A. (2010b). How virtual machinery can bridge the “explanatory gap”, in natural and artificial systems. In S. Doncieux, B. Girard, A. Guillot, J. Hallam, J.-A. Meyer, J.-B. Mouret (Eds.), Natural and artificial systems. From Animals to Animats 11 (pp. 13–24). Berlin, Germany: SpringerVerlag. Sloman, A. (2010c). Phenomenal and access consciousness and the “hard” problem: A view from the designer stance. International Journal of Machine Consciousness, 2(1), 117. doi:10.1142/S1793843010000424 Sloman, A. (2011a). What’s information, for an organism or intelligent machine? How can a machine or organism mean? In G. Dodig-Crnkovic & M. Burgin (Eds.), Information and Computation (pp. 393–438). Hackensack, NJ: World Scientific. Sloman, A. (2011b). Supervenience and causation in virtual machinery. Retrieved from http://www.cs.bham.ac.uk/. Sloman, A. (2011c/2013a). Comments on Annette Karmiloff-Smith’s (1992) book: Beyond Modularity: A Developmental Perspective on Cognitive Science. Retrieved from http://www.cs.bham.ac.uk/. Sloman, A. (2011d, March 10). The self–A bogus concept. Retrieved from http://www.cs.bham.ac.uk/. Sloman, A. (2012a). The meta-morphogenesis (MM) project (or meta-project?). Retrieved from: http://www.cs.bham.ac.uk/. Sloman, A. (2012b, November 1). Meta-morphogenesis and toddler theorems: Case studies. Retrieved from http://www.cs.bham.ac.uk/. Sloman, A. (2013). Meta-morphogenesis theory as background to cognitive robotics and developmental cognitive science. In J. Fagard, R. A. Grupen, F. Guerin, & N. Krüger (Organizers), Mechanisms of ongoing development in cognitive robotics, Schloss Dagstuhl, Wadern, Germany. Sloman, A., & R. L. Chrisley, (2003). Virtual machines and consciousness. Journal of Consciousness Studies, 10(4-5), 133–172. Sloman, A. and Croucher, M. (1981) Why robots will have emotions. In P. J. Hayes (Ed.), Proceedings of the 7th international joint conference on artificial intelligence (pp. 197-202). San Francisco, CA: Wiliam Kaufmann Small, T. (2008, May 20). Brain bulletin #38 - 8 simple strategies to boost your brain power. Retrieved from http://www.terrysmall.com/. Smolin, L. (2006). The trouble with physics: The rise of string theory, the fall of a science, and what comes next Boston, MA: Houghton Mifflin Harcourt. Sonnentag, S., & Kleine, B. M. (2000). Deliberate practice at work: A study with insurance agents. Journal of Occupational and Organizational Psychology, 73(1), 87–102. Sparks, D. (2012). Paperless. [E-book.] Available at http://macsparky/com/. Speelman, C. P., & Kirsner, K. (2005). Beyond the learning curve: The construction of mind. New York, NY: Oxford University Press. Spencer, C. (2006). Research on learners’ preferences for reading from a printed text or from a
360 computer screen. Journal of Distance Education, 22(1), 33–50. Spitzer, H. F. (1939). Studies in retention. Journal of Educational Psychology, 30(9), 641. Stanovich, K. E. (2009). What intelligence tests miss: The psychology of rational thought. New Haven, CT: Yale University Press. Stanovich, K. E. (2011). Rationality and the reflective mind. New York, NY: Oxford University Press. Stanovich, K. E. (2010). How to think straight about psychology (9th ed.). Upper Saddle River, NJ: Pearson Education. Stanovich, K. E., West, R. F., & Toplak, M. E. (2011). Intelligence and rationality. In R. J. Sternberg & S. B. Kaufman (Eds.), The Cambridge handbook of intelligence (pp. 784–826). Cambridge, UK: Cambridge University Press. Starbuck, W. H. (1992). Learning by knowledge-intensive firms. Journal of Management Studies, 29(6), 713–739. Starbuck, W. H. (2010). What makes a paper influential and frequently cited? Journal of Management Studies, 47(7), 1394–1404. doi:10.1111/j.1467-6486.2009.00903.x Staw, B. M. (1981). The escalation of commitment to a course of action. Academy of Management Review, 6(4), 577-587. Sternberg, R. J. (1981). Intelligence and nonentrenchment. Journal of Educational Psychology 73(1), 1–16. Sternberg, S. (1969). Memory-scanning: Mental processes revealed by reaction-time experiments. American Scientist, 57(4), 421–457. Sternberg, S. (1975). Memory scanning: New findings and current controversies. The Quarterly Journal of Experimental Psychology, 27(1), 1–32. Sterner, T. M. (2006). The practicing mind. Wilmington, DE: Mountain Sage Publishing. Stich, S. (1992). What is a theory of mental representation? Mind, 101(402), 243–261. Still, A., & Dryden, W. (1999). The place of rationality in Stoicism and REBT. Journal of RationalEmotive and Cognitive-Behavior Therapy, 17(3), 143–164. http://doi.org/10.1023/A:1023050427199 Stix, G. (2013) New study: Neuroscience research gets an “F” for reliability. Retrieved from http://blogs.scientificamerican.com/. Strohmaier, M. (2008). Purpose tagging: Capturing user intent to assist goal-oriented social search. In I. Soboroff, E. Agichtein, & R. Kumar (Eds.), Proceeding of the 2008 ACM workshop on search in social media, SSM 2008 (pp. 35–42). Napa Valley, CA: Association for Computing Machinery. Strohmaier, M., Körner, C, & Kern, R. (2010). Why do users tag? Detecting users’ motivation in social tagging systems. In M. Hearst, W. Cohen, & S. Gosling (Eds.), Proceedings of the fourth international conference on weblogs and social media (pp. 339-342). Palo Alto, CA: Association for the Advancement of Artificial Intelligence. Surprenant, A., & Neath, I. (2009). Principles of memory. New York, NY: Psychology Press. Sziraki, M. (1978). Mental and biological assimilation and accommodation. Canadian Psychological Review/Psychologie Canadienne 19(1), 67–73. Tavris, C., & Aronson, E. (2008). Mistakes were made (but not by me): Why we justify foolish beliefs, bad decisions, and hurtful acts. Boston, MA: Mariner Books. Templar, R. (2006). The rules of life. Harlow, UK: Pearson Education Limited.
361 Thagard, P. (2007, April 30). Cognitive science. In E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy. Retrieved from http://plato.stanford.edu/, on November 27, 2012. Thagard, P. (2012). The cognitive science of science: Explanation, discovery, and conceptual change. Cambridge, MA: MIT Press. Thayer, R. E. (2001). Calm energy. Toronto, ON: Oxford University Press. Thompson, J. J., Blair, M. R., Chen, L., & Henrey, A. J. (2013). Video game telemetry as a critical tool in the study of complex skill learning. PLoS ONE, 8(9), e75129. doi:10.1371/journal.pone.0075129 Toplak, M. E., West, R. F., & Stanovich, K. E. (2012). Education for rational thought. In J. R. Kirby & M. J. Lawson (Eds.), Enhancing the quality of learning: Dispositions, instruction, and learning processes (pp. 51–92). Cambridge, UK: Cambridge University Press. Torneke, N. (2010). Learning RFT: An introduction to relational frame theory and its clinical application. Oakland, CA: New Harbinger Publications. Tulving, E. (1972). Episodic and semantic memory. In E. Tulving & W. Donaldson (Eds.), Organization of Memory (pp. 381–403). New York, NY: Academic Press. Tulving, E. (2007). Are there 256 different kinds of memory? In J. S. Nairne (Ed.), Foundations of remembering: Essays in honor of Henry L. Roediger, III (pp. 39–52). New York, NY: Psychological Press. Tulving, E., & Schacter, D. L. (1990). Priming and human memory systems. Science, 247(4940), 301– 306. Turing, A. M. (1937). On computable numbers, with an application to the Entscheidungsproblem. Proceedings of the London Mathematical Society, 42(2), 230–265. Tversky, A. (1969). Intransitivity of preferences. Psychological Review, 76(1), 31–48. United States Department of Labor, Occupational Safety and Health Administration. (2013, April 9). Computer workstation evaluation checklist. Retrieved from http://www.osha.gov/. Unsworth, N., & Engle, R. W. (2005). Simple and complex memory spans and their relation to fluid abilities: Evidence from list-length effects. Journal of Memory and Language, 54(1), 68– 80. doi:10.1016/j.jml.2005.06.003 Unsworth, N., & Engle, R. W. (2007). The nature of individual differences in working memory capacity: active maintenance in primary memory and controlled search from secondary memory. Psychological Review, 114(1), 104–132. doi:10.1037/0033-295X.114.1.104 van der Vlist, E. (2002). XML Schema. Sebastopol, CA: O’Reilly. van Merriënboer, J. J. G. (1997). Training complex cognitive skills: A four-component instructional design model for technical training. Englewoods Cliffs, NJ: Educational Technology Publications. VanLehn, K. (1989). Problem solving and cognitive skill acquisition. In M. I. Posner (Ed.), Foundations of Cognitive Science (pp. 526–579). Cambridge, MA: Bradford Books. VanLehn, K. (1996). Cognitive skill acquisition. Annual review of psychology, 47(1), 513–539. doi:10.1146/annurev.psych.47.1.513 VanLehn, K. (1999). Rule learning events in the acquisition of a complex skill: An evaluation of Cascade. Journal of the Learning Sciences, 8(1), 71–125. VanLehn, K., Arbor, A., & Jones, R. M. (1993). Better learners use analogical problem solving sparingly. In P. E. Utgoff (Ed.), Proceedings of the tenth international conference on machine learning (pp. 338–345). San Francisco, CA: Morgan Kaufmann
362 Visser, C. (2009, November). Interview with Keith Stanovich. Retrieved from http://articlescoertvisser.blogspot.ca/. Vygotsky, L. S. (1978). Mind in society. Cambridge, MA: Harvard University Press. Wagner, J. F. (2006). Transfer in pieces. Cognition and Instruction, 24(1), 1–71. Wagner, R. K., & Stanovich, K. (1996). Expertise in reading. In A. Ericsson (Ed.), The road to excellence: The acquisition of expert performance in the arts and sciences, sports, and games (pp. 189–225). Hillsdale, NJ: Lawrence Erlbaum Associates. Wallace, M., & Wray, A. (2006). Critical reading and writing for postgraduates. London, UK: Sage Publications. Wallace, M., & Wray, A. (2011). Critical reading and writing for postgraduates, 2nd Ed. London, UK: Sage Publications. Watson, J. B. (1913). Psychology as the behaviorist views it. Psychological Review, 20(2), 158–277. Waytz, A., Morewedge, C. K., Epley, N., Monteleone, G., Gao, J.-H., & Cacioppo, J. T. (2010). Making sense by making sentient: Effectance motivation increases anthropomorphism. Journal of Personality and Social Psychology, 99(3), 410–435. doi:10.1037/a0020240 Wechsler, D. (1925). What constitutes an emotion? Psychological Review, 32(3), 235–240. doi:10.1037/h0074516 Wells, A. (2005). Detached mindfulness in cognitive therapy: A metacognitive analysis and ten techniques. Journal of Rational-Emotive and Cognitive-Behavior Therapy, 23(4), 337–355. doi:10.1007/s10942-005-0018-6 Wells, A. (2008). Metacognitive therapy: Cognition applied to regulating cognition. Behavioural and Cognitive Psychotherapy, 36(06), 651. doi:10.1017/S1352465808004803 Wells, A., & Mathews, G. (1994). Attention and emotion: A clinical perspective. Hillsdale, NJ: Lawrence Erlbaum Associates Publishers. Wertheimer, M. (1959). Productive thinking, Enlarged Ed. New York, NY: Harper & Brothers. Weststar, J. (2011). Staying current in computer programming: The importance of informal learning and task discretion in maintaining job competence. In D. W. Livingstone (Ed.), Education & jobs: Exploring the gaps (pp. 185–209). Toronto, ON: University of Toronto Press. White, A. R. (1975). Conceptual analysis. In C. J. Bontempor & S. J. Odell (Eds.), The owl of Minerva (pp. 103–117). New York, NY: McGraw Hill. White, R. W. (1959). Motivation reconsidered: the concept of competence. Psychological Review, 66(5), 297–333. Whittaker, S. (2011). Personal information management: from information consumption to curation. Annual Review of Information Science and Technology, 45, 3–62. Whittaker, S., Kalnikaité, V., & Ehlen, P. (2012). Markup as you talk: Establishing effective memory cues while still contributing to a meeting. CSCW’12 (pp. 349–358). New York: ACM Press. Wile, D. (1993). After the fight. New York: Guilford. Wiley, J., & Voss, J. F. (1999). Constructing arguments from multiple sources: Tasks that promote understanding and not just memory for text. Journal of Educational Psychology, 91(2), 301–311. doi:10.1037//0022-0663.91.2.301 Winne, P. H. (2006). How software technologies can improve research on learning and bolster school reform. Educational Psychologist, 41(1), 5–17.
363 Winne, P. H. (2012). Should we expect experiments to advance cognitive science? Four criticisms and a remedy. In S. DiPaola (Director), Defining Cognitive Science Talks, Simon Fraser University, Burnaby, BC. Retrieved from http://www.sfu.ca/. Winne, P. H. (2014). Issues in researching self-regulated learning as patterns of events. Metacognition and Learning, 9(2), 229-237. doi:10.1007/s11409-014-9113-3 Winne, P. H., & Baker, R. S. (2013). The potentials of educational data mining for researching metacognition, motivation and self-regulated learning. Journal of Educational Data Mining, 5(1), 1-8. Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 279– 306). Hillsdale, NJ: Lawrence Erlbaum Associates. Wissman, K. T., Rawson, K. A., & Pyc, M. A. (2012). How and when do students use flashcards? Memory, 20(6), 568–579. doi:10.1080/09658211.2012.687052 Wolf, M. (2008). Proust and the squid: The story and science of the reading brain. Cambridge: UK: Icon Books. Wolfe, J. L. (2000). Effects of annotations on student readers and writers. In Proceedings of the Fifth ACM Conference on Digital Libraries (pp. 19-26). New York: ACM Press. Wolfe, J. L. (2001). Pedagogical uses of annotations and annotation technologies (Doctoral dissertation). The University of Texas at Austin. Wolfe, J. L. (2002). Annotation technologies: A software and research review. Computers and Composition, 19(4), 471–497. Wood, R., & Bandura, A. (1991). Social cognitive theory of organizational management. In R. M. Steers & L. W. Porter (Eds.), Motivation and work behavior, 5th Ed. (pp. 179–191). New York, NY: McGraw Hill. WorkEZ Light. (n.d.). Retrieved from Uncaged Ergonomics: https://www.uncagedergonomics.com/, on April 9, 2013. Worthen, J. B., & Hunt, R. R. (2008). Mnemonics: Underlying processes and practical applications. In H. L. Roediger (Ed.), Cognitive psychology of memory, Vol. 2 (pp. 145–156). Oxford, UK: Elsevier. Wozniak, P. A., & Gorzelanczyk, E. J. (1994). Optimization of repetition spacing in the practice of learning. Acta Neurobiologiae Experimentalis, 54(1), 59–62. Wright, I., Sloman, A., & Beaudoin, L. P. (1996). Towards a design-based analysis of emotional episodes. Philosophy, Psychiatry, & Psychology, 3(2), 101–126. doi:10.1353/ppp.1996.0022 Yamamoto, S. (Ed). (2013). Human interface and the management of information (Proceedings of the 15th International Conference, HCI International 2013). New York, NY: Springer. Yates, T., Khunti, K., Wilmot, E. G., Brady, E., Webb, D., Srinivasan, B., et al. (2012). Self-reported sitting time and markers of inflammation, insulin resistance, and adiposity. American Journal of Preventive Medicine, 42(1), 1–7. doi:10.1016/j.amepre.2011.09.022 Young, M. N., & Gibson, W. B. (1976). How to develop an exceptional memory. Chatsworth, CA: Wilshire Book Company. Zhang, Y., Huang, S. C., & Broniarczyk, S. M. (2010). Counteractive construal in consumer goal pursuit. Journal of Consumer Research, 37(1), 129–142. doi:10.1086/649912 Zimmerman, B. J. (2000). Self-efficacy: An essential motive to learn. Contemporary Educational
364 Psychology, 25(1), 82-91.
Index 10,000 hour rule Abatis Systems Corp. abbreviation expanders abstract artifacts acceptance and commitment therapy (ACT) accessing information accommodation ACT (acceptance and commitment therapy) action tags active reading active study Adler, Mortimer adult mental development agile processes (lean) aging alarm systems Alfred algorithmic mind algorithms, anytime aliases Allen, David alphabet “Alphabet Song, The” Amazon analogical reasoning analogies analysis of concepts assessment cause and origin characterize the concept control examples of questions, miscellaneous template
366 template example andon cord andon cord principle Anki Desktop (flashcard software) annotation annotation services in books browser and goals lack of in information technology multimedia short-hand software software, third party annotation services anytime algorithms appeal in knowledge resources criticisms of definition and emotions impressions of and mathematics and surprise Apple’s Automator applying knowledge applying knowledge, failure to architectural modeling architecture of the mind architecture, mental areas of responsibility (OmniFocus) artifacts, abstract artifacts, conceptual artificial intelligence assessment of documents (information technology) assessment of explanatory theories assessment of information appeal assessment of knowledge resources complications in criteria for and CUPA (caliber, utility, potency, appeal) difficulties of
367 evaluating and values assessment, taxonomy of assimilation associative conditioning attitudes attitudes, changing audio as a knowledge resource autonomous mind backward-reaching-transfer basal ganglia BBEdit Behavior and Brain Sciences (BBS) Bereiter, Carl Beyond Modularity (Karmiloff-Smith) biases, cognitive BibTeX bi-directionality of cognitive development bid-response Bjork, Robert Bloom’s taxonomy Boden, Margaret books vs technology navigating bootstrapping strategy brain mechanisms brain structure Bratt, Sharon broad cognitive science Bugzilla build it, and they will come Build-Measure-Learn loop caliber of knowledge resources Calibre capture Carpenter, Shana Carr, Nicolas challenge templates challenges (instiller) andon cord example concepts, new
368 consider the opposite cramming definition difficulty level examples of practice examples rating ease of questions responding to re-testing schedules schedules, spacing vocabulary terms chess and expertise and memory child vs adult mental development childhood mental development language open- vs closed-classed words the/my word choice citation manager classical cognition classical cognitive processes classification classification of documents cognitive aging cognitive biases cognitive defeatism cognitive defusion cognitive fitness cognitive miserliness cognitive parsimony cognitive potency cognitive productivity cognitive shuffle challenges of definition shallow vs. deep processing and education software cognitive reflexes cognitive shuffle
369 cognitive science criticisms of definition lack of psychology, lack of in and technology, applying terminology differences untapped in information technology cognitive skills and chess and mastery phases of acquisition and practice training training cognitive strategy cognitive terms CogSci Apps Corp. CogZest collections of information, mastering challenges mastering cues harsh startup example practice practice principles RD cue system commenting in documents comparative analysis competence development of feeling of illusion of component processes comprehension computer workstations concept maps concept of goal Concept of Mind, The (Ryle) concept specifications concepts defining distinctions of new
370 instillers of new mastery of new potent conceptual analysis definition conceptual artifacts conceptual progress conceptual understanding template consider the opposite constructible cue system consuming, as metaphor for information contexts, knowledge resources counteractive construal criteria, for assessing knowledge resources criteria, rhetorical critical reasoning crystallized intelligence cue chaining cue mnemonic cue overload cued recall cues CUPA: caliber, utility, potency, appeal curation, as metaphor of information processing daemons decision making declarative memory deep processor delegation model deliberate performance deliberate practice amount needed concepts, new and expertise and knowledge workers Schön on types of vocabulary terms deliberation scheduling deliberative processes Delicious delving
371 audio definition e-books examples of and memory multimedia multimedia, other vs surfing effectance preliminary description of, White’s concept of, generalized. Dennett, Daniel designer stance desirable difficulties hypothesis desktop search engine developing (level of processing information) development of the adult mind DevonAgent DEVONthink digestion, as metaphor of information processing Diigo discriminative cue system dismantle mindware dispositions distractibility distributed recall practice documents filing organizing, project related organizing, third party documents, assessing domain reading Dragon Dictate Dropbox dry cognition EagleFiler e-books editing tools education cognitive productivity learning objectives and memory and memory reading
372 transfer problem effectance effectiveness information, using to earn processing knowledge effectiveness, improving and cognitive aging efficiency elaborate retrieval hypothesis emotional command centres emotions emotions, secondary episodic memory epistemic criteria e-reader software e-readers ergonomics Ericsson, K. Anders Ericsson’s theory of expertise Ericsson’s theory of expertise criticisms with errors in mental representations evaluating knowledge resources EverNote evolution examples (learning from) excelling executive functions experience expert judgments expert memory expert reading expertise in chess and education and effectance fluid and intelligence and memory and motivation and novices and talent explanatory theories
373 extended mind factual memory fan effect fascination feedback (as learning tool) feeling of competence file systems aliases desktop search engines tagging documents fine-grained mental representations fixed-action patterns flashcard applications flashcard software flashcards flaws, knowledge flow fluid expertise fluid intelligence fluid rationality focal resource and meta-information folders, organizing project related folk psychology foresight bias forgetting framework, productivity free recall Freud, Sigmund functional autonomy gem General Problem Solver (GPS) generation effect Getting Things Done (GTD) (Allen) and knowledge organization as personal management system knowledge gems OmniFocus criticism of glial cells goal processing systems goals Gottman, John
374 GPS (General Problem Solver) Grant, Seth graphic tools GTD (Getting Things Done; Allen) habits harsh startup example harsh startups H-CogAff Theory (Human-Cognition and Affect) and ACT (acceptance and commitment therapy) emotion, classes of and emotions goals illustration meta-management processes motivators heuristic relevance-signaling hypothesis hierarchical organization of information highlighting How to Read a Book (Adler) iBooks IDs illusion of competence illusion of rationality illusions illusions of (future) recall illusions of comprehension illusions of helpfulness of information illusions of meta-effectiveness illusions of rationality imagery mnemonics implicit information implicit understanding inert knowledge inert mindware inferring information assessing information, processing and complexity levels of information to effectiveness funnel information, quality of inner motivators
375 inspection of knowledge resources instiller stubs instillers challenge challenges examples concepts, new creation of creating definition designing design rules and knowledge gems motive generators practice smart, folder template types intellectual macho intelligence vs rationality intentional stance Intentional Stance, The (Dennett) intentional tagging intentional talk intermediate effect internal monitors internal motivators Internet attention spans as a distraction and memory rewiring brains interpersonal relations interpretation of knowledge interrupt filters intuition intuitive understanding IQ (Intelligence Quotient) irrationality issue (ticket) processing system James, William Jobs, Steve judgment of knowledge resources
376 judgment of learning junk information Karmiloff-Smith, Annette Karpicke, Jeffrey keyboard shortcuts Kindle know how knowledge abundance of application of definition failure to apply levels of mastery organizing and self improvement processing for effectiveness processing, levels of Knowledge as a Design (Perkins) knowledge flaws knowledge gaps knowledge gems capturing definition design instillers extracting identifying instillerizing mastering practicing with instillers producing knowledge resources knowledge work knowledge work knowledge workers access to knowledge assessment of information cognitive science, lack of knowledge about and cognitive science, problems with definition flexibility in thinking identifying as IT burden at home
377 learning and producing rapidly and meta-effectiveness organizing work and practice print preference and self improvement smart people, surrounded by and time pressures Koriat, Asher labyrinthine lag effect language, childhood development LaunchBar launcher programs layering layers of human mind lean processes Lean Startup, The (Ries) Leap learning learning linking information to information lists, mastering logic long-term memory long-term working memory Lord, Charles machinery management processes marriage mastering collections of information challenges cues harsh startup example practice practice principles RD cue system Mavericks mediator shift hypothesis mediators Mekentosj Papers memes
378 memory and the alphabet declarative and education episodic factual H-CogAff Theory long-term long-term working memory and music principles of prioritizing information procedural quizzing and recall semantic short-term working and technology working memory judgment skewing of word pairing experiment word pairing experiment memory-indexing mental architecture mental development Mental Development Challenge, The mental development, adult mental development, childhood mental reflexes mental representations meta-access problem meta-cognition meta-computation meta-doc (meta-document) accessing analysis section creating definition examples of index Notational Velocity
379 sections templates meta-effectiveness cognitive productivity as contribution to cognitive science definition and designer stance and psychology and psychotherapy meta-information accessing external internal managing tagging types of meta-level reasoning meta-management metamemory meta-semantic competence method of loci micro cognition microdevelopment microdomains of cognition Microsoft OneNote Microsoft Project mind, as a term mind, as virtual machines mind, autonomous mindware categories of definition development dismantle and flashcard software and productive practice inert instiller vs mental concepts motivational aspects of development personal and physics
380 reactive software analogy Stanovich on unhelpful mnemonic system definition and designing instillers RD cue system mnemonics imagery instiller template instilling mindware prioritizing RD cue system mobile cognitive-productivity modifiability modular architecture monitors bid monitors building detecting violations developing growing internal novelty monitors, computers morphogenesis motivation for increased competence motivational aspects of mindware development motivational process motivational state motivators attributes of developing inappropriate internal and management processes tertiary emotions motive motive generators motor multimedia annotation
381 multiple-choice test questions music and memory and practice mySleepButon natural reactive systems natural selection neurons neuroscience cognitive psychological processes nodes non-contradiction, principle of normal learning vs expertise norms Notational Velocity note-taking note taking, audio nStudy nvALT objective knowledge Objective Knowledge (Popper) observation, self obsessions OmniFocus OmniGraffle OmniOutliner OmniPlan open access movement open- vs closed-classed words, in childhood mental development OpenMeta opinions, differing organizing knowledge organizing work OS X Mavericks outliners outlining Panksepp, Jack paper vs technology Paperless (Spark) Papers (software) parallelism
382 PDF apps PDF files PDF reader PDFPenPro pedagogical utility perceived competence perceived self-efficacy perception Perkins, David personal development personal mindware perturbance PhraseExpress physical world (World 1) Piaget, Jean Pinker, Stephen plasticity Pocket podcasts Popper, Sir Karl potency of a knowledge resources definition and mental development as a subjective notion and understanding and usefulness practical books practical knowledge practice by answering questions and chess concepts, new and forgetting and memory and music and skill acquisition spacing schedules time Practice Zealously experiment predictability prediction preferences
383 Preview (Apple’s PDF) principle of non-contradiction principles of expert memory printing information from technology printing vs on-screen problem solving problems of transfer problems of understanding problems, identifying procedural knowledge procedural memory process of modularization processing knowledge resources process-motivator index process-purpose index product startups productive information-processing productive laziness productive practice concepts, new definition developing propensities example of and flashcard apps goals objectives rules steps software and technology tips for vocabulary terms productive processor Productive Thinking (Wertheimer) productivity framework productivity literature productivity software productivity systems productivity tools productivity training products of World 2’ (World 3) professional practice
384 project information, organizing across different files project planning system project view, OmniFocus projects, identifying propensities prophesy, self-fulfilling psychological challenges Psychopathology of Everyday Life, The (Freud) psychotherapy and meta-effectiveness Pyc, Mary quality of information questions and practice questions, in delving quizzes R&D rating scales rational behavior rationality and decisions definition fluid growing monitors illusion of vs intelligence non-contradiction Stanovich on taxonomy of training RD cue system applying challenge templates challenges of challenges of collections and lists described principles of structure for concept instillers reactive mechanisms reactive mindware reactive processes reactive systems Readability (software)
385 reading realms of thinking reason reason recall collections and lists distributed practice illusions of (future) and memory being cue-driven practicing practice lazily experiment as a skill RecentX Reddit reflecting-in-action reflection reflection, in learning reflective abstraction reflective intelligence reflective mind reflective practice Reflective Practitioner, The: How Professionals Think in Action (Schön) Relationship Cure, The (Gottman) relationship problems remembering reminiscence representation representational machinery representational redescription (RR) Representational redescription (RR) in reverse representations resource-rating tags retrieval (of information) retrieval strategies retrieval structure principle review (of information) rhetorical criteria Ries, Eric right vs wrong Rodeiger, Henry, III rote learning RR (representational redescription)
386 RR (representational redescription) in reverse Ryle, Gilbert scalar ratings scheduling, deliberation schema activation exercise Schön, Donald A. scratch pad screen vs printing Scrivener search engines, problems with secondary emotions self improvement self-efficacy self-fulfilling prophesy self-help books self-modification self-monitoring self-observation self-regulation self-testing semantic memory sense-making ability Seven Principles for Making Marriage Work (Gottman) shallows Shallows, The (Carr) short-term memory skill acquisition skills Skim (PDF reader) sleep sleep onset Sloman, Aaron smart instiller-folder smart people and mental architectures and self-destructive beliefs surrounded by smart people who do dumb things smartphones software andon cord principle and annotation
387 flashcard meta-docs access meta-docs annotation OmniFocus outlining project planning system tagging task management TextExpander solutions somnolent mentation hypothesis spacing practice Spark, David speed reading speed up principle Spitzer, Herbert F. Spotlight standards Stanovich, Keith statable knowledge students studying subjective knowledge vs objective knowledge Successful Investor, The (McKeough) superficial processor surface processing surfing information surfing vs. delving surprise (in appeal of knowledge resources) Swahili word experiment synapses synaptic connectivity sync technology System 1 systems biology table of contents tablets tagging action benefits criticisms of while delving
388 documents and highlighting information categories information you don’t understand intentional knowledge gaps meta-information needs in software resource-rating with Skim (PDF reader) software software faults system (for information) temporary term tag topic websites tagging system IDs PDF reader Tags (software) task management system task manager tasks taxonomy of assessment taxonomy of rationality taxonomy, Bloom’s technical rationality technology 1950’s attention spans information processing and memory vs paper and perceived competence proficiency with shallow use of and time pressures tools to remove distractions temporary tags term tag terms, finding later
389 tertiary emotions test questions test-enhanced learning testing effect text expansion software TextExpander TextWrangler the/my word choice in childhood development theory of expert memory theory of mind theory of the development of expertise thesis writing thinking disposition thinking strategy ticket (issue) processing system time management time pressures time tracking times to practice topic tagging TrackTime transcription transfer transformational processing true-false test questions two-strike principle Type 1 process understanding knowledge understanding, concept of understanding, implicit unlearning urgency usefulness of knowledge resources usefulness vs utility utility theory UVOutliner value judgments VanLehn, Kurt vestibular system vignettes agile project management bids in marriage
390 investing virtual machinery virtual machines (World 2’) vision vocabulary terms deliberate practice mastering new productive practice voice-driven task list volition web browsing web surfing Wertheimer, Max What Intelligence Tests Miss: The Psychology of Rational Thought (Stanovich) White, Robert work management working memory World 1 (physical world) World 2 World 2’ (virtual machines) World 3 (products of World 2’) worlds (domains) wrong vs right Yep Yojimbo zone of proximal development