[1st Edition] 9780128121689, 9780128121177

Table of contents :
Content:
Front MatterPage ii
CopyrightPage iv
ContributorsPage ix
Chapter One - New Wine in Old Bottles: “Repurposed” Methodologies for Studying Expertise in PhysicsOriginal Research ArticlePages 1-34José P. Mestre, Jason W. Morphew
Chapter Two - The Interaction Between Knowledge, Strategies, Metacognition, and MotivationOriginal Research ArticlePages 35-52Mitchell Rabinowitz
Chapter Three - Understanding Sound: Auditory Skill AcquisitionOriginal Research ArticlePages 53-93Shannon L.M. Heald, Stephen C. Van Hedger, Howard C. Nusbaum
Chapter Four - Causal Knowledge and Reasoning in Decision MakingOriginal Research ArticlePages 95-134York Hagmayer, Cilia Witteman
Chapter Five - Domain-Specific Versus Domain-General Maintenance in Working Memory: Reconciliation Within the Time-Based Resource Sharing ModelOriginal Research ArticlePages 135-171Valérie Camos
Chapter Six - A Framework of Episodic Updating: An Account of Memory Updating After RetrievalOriginal Research ArticlePages 173-211Bridgid Finn
Chapter Seven - Is Prospective Memory Unique? A Comparison of Prospective and Retrospective MemoryOriginal Research ArticlePages 213-238Dawn M. McBride, Rachel A. Workman
Chapter Eight - Context as an Organizing Principle of the LexiconOriginal Research ArticlePages 239-283Michael N. Jones, Melody Dye, Brendan T. Johns
Chapter Nine - Edge-Aligned Embedded Word Activation Initiates Morpho-orthographic SegmentationOriginal Research ArticlePages 285-317Jonathan Grainger, Elisabeth Beyersmann

Citation preview

Series Editor

BRIAN H. ROSS Beckman Institute and Department of Psychology University of Illinois, Urbana, Illinois

Academic Press is an imprint of Elsevier 50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United States 525 B Street, Suite 1800, San Diego, CA 92101-4495, United States 125 London Wall, London EC2Y 5AS, United Kingdom The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, United Kingdom First edition 2017 Copyright © 2017 Elsevier Inc. All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions. This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein).

Notices

Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods, professional practices, or medical treatment may become necessary. Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information, methods, compounds, or experiments described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility. To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein. ISBN: 978-0-12-812117-7 ISSN: 0079-7421 For information on all Academic Press publications visit our website at https://www.elsevier.com/books-and-journals

Publisher: Zoe Kruze Acquisition Editor: Kirsten Shankland Editorial Project Manager: Charlotte Rowley Production Project Manager: Magesh Kumar Mahalingam Designer: Mark Rogers Typeset by TNQ Books and Journals

CONTRIBUTORS Elisabeth Beyersmann Macquarie University, Sydney, NSW, Australia Valérie Camos Université de Fribourg, Fribourg, Switzerland Melody Dye Indiana University, Bloomington, IN, United States Bridgid Finn Educational Testing Service, Princeton, NJ, United States Jonathan Grainger Aix-Marseille University and Centre National de la Recherche Scientifique, Marseille, France York Hagmayer University of Goettingen, Goettingen, Germany Shannon L.M. Heald The University of Chicago, Chicago, IL, United States Brendan T. Johns University at Buffalo, Buffalo, NY, United States Michael N. Jones Indiana University, Bloomington, IN, United States Dawn M. McBride Illinois State University, Normal, IL, United States José P. Mestre University of Illinois at Champaign-Urbana, Champaign, IL, United States Jason W. Morphew University of Illinois at Champaign-Urbana, Champaign, IL, United States Howard C. Nusbaum The University of Chicago, Chicago, IL, United States Mitchell Rabinowitz Fordham University, New York, NY, United States Stephen C. Van Hedger The University of Chicago, Chicago, IL, United States Cilia Witteman Radboud University, Nijmegen, The Netherlands Rachel A. Workman Illinois State University, Normal, IL, United States

ix

j

CHAPTER ONE

New Wine in Old Bottles: “Repurposed” Methodologies for Studying Expertise in Physics José P. Mestre1 and Jason W. Morphew University of Illinois at Champaign-Urbana, Champaign, IL, United States 1 Corresponding author: E-mail: [email protected]

Contents 1. Introduction 2. Expertise in Physics 2.1 Previous Studies of Expertise in Physics 2.2 Can ExperteNovice Findings Translate Into Instructional Practice? 3. New Approaches for Exploring Expertise in Physics 3.1 Categorization of Problems by Difficulty 3.1.1 Study Design 3.1.2 Findings 3.1.3 Conclusions From Categorization Study

7 9 15

3.2 Change Blindness 3.2.1 3.2.2 3.2.3 3.2.4

17

Use of Change Blindness to Study Expertise Study Design Findings Conclusions From Change Blindness Study

17 18 22 22

3.3 The Flicker Paradigm 3.3.1 3.3.2 3.3.3 3.3.4

2 3 3 4 6 6

23

Use of Flicker to Study Expertise Study Design Findings Conclusions From Flicker Study

24 26 27 29

4. Conclusion References

31 32

Abstract The study of expertise has a long tradition in physics and has led to a deeper understanding of how content expertise is developed and used. After a brief overview of previous methodologies used to study expertise in physics, accompanying findings, and how those findings have enriched classroom practices, we describe three new approaches for exploring expertise in physics. The three approaches rely on methodologies common in cognitive science but are repurposed, or adapted, to study Psychology of Learning and Motivation, Volume 67 ISSN 0079-7421 http://dx.doi.org/10.1016/bs.plm.2017.03.001

© 2017 Elsevier Inc. All rights reserved.

1

j

2

José P. Mestre and Jason W. Morphew

expertise in new ways within a science content area. We discuss findings from these studies, some surprising and some expected, then conclude with commentaries about the value of these approaches for studying expertise in science, and speculate on the implications of the findings for improving educational practices.

1. INTRODUCTION The study of expertise and expertenovice differences has a long tradition in physics and has led to insights into how knowledge is structured in memory, and deployed, to analyze and solve problems. After a brief overview of previous methodologies used to study expertise in physics, the accompanying findings, and how those findings have enriched classroom practices, we describe three “new” methodologies for exploring expertise. The word new was placed in quotes since the methodologies are not new (they have been used extensively in psychology), but rather “repurposed” to explore expertise in new ways within the context of a science content area. Some findings were surprising, some expected, but in all three cases the methodologies proved to be useful tools for the study of expertise in the sciences. We also discuss some important insights from our findings for improving educational practices. Why has the study of expertise been prominent in cognitive science? By investigating how experts perceive problems and situations in their domains of expertise, and how they organize and deploy knowledge to reason about problems and situations, not only can researchers gain insights into efficient ways of storing knowledge in memory and solving problems, but also instructors can structure learning activities that specifically target more efficient ways of developing expertise. The study of expertise is also a complicated endeavor. In science domains, such as physics, the knowledge is rich and extensive, taking a long time to acquire, and the problems one attempts to solve are complicated and involve the coordination of considerable declarative and procedural knowledge (see Mestre, Docktor, Strand, & Ross, 2011, and references therein). Nevertheless, a lot is known about the nature of expertise from empirical work using various paradigms. This chapter discusses three recent research studies from our lab that explore expertise in physics. To the best of our knowledge, the methodologies used in the three studies have not been used to study expertise in the sciences and reveal some interesting and unexpected findings. We begin by discussing previous methodologies used to study expertise, the findings from those studies, and examples of application of some findings to improve instruction.

New Wine in Old Bottles

3

2. EXPERTISE IN PHYSICS 2.1 Previous Studies of Expertise in Physics Since studies of expert and novice behavior are extensive, we will only provide a sample here of some common methodologies used to study expertise in physics and the findings that have emerged. One of the most highly cited studies of expertise was the card-sorting task used by Chi, Feltovich, and Glaser (1981). Experts (graduate students in physics) and novices (students having just completed an introductory course) were given a stack of physics problems written on index cards and asked to form piles of cards that had similar solutions. Experts’ piles reflected a focus on the problems’ deep structure (the major principle needed to solve the problems), while novices’ piles tended to reflect problems that shared similar surface structure (surface attributes such as inclined planes or pulleys). Other categorization tasks have been devised that are much easier to analyze than card-sorting tasks. For example, three-problem categorization tasks contain a model problem and two comparison problems, and participants are asked to pick the comparison problem that is solved most like the model problem (Hardiman, Dufresne, & Mestre, 1989). Twoproblem categorization tasks have also been used where participants are asked whether or not the two problems are solved similarly and to state the reasoning behind their answers (Hardiman et al., 1989). These tasks allow manipulation of competing featuresdfor example, problems can match on surface attributes, on the underlying principle needed for solution, both or neitherdthereby allowing nuanced explorations of categorization behavior among experts and novices. Expertenovice differences in physics problem solving are also studied with think-aloud interview techniques. For example, experts and novices are given a problem and are asked to discuss the approach they would use to solve it. Experts typically begin by providing a qualitative analysis of the problem, which then leads to possible solution strategies, and eventually equations that could be applied, whereas novices begin with equations that match the given or desired quantities contained in the problem (Chi et al., 1981). Other techniques used to study expertenovice differences in physics include memory recall of relevant information for solving problems, use of self-monitoring and metacognitive strategies in problem solving, and explorations of what experts and novices attend to when engaged in problem solving (sometimes using eye-tracking methodology); a review of these techniques and other expertenovice

4

José P. Mestre and Jason W. Morphew

research findings can be found in Docktor and Mestre (2014) and references therein. Generally speaking, expert approaches to physics problem solving can be characterized as strategic, whereas novice approaches can be characterized as tactical.

2.2 Can ExperteNovice Findings Translate Into Instructional Practice? Although expertenovice research focuses on describing behavioral differences between experts and novices and the reasoning that underlies those differences, there are notable examples where findings from expertenovice research have led to the development of interventions that help novices develop habits and behaviors that reflect more expertlike performance. We mention four interventions, developed in our lab based on research findings of expertenovice differences, which were implemented and met with some success. The first intervention used a rudimentary menudriven computer tool that constrained novices to follow an expertlike, conceptual analysis prior to solving introductory physics mechanics problems. The computer-based tool was modeled after research findings from Chi et al. (1981) indicating that experts, when asked to state an approach for solving a physics problem, began by identifying a principle, then justify why the principle is applied to the specific problem context and then describe a procedure for instantiating the principle. A series of studies (Dufresne, Gerace, Hardiman, & Mestre, 1992; Mestre, Dufresne, Gerace, Hardiman, & Touger, 1993) using this tool resulted in novices displaying expertlike attributes, such as improved ability to categorize problems according to principles, and improved problem-solving performance, compared to control treatments. The second intervention (Leonard, Dufresne, & Mestre, 1996) was conducted in a large introductory college physics course and focused on the effect of a different implementation of the Chi et al.’s (1981) findings. This study tried to get students to generate strategies for problem solving as experts do. It compared two large introductory physics classes, one taught traditionally, the other requiring students to write strategies to accompany problem solutions, which consisted of a conceptual analysis of a problem containing three pieces: the major principle(s), a justification for why the principle applied to the specific context, and a procedure for applying the principle. Strategies were modeled during lectures when problems were worked out and students practiced them in homework and some

New Wine in Old Bottles

5

exam problems. When compared to students in the traditional course, students in the strategy-writing course were significantly better at categorizing problems according to the principles needed for solution and demonstrated better long-term retention of the major ideas covered in the course months later. The third intervention, which represents a similar approach, was utilized in high school physics classes (Docktor, Mestre, & Ross, 2015). In this approach, “conceptual problem solving” (CPS) was used in teaching problem solving, again trying to get the student to approach the problem strategically in terms of the underlying principles. CPS highlighted the “principle” to be applied, the “justification” for why it could be applied to the specific context, and the “plan” which was a list of procedural steps for applying the principle. This was followed by a “two-column solution” where each step in the plan was written on the left and the executed step (e.g., a free body diagram or equations) was presented on the right. Results indicated that students produced solutions of higher quality than prior to using CPS and scored higher on various conceptual and problem-solving measures. Finally, we tested whether it was possible to improve novices’ ability to categorize problems according to the principles needed in the solution using a short, 1-h intervention (Docktor, Mestre, & Ross, 2012). In the intervention, students who had finished an introductory mechanics course were given two-problem categorization items (as described in the previous section), asked if they would be solved similarly and to provide a reason for their decision. Two conditions were explored; in one condition, students were simply told if they were correct or wrong after each categorization item; in the other, in addition to being told whether or not they were correct, they were also given a reason. For example, if they were correct, the reason stated they were correct and mentioned the principle that could be used to solve both problems; if they were wrong, they were told so and the reason stated that one problem was solved with one principle (explicitly stated) and the other problem was solved with a different principle (also explicitly stated). We found that the students in the latter condition attempted to apply principles as their categorization criterion significantly more often compared to the former condition, although both groups performed about the same in their ability to correctly categorize problems according to similarity of solution.

6

José P. Mestre and Jason W. Morphew

3. NEW APPROACHES FOR EXPLORING EXPERTISE IN PHYSICS In the remainder of this chapter we discuss three approaches from cognitive science that we have repurposed to investigate expertenovice differences in physics. The first methodology used categorization with a new twistdnovices and experts were asked to categorize problems according to their level of difficulty (as measured by performance of a large-N sample of students solving the problems in a real test scenario). The other two methodologies were borrowed from visual cognition. One methodology utilizes change blindness in which a change is made in a situation that an unsuspecting participant is consideringdin our case the participant was formulating an explanation to a physics situation and the situation was slightly changed to see if she/he noticed. In the other methodology, we used the “flicker technique” where the participant was asked to find the change between two rapidly changing physics diagrams on a screen as fast as possible.

3.1 Categorization of Problems by Difficulty It is not difficult to argue that the ability to determine a problem’s level of difficulty is a useful skill for both experts and novices in the sciences. From a student’s perspective, determining the difficulty of problems can help time management both during study and during an exam. Although studying/solving problems that a student finds easy would not go far in honing problem-solving skills, focusing on harder problems would. This is a study strategy found to be effective since students studying harder problems outperform those who focus on easier problems (Koriat, Ma’ayan, & Nussinson, 2006; Tullis & Benjamin, 2011), but only for those students who are accurate in judging problem difficulty. During an exam, accurately judging which problems are easier and completing them first would allow more time to be spent on the harder problems, but not at the expense of leaving no time for doing the easy problems. For instructors (the experts) being able to judge problem difficulty is a great skill in test development. A good test is one that has a good balance of easy, medium, and hard problems to accurately assess students’ proficiency with the subject matter. Yet, little has been done in studying novices’ and experts’ ability to judge problem difficulty. One study by Gire, Rebello, Singh, Sabella, and Rebello (2010) had both experts and novices in physics rank problem difficulty on a 1e10 scale,

New Wine in Old Bottles

7

but it was difficult to discern a pattern in the difficulty ratings. Perhaps the general finding that students in introductory physics (Rebello, 2012) and other subjects (Dunning, Heath, & Suls, 2004; Dunning, Johnson, Ehrlinger, & Kruger, 2003) overestimate their performance on exams, with worse predictions among low-performing students, is related to inability to judge problem difficulty. In large introductory physics courses, exams will often consist of “problem sets,” which are groups of two to four problems that share a common physics scenario. Data from large numbers of students taking these types of exams afford the opportunity to determine actual problem difficulty as measured by student performance in these problem sets. Instead of ranking an individual problem on a 1e10 difficulty scale, problem sets allow an individual to judge relative difficulty among sets of problems related by the same context. Our strategy was to draw on the two-problem categorization paradigm discussed above to construct pairs of problems that shared the same storyline but differed substantially in difficulty as measured empirically in real exam scenarios. By giving many such pairs to novices and experts and asking them to state which problem in the pair was found to be more difficult by students taking an exam, we could measure this skill and determine if judgments of problem difficulty tracked with expertise (Fakcharoenphol, Morphew, & Mestre, 2015). More specifically, we conducted two experiments that explored the ability of experts and novices to make predictions about the difficulty of physics problems, focusing on two main questions. First, what aspects of problems do experts use when determining problem difficulty? Second, does the ability to judge problem difficulty improve with experience or content expertise? In other words, how accurate are both experts and novices in judging problem difficulty? 3.1.1 Study Design To investigate these questions, we conducted two experiments that asked individuals to make judgments of problem difficulty. The first explored how experts reason when asked to think about problem difficulty, while the second examined the accuracy of problem difficulty judgments as a function of expertise. To examine the criteria that experts use to make judgments of problem difficulty, we conducted an experiment where experts in physics education (advanced graduate students and faculty members) with teaching and research experience related to introductory calculus-based

8

José P. Mestre and Jason W. Morphew

mechanics courses judged the difficulty of problem pairs. These experts were given a packet of 78 problem pairs from previously administered exam questions. Each problem pair referred to the same physics scenario, allowing us to obtain a numerical assessment of their ability to judge problem difficulty (percent correct judgments). Since there was a need to have problem pairs differ significantly in difficulty, we selected problem pairs that differed in student performance by at least 15 percentage points. The experts were asked to indicate which problem was more difficult for students to solve and to explain the reasoning behind their judgment decision. The experts were given the packet of problem pairs and could perform the task wherever and whenever they desired and could take as long as they needed to complete all problem pairs. They were also asked to indicate the time at which they started reading the problem pairs and the time at which they finished writing their explanation. By having experts provide judgments, timing, and reasoning, we were able to ascertain the accuracy of their judgments of problem difficulty, how long individuals spent making judgments, and the extent to which differences among experts’ accuracy were due to different judgment strategies/criteria. To examine how experts and novices compare in their ability to predict problem difficulty, we conducted a second experiment with three groups. Two groups were comprised of novices, and the third group was comprised of graduate students (heretofore, “near-experts”) who had taught in the introductory physics courses. The novices used in this experiment were undergraduate students who had finished an introductory calculus-based mechanics course. One group of novices was asked which problem would be more difficult for them to solve, while the other group was asked which problem would be more difficult for their classmates to solve. The “nearexperts” were asked which problem would be more difficult for a student to solve. All of them read related problem pairs and indicated which problem was more difficult. From the set of 78 problem pairs described above, a set of 28 problem pairs were selected for this experiment. The 28 problem pairs were selected to cover a wide range of physics concepts typically found in an introductory mechanics physics course and had at least 80% of the experts accurately predict which problem in each pair was the more difficult one. All of the participants were seated at a computer screen and presented with the pairs of physics problems. After viewing each pair for 90 s, the individuals were asked to indicate which problem of the pair was more difficult.

New Wine in Old Bottles

9

3.1.2 Findings When individuals make judgments of problem difficulty under time pressure, they often rely on implicit heuristics when making their decision (Benjamin, 2005; Reder, 1987). In the first experiment, the experts were not under external time pressure, but since they were asked to categorize 78 problem pairs, the experts were limited in the amount of time they could spend on any problem pair, spending an average time of 2 min to make their judgment and write down their explanation underlying their decision. We identified three main themes concerning the problem features that experts used to predict differences in problem difficulty. Experts used reasons that focused on the question context (21.0%), the content type (43.9%), and student characteristics (27.4%). The remaining comments (7.8%) either indicated that the experts were guessing, were unsure of their reasoning, or were unclear and could not be categorized. Further analysis indicated that within each major theme the rationale experts used to predict the more difficult problem within a problem pair tended to cluster into categories. Example rationales are included in Table 1, and the percentages for each category can be found in Table 2. Experts reported a single reason (68.6%) more often than multiple reasons (31.4%) when explaining why they selected a certain problem as being more difficult. However, when experts identified more than one reason to support their prediction, they were more accurate in identifying the more difficult problem in the pair. Certain rationales were more successful in allowing the experts to determine which problems are more difficult for students. Experts were more accurate than chance when they used rationales that focused on the presence of strong distractors, the number of steps or level of math needed to solve a problem, the need to attend to the direction objects move to use the correct signs, the differences in conceptual understanding needed to solve a problem, how familiar students are with a problem type, and how intuitive a problem is for students. The results for all subcategories can be found in Table 2. While the experts performed much better than chance at identifying the more difficult problems, findings suggest that experts may have difficulty taking the perspective of novices in some instances. This difficulty in being able to take the perspective of someone who does not have the knowledge that you have has been studied under names such as “the expert blind spot” (e.g., Nathan & Petrosino, 2003), “the curse of knowledge”

10

José P. Mestre and Jason W. Morphew

Table 1 Physics education research expert rationale category examples Theme Category Examples

Question context

Content type

Student characteristics

Question type “It seems that students would tend to do better on more conceptual problems.” “#1 is pure calculation. #2 is conceptual.” Distractor “The distractor is more powerful in this one.” “a, b, and e are good distractors.” Wording “#1 involves interpreting the expression ‘maximum,’ which some students are not very good at.” “#2, picture suggests block 2 will go down, and block 1 up. So even [without] calculation you get correct answer.” More steps “You need the answer to #1 to correctly determine 2.” “#2 requires knowing the answer to #1.” Math “Math is more difficult to set up properly for #2, plus students struggle with ratios. #1 is glorified plug and chug.” Direction “Students must set up with appropriate sign changes in #1. In #2 the direction of [acceleration] is apparent.” Content “Stationary systems are easier for students than dynamic systems.” “#1 is Newton’s 3rd [Law], very simple.” Familiarity “Students are not trained to think about [acceleration] on the side of a vertical track. They are trained to calculate things at the top.” “This seems to be a point that is not practiced nor emphasized as much.” Misconceptions “[Question] 1 prompts several misconceptions.” “I think students would tend more to the force causes velocity p-prim in #2.” Intuition “Changing masses is more intuitive than changing springs.” “I think that conceptual question would be easier because it seems more intuitive.” Carelessness “Students may easily forget the friction” “Students are careless.”

From Fakcharoenphol, W., Morphew, J. W., & Mestre, J. P. (2015). Judgments of problem difficulty among experts and novices. Physical Review-Special Topics: Physics Education Research, 11(#2), 020128 (14 pages). http://dx.doi.org/10.1103/PhysRevSTPER.11.020128. https://journals.aps.org/prstper/ pdf/10.1103/PhysRevSTPER.11.020128.

11

New Wine in Old Bottles

Table 2 Percentages and chi-square tests of accuracy above chance for physics education research experts Percent N (percent) accuracy Chi-square P

Question context Question type Distractor Wording/Pictures Problem content More steps Math Direction Content Student characteristics Familiarity Misconceptions Intuition Carelessness Uncategorizable Guessing/I don’t know

68 (11.0) 35 (5.7) 27 (4.4)

60.3 85.7 63.0

2.882 17.857 1.815

NS