Research Methods for English Language Teachers [1st ed.] 9781315832548

Table of contents :
Cover......Page 1
Title Page......Page 2
Copyright Page......Page 3
Table of Contents......Page 4
Preface......Page 8
Dedication......Page 9
Acknowledgements......Page 10
Introduction: setting the scene......Page 12
Part 1: principles and perspectives......Page 16
Introduction......Page 18
Context and roles......Page 19
Instances......Page 21
Patterns of research: implications......Page 25
Conclusion......Page 28
Discussion notes......Page 30
Introduction......Page 32
Action and reflection......Page 33
A short educational excursion......Page 36
Teacher research and English language teaching......Page 39
Conclusion......Page 45
Discussion notes......Page 46
General views......Page 48
Common views of what research in language teaching does......Page 52
Basic and applied research......Page 53
Description and intervention......Page 55
Normative and interpretive research......Page 58
Conclusion......Page 65
Discussion notes......Page 66
Features of good research......Page 68
Comparison of research traditions on these features......Page 80
Aspects of design of research......Page 81
Conclusion......Page 83
Discussion notes......Page 84
Beginnings......Page 86
Approaches......Page 89
Content: teachers' choices......Page 92
Research and research: existing work......Page 95
Conclusion......Page 99
Discussion notes......Page 100
Part 2: topics and methods......Page 102
Principles, methods, techniques......Page 104
Introduction to methods and techniques......Page 106
Out there: discovering other people's work and telling them about one's own......Page 107
Observation: uses and perspectives......Page 112
Systematizing observation......Page 116
Alternatives to coding schemes......Page 122
Naturalistic observation......Page 125
Conclusion......Page 129
Discussion notes......Page 130
Some definitions......Page 132
Diary data......Page 133
Diaries in language learning and language teaching......Page 138
Keeping a diary......Page 145
Discussion notes......Page 147
Introduction......Page 148
Why count, and what to count......Page 149
Describing the numbers with other numbers......Page 150
Inference from chance - 'significance'......Page 161
Computational aids......Page 163
Discussion notes......Page 164
Why experiments?......Page 166
Causality and the method of detail......Page 168
Experiments and quasi-experiments......Page 169
Some examples......Page 172
Reflections on the experimental approach......Page 176
Conclusion......Page 179
Discussion notes......Page 180
Introduction......Page 182
Questionnaires......Page 183
Interviews......Page 192
Issues in interviewing......Page 196
Conclusion......Page 198
Discussion notes......Page 199
Introduction......Page 200
Introspection......Page 201
Verbal report and think-aloud......Page 202
Research on oneself......Page 205
Research on learners......Page 207
Some examples......Page 211
Conclusion......Page 212
Discussion notes......Page 213
What is a case?......Page 214
Methods in case study research......Page 218
Case studies in language learning and teaching......Page 220
The controversy of case study......Page 227
Discussion notes......Page 229
Principles in mixing methods......Page 230
Teachers' research: some continuing case studies......Page 236
Conclusion......Page 242
Discussion notes......Page 243
The role of research by teachers......Page 244
Appropriate methodologies......Page 245
Development of criteria......Page 246
Teacher research and higher-degree research......Page 247
Evaluations of teacher research......Page 248
And finally .........Page 250
Appendix......Page 252
References......Page 258
Index......Page 268

Citation preview

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Researchmethods for English languageteachers

jO McDONOUGH SeniorLecturerin English as tl Foreign Language.University ofEssex.UK

STEVEN McDONOUGH Lecturerin AppliedLinguistics. Univers4yof Essex.UK

ROUTLEDGE

Routledge Taylor & Francis Group

LONDON AND NEW YORK

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

First publishedin 1997 Impressionpublishedin 2008 by Hodder Education This edition published2014 by Routledge 2 Park Square,Milton Park,Abingdon, Oxon OXI4 4RN 711 Third Avenue,New York, NY 10017,USA

Routledgeis an imprint ofthe Taylor & Francis Group, an informa business © 1997 Jo McDonough & StevenMcDonough

All rights reserved.No part of this publication may be reproducedor transmittedin any form or by any means,electronically or mechanically, including photocopying,recordingor any information storageor retrieval system,without either prior permissionin writing from the publisheror a licence permitting restrictedcopying. In the United Kingdom such licences are issuedby the Copyright Licensing Agency: Saffron House,6-10 Kirby Street, London EC1N 8TS.

British Library Cataloguing in Publication Data A cataloguerecord for this book is available from the British Library Library of CongressCataloging-in-PublicationData McDonough,Jo Researchmethodsfor English languageteacher/ Jo McDonough. StevenMcDonough. p. cm. Includes bibliographical references and index. ISBN 0-340-61472-2(pbk.) 1. English language-Studyand teaching-Foreignspeakers-Research-Methodology.I. McDonough.StevenH. II. Title. PE1l28.A2M384 1997 428'.0072- dc20 96-35143 ISBN: 978 0 340 61472 3

CIP

Typesetin 10'12/12'12Ehrhardt by SaxonGraphicsLtd, Derby

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Table of contents

Prefoce Acknowledgements

Vll IX

Introduction:settingthe scene

1

Part 1: principlesand perspectives

5

1. Teachersin action Introduction Contextand roles Instances Patternsof research:implications Conclusion Discussionnotes

7 7 8 10

2. The teacherresearcherin focus

21 21 22 25 28 34 35

3. What is research?

37 37 37 41 42

Introduction Action and reflection A shorteducationalexcursion Teacherresearchand English languageteaching Conclusion Discussionnotes

Introduction Generalviews Commonviews of what researchin languageteachingdoes Basicand appliedresearch Descriptionand intervention Normativeand interpretiveresearch Conclusion Discussionnotes

14

17 19

44

47 54 55

IV

TABLE OF CONTENTS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

4. Principlesand problems:what makesgood research? Introduction Featuresof good research Comparisonof researchtraditionson thesefeatures Aspectsof designof research Conclusion Discussionnotes

57 57 57 69 70 72 73

Introduction Beginnings Approaches Content:teachers'choices Researchand research:existing work Conclusion Discussionnotes

75 75 75 78 81 84 88 89

Part2: topics and methods

91

5. Generatingresearch

6. Definitions and overview Principles,methods,techniques Introductionto methods andtechniques Out there: discoveringother people'swork and telling them aboutone'sown

7. Observinglanguageclassrooms Introduction Observation:usesand perspectives Systematizingobservation Alternativesto codingschemes Naturalisticobservation Conclusion Discussionnotes

8. Diaries and diary studies Introduction Somedefinitions Diary data Diaries in languagelearningand languageteaching Keepinga diary Conclusion Discussionnotes

93 93 95 96 101 101 101 105

III

114 118 119 121 121 121 122 127 134 136 136

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

UBUmCONTINTI

9. Using numbers Introduction Why count,and what to count Describingthe numberswith other numbers Inferencefrom chance- 'significance' Computationalaids Conclusion Discussionnotes

v

137 137 138 139 150 152 153 153

10. Doing experiments Introduction Why experiments? Causalityand the methodof detail Experimentsand quasi-experiments Someexamples Reflectionson the experimentalapproach Conclusion Discussionnotes

155 155 155 157 158 161 165 168 169

11. Asking questions Introduction Questionnaires Interviews Issuesin interviewing Conclusion Discussionnotes

171 171 172 181 185 187 188

12. Looking inside: methodsfor introspection Introduction Introspection Verbal report and think-aloud Researchon oneself Researchon learners Someexamples Conclusion Discussionnotes

189 189 190 191 194 196 200 201 202

13. Studyingcases Introduction What is a case? Methodsin casestudy research Casestudiesin languagelearningand teaching The controversyof casestudy Conclusion Discussionnotes

203 203 203 207 209 216 218 218

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

VI

TABLE OF CONTENTS

14. Mixing researchmethods In troduction Principlesin mixing methods Teachers'research:somecontinuingcasestudies Conclusion Discussionnotes

219 219 219 225 231 232

Conclusion The role of researchby teachers Appropriatemethodologies Developmentof criteria 'Popularity'of varioustechniques Teacherresearchand higher-degreeresearch Evaluationsof teacherresearch And finally ...

233 233 234 235 236 236 237 239

Appendix

241

Reftrences

247

Index

257

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Preface

'Research'and 'teaching'are in many ways quite differentspheresof activity, eachadheringto its own principles, procedures,objectivesand methodologies. Indeed, the two terms have often been polarized, contributing to the unhelpful and rather tired distinction between'theory' on the one hand and 'practice'on the other. In recentyears,however,the interfacehas beenboth challengedand productively explored by practitionersin the field of ELT: there is a growing literature on the researchcarried out by teachersand on the kinds of modelsfrom which suchresearchis drawn. The presentbook is intendedas a contributionto this literature. Its primary addressees are English languageteachersin their everydayprofessional context. It is based onthe view that teachershave a huge numberof issues and questionssurroundingthem in that context; that there are available many appropriateresearchtechniquesfor exploring those issues;that professionalresearchers and teachershavemuch to sayof interestto eachother; and that it is nowadaysnot unusualfor English languageteachersto pursue periodsof training and professionaldevelopmentaway from the classroom and thus sometimesto entertemporarilya different valuesystem. Teacherresearchhas becomesomethingof a 'buzz' word. Although this book is divided into two sections,one that is largely concernedwith 'principles' and anotherwith 'method',we have attemptedto show how thesetwo aspectsare linked. The principles informing the kinds of researchwith which teachersare becomingincreasinglyfamiliar have powerful derivations and long histories in other disciplines, in the social sciencesas well as in generaleducation;and researchmethodsand techniquesare themselvesnot free-floating, but are embeddedin debatesabout philosophiesand paradigms, and abouthow the social world might be construed.We hope,simply, that readerswill find here some stimulus, or support, for exploring their own ideasand questionsabout teachingand learning, and will wish to peel away someof the layersof both contentand methodthat havefascinatedus. )0 and StevenMcDonough University ofEssex /996

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

For Dorothy Martha Gladys Jepson

1905-1994

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Acknowledgements

We are grateful to Longman Group UK Ltd and L. van Lier for permission to reproducethe box 'Types of research'from The classroomand the languagelearner, 1988,p. 57. We particularly wish to acknowledgethe ideas and hints and feedback from many colleaguesand studentsover the last few years. We are also grateful to Dilly Meyer for helping us to preparethe final manuscript. Only we are to be held responsiblefor the outcome.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

This page intentionally left blank

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Introduction: setting the scene

This book is about teachersand research,more specifically about the relationship betweenthem, and with the primary focus on teacher-initiated and teacher-executedresearch.It is therefore not in itself intended as a generalmanualcovering all researchtopics and techniquesof relevanceto languagelearningand teaching(there are indeedmany pertinenttitles that do precisely this), which would include, for example, second language acquisition (SLA) research,needs analysis, programmeevaluation, languagetestingand a numberof other major areas.Our perspectivedoesnot, of course,exemptthesefrom discussion,but they are drawn on in so far as the teachermay, in senseswhich it will be importantto define, harnessand make use of them. The book is embeddedin the professionof Teaching English as a Foreign Language(TEFL), from which examplesand case studieswill be taken.At the sametime, however,a greatmany of the issues and methodologiesdiscussedderive from a broadereducationalbase,and by implication from a diversity of philosophies,disciplinesand procedures. The book has been written in order to bring togetherthe possibilities for researchinherent and feasible in the English as a Foreign Language (EFL) classroom,and the rangeof approachesand techniquesavailableto teachersfor carrying out or being involved in such research.Thus the starting point and, we hope, the conceptualcoherenceoffered, is the simple questionof what practisingteacherscan actively do to try to develop a 'researchstance',and to solve someof the open questionsin their classroom environment.We would wish to argue that this is broad-basedand not restrictive, first becauseof the numberof topics that can in principle be the focus of attention,and secondbecauseof the wide rangeof research methodsthat can be called upon. This view, then, is underpinnedby severalassumptions: • The frequently cited dichotomy between, on the one hand, teachingas a 'practical activity' and, on the other, researchas a 'theoretical'endeavouris regardedas unhelpful.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

2

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

• Furthermore,althoughthere are someplausiblereasonsas to why teachersmay often feel alienated by conventionalresearchoutcomes and approaches,it is argued that researcher-generated researchis not by definition irrelevant,and that teacher-initiated, applied, and even pure researchparadigmscan and should be brought togethermore than is currently the case.Teachers,as we shall see, can even change camps during the course of their careers.What is feasible therefore in principle includes understanding and applying pre-existing research results as well as possibilitiesfor reciprocity and collaboration. • On a cautionary note, however, the potential tension betweena definition of teachingas 'action' and of researchas 'understanding' cannotbe fudged: much of the debateabout action research, for example,revolvesaroundthis apparentconflict, as is signalled in its very name. • It is clear that teachershave both explicit formulations of, and implicit attitudes to, issuesand events in their own professional lives which a researchperspectiveis able to addresswithin the reality of the classroomcontext. • These'issuesand events'potentially include aspectsof the classroom, groups and subgroups,individual learnersand their own purposesand agendas,the teacher'spersonal-professional development, the wider managementand administrativecontext, syllabus and materials, collaboration with others, change and innovation, and are not restricted to the apparently dominant researchconcern of how languagelearning takes place, however important that is and, indeed, however much teachersmight be able to contributeto it from sourcesand resourcesas yet not fully tapped. The major and obviousquestionthat is still beggedis, of course,that of what is meant by the term 'research'.The beginningof an answer,albeit evasive,is to say that it dependson why we want to know: to the extent that reality is a personaland social construct, there is no exclusive and definitive answer. Nevertheless,definitions abound,and we offer a small selection here just to establishan initial framework (to be firmed up in Chapter 3 by when some key implications and manifestationswill have been discussed).Brumfit and Mitchell (1989: 6), for instance,start with the idea that 'a "researchingattitude" may be defined as the systematization of curiosity', a view echoed by Boomer (1987: 9) who writes: 'Researchis simply institutionalisedand formalised thinking. It is doing self-consciously what comes naturally'. Nunan (1992b: 3) offers the following more concretedefinition: 'Researchis a systematicprocessof enquiry consistingof 3 elements... (I) a question,problem or hypothesis

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

INTRODUCTION

3

(2) data (3) analysisand interpretationof data'. This is a useful view but, as we shall see during the courseof this book, further questionswill be raisedas to whetherall threeare in fact necessary,and if so in what varying sequencesthey might appearin any individual researchdesign. Stenhouse (quoted in the Rudduck and Hopkins, 1985, collection of his writings) considers that 'researchshould underwrite speculationand undermine assertion' and, more contentiously (Stenhouse, 1988) 'using research means doing research'. A broad view is taken by Stake (1995: 97): 'Researchis not just the domain of scientists,it is the domain of craftspersonsand artists as well, all who would study and interpret'. The Guardian (8 November1995) has yet anotherperspective:'reform [in education,law, social housing... ] without researchis a dangerousexercise'.As a final quotation for purposesof this introduction, Hopkins (1993: 9) conceives of research- by teachers- as 'systematicself-con!)ciousenquiry with the purposeof understandingand improving their practice'.There are obviously a numberof recurring thematicstrandshere, particularly to do with formalization,self-awareness, change(Hopkins' 'improvement'),questioning and accessibility.A short brainstormfor the term 'research'amongthe authors'colleaguesand studentsturned up similar points, and revealeda numberof beliefs and assumptions:systematization;problem or data first; the testingof hypotheses;proof; objectivity; researchrelevance;replicability; going public on outcomes;accountability;and the distinction between research,evaluationand development. Clearly, there is a multiplicity of perspectiveswhich are not amenableto a ready-madedefinition nor to a consensus.For the moment, then, the readeris invited to note the rangeof associationsof the term, perhapsto add his/her own to the list, and then to hold in at least temporary abeyanceany firm commitmentto one particularview. This book is divided into two main sections.The first part is concerned with 'approaches':it is primarily concernedto examinethe various interpretationsof the notion of researchin a teachingcontext, and to relate the discussion to the broader framework of research issuesand traditions. Chapter 1 is largely descriptive, using a few disparatebut typical EFL teachingsituationsto illustrate the contextsin which researchissuesmight be formulated as well as to give examplesof teacherscarrying out research projects of their own. Chapter 2 attempts to delineate the 'teacher researcher'notion, with its possibilitiesand limitations, and looks too at the central conceptof 'action research'.Chapter3 then broadensthe discussion, reviewing a rangeof possibilities for defining researchand evaluating ways in which particularresearchparadigmsand procedurescan be said to reflect attitudesto the discoveryand applicability of knowledge.It will also unpackthe terminology of 'quantitative'and 'qualitative'research.Chapter 4 is concernedwith a numberof key researchprinciples, and examinesthe extent to which different kinds of researchaddressthem, paying particular

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

4

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

attention to the controversialissueof whether researchoutcomesneed to be generalizable.Looking ahead to Part 2, Chapter 5 examinespossible areasof researchfocus for the EFL teacher,and considersthe processes wherebyresearchquestionsand topics might be generated. The secondpart of the book is directly concernedwith both topics and methods.After a scene-settingchapter,eachsubsequent Chapter discusses and exemplifies researchtechniquesthat we argue are consideredappropriate and feasible for the English languageteacher,taking into account the qualitative-quantitativespectrumand arguing that no one method, whetherexperimentalor descriptiveor anythingelse,is automaticallyto be preferredor hasa monopolyon the truth. It all dependson teachersthemselves - their values and objectives, their working environment, their views of their own role, their underlying professionalattitudes,and what they want to know.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Part 1: principlesand perspectives

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

This page intentionally left blank

1

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Teachersin action

Introduction A worldwide professionlike that of TEFL is much too richly diverseto be neatly capturedin an introductory chapterof this kind. Rather, the aim here is to offer a small variety of teachingsituations- 'instances'- which, if not representative,at least give a flavour of teachers'working environments and professionalconcerns.These 'instances'are not case studies carefully chosento illustrate modelsand possibilitiesfor teacherresearch, and much of the rest of this book is anywaygiven over to this goal: indeed, in a number of theseinstances,'research'is not obviously centre-stageat all. Our starting point, then, and a theme that underpinsour arguments throughout,is a belief that researchin languageteachingmust be predicated on an understandingof a wide rangeof contextualvariablesthat will interact with and even determineboth researchperspectivesand research methodology: they may of coursealso help to explain an absenceof any researchinterestby participantswhatsoever. We hope to be able to show from the six instancesbriefly describeda little later in this chapterthat researchpossibilities for English language teacherscan be seen on a broad spectrum.At one end there are wellformulated researchquestionswhich are then implementedas a concrete set of procedureswith an actual researchoutcome.At the other end, and undoubtedly representingthe majority of teaching situations, research remainsan unrealizedpotential, though we would arguestrongly that this research potential is in fact inherent by definition in every context, becauseno classroomand no group of peopleworking togetheris without problemsto solve, questionsto resolve, grey areasto clarify and development areasto pursue.We have seenin the introduction to this book that a minimalist view of a 'researchstance' requires the systematizationand formalization of professionalissuesout of the complexity of day-to-day action: the idea that researchpossibilities are embeddedin any teaching situation is well expressedby Fujiwara (quoted in Richardsand Lockhart, 1994: 81),-a teacherwho writes: 'It is only when I look at thesevisions [of the class] that I can begin to analyze why I am doing what I'm doing ... so my planning process is based on layers and layers of assumptions,

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

8

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

experiencesand knowledge. I have to dig deep down to find out why I make the decisionsI do.' We shall also come to see this spectrumas the book developsin termsof the crucial conceptsof 'reflection' and 'action'. We now turn to a short discussionof teachers'roles and contexts,which leads to the illustrative 'instances'each with a short commentary.In the final part of the chapter we draw out their key implications as far as researchis concerned.

Contextand roles The notion of a 'context' is at the sametime a simple and yet far-reaching one. It providesa frameworkin which to describethe diversity of approaches and methodsin languageteachingand their more specific and local interpretations (see, for example, Richards, 1985). It goes some way towards explaining the possibilitiesopen to teachers,their freedom to manoeuvre,as it were, as well as the inevitable constraintsand pressuresupon them. It is through thisroute crucially linked to the teachers'perspectiveson research, whetherin terms of researchopportunity,contentareas,or methodsacceptable and available.To stressthe importanceof contextualreferencepoints for a greatdeal of the researchactivity carriedout by teachersis also to confront directly such problematic and controversial issues as generalizability and validity. Definitions of 'context' also connect paradigms in educational researchto their foundationsand derivationsin sociologyand anthropology. We shall be exploring thesebroaderareasof principle in subsequentchapters in this part of the book, and the implicationsof what Bryant (1993: 3) neatly refersto as the 'readingof settings'. Context

For the moment it will be useful to note the very large numberof factors operatingat national,institutional and classroomlevel (Malamah-T homas, 1987) that cluster togetherin various permutationsto give eachcontextits own particular set of characteristics.There are factors to do with the setting itself, which include: • • • • • • • • • •

sourceof policy decisions statusandtraining of teachers role of English in the country and the curriculum time available physicalenvironmentof classroomand school student:teacherratio classsize resourcesavailable anticipatedmethodology choiceor imposition of coursebook

and so on. Other variablesconcernthe learnersthemselves,for example:

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

TEACHERS IN ACTION

• • • • • • •

9

proficiencylevels age interests motivation and attitude needsand goals learningstyles mothertongue.

(For a fuller discussion,seeMcDonoughand Shaw, 1993.)

Roles Although a teacher'srole is unlikely to be unique, it will clearly be influencedand in varying degreesdeterminedby the natureof the context in which helshe works. All human beings (except presumablyhermits and recluses)operatewithin a 'role set' inhabitedby others(Handy, 1985), and a teacher'stypical professionalrole set will therefore include colleagues, students,seniorstaff, secretaries,technicians,parents,sponsorsand so on. This interactivenetwork affects not only the details of a job specification but also individuals' perceptionsof their roles. Role theory is a complex subject,and this is certainly not the forum to elaborateon it. An awarenessof 'role' is, however, important for the present discussionbecausea teacher'sprofessionalactivities - researchpossibilities thereforeincluded - are conditionedby what is expectedof them and how they seethemselves,through socioculturalnorms,rules and regulations, attitudes, statusand training. Classroominteraction patterns,for instance,may be basedon a norm of social distancebetweenteachersand learners(Wright, 1987; seealso Richardsand Lockhart, 1994), or alternatively one of a student-focusedenvironment,so the kinds of issuesthat might be prioritized and investigatedby teachersare likely to be very different in eachcase.Again, the apparentlystraightforwardmatter of where researchis initiated and how it is subsequentlypursuedwill be related to the teacher'sposition in the educationalhierarchy, as will the associated possibilitiesfor changeand innovation. This, then, is the framework for teachers' professional actions and of the relative importassumptions,and a startingpoint for an assessment ance and appropriatenessof different approachesto researchin TEFL. The 'instances'that now follow are intendedto show that typical teaching situationshave researchimplications of many disparatekinds, determined to a considerableextent by the variablesof context and teacherrole. The authorsoffer a brief commentaryto accompanyeach, and readersmight also wish to draw their own implications. (It must be added here that no one type of situation is necessarilyrestrictedto, or characteristicof, a particular geographicalarea,and no suchimplication is intended.)

10

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Instances

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

For eachof the instancesdescribedherethe authorsoffer a brief commentary, intendedsimply to highlight points that seemto us to standout: readers, of course,may well makeother inferences. Kenji Matsudahastaughtin aJapanese Junior High School for eight years. He teachesfor about 18 hoursper week, Monday throughSaturday,and, like all his colleagues,has a rangeof ancillary duties related to the smooth running of the school. There are regular staff meetings,and quite a lot of the time is spent discussingthe new syllabus and the revised coursebooksthat accompanyit. This new syllabus derives from a policy decision made at Ministry of Educationlevel that the country'sEnglish-languageprogrammes should be designedto introduce a more communicativemethodologyinto the school system alongside the rigorous teaching of English grammar. Examinations neverthelessremain an important element. K.M: uses both English and Japanesein the classroom,the latter particularly as a metalanguagefor classroominstructionsand linguistic explanations,as well as occasionally for the translationof new vocabulary.There are 35 studentsin his class. He sometimeshas the opportunity to work alongsidea native speaker AssistantEnglish Teacher(AET), who visits the school for two weeksevery three months on a peripatetic basis and who acts primarily as a language informant and 'resourceperson'.K.M: is occasionallyrequiredto attendtalks at a local teachers'group, the most recentone given by a teacherjust back from a training coursein the USA. K.M: is himselfinterestedin this scheme, arrangedat Prefecturelevel throughthe Ministry. Comment: On the face of it, KM's working environment is relatively constrained,and overtly 'top-down' in the sensethat main coursematerials derive from centralizeddecision-making.Examinationsare important. This teacheris also very busy 'at the chalk face' for six days a week. There is neverthelesssome space for interpreting the national syllabus, and plenty of opportunity for peer contact and disseminationof information within the school and also locally. A chancefor 'time out' to study abroad may be available,and it is noticeablethat KM would be motivated to take advantageof this. Ann Barker worked for several years on short contracts in various Europeancountriesand then, after successfullycompletingthe RSA Cert TEFLA,' was offered a longer-term post in a private languageschool in Oxford teaching 25 hours per week. The school mainly organizesyearround courses for adults, with a big summer programmefor younger learners.Managementis currently looking at the possibility of developing , Royal Socicty of Arts Ccrtificatc in TcachingEnglish as a Foreign Languageto Adults. One of a small numbcrof initial qualificationsfor teachingEFI., and now administeredby the icate Uni\·crsityof CambridgcLocal ExaminationsS~·ndicatc.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

TEACHERS IN ACTION

11

more specializedprogrammes,particularly - following suggestionsmade by teachersinformally and at staff meetings- in English for doctorsand for businesspurposes.Within the school's budgetary constraints,some teachersmay be given a reduction in teaching load in order to develop these new courses.AB's original background in nursing makes her an obvious candidateto contribute to this initiative. The school is a member of ARELSl and of the local EFL teachers'association,so it is at least possible to find out what other languageschoolsoffer in theseareas,with obvious commercial restrictions. The school does not belong to any other national organization,and doesnot subscribeto EFL publicationssuch as the EL Gazetteor ELTJournal, so accessto a broaderinformation baseis somewhatlimited. Comment:AB is on quite a typical EFL careertrack for native-speaker teachers,moving from the gatheringof somelimited experience(probably immediatelyafter graduation)to an internationallyacceptedqualification, the first step on a recognizedroute into the profession.She has an interestingbackgroundin a different work areawhich, juxtaposedwith TEFL, suggestsa possible niche in ESP (English for Specific Purposes).It is worth noting that the school is willing to provide developmentopportunities, though so far in a rather limited way, and also that managementis responsiveto proposalsfrom the teachers'group - an incipient symbiosis of individual and institutional opportunities.

Irina Petrov is one of 10 English language teachers attached to the EngineeringFaculty of a large city university in central Europe.Other big departments,such as the School of Medicine, also have their own language teachingstaff. There is no one section in which all English teachersof the university are based,with its own premisesand resources.IP teachesfor 16 hours per week, with an averageclasssize of 15-20,althoughthis is likely to increase.Studentsrange from first year undergraduatesto postgraduates (and even staff) hoping to spenda period in an English-speakingcountry. There is a growing demandfor English throughoutthe country, following the rapid paceof political and social changein easternand central Europe. IP is expectedto produceher own teachingmaterialsgearedto the needsof engineers.She typically writes tasksand exercisesbasedon readingstaken from subject-specifictextbooks, sometimesdrawing on English language teachingmaterial publishedsome years ago and focusing on specialistterminology. The only materialotherwiseavailableis a small collection of commercially producedcoursebooksdonatedby publishersor bought from the Faculty's limited budget. IP has once visited the UK, on a three-week SummerSchoolfor teachers. 2

Ass()ciati()n ()f RecognisedEnglish I.anguageSenices, the largestnational ()rganizati()n()f lan-. guagesch()()ls in Britain.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

12

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Comment:There are a numberof attractiveaspectsto working in a situation like that of IP. Thereis a clear subject-specificfocus and thereforean obviousdirection for researchand development;studentsare highly motivated;and hoursof work and classsize are at the momentmanageable,leaving time outside formal contact hours for courseand materialsplanning. At the sametime the environmentis professionallynarrowedby the lack of feasibility in making contactswith English teachersin other fields, and there is not yet either a clear policy for materialspurchaseand development, nor an opportunity for IP to follow a systematiccourseof training in this area.Expectationsand reality may not be entirely in line with each other here. Antonio Lopesis a tertiary-levelcollegeteacherwho has just returnedto Brazil after spendinga year in Britain completingan MA in TEFL on a scholarshipfrom the British Council. During his year away he studied a wide rangeof subjects,from principlesof languageand languagelearning through to more applied areasof methodology.He becameparticularly interestedin readingskills, and hopeseventuallyto develophis small-scale MA dissertationinto a researchproposal·for a PhD, which will involve large-scalesampling and statistical analysis of data. Meanwhile, he has returnedto his old job, which involves teachingEnglish to young college students.The college itself is one of severalin the region that has been instrumentalin implementinga major ESPreadingproject, which hasalso attracteda lot of external and consultancysupport. Staff are encouraged to participatein the project, to attend and give presentationsat relevant conferences,and to contribute to publications.If AL managesto achieve his ambition of gaininga PhD, he will be eligible for promotionto the university, where his job specificationwill include a researchrequirementin addition to classroomteaching. Comment:AL has had a major opportunity to work in his own field at quite a high level away from the day-to-dayrigours of the classroom,to come into professionalcontactwith ELT worldwide via his peerson the MA programme,and to deal in theories and ideas not always directly related to his work that may simply have been stimulating for their own sake (for some people: for others they may have been felt to be 'irrelevant'). He has obviously becomeinterestedin researchmethodologyas such, partly motivatedby the careerstructurein his own country and the existenceof a major and well-establishedproject. David French is an expatriateteacherworking in a medium-sizedlanguageschool in Korea after a degreeand experiencein other parts of the world. His contract obliges him to teach a variety of classroomcourses mainly for adults wishing to pursue businessopportunitiesabroad, and some who are candidatesfor proficiency tests enabling them to study in North America and Australia in particular. It also requireshim to engage

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

TEACHERS IN ACTION

13

in researchand developmentin areaswhich should loosely be of commercial benefit to the school,and a small part of his working week is reserved for such activity. Many of the courseslead to externalcertification via the Test of English for International Communication(TOEIC), the Test of English as a Foreign Language (TOEFL) and to a small extent the International English LanguageTesting System (IELTS), and company sponsorsalso specify relevant competencewhich they wish their employeesto attain.The school is well resourced,with materialsfrom foreign publishing housesmainly in the USA. There are adequatebut not purpose-built teaching rooms, and plenty of educationaltechnology in the form of languagelaboratories,video facilities, computing servicesand a self-accesscentre. Comment: For some, DF has an enviable work situation. However, the company's enlightened policy on research and developmentdoes not include researchtraining, and the busy timetablemakescollaborativeprojects with other teachersvery difficult to organize.Consequentlyinitiating and sustaining a research project within the school involve major headachesand difficulties, leading to a degreeof frustration amongthose who are interested.Time out in the form of a courseof study leadingto a higher degreeaffords time for re-appraisaland re-orientation,and the nurturingof a researchableproject. Carol Turner teachesEnglish as a SecondLanguageas a special needs teacher in a provincial English town. She is attached to the Local EducationAuthority and goes into whichever schoolsin the area require her expertiseto give additional help to the pupils who havelanguageproblems. There is a substantialnumberof ethnic minority families in the district and some of the children are losing out on mainstreameducation becauseof their difficulties with English. Most of the children need help with their reading proficiency, some also have difficulties with oral work. Shemostly works with individual pupils but runs somegroup sessionsin a couple of schools she visits. As a special needs teachershe works with pupils identified as needingher expertiseby the schoolsconcerned,but the nature of the job is peripateticand she is a (welcome) visitor rather than a core member of staff at any of the schools. She works with the pupils for as long as is necessaryto return them productively to the mainstreamclasses.Shehasaccessto all the relevantrecords,and is able to consult the teachersand the parentsconcerned.She rarely teachesa whole class,but is often invited to staffroomsto advise. Comment:CT, who hasa ratherspecialkind of experience,hasthe opportunity and the inclination to do researchin her situation, but the very opportunitiesoffered by her peripateticaccessto all the schoolsin her district meanshe has very little time. Her own position has been under threat from budgetaryconstraintsand educationcutbacks,but the problems her learners have do not go away, and neither has the number of such

14

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

disadvantagedlearnersreduced.She seesscope for researchinto both the individual students'learningdifficulties and into the institutional systemof supportfor suchpupils in the area.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Patternsof research:implications The professionalworld of teachersencompassesmany and varied attitudes, behaviours,expectations,actors, possibilities, pressuresand constraints.Any specific researchprofile will thereforeinevitably be linked at the leastto questionssuchas who initiates the research;who carriesit out; how if at all its outcomesare reported;its contentand focus; whereit takes place; how many peopleare involved; what the control and decisionpoints are; its timescale; its funding source; and its possible applications. We argue, then, both here and subsequently,that researchinvolvement by practising teachersis not restricted to anyone particular paradigm, but can take placealong a numberof different and interactingdimensions.We offer in this sectionan overview of what can and does happen:Fig. 1.1 is an attempt to sum up the discussionin diagrammaticform rather than offer a comprehensivemodel, and anyway does not at this stage include much commenton methodology. We use simple presentforms in opening assertionsin the interestsof being straightforward,but they should be taken to stand for what is both actualand potential. 1. Researchis sometimescarried out by externalagents.Thesemay be officially appointed, perhapsby a Ministry of Education, to undertakea large-scaleproject acrossa numberof schools.These 'agents' may alternatively be university-basedresearcherswho visit an institution just to collect data, initial formulation and subsequentanalysisand write-up being done elsewhere.Teachersare often the objects of such researchrather than participants,and may well not have sight of the final outcome.There are, however, many examplesof this 'outsider research'that involve teachers more closely both in formulation anddirection - if not in conception - allowing, as it were, their voices to be heard. (The work of Allwright, 1988, is an exampleof this and, as we shall see later, belongs in a tradition of largely naturalistic and interpretive research.) 2. Researchis initiated at a higher administrativelevel outside the institution but is actually carried out by its staff in situ. This is often the casein a centralizededucationsystem,where nationalor local governmentspecifieswhat it wants to find out (such as the efficacy of communicativeteaching methodology, or the merits and drawbacksof newly introducedmaterials).In this caseparticularly there is an obvious issueof the 'ownership'of researchtopic

TEACHERS IN ACTION

15

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

and methodology,in that these are largely out of the control of individual teachers.In someways KM fits into this perspective. 3. Researchis initiated by teachersthemselvesperceiving queries, problemsand gaps in their own practiceand setting about finding solutions.Choiceof topic and methodologyare then up to the individual. This may, either entirely or more usually within limits, be Ministry external agents Researchers

BY WHOM

as directed as individuals ingroups as participants

teachers

FOCUS

needs specifications classrooms individuals, groups institution, management learning styles teaching behaviours language programmes, materials preparing for action recording change

GOALS

change and innovation personal and institutional development appraisal knowledge and understanding public and disseminated private

LOCATION

within institution outside own situation MIOIstry University

RESEARCH

positive ATTITUDES

Fig. 1.1 The full rangeof \ ariahlcs

negative

teachers of

institutions Ministries learners

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

16

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

supportedby the employer,perhapsto the extentof providing facilities, funding, or someteachingtime ofT in lieu. The institution may actually encourageresearchby teachers,both for their and its own benefit, whilst not making it mandatory.AB is one of the teachers who works in this kind of environment,where 'bottom-up'opportunities areavailable. 4. More negatively, some working situations discouragea research stanceamongstaff, implicitly if not even overtly, whetherbecause of perceived lack of time, attitudes to 'research'in relation to other duties, disruption to teaching schedules,hierarchical barriers and so on. This particular scenariois wryly documentedby Walker (1985: 192-3), togetherwith a list of suggestedstrategies to circumventsuchapparentconstraints. 5. Equally negatively, the whole idea of 'research'is rejected by teachers(even, as with DF, where the school encouragesit), becausethey feel it to be 'irrelevant'1:otheir practiceand as something that 'researchers' do - the 'big R', in Boomer's(1987) terms. More particularly, they lack a broad and articulatedview of what researchis and what it can do for them - and what they can do which will 'count' as research.This perception,which compartmentalizesteachingand research,is significant and widespread.It has also been pivotal in the re-orientation of educational and applied linguistic researchthat we start to explore in detail in Chapter2. 6. Researchis sometimescarried out by teachersworking alone, sometimeswith one or more peersin a researchteam, and sometimes collaboratively with outsideresearcherswhere the differing perspectivescome together in the formulation of an agreed researchagenda.Learners,too, may be included in the enterprise. (A numberof possibilitiesfor suchtwo- and three-wayresearchin ELT are set out in Nunan, 1992a,for example.)It is characteristic of many teacher-research projectsthat they are institutional rather than individual, sometimesinvolving collaboration across state and national borders.Many of the benefitsfelt by the participants accruefrom the experienceof working on commonproblemswith distantcolleagues. 7. Research,as was flagged in the introduction to this book, takesas its object of study classrooms,individuals, groups, the whole school environment, managementstyles, self and colleagues;it focuses on learning styles and strategies,learner characteristics, teachingmethodsand behaviours,whole programmesor components,language,materialsand so on. 8. Researchis both individual and private, on the one hand, and in the public domain, on the other. In other words, teachersmay·

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

TEACHERS IN ACTION

17

undertakean investigationfor themselvesin their own classroom, or they may considerthe outcomeor indeed the processto be of sufficient interestto sharewith others in an appropriateforum meetings,conferences,journalsand the like. 9. There are many possible motivations for undertaking research, including a wish to effect changeand innovation, howeversmallscale,courseevaluation,personalor institutional development,or simply a desirefor knowledgeand understanding.Someexternally controlled researchis, of course,also driven by an appraisalethos wherestaff themselves are monitored,evaluatedand judged. 10. Teachersalso do researchoutside their working environmentby, for example, undertakingpostgraduatestudy at Mastersor PhD level. Such researchmayor may not be directly related to their own situation and mayor may not be intendedto be taken back and applied there. In AL's case, the study period abroad has served pragmatically to satisfy promotion chancesand to feed directly into an ongoing large-scaleproject, but also more broadly to encouragepersonal intellectual development.Such research may not, however,always be doneentirely on a teacher'sown volition, in the sensethat it is just anotherrequirementattachedto their job description. Moreover, to the extent that it forms a researchtraining, the methodsand format required of a doctoral candidatemayor may not be suitable for conductingcontextual researchin a working teachingenvironment.

Conclusion This chapter has attempted to locate the discussion of appropriate researchmethodsand to follow it within a view of the languageteachers' world, with particular referenceto English as a foreign/secondlanguage. It has recognizedthe diversity of that world by a small set of examplesof teachersin action, who are of coursefictitious, but who exhibit characteristics of context,opportunityand motivation which will be recognizedby a number of real acquaintancesand former studentsof ours. These examples could easily be multiplied, but they serve to illustrate several importantfeaturesof the role of researchand the attitudesof teachersand employersto researchin the real world of institutional languageteaching in many countries. Even these six examplesshow a striking diversity of opportunitiesand facilities for doing research,and incidentally of the roles of higher qualificationsin this area.Someworking contextsallow accessto a single kind of classroom,othersto a variety; someto one kind of course and teaching style, others to a variety. Some institutions are willing to sponsorresearchand give time but wish to seea tangible benefit, usually in terms of incomeor perhapsprestige;someregardresearchas a wasteof

18

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

income-generatingtime. Some teachers have the opportunity but no motivation for research,othersmotivation but no clear idea of what to do; othershaveprojectsthey would like to follow but no opportunity. Put another way, the chapterhas tried to highlight the significancefor what follows of such termsas the following.

Ownership Who owns the ideas and the results of research?How does a classroom teachercometo feel ownershipof a curriculum change?

Initiation Getting a piece of researchoff the ground, having the idea, persuading colleaguesto collaborate,obtaining sanctionsand accessfrom authorities, requiresmore time and energythan many teacherscan spare.

Control Both in the senseof taking control of one'sown teachingsituation, to the extentthat is feasible,and in the senseof the discipline and staminanecessary to carry out a researchproject.

Relevance Much Researchwith a big R is perceivedby practitionersas irrelevant. This may be becausethe original connectionsof the researchwith practice have disappeared,or becauseit never originated in a classroomproblem but within a relateddiscipline. Teacherresearchcan addressrelevantproblems in their own context.

Subjects This may refer to topics of interest, or more perniciously to the people from whom data is elicited.

Change Much teacher researchis about effecting and documentingchange, 10 practice,in institutional systems,in methods.

Knowledge All researchis about finding out new knowledge; whether it

IS

highly

TEACHERS IN ACTION

19

context-specificor generalizabledependson the aims and the methodsof the research.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Discussionnotes NB Both here and throughout the book, discussion notes are offered simply as a stimulus to thinking further about the issues.They are not intended to be worked through rigorously, and should be regardedas an 'optional extra'. 1. Try to briefly describeyour own teachingcontextor one of which you have experiencealong the lines of the six 'instances'set out in this chapter.Then constructyour own commentary,highlighting both the most interesting and indeed frustrating aspectsof the job, and then reflecting on the researchopportunitiesinherentor perhapsexplicitly availablein that situation. If you are working in a group togetherwith other teachers,you might find it fruitful to have someoneelse read and commentyour own 'instance',and to do the samefor them. 2. What is your own position in relation to the much-quotedidea of research relevance? For example, do you have experience of researcheither yourself or as done by others?Do you feel that for teachersin our field there is some kind of sliding scale of 'relevance'?And - if so - what kinds of topics would fit in at various points of your personalspectrum?

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

This page intentionally left blank

2

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

The teacher researcher in focus

Introduction A view of the teacheras an extendedprofessionalwhoseportfolio of activities can in principle encompassan enquiringattitude leadingto researchis an intuitively attractiveone. In its most basicformulation, it carriesthe clear implication that 'research'is not only somethingthat is done 'on' or 'to' teachers,but is also an undertakingin which they can themselvesbe actively involved, by for exampleidentifying interestingor problematicissuesand topics, choosing suitable investigative instruments,and pursuing answers and outcomes.Teachersare thereforemore in control in sucha perspective, closer to the sourcesof decision-makingand - in the current jargon - have greater'ownership'of their own professionalenvironment. We shall see in subsequentchapters of this part of the book that researchby teachers,both in practiceand in its theoreticalunderpinnings, is complex and indeed controversial,directly impinging on such fundamental researchquestionsas validity, generalizabilityand the natureof its contribution to a wider store of knowledge.Thesechapterswill also comment more fully on derivationsfrom researchin anthropologyand social sciencein particular. First, however,it is necessaryin this chapterto show that researchby teachersis embeddedin a numberof broadertraditionsin education,and in the course of the presentdiscussionwe shall be concerned with the much-mentionedquestion of the relationship between 'theory' and 'practice' and the 'relevance'of the former; with the nature and role in researchof teachingconceivedof as 'action',and with the possibilities inherentin teacherresearchfor effectingeducationalchange. The presentchapter,then, is largely descriptiveand historical, the delineation of a concept. First of all we set the sceneby examiningthe crucial notion of 'reflection' in helping to define what is meant by researchin a practical professionalcontext.There follows an overview of the background to someof theseargumentsin mainstreameducation,including the important tradition of 'action research'.The last main sectionsof the chapterdiscuss and illustrate teacherresearchin our own field of ELT, including an

22

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

examinationof how key conceptshave startedto permeateteachertraining and educationprogrammes.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Action and reflection Brumfit and Mitchell, in their introductory chapter to a collection of papersthat takes a very broad view of researchpossibilities in language classrooms,make the following assertion:'There is a particularly strong contradictory pull [between researchand teaching] in that researchis a type of contemplation... while teachingis a type of action' (1989: 10). This is a standardand much-cited polarization (and incidentally representsa view to which the Brumfit and Mitchell volume doesnot on the whole subscribe): it places the terms 'action' and 'contemplation'(or 'reflection'), teachingand research,in two quite distinct and incompatibleuniverses.We can, on the one hand, visualizea teacherwhoseonly concernis to follow a real-time lessonplan to take his or her lower intermediateclassthrough to the next level, and, on the other, a researcherusing amasseddatato develop somenew elementin the theory of secondlanguageacquisition.This is, of course,a caricatureof a situation in which teacherand researcherapparently have no meetingplaceand quite different agendas. The following sectionshowshow the terms'action'and 'reflection' have beennot only juxtaposedbut amalgamatedto provide a principled foundation for researchby teachers:the whole, in other words, will be seento be more than the sum of its two constituent parts. The argument hinges around the idea that reflecting on action allows the developmentof a critical distancefrom the real-timeaction itself. Reselln-hliS re.flection The terms are now consideredhere specifically from a practitioner'sperspective. An obvious starting point for exploring the interface between action and reflection is teachers'professionalknowledgeabout the central aspectsof their jobs. The precedingchapteroffered a range of possible 'action contexts' for the ELT profession, from which it can readily be gleaned that teachers'know about' coursebooks,other materials, audiovisual resources,syllabuses,proficiency levels, examinations,working with colleagues,classrooms,methodology,individual learners,and many other aspects.Bolster (1983) characterizesthe nature of much of this kind of knowledge as 'idiographic' and 'particularistic', primarily concerned,in other.words, with the uniquenessand specificity of eventsand individuals in one's own professionalcontext - although knowledge of materials,or examinations,and so on is of coursepotentially transferableand not only situation-specific_Teachers,then, are not on the whole concernedwith

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

THE TEACHER RESEARCHER IN FOCUS

23

large-scale comparisonsacross different situations. Rather, as Bolster (1983: 298) puts it: 'Every teacher... knows that although thereare many similarities betweenclasses,eachgroup has its own specialcharacteristics, and that successfulteaching requires the recognition ... of this uniqueness.'This should not be taken to imply that teachersare by definition unableto deal with other kinds of knowledge,merely that everydayprofessionallife hascertain typical epistemologicalattributes. It is essentiallyfrom this knowledgebaseof everydayaction that Schon (1983) developedhis seminal theory of the 'reflective practitioner' in the book of the samename.His work, althoughconcernedwith the professions in general,has beenextremelyinfluential in the formulation of the teacherresearcherparadigmbecauseit lies at the root of the idea that reflection is much more than just 'thinking about': he hasshownthat it can be rigorously conceptualizedto provide firm and appropriateresearchprinciples that are in tune with the teacher'sreality. Schon is critical of a view (of 'technical rationality') that assumesthat theory merely needsto be applied in order to be of use to practitionerson the grounds that 'there is a swampy lowland wheresituationsare confusingmessesincapableof technicalsolution' (p. 42). He arguesinsteadnot that scientific method is inherently 'wrong' but that the unique and the particular are best dealt with by converting a professional's tacit knowing-in-action to an explicit reflection-in-action, so that 'when someonereflects-in-action,he becomesa researcherin the practice context'(p. 68). This is the amalgamationreferredto at the beginningof this section, where doing and thinking are interwoven.There are parallels here with Giroux's (1988) view of the teacheras a 'transformativeintellectual',a participantengagedin critical thinking: he arguesstrongly in favour of 'theory' to 'enableteachersto see what they are seeing'. In terms of our own professionthis might imply, for example,a teacherbecominginterestedin the learning habits of an individual and carrying out a casestudy (and then perhapsothers),when a more generalizedtheory of secondlanguageacquisition patternswould be less obviously applicable.Part 2 of this book will be exploring in much more detail the translation of these general principles into viable researchmethodologies:reflecting is the basis for research,not the researchitself.

Theory. practiceand relevance It might be useful to take a brief pausefor thoughtat this point. On the face of it, the dialecticalrelationshipof researchto action that has just beenbriefly outlined seems to challenge head-on the frequently referred-to, almost cliched dichotomy of 'theory versuspractice'.It is also commonto describe it as a relationshipof re/lexivily, where elementsare mutually illuminating (see, for example, Bolster, 1983; Hammersleyand Atkinson, 1983). The

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

24

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

dichotomy, on the other hand, is usually expressedin negativeterms, and can be experiencedas resistanceto what is perceivedas professionallyirrelevant: a lectureon schematheory (for instance),howeverintellectuallystimulating, doesnot inherently reachthe teacherqua practitionerin his or her day-to-day sphere of action, nor are articles on universal grammar (for instance)of obviousimmediateapplicability when planningtomorrow'slesson for the lower intermediategroup working throughHeadway. The situation describedhere, however, is still not entirely straightforward. First of all, there are increasingsigns that the reflective paradigmis itself forming the basis for a new and unhelpful dichotomy where teacher research(or 'action research',as will be clarified in the next section) is being advocatedin opposition to researcherresearchand technical rationality. We discussthis in more depth in Chapter4. Second,if the notion of reflection in action as the basis for researchon practice does challenge researchersto redefinerelevanceand application,at the sametime it undermines an attitude to teachingthat seesit merely as 'craft knowledge'to be acquiredas a set of usabletechniques:in our own field, how to set up a role play, or deal with errors,or activategrammar,and so on. This point is particularly well madeby Elliott (1991: 45-8). In a short chapterentitled 'The theory-practiceproblem', he tries to explain why teachersfeel threatened by theory. (1) It symbolizes the power of the researcherto define valid knowledge(2) this knowledgeis often couchedin the form of generalizations so that localized experienceis invalidated;and (3) there is an implication that teachersare inadequatebecausethey do not conform to idealized modelsof curriculum.Thereis, however,a sting in the tail in Elliott's argument, because'theory' and 'research' protect teachers'practicesas individual realms of private, esoteric, intuitive craft knowledge, and it is preciselytheseconservativevaluesthat are also threatenedby what he calls 'the new conceptof professionalismembodiedin the action-researchmovement', the 'counter-cultureto the traditional craft culture', and the complex belief systemsthat are built up to becomepart of the professional ethos.What Elliott is taking issuewith is the ideaof craft knowledgeas sufficient in itself, not of coursewith action-knowledgeas generativeand as the basisfor research.Furthermore,this 'idiographic'kind of knowledgeis concernedwith skills and techniques,but also with the judgementneeded to apply them appropriately(Pennington,1990). From anotherperspective, it can also be classifiedinto subject-matterknowledge(grammar,functions, topics, situationsand so on in our field) versuscontent-independent 'action system' knowledge (Day, 1990). In other words, we need to be wary of regarding 'craft knowledge' as a monolithic concept. Elliott's view resonates in Schon: 'When practice is a repetitive administration of techniquesto the samekinds of problems,the practitionermay look to leisure as a source of relief, or to early retirement; but when he functions as a researcher-in-practice, the practiceitself is a sourceof renewal'(1983: 299).

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

THE TEACHER RESEARCHER IN FOCUS

25

To return now to the central purposeof this chapter,it will be useful to recap the argumentsfor teacher-initiatedresearch,becausethey are a starting point both for the remainderof this largely descriptive chapter, and for our subsequentdiscussionson the parametersof researchas a concept. Theseadvantagesare clearly set out in Beasleyand Riordan (quoted in Nunan, 1989: 17-18)and are cited (almost)in full here: • It begins with and builds on the knowledge that teachershave alreadyaccumulated. • It focuseson the immediateconcernsof classroomteachers. • It matchesthe subtleorganicprocessof classroomlife. • It builds on the 'natural' processesof evaluation and research which teacherscarry out daily. • It bridges the gap betweenunderstandingand action by merging the role of researcherand practitioner. • It sharpens teachers' critical awarenessthrough observation, recordingand analysisof classroomevents. • It helps teachersbetter articulateteachingand learning processes to their colleaguesand interestedcommunitymembers. • It bridgesthe gap betweentheory and practice.

A shorteducationalexcursion The teacher-researcher movementhas a much longer tradition in mainstreameducationthan in ELT: as long ago as 1970, for instance,Caneand Schroederpublished a monographanalysing via questionnairedata the kinds of researchteacherswould like to seecarried out and ways in which they might themselvesbecomeinvolved. Although their work focused as much on teachershaving a say in researchdoneby othersas on the teacher as active researcher,it laid important foundationsfor subsequentdevelopments. It is, then, the educationalbackgroundthat has largely provided the feeder into current work in ELT, and this section will now set out someof the key factors in that process. In the UK, probably the major figure in stimulating both the principle and practice of teacher researchwas Stenhouse,formerly Professorof Educationat the University of East Anglia (itself one of the best-known centresinternationally of applied educationalresearch).As expressedin his seminal book An introduction to curriculum research and development (1975), Stenhousewas very critical of an end-product,objectivesmodel of educationbecause'it assesseswithout explaining' (p. 120). He advocated insteada processperspectivewhich takesknowledgeas 'the focus of speculation, not the object of mastery'(p. 85), in other words as evolving rather than preformed. From this he developed a view of the teacher as an extended,not a restricted,professional,engageddirectly in the discovery

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

26

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

and creationof knowledgethrough the medium of the school curriculum and their own involvementwith it. Research,then, is 'the meanstowardsa disciplinedintuition, fusing creativenessand self-criticism' (p. 223). A greatdeal of the discussionof teachersas researchershas beenclearly allied to the conceptof action research.Although this linkage is a rather loose and somewhatdiffuse associationrather than an equation,so much educationalresearchhasbeencarriedout underthis headingthat it will be convenient to conflate the terms at this point as essentially belonging within the sameparadigm.

Action research: backgroundandprinciples The field of action researchis a broad and complex one, and there is only spacehere to pick out the featuresthat havebeenmost salientin educational terms, although its philosopqical underpinnings and its expression in researchmethodswill be evident in subsequentchapters.The term itself is usually attributedto Kurt Lewin, a Germansocial psychologistwho, working in the USA in the 1930s and 194Os, sought to develop a researchmethodology basedon people'sreal-world experiencethat he felt experimental methodswere unableto address.He is particularly quotedfor his work with factory apprentices,where he was able to show the highestlevel of output in the subgroupthat was allowed to formulate its own 'action plan' of production: this was felt to provide a persuasiveargumentin favour of group decision-making(Adelman, 1991; McNiff, 1988). Lewin was also concerned with other kinds of social groups,including the subculturesof USA city life: it is important to note that action researchhas maintainedthis broad framework of reference,and is applied in many areasof work and social life as well as education(hospitals,prisons,and so on). Following Lewin, there hasbeen much discussionof definitions and modelsof action research.Centrally, it is conceivedin termsof a self-reflectivespiral, or cycle: Initial idea -+ fact-finding -+ action plan -+ implementation-+ monitoring-+ revision -+ amendedplan -+ and so on throughthe cycle. To put this briefly in context, in ELT this could be taken to mean, for example,that a teacheris concernedabout apparentlydifferent uptakeon coursebookactivities, collects data via observation,field notes and questionnaires,decic\esto vary the sequenceof presentation,monitors success rates and quality of response,and so on until some useful changeshave beeneffected.(A useful discussionof the principal models- Kemmis and Mc'Iaggart, 1988; Elliott, 1991 - can be found in Hopkins, 1993: Ch. 4). The most quoted (even over-quoted)definition of action researchis that of Carr and Kemmis: 'a form of u(F-re./leclive enquiry undertaken by

THE TEACHER RESEARCHER IN FOCUS

27

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

participants in social situationsin order to improve the rationality and justice of their own practices,their understandingof thesepractices,and the situationsin which the practicesare carriedout' (1986: 162, italics added).From this definition, at least indirectly, flow the essentialcharacteristicsof 'pure' action research: • it is participant-drivenand reflective • it is collaborative • it leads to change and the improvement of practice, not just knowledgein itself • it is context-specific. These by no means always obtain so strictly, which is one reason why action researchand teacherresearchmore generallyoften take on convergent characteristics.

Someapplications The educationalliterature contains large numbersof reports of teacherresearchand action-researchprojects, both modest and ambitious. Many derive directly from teachersthemselves(see, for example,the reports in Hopkins, 1993; Goswamiand Stillman, 1987; Hustler el al., 1986). Others -a little paradoxicallyperhaps- have been generatedand run by educational researchers.Among the best known of these are the large-scale HumanitiesCurriculum Project, initiated in the 1970sby Stenhousewith the overall aim of changingteachingstrategiesto give pupils more involvement in their own learning. Another example is provided by the Ford TeachingProject(Elliott and Adelman, 1975; Elliott, 1991), primarily concerned with the implementationof discovery methodsin the classroom; and yet another by the Open University/SchoolsCouncil Curriculum in action project (1981), concernedwith the kinds of questionsteacherscan ask, and the techniquesthey can use, to investigatetheir own classrooms. A numberof theseresearchers,particularly Elliott, were also instrumental in the foundation of the ClassroomAction ResearchNetwork (CARN, University of EastAnglia) which, as its nameimplies, disseminatesinformation on teacherresearch. Much of the educationalresearchthat we have touchedon here is selfevidently concernedwith teachersand learnersin classrooms,often with microstudies of very specific aspectsof this context. However,.action researchin particular is also concernedwith broader curriculum issues, and often with the administrationand managementof schoolsand institutional change(Hutchinsonand Whitehouse,1986; Lomax, 1990). To take us into the final section of this chapter,it is instructive to note the kind of languageused to explain the central tenetsof action research. A glanceat the extensiveliteraturerevealssuch termsas:

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

28

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

• • • • • • • • • • •

empowerment hierarchy power emancipation subversive change statusquo enfranchisement reactionary intervention destabilization.

This, then, is someof the thinking that hasinformed mainstreamnational educationand its associatedtraining patterns.We now turn to an examination of the implicationsfor ELT.

Teacherresearchand English languageteaching The purposeof this final sectionis to chart someof the ways in which the teacher-research ethoshascometo be expressedin the teachingof English as a foreign/secondlanguage.Partly becauseof the nature of ELT as a worldwide professionoperatingin an enormousvariety of different political, social and cultural contexts,it is, as we tried to portray in Chapter1, a many-headedhydra with obvious problems of generalizationcompared with the national educationsystemof anyonecountry. Clearly the influence on ELT of the 'applied linguistic' tradition has been of paramount importancein terms of the designof teachingprogrammes,and it is part of the job of this book to explore possibilitiesfor the uptakeby teachersof this kind of research.For the purposeof the presentchapter,however,we shall look, first, at the 'reflective' paradigm with particular referenceto teachereducation,and then at how this hasbeenharnessedto the development of and researchby practising teachers,often by invoking action researchexplicitly. We conclude the section by commentingon some of the more problematicissuesraised in the transplantingand applicationof teacher-research and action-researchmodelsin ELT.

Reflectivemodelsin languageteachertraining It is usual to make a distinction between'teachertraining' and 'teacher education',often in parallel to pre- versusin-servicetraining. Looked at in anotherway, teachersare expectedto acquire both a trainable 'repertoire of skills' (Pennington,1990) and more generativelythe educatedjudgement to apply and transferthoseskills. Elliott commentsmore critically on the mechanisticaspectsof competency-based training, quoting Pearson's

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

THE TEACHER RESEARCHER IN FOCUS

29

distinction between'habitual' and 'intelligent' skill knowledge,the latter allowing for discernment,discrimination and intelligent action. Since a 'reflective' view has influenced both areas, they are conflated here as 'training': in any caseBolster's (1983) analysisof the nature of teachers' knowledge as the basis for research remains appropriate whether that knowledge is already acquired or embryonic. Languageteachertraining programmes,then - whether short courses,introductory, or at advanced postgraduatelevel - are likely to contain a variety of types of input from explanationof techniques('craft knowledge'), such as use of visuals, or using classreaders,to information on applied linguistic researchoutcomes ('technical rationality'), such as the psychology of learning, linguistics, sociolinguistics.Increasingly,however, attention is being paid to the kind of perspectiveoutlined in the precedingsectionsof this chapter. Wallace (1991), drawing directly on the work of Schon, puts forward threecurrent modelsin languageteachertraining: (1) the essentiallyimitative Craft Model, when trainees learn from experts; (2) the Applied ScienceModel (Schon's'technicalrationality'), a one-wayprocedureoften leading to the much-criticized separationbetweenresearchand practice; and (3) the Reflective Model, where knowledgeis experientialrather than received.Although he arguesthat all three are necessary,particularly the reciprocity between (2) and (3), he is mainly concernedto explore the potential inherentin (3), and much of his book is an attempt to establish this as a coherentframework for training: 'research'is then seenas a logical extension of reflective practice. One might consider, for example, training proceduresfor the introduction of communicativemethodology, or for teachingreadingcomprehension(seeFig. 2.1). It is Wallace's contention that reflective training would be concerned both with a critical evaluationof techniquesin context,and with an understandingof the reciprocity of received knowledgeand classroomexperience(1991: 55). methodology

Communicative

Reading comprehension

Craft knowledge

Pair/group work Role play Building conversations Question and answer Useful phrases Games

Pre-questions Use of pictures Guessing in context Predicting Skimming and scanning Jumbletexts

Applied science

Semantico-grammatical categories Notions, functions Text and discourse Theories of communicative competence

Schema theory Psychological processing models Top-down/bottom-up strategies Perception and cognition

Fig. 2.1

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

30

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

A comparableapproachis offered by Richards and Lockhart (1994), essentiallya pre- and in-service training manual for reflective teaching.It startsfrom the assumptionsthat teachers'know' a great deal about teaching, that their practice is· basedon many assumptionsand beliefs, in other words that all teachershave a 'personalconstruct'but that 'unaware'experience alone is insufficient for development.Arguing for the central importanceof critical reflection, they identify the range of the language teacher'sdomains of operation and propose a variety of approachesto enhance practitioners' explorations of their own teaching. These approaches- observation, questionnaires,audio and video recording, diaries- are amongthosediscussedin Part 2 of this book. Thereare many other reportsin the ELT literatureof the use of the reflective paradigmin teachertraining: see for exampleSwan (1993); Belleli (1993); Ellis (1993); Thornbury(1991). Any discussion of teacher training programmeshas implications at some point for the nature of supervisionof traineesand the supervisor's role, and there are a number of interesting ways in which reflective approachescan be incorporatedin this areaof activity. Obviously a major goal of the supervisor-supervisee relationshipis the learningand improvement of practice- action knowledge- rather than the formulation of overt research plans, but again the notion of reflection-in-action potentially allows the developmentof a researchstanceas an integral part of training. Handal and Lauvas devote a whole book to the relationship between supervisionand reflective teaching.Their main thesis,and one which has clear kinship with the educationalframework and the notion of a 'personal construct'outlined in this chapter,is that 'every teacherpossesses a "practical theory" of teaching which is subjectively the strongestdetermining factor in her educationalpractice' (1987: 9). They promote a model of supervision derived from a counselling approach in which the starting point is this practical theory and teachers'own professionalconditions: teachers then retain some control over, and ownership of, the whole process.Smith (1995) discussesan experimentin introducing guided and self-directed reflection as part of a teacher-trainingcourse. The course involved diary-keeping,planned time gapsbetweensupervisionand feedback for self-reflection, and involvement in action research.Smith comments on the enthusiasmof the traineesand on the levels of reflection they achieved:from practical solutions through theoreticalquestioningto changingattitudesor generaleducationalorientation. This view is echoedstrongly in the related notion of 'clinical supervision', carried out by systematic observation and following a cyclical processof pre-observation(the 'planning conference'),observation,and post-observation(the 'feedbackconference').It is a model that is essentially collaborative rather than more traditionally directive, involving supervisor-traineeteacher in formulating together a hypothesis about

THE TEACHER RESEARCHER IN FOCUS

31

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

problematicareasin the teacher'sclassroompractice,analysingthe lesson, and proposingchangesthat could be implementedin future classes.The procedureis rooted in the teacher'sown needsand problems,and can lead to the systematicinvestigationof aspectsof teachers,learners,classrooms - to researchin the sensethat the term is usedthroughoutthis chapter. Researchand teacherdevelopment

'The shift in terminology from "education" to "development"is significant becauseit marks a shift in perception.'Although Swan (1993: 243) is here talking about student teachersspecifically, she is indirectly making the more generalpoint that teacherresearchhas taken much of its impetus from the congruenceof 'reflection' and 'development'. Over the last decadeor so, the notion of teacherdevelopmenthas becomeprominentin ELT: it portraysteachersas active and questioningprofessionalswilling to reflect and change in a long-term way over the course of their whole careersand not only when engagingin explicit training programmes.This whole areahas beenincorporatedinto various teachers'organizations,one of the largestbeing IATEFL (The InternationalAssociationof Teachersof English as a ForeignLanguage).IATEFL hasa numberof SpecialInterest Groups ('SIGs'), one of which is concernedwith TeacherDevelopment (TO) (McDonoughand Shaw, 1993). Its regular newslettersgive a flavour of the wide rangeof professional-personal areaswhich its title subsumes, including cooperativedevelopment,training programmes,attendanceat workshops, action-researchprojects, self-understanding,research with large classes,staff development,learninga new language,and many more. Somewhat more recently, the linkage between teacher development, teacherresearchand action researchin ELT has been made increasingly explicit, in particular by making connectionswith parallel trends both within and beyond education.The most concreteand significant illustration of this is provided by the collection of conferencepapersentitled, with deliberate ambiguity (if a little grammatical inaccuracy), Teachers Develop Teachers Research (Edge and Richards, 1993). (The second 'TDTR' conferencewas held in 1995 and, at the time of writing, a third is planned for 1997.) The collection - like the subsequentconferencesincorporatesreportson small-scale,personaland collaborativeprojectsby ELT practitioners,but also discussesprinciples and definitions, as well as drawing on teacher-research traditions in mainstreameducationand staff developmentpatternsin, for example,large multinationalcompanies. The current burgeoningof teacherresearchin ELT has, not surprisingly, an international dimension: just the few publications cited in this section so far describework being undertakenin - for instance- Poland, Romania,Cuba,Britain, Uruguay,Estonia,Sweden,Austria, Argentina ... the list could be considerablyextended.Further instancesare provided by

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

32

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

the extensivework carried out in the USA, both in ELT and in mothertongue education (Chamot, 1995; Goswami and Stillman, 1987; Green and Wallatt, 1981; Strickland, 1988). Australia has been particularly prominent in the development of large-scale, collaborative teacherresearchprojects, to a large extent in the context of the Adult Migrant Education Programme (AMEP), described by Nunan (1992a) in an extendedcasestudy.The collection Teachers'Voices(Burns and Hood, 1995) presentsthe published outcomes of a national action-researchproject designedto investigatethe impact of newly introducedcurriculum frameworks. Burns' introductorypaperconvincingly explainsthe reasonsfor the choiceof the action-researchmodel in a context of educationalchange,its associatedmethodology,and the constraints(of time, recognition,expertise) on practisingteachersinvolved in suchresearch. Thereare, then, many instancesof English languageteachersworldwide becominginvolved in researchin their own contextsand classrooms,and it is clear that this is a rapidly growing field. At the sametime, a numberof practitionershave attemptedto examine more broadly the principles on which researchin ELT is based. Brindley (1990) sets out an ambitious 'research agenda' which, although intended initially to be applied in Australia, has wider implications. His paper, after rehearsingthe background argumentsin favour of teacher involvement, discussesTESOL researchin termsof: • • • • • •

initiation, executionand dissemination purposes recipients areasof study planning,datacollection, monitoring follow-up

from the varied perspectivesof teachers,learnersand administrators. For the practising teacher, these published reports of context-bound research projects make stimulating reading, and involvement in such projects is undoubtedly professionaily and personally developmental. Neverthelesswe leave this brief overview on a cautionarynote (and one which looks aheadto Chapter3 where the teacher-research ethosis examined more critically). It is simply this: in encouraginga researchstanceby teachersand arguing for its benefits,there is a needto be alert to the danger of establishinga researchparadigmso 'alternative'that it fossilizesan either/ or attitude in which researcherresearch is too often rejected. Brindley (1990: 8) makes this point almost in passing: 'Both "basic" research... and "applied" research... will have a role to play. Moreover, it is not as if the resultsof basicresearchare nevermobilized to solve practical problems',and he goeson to quote Lett: 'applied researchwithout a theoretical base can be random, inefficient and "Confusing, while basic

THE TEACHER RESEARCHER IN FOCUS

33

researchwithout the requisite applied researchhas little or no positive effect on practice.' We will examine this distinction more closely in Chapter3.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Discussion There are many implications to be drawn from approachesto researchin mainstreameducationfor the field of ELT, and we have also noted some aspectsof the internationalism,even internationalization,of the so-called teacher-researchmovement. However, one of the central tenets of this kind of research,namely the importanceof context, is at the same time problematic in evaluating researchopportunities realistically. Chapter 1 outlined just a few of the working environmentsand roles of English language teachers,and the concomitant differing possibilities inherent in those contextsfor doing researchor simply expandingone's professional activities. ELT takes place in private institutions, colleges, universities, primary and secondaryschoolsthroughoutthe world, so we are inevitably dealing with an enormousdiversity of culture, social structures,political institutions and educationalframeworks that is very different from the kind of nationalsystemfrom which much of teacherand action researchis derived. No one interpretationof the paradigmcan thereforebe universally applicable or generalizable,nor is the paradigm itself necessarilya panaceaalwaysleadingto growth and innovation. A little earlier in this chapter,we noted someof the terminology that is typically usedto characterizeaction research- empowerment,destabilization, changeand the like. Indeed,it is one of the goalsof action researchin particular to challengeentrenchedstructuresof power and authority, to subvert autocratic and top-down decision-making procedures,and 'to [emancipate]individuals from the dominationof unexaminedassumptions embodiedin the statusquo' (Crookes, 1993: 131), a clear echo of Freire's (1972) radical position. Clearly, however,this view of researchis not valuefree but heavily value-ladenso, given that values differ acrosscontexts, self-evidentlycannotbe directly transplanted. Even Somekh's(1993: 37) apparentcompromiseposition would be inappropriate in many contexts: 'To have an impact on institutional development, individuals at different levels in the formal and informal power hierarchiesneedto carry out action researchcollaboratively.Although this is clearly much easierto establishin democraticinstitutions, the problemsin establishingsomeform of collaborativeaction researchin a more hierarchical institution needto be balancedagainstthe difficulties in bringing about developmentand changeby any other means.'Teacherswork within different kinds of managementand institutional structure, and the locus of changevaries accordingly. In Kenji Matsuda'ssituation, for instance,the

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

34

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

fact that decisionsabout innovationsin syllabusare madeat Ministry level doesnot invalidatechangesmadein that environment,nor doesit necessarily mean that individual teacherscannot undertakeinvestigationsinto the pedagogyof their own classrooms.Conversely,a situation where a teacher can more readily effect £hanges does not necessarily mean that those changeswill be appropriateand effective. Roberts(1993) rightly stressesthe needfor more empirical studiesto evaluatethe efficacy in contextof the theoretical claims made for self-directedclassroom-based research.He offers two such studies- one carried out in the UK, the other in Israel - that lead him to be cautious,though not pessimistic,about its generalapplicability. His mixed bagof conclusionsis worth summarizing: • Collaborativeaction researchis just one of several processesthat can enableteachersto look productivelyat their own action. • Changethroughout a whole school via teacherresearchdepends on the extent to which teachershave the power (sic) to effect it, and is only partially underteachercontrol. • Teacher researchcan be illuminative for the classroomcontext evenif outsidechangeis imposedat other levels. In the worldwide professionof ELT, the nature and focus of teacher research,whether actual or potential, are clearly widely divergent, with very variable 'freeroom' (Handal and Lauviis, 1987). It will depend on such interwoven factors as the managementstructureof institutions and educational systems, on culturally and socially determined beliefs, assumptionsand expectations,on teachertraining patterns,on resources available, on opportunities, and on self-perception. Within each local interpretation,teacherscan, as we have seen,investigateclassroominteraction and behaviour, individual learners,colleagues,self, syllabusesand materials,whole school policies and so on, whetherthoseinvestigationsare intended to further knowledge and understanding,or whether they are more interventionistand orientedto change.It will be our task in Part 2 to explorethe methodologiesavailablefor doing this.

Conclusion This chapterhasexploredsomekey notions in the developmentof teacher researchin English as a foreign language.The contrast between'action' and 'reflection' was shown to be only apparent,and Schon'sresolutionof this in the conceptof 'reflection in action' was taken to be a keystonesupporting the whole venture.Teacher-initiatedresearchin this tradition was seento haveestablishedand venerableroots in mainstreameducation,and both the advantagesand somedangersof this approachwere aired:

THE TEACHER RESEARCHER IN FOCUS

35

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

on the positive side: • the immediate relevance to classroom teachers' problems and modesof work • the bridgesbuilt betweentheory and practice • the developmentof critical awareness • the appropriacyfor subtleprocesseslived through by the participants • the challengeto the comfortable'craft knowledge'view; and on the minus side: • the unhelpful growth of oppositionsbetweenresearchby teachers and by researchers • the difficulties of evaluatingits ultimate effectivenessas an agent for either personalor institutional change. The chapter then briefly reviewed the work in mainstreameducation using action research,and pointedout the aims, someof the achievements, and someof the language,of thesemainly large-scalecurriculum innovation projects. A major section of the chapter was devoted to the use of teacherresearchin English languageteaching, bearing in mind particularly the worldwide nature of the enterpriseand its involvement in so many different cultures and educationalsystems.We saw how the principles of reflective training are being applied by severaldifferent trainingprogrammedesigners,and looked at examplesof researchby teachers working individually, institutionally or in teams,which have been carried out and are currently in progressin many countries.Finally, the problem of cultural and situational effectivenessof teacherresearchin worldwide ELT was raised.It may be difficult to introducean approachwhosepower for innovation and change may be feared by local and governmental authorities,but its capacityfor stimulatinglocal changemay yet be preferable to either conservatismor the importation of culturally inappropriate methodsvia the 'appliedscience'model of training.

Discussionnotes I. Assuming that researchby teachersis a possibility in your own teachingsituation, enumeratewhat you consider to be the main variables that would affect the kinds of topics that could be addressed,and the ways in which researchcould be implemented. For example,what can be done may be affectedby factors sudh as the amount of responsibility you have; patterns of cooperation with colleagues;contacts outside your own institution; or the questionof how changein policy and methodologycomesabout, and who initiates it.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

36

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

2. To what extentis the specific conceptof action researchfeasiblein any working context(s) with which you are familiar? Is such an attitudean integral part of your professionalenvironment,or is it regardedas undesirablein termsof the statusquo? 3. Even if you do not label yourself as a 'researcher',what are the and other professionalissueson which kinds of classroom-related you mainly find yourself reflecting?On individuals in your class? On your own teaching style? On the materials available to you? ... and soon.

3

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

What is research?

Introduction In the last chapter,we saw that many teachersdo havethe motivation to do research,and the problems that need research,and that some teachers have the opportunityto do so. This chapterfirst stepsback from the classroom to examinegeneralideasabout what researchmeansin various realworld contexts, and then looks at what researchspecifically in language teachingdoes, and what special characteristicssuch researchdisplays. In what follows, we will compareand contrast views of researchin other fields; evaluatecommon views about researchin languageteaching; discuss the contrastbetweenbasic and applied research;look at the relative importanceof descriptionand intervention in the context of the problem under investigation;and explore the differencesbetweenquantitativeand qualitative kinds of researchboth in terms of methodologyand also in terms of underlying assumptions.This will be followed in Chapter4 by a fuller account of the criteria by which researchitself is evaluated,with some discussion of how 'teacher research'compareswith researchby researchers(usually in universities),on thosecriteria. Chapter4 will then go on to deal with someimportantgeneralprinciplesof researchdesign.

Generalviews The view of researchimplicit in the foregoing chaptershas been one of professionalenquiry. In other fields, the term 'research'has other senses. One could draw a crude (and probably ultimately invalid) distinction betweentwo sensesof the word research in the ordinary language.In the first sense,the outcome of researchis the establishment,publicizing, or utilization of somethingthat somebody- not the researcheror the person commissioningit - already knows. In the secondsense,the outcome is knowledge nobody had before. This is the general aim of academic research.The word researchis used in different fields in one or other of thesesenses,and in somefields in both senses.The title of this book uses

38

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

the word in this second sense: however, teachersmay be involved m researchin both sensesin their professionallives. Someexamplesof the first senseare the following:

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Fiction For many novelists,it is essentialto preparefor their writing by researching the backgroundin which they wish to set their narrativefor accuracy of description,sensitivity to atmosphere,history of the period in which their fiction takes place and authenticityof the language.The final criterion for them is the authenticityand artistic conviction of the final product in their readers'eyes.Researchin this sensemay take many forms, and involve considerableexpensein terms of effort, time, and money: but the originality of the novel doesnot lie in the researchbut in the artistic creation for which it providesa background. Journalism For journalists,especiallyin investigativejournalism, long periodsof time are spentresearchingtheir storiesto uncoverfacts and secretswhich their editorial policy judgesto be in the public interest,and in cross-checking what they are told or discoverto establishits truth. This is not usually new truth, but information otherwise not public or deliberately withheld by anotherperson. For them, the final criterion of truth may be bound up with legalities: the laws of libel and slander,or the Official SecretsAct. Police work For the police, thereis an obviousparallel betweenpreparinga caseagainst a suspectby detectivework and researchin this first sense:the criminal they seekhas a secretwhich the police are bound to uncover. Here again, the criterion of validity of the researchis circumscribedby the law: laws of evidence,police procedure,protection of the rights of the innocent, the criminal and the victim. However, it is interestingthat in this connection the languageprefersthe term 'detection'or 'investigation'to research. Businessand commerce In the commercialworld, much moneyis investedin productdevelopment and even sponsorshipof basic research,and here one is referring to the secondsenseas describedabove.However,researchis also conductedinto the peoplewho will buy the products:marketresearch.Market researchis usedto establishwhat can be sold and who will buy it, how a productcan

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

WHAT IS RESEARCH?

39

be packaged,advertisedand priced to makeit commerciallyattractive,and, linked to advertisingcampaigns,evento createa market- to persuadepeople to buy somethingthey did not know they wanted.The final criterion for researchin this field is not thereforesimply truth - the descriptionof a market situation- and the authenticexpressionof that truth, but also the successof an interventionor manipulationof the marketmeasuredusually in profit terms. Someobvious examples of researchin the secondsenseare the following:

Medical research In the medical world, researchis conductedat many levels, and, apart from the commercial aspectsof competition to sell drugs, it includes methodsof assessing diagnoses and treatments,drug development,sideeffects, methods of general practice, methods of surgery, epidemiology and common and rare illnesses.This is researchin the secondsense:to find out things nobody knew before.The final criterion is the discoveryof new truths, but also the translationof the new truths into practical treatments,and the developmentsof economiesof scaleso that the treatmentis available for the largest number of people. Here ethical considerations receivea great deal of attention,both in terms of the humanpatientsand the animalswhich are used for trials, but the ethicsof researchare important generallyas well, as we shall seein greaterdetail later.

Scienceand technology Everyonenaturally associatesresearchwith scienceand technology,white coats and laboratories,but it should be rememberedin this context that advancesin the sciencesare not restricted to the discovery of new facts: advancesin theory developmentand in researchmethodsand approaches occur in parallel. It is of course a commonplace,though nevertheless important, to highlight the ethics of the developmentof scientific knowledge, both in the processof gatheringthe knowledgeand in the fields of application. Before we turn to languageteachingand languageteachers,this simpleminded surveyof researchin other fields has shown that 'research'is used both for the discoveryand publication of concealedknowledgeand for the creation of new knowledge. There are different kinds of criteria which dependon the field; it is also involved in interventionand manipulationof its sphereof interest;and it raisesconsiderableethical implicationsboth in the processand in the applications.

40

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Languageteaching

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Languageteachersalso do researchin the first sense,but do not tend to call it research,though what they do is analogousto what a journalist or doctor doesin preparingfor their professionalactivity. They haveto: • scan textbooksand libraries for appropriateexamplesof the languageand targetedexercises toinclude in their intended lesson plans • find materialsfor conveyingthe points they wish to make • locateappropriatetexts and examples • checktheir understandingof grammatical,pragmaticand cultural descriptionsin preparingexplanations • find out why studentshave made certain kinds of mistakesand why they havebehavedin certainways, and • look for remedies. In all of this they are guided by what will work with their particular students in their context. Thisis very like what a journalist, novelist or market researcherhas to do, or even a general practitioner choosing an appropriatetreatmentfor a patient with a particular set of symptomsand prehistory. It is concernedwith detection,with authenticity, with utility, and in a sensewith product development,if the teacher'sclassroomactivity can be seenas a productwhich is deliveredto the students. This kind of 'research'(sense1) usually lacks the elementsof originality, innovation and theory-buildingwhich characterizeresearchin the second sense.The argument elaborated in Chapter 2, however, strongly suggeststhat language teachersare uniquely placed to be involved in researchin this secondsenseas well. Thereare many questionsabout languagelearningand teachingwhich can only be answeredby investigations conductedin the normal context of teaching and learning. But getting involved in research(2) also incurs risks - of addingcommitmentsof time and application, of altering the teacherresearcher'sconceptionof their professionalactivity, of challengingtheir understandingof their own and their students'context of work, of changingthat context from the inside. As Elliott (1991) has said, such risks may be seenas threateningor be welcomedby the teachersthemselves,their studentsand their employers,and thereforemost teacherswill want to undertakethis kind of researchwith good preparationand a clear ideaof what is going to be involved. But what sorts of activities would be classedas this kind of research? Sometimesprospective research students propose course development projects as researchfor the degreeof PhD, and they are often refused. Coursedevelopment,though clearly an activity many teachersengagein outsidetheir classroomteachinghours, and which may, obviously, involve innovation in many ways, is not primarily gearedto discoveringnew knowledge,but rather,to encouragingthe bestperformanceout of the students.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

WHAT IS RESEARCH?

41

However, the processof coursedevelopmentmay, perhapseven should, involve researchactivity in threeareas,becausecoursedevelopmentwhich is basedon empirical evidenceought to be more successful.Thesethree areas are needsanalysis,delivery and evaluation.In eachof theseareasa teacher may simply use a pre-existingor even prescribedmethod,questionnaireor performanceindicator to gather the information identified as necessaryfor the developmentof the course,or may approachthe problemin an altogether more imaginativeway which may be indistinguishablefrom the proceduresof researchin the secondsenseoutlined at the beginningof the chapter.

Commonviews of what researchin languageteachingdoes As part of a coursein applied linguistics and English languageteachingwe regularly ask experiencedteachershow they conceiveof research,and we I have polled teachersat an internationalconferenceon the samequestion (McDonough and McDonough, 1990). There are, of course, as many answersas thereare respondents,but they tend to group arounda number of popular issues.Attitudes to researchin languageteachingcan also be sharply contradictory, with widely differing views between individuals, and sometimeswithin the sameindividual until he or she has sortedout a coherentview, usually after someexperienceof readingrelevant research, for exampleon an MA course.Here follows a selection: 'Researchis systematicand is basedon data'

This attitude provokesseveralinterestingquestions.The referenceto systematicity could imply the application of coherenttheory, but it is more likely to mean the use of a more or less meticulously plannedmethod for carrying out the research.The belief that researchis basedon data begs questionsabout what actually counts as data: numbers,statements,questionnaire responses,errors, learner language,but perhapsalso pictures, films, videos,tape-recordings,or evenfeelings and emotionalencounters. 'Researchusesexperimentalmethodswith treatmentand control groups'

The identification of true researchwith experimentalmethodsas used in the hard sciencesor in laboratory psychology which involve controlled comparisonsis a frequentattitude. It presumablyexcludesdescriptionand classificationas valid kinds of research,since it dependson comparison. Contemplation of the many difficulties of executing such a model of researchin normal classroomsprobably meansthat this attitudecorrelates with a belief that researchis actually irrelevant to the normal professional activities of a teacher.

42

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

'Researchinvolvesa ~ypothesis

which is then tested'

Again, this quite sophisticatedview reflectsan importanttraditional principle: that research,as perhapsopposedto mere curiosity, requiresa formal statement,gracedby the technicalword hypothesis,whosetruth is put to the test. Pushingthis notion further it is possiblethat this attitude underlies one of the distinctions betweenteachingand researchinghighlighted by Cohenand Manion (1989: 229) where they posit that 'researchvalues precision, control, replication and attempts to generalisefrom specific events.Teaching,on the other hanq\is concernedwith action, with doing things, and translates generalisationsinto specific acts'. Cohen and Manion makethis point in a discussionof action research. This attitude also illustratesa frequent preferencefor traditional types of researchrather than some of the principles of qualitative research,in particularthe idea that theory can be derived from datacollected.

'Researchis objective' The quality of objectivity usually refers to imperviousnessto the actions or judgementof the researcher,that is to an externalreality which is independentof the wishesor hopesof the personin the situation.Again, much of the methodology of traditional quantitative researchis intended to ensurethat the resultsand interpretationsof the researchhold good whoever actually performs it. Objectivity also suggeststhat those results and interpretationsgainedfrom one context of researchwill be true of others: thereforeobjectivity and generalizabilityare closely related. However, to the extent that objectivity involves a denial of the individuality of the participants, much activity by reflective practitionersand action researchers challengesthe doctrine of dissociationbetweenthe roles of active participant and objectiveresearcher.The roles and the individual ways the participant, here the teacher researcher,carries out those roles within the context in which the researchtakes place becomea factor in the research itself.

Basic and appliedresearch Basic researchis often describedas researchwithout immediatepractical utility, driven only by the advancementof theory, whereasapplied research involves some kind of applicability. SharwoodSmith suggeststhat there are two strandsof secondlanguageacquisition research,a pure and an applied one. Pure SLA developsa body of knowledgeabout secondlanguagelearnersbecauseit is 'interestingas a phenomenonin its own right' (1994: 4). It may make referenceto, borrow theoriesand methodsfrom, and expectto influence theoreticaldevelopmentsin a variety of other disciplines, such as linguistics, sociology and cognitive science.Applied SLA

WHAT IS RESEARCH?

43

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

would, for SharwoodSmith, constitutean answerto the question 'What does secondlanguageresearchhave to say to languageteachers?'Applied researchmay develop along at least three different kinds of path, all of which havebeenusedin languageeducationat sometime: 1. The applicationof researchresultsand the theory they supportto the solution of languageteachingproblems.A distant exampleof this is afforded by the developmentof patterndrills on the basisof learning experimentsin the behaviouristtradition (Rivers, 1964); a more recent exampleis the adoption of schema-theoreticreading exercises(involving extensivepre-text exercises,the application of backgroundknowledge)on the basisof researchon second languagereaders(Carrell and Eisterhold,1983). .:.. A secondview of applied researchsuggeststhat what is applied is not so much the productsof existing researchbut the methodology of obtaining those results. Thus, much applied psychology researchtakesthe 'scientific method'as embodiedin experimental psychologyand applies it to problemsof human performancein the real world. 3. Yet a third view arguesthat applied research- which should be distinguished from action research(Cohen and Manion, 1989: 218) - developsits own body of knowledge and theory-building for its particular set of problems.It is this approachwhich blurs the distinction between basic or pure research and applied research,for the set of problems which the applied researchis designedto help solve do not admit of a more basic approach. Researchon and by the participantsin languageteachingcan be of this third type, since it can be argued that only the kind of researchwhich satisfies the conditions of validity and contextual relevancecan be appropriatefor the peculiar circumstancesof the problemset we are trying to understand. It is thereforesuggestedthat the activities of classroomlanguageteaching and learning contain the kind of challenge to our understandingand therefore to our attemptsto introduce changeand innovation which can only be resolvedby doing researchon and in thoseactivities, using methods which are appropriate to those researchaims and the problems under study. Neither the extrapolationof knowledge from 'pure' research,like comparativeor universal linguistics, or learning behavioursin specially designedsituations,or secondlanguageacquisition in other contexts(for example, naturalistic or self-taught) nor research 'applying' methods found appIopriatefor other kinds of problems,whether psycholinguistic, or sociological, or hard science, are likely to adequately represent the essentialsand the variability of the problem.

44

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Description and intervention Whateverthe planned,or unplanned,utility of the research-based knowledge might be, it is usual to distinguish two kinds of aims, or possible routes to the common goal of improving understanding.These two are, broadly speaking,descriptionand intervention. Much researchproceeds by intervening in the activities or processesin question by manipulating the variablesthat can be identified and attemptingto isolate the influence of one or more on the process.As we discussmore fully in the next section, experimentalmethodsin particular use a variety of techniquesfor separatingout the variables in a situation so that the conclusion about what affected what can be drawn. For example, if one wanted to know whethersuccessin readinga foreign languagewas more likely for learners who were good readersin their mother tongue, a way of distinguishing good and bad readers in each language, perhaps by a test, would be required. Foreign language reading would be termed the Dependent Variable, mother tonguereading theIndependentVariable. Simple countmg of the frequencyof peoplewho were good or bad readersin both compared to those whose readingscoreswere very different in each language might answerthe question,as in the following: good L1

bad L1

good L2

a

b

bad L2

c

d

To uphold the conclusionthat good L I readingis important for good L2 reading it is obvious that you would want to find a larger proportion of peoplein cells a and d than in cells bandc - more peoplein whom reading proficiency in their two languageswas similar than peoplewho were good in one and bad in the other. This difference in proportionscan easily be evaluatedusing statistics,but the important point here is the immediate associationbetweenthe evaluationof this interventionisttechnique(of isolating reading scores)and measurementby the use of tests for estimating the valuesof the dependentand the independentvariable. However, there would be other factors involved which might prevent any conclusionfrom sucha simple procedure.Theseother variablescould be confoundedwith the independentvariable so as to bring into question the certainty of the connection apparentfrom the figures. Such 'confounding variables' may arise from characteristicsof the people themselves (in the imaginary example one would be reluctant to accept the conclusionif it was discoveredthat all of the peoplein cells a and d were also teachersof readingbut none in cells band c were), or characteristics

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

WHAT IS RESEARCH?

45

of the situation.Again, in our imaginaryexamplethe conclusionwould be unsafe if it was discovered that the people had performed the reading tests in different places under different lighting conditions, noise levels, as individuals or groups, taking different completion times and so on. In the design of interventionist research, controls and counterbalances (which we will look at in detail in Chapter 10 on doing experiments)are built in, by using randomization,careful measurementof confounding variablesand strict adherenceto procedure,in order to allow the conclusionsto be unambiguous. Sometimes,this kind of research,when applied to languageteaching and other educationalproblems,leadsto procedureswhich distort the situation in which the researchis being conducted.Green's(1975) study of the efficacy of languagelaboratoriesrequired both groups of children to go from school to the university where the languagelaboratory group performed lessons in the laboratory and the comparison (or 'control' group) did ordinary lessons- but in what was for them an extraordinary situation. The reasonwas to control for the possibleconfoundingof any benefit accruing from the actual languagelaboratory teaching with the general excitement of being chosen to go off to the university for a researchpurpose.Thus, the learners'personaland educationalcontext is controlled and counterbalancedso that particular effects can be made manifest. Descriptiveresearch,on the other hand,aims at making explicit the significant effects within the context itself. To this end the researchattempts to provide a rich accountof the whole situation rather than minimizing it. There are two reasonsfor this. The first is that in most educationalsituations the list of possibleconfoundingvariablesis so large, with somesystematic and some unsystematicones,that realistic and satisfactorycontrol and counterbalanceare nearly impossible.The second,and more positive reason,is that it is increasinglyrealized that individual effects which can be isolated rarely work alone: therefore the pattern of context in which they are embeddedis all important. Context

The importanceof the role of 'context' in formulating the teacher-research paradigmhasappearedseveraltimes in the discussionso far, both explicitly and by implication. We have,for instance,consideredboth the concretecontextual variablesat play in teachers'working environments(Chapter 1), as well as ways in which the notion of context can be linked to the nature of professional knowledge and to issues of specificity and generalizability (Chapter2). A simple examplewill serveto illustrate. One of the authorswent to the classroomto preparefor the next lesson

46

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

(an EFL Study Skills classfor studentsin university). The previousteacher had not cleanedthe whiteboard,on which the following was written: nationality naturalizedcitizen mlSO soup he is a descendantof the Emperor invisible Although the presentwriter knew the group taught by her colleaguevery well, it was neverthelessimpossible to reconstructexcept in the vaguest terms what had been the substanceof the lesson.Clearly the teacherhad not merely stuck to a fixed lesson plan (the 'objectivesmodel'), but had also respondedto what Richardsand Lockhart (1994: 84) call 'the interactional dynamicsof the teaching-learningprocess',thus creatinga uniquenessof contextfor this particularlesson.Calderhead(1987, quoting Doyle) offers six featuresof the classroomenvironmentthat are relevant here. It has: 1. Multidimensionality

2. Immediacy

3. Simultaneity

4. Unpredictability

5. Publicness

6. History

In this view, then, reality is seennot as fixed and stablebut as socially constructed,so what Sevigny(1981: 72) calls the 'social order' - in our case,of a classroom- is perceivedas 'an emergentphenomenon'.This view is echoedby Erickson and Schultz (1981: 148), who describesocial contexts as 'interactionally constituted environments'where there is a constant processof changeand re-adjustment. This perspectivehas a long pedigreein social science,through such major related traditions as ethnomethodology,phenomenology,grounded theory, and symbolic interactionism.There is no space here to go into sucha rich and complex field in any detail, but the importanceand attractivenessof thesetraditions for educationalresearchshould not be underestimated. HerbertBlumer, for example(quotedin Hammersley,1989), who is particularly associatedwith symbolic interactionismand the Chicago School of social science,wrote that meaningsare not fixed and universal but variable, 'fabricatedthrough the processof social interaction'. In this view (as Denzin, 1970, underlines)social life is in a stateof flux, and open to multiple perspectivesand interpretations.It is, as he says,'a world that refusesto stand still'. Not only, as we shall see in Chapter4, does this throw a problematiclight on standardresearchquestionsof validity and

47

WHAT IS RESEARCH?

reliability: it is also likely that the view it encapsulateswill be recognized by the majority of teachersas what happensin their everydayreality.

In the previoussection,two modesof researchwere describedbriefly. The distinction betweenthem is not just methodological,that is to say not just a questionof selectingthe most appropriatemethod for the subject under study. It is not merely a questionthereforeof 'horsesfor courses'.Van Lier (1988: 57) setsout the various types of researchin a useful table, in which two dimensionsof control and structuregive four broad types: measuring, controlling, watching, and asking/doing.His table is reproducedin Fig. 3.1 for illustration. Van Lier's dimension of 'control' is roughly equivalent to what was described above as description versus intervention; his dimension of

Measuring

+

Surveys Coding 'Systematic' observation

Controlling

Structured

Experiments Quasi-experiments selectivity

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Normativeand interpretiveresearch

intervention _

+ Control/ed

Observation Case study Protocols Stories Diaries

Watching

Action research Interviewing Elicitation

Asking/Doing

Fig.3.1 Typesof resel}n:h (Reprintedhy permissionof Addison Wesley I,ongroanI.td.)

48

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

structure is roughly equivalent to the traditional distinction between quantitative and qualitative approaches.We now turn to this difference, but widen the discussiona little to take into account the implications of the parallel but slightly more generalterms'normative'and 'interpretive'.

Normativeapproaches Normativeapproachescan be characterizedby the following: • • • • •

descriptionby numbers significancein termsof probability useof experimentalor quasi-experimentaldesigns generalizationfrom sampleto population the searchfor causes.

Typically, as Cohenand Manion (1989) point out, they have beenassociated with large-scale research, with 'objective' judgements made by researchersfrom the 'outside',and have claimed to isolate facts which are true of more peoplethan the researchwas actually performedon. In addition, it is usually claimed,at least in experimentaldesigns,that the effects isolatedare causalin nature,and althoughwith certain designsthis is precluded, the eventualgoal of such researchis to come up with valid causeand-effectrelationslike thosesoughtin the naturalsciences.

Descriptionby numbers In our imaginary exampleabout reading proficiency in the previous section, we noted that testswould probablybe usedto give a measurementof the proficiency eitheras a simple good versusbad judgementor as a score. One advantage of this would be that such measurementscan be counted, and this reflects an important assumption of the normative approach: that truth resides in the degree of agreementbetween many of the sametrait observations.Thus it is betterto take many measurements from one personand better to take measurements from many peoplethan to useone-off observations.Proceduressuchas finding the mean(or average - the figure which best characterizesagreementof all the measurements) and the deviation (the degree to which those measurements actually do not agree)can be applied to find the most valid and reliable way of describingall the measurementsby using just one or two figures. Such manipulationsallow a kind of truth to be revealed,but notice it may be an abstracttruth, one for which no real example exists. A (probably mythical) example of this is the description of the 'average'family in Britain as being constitutedby l.5 adults, 2.4 children and 0.5 dogs. Of course,sucha descriptionrepresents,probablyaccurately,the fact that not all nuclearfamilies consistof two adults, that there is a rangeof offspring

WHAT IS RESEARCH?

49

numbersfrom none to double figures, and about half the families own a dog: but equally obviously no family answeringthat descriptionwill ever be found.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Significancein termsofprobability Technically, the 'significance'of a result is the likelihood that it could have been produced by chance, rather than by some systematic influence. (Whether the systematic influence was what the original hypothesis or theory said it would be can only be determinedby the design- hencethe controlsand counterbalances mentionedin the previoussection.)In experimental, survey, and correlational designsas used in psychology and psycholinguistics,therefore,the definition of a fact is a matter of likelihood, of distinguishing'fact' from 'accident'.

Use ofexperimentaland quasi-experimental designs Normative researchtypically uses designswhich attempt to identify independentand dependentvariables and incorporatecontrols for systematic confoundingvariables,known featureswhich might bias the results,such as previouslearning experience,languageproficiency level, intelligence, sex or whatever,and counterbalances like randomizationfor unknownbut suspected variables.

The searchfor generalizationfrom sampleto population The aim of this kind of researchis to make generalizations,and find evidencefrom the particularsamplestudiedto the populationof peopleat large, or at leastto somerecognizablesubsetof them, like peoplein the samekind of learningsituationelsewherein placeor time. In itself this aim is sharedby most kinds of researchincluding, as we shall see, qualitative methods. However,thereare two ratherseriousproblemswith it for languageteaching and teacherresearch.The first is that the quality of the generalizations possible,given the inevitably narrow data basein the languagearts, tendsto be restricted. McLaughlin (1987: 155, Table 7.1) lists 10 generalizations basedon secondlanguageresearchwhich he seesas hypotheseswhich the general theory of second language acquisition he seeks must.explain. However, close inspectionshows that they are all remarkablyvague,and he himself points out 'then! is not universal agreementas to their status'. Second,practitionerswishing to act on these kinds of generalizationsfor anothergroup of learnersin anotherplace have an almost impossibletask relating the generalto the particular.

50

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

The search}i}r cause Van Lier (1989) criticizes normative approachesfor espousinga positivistic belief in determinism- that is to say, the searchis for causaleffects, and the test of the searchis the successof the prediction. If, therefore(to use one of his own examples),you can say that a particular strength of wind will causea tree to fall down, you can test this by observingthe effect on an experimentaltree of winds of different strengths.He points out that even classical physics is hard put to it to guaranteesuch a prediction, becausethe causal chain is complicated,involving characteristicsof the wind, the tree itself (age, type of wood, foliage, health), the soil it is planted in, the proximity of other treesand objectscausingeddiesand so on. He also points out that such a causal model is irrelevant in language teachingbecauseteachingdoesnot causepeopleto learn anyway. With these argumentshe criticizes a whole tradition of researchon teachingwhich endeavoursto specify elementsof the process- teachingwhich affect the product - learning. However, it is still legitimate to ask why as well as how things happen,and he distinguishesbetweencausesand reasons.Sucha distinction takesus into the realmsof ratherabstrusephilosophy except for the parallel distinction betweenexplanationand prediction. The failure of many kinds of theory to predict in all real-world circumstances(meteorologyis one familiar example) is not held to be a major critique of the theory sincein generalterms the theory may well give satisfactoryexplanations.Prediction is, however, not necessarilya tougher criterion than explanation:we can all predict that a child will acquire the languageof its environmentbut nobody can yet explain how. Skinnercould predict with some accuracythe learning curve of an animal in a Skinner box but could not explain it in termsother than his predictions.

Interpretiveapproaches In this categoryare often collected together a number of researchtraditions which actually differ considerably among themselvesas to their major assumptionsand fields of interest and applicability, favourite data types and methodsof analysis.They are participantobservation,symbolic interactionism,ethnography,ethnomethodologyand others. However, the contrast with the normative approach highlightsthe shared features of these traditions. Many of these features hold considerableattraction for researchin language teaching and learning, collaborative researchand researchby teachers,since they suggestalternativesolutions to the problems of context, generalizability and confidencein the conclusions,and offer canons of methodologicalrigour to the investigation of local and often small-scaleproblemsof learning communities.After the main characteristicsof this kind of researchhave been describedin what follows, a further section will point up some interesting parallels in the two

WHAT IS RESEARCH?

51

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

approacheswhich exist despite their very different origins and basic assumptions.Later, in Chapter 4, the increasingly frequent strategy of using elementsof both approacheswill be discussedunder the headingof triangulation. EthnograPky

As Watson-Gegeo(1988) points out, ethnographyhas become,for some researchersin languagelearning,almosta synonymfor qualitativeor interpretive research,although this assumptionis in fact unwarranted.Brown (1984) also introducesa distinction betweenexperimentaland naturalistic researchin languagelearning, and discussesethnographyas an important variety of naturalistic enquiry. Following Ochsner,she presentsa useful tabulationof the contrastsbetweenexperimentaland naturalisticresearch which, although polarized, clarifies the comparison.This is an excerpt from the tabulation: Experimental Nomothetic Empirical Quantitative Scientific

Naturalistic Rational Qualitative Grounded Ethnographic

Van Lier (1988; 1989), as already mentioned,has performedan ethnographyof classroomlanguagelearningand discussesethnography,if not as a synonym, then perhapsas an emblem of interpretive or naturalistic research. Watson-Gegeoand Van Lier highlight several principles in ethnographicresearchwhich are exploredin what follows: the 'emic' principle, which refers to the 'rules, concepts,beliefs, and meaningsof the people themselves,functioning within their own group' (Van Lier, 1989: 43); the 'holistic' principle, by which ethnographersattempt to understand phenomenain connectionwith the location and situation of those phenomenaand not abstractedfrom them; the accent on 'context', as describedearlier in this chapter;the useof 'groundedtheory'; and the idea that the processof researchshould be open and explicit about its proceduresand designdecisions. Parlicipanl research

Many, but not all, of these qualitative approachesclaim the virtue of the researcherbeinga participantin the situation being researchedas compared to an 'objective'outsider.The participantobservermay be a researcherwho takes a role within the community being studied,as in the anthropological tradition stemmingfrom Malinowski, and much ethnographicwork; or may

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

52

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

be a 'natural' participantin the situation like a teacher- or a student,for that matter- who addsto their normal role that of data-gathererand analyst. The fieldwork involved will be intensive,conductedaccordingto rules of evidence,and reflective. However, being a participantobserverdoesnot necessarilyguaranteeprivileged entry to the most appropriateinterpretation: indeed,it could be a disadvantagerequiringconsiderableeffort to overcome,because,as Erickson (1986: 121) says,everydaylife is often 'invisible' to us preciselybecauseit is familiar, so to posethe question'what is happening here?'is to posea considerablechallenge.What is commonplaceto the participant may be problematic to the researcherwho aims to produce a plausibleinterpretationof a whole situation. However, Erickson also points out that the participant observeris in the best position to note 'concrete details of practice'- in other words a fully detailedrecord of actions which can be usedto provide the datafor interpretation. Local and non-localorganization Interpretive approachesdeal particularly with what actions mean to the peoplewho engagein them - with the local meanings(an exampleof the 'ernie' principle). Thus, such researchis necessarilysituated in natural groupings:real classes,normal schoolorganizations,ratherthan concocted or randomizedorganizationsand assignmentto treatmentgroup for the purposeof control or counterbalancingof 'confoundingvariables'.That is, however, not to say they do not pay attention to more generalmeanings, which are non-local meanings.Interpretationsof a particular context and what cooperativeaction meansto the participantsin it is both to be compared to similar actions in other contexts - 'how do they do this elsewhere?'- and usedto illuminate thoseother contextsas well.

Generalization It might be thought thatan approachrelying so heavily on the explication

of contextand the local meaningsof action could not produceuseful generalizations.Interpretiveresearchindeeddoesaim to uncovergeneric,universal features of the phenomenonin question through comprehensive and detailedstudy of individual contexts,of 'local microcultures'-a classroom, or learning group. Erickson comparessuch 'concrete' universals with the,'abstract'universalsproducedby statisticalresearch:'The paradox is that to achievevalid discoveryof universalsone must stay very close to concretecases'(1986: 130).

The searchfor meaninginterpretations This researchtradition looks at the ways in which the participantsinterpret their own situation to themselvesand to eachother. Thus, what contributes

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

WHAT IS RESEARCH?

53

to achievementis seen in the pattern of interaction betweenmore or less skilled teachersand more or lessskilled learnersand how the learningcommunity utilizes its resources.Connectedwith this distinction is that between behaviourand action: a billiard ball might be said to behavein a minimal kind of way as a result of forces on a flat bed; but a learneris acting in the situationand making sense(or failing to makesense)of it. 'The billiard ball doesnot makesenseof its environment'(Erickson, 1986: 137).

Qualitative data It follows from the abovethat the kinds of datacollectedare very different to thosein normativeresearch.Whereasnormativeresearchrequiresa numerical evaluation,qualitative researchusually gathersobservations,interviews, field data records,questionnaires,transcripts,and so on. The rangeof data is wide, but this also requiresrules of evidenceso that rigour is preserved. This comprehensiveattitude to data also meansthat it is typical for qualitative data collection choices to evolve, as in the action research spiral (Chapter2), as insights and tentativeconclusionsby the researcherindicate the needfor otherconfirmation.

Plausibleinterpretations To ensureplausibility, there have to be rules of evidence.Erickson (1986: 140) gives a straightforwarddescriptionof five rules, here reproducedas positive statements. 1. Evidencemust be adequatein amount to supportinterpretations.

Central importanceis given to completeand detailedobservation. 2. Evidence should come from a variety of data types. Reliance on one type can miss important featuresand, more seriously, leave conclusionsunvalidated. 3. Data must have good 'interpretive status': in other words, researchershave to guard against misunderstandingfeatures in the data, becauseof inadequacyof amountor even deliberatemisinformation. 4. Disconfirming evidenceshould be includedand actively sought. 5. Discrepantcasesshouldbe analysedcarefully. Qualitative research usesdiscrepantcaseanalysisas a powerful antidoteto looking only for evidenceto supportthe researcher'sconclusion.

Ethical considerations Qualitative researchis not alone in valuing ethical considerationshighly. However, arguably, this tradition has formulated more explicitly than

54

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

others how researchethics work, both (a) to protect the validity of the research- for example,the achievementof good data by recognizingthat data provided by informants is owned by them, and its use is with their permission only; and (b) to protect the participants of the research through rules of confidentiality and consentto particularusesof the data.

Someparallels hetweennormativeand interpretiveapproaches Despitethe many deep differences,it will be realized that both traditions sharea numberof concerns,but provide different solutions. In both, thereis a concernfor quality of data: in normativeresearchit is expressedin terms of pre-testingthe researchinstrumentsand tests for validity and reliability, in interpretive researchin terms of quantity and variety of datatypes. In both, there is a concern for potential falsification. In normative research,this is enshrinedin the use of empirical data to reject a null hypothesis;in interpretive research,it is expressedas the searchfor disconfirming evidenceand 'discrepantcaseanalysis'. In both, there are proceduresfor determiningwhat is significant, but in the former the significanceof a finding lies in probability, whereasin the latter, it is a questionof revealingthe universaland genericfrom comparisonsof datasetswhich are as completeas possible.

Conclusion This chapterhas attemptedto follow through and extendthe issuesraised in the discussionof teacherresearchin Chapter2. It hasin a sensestepped back a little, in order to look at some of the key conceptsthat inform teacher-initiatedresearch,togetherwith their derivationsand background, as well as to set this particularorientationto researchwithin a broaderdefinitional framework. Since methods and techniquesdo not exist in a vacuum, this exploration of the paradigmsthemselves,and their implications in generalterms for languageteaching,are also a necessaryprecursor to the whole of Part 2 of this book. The chapterbeganby invoking a simplified but convenientdistinction betweentwo sensesof the term 'research':uncoveringand perhapsreinventing the known world, and discoveringnew things. We then reported somecommonlyexpressedviews from teachers onthe assumedcriteria for researchin languageteaching.This led to a more detailed exploration of somefamiliar and often polarizedterms,in particular: • basic(or 'pure') versusappliedresearch • interventionistversusdescriptiveapproaches • normative versusinterpretive traditions, in particular the much-

WHAT IS RESEARCH?

55

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

quotedquantitative/qualitativedistinction and the natureand role of ethnography as well as the pivotal notion of 'context' within the interpretivedimension of social scienceand educationalresearch. In the courseof the discussion,we have suggestedthat, althoughthese representapparently very different ways of seeing the world, it is not always helpful to see them as mutually exclusiveforms of knowledge,and the methodologyusedfor a particularresearchundertakingmay well combine facetsof each. Nevertheless,an understandingof the epistemological distinctions is important when making informed and reasonedchoices aboutresearchcontentand method.

Discussionnotes 1. What are your own associationsof the term 'research'?To what extent are they comparableto those listed as quotations in this chapter? 2. Can you identify areasof 'applied research'that in your opinion have had a direct impact on aspectsof methodology,or materials? (For example,many coursebooksclaim to derive activities directly from researchon reading comprehension,or discourseanalysis, and so on.) 3. Do you think that teacherstypically make 'causal' judgements about ways in which their learners learn (or apparently fail to learn?)

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

This page intentionally left blank

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

4 Principles and problems: what makes good research?

Introduction The first, and largest, section of this chapterdiscussessome 13 features which are traditionally associatedwith good research,and attempts to explain them. They fall loosely into four groups concernedwith the initiation and undertakingof research,the internal workings of research,the applications of research, and one all-pervading issue: the ethics of research. There are many areasin which thesefeaturesare potentially in conflict, for example,the tension betweenconfidentiality of data and publication, or betweensensitivity and objectivity, or betweenoriginality and research history, or betweencontextual accuracyand replicability. The quality of researchcan thereforebe seento someextent in the adequacyand sincerity of the compromisechosenby the researcher. The secondsectioncomparesvarious actual researchtraditions, mainly the normativeand hermeneuticas introducedin the previouschapter,and the researchthat can be performedin the academiccontext,often in pursuit of higher degrees,and participatoryresearchby teachersin their own context. Each kind of researchhas its own advantages;no one tradition or situationcan claim supremacyover the othersin all circumstances. The third section considerssome important issuesin researchdesign, and exploresthe advantagesand somedangersin the use of confirmatory strategiessuchas triangulationand multi trait, multimethodresearch.

Features of good research The following 13 featuresseemto fall loosely into four rough categories, and we shall deal with eachof them in turn. First, there are features to do with the initiation and undertakingof research- the interest, originality,specificityand publication or dissemination of the researchquestionsand findings.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

58

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Second, there are features of the design and methodology of the research- the sensitivity,objectivity, validity and reliability of the methods chosen,and the key principle of folsifiability, which appliesto hypotheses, conclusions,and theoriesequally. Third, there are featuresconcerningthe applicationto other situations, of practical exploitation, use in effecting changeand innovation, and of further or parallel research- replicability, generalizabilityand·utility. Last, thereis the questionof the ethicsof researchand how the participants' rights are affected,how both the quality of the researchdataand the confidentialityof disclosuresmadeby respondentsmay be protected.

Initiation and undertakingofresearch 1. Interest It need hardly be said that good researchis interesting to somebody. Although interestis a notoriouslydifficult conceptto captureon paper,in researchterms it often has to be defendedexplicitly, and in writing. For example,it might be necessaryto persuadean authority to make time or funds availablefor doing the research,or arguefor a proposalfor higherdegreeresearchat a university. The reasonsfor finding somethinginterestingmight obviously be anything,for example: personal • an observationof somethingunexpectedor incongruous • differencebetweentwo versionsof the sameevent • a difference in the reception of the same lesson by two parallel groups arising out of somethingread • a prediction froman establishedtheory • implicationsfrom a descriptionof someoneelse'sresearch • an unexplainedresult in previouswork • a puzzleunexplainedby existing theoriesand ideasaboutlearning • a discrepancybetweennormal assumptions(for example,from a methodologytext) and actualoutcomes basedon possibleconsequences • the pay-off from adoptinga new approachin terms of efficiency, useof resources,cost, and so on • the inside story of the implementationof a new decision. As we shall seein Chapter5 when discussingthe generationand developmentof researchideas,the original idea that sparksoff the venturemay evolve as a result of subsequentconsiderationof what has already been establishedabout the problem in the extantresearchliterature,or from the

PRINCIPLES AND PROBLEMS

59

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

preliminary resultsof the actualresearch.However, the original interest,if strongenough,will remain to guide the executionof the researchand help to anchorthe results and interpretationin the original question:and also to motivatethe researcherin the depressingtimes when all seemsdataand no sense.

2. Originality This quality is closely allied to the first, though it is possibly even more elusive. The original contribution in much published researchis in fact quite small. While it would be everybody'sdream to come up with a totally new explanation to account for a totally new set of data about a question nobody had thought of before, in practice it does not work out like that. Originality may residein anyone,any combination,or (rarely) all of: • a new question • new data- perhapsfrom a different kind of class,a lessonwithin an educationsystemnot previously researched(for example,trying out writers' conferencingin a school systemwhere it had not beenevaluatedbefore) • a combinationof data- for example,addinginterviewsto observations • a new theory or modification of theory • evaluatinga new teachingmethodor learningstrategy • a new analysisof previouslyobtaineddata • using a recognizeddatacollection methodin a new context • (even)replicatinga previousstudy to comparethe resultsand thus establishits generalizability. 3. Specificity

Empirical researchof all kinds uses observationsof specific events to uncover generalprinciples. If a teacherwas interested,for example,in how participation in class affects learning, then the teacherwould want to be very clear as to what kinds of classroombehaviourconstitutedparticipation, what signsof learningto be looking for, and what indicatorsof a relationship betweenthem to spot. Cook (1986: 17-19) gives an illustration of the processof concretizinga generalquestioninto a specific questionthat can be answered,by increasing in eight stagesthe specificity of each con'cept,or operationalizingthe general concepts(in his example, learning by speakingand learning by writing) by reducing them to actual tests and measurements(particular learners, particular tasks and achievementmeasures).Researchin the

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

60

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

normativetradition requiresthis kind of specificity, for otherwiseno measurementscan be taken. However, replacinga generalconceptlike 'speaking' or 'participation' with a speaking test or an index of student turn-takingbegsthe questionof how well the test or index chosenactually representswhat the researcherwanted to enquireabout. This 'methodof detail' standsor falls by the appropriacyof the detailschosen. Q!Jalitative research,however,requiresspecificity in a slightly different sense:specificity of context, a sufficiently detailed descriptionof the personal, institutional, and educationalcontext for the researchto represent adequatelythe individuality of the situation. A qualitative approach to Cook's questionmight thereforelook at different individuals in the same class and record all the different types of learning mode and spokenand written utterancesproducedby them and relate the observationsto pupils' self-reportsor interview data, placing the datain the contextof the history of the classor the aims of the programme.This particular issueis crucial to the questionof how researchquestionsare generated,to be examinedin Chapter5.

4. Publication Publicationmay seema strangefeature to be groupedunder initiation and undertakingof research,but researchactivity needspublication and publications in two seriousways. First, researchideasand ways of doing the researchare often suggested by readingthe professionalliterature.The originality of a pieceof research is usually evaluatedby how it compareswith what hasbeendonebeforeon the topic. In a thesissubmittedfor a researchdegree,the literature review is an integral and necessarypart: the issuesidentified and pursuedby previous researchersand the methodsthey usedare evaluated,and the thesis writer continuesto demonstratehow the new researchwill contribute to the sum of knowledge. But, although researchconductedby teachersin tHeir own classroomsdoes not need to follow the samecanonsof display and presentationas a PhD, the topic is not usually without some kind of history. The teacherresearcherwill still want to know how other teachers' thinking compareswith their own, how previous attempts to solve the problem have fared, what the resultsof similar studieshave been,in order to make a judgementabout the direction, methodsand perhapsgeneralizability or at leasttransferabilitymtheir own research. Second,any researchrequires disseminationthrough conferencepresentationor discussionor publicationso that it may be evaluatedby others. Public perusal,in particular peer perusal,is part of the processof validation. A teachermay discoverto his/herown satisfactionthat, for example, conferencingworks reliably for their studentslearningto write in a second language.Until the details of the work are available for other people (in

PRINCIPLES AND PROBLEMS

61

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

some suitable and accessibleforum, it does not have to be a research journal in a university library!) no one elsecan questionthe methodsused or check the plausibility of the conclusions:becauseof that, no one else need believe the research,and, crucially, neither can the researcherhave confidencethat his or her conclusionscan be accepted. Designand methodology 5. Sensitivity

Researchaims to discoverboth broadgeneralizationsand subtledifferentiations. In languagelearning researchthere have been many attempts to demonstratelinks between broad categoriesof personality and achievement, and concurrentlythere have been many investigationsof individual learnersfollowing particular courses.Sensitivity is, however, more than a questionof scale, it is primarily a questionof quality of data, discriminatory power of tests,and the useof 'insider knowledge',particularly in the caseof participantobservatoryresearch. It may take some time to learn the skills necessaryto obtain goodquality data: dependingon the project, the researchermay needinterviewing skills, practicein constructingquestionnaires,careful observationand recording techniques,all of which we discuss in Part 2 of this book. Allwright refers to the problem of skill-learning time in his report of a workshop in 'exploratory teaching' (1993: 129-30); perhapssurprisingly, his workshopcollaboratorsdid not seemto regardit as seriouslyas he did. Perhapstheir enthusiasmcoloured their judgement,or perhapsthey saw ways of incorporatingresearchskill learninginto the developmentof their teachingskills. There have been a number of researchprojects whose inconclusiveor disappointingresults have been blamed on measuresand tests that were too crude to show up the sought-afterdifferences,particularly in the area of individual differences in learning. One such issue is the controversy surroundingthe supposedadvantageof extrovertsover introvertsin learning a foreign language(Skehan,1989). Similar problemsmay arise when using standardtests as measuresof achievementon a course: the unreliability of the testsmay maskany true gainsin proficiency. As Burton and Mickan remark (1993: 113), discussingteacher'sclassroom research,'the closenessof teachersto the eventsbeing researched'is a frequentargumentfor teachersto be involved in research.They identify particularly 'information from the front line: teachersare most familiar with the complex circumstancesinvolved in the teachingand learning of languages;what they experiencein classroomsis a direct sourceof information aboutlanguagelearning.'

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

62

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

This kind of closenessshould promotesensitivity, but it is striking that later in their list of reasonsfor languageteachersto do classroomresearch is 'learningthrough research:teachers'researchpromotestheir own learning about what is happeningin their classrooms,and their understanding of why it is happening;through researchteachershypothesizeabout their experience'(1993: 114). Thus sensitivity to issuespeculiar to languagelearningin classroomsis a great asset to the teacher as a participant researcher;but doing the researchmay be the bestway of maximizing that sensitivity.

6. Objectivity Classicalresearchdesignsstrive for the elimination from the resultsof an experimentof any biasesfrom the researcher:the knowledgegainedis seen as objective and independentof any particular humanagent,with personality, emotions,careeraspirations,hopesand desires.Thus, the messageis important, not the messenger.It is obvious that in educationalresearchas well, particularly in evaluation, the possibility of communicatingto the learninggroup what the researcherwantsto hearis only too open. If a teacherwanted to compare,say, teachingvia group discussionwith teachingusing video clips, and had strongreasonsto prefer one to the other, his/herresearchmight unconsciouslylead the studentsto prefer the alternative he or shepreferredby making it more interesting,being more relaxedin it, or devoting more time to it. An objective approachwould build in safeguardsagainstsuch 'experimenterbias', perhapsby substitutingthe teacher by otherswho did not havesuchpreferences.However,any objectivity gained by using different teachersmight be neutralizedor at leastcompromisedby the effects of disrupting the normal arrangementsfor the class or classes. Thus objectivity is a laudable goal, but so is contextual specificity: the problemis to satisfyboth concurrently,not to sacrificethe one for the other. One argumentfor action researchis that it has typically acceptedthat the actors, including the researchersas observers,are an integral part of the situation being researched,not simply nuisancesto be eliminated or controlled out of the way. Dadds (1995) argues that there are serious reasonsconcerningcompletenessof descriptionand the recordingof significant factors for participantobserversor teacherresearchersto carry out 'first-person research'into their own reactionsto events, including even the recordingof emotional reactions.This is one justification for treating diaries,which are the leastobjectiveof all data,as researchdata.

7. ValidiZy Supposea researcherfollows a particular student through a language courseand beyond,perhapsinto the communityto look at languagemain-

PRINCIPLES AND PROBLEMS

63

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

tenance or successin a job requiring that language. The researcher observesthe student in the class, holds interviews, records achievement scores,takesinto accountscoreson IQand individual differencemeasures, reads his diaries, observeshim after the course.Two questionsimmediately ariseout of any descriptionthe researcherwrites: 1. Does the interpretationof all this data accordwith reality - presumablythe student'sreality in this case? 2. Is it possible from this interpretation to suggestthat other students in similar circumstancesmight learn, react to the course, meetproblemsafter the course,in the sameway? Question 1 is a questionof internal validity, or credibility; question 2 is one of externalvalidity, or generalizability.Another illustration of internal validity is afforded by a famous study of learner strategies,the 'Good LanguageLearner'study (Naimanet al., 1975). In this, a numberof kinds of data taken from school learnersof French,by observations,mental tests and interviews,were relatedto achievementvia two kinds of tests:a standardized listeningcomprehensiontest and an 'oral dictation' task in which the studenthas to repeatsentencesspokenby the teacher.The extent to which the readerwill acceptNaiman et al. 's conclusionsis dependenton how happily they acceptthe validity of measuringlanguageby thosetests. Thus, validity is essentiallyto do with credibility, which is the term used by Lincoln and Guba (1985). However, an even stricter criterion of validity can be entertained,namely that researchneedsto be credible not only to consumersbut also to the original participantsin the situation under review. How this might be achievedis a separatequestion. 8. Reliability

In any kind of measurement,reliability concernsthe confidencethe user can have that the measurewill give the sameanswergiven the samething to measure.For example, nobody would trust a languagetest that produced wildly different answersif given to the samepersontwice, just as one would expect apair of scalesto weigh a pound of sugaras a pound, however often one weighed it. In practice, of course,no one expectsthe complicateddata arising out of languagelearning situationsto be as reliable as a pair of scales,partly becauseit is just a different order of complexity and partly becauselots of the dataare not, in any meaningfulsense, measurements. In languageproficiency testing, for example,there are recognizedproceduresfor estimatinghow unreliablethe test scoreis: that is, what its rangeof accuracyor 'spreadof error' is. If a test scoreonly claims to be reliable within two scale points either side of the actual score (imagine a scoreof 50 ± 2), it is effectively sayingthat it cannotguaranteethat the true scoreof that testeeis any particular one between48 and 52, but

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

64

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

the best obtainableestimateis somewherein that range.To put it another way, there are limits within which a test receivercan be confident the true score lies, and statistically thoselimits can be given a quantity, like 68 per cent, 95 per centand 99 per centconfident. To take a non-measurement example,a researchermight wish to ask pupils to write diaries: but there is no guaranteethat the picture of the class so gained will be the best possible, simply becausesome of them may not cooperate,somemay lose them, or not be in classon the day they are collected.Data-collectionproceduresall suffer from problemsof reliability, which, again following Lincoln and Guba, might more appropriately be glossedas 'dependability'. Another instance of reliability concerns data analysis. Just as essay marks might be made more reliable by double marking, so coding, or labelling, a sequenceof classroommoves,or themesoccurringin an interview transcript, may be made more reliable if the task is done by two or three people independentlyand the decisions with the least agreement rejected. However, we should bear in mind Elton's (1995) stricture that there is no contestbetweenvalidity and reliability: validity is far more important. He relatesa tale about a scientistwho looked for the watch he had lost not where he had droppedit but under the table lamp, becausethat was where he could see best - clearly a reliable, well-illuminated situation, but doomedto failure.

9. Falsifiability Knowledgedevelopsby finding interpretationsand understandingswhich supersedeolder interpretations,understandingsor descriptions(looseh theory), and empirical researchis usedto provide evidenceto supportthat development.A classical principle in empirical work is that progress comes through finding evidencethat tells us that a theory is wrong, not simply by finding evidencewhich confirms what was alreadythought. For this to be possible,a theory (again,without attachingany technicalsophistication to this term) needsto be sufficiently preciseto be capableof being shown to be wrong. A theory which says'sometimesx happensand somebe falsified unlessit can statethe conditionsprotimesy happens' cannot ducing outcome x and distinguishesthem from those that producey. If this holds, a theory can be said to be falsifiable. Obviously, it is a measure of the strength and importanceof a theory that it may take a very long time and much ingenious researcheffort to prove that it is wrong, and replaceit with anotherthat accountsfor the new evidenceplus all the evidencethat the older theory accountedfor. This principle is the reasonwhy most statistical researchattempts to reject a null hypothesis (that, for example,there is no differencebetween two treatmentson somemeasure)

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

PRINCIPLES AND PROBLEMS

65

by showing that the averagedifference observedis so unlikely by chance that it must be true. However, although falsifiability is a respectedgoal, it has beenarguedthat in reality most researchefforts proceedby confirming existing notions: by looking for supportingevidence.Changecomes about when the search for confirmation in fact producesa build-up of anomalousand disconfirming results which then provoke a switch to a new, more widely applicabletheory. In languagelearning researcha number of authors(McLaughlin, 1987; Ellis, 1994) have pointed out that the principle of falsification is often ignored,or at leastnot fulfilled. Schumann(1993) explores the implications of this principle for languagelearning research,and similarly concludesthat it is not feasible in most situationsto adhereto it. The problem is a generalone. As we saw in Chapter3, naturalisticmethodsalso look for discrepantcasesor evidence disconfirming their interpretationsin order to test and refine those interpretationsfor the most accuratestatementpossible;but they are necessarily dependenton what data might occur naturally since they do not manipulatethe contextfor the purposeof the research.

Applicationsto other situations: three aspectsof transferability 10. Replicability A piece of researchwill be highly valued if it is so clearly reported that somebodyelse can do the samething again and obtain the sameresults. Accuratereporting of a piece of researchis thereforenot just a matter of good manners:the practical possibility of repeatingthe researchmeans that it is opento a test of reliability. In fact, replication studiesare surpris- . ingly infrequent in the standardapplied linguistic literature, and at least one author (Brindley, 1990) has arguedthat this reducesthe value of the literature. Naturally, there is a prima focie conflict betweenreplication and originality, and it is understandable that peoplelooking for a researchtopic (discussedin much greaterdetail in Chapter5) prefer to go for what looks like a new thing to look at rather than repeatingsomeoneelse's work. However, to repeatsomeoneelse'sstudy in a new situation on the basisof an explicit and completereport is itself a useful activity, and if the results are different, interestwill be high. The replication issuetherefore touches directly on other featuresmentionedhere: originality, reliability and falsifiability. Furthermore,there is a genreof research,called, by CohenandManion (1989), 'meta-analysis',which is related to the replicability issue, but which involves comparingas many studiesas possibleof a particular phenomenonto establishthe generaltendencyand the weight of evidence In languagelearning studiesthere .have been several: Peal and Lambert's (1962) examination of cognitive deficit in bilinguals and Long'S (1983) examination of studies comparing 'instructed' and 'natural' language learning are often quoted as examples.·The significance of replicability

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

66

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

depends,however,on anotherfactor: the uniquenessof teaching/learning situations. Doing the samekind of observationin a class of, say Algerian learnersin Algeria, Japaneselearnersin Japanor a multilingual group in Bournemouthmay not produce the same results: that cannot mean that the three observationsare any less valid. Thus, in context-boundresearch, failure to replicate does not of itself invalidate. However, tpis problem gives a very strong reasonfor reportsof how the researchwas conducted to be as completeas possible,so that the crucial differencesin the contexts may be identified. (For further discussion,see Chapter 13 on casestudy research.) 11. Generalizability

In one sense,all researchis about making significant generalizations(i.e. the most general and accuratestatementpossible) about some problem area. Educational researchand researchin languagelearning situations are no exception.Thus, a linguist will be looking, for example,for the mosteconomical way to capturegrammaticalregularitiesin a rule, or the set of primary and secondarymeaningsof a word in a lexical entry; a classroomresearcher will be looking for the most importantfeaturesof the classroomsituationand how they interact. In this sense,researchworks from the particular, specific data, to general statementsthat data can support. In applied linguistic research,much activity hasbeendevotedto seekingthe 'conditions'in which languagelearning generally occurs - Spolsky devotes a whole book to it (1989). 'Condition-seeking'research,often with regard to individual differenceslike aptitude,motivation, and so on, has attemptedto establishgeneral statements(Skehan, 1989). There is another sense of generalization,to anothercontext or situation, which is much less congenial.As we discussed in Chapter2, in many casesteachingrequiresthis kind of generalization:taking principles derived in one situation and generalizingthem to new situations. This senseis more properly a matter of 'action' than 'research',and therefore will be discussedlater under 'utility'. Here we should explore a little deeper the notion of making general statementsin context-bound researchand researchconductedby participantresearchers.The purposeof this kind of researchis to achievedeeperunderstandingof those particular contexts;suchan understandinginvolves, necessarily,recognizingboth what is special and unique and what is characteristic,normal and commonplace. Erickson (1986), in discussingqualitative research,draws a rather surprising parallel betweenthis aim and that of a linguist looking for universal principles of language,and describingparticular languages.His point is that, in both cases,the universaland the particularare establishedby exactand complete descriptionof the phenomenaconcerned.This point will be taken up in Chapter 13 where the notion of naturalistic generalizationis discussedin relation to casestudy research.

PRINCIPLES AND PROBLEMS

67

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

12. Utility

The second,and in one way more practical,senseof generalizabilitymentioned aboverefers to the utility of the research.Can the findings be used, either in the future in the immediatecontext,or applied to other contexts? Commonly, there are two generalpositionson this. One is that empirical researchfindings are in fact of little practical use: 'The applicationof basic researchfindings to professionalpracticeis alwaysa risky business.This is no less true in the case of languageeducation ... researchfindings are often too incompleteand limited to indicate substantialand unqualified practicalimplications'(Beebe,1988: 106). The other is that the function of researchis to help us draw out assumptionswhich we can question: 'Researchfindings can challengeour sometimescomplacentand poorly substantiatedbeliefs aboutthe way learningoccurs'(Beebe,1988: 107). Beebe'sremarkswere made in the context of the relationshipbetween secondlanguageacquisition researchand secondlanguagepedagogyand teachertraining, and Ellis (1994) concludeshis massivestudy of second languageacquisitionwith a careful review of currentopinion on this issue. Ellis notes in his final remark, after discussingvarious positions from 'don't apply' to 'go aheadand apply': 'Furthermore,someeducationalists might feel that researchundertakenby professionalresearcherswill always be of limited value to languageteachersand that a more worthwhile and exciting approachis action research,where teachersbecomeresearchers by identifying researchquestionsimportant to them and seekinganswers in their own classes'(1994: 689). It may well be that the main utility of researchis not the commonplace of direct applicationbut the experienceof challengedassumptions,which seemto make otherwisefamiliar situationsmore interesting,curious, and indeed less familiar. Researchcan put new perspectiveson to old situations, which is exactly why it can contributepowerfully to innovation and the maintenanceof innovationin context. 13. Ethics

There are, mercifully, few parallels in languageeducationresearchwith headlineethical issuesin sciencesuch as military applicationsof nuclear physics, biology, or chemistry,or the use of animals in the drug and cosmetics industries.However, there are important ethical questionsregarding the collection, interpretation and publication of researchfindings, which affect both the researcherand the clients or subjectswho provide the data.Thesequestionsconcern: I. accessto the situation in which the researchis to take place 2. protectionof the clients who provide the data 3. protectionof the validity of the datacollected

68

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

4. ownershipof the data 5. agreementsaboutdisclosureand publication 6. codesof practice about generalizability and application in other contexts. 1. Access.In most languageeducationsituations,and in all institutional ones,some kind of permissionis neededto carry out research.This permissioncan usually only be grantedby a seniorpersonwho actsas a gatekeeper. One concernthat such a personmay have is, of course,that the researchmay interfere with the teachingand learningactivity or, depending on design,disrupt the normal running of the institution beyondtolerable levels. 2. Clients need to be protectedby assurancesabout all proceduresto ensureconfidentiality. Thesemay be situationsin which the researcheris vouchsafedprivate information concerning,say, opinions about another participant (the teacher,a fellow teacher,anotherstudent) which would if passedto that person. causeembarrassment 3. The data needsto be protectedso that it is of the highest quality: simple proceduressuchas askingintervieweesto vet a transcript,give permission for it to be usedin specifiedways, and so on, can ensurethis. 4. Data providedby peoplein a social learningsituation,an observation, or an in-depth interview, should be considered the property of the providers.Recently,this principle hasevenbeenincorporatedin law. 5. None of the abovewould be possibleunlesssomeexplicit agreement is made with the data providers or those responsiblefor them about the conditionsof disclosureand publication. Publicationand disseminationof resultsare themselvespart of research,and the enterprisewill not get off the ground unless the participants'agreementcan be obtained on such conditions,whetheropenor restricted. 6. Somepoor decisionsabout programmedesignhave beentakenin the past when choiceshave beenmadeon the basisof inappropriateextension of researchfindings to new contexts.A code of practicedesignedto limit this kind of damage has been proposedby EUROSLA (the European Second Language Association), for use by its members, who are researchers.In the currentclimate, further initiatives of this sort are to be expected. Many social research organizations such as the British SociologicalAssociationhaveestablishedcodesof professionalethics. As mentionedbefore, thesefeaturesof good researchare not independent, they interact: the quality is in the balanceand compromise.There may be tensionsto be resolvedfor any particularpieceof researchbetween confidentiality and protection, and publication; between sensitivity and reliability; betweenoriginality and publication; betweenreplicability and context-specificity;and possibly betweenvalidity and reliability in some cases.

PRINCIPLES AND PROBLEMS

69

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Comparison of research traditions on thes(. features Educationalresearchershave used many researchmethods,as is evident from manualssuchas Cohenand Manion (1989). Stenhouse(1975) points out that in geography,different projectionsof the earth are useful for different purposes- large-scalecharts use the 'Mercator' projection for crossingseasand oceansand the 'Gnomonic'projection is usedfor smallscaleharbourcharts.Both suffer from the error of representingan irregular sphericalsurfacein two dimensions,but that distortion is quantifiable. Thus different researchmethodsserve different kinds of problems and different purposes.However, that is not a purely relativist position, for thereare generalcriteria. How do the variousresearchtraditionscompare? Hopkins' view is as follows (1993: 171): Criteria such as validity, reliability, generalizability,are necessaryif teacher-researchers are to escapethe sentimentalanecdotethat often replacesstatistical researchdesignsin education,and gives teacherresearchsuch a bad name. Enquiry, self-monitoring, and teacherresearchneedto establishstandardsand criteria that are applicableto their area of activity, rather than assume(and then reject) criteria designedfor different problems. Without necessarilysubscribingto the view that teacherresearchis often sentimentalanecdote,we can, however, agree that appropriatestandards can be established. By and large, traditional numerical designs aregood on objectivity, reliability, falsifiability and replicability, and weaker on interest and originality, context-specificityand utility. Qualitative approachesare good on interest, originality, sensitivity, context-specificityand validity. They are sometimes thought to be weaker on falsifiability and generalizability.Recalling the discussionof 'discrepantcaseanalysis'in the last chapter,it is clear that qualitative designsdo not haveto be any weakerin falsifiability than quantitativeones. The commentsabout utility and ethics apply in equal measureto both broad traditions; individual pieces of research may well have variable strengthsand weaknesses in this regard. Researchby participantteachersand action researchmay be very strong on interest,originality, context-specificity,validity and utility, but weaker on publication, reliability and replicability. On publication, there is often a feeling'thatthis kind of researchmay accentuatethe uniquenessof the situation and play down the history of the topic as developedin other contexts. But there is also a dangerof re-inventingthe wheel, of unwittingly replicating, without referenceto the history of thought which lies behind every researchtopic. A quotation from Bassey(1986: 24) may illustrate a prevalent attitude: 'The only referenceto the educationalliterature is to P's sourceof inspiration.There is no tediousreferenceto queuingtheory

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

70

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

or to physiologicalexplanationsof fidgeting. Action researchersuse the literatureonly to the extent that there is somethingsignificant and germane to the issue under study: they do not genuflect to Pavlov and to Piagetin order to impresstheir readers.' In so far as this quotationputs a higher valuation on relevanceand significance, it cannot be criticized: but the suggestion that a literature search,and the typical literature review that is normally part of a research report, are there to impressthe readerscannotgo unchallenged.A literature review is about writing the history (briefly, of course)of approachesto a subject and to the methodologyof previous investigationsto establish the crucial questionslurking under the surface.It is there to establishthe ancestryand the originality of the new work. Schecterand Ramirez(1992: 192) discusstheir findings after studying a teacher-researchgroup associatedwith their university in the USA, by monitoring meetingsof the group and analysing the in-house reports of their work. It is thereforea kind of meta-analysis.They found a numberof important differencesbetweenteacherresearchersand university 'professional' researchers.The first of these concernedthe role of the literature search: the university researchers,whatever researchtradition they subscribed to, neededto demonstrateexplicitly that their work contributedto an existing body of knowledge;the teacherresearchers,on the other hand, quoted other people'swork on their topics where it had been useful for them, but without undertakingsystematicreviews. The secondconcerned reliability and validity judgements,in particular the role of personalexperience. For the university researcher,this would not suffice; for the teacher researchers,it was regardedmore highly, although many also attemptedto supportit by other forms of evidence.Third, they found that severalpieces of teacherresearchdid not 'addressa problem' but rather took teaching methods and described their implementation and success. University researchusually requires question areasto be painstakinglyhoken down into crucial problemsor issueswhich the actual researchcan be performed on. A further very interestingpoint made by Schecterand Ramirez is the difference in standardizationof forms of reporting, and that, as teacher researchmaturesand becomesmore widespread,so such standardizationof rhetorical forms may increase.They describethe 'double narrative',i.e. the descriptionof the researchinterwovenwith the narrativeof the researchers' development,and point out that a frequent style of reporting usesthis, for the reasonthat teacherresearchersare strongly awareof the reflexive relationship betweenthe roles of teacherand researcher,which contrastswith the university researcher's claimsof objectivity.

Aspectsof designof research It will be evidentfrom the foregoing that thereis no simple formula which

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

PRINCIPLES AND PROBLEMS

71

guaranteesgood research, andthereis no necessityfor researchto useonly one method. In fact there are good reasonsto incorporateseveral techniquesin data-gathering.This allows the opportunityof greatercredibility and greaterplausibility of interpretation.For example,a researchermight decide to find out the opinions of a group of learnersabout a particular approach through a questionnaire,which would elicit easily countable answerson a rangeof standardquestions,but supplementthis either with interviews with a sampleof learnersor a contentanalysisof diaries written at the time of the introduction of the new approach.The two kinds of data, quantitative and qualitative, may coincide on a number of points, thus strengtheningconclusionsdrawn from them, and diverge on others. In general,such a procedureis called triangulation- anotherterm taken from geography,referring to finding an unknown position from the intersectionof threecompassbearings.Denzin (1978) arguesthat triangulation takes place in four different areasof the researcheffort: combining data sources,using comparisonsof theory and individual accounts,using multiple methods, and several observerswhere possible. We return to this important conceptin Part 2. An obviousexamplewould be as follows: in a small-scalestudy of studentopinion, a researchermight conduct both a sessionof individual interviews and a focus group discussion,in order to test the depthsof feeling on particular points and its resistanceto change in social interaction. In general, mixing methodsof data collection aids validity (seeChapter14 for a fuller discussion). Methods of analysismay also with profit be mixed, to check on interpretation. In a study of studentslearning to write in a secondlanguage,a researchermight want to look at the written productsboth in terms of the frequenciesof certain grammaticaland rhetorical structures,and at the global impressionsof comprehensibilityrecordedby a sampleof readers. Some large-scale multimethod studies by academic researchershave reachedthe press (for example, Bachmanand Palmer, 1982, on testing; Andersonet at., 1991, on test-takingstrategies).Bearingin mind Schecter and Ramirez's(1992) point about the strengthof the argumentfrom personal experience,it seemsthat teacherresearcherscould profitably design into their researchseveralkinds of data collection and analysisprocedures to test and supporttheir own convictionsfrom experience. A further problem in researchdesign of any kind is the nature of any comparisonsto be drawn. Purely descriptiveresearchtakesa situation and describesit: it is sometimescalled 'one-shot' research.There may be a numberof possibleother comparisons.An examplewould be a time series in which the developmentof a situation over time is charted. Another would be the use of different treatments,either within the samepersonor group at different times, like a study of reading strategiesfor different kinds of text in a foreign language.In this case, there may be a need to control order effects, so that reading the different texts is not confused

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

72

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

with readingthem in a particularorder. Another kind of treatmentmight require two different groups at the same time, as when two different classesare observedreceiving the samelesson.Yet anotherkind of treatment comparisonmight involve two groupsreceivingdifferent treatments, which becomesthe classic'Group A Group B' or 'experimentaland control group' design. Manuals of researchdesign contain a large variety of possible design featureswhich may get very complicatedin order to allow unambiguous conclusionsto be drawn: some of the more common ones will be introducedlater. The main point to be drawn out here is that considerationsof good design affect all kinds of research,by anybodyengagedin research. An exploratory 'one-shot'design may be correct for the particular situation and purpose,but, if so, the researcherhas to considercarefully the completenessand the quality of the data which she or he can use in the exploration,just as a researcherusing a complex comparativedesignlooking at changesover time in severaltreatmentgroupshasto be careful that the data collected is in a form that will allow the planned comparisons actually to be madeat the end of the day.

Conclusion This chapterhas introduced 13 key terms that are conventionallyusedto discussand categorizethe natureof 'research'as a concept,althoughthey are not usually brought together in this way. We have, to put it another way, attemptedto provide a metalanguagethat can be used for talking aboutdifferent approachesto research.It is probablytrue to say that validity reliability and generalizabilityare thosethat surfacemost frequently and often quite technically - in the literature on researchmethod: nevertheless itis importantto rememberthat eachindividual featurehasits own kind of significancedependingon the type of researchproject undertaken. Thus, for example,JalsifiabiliO' is a major concern within a normative approach,ethical considerationsplay a large and explicit role in many interpretivetraditions,and publication (not quite the sameas going public) is a sine qua non in many academiccontexts. The 13 criteria have been proposedhere as a framework for evaluating 'good' research.However, given our earlier descriptionsof different, even competingparadigms,it should be clear that eachcriterion is open to different possibleinterpretations,and this we have tried to draw out in the courseof the discussion.Objectivity for instance,is a complex andcontroversial term, difficult to define and sometimeseven rejectedas a goal of good research.All the featuresset out here 'in principle' will recur in various ways in the subsequentmethodschapters,as will the issueof mixing methodsand the role of triangulationin dataanalysis.

PRINCIPLES AND PROBLEMS

73

Discussionnotes

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

1. Taking any piece of researchwith which you are familiar (some-

thing you havedoneyourselfor beeninvolved with, either as participant or subject, or perhapssomethingyou have read about), try to evaluatethe extent to which the researchaddressedthe 13 criteria. It may have done so explicitly, or alternativelysome criteria may only havebeenimplied, or beenabsentaltogether. 2. Do you consider that certain criteria are best suited to certain kinds of research?For example,are someof them more applicable than othersto an action-research approach? 3. How would you define the term 'objectivity'? We will seethat it is an importantone for severalof the methodsto be discussed.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

This page intentionally left blank

5

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Generating research

Introduction This chapteris the final one in which researchis discussedin terms of its underlying principles, though at the sametime it is intendedto provide a springboard for a subsequentexploration of the methods which operationalize those principles. The chapteris basedon the assumptionthat a researchstanceis integral to the view of the extendedprofessionalthat underpinsthis book, and not a kind of 'bolt-on extra' only obliquely related to one'severydaymainstreamduties. Its centralaim, then, is to examinethe questionof where researchideas come from, and the kinds of topics that might appropriatelybe addressed.By and large, for the purposesof the presentdiscussion,we adopt the convenientposition that contenttends to go in searchof methodratherthan vice versa,though we shall seelater that this is by no meansalways the caseeither in theory or practice,particularly with data-richsources.The approachtaken here is intentionally an eclectic one, coveringboth researchthat is teacher-generated and executed,but also the useand uptakeof researchdoneby others. We begin by reviewing someof the factors on which the origin and execution of researchin a working contextdepend.This is followed by a runthroughof the key elementsin the debateabout the relative roles of theory and data and their relationshipto paradigmdistinctions.The next section illustrates the kind of content that is nominatedby (a small sample ot) teachersof English as a foreign language.The chapterendsby askinghow existing researchmight impinge on the activities of practisingteachers.

Beginnings In Chapter I we looked at some typical teachers'in action' in theit working situations,noting particularly the ways in which factors of contextand role affect, constrainor facilitate their room for professionalmanoeuvre.A key point there was the assertionthat all theseteachingsituations,however disparate,have an inherent potential for researchto take place. This section is concerned,in fairly concreteterms, with how that potential might

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

76

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

becomeactual. Let us therefore supposethat the teachersin action in Chapter1 have, as it were, developeda researchconsciousness such that they are now working on specific projects. In other words we will tell a little more of the stories of a few of them, with a couple of new people interspersedto broadenthe spectrum:the aim is to give a senseof the variety of ways in which a teachermight be stimulated to do research. There is of coursea jump in logic here, becauseit will not be until the subsequentsectionthat the genesis,sourcesand developmentof research ideasare moreexplicitly discussed. Kenji Matsudahas now beenon a six-month languagedevelopmentand teachertraining programmein the UK. During his stay he was required (by the Ministry) to write a project directly addressingsomeaspectof his own situation.He choseto work on the methodologyof vocabularyteaching. Now back home, he is linking his project work with everydayreality by analysingthe set textbook and tabulating its methodsof introducing and reinforcing vocabulary,and by selectivelyintroducing new techniques then testing his classat regular intervals to seewhich methodsencourage greaterretention. He plans to persuadecolleaguesto tryout something similar, and possibly to discoverwhetherdifferent methods,including use of the mothertongue,are suitableat different proficiency levels. Anna Garcia is a Colombian university teacherof English with many years'classroomexperience.The main part of her job is to teach reading skills to undergraduates in different subjectareas.Although the materials used are relatively up-to-datein the senseof incorporatingcurrent views on the natureof comprehension,this teacherfeels that her students'reading skills in English remain at a rudimentarylevel. She suspectsthat this may be related to, though not necessarilycausedby, inefficient reading skills in Spanish,and in turn to the fact that languageis typically taughtin the schoolsas a grammaticalsystem.Shehasmanagedto get a scholarship for a PhD to researchthis issuein depth. Her plans include an investigation of what other researchershave found, the developmentof instruments to measurereadingefficiency in two languages,and an analysisof pedagogicpractice. Ann Barker'sschool decidedto go aheadwith its plans for more specialized programmesto run alongside its general English courses,both as whole packagesand as part-course options. They have started with 'BusinessEnglish' and 'Medical English', and AB, in view of her background, has been put in chargeof the latter. Existing materialshave been useful up to a point in getting started,but sheis uncomfortablewith them becausethey either overgeneralizeor focus on one professionalsubgroup in a way that is not obviously relevantto another.Shewould thereforelike to preparein-housematerial that is more focusedin termsof topics, skills,

GENERATING RESEARCH

77

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

discourseand languagestructure.She has startedby contactingher local hospitalwith a view to collecting (a) typical written dataand (b) samplesof spokeninteraction,in the first instanceas it relatesto nursingstaff FrankJonesworks in a British university in the areaof teachingEnglish for Academic Purposes (EAP) to non-native postgraduatestudents. study, Togetherwith a colleaguehe has completeda questionnaire-based involving about 300 students,designedto investigatethe attitude changes of new studentsin relation to both academicand social-personalmatters as they adapt to a new culture and progressthrough the academicyear. Initially intended for internal university circulation, the results when reported at a national conferencefound sufficient resonancethat colleaguesin other universitiesare now seekingto replicatethe original study and compareresults. Carol Turner has become increasingly interested in looking in more depth at the learning patternsof her pupils. She knows that she will have to set aside some time to review and update her earlier training in the teachingof young learnersin general. However, she also plans to collect empirical data on her small groups in each school, and on individuals within them, probably by keeping a log over an extendedperiod of time backed up with samplesof the children's work. Her accessto staffrooms means that she can liaise closely with mainstreamteachers,who have agreedto make availablethe normal classwork of CT's pupils and others, and to be interviewedfrom time to time. Although theseresearchinstancesare necessarilyselective,we can now, so to speak,squarethe circle with Chapter I and briefly enumeratesomeof the parametersof researchactivity within the broaderframework outlined there. (It is worth noting that Hopkins (1993) offers a comparableselection of casestudiesof teachers- he calls them 'reflective professionals'carrying out research within mainstream education in Britain and Canada.) All theseteachershave madechoiceswhich, whateverthe eventualoutcome, have an organic relationship with issues that are pedagogically familiar to them. At least from their perspective,then, researchentails questionsof: • • • • • • • •

methodschoice(the substanceof Part 2) focus and topics timescale time available participation(individual, collegial, collaborative) access(to institutions,participants,literature) scope review of the work of others

78

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

• outcome (personal/institutional change, interest, conference, higher degree...).

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

All these will surface in one form or another in subsequentchapters. Thereare also questionsof: • the initial stimulusfor the research • paradigmchoice which are both relatedto the wider issuesof approach.

Approaches The logical leap admittedto earlier meansthat we havenot really enquired as to where these teachersgot their researchideas from, nor how those ideas developedinto a researchplan. Behind this seemingly straightforward statementis a question that has been extensively discussedin our field (as in many others), whether in linguistics, secondlanguageacquisition studies, classroom research and many other areas. Any research undertakingtherefore will need to have an angle, at least implicitly, on such a pervasiveissue,not leastbecauseit is underpinnedby basicchoices in researchphilosophy. Theoryfirst?

The 'classic'debateis well coveredin the literature. It is relevant here in so far as it points up the particularlocus/loci for the kinds of researchthat teachersmight do in the broad senseintended here, and becauseit taps into a long tradition of debatein the philosophyand practiceof both the natural and social sciencesfrom which even a small-scaleand modestly conceivedproject cannotreally be separated.We restrict ourselveshere to a brief and very simplified overview: lengthierand accessiblecommentcan be found in Allwright and Bailey (1991), McLaughlin (1987) and Seliger and Shohamy(1989). A long-established,historically dominant view of research(albeit possibly as much honouredin the breachas in the observance)seesit as beginning from a theoreticalposition with the goal of testingout or refining that theory. It is an essentially'nomological' approachthat works from and towardsthe establishment of laws of varying degreesof specificity. Clearly it is an approachmost closely associatedwith experimentalscience,with concomitant proceduresof control of variables and the setting up of hypotheses.It is often referred to as 'hypothetico-deductive'or 'analytic', typified by a set of researchstagesalgorithmically followed throughon a 'if x theny' formula. SharwoodSmith (1994), for example,a secondlanguage acquisition (SLA) specialist, provides many illustrations of how much

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

GENERATING RESEARCH

79

researchin this areais concernedwith theory-buildingand theory-testing, such as modelling the mental representationof linguistic knowledge, or the delineationof the conceptof Universal Grammar(UG) (seeChapter3 for a relateddiscussion). There is no reasonin principle why a teachershould not undertakethis kind of research,or find its ethosuncongenial,given a familiarity with the theoretical bases,though it may be more appropriately carried out in a context where it is the sole focus and where there is plenty of accessto the literature, such as when working for a higher degree. For a number of reasons,however,a teacher'sworking life hasa complexity that is different in kind from the undoubtedcomplexity of the 'theory first' approach.

Sourcesof researchideas First of all, the notion of 'theory' will probably have a different connotation from that of 'grand theory' and abstractedlaws. As we have commentedbefore, teachers'theory is more likely to consistof their personal constructs,assumptionsand beliefs, all of which will be aI} amalgamof many factors - experience,value system, training,reading, other people, and so on. Second,a hypothesiscan be formulated rather more loosely so that it becomesan idea that is a starting point to be tested out. Third, teachersin action are groundedin the context of their everydaypractice, surroundedmost obviously by raw data rather than overt theory, so that a qualitative paradigm- though by no meansalways qualitative method- is likely to be intuitively more appropriate.McLaughlin (1987) is one of several writers to explicate the data-driven,inductive approachto research which starts from empirical study (before, at least for the professional researcher,proceedingtowardstheoreticalstatements).This approachmay also be referred to as 'bottom-up', or 'heuristic', generatinghypotheses ratherthan testingthem. How then, in practice,might a teacherget started?Most likely it will be triggeredby an aspectof practice,such as looking at a class,critical reflection after a lesson or while preparing one, or chatting with colleagues. Often there will simply be a senseof dissonanceor uneasewith what Berthoff (1987: 33) calls 'tight schedules,leakproofsyllabi, the instructor's manual,and the gilt-edgedstudy guide' (and we can think back to Schon's 'swampylowlands' instead).This uneasemay be formulated perhapsas: • Why doesthe classnot respondto (certain kinds of material)? • How can I1we know (when bestto switch to anotheractivity)? • I1she wasn't satisfied with (the way I handledthe student'squestion). • This coursebookhas(inappropriatetexts). • X alwaysgoeshis own way - it's really irritating.

80

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

• Does everyoneelse find (that the level is different from the test results)? • What if (I do somethingdifferent from what they'reexpecting)? This kind of starting point has beenformalized in terms of 'critical incidents' in teaching,momentsor eventswhich trigger a step-by-stepinvestigation and understanding of aspectsof one'spractice(Tripp, 1993). There are, of course, further sourcesfrom which ideas might derive. One is readingthe work of others,or hearinga paperat a conference,possibly linking up with what was alreadya latent idea, or alternativelysupplying a 'new' notion that an individual teacherfinds stimulating enoughto follow up. A paper on the nature of people's behaviour in 'natural' as opposedto pedagogicallyconstructedgroupings,for example,might trigger a fresh look at one'sclassroompractice in this area; or an article on 'display' questionsmight encouragea teacherto examinemore closely the ways in which he or sheposesquestions. Finally (for purposesof this section),and despitethe frequentexhortation (Allwright and Bailey, 1991, for instance)that it is preferableto start with the focus providedby somekind of question,the datacollectedfrom, say, diaries, or from observationof recordings,and then reflected upon, can suggesttopics for investigationfor which other researchtools are then invoked. Hopkins (1993) makes the point that researchmay start with 'closed'questionswhere a hypothesisor idea is alreadyquite well formulated, but also with 'open'questionswhich begin either with dataor with a broad area within which more specific issueswill be identified and then explored.

Researchin progress One view of a researchendeavourmight be that it progressessystematically somethinglike this: hypothesis/idea-+ choiceof methods-+ investigation-+ results Most researchers,however, whether they work within a normative or interpretive framework, are unlikely to be able to follow such a neat and logical set of stages:there will always be dead-ends,new data,datathat do not seemto 'fit', uncooperativesubjects/participantsand much - fortunately- that is unexpected. We have spent some time in this book so far setting out the qualitative researchparadigmthat, whateverthe specific orientation,regardscontext and socially constructedmeaningsas particularly significant. Taylor and Bogdan(1984: 5) make the following point: 'Qualitativeresearchis inductive. Researchersdevelop concepts, insights, and understandingfrom

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

GENERATING RESEARCH

81

patternsin the data, rather than collecting data to assesspreconceived models, hypotheses,or theories. They begin their studies with only vaguely formulated research questions' (italics added). This is echoed by Dillon (cited in Hopkins, 1993), who conceptualizes'problems'as either existent, emergent or potential. Education, like the human behaviour it deals with, is a necessarilyuncertainscience,so that emergentquestions will always need reformulation on the road to their increasingspecificity. For example,a case study of an individual learner started only with his teacher'sfrustration at his apparentinability/unwillingnessto learn. The teachergot interested,askedherselfwhy this 'non-learning'was happening, then collected different kinds of data that gradually began to shed light on the problem which eventually becamea reflection on evaluation procedures(McDonoughand McDonough,1993). Seligerand Shohamy(1989) offer a common-sense model incorporating four phasesof preparationfor researchin which systematicprogression,a variety of starting points, and the possibility of changing direction,are all allowed for. The phasesare: • • • •

formulating the generalquestion focusing the question decidingon an objective formulating the researchplan (or hypothesis).

In this section we have put forward some possible starting points for research.We next survey by way of illustration the kinds of content that might provide the substance.

Content:teachers'choices The purposeof this sectionis to set out someof the kinds of contentthat EFL teachersidentify as being worthy of investigativeattention.It would self-evidentlybe an absurdundertakingto attempt to give a comprehensive rundown of all possible topics and all known researchprojects by teachersgoing on in our field. So, short of an unattainableuniversality, we will mainly exemplify by using a specific circumstancein which teachers were askedto focus on themesof potentialinterest. Themes

In the context of a postgraduatecoursemodule on researchmethods,it is arranged for the participants- all practising languageteachersfrom a number of different countries and backgrounds- to observe ongoing English languageclassesin the same institution. It is not an exercisein

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

82

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

observationas such, except for the instruction to participants(who go to the classesin pairs) that they should regard it as a descriptiveand not an evaluativetask, and should find ways of noting down as many of the classroom events as possible. Both conceptuallyand in terms of design, the exercisehas obvious flaws: we are not concernedwith possible research outcomesnor with the meaning perspectivesof the observees,and the detailsof the methodologiesof observationare consciouslyfudged so as to provide an easily managedvehicle to trigger focal topics. One further instruction reads:'Generatingissuesfor researchinvolves noticing interesting events in the class. They may be interesting becauseyou cannot immediatelyexplain them, or becausethey illustrate familiar themesfrom your theoreticalknowledge,or becausethey do not ... , or becausepossible explanationsare unsatisfactoryin the contextyou haveobserved.' Here then are some samplesof what these participants have found worth noticing. • How to recognize/dealwith 'false friends' in a multilingual classroom? • What do studentslearn from listening to eachothers'role-plays? • What is the role and value of reformulatinglearnerresponses? • How do teacherschangethe way they use languagewith different addressees? in a lesson? • What are the sourcesof misunderstanding • Are the attention-seekers necessarilythe strongestlinguistically? • The variable accuracy of explanation when teachersdeal with vocabularyqueries'on the spot'. • What doessilencesignify? • What are the characteristicsof good monitoring? • Patternsof participation. • How to deal methodologicallywith latecomers. • Distinguishing and evaluating explanation versus discovery of meaning. • To what extent do students do pair work according to the teacher'sinstructions? • What do studentsget out of a lesson,and to what extentdoesthis matchwith teachers'perceptions? Comparablelists, though not drawn explicitly from observation,are to be found in Brindley (1990), Chamot (1995), Johnson(1992), Nunan (1989; 1992b)and Richardsand Lockhart (1994). It is noticeablethat the majority of themesthat they (or their informants) nominateare likewise formulated as direct questions,suchas 'What aspectsof learnerbehaviourdo I respond to?', 'Do I interact with somelearnersmore than with others?'(Nunan),or 'How are objectivesmodified in the light of ongoing assessment?', 'What

GENERATING RESEARCH

83

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

differencesare there betweenthe type of languageproduced by learners insideand outsidethe classroom?'(Brindley). Even from this rather rough-and-readylist it is clear that there are a numberof categoriesinto which teachers'choicesfall, including: • • • •

classmanagement learningstylesand strategies teacherbehaviour stylesof languagepresentation.

To thesemight be addedfurther categoriesand subcategoriesidentified by the authorsjust mentioned: • • • • • •

Testingand assessment Courseevaluation Planning andsettingobjectives Talk and interaction Needsassessment Leamerlanguage

as well as more 'macro' issuessuch as socioculturalconcernsand teacher training. The potentialfor topic selectionis obviously considerable.

SpecifYingfurther Thereare a numberof reasonswhy the kinds of topics just touchedon are unlikely to lead directly to a researchproject without further formulation. Someof thesereasons- resources,timescale,likely goals, patternsof participation of those involved and so on - were listed earlier in this chapter as important factors affecting both approachand execution. There are three more specific reasonsbriefly commentedon here: the first two relate loosely to what might be said to constitute'researchable'. First, some topics are clearly very wide-ranging,such as 'participation of learners','good monitoring' or 'what does silencesignify?'. As triggers they are probably interesting:as 'emergent'issuesthey need to be further broken down, focusedand modified, so that a tighter formulation becomes a more manageableundertaking. 'Participationof learners',for instance, may involve asking whether participation differs according to task type, nationality or proficiency, and possibly the interactionbetweenthem. This is related to the questionof the feasibility and penetrability of the topic itself: 'teachingand learninggrammar'or 'using the mother tongue',however central, are so large-scalethat it is difficult to seehow a way in could be found to investigatethem at all. Nunan makes the further point that topics need to be worth addressingin the first place, and is tongue-incheekdubiousas to whether'a study to determinethe numberof Spanish interpreters who wear designer jeans' (1992b: 213) would be. Similar

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

84

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

doubts- the 'so what?' problem- might be raisedover a researchexercise that collected numericaldata for its own sake,for examplethe nationality, age and sex of studentsreturning a. questionnaire.Hopkins (1993: 64), albeit from an action-orientedperspective,advises:'when choosinga topic or focus for classroomresearch,make certain ... that it is viable, discrete, intrinsically interesting,involves collaborationand is related in some way to whole schoolconcerns.' Second,any researcherwill need to make a realistic match between topic and method of investigation.The postgraduate-teacher participants who provided the authors'list of topics (see previoussection) were subsequentlyinvited to shortlist themesof their own choosingand sketchout a possibleresearchprocedure,coveringsuchquestionsas: • What datawould you want, and from what sources? • How would you collect it? • How would you analyseit? For the record, one group chose 'the role of feedbackand praise', 'how teachersadapttheir languageto different proficiency levels', and 'observable patterns of monitoring'. A range of methodology was proposed, including observation and recording of classes,interviews, coding and counting,and field notes. Finally, somepopularnominatedtopics haveoften beenthe subjectof a considerable bodyof previous research,for instanceteachertalking time (TTT), the treatmentof learners'errors,turn-taking,and many more. We are certainly not suggestinghere that the insights provided by a teacherresearchproject in suchan areawould not contribute'new knowledge':on the contrary, and for the momentas an aside,there is much to be said for the establishmentof collaborative researchcarried out by teachersand professionalresearchersin partnership.What this does suggest,however, is the needfor both the availability of, and critical opennessto, the existing literature,and it is to this point that we now turn.

Researchand research:existingwork This section is in no senseintended as a review of areas of potential teacher-generated researchlined up againstwhat hasalreadybeendoneby others.Our concernhereis to addressthe issueof the possiblerelationship betweenwhat a practising teachermight do within the main framework that we have dealt with so far - the context of their everydayprofessional work. The 'elsewhereness' of 'the big R' that we commentedon in Chapter 2 is indeedoften unattractiveto practisingteachers,partly it must be said becauseof a limited and limiting view of the natureof 'theory' and 'practice', and partly, to put it more positively, becausedifferent professionals (in our caseteachersand 'academic'researchers)may quite properly have

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

GENERATING RESEARCH

85

different agendas,expectations,proceduresand targets.(After all, if I buy a motor boat I can't complain that it doesn'thave sails.) It is nevertheless one of our contentionsin this book that it would be both unrealisticand shortsightedfor a teacherinvestigating a personally motivating topic to ignore the availability of existing work, even if that work were of background interest rather than direct application and even if - as is likely that work will be harnessedto servethe teacher'sown researchgoals. (The reciprocal, i.e. the researcher'sneed for a deeperunderstandingof practice, is itself far too often overlooked,and it is an important reason for teachersto producepowerful descriptionsof their own teachingcontexts for researchersto absorb. For researchersto give time to doing teaching will in certain circumstancesbe equally useful.) It is also presumptuousto supposethat teachershave no interestin backgroundreading,only being focused on local concerns.In this section, then, we commenton ways of using the work of others, whether professionalresearchersor teachers, first from a particular perspectivein teachertraining, then from the point of view of the experiencedteacherat work. Theory through training

It is common practice in many teachertraining/educationprogrammes (seeChapter2) that information is given, often in the form of lecturesand set readings,about the establishedresearchbase of the subject. In our field, of course,this would particularly be referred to as 'applied linguistics'. A typical programme,for example,would containsuch topics (to take a few at random)as CommunicativeCompetence,UniversalGrammar,the Monitor Model, Errors and Interlanguage,SchemaTheory, First and Second Language Acquisition, Discourse Analysis, Semantics and Vocabulary,and so on. In our view the appropriacyof this methodologyas one segmentof a training programmeneeds no defence if it helps to preventthe theory-practicedisjunction from being perpetuated.However, there are other ways of looking at this such that 'teachers'theory' and 'researchers'theory' are more explicitly linked. Marshall and Rossman (1989: 22) put forward this point of view: 'Personal,tacit theory and formal theory ... help to bring the question,the curious phenomenon,or the problematic issue into focus. The potential researchmovesfrom a troubling and/or intriguing real-world observation... to personaltheory ... to formal theory, conceptsand modelsfrom literature ... which frame a focusedresearchquestion.'This is by no meansa universal or necessaryprocedure,but it is one that hasa well-known parallel in the literature on teachertraining in EFL, and it is worth setting out in a little more detail. Ramani (1987) seeks an antidote to the 'theoretical-input model', and finds one in an eight-stageprocedurethat encouragestrainees

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

86

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

to conceptualizeand formalize issuesin their own practice. The starting point is the trainees''pre-theoreticalmap', tapped into by inviting a previewing discussionof personalcriteria for evaluatinga lessonwhich is then shownon video. The post-viewingdiscussionconcentratedon eliciting subjective responsesand then relating them to the establishedand agreedcriteria. For example, by the end of this process the par.ticipants had foregroundedas worth commentingon the ways in which teachersask questions, and were able to formulate their observationsquite sharply with the trainer as facilitator/ reformulator. The final stageconsistedof a handout from the trainer setting out the issuesas the traineeshad perceivedthem, and recommendingrelevant readings from the existing literature on the argumentthat these were now an organic part of the whole process.The whole procedureshows how traineescan begin with informal reactionsto data,and work towardsthe personal'discovery'of existing work. It also illustrateshow individual researchcan be relatedto a broaderknowledgebase.

Starting research- somebrief casestudies It will be obvious from the above discussionthat researchplans do not spring fully fledged into existence. Before every piece of completed researchthere has beena long and often depressingperiod of changeand development,often with discardeddata and thwartedattemptsat analysis. ImagineAnna Garcia, who wantedto investigatethe English readingskills of her Spanish-speaking studentsand had a hunch that there was a connection with levels of reading skill in their native language.She also saw that some of the comprehensionerrors were motivated by grammatical ignorance. Her first researchplan involved a contrastiveanalysis of the two languages,but then she realized that she was not going to be able to relate the resultsof the linguistic analysisto any readingability measures. At this point, she beganto read more widely in the professionalliterature on foreign languagereadingskills. This showed her some newer methods of describingreadingskills, and also that other people,in this casein three different countries, had investigated first and second languagereading ability. This reassuredher that the topic was manageable,but she had not suspectedthat other people'sthinking aboutit was quite so advanced,even to the extent of the existenceof a theory, the 'threshold'theory of transfer of first languagereading skills. She was at first dauntedby the international dimension to her original problem, believing that it took her away from the possibility of improving the performanceof her studentsin her own context. Then she realized that the work she had read was itself flawed and contradictory,so she decidedto test out a modified 'threshold' hypothesiswith her own students.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

GENERATING RESEARCH

87

Anna Garcia'sexperienceis interestingin the presentcontext because of the role of the codified experiencein the 'literature'in shapingher own researchplans,and helping her to substitutesomethingpracticaland manageablefor the generaland unworkableideasshe set out with. In her particular case,sheobtainedfunding to go to Britain for researchtraining. In another case,a teacherwanted to introduce a particular technique which he thought would be advantageous:peer conferencingto improve pupils' writing skills. He had noticed that the pupils did not respondvery effectively to their teacher'susual commentson their work and he wanted to evaluatethis technique.Unfortunately,it had not been tried in his own educationsystem,so he decidedto researchit in his own context to assess if it was worth recommendingmore widely. In his case,the literature on learning to write in a secondlanguagewas remotebecausemost of it concernedlearning situationsin countrieswhere the role of the foreign languagewas very different from that in his own African context. However,it did give him hints towards the design of the research,for examplewhat indicationsof improvementin the writing to look for, what comparisonsto set up, and how to combinea numericalapproachcountingaspectsof performancewith a qualitativeanalysisof the actual peerconferencesessions. A third teacher'sexperienceof starting researchwas different again. This teacher,working in a special kind of secondaryschool in a South American country, wanted to conduct an evaluation for the English for Specific Purposescomponentof the course.This componentused a great deal of materialoriginally developedfor the university systemin her country, in an internationally supportedproject. The university project had beenevaluated,with positive outcomes,but there were obvious questions about its extension 'downwards' in the secondaryschools. After much effort looking at ways of measuringachievement,resources,teachertraining, costsand implementation,this teacherrealizedthat sheneededsomething much more focused and in one way less ambitious for her own project. Shedecidedto focus on technicalreading,and to comparereading strategiesused by successfuland unsuccessfulstudents.For her method, shechoseto combinetwo existingapproachesshediscoveredin the literature: an 'introspective'approachcalled protocol analysis,and a quantitative questionnaireapproach.Thus in this case,an explicit wish to replicatean evaluationin a new context turned into an original thesiswhich eventually won her a doctorate. In eachof thesepersonalstories,there is a different outcomefrom the interactionbetweena teacher'soriginal thoughtsabout the researchtopic, and the work of othersthat they consulted.The first teacherexperienced somedegreeof shock that the questionshe had come up with in her own classeshad beenresearchedin other contextsat all; the literatureshe read showedthat she was in touch with a widely recognizedproblem, but also

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

88

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

that attemptingto do somethingabout it would requirea greaterdegreeof sophisticationin analysisand designthan sheoriginally expected.The second teacher found that the question he wanted to ask was somewhat devoid of interestingand relevantprehistoryin language-learning contexts similar to his own, and decidedto extrapolatemethodsand insights from other cultural and educational contexts in order to establish a viable design, and in order to argue for the innovation on the basis of plausible comparisons.The third teacherfound the literature in a particularareaof enquiry gave her the meansto developa 'do-able'piece of researchwhen she was beginningto despairthat her original idea was too unfocusedand difficult to carry out. These three personal histories were based on the experiencesof real people; in each case they were pursuing researchin order to obtain a higher degree.As pointedout by Schecterand Ramirez(1992: seeChapter 4 in this book), this aim introducesa parameterof explicit incorporationof the researcher'swork into the history of the subject which teacher researchersmay not feel bound by. However, the teacherresearchermay neverthelessexperiencethe samesenseof shock,surprise,lack of support, and stimulusfor developmentin interactingwith the relevantliterature. In Chapter6, somesourcesfor literaturesearchand ways of discoveringwhat hasbeendoneaboutparticulartopics are listed.

Conclusion This chapter has looked at where researchideas come from, how individuals begin to test their ideasand developtheir own researchprogrammes, and how those researchactivities relate to their normal professional lives. The chapter explored the variety of approachesthat are possible, from testing predictionsfrom establishedtheory in one'sown classroom, through the use of a teacher'sown experience,to inductive or data-driven researchin which explanationsare soughtfor problemsthat are allowed to emerge,as it were, from data collected.The point was made that just as different groupshave different ideasof theory, so they have different attitudesto the desirabledegreeof precisionin the initial researchquestions. An overview of likely themes for researchby English languageteachers was offered, basedon actual observationof classrooms.Thesethemesare refined in discussionas the central issue which underliesthe question is identified. Often, several apparently different questionsmask the same issue, sometimesquestions need to shift as the most important issues emerge.The strategy of training teachersto discover theoreticalcontroversy through classroomobservation- to develop an awareness f theory through educatedobservation- was discussedas a way of generatingnew researchquestions.Finally, somepersonalhistoriesof teachersdeveloping researchprogrammeswere given which illustrate the variety of effectsthat

GENERATING RESEARCH

89

interaction with the history of their chosentopics can have - sometimes depressing,but mostly formative and encouraging.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Discussionnotes 1. Reflectingdirectly on your own teachingsituationand experience, what events,or incidents, can you identify that have given you a senseof 'unease'?These may be to do with individual learners, your own planning, the whole class, colleagues,materials and many more. When you have identified someincidents,chooseone and try askinga seriesof questionsaboutit, for example: • • • • •

What exactly happened? Why do I rememberit? Why do I feel it was important? What did I learn from it? Did anythingchangeas a consequence?

It is useful to work on thesequestionswith a colleague. 2. In your reading of the published professional literature, what works (books,articles) have beenparticularly memorableor influential for you?

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

This page intentionally left blank

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Part 2: topics and methods

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

This page intentionally left blank

6

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Definitions and overview

This chapteris intended as a bridge betweenthe two main parts of this book. It begins with a brief commentaryon the relationshipof research principle to researchdesign,including someobservationson the terminology involved, and then introducesthe methodsand techniquesthat will be discussedin subsequentchapters.The final sectionof the chaptersuggests in practical terms some of the channelsopen to teacherresearcherswho wish to 'go public' and disseminatetheir own work to a wider audience, and read up on publishedwork.

Principles, methods, techniques A considerableproportion of this book so far has beenspenton a discussion of the major paradigms underpinning educational research,their derivations (particularly in the social and natural sciences),and some of the key issuesthat arise as a consequence - how to approachfundamental questionsof validity, reliability, generalizabilityand so on. On the face of it, it would seemintuitively sensibleto assumethat a particularorientation to researchwould have straightforwardimplications for design: a preference for a qualitative paradigm would, one might expect, lead to datacollection techniquesat the naturalisticend of the spectrumand to interpretive rather than quantitativeanalysis.Thus, for instance,an interestin the 'local meanings'of participants in the classroomcontext (following Brown's, 1984,tabulationof the featuresof naturalisticversusexperimental researchtouched on in Chapter3) might involve selectivenon-evaluative observationtriangulatedby teacherand learnerdiaries; in-depth interviews and case studies might be preferred to questionnairesadministeredto large populations.There is obviously much logic in a researchprocedure that matchesin kind, as it were, a researchethosand the dependentpracticalities of its design. However,a caution is also in order, and it is useful to makea parallel with languageteaching here. Most languageteacherswill be familiar with the debatesurroundingcommunicativemethodologyand its claimed advantages over the teachingof a languageas a formal linguistic systemthrough gram-

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

94

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

matical patterns (Widdowson's,1978, well-known use/usagedichotomy). We know that a communicatively-based coursebookor lessondoesnot preclude systematicattention to grammaticalexponentsdirectly or indirectly related to the context of use. Likewise, a glanceat a structurally organized textbook will almost certainly reveal opportunitiesfor learnersto practise the structurein a 'communicative'situation. In other words, the choiceof a principle does not automaticallymean that certain methodsare ruled in or out, and so it is with researchdesign. Erickson (quoted in Denzin and Lincoln, 1994: 213) points out that researchis qualitative (or quantitative) becauseof its 'substantivefocus and intent', not just becausequalitative techniquesare being used.Taking the exampleof narrative description,he arguesthat a qualitative researcher,on the one hand, and someonewith a positivist, behaviourist orientation, on the other, jf confronted with the 'same'phenomenon,'would write substantivelydiffering accountsof what had happened,choosingdifferent kinds of verbs, nouns,adverbsand adjectives to characterizethe actions that were described',reflecting varying degreesof researchinterestin the meaningsof the 'actors'in the situation. We shall try to show as we work our way through the availablemethodology of research(for instance,coding systemsin observationanalysis,or content analysis of verbal data) that quantitative techniquesare often used in an interpretiveas well as a positivist framework without 'damaging'the paradigm choiceitsel( Marshall and Rossman's(1989: 42) common-sense advice is worth quotinghere: 'Researchers should designthe studyaccordingto the researchquestionsthey seekto answer.' There is one further factor that requires comment: again there are rough parallelswith languageteaching.Many readerswill be familiar with the three-stagemodel of course design associatedwith the work of Richardsand Rodgers(1986, though they basedtheir discussionon earlier work by Anthony). The model is intendedto show how 'theory' becomes 'practice' through a processof increasingspecificity and concretization. The levelsareApproach,Designand Procedure.'Approach'refersto general views - theories- of languageand learning on which the next stage is based.'Design' is the stageat which axiomsat the first level are converted into syllabus and materials construction, and 'Procedure'refers to the practical techniquesused in the classroom.Research,too, is often written about in a comparableway, a sequentialdistinction being made between interdependentlevels: Principle -+ Method (or methodology)-+ Technique Taylor and Bogdanequatethe middle level not so much with practicalities as with the logic of researchdesign: 'When stripped to their essentials, debatesover methodology are debatesover assumptionsand purposes, over theory and perspective' (1984: I). Walker (1985) refers to

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

DEFINITIONS AND OVERVIEW

95

'methodology'in a similar way, and further distinguishesit from 'method'. Observation,or interviews, for example,would be methodsin this definition, leaving the term 'techniques'to denotethe details - in the caseof observation,checklists,coding systems,descriptivestrategies,and so on. As with much terminology, usageis often rather loose and interchangeable: we note the conceptshere becausethey indicate a useful general framework for conceptualizingthe whole researchprocess.We shall ourselvesapproximateWalker'smethod/technique distinction.

Introductionto methodsand techniques The literature on researchmethodsis extensiveand growing. In the field of English as a foreign languagealone,publishedwork dealswith language learning, often but not always from the perspectiveof applied linguistic researchersas much as teachers,and more specifically with what is potentially feasible for language teachers. (See, for example, Allwright and Bailey, 1991; Brown, 1988; Nunan, 1992b; Seligerand Shohamy,1989). It is useful to set thesealongsidethe more explicitly teacher-orientedwork from mainstreameducation (for example Hopkins, 1993; Nixon, 1981; Walker, 1985), as well as standardreferenceworks such as Cohen and Manion (1989). A typical profile of methodscoveragewould include at leastthe following, in no particularorder: • • • • • • • •

questionnaires interviews observation(direct and recorded) field notes, diaries,documents casestudy (not strictly a 'method',as we shall see) experiments think-aloud numericalanalysis.

As we have often commented,choice of method and associatedtechniques will dependon many factors, often of a very practical nature (see Chapter5 for a more detaileddiscussion).When we take into accountthat each of these methodsis a 'composite',with considerablepossibility for internal variation, then the permutationsare many. For example, interviews can be structured,semistructuredor unstructuredand open-ended, and can take place betweena number of different participants(learners, teachers,outsiders,and so on); observationcan be evaluative,descriptive, or 'coded' in some way, as well as being carried out by different people with different purposes; and all methods can interact. Walker (1985: Chapter3) is one of severalwriters to set out the advantagesand disadvantages to teacherresearchersin particular of some of the main methods, using datain the form of feedbackfrom teachersparticipatingin the Ford

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

96

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

TeachingProject(FTP). Just from his discussionwe can invoke suchconsiderationsas the participantsinvolved, their numberand role; the amount of time needed;relevanceand applicability; follow-up possibilities; bias and objectivity; equipmentneeded;researchtraining needed;confidentiality; degreeof specificity; administrativelogistics; cost and scale.As Walker comments:'The accumulationof this kind of practicalknowledgeprovides an important backgroundto decisionsthat need to be made concerning the selectionof researchmethodswhen faced by particularresearchproblems' (1985: 60). In the chaptersthat follow, we look at the principal researchmethods and their associatedtechniquesfrom the teachers'viewpoint, but at the sametime discussingthe varying extent to which they have beenimportant in the researchmethodologyof languageteachingand learning more generally.There are, in other words, emphaseson the frequencyof use of methods depending on the type of research and the training of the researcher(observationand transcription,for example,being more widely usedacrossthe whole spectrumthan diary studies).We begin in fact with observation in the languageclassroom,becauseit is particularly widespreadand pervasiveas a method. The subsequentchapters,to use Van Lier's schematization(1988, and quoted in Chapter3 of this book), sit at various points within his two main parametersof interventionand selectivity and a fourfold division of methodsinto thosethat mainly involve one of Measuring, Controlling, Watching and Doing/Asking. Since we hold the view that all methods in principle should be available to teachers,the chaptersare not to be seenas occurring in any systematicsequence.We thereforecover: • • • • • • •

observation diary studies numericalmethods experimentalmethods questionnairesand interviews think-aloudand other introspectivetechniques casestudies

and concludewith a chapterexplicitly discussingsomeof the possibilities for combining methods.First, however,as a window on methods,a short digressionon ways of informing othersaboutone'sresearch.

Out there: discoveringother people'swork and telling them aboutone'sown The point hasbeenmadeseveraltimes, and particularly in Chapter4, that researchneedspublication and disseminationboth in its formative stage, learning from what other people have done, and as an outcome, telling

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

DEFINITIONS AND OVERVIEW

97

otherswhat the researcherhas found. There are many different channels and modesof disseminationand discussionof researchavailable.Here follows a necessarilyincompletelist. First of all, at the most local level, there are someschoolsand colleges that run in-houseconferences,under the headingof 'staff development', at which thereis the opportunity to invite outsidespeakers,and to present an individual's work - either in progressor as a final report - to colleagues.This opportunity doesnot exist in all teachingcontexts.In many casesthereare, however,in the local area,organizationsor loosegroupings of teacherswho meet more or less occasionallyto exchangeviews. Such teachergroups usually flourish longer if they are in contactwith and can draw on the resourcesof regional or national organizations.Based in Britain, there is one large organization relevant to EFL teachers: the InternationalAssociation of Teachersof English as a Foreign Language (IATEFL). This organizationruns a large annualconference,publishesa newsletter,and offers a numberof useful services.In particular, its members can join a number of Special Interest Groups or SIGs, which have specialmeetingsat national and local level, and publish their own newsletters which are often hungry for copy from practisingteachersengagedin researchand development.The SIGs most obviously promoting research are 'Research','TeacherDevelopment'and 'Testing';but researchfeatures regularly in all the othersas well. Outsidethe EFL field there are similar organizations for other languages,and for those interested in action researchthere is CARN, the ClassroomAction ResearchNetwork. The United States' organizationTESOL (Teaching English to Speakersof Other Languages)plays a local, national and internationalrole, and many national organizationsare themselvesaffiliates of the 'mother' TESOL. TESOL also runs a massiveannual Convention, many "Special Interest Groups,and promotesand aids more local events.The British Association for Applied Linguistics (BAAL) also runs an annualconferenceand promotes researchon teachinglanguages.The addressesfor theseorganizations are in the Appendix. There are many publications which carry reports of researchin languagelearning, teachingand education.The major internationaljournals in the field are: Applied Linguistics, English for Specific Purposes, InternationalJournal ofApplied Linguistics, International ReviewofApplied Linguistics, Language Awareness, Language Learning, RELC Journal, Studiesin SecondLanguageAcquisition, SecondLanguageResearch;System and TESOL Quarterly. There are a number of journals which specialize rathermore in teachingEnglish as a foreign languageand carry articlesby teachers:ELT Journal, Modern English Teacher, Practical English Teacher, Prospect,and The TeacherTrainer. In the UK thereis also a monthly newsof conferencesare often paper,the EL Gazette, in which announcements

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

98

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

made,and reportsof interestingeventsin the field are carried, as well as sometopic-basedbibliographicarticles.The addresses of the publishersof all theseperiodicalsare in the Appendix. For those with accessto full library facilities, there are several useful sourcesof summariesof particular topics. An annual publication is the Reviewof ELT (Macmillan). Thereare Dictionary of Linguisticsand Phonetics (Crystal, 1991), A Linguistics Encyclopaedia(Malmkjaer, 1993) and Handbookof Applied Linguistics (H. and K. Johnson,in press,Blackwell). A recent publication of relevanceis The Cambridge Encyclopaediaof the English Language(Crystal, 1995). Most useful are the abstractingjournals, such as Linguistics and Language Behavior Abstracts (published in the USA) and LanguageTeaching(publishedin the UK). Thesepublish oneparagraphreviews of newly publishedpapersin other journals, with full references,groupedundera whole rangeof key words. Both featurea regular spot for a commissioned'Stateof the Art' article with an explicitly comprehensivebibliographic brief, so a glanceat recenteditions will produce detailedreviews of work in a multitude of topic areas.Full details of theseare given in the Appendix. In the UK, the Centre for Information on LanguageTeaching(CILT) maintainsa register of ongoing language-relatedresearchprojects, has a dedicated library and provides a bibliographic search service. In the United States,the ERIC (EducationalResearchInformation Center)also provides arangeof bibliographic servicesfrom short pre-prepareddocuments to full individually commissionedsearches.Both of these can be accessedfrom abroad,thoughsomeof the servicesattractmodestcharges. Bibliographic searchesthrough hard copy and electronic databasescan also be commissionedthrough a local university library, again there are usually costsinvolved. Theselibraries also havethe computerizeddatabase of 'Dissertation Abstracts' where unpublished graduate work can be accessed,usually going back to about 1985. Rapidly increasingin importanceand scopeis the amountof information availableelectronically.At the time of writing, thereis an explosionof interestand developmentavailableto thosewho haveaccessto the Internet via e-mail or the World Wide Web. However, it can be quite frustrating to find that after somehours of 'surfing the net' either somecrucial connection breaks down or there is no useful information at the end anyway. Enthusiastsclaim that both of theseproblemswill reduceas the technology improves and the quality of the information placed on the network gets better. Already, usersof e-mail can sign up to a free electronicmail service, 'TESL-L', and a paper-freerefereed journal, 'TESL-EJ' dedicated to English language teaching worldwide. Seedhouse'sarticle in IATEFL ResearchSIG newsletter(1995: 6) explains lucidly how to do this. The instructions aregiven in the Appendix. Seedhousepoints out that such resourcesgive accessto huge numbersof people,and in the case

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

DEFINITIONS AND OVERVIEW

99

of the journal promisequick returns- your work, if accepted,can be disseminatedwithin months rather than the typical two years of a conventional journal. Such resourcescan be used to find out if usersare working on similar problems, to get others to fill out your questionnaires,to get volunteersfor your researchprojects,and possibly to obtain other people's opinionsof your ideas. Other material availableon the Internet include even a Virtual Library, run by Birkbeck College of the University of London, and Home pages for organizationssuch as the British Council and IATEFL. At the time of writing, there is not one for TESOL but it is being developed.Library catalogues can also be directly accessed.The IATEFL newsletters have already carried useful articles by Seedhouse (1995) and Blackie (1995) guiding potential users. The American ERIC organization can also be accessedbye-mail. All these electronic addressescan be found in the Appendix. Information exchangeis becomingeasierand easier:naturally, no one will confuse quantity for quality of information, and it must be most users' experienceof electronicinformation that there is a lot of drossto be waded through before anything interesting crops up. However, as we saw in Chapter 5, there are many ways in which other people'swork and other interestedpeoplecan be useful to an enquiringteacher,and the information explosion has to be welcomed: it means wider and easier accessto other people'sthinking, their problems,and their contributions,for everybody.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

This page intentionally left blank

7

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Observing languageclassrooms

Introduction 'Observation',as it stands,is a monolithic label, a broadand evenamorphous umbrella term subsumingmany and varied purposesand interpretations. Hopkins (1993), for example, gives a positive sense of this breadth of application by describingit as a 'pivotal activity' with a crucial role to play in classroomresearch,teachers'personal-professionalgrowth, and school developmentas a whole. Observationwith its associatedtechniquesis also often embeddedin a larger-scaleresearchplan as one methodamongothers, when perhapsa variety of datasourcesis appropriate.Here, however,we take it as by and large discrete,returningto it in the final chapterin the discussion of ways in which methodsmight be usedin conjunctionwith eachother. Thereare a numberof categoriesinto which the world of classroomobservation is divided up in schemesproposedin the literature.To take just three: Wallace (1991) writes about system-based/ ethnographic/adhoc procedures; Hopkins (1993) has a more detailed breakdowninto systematic/structured / focused/openobservation;and Seligerand Shohamy(1989) encapsulatethe samespectrumin a binary structured/opendistinction. As we shall see,all such frameworksbega numberof questions,to do with the observer'sdegree of participation,the issueof 'objectivity', inductive (heuristic) and deductive (analytic) parameters,amongothers.The presentchaptermakesa straightforward distinction between'systematic'and 'naturalistic'observation.We begin by attemptingto delineatethe term as a way of focusingon its specific role in research.There follows a discussionof the kinds of frameworks teachers might develop for structuringobservations,and a commenton the developresearch.After a bridging secment of coding systemsfor classroom-based tion on the interfacebetweenquantitativeand qualitativeapproaches,we turn to the more 'open' methodologyof descriptiveobservation.Throughoutwe take observationto be an intentional activity rather than a more reactive 'noticing' of classroomevents.

Observation:usesand perspectives Considerationof context,as discussedin Chapter3, implies that any claim of typicality has to be taken with caution. Nevertheless,to provide a simple

102

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

window on the classroom,we might agreethat the following description of a segmentof a lessonrepresentsa reasonablyrealistic if unexceptional scenariofor an EFL classin someparts of the world. (In the absenceof other available meansof presentation,a verbal description will have to suffice.) The teacher(T) has a lower-intermediateclass of 16 students(Ss) aged 18-23 from a numberof countries.This is the first lessonof the day. He chatsa little aboutthe previousevening'sactivities, then nominatesa few Ss in turn to say what they did. He occasionally correctspast tenseforms and pronunciation.He then asks them to open their coursebooksat the new Unit, which beginswith a narrative abouta family's experienceson their holidays.This is readaloud by Ss who self-select,and is followed by vocabularychecks:someSs write down the new words, sometimeschecking in their bilingual pocketdictionaries.The text is followed by questions,which Tasks Ss to work on in pairs: somedo so, otherswork alone. Answersare then checkedin plenary.The coursebook has somegrammarexplanationsand exerciseswhich the classworks through with T. For the final activity Ss listen to a cassettewith two peopletalking in a question-and-answerformat about an accident one of them had witnessed.Ss fill in details in a table as they listen. With a few minutes to go before the end of the lesson,T does a quick verbal test of irregular verb forms. This is not in itself particularly illuminating, becausethereis no specification as to observernor as to the purposeof the observation,so without this information thereare many possibleinterpretationsof what is going on. As Lincoln and Guba (1985) point out - and here we anticipate'naturalistic' argumentsaboutperspectiveto be addressedbelow - reality is constructed and interpreteddifferently by different individuals, whetherthe entity is (their examples)communism,good manners,God, Harvard University, the Middle Ages, urban blight, social science, trickle-down economics, Watergate... and, we might add, a segmentof an EFL lesson. There are, then, at least three key parametersthat need to be clarified. Theseare (1) the observer,(2) the goals of the observation,and (3) the procedures;in other words who?, why? and how?, which will in turn be determinedby the natureof the settingin which the observationis taking place (see Chapter1 for a fuller discussionof thesefactors; a useful discussionof the interactionof various featuresas applied to observationcan also be found in Walker, 1985: 120ft). Theseare just a few of the possibilities undertheseheadings,and the permutationsare many:

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

OBSERVING LANGUAGE CLASSROOMS

103

Observer Coursedirector Head/seniorteacher Colleague/peer Trainee/juniorteacher New teacher Researcherfrom outsidethe institution Externalassessor Goals Placementof students Evaluationof efficacy of materials Apprenticeshipfor noviceltraineestaff ('craft' learning) Staff appraisal:formative/developmental summative,e.g. managementdecisionson employment Quality assuranceand control Personaldevelopment Improvementin methodology Procedures Checklistsand written criteria: observer-constructed; agreedin advance with thoseobserved;preparedby externalagency Audio- and video-recordings Notesand logs Participation/ non-participation With/without feedback. Just this brief and selective overview should be enough to show that 'observation'is multifaceted,and without clear cut-offs betweenapplications. Most combinationscould in fact be invoked as a way of looking at the EFL lesson describedabove. We now need to considermore specifically the usesof observationas a researchtool. Observationfor research Even with this apparentnarrowing-downof the role of observation,there remain many and varied possibilitiesfor both approachand execution:the researchapplicationsof observationtherefore need to be situated in the context of the lines of argumentin the presentbook. Let us remind ourselvesthat we are concernedprimarily with the kinds of investigationsthat teacherscan undertakeas an integral part of their professionallives, but also with the uptake,whetherreceptivelyor by application,of the work of others(researchers,other teachers),and with what can be done when there

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

104

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

is an opportunity to take 'time out' to concentratefully on a researchproject. Although teachersworking directly from their day-to-day teadiing experiencemay well feel more in tune with a naturalistic, interpretive many researchparadigm,the spectrumtaken as a whole still encompasses choices.The discussionin this chapteris restrictedto observationcarried out in the classroomitself, thoughof coursethereis no reasonin principle why it should not be able to handlelooking at staffrooms, shadowingindividuals at work and so on, as indeed is often the casewith, for example, participantobservationstudies. A well-known volume in the educationalliterature on observationis entitled Looking in Classrooms(Good and Brophy, 1978). The overall tone of the book is prescriptivein that a major goal is to increaseteacherawarenessand effectiveness:nevertheless,the title is memorableand the preposition significant. Everyday languageallows a number of possibilities: looking in, looking at, looking jor, looking with, looking into ... , each reflecting a somewhat different orientation, as the work of Stenhouse (1975; 1985, in the Rudduck and Hopkins collection) reminds us. Researchersand policy-makerscan look at teachers,who themselvescan look at their learnersor their colleagues;teacherscan look with eachother, or a researcherwith a teacher;any of thesepeoplecan look for something specific once the issue for investigation has been formulated; looking in can be doneby someonestandingoutside,but also by an insider in the situation. Most of the approachesand attitudesdiscussedin Part 1 of the book shouldbe recognizablehere. The main interrelatedareas thatthe remainderof this chapterwill be concernedwith under the two headingsflagged above (and they will of courserecur in subsequentchapters)are simply: • researchin the interestsof increasingknowledgeand understanding of a phenomenon,or of creating 'new' knowledge, whether that knowledgeaspiresto be idiographic and particular, or transferableand general • researchundertakenexplicitly to bring about change,innovation and action, often at institutional level; this is an importantpoint at which researchand evaluationclearly interact development. • researchfor personal-professional Within thesebroad categories,we shall needto invoke a numberof further issues,including: • the observer'srole on a spectrumof objectivity-subjectivity,and the whole questionof bias; degreesof participation/non-participation • the role of thoseobserved

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

OBSERVING LANGUAGE CLASSROOMS

105

• the natureof the data derived from observationand the timing of its collection; the use of data to trigger researchquestionsversus its role in answeringquestionsalreadyformulated • different ways of counting, systematizingand analysingdata, and some of the problems associatedboth with selectivity and data reduction,and the converseof 'rich' data • the nature of the setting itself, including the uses in researchof the 'normal' classroomand opposedto the specially constructed or eventhe experimental.

Systematizingobservation Preliminaries

In this section we shall look at ways of observingclassroomswhich were very popular at one time and try to evaluate them as tools for teacher research.We refer to the useof preplannedobservationalcategories,sometimes coding schemes,or 'systematicobservationschedules'.To use them, the observer has to learn how to recognize instancesof particular categoriesof classroombehaviourand note them down as they occur, either live or from some kind of recording. Clearly, observingone'sown classis difficult, but not impossible,to reconcilewith teachingit at the sametime: the Open University's course on Curriculum in Action (1981) contains a video of several teachers doing just that. However, in practice, selfobservation by teachers is likely to involve video- or audio-recording for later analysis, but collaborativeobservation- going into a colleague's class- is in someways easierto conductin real time. Why systematize?

Earlier in the chapteran important distinction was drawn betweenobservation as an 'intentional activity' and 'reactive noticing' of classroom events.The second is a necessarypart of generatingresearchquestions (and of teaching anyway!); but intentional observationimplies planning and the use of some previously establishedcategories.The importanceof systematicitywas neatly put by Fanselow(1977) who deviseda particular coding schemecalled FOCUS, when he pointed out that we all seedifferent featuresin the samesceneand thereforegive different accounts:in his view serious investigation needed agreementbetween observerson the basic categoriesinvolved. So systematicityimplies prior decisionsabout what to record. Furthermore,it implies agreementor at least prior decisions about methods:when to record, for example: • on a regular time base(every 3 seconds,30 seconds,60 seconds, and so on)

106

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

• using a recognizableboundarymarker - like 'OK' - to signal a segmentof a lessondevotedto one topic • to categorizeone turn of speakingat a time and so on; and what kind of temporalinformation to record, such as frequenciesor sequences. Coding systemsmay be publishedand in widespreaduse, such as the venerable'Flanders Interaction Analysis Categories'(FIAC) (Flanders, 1970) or the 'CommunicativeOrientationof LanguageTeaching'(COLT) scheme(Frohlich et al., 1985), or they may be set up for particular purposesby individuals. The useof systematiccoding schedulesis thereforevery much an elaborated checklist approach.Checklistsare used to ensurethat relevant steps are noticed and remembered,like a week'sshoppinglist or a pilot's landing checks; but since they thereby systematically reducethe raw data, interestingeventsin that data that are not included on the checklist will not be noted. The advantagesof a coding systemapproachmay be easily summarized as follows: 1. They are easier to use than on-the-spot description or paper recording. 2. The use of agreedand even published category systemsenables comparisonswith other studiesand thereforegeneralizability. 3. They can be tailor-madefor a particularproblem. 4. Analysis can be by simple frequency counting and numerical analysis. 5. Patterns of interaction and developmentthrough time can be established. 6. Personalpatternsof a particular teacheror learnercan be established. 7. It is possible,indeedvital, to train observersin the coding system. A measureof reliability of the schemeis given by the relative ease different observershave in agreeing that actual behavioursare instancesof one or other category. The disadvantages of the useof codingsystemsin generalare as follows. 1. They involve editing, i.e. reduction, of the data, in however systematica way. Dependingon the unit of analysis,for instance,taking a time baseor a segmentof a lesson,what is going on outside that unit doesnot count directly in the analysis.Thus, noting what is happeningevery 30 secondsessentiallydiscardswhat has happenedin the interveningtime. 2. The categoriesare preselected,and may not be the relevantones for the lesson being observed. Put more strongly, observation

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

OBSERVING LANGUAGE CLASSROOMS

107

schemesneed to be validated againstsome independentcriterion of relevance,of relation to achievement,or whatever.How otherwise do we know what the analysiscategoriesare that are significant for learning?This has rarely been done even with category systemsthat are in widespreaduse. 3. Very often, observinga class using a categoricalchecklist cannot take into account relevant but unobserved shared knowledge among the actual participants.Thus going into other teachers' classesand using a systematicobservationschedulecan be blind to featuresof the history of that class and its institutional context which are presupposedby the participants themselves. This objection may not hold for observingone's own classes,but the use of 'insider knowledge' in interpretationhas to be consistent and rule-governed.

Kinds ofanalysis In looking for behavioursto slot into categories,the researcheris ultimately interested in what the observablebehaviour tells him about something deeper: the aspect of learning or teaching under study. Categoriesvary thereforein the degreeof precisionthey exhibit. A frequentdivision hereis betweenhigh-inferencecategoriesand low-inferencecategories.Using highinferencecategories,an observerhas to makea considerableeffort of judgement to decide whether a range of behavioursconstitutesa category. A long-standing example of this is the first category of the Flanders InteractionAnalysis Categories:'Acceptingthe feeling tone of the students'. A teachermight do this either verbally or non-verbally;by a major or minor changefrom his/her plannedlesson;by cedingturns of speechto a student expressingthat mood; or by explicitly referring to the 'feeling tone' or ambient atmosphere.The observerhas to make a number of judgementswhich concernthe intentionand probableoutcomeof the behaviour;the teacher's accuracyof perceptionof what the studentsare feeling; and a host of other context-boundvariablesto assignthe eventto a category.By contrasta lowinferencecategory like 'hands-up'is relatively easy to code, but the later interpretation might prove more difficult. In particular, even with lowinferencecategories,the observerprobably has to decide whether they are looking at somethingdeliberate,possibly symbolic and meaningful,pr accidental,unintentionaland without symbolicsignificance.

Publishedor inventedcategories Thereis no needto rely on publishedliteraturefor categories:a researcher can develop his or her own for some particular researchpurpose. For

108

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

example,a teacherresearchermight be interestedin the possibleeffectsof different kinds of strategiesfor giving oral feedbackor interveningabout an error in spokenperformance.Rather than taking a global categorylike 'praise'or 'criticism', a project might choosemore detailed categories,as follows: • • • • • • • • • • • • •

identifying the location of an error by prompt or non-word- oh! giving a location by repetition giving a prompt askinga grammaticalquestion askinga meaning-relatedquestion pretendingnot to understandthe utterancecontainingthe error askingfor a translation giving a paraphrase askingfor a paraphrase interveningimmediatelythe error occurred interveningafter the end of the student'sturn to speak interveningat a topic shift boundarymarker('OK') non-verbalreactionsuchas gesture

and so on. In other contexts,a teachermight want to developa set of observational categoriesfor eventslike 'handlingquestionsfrom the class'or 'initiating a group work activity' or 'setting up a readingcomprehensiontask' or 'giving grammaticalexplanations':in principle the possibilitiesare endless. Experienceof using the in verited-categorysystem would show how workable it was, and how consistentthe researcherwas in applying the codings;trying to teachsomebodyelseto useit would help establishwhich categorieswere more reliable. The usual testsfor reliability of coding systems is the percentageof agreementbetweenindependentcoders(teachability), and the percentageof agreementbetweencodings at different times by the samejudge (consistency).Notice in this mocked-upsuggestion there is still a considerableamountof judgementinvolved: even these are relatively high-inferencecategories.Another difficulty the researcher would have with thesecategoriesis time: since each of thesemay occupy one or even part of a turn of speech,recognizingand noting them may be too difficult for real-time work, and resort would have to be made to a recordingand transcript. A further set of decisionswhich would bear on the choice of categories would be their suitability for different kinds of participant groups. For example,the teacherresearcherin our examplesmight be interestedin: • the teacherin a lockstepteacher-frontedclass • teacher-student interaction • a selectedgroup of studentswithin a co-actingclass

OBSERVING LANGUAGE CLASSROOMS

109

• a small working group of studentsin a group-workclass • the teacherworking in turn with several groups working on the sameor different materials.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Analysisoj codings Codedtranscriptsof classescan be analysedin severaluseful ways. 1. By counting frequency, in which the relative frequency of each kind of behaviourcan be set out in tabular form and conclusions drawn from the overall picture. This might indicate, perhapsto the surpriseof the classor the teacher,particularly favouredkinds of behaviour or mannerismsassociatedwith a particular class, studentsor teacher;or might demonstratethe proportion of time spent on certain kinds of linguistic exchangesat the expenseof others, or the proportion of time spent in the target languageas opposedto a commonlanguage.Before drawing many conclusions it would be possibleto check how any markedly over- or underused behaviourscomparedwith what chance would predict by using an appropriatestatisticaltest (for the significanceof propor.:.. tions, seeChapter9 on 'chi-square'). 2. By finding patterns, in the sense that there may be certain of behaviourthat are characteristicof the lesson:e.g. sequences

soliciting answers-+ nominating a pupil checkingwith anotherpupil -+ praise.

-+

receiving an answer -+

These patterns may be individual to a particular teacher or general depending on what is under observation. As the teacher researcher inspectsthe datafor suchpatternsthere will be two concernsin particular: first, to use any 'insider knowledge' carefully and defensibly especially when observing one's own behaviour or one's own class; second, to rememberthat one swallow does not make a summer,and not to believe thereis a patternuntil thereare severalexamplesof it.

Relationshipswith other infiJrmation Instancesof particular codings, frequencies andpatterns,are difficult to interpret only by referenceto themselves.Lessons,in particular, usually have a plannedstructurewhich at the least can be comparedto the actual sequenceof the lesson to observeand explain differences. Lessonsand their participantsalso have a temporal and a structural place in a programmeor curriculum which itself may be significant in explaining what is or is not observed.Furthermore.lessonsand the people co-producing

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

110

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

them - the teachersand the students- have both individual and corporate histories,and any outsideobserverusing a coding schememay miss very important events by not knowing those histories. An inside, participant teacherresearcherwill probably know some of it, but it is a weaknessof the method that there is no systematicway of including this kind of information. Lastly, behaviourswhich can be codedmay be expressedin all sorts of different ways, both linguistic and non-linguistic, so somerecord of the type of expressionmight be kept, especiallyin the caseof a language class. An important issue in many early language-classroom observations was simply how much, and when, and for what purpose,either teacheror studentsusedtheir mothertongueor the foreign language. Live observationor recording? Any form of observationis going to introducesomedistortion from normality. It is incumbenton the teacherresearcherto satisfy himself or herself that any distortion is evaluatedor taken accountof. At its worst, the distortion of the datacould invalidate the observationsas a true picture of what normally occurs; at best, the amount and kind of distortion can be known and the conclusionsmodified accordingly. Sitting at the back of someoneelse'sclassroom- or of one'sown while the studentsare working on their own - conductingan observationand marking down instancesof particulareventsin codecan distract the attentionof studentsand teacher alike, howeverused they may be to the presenceof anotherteacherin the class. Introducing a tape-recorderinto the class to listen to later can also distract attention, and the recording can also distort: microphones are much more limited devicesthan the human ear, voices on the recording are much less easyto identify, and distant participantsmay not be picked up. However, such a recording can be used as a check on a coding conducted live, without necessarilywriting a transcript. Writing up a transcript of an audio-recordingmakesit availablefor careful analysis,freeing the researcherfrom the constraintsof real time, but it is itself a very timeconsumingexercise,and naturally loses more information which may be necessaryto decide between coding categories,such as intonation and other paralinguisticcues and non-verbal activities, unless the recording can be supplementedby field notes. Video has naturally been used a great deal in observationalwork, but any fixed cameraonly seeswhat it is pointing at and a movable camera requires an operator who is necessarilyediting the record as he or she chooseswhat to record. Even the comparativelysimple problemof recording interaction betweenstudentsand teacherin a teacher-frontedclassroom is quite difficult to accomplish.Cameraequipmentin a class may also distract and possibly invite strangebehaviour;leaving it unusedfor a

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

OBSERVING LANGUAGE CLASSROOMS

111

time to let the class get used to it has been advocated.Siting audio- or video-recordingequipmentout of sight of the participants,as with the use of one-wayvision screensset in classroomwalls, can be criticized reasonably on ethical grounds.All the participantscan be consideredto have a right of consentto the research,so concealedobservationmay be interpreted as bypassingthat right, unless the researcherrevealsits existence and location - which rather negatesthe point of concealmentin the first place.

Alternativesto codingschemes Using a coding schemeor a systematicobservationschedule,whether in real time or on a recordingor transcript,is a way of separatingout significant events from the mass of data, spotting patterns,and arriving at an interpretationof the structure of what has been observed.It necessarily involves reducingthe data by somepreconceivedplan, hencethe checklist of categories.The validity of the resulting interpretationhas a lot to do with the validity of the coding categories,establishedon other data in other classrooms,and it follows that it has lessto do with the uniqueness of the particular observation. Nunan (1989: 16) demonstrates,using a transcript of a lesson excerpt, how three different ways of representing and analysing the data give quite different insights into the progressof that lesson,which is in fact the openingsequenceof greetingand warmup. He usesan adaptationof the FlandersFIAC, anothersystemfor categorizing verbal behaviourdevelopedby Bowers for English for Specific Purposesclasses;and a narrative description. They also differ in 'userfriendliness'.The reduction of the data for the FIAC abstractsand highlights certain processescharacteristicof the opening of a lesson; the labelled transcript using the Bowers descriptorspresentsa ready-made interpretation,but the narrative caststhe data with all its human signals, messagesand nuancesas the author(perhapsonly the author) saw it. This version is, Nunan tells us, perceivedas the most user-friendlyby teachers. It describesthe teachernot just as coming in and sitting down but 'slumping in his chair'; studenttalk is describedas 'muttering','echoing','interjecting" 'suggesting'and 'venturing': a rich vocabulary for a rich set of nuancesof interpretation.How accuratesuch a narrativecan be is open to evaluation. Recordingsandfield notes

Earlier, we discussedthe use of recordingsfor the purposeof running a check on coding accuracy.An extensionof the practiceof recording,bearing in mind of coursethe caveatsabout recordingand dataselectivity, is to

112

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

record a lessonor part of a lessonand then transcribeit using field notes takenat the time of the recording.Putting the two sourcesof datatogether representsa move away from reductionist observationmethods towards somethingone might usefully call elaborativedescription.

Text analysis Allwright, in his influential paper 'Turns, topics and tasks' (1980), presentsa method of analysis that invoked both a (tentative) coding framework at each of three levels and a line-by-line commentaryon the coded transcript of the lesson. The commentaryacted as an immediate interpretation of the flow of negotiationsthat constitutedthe managementof the interactionby all the participantsin the lesson.This paperis also discussedin Allwright's book (1988), which describesthe developmentsin observationalresearchin languageclassroomsover the previousdecadeor so. An interestingtheme drawn out by Allwright is the extent to which observationcan in principle reveal a complete picture of the classroom dynamic.

Stimulatedrecall Another techniquewhich researchershave used with teachersbut teachers could use with their colleaguesand studentsis known as 'stimulatedrecall'. Here, a recordingis made of the lessonor class to be observed,but interpretationof the observationis made with the collaborationof the participating teacheror students,the researcherstoppingor freezingthe recording at points of interestand asking the participant to think back to the lesson and reveal what had beengoing on, their perceptionsat that point,and their decision processes.This technique has been used in published work by Nunan,working on lessonrecordingswith a teacher,but also for exampleby Poulisse et al. (1987) working on native-speaker Ilearner dialogues with learnersof English. Taking the example narrative of a class given at the beginningof this chapter,this techniqueallows a researcherto ask why, for example,the teachernominatedstudentsto say what they had donethe previous eveningbut allowed the studentsto self-selectwho was going to read in the class,on the face of it a curiousdiscrepancythe motivationof which is unrecoverablefrom the narrative.

The TlJork ofPeck An exampleof language-teaching researchon quite a large scale,which does not use a systematicobservationschedulebut at the same time does not espousea full naturalisticenquiry philosophy,is given by the work of Peck

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

OBSERVING LANGUAGE CLASSROOMS

113

(1988). Peck'smethodologywas developedto enable him to comparethe realities of languageclassroomsdevoted to several different foreign languagesin five countries(Britain, France,Germany,Norway and the USA; English, Frenchand German)and yet it is readily adaptablefor useby individual teacherresearchersinvestigatinglessonsin their own schools. Peck intended to provide a workable schemefor describing languageteaching: 'categoriesfor the interpretationof current practice,and [to] makeavailable a numberof useful comparisonsbetweenlessonsand teachers'(1988: 4). Tellingly, his introduction containsthe statement:'I could not observe, or describe,languageteachingfrom the point of view of an outsider, like for instancea Martian observinga cricket match; I tried to bring to observation and explanation all the knowledge and experiencewhich I have acquired myself over a number of years as a languageteacher,textbook author,and teachertrainer' (1988: 4). Peck observesand describesa fair numberof lessonsand teachersusing his 'insider' approach,without placing strict and difficult restrictions on himself as to his choice of unit of analysisor his observationaltechniques. His objects of study were normal categoriesof teaching,like presentinga text, new grammar,free expression,narrative, pronunciation,oral practice, teachersupport. His unit of analysis was the 'part' of a lesson. His categoriesfor observationare often expressedas comparisonsor tradeoffs,plotting them on coordinatesto show how a particularlessonor teacherrated: Rangeof linguistic structures

versus

Paceof oral practice

Technicalvariety

versus

Intensity of language

Teacherintervention

versus

Studentparticipation

Fluency

versus

Preparationfor oral work

In many instanceshe plots a categoryover the time courseof a lesson(see Fig. 7.1). Very difficult

Moderately difficult

Easy

Very easy 1st quarter

2nd quarter

3rd quarter

Fig. 7.1 Lessondifficulty (Peck, 1988: 69, Fig. 1.20, adapted)

4th quarter

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

114

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

A very useful feature of his book is that many of the suggestionsfor analysisare deliberatelyfree of copyright so that any teachercan use them in examiningtheir own or a colleague'steachingstyle. Discussionof theseapproachesto interpretationof the processof languageclassesusing semicontrolledbut rich dataservesas a bridge between the quasi-scientificreductionist paradigmsof the systematicobservation schedulesand full naturalisticobservation,to which attentionnow turns.

Naturalisticobservation Somedefinitionsand implications At the simplestand most generallevel, 'naturalistic'is to be taken in contrast to contrived, manipulatedor experimental:it refers, in other words, to a concern with the understandingof natural settingsand the representation of the meaningsof the actors within that setting. In the caseof EFL, a naturalistic observationwould need as its databasethe everyday lessonwith its usual participantsin real time, rather than, say, a classconstructedto tryout a particularpedagogicmethod,or a 'one-off' with volunteersnot normally together,or a teachernot known to the students,any of which - whatevertheir other researchmerits - would constituteinherent distortionsof naturally occurring phenomena.(In passingit is worth noting Nunan's(1992b) somewhatsurprisingstatistic that of 50 well-known classroomresearchstudiesonly 15 were carried out in 'genuine'language classrooms.)Lincoln and Guba (1985) further remind us that naturalistic enquiry is itself defined at the paradigm and not necessarily at the methodologicallevel, though clearly the tendencyof such researchwould be weightedtowardsa preferencefor qualitativemethod.Thereare implications, therefore, not just for goal and setting, but also for the relative roles of participantsand for the nature,collection and analysisof data. In principle, then, in terms of the logic of the paradigm,there are at least two important corollaries. First of all, the notion of context becomes crucial, becauseit sites the phenomenonof study in spaceand time, and can thereforetap into the constantlyfluctuating interactionsand relationship patternsin a group of peopleworking together.(This is the holistic, 'emic' world of Van Lier, 1988, where the setting speaksfor itsel() As we saw in the short example in Chapter 3, a 'classroomculture' is usually built up over quite a long period of time, so even a short interactionmay contain some complex verbal and behavioural presuppositions.The teacherin the lessonsegmentin the first sectionof this chaptermay have 'insider knowledge' reasonsfor choosing not to nominate certain individuals to answer(which an outside observermight well misinterpret);a referenceby a learnerto 'that prepositionproblem' could be retrospective by severalweeks. Walker and Adelman's(1976) exampleof the class who

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

OBSERVING LANGUAGE CLASSROOMS

115

laughedwhen a new teachertalked about 'strawberries'is much cited. A researcherusing a prespecifiedcoding schemecould adequatelydescribe neither the students' unexpectedlaughter responseexcept weakly as a 'reaction' nor the teacher'sbafflement at that reaction. (The students were, it transpired,amusedbecausea previous teacherhad often talked about bad strawberrieswhen returning weak marked work.) All teachers would be able to think of a host of such examples,only apparentlycommonplace. Second,the naturalisticobserverwill take as given the existenceof the multiple perspectivesof thoseobserved,and will positively seek to interpret rather than avoid them. The observeras a 'human instrument'also has a perspective,of course.As Schwartzand Ogilvy (quoted in Lincoln and Guba, 1985: 55) put it: 'Perspectiveconnotesa view at a distancefrom a particularfocus. Wherewe look from affectswhat we see... A whole picture is an image created from multiple perspectives.'As a direct consequence,in this view it is acceptedthat an understandingof bias and subjectivity is the norm, and notions of objectivity and neutrality become spurious: the famous 'observer'sparadox' is elevatedto the status of a betweensocial settings principle. This view of the mutual interdependence and the accounts given of them is often referred to as 'reflexivity' (Hammersleyand Atkinson, 1983), where everyoneinvolved is part of the construction.A brief, observableillustration of perspective: The teacherhandsout a summaryin table form of the verb systemof English, including a list of irregular verbs, the intention being to review ground already covered. Learners react in different ways. Someask questions: What'san imperative? What does (X) mean? Is (X) the sameas (Y)? Why can't I usethe sameending? How do you spell ... ?

Others are silent: some are confused(and mayor may not be preparedto say so); someknow it already(and may be bored,or pleased to go through it). There is clearly a rangeof different learning paths,and the lessonhasdifferent 'meanings'for the individuals participatingin it. Dealing with data

In our discussionof 'coding' models, we have already commentedon the research place that systematicobservationcan play in a teacher-designed project. On the whole, however, to capturethe natural setting, the tech-

116

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

niquesusedwill be at the open, less structuredend of the methodological spectrum.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Describingsettings Description is a key technique in ethnographyand anthropology: the researcherusually becomesimmersed in the setting as a 'participant observer', a term which captures the role duality. Ethnographersand anthropologistsare of courseprofessionalresearcherswho will spendlong periods of time intensively studying their chosenenvironment.Furthermore, their studies will be 'macro-ethnographic',concernedwith such large-scalesettingsas whole schools,or even whole cultures.As we have noted, practising teachers(unless taking time out for research)are very unlikely to become full-scale ethnographers,and a micro-ethnographic approachthat focuseson the more manageableentity of a class,or relationships and events within that, is more realistic. Observerroles, too, then becomemore variable.Hereare just two possiblescales: • active participant • privileged observer • limited observer

• • • •

completeparticipant participantas observer observeras participant completeobserver

(Wolcott, cited in Johnson,1992)

(HammersleyandAtkinson, 1983)

Teachershave a built-in advantage:when observingeither their own class or that of a colleaguethey are always in a sense 'participant' or at least 'privileged' observersbecausethey are an organic part of the institutional environment. Whatever the detailed arrangements,the principal data-gathering instrumentfor naturalisticobservationwill be description,with the aim of producing'thick' rather than 'thin' (i.e. restricted)data (Geertz, 1973), in other words a multilayeredpackagethat is as comprehensiveas possiblein terms of individuals, interactionsand behaviours,as well as all facets of the physical setting. Description of a real-time processis an intensive undertaking.It may be a full text, attainablein the observationof others; or it may take the form of field notes written up later, as when observing one'sown classroomin action.

Anafysingdata Thereare two main approachesto the analysisof dataderivedfrom descriptive observationwhich essentiallycomeat it from different directions. First, we need to start with anotherexample.For simplicity let us suppose this is the written outcome from an observationof another lesson

OBSERVING LANGUAGE CLASSROOMS

117

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

with the sameclass as in the first section above - details of the physical settingwill haveto be assumed. The teacherarrives early and changesthe seatingarrangementsso that the classcan work togetherin groupsof four. Studentsgradually drift in, sometalking to their compatriotsin their L 1. When most Ss are there, T returns corrected homework, which not everyone has done,and puts recurringerrorson the board.Thereis somerestlessness until he calls for attention and explains the task. It is a wordbuilding exercisewhere one form of a word is given and Ss have to write down the adjective, noun, verb and so on. It is competitivein time betweengroups.After someinitial confusionSs get on with the task, thosewith the sameL 1 referring to their dictionariesand each other to clarify meanings.T does a mix of desk work and walking round to listen, spending differential amounts of time with each group and requesting'English only'. One group finishes and 'wins', though their answersmay not be correct. Others seem to be some way behind,poring silently over their dictionaries. This is clearly a very broad description:spaceconstraints preclude greater amountsof detail. From the observer'spoint of view, issuesof potential interest are as yet unformulated,and it is the data itself that will suggest productive lines of enquiry. Even our rather minimal data would show a few themeshidden in theseroutine classroomevents,suchas: • • • • • •

the frequencyof certainbehaviours LI/L2 useincluding patternsof dictionary reference individual stylesand preferencesin working patterns T's methodologicalpreferencesand their execution variabletime-on-task contextualizedldecontextualizedlanguagepractice

and so on. Essentiallythe data examinedand re-examineduntil recurrent patternsbegin to be foregrounded.(More detailed data would show linguistic patterning as well, perhaps students' questioning style, or a teacher'sclassroommanagementphrases.)Walker discussespattern analysis as a formal technique,basedon audio recording and transcription, then 'looking for patterns... regularitiesof behaviour,forms of interaction which occur over and over again' (1985: 140). Once foregrounded,such patternsare then open to structuredscrutiny. It should be noted that this kind of observationis in tune with ·the 'emergentdesign' dictum of naturalistic research,where anyone theme can be successivelyreformulated and focused,often by using different kinds of researchtools onceobservation has elicited the area of interest.This kind of descriptiveobservation in a qualitative tradition is discussedby Day (1990), who also provides a lengthy descriptivesegmentof a class.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

118

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

We have had to limit ourselveshere to the one-ofT kind of observation, but regular observationsover a longer period of time would arguably be more useful, partly becausethey would allow the researcherto build a picture of the developingclassroomculture and its insider-knowledgeand presuppositions,and partly becausechangeover time - in learningpatterns,or interactions,for example- can emergemore clearly. The 'thick descriptions' from this kind of observationare, in Lincoln and Guba's(1985) terms (a) inductive, (b) generativeand (c) constructive.For all data analysisof this kind, Miles and Huberman's(1994) sourcebookis a key referencework. They collate in fascinatingdetail the methodsavailableto datefor looking at qualitativedatastartingfrom the threeinterrelatedkey principles: • data reduction:coding, clustering,writing 'stories' • datadisplay: networks,structuredsummaries,synopses • conclusion drawing and verification: triangulation, comparison/ contrast. Second,descriptiveobservationand a data-first approachdo not necessarily go hand in hand. Descriptioncan also be selectiveand focusedafter a particular issue has been chosenfor further investigation. For example,a teacher researchermay have already decided that a particular learner exhibits problematiccharacteristicsin uptakeof instruction and materials, apparentlyfollowing his own agenda(looking up words in the dictionary rather than following the flow of the lesson, and so on): he or she takes descriptive notes in class over time and charts the key patterns,perhaps going on later to look at successratesin testsas comparedto others.Again, anotherteachermight invite a colleagueto examinedescriptivelyhow he or she sets up pair work, becausehe or she is uneasy about its efficacy. Mention must be made here of the techniquesof 'clinical observation'(or supervision, in teacher training: see Chapter 2), a cyclical processwith three recurrentphasesof planning, the observation,and a feedbackmeeting (which could lead to further observation).(For more detailed discussion, seeHopkins, 1993; Gaiesand Bowers, 1990; Fitzpatrick, 1995.)This procedurebelongsfirmly in the tradition of researchfor action and change. Seligerand Shohamy(1989) point out that descriptionstarting with an already narrowed-downscopeand preconceivedtopic sharessomeof the featuresof the deductiveapproach,even if the 'hypothesis'to be 'tested'is a more diffuse sense that something is worth looking at. Descriptive researchmay also have features of quantitative analysis, for example by providing measuresof frequency.

Conclusion This chapterhassurveyedsystematic,focusedand naturalistic,rich methods of classroomobservationfrom the point of view of a teacher who

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

OBSERVING LANGUAGE CLASSROOMS

119

wishesto use someof them. It beganby consideringhow a 'window on the classroom'can be openedand how observationcan be used for research purposes.The use of systematicobservationschedules,both general and published, and invented for a specific local purposewas then discussed. Ways of analysingcoding schemesthat have been used in the past in languageclassroomswere then explored.A bridging sectionthen gavea number of approacheswhich have been successfullyused already, and which incorporatemore data than was availablefrom the more mechanicalcoding scheme approach.These methods combined different kinds of data in order to achievea fuller interpretationand a more completedescriptionof the language teaching classroom. The final major section described in some depth the naturalistic and ethnographicapproachesto interpreting classroomevents.It showed how the 'rich' descriptiontakes into account the context, the physical constraints,the individuals, and the interaction, to producean interpretation.Analysis of such data concentrateson establishing patternsof various kinds and on scrutinizing them for evidence either for a general description or for the solution of focused detailed questions,which may have been flagged beforehand,or which emergeas crucial issuesfrom the inspectionof the data.

Discussionnotes 1. Are there any aspectsof the life of your own classroomthat you think could be usefully investigated by devising a systematic observation scheme? For example, are there features that you think it would be helpful to quantify? How would you interpret the outcomein the contextof your classesas a whole? 2. If possible, find an opportunity to sit in on a class together with another observer.Write as full a description as you can of what went on. Look at your description to identify the most frequent/significantevents,and then compareyour analysis with the other observer.Did you 'see'the classin the sameway? (If you do not have accessto a class, use video data, or even Day's, 1990, transcript.) 3. Do you think that being a participant observerin your own class would help you to observeaccuratelyor make it more difficult to perceivewhat is going on? Can 'insider knowledge'aid interpretation and can observationchallengepre-existingassumptions?

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

This page intentionally left blank

8

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Diaries and diary studies

Introduction Diary-writing is a pervasivenarrativeform. It has,of course,always played an important role in many people'sprivate reflections,and has had many famous exponents(Anne Frank, Samuel Pepys, Malinowski, Katherine Mansfield and many more). It is widely used in specializedand nonspecializedways in such diverse fields as history, literature, anthropology, autobiography, sociology, health studies, clinical psychology and psychotherapy.In educationand in English languageteaching,the diary has becomeincreasingly significantboth as a reflective genre in itself, and as one of a battery of interpretive micro-ethnographicresearchtechniques. In Van Lier's (1988) terms, most diary-writing clearly belongsin the categoriesof leastcontrol and selectionof data(seeChapter3). After running through somestandarddefinitions of diaries, we go on to discussthe natureof the data that they generate.The next sectionlooks at ways of analysing that data and at the place of analysis in the whole researchprocess:it also invokes the important distinction betweendiaries and diary studies.We then set out and exemplify the main usesof diaries in researchon languageteachingand learning,including a short sectionon the 'reflective' ethos in teachertraining and its researchunderpinnings. The final discussionsectionis concernedwith both the benefitsand problematicaspectsof diary-keepingas a researchmethod.

Somedefinitions Most peoplewho have kept a personaldiary would recognizethe following way of looking at it: 'It is such a rough, impulsive study of life itself, it will prove perhaps... that life is not one thing but changesconstantly, that characterhas not one face but a thousand,changingas thesepageschange' (Anais Nin, quoted in The Guardian, 4th January,1996, p. 6). Indeed,the themeof changeover time and the senseof writing about a processis one that resonatesdirectly with the useof diariesin educationalresearch.This is echoedin Elliott's prescription that a diary should contain 'anecdotes; ... accountsof conversations... ; introspectiveaccountsof one's feelings,

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

122

RF.'iEARc/! I\IETHODS FOR ENGLISH LANGUAGE TEACHERS

attitudes, motives, understandingsin relation to things, events, circumstances'and should be kept on a continuousbasis(1991: 77). Bailey's wellknown definition is more formalized: 'A diary [ study] is a first-person account of a language learning or teaching experience, documented through regular, candid entries in a personaljournal [and then analysed for recurring patternsor salient events], (1990: 215; we commenton the significanceof the quotesin squarebracketsa little later in this chapter).A diary, then, is personal,with oneselfas addressee,long-term,and may also be relatively unboundedin the kinds of facetsit records,at least within the broad areawith which it is concerned.At the sametime, a diary is not only a re-creationof immediateexperiencebut is also a written record: the act of writing itself is a way of structuring, formulating and reacting to that experience,which is then availablefor reflection and analysis. The diary as an open-endednarrativegenre,which is the principal type that we shall be discussingin this chapterbecauseof its relative frequency, is, however, not the only possibleformat. First of all, a numberof writers (for exampleHolly, 1984; Hopkins, 1993) make a distinction betweenlogs, journals and diaries. Holly regards(a) logs as a record of factual information, (b) diaries as the kind of subjective text describedin the previous paragraph,and (c) journals as an amalgam of the two, containing both 'subjective' and 'objective' data. In practice the distinction is somewhat blurred: a conventionalship's logbook, for instance,will often contain a narrativesectionas well as essentialdetailsof coursesteered,distancerun and weatherconditions. In this chapter, we shall for conveniencesimply refer to 'diaries' and shall assumethat any kind of information - factual, feelings, attitudes, reactions- may in principle be included. Second,a somewhatdifferent perspectiveis offered in many areasof social research, where it is common for diaries to be commissionedfor researchpurposes, as opposedto the non-reactiveformat kept of one's own accord. In such cases,although the diary remainsa personalaccount,the domain is quite tightly specifiedby the researcherand the record kept is often behavioural, orientedto facts about activities rather than being evaluativeand affective. Examplesof this kind of researchapplication include the investigationof patternsof consumerexpenditure,how people spendtheir time or travel, diet and nutrition, alcohol and drug consumption(see Corti, 1993, for an overview). Clearly, as we shall see,there are implications here for the ways in which data are analysed.This kind of precodeddiary is to date less common in reports of diary-writing in language teaching and learning contexts,though thereare someinstancesof its use.

Diary data One of the problems in reproducing diary material as raw data is the virtual impossibility of providing long enough samples.Diaries are best

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

DIARIES AND DIARY STUDIES

123

\Hitten over an extendedperiod, and 'snapshot'extracts cannot capture changesover time which can often be very marked, so anyone segment belongs organically in a broader temporal and contextual picture. .\ievertheless,we begin with two short diary extractswhich must suffice to give at least a stylistic flavour of a narrative diary entry. (There is more detailed discussionof thematic content in the next main section of this chapter.)The first extract is from a teacher'sdiary, the secondby an elementary learner of modern Greek. Both diaries were kept by one of the presentauthors.A further problem in publishing diary data is of course the issue of confidentiality and ethics, and no material should be reproduced without the writer's permission,and indeed approval of the whole text in which it appears,if necessarywith due regard to anonymity. That said, some publisheddata is generally available (see, for example,Nunan, 1992b, and the individual sourcescited in the later review of teacherand learnerdiaries).

Extract A: Teacher'sdiary Am looking forward to having more students next term, particularly in this group which seems to operate so much from shifting balances of personalities and others' reactions to them. Like introducing new ingredients and seeing what happens ... Worried today about not 'teaching' enough, at best giving them opportunities to explore in English. Not that the class went badly, but I felt I did rather little of the work. Maybe I should volunteer to teach grammar or something next term ... I wonder why X often doesn't understand the simplest classroom instructions and exercises, and why he refers to himself in the 3rd person? He seems to want to 'explore' a lot in English, but reverts constantly to the electronic dictionary ...

Extract B: Learner'sdiary Chatting before the beginning of the class, some people said they felt they weren't making any progress. Personally I feel on a bit of an upward swing at the moment, though it will no doubt change. The language feels like a constant discovery process as things gradually fit together - am more confident about talking, and I catch myself at odd moments trying to work out what I might want to say in Greek, though I still have to wo.rk it out in English first. I'm pleased that things that seemed to 'crowd in' too much at the beginning of the course are now becoming more routine, though of course there are new 'blocks' to fill their place. This time it's the future of the passive verbs, which I've decided not to learn till later ...

Let us first review the kind of data that in principle becomesavailable from a diary.

124

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

The nature oj data There are at least four featuresthat can be identified as characteristicof diary-generatedtext, apart from the fact that it is rathercasualand unpolished. We refer particularly to the more prevalent open-endednarrative type, though we also commenton more prespecifiedkinds of data. First of all, the textual material (like ethnographic-styleinterviews or descriptive observation,for example) is very 'rich' both quantitatively thereis a greatdeal of it - and qualitatively, in that more than one themeis addressed.This is evident even in theseshort extracts:the teacher'sdiary talks aboutan individual, the whole group, her own methodology;the language learner talks about learning curves, other people's patterns of learning, cognitive overload, specific strategies.Second,the diary is selfevidently subjective and introspective where the perspectiveof the 'I' dominates,and this includes filtering the assumedand reportedattitudes of others. However, the nature and focus of this introspectionmay well differ dependingon whether the self is the sole intendedaddresseeas in the most free styles of diary-keeping,or alternatively a researcher(who might be a teacheror colleague) who has provided some guidelines for writing with a specific line of investigationin mind. Open-endeddiaries too can subsequentlybe madeavailableto othersfor analysis. Third, diariesare necessarilywritten retrospectively,even if the timelag between theevent and the recording is very short - just after a lesson, sameday. As with any kind of autobiographicalmemory, there is likely to be some decay in accuracyover time, at least as far as factual logging of activities is concerned.However, few people would claim that a diary is a 'true record' (a notion which our earlier discussionof objectivity and bias has anyway called into question), rather that, in Elliott's words, it can 'convey a feeling of what it was like to be there participatingin it' (1991: 77). Finally, the minimal data in the aboveextractsis still able to demonstratethat diaries can record what happened,what the writer felt aboutit, what might or should have happened,what could change,opinions, anticipation and immediatereactions,as well as a more reflective tone. These characteristicsof the raw data haveimplicationsfor the proceduresthat an analystcan useto deal with it.

Analysingdiary data Particularlywith the more open-endedstylesof diary writing, thereis obviously a problem of seeingthe wood for the trees.So 'thick' is the data that its density doesnot allow immediatelytransparentresults. However, as we saw in Part 1, many researchersseethis as positive: Saville-Troike, writing about the ethnographyof communication,commentsthat 'one kills something thin only to discoverthat it is fat' (1989: 129), a backhandedcritique

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

DIARIES AND DIARY STUDIES

125

of overly normativeanalysis.Themesmust be 'teasedout' from the massof data. In termsof researchmethodology,diary material of this kind fits naturally into the classicdata-first procedures.Scrutiny of the text may lead to a self-containedanalysis- of behaviour,interaction,learningprocesses,and so on - as a diary study in its own right, or alternatively may be, in Nunan's(l992b) terms, a 'ground-clearing'preliminary, generatingtopics worth pursuing with other researchtools both qualitative and numerical. To take just one possibleexample,studyingone teacher'sdiary could lead to a set of interviews with that teacher,to observationof his or her classes, to the distribution of a larger-scalequestionnaire,or to the collection of comparativediary data. Whateverthe objective, the approachto analysis, as Seligerand Shohamy(1989) remind us, is essentiallyheuristic and one of discovery,not deductiveand hypothesis-driven. The most commonly reported procedurefor analysis is a very simple one, consisting of reading and re-reading the text to allow significant themesto becomegradually foregrounded.The reader(of one'sown diary, or thoseof others)may notice, say, that certain usesof languagerecur, perhapsstyle features,semanticgroupingof lexis, tenseusage,and-soon. Or a particular focus may emerge- on individuals, or aspectsof methodology, or preoccupationswith learning specific aspectsof a language.Allwright and Bailey (1991: 193) have proposedthat three key features need to be takeninto account: 1. frequencyof mention 2. distribution of mention (across writers, when several diaries are

beingexamined) 3. saliency: the strength of the expression with which a topic

IS

recorded. Mention of frequencyand distribution, as Allwright and Bailey point out, implies that content analysisis likely to be quantitative: they even suggest this as a possibility for 'salient' items as long as analystscan agree on a 'strength scale' for languageused. Computer packagesfor the analysisof qualitative data including diary text are also now available (Corti, 1993; Weitzman and Miles, 1995). It is important to remember,however, that a counting approachwill be necessarybut not always sufficient, becausean interpretive analysiswill need to take full accountof the whole context in which the diary-writing took place- featuresof the setting,assumptionsand intentions,the specific classroomculture. It is also importantnot to start the analysis too soon, becauseearly identification of themes may lead to an involuntary precodingthat shapessubsequententriesin a restrictive way. The abovecommentsnotwithstanding,we noted earlier that diaries as a more closed genre are common in some fields of research.With some degreeof pre-specificationa diary can becomeat least an interim research

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

126

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

outcome as much as a trigger, a kind of 'soft' version of an inductive approachwhere analytic categoriesprecededata. As we shall see, in language teaching and learning this is not particularly unusual in teacher training, or where diaries are collected from learners in classrooms, becausetrainersand andteachersare of coursein a position to be directive if they so choose.More generally,Richardsand Lockharthave suggesteda large number of 'reflection questions'that any teachercould addressin a diary format. The list is too long to reproducein full here, but includes questionsabout what happenedduring a lesson,looking at teachingstyle, planning, actual events, what studentsdid, and 'questionsto ask yourself as a languageteacher'(1994: 16-17).

Diaries and diary studies

A major function that a diary fulfils is to offer what Holly (1984) simply calls 'private space',a forum for reflection. This may be extendedinto collaborative journals, shared with colleagues, or into dialogue journals (Peytonand Stanton, 1991), where teachersand learnersrespondin writing to each other through the diary medium. Thesecan all playa central role in many aspectsof professional development.It is, however, only when they have been processedanalytically that they start to become instrumentsfor research.This point has been made particularly strongly by Bailey (1983; 1990). Her argum'ent,briefly, is that, while a diary is private, a diary study meansgoing public, contributing to a growing body of 'different insights into the largely unobservableprocessesof second language learning and teaching' (Bailey and Ochsner, 1983: 191). The addresseeis then not only the self but a wider professional audience, whether reached through conferences, journals, newsletters or local teachers'groups.The goal of the researchmay be an enhancementof our knowledge in a particular area, or an action researchone of changeand innovation. Bailey (for example, 1990: 219) diagrammatizesthe production of a diary study into five steps: 1. 2. 3. 4. 5.

accountof diarist's professionalbackground confidentialdiary revision of entriesfor public version analysis interpretationand discussion.

The unpolisheddiary is thus embeddedin a broadermore formal and analytic text, and researchbecomesa public activity within a critical professional community.

DIARIES AND DIARY STUDIES

127

We now turn to an examinationof ways in which diaries have beenused in languageteachingand learning research,with examplesfrom a number of publishedstudies.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Diaries in languagelearningand languageteaching Along with the growth of interest in naturalistic modesof enquiry, so the use of diaries to investigatea variety of aspectsof teachingand learning has likewise burgeoned,with a concomitantmore widespreadavailability of reporteddiary studies.In this section we look first at different types of learner diaries, then at diaries written by teachersin the context of their regular teaching. We conclude with a comment on the use of diaries in teachertraining. We are principally concernedhere with diary s.tudies,or at least with diaries that havebeenusedas a platform for research.

Languagelearner diaries Thereare two ratherdistinct usesof diarieswritten by learnersof a language.

Pedagogicuse In many teachingprogrammes,it is now quite common practice for class teachersto ask their studentsto keep a journal. Occasionallythis is unfortunatelyacourserequirementwhich may meet with someresistancefrom individuals in the 'captive audience'environment: data from such conscripts is anyway likely to be unhelpful becauseit will probably be skewed merely towardsexpressingtheir unwillingnessto keep a diary at all. Where participationis voluntary, the usual procedureis as follows: • The diary is confidentialbetweenteacherand individual learner. • It is usually written in the target languageexcept at elementary level. It is interestingin passingto speculatethat, while Ll data will probably be more sophisticatedin expressingfeelings and opinions about the language-learningexperience,it will of course not provide any material to investigateactual L2 languagedevelopment. • It is not normally used as a vehicle for error correction,its overt purposebeingexpressiveratherthan formal. • Any subsequentwrite-up as a diary study requirespermissionand a readingof the researchreport by writers of any diary cited substantially.

128

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

It is characteristicof pedagogicdiaries that studentsare given some guidelinesbefore they start writing. These may be in the form of quite detailedpoints to address,for instance(McDonoughand Shaw, 1993): • • • • • •

lessonsfollowed what you found most/leastuseful and enjoyable what you found easiest/hardest reactionsto your teacher reactionsto other learners commentson specific lessons

or they may be designedto constitutea loose framework aroundwhich the writer has more space to choose topics (Parkinson and HowellRichardson,1989): • • • •

in-classactivities out-of-classactivities my problems what I havelearnt.

This kind of pre-categorizationhas the advantageof giving learners(who are rarely specialistsin language-learningtheory) a kind of 'metalanguage' with which to talk about their experience. Parkinson and HowellRichardsonreport two different diary studies,both intendedexplicitly as researchinstrumentsab initio. One study analysedinformativity, use of English outsideclass, and anxiety levels; the other also looked at out-ofclassuse,and addeda categoryof learningstrategies. Clearly the initial specification will determine to some degree the themesthat emergefrom thesekinds of diaries. Furthermore,the process and outcome of analysis will dependon whether the teacherresearcher choosesto look at a broad spectrumof the whole class,or particular individuals: an in-depth examinationof learning styles and strategiescan be more fruitfully carried out by selectingindividuals, whereasthe diaries of the larger group will more readily show patternsof reactionto the lessons and the teacher. A further dimension is whether the investigation is intendedas a form of action research,carriedout in order to makechanges to aspectsof one'spractice.Finally, even though error correctionis rarely the goal of a learnerdiary, there may well be an interestingspin-off for an individual's languagedevelopmentthrough the processof regular diary entries.(See,for example,Jones',1994, experienceof writing his diary in Hungarian, or the learner reported in McDonough and McDonough, 1993, who moved from simple sentencereporting of daily activities to more complex exploration of his whole languagelearning experienceand the contextin which it was taking place.)

DIARIES AND DIARY STUDIES

129

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Expertdiaries By 'expert' here we refer to languagelearning diaries kept by language specialistswho may be teachers,applied linguists, or second language acquisitionresearchers,and all of whom are used to dealingwith language as a central part of their professionallives. Their diaries, even if not consciously crafted artifacts, are typically written from their inception with the intention of completinga diary study of their own language-learning processes.Quite an early form of this kind of self-reportcan be found in Pickett (1978) who, using a detailed set of prompts, invited successful adult languagelearnersto write to him about their learningexperiencein a diary-letter format: the data, which was principally concernedwith learning processes,was clearly (very) retrospective.In most cases,however,the 'expert'writer is his/herown initial addressee. There are a number of published examplesof language-learningdiary studiesof this kind. The most quoted is Bailey's (1983) work on competitivenessand anxiety as evidencedin the journal she kept while learning French.Other well-known studiesare thoseof Schumannand Schumann (1977), who reportedon learningFarsi and Arabic; and Schmidtand Frota (1986), who documentedthe experience of one of them of learning Portuguese in Brazil. Jones (1994) analysed his own self-study of Hungarianusing teach-yourselfmaterials. As might be expected,second languageresearchersin particular are interestedin exploring their own language-learningprocessesand thus making links with the researchfield more generally.Their raw diary datais itself quite revealing(the short quotationsare taken from Bailey [B], Jones U), and Schmidt and Frota [SF], and where unacknowledgedfrom the languagelearningdiary of one of the presentauthors): Fantastic feeling: I can read my magazine articles without a dictionary ... [J] ... in the beginning I was very shy, I didn't even dare open my mouth ... [J] I was really up, self-confident, feeling fluent ... [SF] I feel very anxious about this class. I know I am (or can be) a good language learner, but I hate being lost in class. I feel like I'm behind the others and slowing down the pace ... [B] I really like doing verbs and guessing the 2nd stem - more like a logic game ... ... his face showed complete non-comprehension. I grabbed my dictionary ... [SF]

130

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Some people said they felt they were getting worse and I didn't like to say I don't!

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

I hate the feeling of being unable to talk to people around me ... I don't like the silence ... [SF] It seems that it's easier to learn sentences or expressions insfead of lone words ... [J] My grammar background is probably stronger than most of the students'. I'm just having trouble in recognizing and producing the spoken language

... [BJ The remainder of the class was choral repetition ... I didn't like that much either ... what a sour start! But I think I'll stick with it ... [SF]. Today I decided to speak to the man who is so uptight about his ... test. I was sad that he didn't come to class. I hope he doesn't drop the course. I said hello to another student in the hall ... but he just nodded. I would have liked to have someone to commiserate with ... I am absolutely worn out ... [B]

Certainly thesediaries turn out to havea great deal to reveal about language-learningstrategiesand processes:indeed, most of Jones' formal analysisis concernedwith these factors, being divided into categorieshe calls thresholds/grammar / writing/ reading/pronunciation/speaking/listening, with just a short section at the end attending to motivation. However, it is clear that expert writers are also preoccupiedwith affective, social, interpersonaland classroom-methodological influences and reactions. Allwright and Bailey (1991: 38) make the following point, which is worth quoting in full, about Bailey's French class diary. Bailey, as a researcher,was particularly interestedin error treatment.However: error treatmentapparentlyhad not beenimportant enoughto her as a languagelearner to be recorded in the journal entries. What she found insteadwere numerousreferencesto competitivenessand anxiety ... , a topic that arosefrom the data rather than from any preconceivedhypothesisor theory of classroomlanguagelearning. This topic turned out to be a productive focus in itself, but a certain amount of frustration aroseand time was lost in the searchfor the learner'sreactionsto error treatment-a topic for which there were no pertinentdatain the journal. Rathercuriously, Allwright and Bailey regard this as a reasonfor at least novice researchersnot to collect data first: in fact it is fascinatingevidence of a tension betweena theory- versusdata-first paradigm,and betweena learner(or pedagogic)versusresearcherrole.

DIARIES AND DIARY STUDIES

131

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Teachers' diaries In comparisonto learner diaries, particularly the secondtype, there are rather fewerpublishedaccountsin EFL to dateof diaries kept by teachers during their everydaywork in Schon's(1983) 'swampylowlands'- another kind of 'expert' diary. The genre is somewhatmore endemic in teacher research in school-basededucation (see, for instance, Hitchcock and Hughes,1995, who discussthe use of diaries in this context in their book on qualitative research;also Enright, 1981, and Tranter, 1986). The relative lack of teacherdiaries in EFL may be due in part to the historic dominanceof the applied linguistic tradition and the 'scientific' paradigm,or perhapsto the timelag betweenlearningabout diariesand relatedmethods on teacher-trainingprogrammesand actual uptake in practice. Whatever the reasons,diary-keepingis arguably one of the ways in which teachers can get closest to their own work and hence, via critical reflection, to researchingit. Potentially thereare many and varied themesthat could emergepost hoc from a teacher'sdiary, and which tap directly into the researchableissues that we discussedin Part 1. As Holly (1984) puts it, what to write about is 'limited only by your imagination'.She lists threeareas: 1. teaching:what you do and why 2. students:what they do, circumstances,descriptionsof behaviours 3. collegial interactions. To this we could add a further category: 4. metacomment:thoughts about the processof diary-writing, and reactionsto one'sreadingsand the researchof others. Each of thesecan be broken down to becomemuch more specific, so the very generallabel of 'teaching'might include (to nominatejust a few): • planningthe lessonand selectingmaterial • dealing with different typesof learnerin the class(mothertongue, learningstyles,attitudes,pacing,proficiency ... ) • whether the plan matched the reality; why some things worked, othersnot • coping with the unexpected;changingdirection • using availableresourcesand technology • speculation(aboutlearning;behaviour... ). Turned on its head, Richards and Lockhart's (1994) list of prewriting prompts referred to earlier provides a useful checklist of themesto look for when diary data havealreadybeenproduced. The potential range of topics is reflected in the diary studies publicly reported.Bailey (1990) hasa useful survey of a numberof these(Telatnik; Butler-Wall; Deen; Ho), which cover such diverseareasas the relationship

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

132

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

betweena teacher'sideason teachingand thoseof expertsand theorists; teacherpower; teacher'schoice of classroomlanguage.McDonough and McDonough (1993) report on a teacher'sdiary that both reflected the gathering and interpretationof qualitative data and, by focusing on an individual learner, turned into a casestudy subsequentlyinvoking other methods.Llewelyn (1995), working in the contextof a collaborativeaction research project on the Australian Adult Migrant English Program (AMEP), chartedher coursedesign processesand her developingunderstandingof functional grammarthrough the mediumof a teachingjournal. Burns (1995) in her introduction to the collection of papers in which Llewelyn's is included,situatesjournalsin the framework of all the action researchmethodsused in the AMEP project. McDonough (1994) examined the diaries of four teachers(including herself) on the samelanguage programme,and found that the analysisyielded themesto do with individuals; the whole group; methodology;teacherroles; style and language of diary entries;and metacommenton the value (or otherwise)of writing a diary. The most extensivereport of a teacher'sdiary is to be found in Appel (1995) who looks back over and commentson the diaries he kept during his first six years of teaching English at secondary school level in Germany. The book is divided into three sections, entitled 'Survival', 'Change' and 'Routine', charting Appel's own 'journey' as a language teacher.He writes: 'I was sifting my personaland subjectiveevidencewith certain guiding questionsand conceptsat the back of my mind. At the sametime the conceptsthemselveswere sifted and modified as they were confrontedwith the evidence'(1995: xiii). Eachsectionis precededby the diary entries, and is followed by analysis. The presentationis a little crafted, and on occasionsthe diary entriesdo not so much generateanalysis as suppor.tthe author'sbroaderline of pedagogicargument,and the juxtaposition of individual entries with generalizationsabout teaching sometimesstretchescredibility a little. Nevertheless,the record of Appel's experiencegives a powerful sense'of what is was like to be there participating in it', and his accumulationof experienceover time offers a fascinating thematic transition from 'novice' to 'expert' concerns.He moves, for instance,from a preoccupationwith discipline and control to a position wherelanguagelearningand methodologybecomemore centre-stage. Teachertraining

Diaries kept by trainee teachershave severalobvious parallels with what we have termed 'pedagogic'learners'diaries (Bailey, 1990, choosesto use the term 'academicjournals'). In particular,they are usually in somesense a requirementof the training programme,and they almost invariably have

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

DIARIES AND DIARY STUDIES

133

prespecifiedcategoriesas a framework for writing. It is also unlikely that the traineesthemselves,evenon INSET courses,will carry out a researchorientedanalysisthat will eventually go public: rather, they are exploring in the foothills of that research,being sensitizedto the valuable role of critical reflection on practice at the sametime as learning about at least one researchtool. Porter et al. (1990: 240) make the further point that 'The journal encouragesstudentsto go beyondlearningcoursecontentin isolation and to strive to link this information to theoriesand knowledge beyond the particular assignmentand the particular course ... In sum, it teachesthem to do what we do as professionals- to work to integratenew ideaswith what we alreadyknow and to talk with eachother as we do so', a perspectivethat makesa direct link with Ramani's(1987; Chapter5 of this book) 'theorizingfrom the classroom'.There would, incidentally, be much scopefor comparingthe kinds of topics addressedby experiencedteachers, on the one hand,and 'novice' trainees,on the other. As with the teacher who carries out a diary study based on his/her learners'writing, so of coursea teachertrainer'sanalysisand write-up of trainees'logs and journals is a researchoffering to the wider professional community. Murphy O'Dwyer (1985) looked at the attitudes to learning reflected by participants'diaries on a short training course, and in her analysis identified the four categoriesof group dynamic, generaladministrative constraints, personal variables and presentation/ content. Thornbury (1991) used teachingpracticelogs for traineeself-assessment, and found that they talked about perceptionof self, subject-matter,tasks and learners/learning,and also about the developmentof their own 'craft knowledge'.Other reports of trainee diaries, and generaldiscussion,can be found in Jarvis (1992), Bailey (1990), and Porter et al. (1990). Finally, Richards(1992) proposesan interestingvariant where he askedtraineesto learn a foreign languageand keepa written log of their experiences.Major themes recorded were the teacher; the learning experience;motivation; authenticityof materials;vocabulary;cultural factors; phonology. In this section,we have consideredthe following types of diary writing in languagelearningand teaching(seeFig. 8.1).

pedagOgic

learners in classrooms trainees

Diaries langUage learning

expert

experienced teachers teacher trainers

Fig. 8.1

134

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

We concludethis chapterwith a brief overview of practical proceduresfor diary keeping,and a short discussionof someof the broaderissues.

Keepinga diary

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Methodology Referencewas made earlier in this chapter to Bailey's (1990) five-stage procedurefor carrying out a full-scale diary study. On a practical level, keepingthe diary itself is in principle straightforward.No specialarrangementsare required,just pen and paper(or a wordprocessor)and a modest amountof time that is anyway organically related to the teachingor learning activity that it records. With precodeddiaries, particularly where the writer (or 'respondent'in this case)is writing underthe guidanceof a researcher,adviceon the 'how' is likely to be quite detailed, perhapsincluding the type of notebook, a model entry, standardizationof the time period allocated to particular pages,and a checklist of items to jog the writer's memory (Corti, 1993). This kind of systematizationis clearly helpful to the researcherin sifting through a large amount of data collected with specific researchquestions and hypothesesin mind. With all kinds of diaries, whether open-ended narrativeor with prespecifiedcategories,the standardadviceis in line with that given by Bailey (1990), andAllwright and Bailey (1991): • set aside a time each day (or each lessonif it is less frequent) to write the diary • tape-recordingis an option, but may be more time-consumingif it requirestranscription • allot roughly the same amount of time to the diary as spent in class(less feasible if one is teaching/learningon an intensiveprogramme) • concentrateon recordingdata, not on style, grammaror organization • record facts and eventsas well as reflections. Richards and Lockhart (1994) also suggestreviewing entries 'regularly', but do not specify what this might mean: we have noted the possibledanger of inadvertentlyrestricting subsequentwriting by doing a preliminary analysistoo soon into the process,especiallywith self-driven diaries. It is also worth speculatingthat diarists may use a different style of writing whetherthe addresseeis the self or a teacher/researcher. Discussion

Those who keep and/or work with language teaching and language learning diaries are, not surprisingly, advocatesof the genre, linking it

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

DIARIES AND DIARY STU DIES

135

particularly to naturalisticparadigms:'The journal holds experiencesas a puzzle frame holds togetherits integral pieces... clues lead to new clues, partial perspectivesto holistic perspectives'(Holly, 1984: 8). In the course of this chapterwe have alreadyidentified a numberof salient issuesto do with diaries in research,including the quantity of data, the benefits of sharinginterpretationsin a public domain, and the nature of exploratory processwriting itself (seeRaimes,1983; White arid Arndt, 1991). Here we briefly add to these,noting that a countercan be found to most pros and consdependingon one'sbeliefs and assumptions. In termsof researchmethod,teachingand research,teacherresearchers and professionalresearchersare in particularly close approximationin the diary mode,whetherresearchersare keepinglanguage-learningjournalsor teachersare critically reflecting via their diarieson the many facetsof their role. This is, then, one significant instancewhere everyoneis engagedin a comparableprocessand working within a comparableparadigm.The perceived advantagesof the techniqueare also shared:the diary, it is argued, is a primary vehicle for processresearch,for getting 'underthe skin' of the psychological,social and affective factors involved in teachingor in languagedevelopmentin ways that cannot readily be reachedby staff meetings or tests or population sampling or experiments. Progoff, who pioneeredthe use of journals in psychotherapyand whose work has also been influential in education,writes of 'experiential feedback',and puts the point thus: 'The journal enablesour subjectiveexperiencesto become tangible to us. Experiencesthat would otherwisebe intangible and therefore too elusive to grasp thus becomeaccessibleto us so that we can work with them' (1975: 37). We still, however, need to add a cautionary reminder that introspectionof the kind recordedin diaries is not universally suitableand not everyonefinds suchself-scrutinyilluminating. With diary studies, as with much interpretive research, there are inevitable chargesthat they lack reliability and validity. Some researchers attemptto deal with this on its own terms by using parallel diariesor other forms of triangulation, such as data gatheredfrom other sourcesor other kinds of documentsas part of a single study. Others reject the concepts themselves:Lincoln and Guba (1985), arguing that 'bias' and perspective' are of central importancein social and human research,prefer; as we have seenin Chapter4, to think in terms of dependabilityand credibility. Jones (1994: 444) asks: 'But is subjectivity forever wicked? If the object of a study is to discover individuals' reactionsto the learning process... then one might arguethat "subjectivity" (how one perceivesthe processes,what one choosesto record, etc.) is a prime researchaim. More good fairy than bad, in other words'.

136

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Thesequestions,and the crucial one of generalization,will resurfacein the later chapteron casestudies,to which diaries are conceptuallyclosely related.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Conclusion This chapterhas shown how important diaries can be as a researchtool. There are a numberof different formats for diaries, all sorts of categories of information can be included, and there can be varying degreesof advancespecificationof what is to be included and how often they should be written up, dependingon the researchpurpose.However, the natureof all diary data is that it is 'rich' and 'thick' and retrospective. Some methodsof analysisof this kind of data were explainedin the chapter.We also highlighted the difference betweena private diary and a public diary study of one or more diaries. A major section of the chapterthen reviewed extant samplesof diary material in languagelearningand teachingand their use in research,making a distinction between'pedagogic'diaries, written to aid reflective language learning and teachereducation,and 'expert' diaries, in which the writer recordslanguage-learningexperiencesor incidents in professional life. A further section highlighted the necessityfor method and indeed personaldiscipline in keeping a diary to be analysedfor a researchpurpose,to protectthe qualityof the data. Finally, the chaptercommentedon the value of diary information and diary studies,on somereasonsfor their popularity and perceivedutility in teacherresearch,and on somecounter-arguments to the inevitable accusations of unreliability and invalidity. Many of theseargumentslink directly also to the issuesof participant observationand casestudy researchdiscussedin Chapters4 and 13 respectively.

Discussionnotes 1. To what extent do you think that a diary kept as a teacher about one's own language-learningexperiencescan shed light on the ways in which our learnerslearn? 2. If you have not kept a teachingdiary, try to set asidesomespecific times each week to record fairly freely your experienceswith a particular class.After a few weeks you will have some useful data which can then be comparedwith the outcomesof other studies suchas thosereportedhere. 3. What, if any, are some of the specific ways in which diary data might be able to deal with particular preselectedissuesin languageteaching?

9

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Using numbers

Introduction This chapter looks at the use that can be made of more or less simple counting techniquesby language teachersdoing research.It also gives somebackgroundto thosetechniquesto harnessthe considerablepower of numerical analysis in describingand interpreting many kinds of data. It also attemptsto uncover someof the assumptionsof numerical methods, which, as we saw in Chapter4, are not sharedby all the approachesto researchcoveredin this book. The chapterthat follows then takes a brief look at the notion of experimentationto establishwhat might be useful to the readersof this book from that approachto research.Neither chapter can act as a comprehensivemanual for numerical techniques;for that the readeris referredto books such as Hatch and Farhady(1982), or Woodset al. (1986).

Numerical methodsare only of use to the researcherwhosedata can be expressedas numbers.A first question,therefore,is to ask: • What advantagesare there in obtaining numericaldata in the first place? There are also several kinds of numbers,each having different assumptions and associatedpermittedoperations.A secondquestion,then, is: • What can be done with the kinds of numbersobtained? Numbersmay be obtainedfrom one respondent(for example,a learner)or from a class,a cohort, a year group, a school region, a county, a host - in other words the third questionis: • How can numberscope with the scaleof the data? The useof numericalmethodsin large-scaleresearchhighlights the primacy of reductionism: ~he reduction of large amountsof data to one or two figures, commonly known as descriptive statistics. The issue of scale also highlights the distinction between'insider' and 'outsider' research:a teacherwanting to know aboutactivities in other school contextsmight be able to senda questionnaireto colleaguesand count up the answers(as an

138

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

'outsider') where doing observationsor casestudieswould be impossible. Numerical data is usually associatedwith a positivist, normativeapproach as discussedin Chapter3, but numerical methodsalso have their place in qualitative research.

Why count,and what to count In a learning context, there are many kinds of data that can profitably be counted.A teachermight wish (for example): • • • •

to look at students'scoreson a test or examination to evaluatethe responsesto a questionnaire to estimatethe relative importanceof items in a needsanalysis to compare the frequencies of different teaching/learning exchangesin a classroomobservation • to investigatethe frequency of mention of different themesin a study of learners'or teachers'diaries.

Counting may be the first step in a purely normative kind of study, or it may be only one way of analysingdata which may be looked at in a number of different ways. Counting may thereforebe useful also in qualitative and interpretiveresearchapproaches.

Different kinds ofcounting The teacherlooking at the codedexchangesin the classroomis most probably counting frequenciesonly. Frequenciesare usually called 'nominal' data - often in fact only counting namesor labels. The teacherlooking at examinationor test scoreswill be looking at somekind of scale,in which a lower figure usually means worse performance than a higher figure. Sometimesall such a scalemeansis a rank order: A is better than B who is better than C who is better than D who is equal to E and F, etc. Theseare 'ordinal' scales,and in many circumstancesare the most appropriateand fairest statementof learnerdifferenceson somecriterion. Mostly, however, people dealing with scoresof various sorts want to be able to distinguish levels of achievementor proficiency with greaterprecision, and specify by how much A is better than B and if B is better than C by the sameor different amount. Examinationand test scoresusually aim at this kind of statement,which is normally called an 'interval' or 'equalinterval' scale. The assumptionis that the differences betweenany two scale points are equal. Thus the difference betweensomeonescoring 60 and someonescoring65 is larger than the differencesbetweenthe first and a classmatewho scored58. However, as can readily be seen,when dealing with most kinds of knowledge, and languagein particular, this does not seemto mean very much. What the 60-scorermust do to emulatethe 65-

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

USING NUMBERS

139

scoreris in no realistic sensetwo and a half times what the 58-scorerhas to do to emulatethe 60-scorer. This equal-intervalscale is therefore some way short of the 'highest' level of numerical measurement,the 'ratio' scale, which has a zero and permitsmultiplication and division, althoughit may look like one. Consider the ESU Framework (Carroll and West, 1989) or the BC/UCLES IELTS Band scores:'the difference betweenlevels 3 and 4 may be, conceptually,much the samein size as that betweenlevels 5 and 6, and might be translatableinto roughly equal numbersof hours of instruction neededto move from one level to another.However, the actual skills and languageitems required, range of text types in reading and listening to be mastered,interactionaldemandsrequired will differ, as defined by the framework scale.Alderson (1991) gives a lucid accountof the notions of scoresand levels in ELT professionalmeasurement,remarkingthat language test scoresand bands are not even really interval scales,although they are often so treated. The ratio scale is rarely relevant for the kinds of researchdiscussed here. Obvious examplesare weights and measures,or money, where there is a meaningful equality of unit and a meaningful zero: but virtually no data collected in educationalsettings can make the assumption,for example,of 'no knowledgeat all'. This sectionhasintroduced,ratherquickly, four levels of measurement: nominal, ordinal, interval and ratio scales.It is, of course,highly debatable whether the kinds of data collected in language-teachingresearchare 'measurement'in any useful senseanyway.After all, most of the discussion in the presentvolume has beenin terms of descriptionand interpretation, not measurement.Certainly some of the researchapproachesmake that assumption,but not all.

Describingthe numberswith other numbers When dealing with reasonablylarge amountsof data, like scoresfrom a whole class,or the results from a questionnairereturnedby 20 people,or much larger-scaleresults from someexperimentalinvestigation,there is a need to find someway of summarizingthe data to reducethe massof figures to an easily understoodgeneral picture, perhapsto find the typical value that more or lesscharacterizesthe whole. In fact thereare two argumentshere: • first, the practical one that finding some kind of 'typical value' enablesthe researcherto think about the group who producedthe

1

BC: The British Council; UCLES: University of CambridgeLocal ExaminationsSyndicate; IEL TS: InternationalEnglish LanguageTestingService.

140

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

data as a group rather than as a collection of independentindividuals • second,the more abstractargumentthat no one casecan be taken as characteristicof a group, but that the truth of the matter is in the distillation of information from many observations. The classic example of the second is the distribution of height among humans:the answeris not anyoneperson'sheight, but an averagewhich lumps togetherdifferencesof ethnicity, nutrition, age, diurnal variation, posture,and so on. This normativebelief in the natureof truth as lying in a statisticalreductionof datafrom many observations,whetherof different people or of the samepersonat different times, is clearly important for many kinds of knowledgebut is the subjectof reasonedattack in educational and sociological research,in particular where the effect of educational variableson individuals' learningis the subjectof study. Central tendency

Having thus rather loosely introducedthe conceptof an averageor mean, we should now look more closely at ways of representingthe typical value or 'centraltendency'of a massof data. With frequencydata,the simpleststatementis the most frequentvalue. Thus, in a questionnaire,one of the questionsmight be agreedwith by most people(seeFig. 9.1). Q: Doing statistics is a drag - really difficult

Frequency

strongly disagree

Fig. 9.1

disagree

don1 care

agree

strongly agree

USING NUMBERS

141

This would be the 'mode' or 'modal' answer, the most popular. It is readily obvious that more than one outcomeis possible- this degreeof homogeneityin a group answeringa questionnairewould be rare. In fact theremight be two most popularanswers(seeFig. 9.2).

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Q: Doing numbers is a breeze - really easy

Frequency

strongly disagree

disagree

don1 care

agree

strongly agree

Fig. 9.2

This would be a 'bimodal' distribution, indicating that the group returningthe questionnairefell into two distinct campson this issue. With rank-orderedor 'ordinal' data, the normal way of choosinga typical value or of representingthe midpoint around which all the other values cluster is the 50 per cent point: the score with half the data points aboveand half below. This is called the 'median'. With interval data the midpoint is determinedby using the familiar averageor mean: the sumof all the values dividedby the numberof values. A moment'sreflection will confirm the equally familiar fact that median and mean are not always the same, and that often the cruder measureis actually a better figure to work with simply becauseit is not affected by extremevalueswhich may distort or 'skew'a mean.

Dispersion Simply finding a 'typical value' or a 'midpoint' is often not a sufficient statistical descriptionof a group or distribution of scoresor results: it is usually important to know how typical it is or how closely the other results clusteraroundthe midpoint. In other words, the researcherwill be looking

142

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

for a measureof 'dispersion'around themidpoint, or the typical rangeof scores- in a sense,the degreeto which, on average,the results disagree with the typical value, midpoint, or mean.This measuregives an indication of the homogeneityof the group (seeFig. 9.3).

o

10

20

same midpoint same means

30

40

50

60

70

80

90

100

different dispersion different degrees of homogeneity

Fig. 9.3

The usual statistic for this purpose borrows a useful principle from probability theory that teachersfamiliar with interpreting test and examination results will also be familiar with: the idea of expressinga scorein termsof its distancefrom the midpoint, of its deviationfrom the norm, or the 'standardscore'.A standardscoreis a scoreexpressedas the difference betweenthe raw scoreand the mean,in terms of the generalshapeof the distribution of scoresaround that mean, which is given by the 'standard deviation'.The useful side of this borrowing is that accordingto probability theory, one standarddeviation either side of the mean encompasses

USING NUMBERS

143

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

approximately68 per cent of all the scores,two either side 95 per cent, threeeither side 99 per cent.This is illustrated in Fig. 9.4.

-3 s.d.

-2 s.d.

-1 s.d.

mean

+1 s.d.

+2 s.d.

+3 s.d.

Fig. 9.4

Thus the most useful descriptivestatisticsfor any group of scoresare the arithmetic mean and the standarddeviation (within certain restrictions); and any individual scorecan be interpretedin termsof the performanceof the group by referenceto thesetwo descriptors. For example,the group mean of a certain test might have been 50; one studentscored65. Was that exceptionalor fairly ordinary? If the standard deviation of the group had been 20 points, indicating that nearly 70 per cent of the studentsscoredbetween30 and 70, then clearly it was on the good side of ordinary (seeFig. 9.5). But, if the standarddeviation had been only 5 points, indicating that nearly 70 per cent of his classmatesscoredbetween45 and 55, very much level-peggingin a very homogeneousgroup, then clearly it was an exceptional performance(seeFig. 9.6).

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

144

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

x

o

10

20

30 -1 s.d.

40

50

60 x 70

80

90

100

+1 s.d.

Fig.9.S

Notice how thesenormativemethodsallow one to interpret individuals in comparisonto groups,and certainaspectsof group behaviour.They do not allow interpretationof an individual's learning experience,nor of the relationship betweenthe individual and his or her own goals, except as they are measuredagainstsomegroup score,or referencenorm.

Correlation Another kind of descriptivestatisticthat is both very familiar to peopleby nameand also very useful is a measureof how well a certain characteristic or property goes with another - measureof associationor correlation. Thereare two kinds of questionhere: • how to establishan associationor correlation • what it might mean.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

USING NUMBERS

145

x

o

10

20

30

40 50 60 x70 -1 s.d. +1 s.d.

80

90

100

Fig. 9.6

It is here that we introduce two of the most widely used and familiar statistics of all. The first is a measureof association. For the purposeof example,it is a frequent observationthat teacherslike gamesin the classroom more than students(cf EFL Services,1992; Nunan, 1988): a questionnaire study could perhapssettle the matter. Imagine a questionnaire about using gamesin the classroomgiven to a group of studentsand a group of teachers.One of the questionsmight be a simple one of liking. The responsescould be set out simply as a 2 x 2 table, with liking and disliking in the columnsand teachersand studentsin the rows (seeFig. 9.7).

Fig. 9.7

like games

dislike games

teachers

lots

few

students

few

lots

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

146

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

In the examplein Fig. 9.7, there is an associationbetweenthe question of gamesin the classroomand participantrole: the greaterproportion of teacherslike games,the greaterproportion of studentsdo not. The question did not ask aboutdegreesof liking, only liking versusdisliking, so the results are in terms of frequencies,in fact proportions.Where frequency data can be assembledin this fashion, a very useful statistic called chisquarecan be usedto test the strengthof any associationpresent.It is also simple, but tediousto work out. Once such an associationhas beenestablished, the real work begins- interpretingit. A correlation is anotherwidely used term that in numericalapproaches refers to a group of people giving results on two rather than only one scalar variable - i.e. not frequenciesbut scores.Many people believe, for example,that learningachievementin a foreign languagegoeswith degree of aptitude or talent. In the heyday of such studiesit was indeed shown that scoreson languageexamsdid co-vary with scoreson so-calledaptitude tests, but not very strongly. In its simplest form, such researchtook two setsof scoresfrom eachlearner:an achievementscoreand an aptituuc score.All the people(herecalled A, B, etc.) were then plotted on a scattergram accordingto their markson both tests(seeFig. 9.8). 100 80

K

M

J

I

H G

60 E

achievement

40

F

0

C

L

20 A B

0

10

20

30

40

50

60

70

80

90

100

aptitude

Fig. 9.8

In the examplein Fig. 9.8, one can readily seethat most of the people, but not all, have scoreson the two teststhat roughly tally. There is a 'positive' correlation.It is not perfect: it is not true of either M or L. It is possible to work out a figure, called a 'correlation coefficient' which gives a descriptionof the overall degreeof agreementbetweenthe two measures or setsof scores.It is worth pointing out that correlationscan be positive or negative.If it were true, which it does not appearto be, that J..:hievement in learning a foreign languagediminishes with advancing age, it would be an exampleof a negativecorrelation.They can also be calculatld for all kinds of data,exceptfrequencies.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

USING NUMBF~~

147

However, interpreting both associationsand correlationscan be difficult, sinceone always wants to know what the two featuresactually have to do with each other and whether one causesthe other. In the caseof the associationbetweenparticipant role and game preference,one could not argue, for example,that the students'reaction was directly to do with the teachers' predilection, becausein that example nobody suggestedthe teachersand students had ever met. Even if the students were being taught by the teachers,the reasonseach group had for their preference might be quite unrelated:the teachersmight have thought in termsof participation, distribution and control of interaction and lightheartedness, whereasthe studentsmight have thought primarily about boredom with long setting-uptimes, constraintson their freedom of expression,or lack of learningpay-off. In the caseof the correlationbetweenachievementand aptitudeone might want to assumethat aptitude'caused'the achievement. There would be no justification for this becausewith two measurestaken at the same time there is no indication of what the actual relationship might be: being better at the languagemight as well be helping with the aptitudescoresas vice versa. Mostly, interpretationof correlationsmeans looking for other factors which might be influencing both sets of results. Causalarguments,if appropriateat all in languagelearning, can properly only be madeon the basisof experimental,not co-variance,data. This is becausethe natureand purposeof experimentsare to reduceall possibleextraneousinfluenceson the resultsto the particulartreatmentor treatmentsbeing manipulated,i.e. to the effect of one action on another (in the simplestcase).In a correlationalstudy, however,it is usually not the casethat all the other possibleinfluenceson the results, the rival explanations, can have been isolated,and it is also usually not the casethat either one of the measurestaken can be consideredto be affecting the other one, becausethey are takencontemporaneously. An interestingissue ariseswhen scoresare taken from the samepeople at different times. An exampleof this is the use of correlationto establish predictive validity of proficiency tests. A major use of tests such as TOEFU and IELTS is to estimatethe chancesof non-nativespeakersof English with certain language levels succeedingin academic study in North American and British and Australian Higher Education.Typically, validation studiestake a large sampleof applicantsand comparetheir test scoresat the beginning of their courseor before the course begins with their academicresultsat somepoint during or at the end of their courseof study. The interpretationof such 'predictive validity' is that performance in the academicstudy, if in generalit co-varieswith initial languageproficiency, is becauseof that proficiency. Academic qualifications of new 1 TOEFL:

Test of English as a Foreign Language(EducationalTestingService,Princeton,New Jersey,USA).

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

148

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

entrants are in this instance held to be equal. Such tests are used by receiving institutions as part of their selectionprocess.A great deal is at stake: acceptinga studentwho is likely to fail becauseof languageproblems is againstvariouscodesof practice(such as that of the Committeeof Vice-Chancellorsand Principals); it will cost the sponsoror possibly the studenta great deal of money for nothing, for fees, living e?,penses,and travel, and the unhappyexperienceof failing to cope in an alien environment could causelong-term psychologicaldifficulties; the waste of time and effort might cost the individual a start in an alternativecareer.While it is undoubtedly sensible to use tests to safeguardagainst these consequences,it is illogical to interpret the correlationto meanthat language levels 'cause'academicsuccessor failure, since variation in academicsuccessoccurs,obviously, with native speakersanyway, in the absenceof the foreign-languageproblem. It is, however, reasonableto suggestthat the correlationreflects some kind of real relationship,the natureof that relationship being quite complicatedand tenuous.This is an instanceof Van Lier's (1989) argument that there are principled differences between causesand reasons.On the specific point of predictive validity of test scores,studieshaveshown (Criper and Davies, 1988) that in fact it is difficult to establishgood correlationaldata on this point, and that statistical evidencefor using particularscalepoints as decisionpoints for acceptance or rejectionor for recommendations for extra languagecoursesof different lengthsis often unreliable. Groups and individuals

This examplehighlights also a generalpoint aboutnumericaldataand correlation in particular: such data can tell us quite a lot about groups, but rather less about individuals. Teachersof English for academicpurposes and English for specific purposesprobably haveas many storiesaboutstudentswho seemedunlikely to succeedat the languagegoing on to be academically successfulas they do about linguistic high-fliers failing their first-year examinations.Statisticscan tell us important information about general trends which may feed properly into institutional plans and resource allocation, but they cannot be held responsiblefor decisions aboutthe propertreatmentof individuals. Bivariate and multivariate research

In eachof the aboveexamplesof associationand correlation,therewere two variables: we discusseda hypothetical associationbetweenparticipant role and game preferencein the classroom,a correlation betweenaptitudeand achievement,and a correlation betweenlanguageentry level and academic

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

USING NUMBERS

149

success.In many, perhapsmost, situations, life is more complicatedthan that, and there are more than two relevant and interactingvariables.More complexnumericalmethodsinvolving correlationsare availablefor analysing more complexsetsof data.Inevitably, suchstudiestend to demanddataon a largerscalethan is availableto most individual teachersworking in one context, so here a few exampleswill be given, only to give the flavour of such research. Multiple regressionand partial correlation can analyse severalsets of scoresto determinethe strengthof the effect of one on the others,or the strengthof the prediction from eachto a final score.A frequentexampleis the tester trying to determinewhich of a battery of tests - of grammar, pronunciation,listening comprehension,cloze reading, stressperception, sound-symbolcorrespondence, vocabulary,oral production,etc. - is having the strongestinfluenceof the final discriminationbetweenthe test-takers. Factor analysis and cluster analysis, which can handle very large amountsof data,are usedto reducethe scoresfrom large numbersof measures to a few underlying influencesor 'factors', and indicate how strong the influence or loading is from anyoneof the factors on the individual scalesusedto derive the analysisfrom. In languagelearningresearch,perhapsthe most widely known use of factor analysisis the work of Gardner and his many associateson motivation in languagelearning (Gardnerand MacIntyre, 1993) which has investigatedthe relationshipof many issues such as orientation to languagelearning, anxiety, strengthof motivation, aptitudeto learningachievementin a foreign language.Gardner'sresearch over the yearssince 1959 has involved thousandsof studentsin many contexts, but not all multivariate studiesare mega-scale.Johnson(1992) cites the interesting study by Ely in her chapter on correlational approaches, which used a sophisticatedcorrelational method to try to teaseout the strengthand direction of influenceof emotionalconcerns,such as discomfort, risktaking, sociability and motivation, on classroomparticipationand proficiency in the languagein one institution. Ely's study did, however, use 75 college-level learnersof Spanishin six classes,so it might not be fair to claim it was small-scale. Condition-seekingresearch

In general,the kind of researchdescribedabove usually involves looking for the interaction betweensome particular learner trait, like motivation, willingness to participate, extroversion, willingness to take risks, and so on, and some treatment or instructional approach. In other words the research looks for the conditions under which certain influences are important or certain approachessucceed.For these reasonsboth Skehan (1989: 134) and McLaughlin (1987: 162) equate 'condition-seeking' researchwith researchbased on 'aptitude-treatmentinteraction'. Most

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

150

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

teacherswould readily acceptthat somestudentsrespondto certain ways of teachingand othersto different ones,and vary within themselvesfrom time to time. Condition-seekingresearchattemptsto clarify and systematize these interactionsbetweenvariables. Skehancomments:'The three major conclusions we can draw from this chapter [1989: Chapter 7, Interactions]are first, that the completedcondition-seekingresearchhas beensomeof the most fascinatingin applied linguistics; second,that there are many studies whose interpretation is not clear for researchdesign reasons;and third, that abysmally little researchof this sort has actually beendone'(Skehan,1989: 134). He goes on to call for more, better-designedresearchin this vein. However, it is instructive to note that, outsidethe numericalor normative approachesto research,condition-seekingresearchis regardedin a much less favourablelight. Thereare basicallytwo reasonsfor this: • it presentsgroup results which cannot easily be applied to individual learners • it fragmentsthe participantsand the situation rather thanproviding an elaborateddescriptionof the learningsituation.

Inferencefrom chance- 'significance' So far in this chapter we have consideredvarious uses of counting and ways of using powerful numerical techniquesto make senseof what has been counted. A core concept in statistical techniques goes beyond descriptioninto the nature of what constitutesa fact in such approaches: the significanceof the result. Significancein this sensedoesnot necessarily equal meaning- that is a questionof the importanceof the fact discovered for the researcher'sinterpretation,hypothesis,theory or whatever. Significancehere refers to the likelihood of obtaining the given result by chanceas opposedto the operation of some effect, presumably theone being manipulatedin the research.If, after all, it were possible to show that the correlation betweenaptitude and achievementreferred to above was just chance, it could not be considereda fact at all: in normative approachesfacts are contrastedwith chanceoccurrences.But how can one tell them apart?For this, we can useinferential statistics. Perhapsit helps to imagine a situation in ordinary life outside the discussionof researchmatters.Considerjaywalking: it would be unfortunate if a bus hit one of us when we attemptedto cross theroad. If we counted how often it happenedto everyoneelse- say we found it was 1 in 20 times on averageon walking out to crossthe road we might comparethat to our own experience.How would we feel if it happenedto us every 2nd time, or every 5th time, every 20th time, or every 100 times?If 1 in 2, in 5, or in 10 times, we might think we were jinxed; 1 in 20 times would be par for

USING NUMBERS

151

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

the course; but I in 100 times we would think we led a charmedlife. In other words, we would define ourselvesby reference to chance levels. Conventionally,normativemethodsstart taking eventsseriously- calling them facts - if they actually occur in the investigation whenthe chances of them happeningare I in 20 or less, and seethem as establishedfacts if the chancesof them happeningare 100 to I against,all other things being equal.

But we do not have infinite time in which to determinechancesof survival of jaywalking in this way, so we have to turn to mathematicsto determine how many accidentsindicated a deviation from normality: whether we were jinxed or charmed.The mathematicianswould look at the accident figures for a 'sample'of the generalpopulation with which to compare our particular case. The larger the sample, the more accuratethe estimate. To return again to the correlation example, the same logic shows that the significanceof the correlation is dependenton the size of the sample from which it is obtained.The more peoplecontributeddata, the greater the likelihood that the resultingcorrelationcoefficient was a real phenomenon and not just an effect of chance.In the caseof the association,the data was in termsof frequencies,that is to say what proportion of teachers liked or disliked gamesand ditto for the students.Chi-squareevaluatesthe significanceof theseproportionsby comparingthem with the distribution one would expect if there was no association(i.e. equal proportions of teachersand studentsliking and disliking). It then evaluatesthe size of the difference between the observed and the expected results taking into accountthe total samplesize: the numberof people polled. As one might expect, the larger the sample size the smaller the difference need be to prove to be significant, and the smallerthe samplesize the larger and more extreme the observedproportions have to be to be distinguished from chanceand thus be significant. Interpretation

Having establishedwhethera result is significant, the researcherhas to go back to the design and purposeof the researchto decide what the result actually means. Dependingon that purpose, both significant and nonsignificant resultsmight meanimportant things. A teachermight be interested in the use of particular kinds of tasks in the classroom,and obtain students'popularity ratings on theseand on alternativetask types. If a significant difference was found between the ratings on the two types of tasks, the teachermight be encouragedto continuedevelopingwhichever set was favoured. If no significant difference appeared,the teacherwould be faced with a very interestingsituation, becauseit would probably mean

152

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

that the principled differencehe or sheperceivedbetweenthe two typesof tasksjust was not relevantto the tastesor experienceof the students.That discrepancyof perceptionbetweenteacherand studentmight be valuable to do somemore researchon.

Computationalaids When one of the authors was taking basic statistical trammg several decadesago, it was typical to teachshortcutsto number-crunchingalongside the standardstatistical techniques.Large data sets were reducedto small ones by grouping - counting numbersof groups rather than individuals - to save on manual computationor card-punchingtime for the computer,the slight loss in accuracymore than compensatedfor by the time saved.Nowadaysthe widespreadavailability of pesmeansthat powerful statistical analysis programmeslike the Statistical Packagefor the Social Sciences(SPSS)or Minitabs are available(at a price in both money and memory terms) on domesticmachines.In somecasesthose who can accessa large mainframecomputerfrom their own machinedo not even need a copy of the programme.The sophisticationof such programmes has increaseddramaticallyeven in the last few years.Data entry and executive commandslook increasinglylike ordinary paperlayouts and natural languagerather than special formats governed by computing and programminglanguageconstraints,reducingthe user'sfear of complexstatistical operationsby making it apparentlyeasy. Naturally this can only be welcomed.However,thereare still somedangers: 1. using tests of greater power and sophisticationthan one needs becausethey are easy to execute,but then finding they are less easyto interpret 2. being presentedby the machine with impressive and complex resultswhich the usercannotunderstand. This seconddangeris evenmore to be guardedagainstwhen someoneelse actually runs the data through the statisticalprogramme.It is betterto use a less powerful test which the usercan interpretthan a more powerful one which only an expertcan interpret. The important principle is that the researcherneedsto be in control of the interpretationand can only acceptthe statisticaladvice that he or she can use in that interpretation.It is a trap which postgraduatestudentsfall into regularly - taking sophisticatedstatisticaladvice and entrustingtheir preciousdata to an expertconsultant,only to find that they cannotunderstandeither the adviceor the expert'sinterpretationat the end of the day. The outcome is that they cannot then defend their conclusionsto their colleagues,supervisoror examiners.However, such disastersare easy to forestall with planningand preparation.

USING NUMBERS

153

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Conclusion In this chapterwe have introducedthe most frequently usedtechniquesof numericalanalysis,and have tried to show that they, too, can form part of the teacherresearcher'savailable approachesto investigatingtheir classrooms and other aspectsof their professionalwork. Specifically, the chapter has looked at the different perspectiveson the notion of 'counting'and at a variety of standardways of analysingnumericaldata dependingon the natureand goal of the research.Theseinclude the conceptof 'mean'and how it can be interpreted,the correlation of individual factors with each other, and the complexity of our field such that multivariate analysis is often more suitablethan bivariate. Throughoutthe chapter,a background('paradigm')questionhas been the extent to which the complexity of humanbehaviourin an areasuch as languageteachingcan in fact be reducedto statementsconcerning'means' and the like, balancedagainstthe equally problematicissuesof generalizability and so on that are raised by more individualized and interpretive researchmodes.Finally, it is worth emphasizing- with the crucial proviso that the researchquestionand purposemust be clearly formulated - that the researcherhimself or herselfdoesnot have to take training in statistics, which are a tool like any other: specialistsupportis availableto handlethe data, whetherfrom statisticiansor computerpackages.

Discussionnotes 1. In your everydaylife as a teacher,what 'level' of measurementare the numbersyou use,suchas marks,roll-calls, costs,and so on? 2. What kinds of eventscould you count to investigatesomeof your students'learning problems,such as errors, turn-taking, strategy use, marks,and so on, and how could you describethem numerically? 3. Try to think of your own examplesof situationswhere we distinguish fact from chancein both professionaland private life. 4. How could multivariateanalysishelp in analysingthe influenceson students'learning achievementin your particular teaching situation?

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

This page intentionally left blank

10

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Doing experiments

Introduction It is not at all self-evidentthat the experimentalmethod as practised,for example,in the psychologylaboratoryor in psycholinguisticresearchis at all relevantfor the conductof researchin languageeducation,whetherby teacherresearchersor anybodyelse.There are severalreasonsfor this, but thereare also reasonsto considerthe experimentalparadigmand somerelevantexamplesto seewhat utility it might have.This chapterwill concentrate first on the positive benefits of the experimentalparadigmand later on someof the problemsit posesfor researchon languagelearning inside and outsideof classrooms,and for the teacherresearcherwho is the probable user of this book. It will not attempt to give a detailed step-by-step guide to experimentaldesign, since there are many texts which discuss such matters to whatever degree of sophistication is required (Brown, 1988; Hatch and Farhady, 1982; Guilford and Fruchter, 1973); but it will discuss several of the most important principles, review the arguments aboutestablishmentof causalityfrom Chapter3, discussfeaturesof design such as controls and counterbalances, and briefly analysesome published examplesof experimentson classroomlearningissues.

Why experiments? Conductingan experiment

An experimentis 'a controlledlook at nature.The experimentersetsup a task in which the structureof the task is explicit, the natureof the performance being studied is explicit, and the question that is being aske9 is precise' (Paivio and Begg, quoted by Cook, 1986: 13). These themes of control, explicitnessand precisionmean that in certain circumstancesan experiment is an attractive option. Data-gatheringitself need not involve a long, protracted period of time; the relationshipbetweenthe data setsgathered,their numericalanalysis,and the researchquestionis decidedin advance;the interpretationof the resultsdependson featuresof the experimentwhich are also decidedin advance.So, if a teacherwants answersto precise-enoughquestions and has the power to manipulate the learning context to design an

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

156

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

appropriateexperiment,there is sensein doing so. Later in the chapterwe shall look at two examplesof classroomexperiment,one on the distribution of question types in teacher-frontedand pair-work modesof class interaction, and the other on the effectsof different kinds of feedbackon writing in a secondlanguage.Of course,in practice it is rare for teachersto have the power to manipulatethe contextsand the tasksto the extentthat true experimentation demands.Outside researchersmay be given such power if the needfor the researchis perceivedas great enoughby thosein authority; but usually a teachermust live in the context after the researchis done and classesare returned to normal, whereasan outside researcherdoes not. Hitchcock and Hughes'(1995: 40) remarksabout'smashand grab ethnography' apply evenmore strongly to experimentalapproaches,wherethe 'smash' element- disruptionof normal classes,regrouping,matchingof studentsby extra testing, and so on - is probably more thoroughgoingthan in ethnography.A teacherconsideringexperimentationwould be wise to ponderthe longer-termwisdom that 'you can'tmakean omelettewithout breakingeggs'. Theseissueswere laid out for educationalresearchersin the first Handbook of Researchon Teaching(Gage, 1963) by Campbell and Stanley, who introducedthe distinction betweentrue experimentsand quasi-experiments.It is probablytrue to say that most studiesclaiming the rigour of experimentation in languagelearning are in fact quasi-experiments.There are usually too many variablesfor them all to be effectively controlled.The sameis also often true of apparently well-controlled psycholinguisticexperimentsin second languageacquisition(seeGassand Schachter,1989) where task performance is often determinedvery preciselyby sophisticatedtheoreticallinguistic considerations,but the actualexposureto the secondlanguageof the participants or subjectscannot be known more than impressionisticallyor in terms of length. However, a quasi-experiment,relaxing someaspectsof control, may still yield valuableinformation and enablea teacherto answersomespecific questionarising from his or her experience.

Understandingother research Brown (1988), in particular, has argued that, while it is unlikely that teacherswill want or be able to get involved in experimentation(exceptas subjectsthemselves),a knowledgeof how experimentsare designedand conductedgives them accessto valuable literature which is difficult without such prior knowledge.Whetherthey want to havesuchaccessdepends on deeperconsiderationsof the validity and utility of knowledge gained through the approach.Accordingly, an acquaintancewith the procedures and design features of experiments,and of the usual format of experimental reports,arms the teacherwith somecritical faculties for evaluating such researchand empowersthem to go beyond an uncritical acceptance of the conclusions,or more likely to take the conclusionseriouslybecause the stepsin the argumentationare clearer.

DOING EXPERIMENTS

157

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

A languagefor talking about design Experimentalmethod has evolved a jargon of its own, which can be used outside its strict boundaries.The different kinds of variable considered, the abstractionsof canonical versionsof different designs,the terms for different aspects of control, have their uses in many other kinds of researchand may serve to clarify the researchquestionand the meansof conducting the researchin other traditions as well. The literature on experimentaldesigndealsmainly in abstractprinciplesand offers a way of looking at researchdesignat one removefrom the detailsof particularprojects,allowing strengthsand weaknesses to be spotted.

Causalityand the methodof detail By and large, experimentalmethod searchesfor effects of certain treatments on given measures.Van Lier's (1989) criticisms of this model for researchon languagelearning and teachingwere given in Chapter3: that (a) teachingdoes not causelearning and (b) any causalchain in language learning is inevitably long and complex, involving many conditions. Secondlanguagelearning is not the only field where such argumentsare increasingly held to be valid: other areasof educationhave moved away from simplistic accounts of achievementbeing influenced directly by treatments,and linguistics has never embracedthe experimentalmethod despitebeing an empirical science.In someareas,for example,computer modelling has beenfound to be a more powerful tool for analysis,and this is perhapstrue in our own field, thoughit has yet to be demonstrated. However,any teacherwho noticesthat certainconsequences follow from certain ways of dealing with a classis entitled to considerdoing an experiment to investigateit. An exampleis the suggestionthat certain question types stimulate and others depresslevels of student participationin the lesson.A teachernoticed (in anotherteacher'sclass) that open questions directed at the whole classseemedto produceless discussionthan closed questionsdirectedat specific groupsof students- this in a classwith several groups of students sitting around tables. Whole-class questions appearedto underminethe purposeof the groupings,and the classdid not respond,whereasquestionsto a subgroupseemedto have the effect of getting each subgroup to work together. This observationstimulated a hunch that therewas a researchableissuehere,and it could be one which a fairly simple experimentcould answer.In this case,Van Lier's generalcriticism is irrelevant, becausethe teacherconcernedwanted to researchthe effect of two kinds of proceduralsignals from the teacheron the level of interaction.It would be a further, and difficult, step to argue that level of interaction affected individual learning outcomes. Nevertheless, this would be an interestingand topical question. It follows that, in order to find precise questions to answer and

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

158

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

constructexplicit tasksand explicit controlsto makesurethat the answeris correctly relatedto the question,the issuehas to be decomposedinto small parts. This is sometimescalled 'the method of detail'. To follow up the question discussedabove, a teacherresearchercould wait a long time for other instancesof the discrepancyshenoticed to arise naturally. An experimental approachwould seek to establishunder precisely what conditions the phenomenonoccurred, and set up a classroomsituation to compare various question types, intended audiencesand namedaddressees,group sizes, languagelearning tasks, perhapsproficiency levels, group composition, and so on. In a (somewhatfar-fetched)sense,the experimentattempts to recreatein a controlled and reliable wayan interesting phenomenon which was observedby chancein the natural classroom,in order to study the effects more closely. The issue and the situation which give rise to it would be decomposedto find the crucial variables in play and how they interacted.To do this, the researcherwould have to look at all the features of the situationand decideif they were relevant,that is if they could plausibly explain the observationand thereforebe a rival for the original idea that questiontype was the determiningfactor. Perhapsthe contentof the questions directedat the whole classand at the groupsdiffered; perhapsthe way in which the studentssat gave them no senseof interactionacrossgroups; perhapsthe co-actinggroup allowed individuals to hide and reducedtheir anxiety levels but whole-class discussion was perceived as too risky; perhapsthe phenomenonwas real but restricted to one cultural group. Decompositionof a problem into details is thus the basisfor manipulating comparisonswhich isolatecause-and-effect relationshipsand remove,as far as possible,rival interpretationsof the results. Isolation of such details and reduction of the researchquestion to one crucial issue which this 'controlled look at nature' exposes are also designedto removethe issuefrom the immediatecontextand investigateit as a general problem. Experimentationtherefore uses tight controls to establish'externalvalidity' or more simply generalizability.Other meansof establishingthis include replicationand testingin other contexts.

Experimentsand quasi-experiments Basicfeaturesofexperiments Experiments are designed to answer precise questions. An experiment thereforeneeds: 1. a precisequestion 2. an explicit set of variables 3. an explicit task which can be measured(at whateverlevel of measurement)

DOING EXPERIMENTS

159

4. sufficient participantsto enablestatisticalinferencesto be made 5. controlsand counterbalances to eliminaterival interpretations 6. a meansof ensuringthe relevanceof the resultsto the question.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Let us look briefly at thesesix features. 1. The precisequestionis usually in the form of a prediction. For our putative exampleabove it might be: 'Whole-classquestionswill stimulate less interaction than questionsaddressedto subgroupswith studentsof x proficiency doing y type of languagelearning task in monolingual classrooms.' In reality, given the requirementof falsifiability associatedwith much positivistic research,the hypothesisis more likely to be phrasedin 'null' form, which the experimenttries to reject: 'There will be no differencein interactionlevels following whole-classquestionsand questionsaddressed to groups,with studentsof x proficiency doingy type of languagelearning task in monolingualclassrooms. ' If the experimentfound there was a difference,this hypothesiswould be rejected.The direction of the difference (favouring one or other of the treatments)would then be evaluated. 2. Variablesusually comein four kinds: a. Moderatorvariables(MV): usually a differencebetweenthe participants which may systematicallyor unsystematicallyaffect the results. In a questionnairestudy, interest might focus on the way two different kinds of pupil, e.g. girls and boys, answeredthe questions.Here, sex would be a moderatorvariable. b. Independentvariables(IV): the crucial variable being manipulated which the hypothesis concerns.Argument about experimental techniquesoften concernsthe validity of the relationshipbetween the IV and the hypothesis. c. Dependentvariables(DV): theseare the measuresof the effect of the manipulationof the IV. In our example,the interactionlevel presumablysome measureof amount of talk, number of student turns of speech,and so on. d. Confounding variables (CV): often it is difficult to separateout variables from one anothersatisfactorily. In the presentcase, the researcherwould have a difficult time sorting out the difference between a groupwork task and a groupwork seating position. These two variables, which might be important, are so far confounded. Of course, confounding variables are undesirablefeatures of experiments. 3. The measurabletask performancerefers to both independentand dependentvariables:both have to be preciselydefined and clearly identifiable.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

160

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

4. Sufficient participants. In the case which is serving here as an example,therewould haveto be sufficient participantsto ensurea meaningful difference betweenthe whole class and the subgroups:three or four subgroupsof four each,giving 12, 16 or 20 studentsin the class.A further considerationwould be if the comparisonwas going to be betweenthe same studentsat different times (a correlatedsampledesign)or betweentwo different groups(an independentgroups design).The former would require fewer people and be more sensitive,but might be subject to problemsof order, becausethe order in which the different kinds of questionswere askedmight be significant; and the latter would require more peopleand be less sensitive,but not be subjectto order effects.Yet anotherconsideration affecting numbersand samplesize might be the representativeness of the peopleinvolved of somelarger grouping thatwas thoughtto be significant. The participantsin experimentationare usually called 'subjects'. 5. Controls and counterbalances. We havealreadyseensomeof thesein the discussionso far. One popular way of controlling for systematicbiases amongthe peopleparticipatingis to assignthem to the different treatment groups at random. Thus, in our experiment on teachers'questioning behaviour,we might have chosento set up two independentgroups and make up the groupsfrom the availablesubjectsrandomly, in order to have an equal chancein eachgroup of peoplewho were, for example,good and bad talkers, friends with the others, skilled or unskilled in social interaction,. favourably or unfavourably disposed to interaction, extrovert or introvert, or any of a whole host of possible variables.Another form of control would involve matchingthe studentsas closely as possibleby having them do tests of proficiency level, attitude, personalitytype, aptitude and so forth. Effects of order of presentationof the experimentaltreatas discussedabove. mentswould haveto be counterbalanced The consequenceof such measuresis to reduce possible rival interpretationsby controlling their effects,and, manifestly, to increasethe artificiality of the situation at least as comparedto the normal context of the learners'and teachers'everydayexperience.The greaterthe influence of such design measures,the nearer the researchis to a true experiment; quasi-experiments, which are more usualin real-world situationsoutsidea psycholinguisticlaboratory,haveto sacrifice somecontrolsbecauseof realworld constraints. 6. Sinceexperimentsaim to answerspecific questionsby rejecting specific hypotheses,anything that compromisesthe relationship between. question and conclusion is a threat to internal validity. Internal validity may be seenin termsof severallinks: a. betweenspecific hypothesisand researchquestion(is the prediction justified?)

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

DOING EXPERIMENTS

161

b. between the measurementsproduced by the data-collection instrumentsand the hypothesis(are they really the right measures?) c. between the result and confounding variables (can rival interpretationsbe eliminated?) d. between the result and the original hypothesis(does the result makesensein termsof the hypothesis?).

Someexamples In this section,we shall briefly review three studiesof aspectsof Englishlanguageclassroomsin the experimentaltradition, to illustrate the kind of questionsattackedand the methodsused. None of thesestudieswas carried out by the teachersresponsiblefor the classes;they are thereforenot to be seenas direct models for teacher-ledresearch.They are mentioned here to demonstratethe natureand problemsof doing experimentsin this field, as discussedearlier in more abstractterms. Becausethe degreeof control over extraneousvariables is very different in all of them (only one used a randomizationprocedure)they should be classedas quasiexperimentsratherthan true ones. They are instructive examples,and some lessonsto be learned from them for the natureof successfulexperimentationare drawn from them in the following section. They may also serve as pointers for how teachers might go about designingexperimentsfor their own researchquestions. Experimentationcould be used, for example,for the teacherquestioning issue mentioned above; for investigation of the motivating effects of authentic and contrived material; for a study of the teacher'soptimum handlingof latecomersin the class;and a whole host of manageablequestions. Theseparticular exampleswere amongthose suggestedby a group of EFL teachersin a researchmethodsclass. An experimenton methodologyjiJr teachingreading

Carrell et al. (1989) conducteda study comparingthe effectivenessof two kinds of readingstrategytraining. Independentvariableswere a technique called 'semanticmapping',in which studentslearnedto draw and then redraw diagramsbasedon their knowledgeof the topic and what is said about the topic in the text, and another called 'experience-text-relationship' (E-T-R) which involved structureddiscussionsessionswith the teacher. The control group took the regularclassesin the programme.The dependent variableswere gain scores,that is the differencebetweenscoreson a specially designedreadingcomprehensiontest before and after the strategy training. The moderatorvariables included a questionnairegiving a

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

162

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

picture of the individual differencesamongthe students,including strategy preferenceand cognitive processing.The studentswere adult ESL studentsin regularclassesat approximatelythe sameproficiency level in a university ESL centre in the USA, and were of mixed national and linguistic backgroundsand ages.The experimenterdid not randomly assign studentsto treatments.The timescaleof the experimentwas nine days: a day for pre-testing,four days of strategytraining, and later a final day of post-testing. The structureof the experimentwas as follows: Group 1 Group 2 Group 3 Group 4

=9 pre-test n =9 pre-test n =3 pre-test n =5 pre-test n

semanticmapping

post-test

E-T-R

post-test

control

post-test

control

post-test

Both types of strategy were taught by the same teacher and another teachertaught the control groups. Unfortunately the semanticmapping group was disruptedon their fourth training day by having to evacuatethe building becauseof a bomb scare. The results showed that there were statistically significant advantages for the strategytraining treatmentsover the controls on open-endedcomprehensionquestionsbut not on multiple-choicequestions,and that these differences were strongly influenced by individual student differences. When the resultswere analysedusing the moderatorvariables,they found that 'deep'cognitive processorsin the control groupsdid make significant gains on the tests, whereasthe strategytraining worked for both 'shallow' and 'deep' cognitive processorsamong the students. Presumablydeep processorscan makethesegains independently,without strategytraining. This relatively simple and timeboundquasi-experimentalstudy showed that such issues are amenable to experimental attack, but the results demonstratea typical featureof all research:a numberof tantalizingquestions are raisedwhich can only be answeredby further researchwith modifications to the design. For example,why did the advantagesonly show up on somemeasures?Why did the 'deep'processorsapparentlynot needthe special training? If they could do without the training, why did they not perform much betteron the first test and not makea gain? A classroominteraction experiment

In the late 1980sthere was a flurry of activity, sparkedoff by the increasing popularity of communicativemethodology,which was directed at the advantagesof group and pair work over traditional teacher-fronted'lock-

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

DOING EXPERIMENTS

163

step' teaching.One quasi-experimentalstudy was conductedby Doughty and Pica (1986) to investigatethe nature of classroominteraction in the new languageusing an information gap task in three kinds of organization. In this study, the IV was class organization: teacher-fronted,groups (of four students),and pairs. Studentsattendingregularclassesin a university ESL programme were assigned randomly to the three classes. In the teacher-frontedclass, the teacherread out the instructionsand stayed to run the class and complete the task; in the group and pair-work classes, the teachersgave out written instructionsand left. The DVs were amount of interaction (measuredby the numberof turns taken by all the participants) and quality of interaction (the proportionsof interactionalmodification such as comprehensionand clarification checksand confirmations) and accuracyon task. No MVs were measured,it being consideredthat randomassignmentwould adequatelycounterbalancethe inevitabledifferencesin attitudes,experienceof the treatments,friendship groupingsto be found in a group of students.Proficiency was counterbalanced by drawing the sampleof learnersfrom regular classesat one particular level. There was no control group as such. The task set all the classeswas to plant a gardenusing a feltboard and a numberof icons of gardenfeaturessuch as flowers, flower beds,lawn, patio and gardenequipment,accordingto some instructions which required some interactional decision-making. The hypothesiswas that interactionwould differ in amountand quality according to the classorganizationadopted. The results showed that both amount and quality of interaction did indeed differ betweenthe three treatmentgroups. Briefly, there was more interaction in the teacher-frontedclass than in the pairs, and more in the pairs than in the groups. The reasonsfor this were open to speculation: perhaps the teachers' directive function stimulated more interaction through nomination and invitation, whereasin the groups it is possible that somestudentswere able to 'hide' and leaveit to the others.Therewas, however, more modification through comprehensionand clarification checksin the groups and pairs than in the teacher-frontedclass. Perhaps the studentsdid not want to challengethe teacherby frequent checking, or perhapsthey did not feel they neededto becausethe teacher'sinput was anyway clearer and more grammatical.Oddly, the accuracyof the task of planting the gardenwas lower in the teacher-frontedclassroomcompared to the others:perhapsthe studentsonly pretendedto understand! This study illustrates the neatnessof an experimentalapproach,but its inability to support a clear interpretationshows that there are too many issues, too many confounding variables, which remained uncontrolled. Perhapsmore kinds of data would have helped, for examplelesson transcripts or interviews with the participants.This study was, however, only one in a seriesby theseauthorsand other co-workerson related problems. This question perhapsaffords an opportunity to combine experimental

164

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

measurement with naturalisticdescriptionin the interestsof greaterprecision. (This is discussedfurther in Chapter14.)

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Feedbackon writing The third example to be discussedis a quasi-experimentalstudy of the effects of different kinds of feedbackon teachercommentson EAP st11dents' writing. In this study by Fathmanand Whalley (1990), the IV was type of feedback:no feedback,feedbackon grammar(locating an error by underlining), on content (minimal and rather unspecific encouraging remarksand generalizedcriticism) and both. The DVs were numbersof grammatical errors and holistic content rating. The learners were in a selectionof six intermediatecollegecompositionclasses,72 studentsin all, from two similar colleges, from mixed languagebackgrounds.The task, presentedto the learnersin the sameway by their own classteachers,consistedof a set of eight pictureswhich representeda narrative,which was talked through by the teachers.The hypothesiswas that writing should improve accordingto the kinds of feedbackreceived.The studentswere given 30 minutesto write the narrativeindividually. The scripts were collected and assignedat random to one of the four treatmentgroups; the feedback was written on the scripts. The scripts were returned a few days later and the studentsrequired to rewrite the composition.Both originals and rewrites were then graded for grammar errors and quality of contentby two independentraters.The report does not mention if the raters knew whether they were rating originals or rewritesor had sight of the feedbackcomments;one presumesnot. The results of this study were analysedin two ways: by averagingthe scoreswithin the feedbacktreatmentgroups, and by counting the numbersof studentsimproving betweenoriginal and rewrite. The first analysis showed that significant improvementin grammaticalaccuracyoccurred with grammaticalfeedback(with and without contentfeedback),but content improvement occurred in all the treatments, including the nofeedbackgroup, althoughat a slightly higher level when contentfeedback was given. The secondanalysisshowedthat the grammarfeedbackhad a more powerful effect on accuracythan content feedbackhad on content. The hypothesiswas, then,only partially confirmed. As with the other examples, this study is subject to a number of methodologicalcriticisms. It is a relatively small-scalequasi-experiment, despiteinvolving six classesin two colleges,because: • only one compositiontopic was used,so there is no way of knowing if topic or type of writing assignmentmight affect the issue • only one rewrite, or 'fair copy' was required, so there is no information on the persistenceof the improvementsclaimed • only one form of feedbackin eachareawas tried.

DOING EXPERIMENTS

165

Reflections on the experimental approach

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

This last sectionwill recapon a numberof issuesconcerningdoing experimentsin generaland the useof experimentationin teacher-ledresearchon languagelearningin particular. Reportingexperiments

One advantageof this approachis its history and conventionsof reporting. Since it has been used for a long time in many contexts,a great deal is known and published about the power and shortcomingsof different designs. One result of this has been the almost universal adoption of a convention of reporting designed to ensure that all the features of the experimentnecessaryfor replicationare mentioned: • • • • • • • • • •

researchquestion literaturereview hypothesis experimentaltask designand method controlsand counterbalances subjects methodof analysis resultsand statisticalanalysis discussionand interpretation.

We have already discussed,in Chapter4, the problemsof replication in languagelearningresearch.However, the frequencyof the format (for some journalsit is a condition of acceptance)meansthat for the initiated it is relatively comprehensible;but like all suchconventions,it is not easyto learn. Externalvalidity,

The feature of generalizability,or external validity, referred to in Chapter 4 is a prized attribute of good experimentation.The fact that this method can attack issuesisolatedfrom other effectsand contaminationsfrom contextual details enablesconclusionsabout those issuesto be made in the form of generalstatements.As we have seen,however,such generalstatements lose their force when there are problems in interpreting exactly what the experimentaldata mean and how strong the support for those statementsreally is from the data. Decontextualization

Much of the apparatusof experimentalcontrols and counterbalancesis directed at the isolation of issues from context, for reasonsof specificity,

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

166

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

generalizationand precision. However, it is obvious from the three illustrations given that decontextualizationalso leads to disruption of normal patterns of classtimetables,activities and teachingpatterns.The 'experimental' treatmentsdiscussedalso were by definition different from what the classes were normally receiving as tuition: in theseexamplesno attemptwas made, or at least reported, to integrate the methods under study into normal classes.It is reasonableto ask, first, if thereis any effect resultingfrom a student knowing they are doing somethingout of the ordinary or special(usually known as the 'Hawthorne' effect) and, second,if the results of such experimentsso carefully decontextualizedcan be generalizedto learning in 'normal' classesor tell us very much about the fate of the innovationsproposedwhen they enterregularteachingprogrammes.The artificiality of this 'controlledlook at nature'may in the end be self-defeating. Time

It will be evident that the experimentalmethodcannotseriouslybe usedas a relatively quick, albeit well-prepared,economicalapproach,sincein each of the illustrations the results only partially confirmed or rejected the hypotheses.The issues addressedtherefore required second and third attemptswith designand materialsmodificationsto actually be settled:any researchis an ongoingprocess.It is probablynot fair, therefore,to contrast the time economy of experiments favourably with the fairly lengthy processof review, replan, reconnaissance and recycle of action research: the advantageis illusory. Randomization

It is worth noting that random assignmentof subjectsto treatments,one of the biggestdifficulties in using an experimentalapproachin a regular educa~ tional context, does not actually have to disrupt normal classesin all cases. Fathmanand Whalley'sexperiment(1990) preservedthe integrity of regular classesin two collegesbut randomizedthe assignmentof the written products to the markers,the postagebut not the people,as it were. Normally, random distribution implies fairly massive interruption to normal timetables and programmearrangementswhich itself may bring serious educational disadvantages and prove a costly item in relation to the benefit to the profession of the results.Any administratoror gatekeeperresponsiblefor allowing accessto the learners for the researcher(s)may well say 'It had better be worth it'. However, it is clear that for somepurposes,with ingenuity, disruption can be minimized. An alternative strategy, frequently used in quasiexperiments,is to use existing classesanyway, and attempt to control the various confoundingvariablesby measurement.This is the strategyused by Carrell et al. (1989), and in that example it is defensible.It is remarkable,

DOING EXPERIMENTS

167

however,that althoughthe regularclasscompositionwas not rearranged,the normal pattern of teachingwas, with one teacherdoing both experimental treatmentsand anotherthe normal programme.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Scale

Clearly all three of theseillustrative experimentsconcernedlarger numbers of studentsthan an individual teacherin a class has responsibility for. The scaleof a conventionalnotion of experimentis usually larger than individual teacherresearcherscan manage.As experiments,theseillustrations could in fact be criticized for not being on a large-enoughscaleto incorporateall the design featureswhich control and objectivity might lead us to desire (we have already seenthat, in all three cases,the conclusionswould have been strongerif more time had beengiven, further comparisonsmade,or larger numbershad beenavailablein one or other treatmentgroups).However,by chance all these illustrations used independentgroup designs, which is expensiveon numbersbecausethereare different peoplein eachgroup. It is also possibleto give the studentsthe different treatmentsat different times, and use a correlatedgroups design. In this kind of design, learnersact as their own controlsbecausetheir performanceis measuredunder the two (or more) treatments.Sucha designmight in somecasesbe more suitable,especially where scaleis a major factor. It is also possibleto solve the scalequestion by using collaborativemethods,where a group of teacherswork on the sameproblemat the sametime. Numericalanalysis

In each of theseexamples,the results were expressedas numbers:scores on reading tests, frequenciesof interaction and linguistic features,numbers of errors or scaleof quality, and the figures were analysedstatistically in more or lesssophisticatedways. The Carrell et al. (1989) study looked at the significanceof the size of the differencesbetweenpre- and post-tests in each treatmentgroup, and then used a more powerful technique,analysis of variance,to look at the differencestaking into accountthe various individual differencesthey had measured.No accountwas taken, in these examples,of qualitative data such as learner or teacherreflection, diary notes,observationsor interviews. Disclosureand ethics

In none of the articles which report the aboveexamplesis there any mention of whether the learnerswho provided the data were given explanations of what the experimentswere about. This is a somewhatdifficult

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

168

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

ethical question, since, on the one hand, learnersmay be consideredto have a right to know what a researcheris doing with them for the sakeof researchand the advancementof knowledge,but, on the other, knowing what the purposeof the experimentthey are subjectsin, not obviously by their own volition, could bias the results. It would have beendifficult for the subjectsin the three illustrations not to know they were in experiments, since unusual things were happening:assignmentto ·new groupings, extra tests,a specialform of feedbackor noneat all. Considerin particular the control groupsin Carrell et al.'s (1989) study. Their only participation in the study was to have to take the same tests twice, the content of which bore no resemblanceto the 'normal' teaching they were receiving. No wonder they did not improve - it is interestingto speculatewhat their attitudeto the extra testingmight havebeen.Erickson's remark(1986) that a billiard ball cannotmakesenseof its surroundingsbut a person can is relevant here: the 'subjects'in an experimentare trying to makesenseof this slightly odd temporaryworld, and it is probablethat they will come to quite different conclusions,which may affect the results.That is the meaningof the 'Hawthorne'effect, which basicallytells us that experimentsin schoolstend to confirm their hypothesesbecausethe learnersgiven the new experimentaltreatmentsbecomemore interestedthan thosegiven the traditional treatmentsand work harderto justify their specialstatus.Of course,it is perfectly possibleto obtain agreementand voluntary participation on the basisthat the full story will be revealedafterwards,but cannotbe beforehandpreciselybecauseof the dangerof bias.

Third-personresearch Kemmis (1991) draws an interesting distinction between third-person research,which is 'positivistic' and strives for objectivity, second-person research,which is 'interpretive' and acknowledgessubjectivity, and firstpersonresearch,which is 'critical' and identifies the researcherwith th'! respondentsand the providersof the data. He is very scathingaboutthirdpersonresearchbasedon statisticalmeasurements and the use of learning achievementoutcomes.Clearly, most experimentalapproachesin language learning fall into this category,and any intending researcherwill want to decide what the purposeof doing researchis and what kind of research they want to do before opting for any particular approach. Experimentation is primarily about objectivity, the suppression of researcherand other personalbias, the isolation of issues,precision and measurement.If theseare neededor considereddesirable,then there are many well-worked-outdesignsavailable.

Conclusion The term 'experiment'is sometimesused rather loosely, with a meaning approximatingto 'trying out' - whetherthat be an innovation, a different

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

DOING EXPERIMENTS

169

teachingmethod or technique,a new coursebookand the like. We might talk, say, of 'experimenting'with a different kind of grouping in the class, to seewhetherwe get different learningoutcomes. This chapterhasset out the most importantcharacteristicsfor the more rigorous designof true experiments.It has also pointed up the distinction betweenexperimentsand quasi-experiments, suggestingthat the latter are more appropriatefor languageteachingand learning research- or indeed in any field of social life where variablescannotso readily be isolated,and where behaviour and outcomesare relatively unpredictable.Beyond the actual details of design,the chapterhas also commentedon the advantages and disadvantagesof experimentalmethod for use in the languageclassroom, relating them to such broader issues as validity, generalizability, contextand ethicsthat underpinmuch of the presentbook.

Discussionnotes 1. Bearing in mind the discussionof design featuresof experiments, take the issue of teacherquestioningbehaviourand sketch out a plan for an experimentto investigateit. 2. Can you imagine ways of controlling (in the senseof the word in this chapter) the variety of influences on the learners in your classroomso that you could isolate one variable and experiment with it? 3. Do you consider that any of the examplesof experimentsmentioned producedknowledgethat you could useas relevantfor your own learners?

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

This page intentionally left blank

11

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Asking questions

Introduction This chapteris devotedto the central topic of asking peoplequestionsin order to obtain researchdata. Although there are very obvious differences betweenthe two general categories- questionnairesare associatedwith survey work and interviews with individual respondents- there are also many points of congruencewhich derive from the simple fact that both are rather specializedforms of conversation.In what follows we shall look at questionnaires~nd interviews in turn, bearing in mind that much of the discussionof one is applicable to both. Taking questionnairesfirst, we shall explore Low's (1991) illuminating distinction betweenquestionnaires as conversationand as tests;look at the rangeof typesof questionnaireuse in ELT research;explain some general guidelines for constructingquestions; discuss some popular methods for analysis and give some recent examples. Moving on to interviews, we investigate the main kinds of researchinterview and their different characteristics,and then draw out somegeneralpoints.

Popularity oj questionnaires Questionnaireresearch seems to be very popular among educational researchersin general(especially in the survey context - see Cohen and Manion, 1989: 97) and ELT research in particular. This is because, although it is quite labour-intensivein construction and analysis, the researchercan benefit from severaladvantages. • The knowledgeneededis controlled by the questions,thereforeit affords a good deal of precisionand clarity. • Questionnairescan be used on a small scale, in-house,and on a large scale, requiring little more extra effort than photocopying and postage(though for large-scalesurvey work this can consume a significant budget,especiallywhen using stampedself-addressed envelopesfor returns).

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

172

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

• Data can be gatheredin severaldifferent time slots: all at once in a class,in the respondents'own time as long as it is easyto return, at conveniencewhen a suitablerespondenthappensto comealong,and in different locationsat different times; but in all of thesethe datais comparable,the questionsare the sameand the format is identical. • Self-completionquestionnairesallow accessto outsidecontextsso information can be gatheredfrom colleaguesin other schoolsand evenother countries.

Sensitivityofinterviews Interviewing is also a populartechniquebut for different reasons.It lacks, obviously, the administrativeconveniencesand in many forms the standardizationof questionnairework. It featuresas the preferredresearchtool in many investigationsbecause(dependingon the format): • it is (usually) one-ta-one,and thereforesensitiveto individual differencesand nuancesof emphasisand tone even if a standardized structuredlist of questionsis the content • it has a potential for opennessand allows control of what is revealedto remain moreor less with the respondent,giving room for individual expressionand broachingof new topics • it is more like ordinary conversationand thereforeto someextent a more personalcontextfor information exchange • comprehensionchecks,hedges,and assurancesof confidentiality can be requestedand given at any point during the interview in order to maximizethe usefulnessof the data • the form of the answer is not (usually) constrainedto ticks and circles, but is ordinarylanguagewith all its freedomand sensitivity.

Q!Iestionnaires Questionnairesas a genre Low (1991) comparesquestionnairesto two other human activities, conversationsand tests,pointing out that eachanalogyexplainssome part of the way people react to questionnairesand holds some lessonsfor the questionnaireconstructor.

As conversations One of the most frequent acts in conversationis asking and answering questions,but in conversationthereis a dynamicof power and trust which concernsthe acceptabilityof certain kinds of questions,the right to ask them, the pressureto provide an answer, and the right to conceal the

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

ASKING QUESTIONS

173

answer.Sinceinterviewsare also conversationsof a kind, using languageto negotiatesuch tensionsis part of the interviewer'sskill, but respondents often have difficulty with questionnairesprecisely becausethere is no, or only inadequate,protectionof their rights. A questionnairemust solicit the respondent'scooperationor it is valueless. Low (1991) continues by discussingthe parallel between conversations with strangersand eliciting information from strangersvia questionnaires. Such conversations,he points out, involve a great deal of hard work, with wary self-revelationon both sides,before important information is entrusted to the other person:no wonderthat impersonalquestionnaires,often through the postbox, with little explanationor personalappeal,areoften not returned. Low further discussesthe 'human-sized'natureof conversationinvolving frequent topic and key term repetition, stories,social talk, tailoring requeststo the conversationpartner,and redundancy.Questionnairesrarely show these featuresand suffer a reductionin their power to elicit good dataas a result. A large amount of questioning also features in another specializedform of conversation: 'People do not like being interrogated' (Low, 1991: 124). Interrogationsfeaturetraps,repetitions,consistencychecksand hints of punishment; questionnairescan as well, but the respondentis in the privileged positionof choosingnot to return the paperif it is perceivedas threatening.

As tests Low (1991) argues that tests are constructed according to particular canonsof validity, reliability, lack of redundancyand repetition, and independenceof items. Designinga questionnairealong those lines produces indeed a familiar kind of documentwith short and simple questions,no irrelevant information, as little as possibleinteraction on paper with the researcher,or often little explanationfor the presenceof certain questions, few verbal links, and little structureeven in terms of divisions into thematic sections.It should not be possiblefor the answerto one questionto be dependenton the respondentbeing able to answeranotherquestion,nor for one questionto suggestthe answerto another.Questionnairesare often evaluatedlike tests- the sametechnologyfor automaticscorereadingcan be used,the samestatisticalmethodsmay be invoked. The central issueexploredby Low, and the reasonit has beenexamined in somedetail here, is the questionof the relevant and proper criteria for good quality data out of a questionnaire.There are choices, sometimes quite difficult ones,to be madebetweenwhat the designermay ask with a reasonablehope of a true and cooperativeresponse,and what might be most valid and reliable but so off-putting that few responsesare yielded; betweenuser-friendlinessand accuracy; betweenrevelation and explanation and minimizing hints about preferred answersand responsebias; betweenlength and conciseness.

174

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Typesofquestionnaire Q!Iestionnairescome in all shapesand sizes.The optimum length of a questionnaire is governedby the expectedyield: the more work respondentshave to do the less likely, given a free choice, they are to return it. However, if the topic (or the researcher!)is important to them, long questionnairesmay be completedand returnedwithout complaint.There is a rule of thumb, with no theoretical researchbacking, that a questionnaireextending over two sheetsof papermay be too long. However,thereis greatvariability depending on purposeand expectedyield. Studentcourse-evaluationquestionnairesat the authors'institution are one pageincluding invitation, explanation,questions and machine-readable answerblocks; a recent internationalsurvey by the British Council for their e2000project was in booklet form. Questionnairesmay contain only one kind of question or a mix. It is usual for the majority of questionsto be answeredby ticking a box or circling an alternative to enable easier counting. For very large samples, machine-readingfacilities may be required. In any case,respondentshave to be given clear instructionson how to answer.If this is not done, either they will not answer,or the researchercannotbelievethe answersgiven. What to include will dependon: • the kind of information needed • the kind of analysisproposed. In general,the questionwill be of the following types.

Factual questions Theseare often usedfor 'moderator'variablesto tap divisions in the population being polled. Theseare usually multiple-choice,but not exclusively, suchas Age

under IS 15-19 20-24 25 and over etc.

Proficiencylevel Beginner Post-beginner Pre-intermediate Intermediateetc. Sex Male Female

ASKING QUESTIONS

175

Yes/no

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

These are often informative but blunt. Very often a respondentwill be uncomfortablewhen unableto hedgewith 'it depends'or 'sometimes'.

Multiple-choice Theseare the usual way of askingquestionswhich in an interview or conversation would be prefaced by Who?, What?, Which?, Where?, Why?, How?, and so on - the Wh-questions.Given a set of choices,the respondent can indicate more divisions in the information: but the questionnaire designershould consider if only one responseis allowed or several. For example: Which teachingmodedid you like? small group pairs whole class discussion lecture video presentation languagelaboratory Patently, these alternatives are not mutually exclusive, so several selfconsistentresponsepatternsare possible. The assumptionthat respondents only liked one is anyway probably unwarranted;the questioncould be re-phrasedas: Which teachingmodedid you like best? or Which teachingmodesdid you like (choose4)? In either case,the questionis constrainingthe choicesof the respondents and possibly compromisingthe quality of the data.The similarity of both yes/noand multiple-choicequestionsto test items is also obvious.

Rankedquestions Another way of asking Wh-questionsis to ask respondentsto rank the alternativeswith an instruction like: Pleaseindicate your order of preferencewith a number between and 7 where 1 =most favouredand 7 =leastfavoured.

176

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

A frequentpitfall with such instructionsis that somerespondentsseemto do it the other way round, thus again compromisingthe data. Prima facie, there is no way of choosingbetweencounting up and counting down. If several ranked questionsare used, they should be consistentin rubric, otherwisethe questiontype will confusethe respondent.

Open-endedquestions Many questionnairesinclude open-endedquestionsto allow the respondents to feel that they can contribute more individual points of view and more detailed information than is elicited in closed questions.To have a majority of such questionslooks like an invitation to write a number of essays,and is likely to reduce yield. However, in certain circumstances, this may not be a disadvantage.Powell (1992) reports a study (to be discussedbriefly below) in which languageteacherswere polled by a seriesof open-endedquestionson their involvementin a curriculum development project. Open-endedquestionsdo presentdata-analysisproblems which may require more sophisticatedtreatmentthan counting yes answersand no answers:contentanalysis,for example,as discussedat the end of this chapter.

Scaledquestions One familiar type, the Likert scale,presentsnot questionsbut statements and asksfor degreesof agreement.Theseare often used to elicit opinions ratherthan facts and are sometimescalled 'opinionaires'.For example: A teachershould wait longer for an answerfrom a poor studentthan from a good student. stronglydisagree disagree no opInion

agree

stronglyagree

Likert scalesmay have three, five or more choices.Their advantageis that shadesof opinion may be given numericalvalues:it might be significant if large numbersof respondentscould only bring themselvesto agreewith a particular statement,showing weak support, but strongly agreed with others.Unfortunatelythey also have two disadvantages.First, the midpoint is often difficult to interpret(no opinion becausethe questionis not relevant or becausethe respondentis not interested?)and second,peoplevary in degrees of caution, and one person's agreementmay be another person'sstrong agreement.In quantitative analysis, the five alternatives may be reducedto three anyway in order to have enoughpeoplein all the cells for the expectedvalues.

ASKING QUESTIONS

177

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Gullman scaling Another kind of scaling is often used to gaugethe strengthof feeling on some issue. Here, each questionis related to the others rather than being essentiallyindependentinformation, and the answersfrom individuals can be summedover the whole questionnaireto give a measureof strength. This techniquehasbeenused,for example,to measurestrengthof motivation in languagelearning. It has also been used extensivelyin personality studies, which in their turn have been used by language learning researchersto investigate various questionsconcerning the relationship betweenpersonalitytypesand languagelearning.

Constructingquestionnaires It is evident from the above discussionthat a good questionnaireis one which is relatively easy to answer, easy to record and evaluate, userfriendly and unambiguous.The designerhas to choosea mix of question types that will maximize the range anddetail of the information elicited. However, there are some fairly obvious difficulties in constructingquestions which Cohen and Manion (1989: 108-9) succinctly describe(in the context of survey research)as a list of outcomesto avoid. They are summarizedhere:

• leading questionswhich suggestthere is one desirableor desired answer • highbrow questions,using portentouslong words which are liable to be misunderstood • complexquestionswith many subparts • irritating questions or instructions, asking for example for responsesin severalcategoryboxesat once • negativequestions,especiallydoublenegatives. Cohen and Manion do warn against open-endedquestions or selfcompletionquestionnairessince they may make too greata demandon the respondents'time or goodwill, and are possibly difficult to interpret. To this list one needhardly add:

• ambiguousquestions. Questionnairedesignerstry therefore to avoid poor types of question, but often the difficulties with individual questionsdo not becomeapparent until a version of the questionnairehas been piloted. A kind of item analysiscan be performed,looking at: • the consistencyof answers,which might indicate a lack of discrimination in the questions

178

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

• the yield on individual questions,since a question attracting a large number of 'don't knows' may indicate too difficult a choice or an ambiguity • any other commentsavailablefrom the respondents. Giving questionnairesin a secondlanguagemight prove less useful than translatingthe questionsinto the respondents' mother tongue(s);if this is done, however, it is usual to have an independentperson translateback into the original languageas a checkon the comparabilityof the translated versIOn.

Methodsofanalysis Most questionnairesare designed to be evaluatednumerically, and are amenableto fairly simple countingtechniques.Thesecan be done by hand for relatively small runs and can be performed by computer when the numbersgrow large. The first step is to draw up a table with all the questions down the left-handside and all the respondentsalong the top:

A

B

C

D

EF

F

G

H

I

J

K

Ql

Q7

Q3 Q4

Q5 Q6

OJ Q8 Q9 Q10 Each person'sresponsecan then be plotted, giving an overview of all the data. Each data field will have values correspondingto the mode of responsedemandedin the question:yes/no;never/sometimes/often; rank 1-10; and so on. Exactly the sameprocedureis required for entering the data into a computer using for example the Statistical Packagefor the Social Science (SPSS). For each individual question, frequencies and therefore proportions can be easily calculated,and it is often clearer to translatetheseinto percentages.Occasionally,the researcherwill want to

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

ASKING QUESTIONS

179

test the significanceof the proportionscomparedwith chance,in which casechi-squareas discussedin Chapter9 is appropriate. Interpreting a questionnairenumerically normally means evaluating each questionseparately.However, a further and very useful level of analysis is 'cross-tabulation',in which the responsesof subgroupscan be blocked together.It is usual to do this with the moderatorvariables. For example,one of the authorsrecently polled a classconsistingof members of two different courses.Cross-tabulationrevealedthe differencesbetween them on severalquestions,to do with the feasibility of doing researchin their normal teaching situations. Cross-tabulationplots one question againstanother: Q10 Would teacherresearchbe feasiblein your situation? Yes

No

Q3 CourseA CourseB By this means, different kinds of divisions and splits in the block of responsescan be analysed.It doesnot haveto involve only moderatorvariThe power of ables,as there may be significant divisions by other answers. cross-tabulationis thatit can be used(by hand or more easily in the computer) to look at more complicateddivisions, for exampleat three or four questionsat once, so extendingthe above example,there might be some interestin seeingif coursechoice(Q3) interactedwith perceptionsof feasibility (QIO) for those teachersinvolved in evaluation(Q13). In principle, cross-tabulationcan cut swathesthrough a massof responseslooking at intersectionsof any number of questions, but in practice it becomes impossibleto representsatisfactorilyon paperafter four levels (Qby Q by Q by Q) and exceedinglytaxing to conceptualize. Finally, a recurringworry aboutquestionnaire-based researchis the validity of the instruments:do they really describeor measurewhat they are believedto? The only way to decidethe issueis to comparethe resultswith similar information obtainedby other methods,through interviews, diaries, casestudies,verbal reportsor tests.A recentlarge-scaleexampleof such validation is by Gardnerand Macintyre (1993) who attemptedto cross-validate their 'Attitude Motivation Inventory', a collection of 13 questionnaireswith two alternativeways of eliciting comparableinformation. Questionnaire-based researchin languageteaching

There have beenmany examplesreportedin the literature. Here is a small samplefor illustrative purposes.

180

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Studyskills

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Jordan(1993) presentsthe resultsof questionnairesurveyson study skills among overseasnon-native speakerstudentsat ManchesterUniversity over a three-yearperiod. Needsanalysis Weir (1988) investigatedthe linguistic demandsof work in different university departmentsas part of a needsanalysis.The purposewas to determine the kind and range of English-languageskills required of new entrantsand thereforederive a specificationfor a new task-authentictest. In ESP there are many further examplesof needs-analysis questionnaires: see,for example,Mackay and Mountford (1978). Assessment Brindley (1991) discussedthe problemsof assessingachievementin the learner-centredcurriculum of the Australian Migrant English Programme. He used a ranking method and comparedthe perceptionsof teachers,programmeadministratorsand learnersof the various possible reasonsfor testingachievementand ways of doing so. Involvementin curriculum development Powell (1992) used open-endedquestionnairescombined with diary entriesto study the perceptionsand emotionsof 15 teachersinvolved in a regional modern languagecurriculum developmentproject. The teachers were participantsin an in-service professionaldevelopmentprogramme, and were also writing languageteachingmaterials.The responsesreveal fascinatingvariety of detail about the way the teachersreactedto the project, to the problemsthey encounteredin writing materials,and to the various extra activities set up in the programme. Writing skills Casanaveand Hubbard (1992) used a questionnaire(adaptedfrom one publishedin a TOEFL researchreport on the Test of Written English) to investigatewriting skills and writing requirementsof first-year doctoral researchstudents(both native and non-nativespeakers)in a large university in the USA. The questionnaireconsistedmainly of Likert-type questions, ranking questionsand open-endedquestions,and was six pages long. The paper analysed the yield of 85 questionnaires,a respectable

ASKING QUESTIONS

181

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

number,but it is worth noting that they sentout 563, a yield of only 15 per cent. The survey investigateda wide range of issues: writing tasks, the importanceof writing skills, evaluationcriteria, writing problemsof native and non-nativeEnglish speakers,explicitnessof assignments,and a bunch of open-endedquestions.

Metacognitivestrategies Carrell (1989) decided to use a questionnaireto investigate learners' strategiesin readinga text in a foreign languagefor an interestingreason. Previous researchhad used sensitive 'think-aloud' methods, essentially asking peopleto talk about what they were thinking while they read specific texts. Carrell wanted to free her data from the constraintsof reading specific texts and look at more generalissues,to tap into the beliefs, intentions and preferred modes of learning of her respondents.She elicited data from Spanish-speakinglearners of English and English-speaking learnersof Spanish,and requestedresponsesto the questionnaireunder two headings:readingin the first and readingin the secondlanguage.The resultswere then evaluatedby quite a powerful statisticaltechniqueto separateout the variouspreferencesoperatingin the different situations.

Programmeevaluation Questionnairesare often used as a meansof data collection in the evaluation of languageteachingprogrammes.They may be given to students requestingfeedbackon coursedelivery, to institutions as a meansof quality audit, and to staff in pursuit of information aboutcoursedevelopment and staff involvement. Many examplesare given in Weir and Roberts (1994).

Interviews Interviews may be usedas the primary researchtool, or alternativelyin an ancillary role, perhapsas a checkingmechanismto triangulatedata gathered from other sources. In education, uses include large-scalesurveys, ethnographicstudies, 'micro' researchon individuals, and the kind of action-researchperspectiveexplicitly favouredby Hopkins, who lists three applicationsof interviewsin classroomresearch(1993: 125 and 127): • to focus on a specificaspectof classroomlife in detail • teacher-pupildiscussion-+ diagnosticinformation • to improve the classroomclimate. For applied linguistics, Nunan's list (1992b) covers SLA research, language testing of oral proficiency, conversationalanalysis, linguistic

182

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

variation and so on. Teachersmay well sharesomeof thesegoals, though to theseuseswe could also add (for example): • • • •

needsanalysis programmeevaluation individual casestudies mini-surveys(within institution).

Interviews,of course,are just anotherway of askingquestions,this time in face-to-faceinteraction,rather than the questionnaireformat discussed above. In fact the label subsumesa numberof diversemodesthat roughly parallel the different orientationsto researchdiscussedin this book. In this sectionof the chapterwe look at the main types of interview available to suit different purposes,the kinds of datathey generate,and someof the issuesassociatedwith using the technique.

Typesof interview Interviewing is a very basic researchtool in social science,from which the conventionalclassificationlargely derives. Most commonly, interviews are divided into (a) structured, (b) semi-structuredand (c) unstructured, and theseare the terms we shall adopt here. (Terminology is, however,varied: Cohen and Manion, 1989, for instance,use (a) and (c) alongsidethe two further categoriesof 'non-directive'!lnd 'focused'.)What is essentiallybeing representedis a spectrum, not hard-and-fastself-containedcategories, from formal and controlledat one end to more openand lesspredictableat conthe other. Lincoln and Guba seethis as a normative/non-normative tinuum: they gloss the structured interview as when the interviewer 'knows what he or she does not know', and the unstructuredinterview when the interviewer 'doesnot know what he or she doesn'tknow' (1985: 269). It is important to rememberthat the categoriesmerge,and can also be combinedin the sameresearch'event'.

Structuredinterviews Theseare closestto the standardquestionnaireboth in their format and the assumptionson which they are based.In other words, using questions tightly specifiedin advance,they aim to survey relatively large populations by askingthe samequestionsin the sameorder. Many peoplewill havehad the experience of being approachedin the street to answer marketresearchquestions,for example,and on this very large scaleit is necessary to usecohortsof trained interviewers.As we write, the Guardian (6 March 1996) has just reportedon the interviewing bi telephoneof 1200 people concerning the future of the British monarchy. Data analysis, seeking

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

ASKING QUESTIONS

183

representativeness and an overall picture of trends, is predominantly quantitativeand often tabularand diagrammatic. Such interviews do have someadvantagesover the questionnaire.They allow the interviewer to seek clarification in limited ways, and to disambiguate questionsif necessary,and they normally yield a better response rate (I can throw a piece of paper into the bin becauseI feel too busy to deal with it, but it is more difficult to sendsomeoneaway in a face-to-face situation). On the other hand, the anonymity of the questionnairemay meanthat more honestresponsesare given. Cohen and Manion (1989: 312ff) set out the three main types of item usedin structuredinterviews (and herethereare clear parallelswith questionnaires): • fixed alternative:i.e. dichotomousyes/no • open-ended:some responseflexibility possible (e.g. 'What TV programmesdo you watch?') • scale: usually degreesof agreement-disagreement. Structuredinterviews have been widely used in researchon language learning, for example in data collection on attitudes or proficiency, and often on a large scale.In more localizedcontexts,they offer practitionersa very useful tool in a numberof areas,particularly when the population is small enoughto make personalinterviewing realistic, rather than requiring a questionnaireformat. For instance,a profile of institutional English-language proficiency requirementscan be gatheredrelatively quickly with suchstandardizedquestionsas: How many non-nativespeakersare therein your department? What proportion of the total studentpopulationdo they represent? Do you ask for a scoreon a standardtest suchasTOEFL or IELTS? How do you deal with languageproblems? Further typical applications are in the analysis of learner needs, methodological learning preferences,evaluation, and so on; indeed any aspectof school and classroomlife where the teacherresearcheris interestedin gainingan overview.

Semi-structuredinterviews

Interviews in this categoryhave a structuredoverall framework but allow for greater flexibility within that, for example in changing the order of questionsand for more extensivefollow-up of responses.So, for instance, answersto the fourth questionabovecould be probedfurther:

184

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

How do you usually deal with languageproblems? I usually try to correctthe grammarmyself Do you think that is an appropriaterole for you?

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

(and so on). The interviewer, then, remainsin control of the direction of the interview but with much more leeway. Although this format has characteristicsof both other types, it is usually regardedas being closer to the qualitative paradigmbecauseit allows for richer interactionsand more personalized responsesthan the quasi-automatoninterviewer armed with entirely precodedquestions. Unstructuredinterviews This style of interviewing still hasto start with somekind of agenda,but it is usually a loose one, a rough checklist built around issuesformulated in outline only. The direction of the interview intentionally follows intervieweeresponses,with someof the characteristicsof natural conversation: 'The qualitative interviewer should arrive with a short list of issueoriented questions... The purposefor the most part is not to get simple yes and no answersbut descriptionsof an episode,a linkage, an explanation. Formulating the questionsand anticipating probesthat evoke good responsesis a specialart' (Stake,1995: 65). In the sensethat less structuredinterviews are more in touch with the social world, being able to tap into everydayreality, they are clearly quite well suited to teacherresearchand to the ways in which teachersinteract with learners,colleaguesand others in their working environment.The interviewer needsa well-developedfeeling for context and some understandingof the concernsof intervieweesas a starting point. For instance, one of the presentauthorsis currently investigatingpatternsof changein the experiencesof overseasstudentsin Britain by meansof interviewswith a small number of people, using issuesas 'triggers' (e.g. 'Tell me something aboutthe peopleyou interactwith' and 'Have therebeenany important turning points for you sinceyou arrived?').Open-endedquestioningis also particularly appropriate in interpretive case study research and, perhapscross-referredto other datasourcessuchas diaries,can contribute to a roundedportrayalof individuals. Qualitative interviewing and ethnographicresearchmethodology are clearly closely related.From a languageteachingperspective,studiesin the ethnographyof languageand communicationare of particular interest. Saville-Troike's survey book explainsthat ethnographicinterviews do not on the whole have predeterminedresponsealternativesbecauseof the need to be 'open to new meaningsand unforeseenpatternsof behaviour'

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

ASKING QUESTIONS

185

(1989: 124). She makes the important point that many conceptsare not universal but language-and culture-specific('friendly' is one of her examples) that have to be explored in the context in which they are used. Although most teachersare instinctively sensitiveto such differentiation in languageuse, there is rich researchground here for someoneinterested in cross-culturalfactors of languageuse and ways in which they affect classroominteraction. One further format of interesthere is the group interview, usually with three or four participantsrather than a whole class. Hopkins (1993) finds this the most productive because,he claims, individuals can 'spark' each other into perceptivelines of discussion.The most obvious, though by no meansthe only, use is in formative evaluation.An exampleof this is EFL Services'explorationof studentattitudesin coursesin British private languageschools(EFL Services,1992), using the 'focus group' method. Finally, we reiteratethat, at leastmethodologically,'structured'and 'less structured'interviews are not mutually exclusive.Q!.1alitative interviewing may lead to more structuredquestionsat later stages,for examplewhen checking details with respondents;conversely,precoding may help with initial 'ground-clearing',or may be the first stagein openingup key issues.

Issuesin interviewing The interview as a researchinstrumentraisesa numberof issuesof both principle and practice. Here we comment on just three areasthat seem particularly to impinge on languageteacherresearch. 1. Interviewing, like any activity that deals with personal data,raises obvious questionsof access,confidentiality and privacy. More specifically - and dependingon the type of interview - there is a further ethical dimension that requires us to take into account the role relationship of interviewer-interviewee.Borrowing the terminology of social psychology, this relationshipmay be symmetrical,as betweenpeers,or asymmetrical, as between(say) teacherand headmaster,or teacherand student.A probing, interpretive interview 'invades'the interviewees'private space:especially in the teacher --+ student interview an element of power is introduced.This is even more salient when teachers/researchers are dealing with non-nativespeakers:not only is the question-and-answer technique itself a control device (as so much of the researchon teachers' classroomlanguage shows),but there is, automatically, another layer of control, this time by the teacherof the medium itsel( This will therefore have implications for the formulation of questionsas well as for their content, and will require a good deal of linguistic sensitivity and adaptability by the researcher.(For a particularly negativeinstanceseeNunan, 1992b: 151, quotingVan Lier.)

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

186

RESEARCH METHODS FOR ENGLISH LA]\iGUGI.

Ii \l.IIFR.'

2. In EFL methodologygenerally, there has been considerablediscussion of the parallelsbetweenteachingand therapyor counselling(see for example Stevick's discussion, 1980, of Curran's model of CounsellingLearning).A similar parallel hasbeendrawn with uns.tructuredinterviewing, likening the researcher-interviewer's role to that of the therapist. Cohenand Manion (1989) discussin particular Carl Rogers'advocacyof the non-directiveinterview, at the sametime sheddingdoubt on its usefulnessin educationalcontexts,where the objective is not to 'cure' by changing behaviour but 'to leave a residue of results rather than a posseof curedsouls'(p. 326, quoting Madge). 3. In the social science and educationalresearchliterature, the best method to record interview data has been much debated.There are three main possibilities. • Write-up after the interview. Despiteobvious problemsof memory,

this method is favouredby Stake(1995) amongothers.He calls it a 'written facsimile', claiming that data written up within a few hourscan bestcapturethe meaningand innuendoof ongoingconversation.It also enablesannotationas to para-and non-linguistic aspectsof the encounter. • Audio recording. A tape/cassette-recorder may be intrusive (and will requirepermissionfrom the interviewee).Data will haveto be transcribed,a very time-consumingundertaking.However, when languagelearnersare being interviewed there is a clear advantage in this technique becauseit ensuresan accurate and detailed record of actual languagedata which may not just be the vehicle for the interview, but its object as well. (For an interestingvariation on qualitative data analysis, see Hycner's 'phenomenological' model, quoted in Cohen and Manion, 1989. It begins with transcription and proceedsthrough 15 stepswhich cluster segments of meaningand include membercheckingand re-interviewing.) • Note-taking. There may be problems with the sheerquantity of ephemeraldata,and the activity itself may distractthe attentionof both interviewerand interviewee. Walker makes the point that these are not just technical nuancesbut represent'different ways of going about doing research... They each imply a different kind of relationshipbetweenthe researcherand the task and betwee:1the researcherand the subject,and a different conceptionof the natureof the task' (1985: 109). 4. The analysis of structuredand semi-structuredinterview data can follow some of the same numerical lines as that of questionnaires.The more open-ended,exploratory and ethnographicinterviews referred to above may be analysedqualitatively by searchingfor themes,by looking for patterns,by looking for interpretationswhich are consistentwith all

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

ASKING QUESTIONS

187

the information revealedin the interview. The criterion of validity of such interpretationshas to be their plausibility and the senseof accountingfor all that the intervieweehas revealed.This processhasbeenlikened to writing fiction, wherethe criterion is also its plausibility and convincingnessto other people; but in this case the researchis discoveringsomeoneelse's truth, not inventing an imaginaryworld. Thereare also methodsof 'contentanalysis'(Cohenand Manion, 1989: 62--4, and Chapter8, for example)for naturally occurring texts which can be extendedto interview data. Contentanalysisis essentiallya way of taking verbal documents,not necessarilyelicited for researchpurposes,and codifying them for quantitativeanalysis.A contentanalysissetsup a number of categorieswhich are appropriateto the specified text type and content, and createsunits of analysis - words, phrases,themes,and so on. Ways of systematicallyidentifying theseare set up and the resulting tallies can be grouped,nested,ordered with respectto each other, countedand otherwise displayed. A languagelearning example is given by Wenden's paper on 'How to be a successfullanguagelearner' (1987). In this, she used contentanalysisof semi-structuredinterviews lasting about 90 minutes eachwith 25 learnersof English (nearly 40 hours of tape!). The topic was their beliefs of how best to learn English. Wendenset up a numberof criteria to identify statementsabout her respondents'beliefs about language learning. These systematicallydefined the 'themes'in the data by focusing in various ways described in her Appendix 3 (1987: 116). Essentially,this involved finding generalizations,characteristicapproaches, justifications, certain verbal contextsor linguistic frames (like 'I think ...'), repeatedphrasesand topics, and lengthy explanations.From the mass of data this procedureallowed Wenden to distil the learners'beliefs into 12 statements,most of which most of the group had contributedto.

Conclusion This chapter has discussedtwo closely related ways of obtaining data. Both questionnairesand interviews require care and subtlety in questioning techniques,but in different ways. Their use rangeswidely acrossthe spectrumfrom normative to non-normative,and also in terms of scale. Q!.Iestionnairesand certain forms of interview can be used in large-scale and small-scalecontexts,by individual researchersor groups working in collaboration, and in different sites. Data-analysistechniquesalso may range from simple numerical ones to subtle and sensitive interp'retive methods. Both kinds of researchcan be seen as somewhatspecialized forms of conversation,and the parallel with conversationreminds us of the many interesting features of such information exchange:power and statusdistribution, risk of giving and taking offence, loss of face, properly formulated requests,self-revelationand disclosureto strangers,rights of

188

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

refusal, and so on. Both questionnairesand interviews are very popular forms of researchtool and may be usedin conjunctionwith eachotherand other forms of data.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Discussionnotes 1. Have you ever had the experienceof writing a questionnairewhether for courseevaluationor any other purpose?If so, were there any aspectsof the designthat in retrospectdid not yield the kind of data you wanted?Try to enumeratesome of the reasons for this. You may also like to consider your own attitude to respondingto questionnairesdesignedby other people. Do you usually completethem? 2. Supposeyou are conducting a needs analysis or a programme evaluationwithin your own institution. To what extent would (a) questionnairesand (b) interviews be appropriatemethods?Might you useboth, for different reasons? 3. What kind of teacher-research project might bestbe servedby the useof naturalistic,unstructuredinterview techniques?

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

12 Looking inside: methods for introspection

Introduction Many interestingquestionscan be answeredby consideringwhat a person is thinking about while they perform sometask. It is a commonplace of skill theory that skilled performanceof tasksrequiresvarying degrees of conscious attention: some aspectsof tasks are virtually automatic, othersrequirecloseconcentrationand the consciousintegrationof many different kinds of information. Similarly, the sametask for an unskilled personmay require more consciousattention than for a skilled person. To the extent that languagelearning and indeed languageteachingmay be regardedas skills, the samepoints apply: attention is neededfor any languagelearning, using and teachingprocesses.Therefore,askingsomeone what they are thinking about when they perform some more or less skilled activity can reveal the contentof thoseattentionprocesses.People may report on the points at which they make decisions and their sequence,their strategies,their perceptions,their meansof monitoring and controlling what they do, their frustration at memory loss or cognitive overload,or sometimesnothing at all - the mind goesblank. This internal aspectof the active participant in languagelearning and teaching may be partial, since there may be automatic and autonomous processesrunning as well, but it is crucial, sinceso often decisionsabout future action are madeon the basisof the perceptionsexpressedin those reports. This chapterwill first look at some of the ways in which this kind of researchhas been characterized,roughly how it is conducted,and what someof the major problemsare. Then it will turn the discussionround to look at doing researchon oneselfby observingoneselfrigorously in various situationsand what might be learnedfrom that. From there the discussion proceedsto outline introspective research methods for other people, mainly, but not exclusively, learners.The final section will give the flavour of this kind of work in a couple of brief accountsof research work.

190

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Introspection There has been a long history of introspective research on learners extendingback more than 20 years.Techniqueshave been developedfor getting learners to reveal their own thought processeswhile or shortly after performing some kind of language activity. One of the earliest researchersin this field, Hosenfeld (1976), commentedthat, after conducting a lengthy seriesof individual interviews with membersof a class learning intermediateFrench,the class teacherexpressedsurpriseat how much more the researcherknew about her studentsthan she did. In the light of subsequentdevelopments,this remark can be seen as a justification, not merely for research,but for teachersdoing research.In fact in the case of Hosenfeld's work the distinction between teaching and researchbecomesblurred, since she went on to developremedialteaching methodsfor L2 reading basedon think-aloud revelationsby studentsof what strategiesthey typically adopted.There has not been the samehistory of researchon teachingusing verbal reportseither by teachersor anybody else: the equivalentis probably the teacher'sdiary and diary studies, which beganseriouslyat much the sametime, and which were described in detail in Chapter8. Secondlanguageacquisition studieshave pursued lines of researchabout autonomousacquisition processesand voluntary learning processesfor some years. Ellis (1994) presentsa comprehensive overview; McDonough (1995) explores the many interesting studies in strategyuse and skilled languagelearning. This area of researchhas also spawneda numberof evaluationstudiesof the effects of teachingstrategy use to languagelearners,with mixed results. Studying languagelearning, or anything else, by using verbal report methods sits fairly firmly in a long tradition of individual psychology studies.Typically numbersof participatinglearnersare few, becauseof the amount of data that anyonelearner producesfor analysisin a couple of sessions.It is for this reasonnot usually bracketedwith the study of social contexts;however,context of learning will manifestly affect what learners will report. Whereverbal report is used for analysingclassroombehaviour or in combinationwith classroomobservation,it could be arguedthat the individual study interacts with the social study. However, just as other techniquescan be used in severalresearchtraditions, so can verbal report protocolsbe analysedeither quantitativelyor qualitatively, and often both kinds of analysis are used to support an interpretation.Verbal report methodshaveallowed learnersas individuals to take centrestagein investigations of languageteachingand learning, rather than languageacquisition theories,teachingtechniques,or materialsdevelopment. Although the methodsdevelopedfor introspectiveresearchare neither purely experimentalnor naturalistic(but they can be usedin both kinds of research),nor do they fit into the elaborateaction researchcycle, teachers

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

LOOKING INSIDE: METHODS FOR INTROSPECTION

191

will find that they producefascinatingdata. This often gives quite unexpectedinsightsinto how a learneris processingthe material,why a persistent error will not go away, what kind of contribution particular text types, activities, classroom tasks and writing demandsmake to the learner's growing competencein the language,and it also enablesteachersto assess the challengethey set for their learnersin the light of their own experience as foreign languagelearnersand users.

Verbal report and think-aloud It is obvious that not all our mental activity is available for our conscious inspection.In many areasof our lives, especiallyin usingour native language, we are only aware of a proportion of the decision processesthat we go through.However,both in usingand learninglanguage,we are awareof some of them, and we often pay close attention to them, for examplein how we speakor how we phrasemessages,and in how we try to notice what seemsto be important in our learningand how we try to ensurewe remembersomething. Verbal reporting methodsare premisedon the idea that theseinsights are not just 'epiphenomena', that is imperfectreflectionsof the real but invisible decisionprocessesin our minds, but real phenomenain their own right. Furthermore,we tend to placesomerelianceon the ways in which we believe we operate,to the extent that we base future actions, toleranceof failure, expectationsof successand more generally our hopesand fears in learning achievement on these cognitive insights. Verbal reporting methods are designedto help us reveal theseso a fuller picture of learning processescan be constructed.Obviously, also, people do not necessarilywant to have all their thoughts revealedto others, so there is a strong ethical dimension to this kind of research,in terms of confidentiality, publication, protection of contributions,as discussedin Chapter4 and elsewhere. There is a chain of eventsinvolved in this kind of research,and there are important decisions to be taken for each link in the chain. It runs roughly as follows. Thoughts Doesdirecting attentionto one'sthoughtsspoil the flow of thought? Doesintrospectionrequireany kind of training? Is attention divided betweendoing the task and looking at oneself doing it? Think-aloud Is the reporting immediate, say within 20 secondsof the mental event(introspection)or later (retrospection)? Doesreportingor giving a verbal report itself benefit from training? When is verbal report requestedand how often?

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

192

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Recording Is the report spokendirectly into a recorder,or written down? Is the report in the form of a checklistor structuredquestionnaire? Is it recordedby an externalobserverusing field notes? How closely can what the reporter says be related to what they are doing? Is time information available? Analysis Can the eventsbeing reported be related to an existing set of categories,e.g. an inventory of strategiesor a codingscheme? Can they be describedqualitatively? Can a contentanalysisbe developed? Comparison How doesone report comparewith others? Are theredifferencesaccordingto moderatorvariablessuch as proficiency level, activity type, text difficulty, topic familiarity, task importance? Interpretation How can the reportsbe bestdescribed? Verbal reporting has had a chequeredhistory. Suspicionsabout the relevance,usefulnessand indeed acceptabilityof such data were elaborated into strongcriticisms by Nisbett and Wilson (1977), who arguedthat our reportsof our own mental life are tooflawed and, amongother things, too influencedby what we believe we want othersto think to be used as data. According to them, our own reports are too unreliable, since we are not necessarilygood observers,and also becauseour explanationsof our ourselvesto ourselvesmay be complexand spurious,not admittingsimple but unprestigiousexplanations.There are, as mentionedalready,limits on the depth of processto which we can have consciousaccess.Lastly, the act of reportingmay alter the event comparedto doing the samething when not required to report. On the other hand, Ericssonand Simon (1987; 1993) have argued (particularlyin the first edition, in 1984, of their 1993 book, on which much verbal report methodologyhas beenbased)that such data is admissible,interestingand usable.They introduceda numberof sensible constraints. First, they emphasizedthe limitations set by human attention.Verbal reports can only be expectedto yield data on 'heeded' processes.Second,someprocessesmay be easyto put into language(called Level 1 reporting) and some may be difficult (Level 2 reporting). Third, they pointed out that some reporting involves going back some way in

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

LOOKING INSIDE: METHODS FOR INTROSPECTION

193

time, called 'retrospective'reporting, while someis concurrent,reporting as closely as possible to the actual performanceof the task. Concurrent verbal reportingis the normal kind of data usednowadaysin this method. The report is usually called a 'protocol', and the method, 'protocol analysis'. Cohen (1992) has summarizedthesevarious differencesin this tripartitedistinction, which he glossedas follows in a later a paper(1995):

Self-report Generalizedstatementsabout learning behaviour, descriptions of what learnersbelieveabout themselves,e.g. 'I tend to be a speedlistener'. Self-observation The inspectionof specific languagebehaviour,e.g. 'What 1 did was to skim through the incoming oral text as 1 listened,picking out key words and phrases'. Self-revelation 'Think-aloud', stream-of-consciousnessdisclosure of thought processeswhile the information is being attendedto, e.g. 'Who does the "they" refer to here?' Thereare many problemswith this kind of data,but the interestinherent in discoveringwhat learnersactually do in their classroomsand places wherethey learn and usethe languagemakesit worthwhile. Often it can be surprisingto a teacherto discover,as Hosenfeld(1979) did with Cora, that the studentis actually subvertingthe instructionsfor a particularexercise but, in the process,making it her own contribution to her own learning. Disobedienceto instructionswas revealedas normal and understandable in the testing situation, where test-takersoften report refusing to read a text through first but going straight to the questions,because(a) maximum time is neededfor the questions,which is where the reward is, and (b) the testers'contentionthat the tests assessability to comprehendthe overall meaningof the passageis dubiousand the testeesknow it (Cohen, 1984). The problems particularly concern reliability, timelag, yield, reactive effects and fullness of reporting. Unreliability is inherent in the data, as we have seen,and is also inherentin using coding systemsand categorization systemsfor analysisof observationaldata for classrooms(Chapter7). The usual answer is in terms of triangulation, here the use of multiple raters or categorizers.Timelag betweenmental event and verbal report may simply meanthat useful data is lost, just forgotten, and the premium is on immediacy, hence the accent on concurrent verbal reporting.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

194

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

However,it has beensuggestedthat a comparisonbetweenconcurrentand retrospectiveverbal reportsis neededto get a more balancedpicture. Low yield can be a problem if the reporterdoesnot want to say much, is confused about the task, or has linguistic difficulties when reporting in L2; high yield can also be a problem becausemany humanbeingsbecomeparticularly loquaciouswhen talking about themselves.Reactiveeffects refer to the effects of performing verbal reporting on the task being observed. These may, as claimed by Ericsson (1988) referring to reading studies, merely slow down but not changethe quality of the task performance,but Cohen(1995) documentsa numberof reportswhere peoplehave said that doing verbal reporting has changedwhat they normally do, both detrimentally and also beneficially, by drawing their own attention to habits of theirs that they can modify with advantage.Fullnessof reporting (by the researcher)can be a problem if insufficient detail is given about the reporters,such as their proficiency level, known languages,experiencein doing verbal report, and other significant pointers, since such research dealswith individuals ratherthan groups.

Researchon oneself Most of the considerationsabovehave arisenin the context of developing methodsof investigatingother people'sintrospections.They are also relevant when consideringhow to reveal and capitalize on one's own observations as a languagelearner, user and teacher,for the potential benefit of one's own teaching. Observingoneselfmay seemto be a strangekind of researchto be advocatinghere, but it can be a serious factor in understanding what might be happeningto learnersand how they might be reactingto the languageand.languagetaskswith which they are faced. Of course, one should not commit the 'egocentricerror', that is assuming that, becausewe act the way we do, everybodyelse does the same. Such insights into oneselfneed to be testedout on others- but thereforethey form a rich source of researchquestions.The discipline implied in the previoussectionabout decisionsto be madeconcerningthe immediacyof recordingthe observation,methodof recordingand analysisapply equally to observingoneselfThey are obviously and closely related to the discipline and rigour describedin Chapter8 about diary-keeping,and fit neatly with the idea of the self-reflectiveteacher. However, what can one observe?Relevantareasfor self-observationand self-revelationwill include one'sown experiencesof languagelearningand foreign languageuse, of languageteaching,and of working in institutions and interactingwith students,colleaguesand personsin authority. Under the headingof languagelearning and languageuse, two autobiographical exampleswhich one of the presentauthorshas noticed but neverseriously

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

LOOKING INSIDE: METHODS FOR INTROSPECTION

195

researchedwhile operating in a foreign languageare the use of mental translationin talking and writing and the completefailure to use the strategy of skim-reading.With regard to translation, the presentauthor has noticed many times clear auditory imagesof the Ll sentencesbeing used as the basis for L2 production even when that has been fairly fluent. In fact the only times he was fairly happy that he was not translating was when the audiencepressurewas very strong: in particular, when talking to a classof schoolchildrenin their language,when any hesitationor lack of accuracywas swiftly penalizedby loss of authority. This may serve as a very obvious exampleof an area in which individual experiencecontrasts with received wisdom. It is possibly an example of acceptedknowledge being defined by researchers(here, old work dating back to the behaviourist tradition of researchon 'transfer') and enshrinedunchallengedin methodologytexts long after the original researchis forgotten or discredited. The other autobiographicalexampleis skim-reading,often advocated in textbooksas a useful learner'sstrategy,but which this author hasalways found extremelydifficult in the few foreign languagesavailableto him and also difficult in his native English. Attempts to skim-readusually result in fast readingand the realizationthat good chunksof the meaninghavebeen missed.Of course,the personalexamplesdo not matter: what mattersis the availability of evidencefrom one's own processingto check against receivedwisdom, other learners'experienceand one'sstudents'own preferred processingstrategies. There may well be instructive experiencesin teachers'interaction as learnerswith other teachers,as commentedon in Chapter 8. While the majority of these are probably best captured by using a diary method, thereare somesituationswherethe teacher-learnermight considerusing a more immediateor concurrentverbal report method.The most obviousopportunitiesfor observingone'sown reactionsto anotherteacher'smethods are when doing a listening exercise, when concurrent written notes or interrupting the tape for spokenverbal report are feasible; carrying out a readingassignment,using a version of protocol analysis,or when attempting to write in the new language,again using a concurrentverbal report. Using concurrentverbal reporting in the activity of languageteaching presentsobvious problems;however, the more immediately a note of the thoughtsin the teacher'smind can be made after conductingsome teaching procedure,trying out somethingnew, using various new strategiesfor classroommanagementor for maximizing work involvement, the more accurateand detailed the evidencefor later reflection and incorporation into a diary can be. One private areain which a full-scale protocol analysis would be feasible, and which to this author'sknowledge has not yet been performed,is in marking and giving feedbackon students'writing. Given the topicality and popularity of studiesof students'writing processesand

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

196

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

strategiesfor interpretingand using teachers'feedback,it is curiousgap in the ELT literature. A third opportunityfor researchon oneselfas a teacherconcernsinteraction within one's own teachingcontext. Time management,staffroom interaction,assemblingresources,preparingmaterial, attendingmeetings and many other activities are the normal sceneof a teacher'sprofessional life: here there is scopefor introspectionboth for one'sown purposesof reflection and in order to complement'outsider'researchon thesekinds of activities. Dadds (1995) has argued for the importanceof such detailed reflection underthe headingof 'first personresearch'.

Researchon learners The suggestionof teachers'self-observationin the previous section may appearunrealisticand evenquirky to some;to othersit will not evencount as research.It is, however,in the spirit of 'teacherdevelopment'and consistentwith the aims of the more familiar methodsof diary research.The main thrust of verbal report methodshas, however,beenin trying to elucidate what learnersare actually doing while they perform activities and tasksthat are consideredto be relevantfor languagelearning.This section outlines the main stepsin eliciting and analysingverbal reports of introspective data. For the purposesof clarity, the researchprocesscan be divided into six steps:any actual investigationwill make different choices at each step consistentwith the researchpurposeor the particular question beinginvestigated.

1. Training It is usual, in this kind of research,to give somedemonstrationof what is meant by giving a concurrentverbal report to the reporters.This varies from simply doing the task in front of the learnersto settingup a number of activities for them to do to give them a senseof what is involved. The usual method is to ask the reportersto perform some simple task which has nothing directly to do with the task to be investigated.The reasonfor that stipulation is that it is important not to suggestto the reporterswhat the researcherwants them to say. The usual tasks used for training in verbal reporting are solving anagramsaloud, doing simple letter puzzles, and doing simple mentalarithmeticaloud.

2. Elicitation The chosenmethodof elicitation will dependlargely on the particulartask being investigated but an important general distinction is between a

LOOKING INSIDE: METHODS FOR INTROSPECTION

197

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

'planned' reportingpoint and a 'free' one. Examplesof 'free' reporting pointsare: • asking for a responsewheneverthe reporter thinks he or she has somethingto say • askingfor a concurrentverbal report while writing an assignment • askingthe respondentto pausea tapeor video in a listening task at will • asking for a report on readingwhen somethingoccursto the person. Free reporting points may be 'democratic'but they also involve a judgement on the part of the respondentas to when they havesomethingrelevant to say: this may concealsomeuseful information. Examplesof 'planned'reportingpoints would be: • askinga readerto report at the endsof sentences,lines, clauses,or wherea markeris insertedinto the text • pausinga tapeat pre-determinedintervalsfor reporting • askingfor a report after eachitem in a test • stopping a class and asking for a report on what people were attendingto at that point. Planned reporting points are a way of ensuring quantity and to some extent quality of data: however,they may break up the flow of the activity in questionin uncomfortableways, even to the extentof altering its nature significantly. When working with languagelearners,it is important to give them the opportunity to report in whateverlanguagecomesto them naturally. It is not always their native language.Even when they choose,or are instructed to use the L2 as the languageof report, they should be allowed to use L 1 for phrasesand words, and code-switchfreely. A subsequentinterview can be used to discoverwhat was meant.The importanceof making reporting easycannotbe over-emphasized. It is normal to requestpeopleto report on what they are thinking at the time of doing the task, not how they think they are performing it. Questionsthat requireinterpretationof mentalactivity are usually 'level 2' and involve more than just reporting: in particular, self-monitoring and interpretation. 3. Recording

Almost universally, verbal reportsare recordedon to audio-or video-tape. It is important to ensure,either by a tim.! code or by field notes by the observingresearcher,or by someother mechanicalmeans,a clear link-up betweenthe tapeof the verbal report and the progressof the activity being

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

198

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

reported on. In a writing task, for example,there must be some way of tying the verbal report directly to the text as it is being written. One of the frequent activities noticed in this situation is 'self-dictation',i.e. speaking while writing, and this can only be recorded if there is some meansof establishingexactly what the writer was writing when he or she said the comments.Sometimesthe link-up can be establishednaturally from the contentsof the verbal report; the method of 'planned' reportingpoints allows more accuracythan a free method. 4. Analysis

Introspectivemethodsproducemessyand obviously qualitativedata.They can be analysedboth qualitatively and quantitatively, as with other data forms discussedelsewhere.Coding systemscan be used; large amountsof work on the writing processhave used versionsof the coding systemfor written protocols set up by Perl (1981) for studying written composition by native writers. In many casesthe coding categorieshave beenelevated from the status of data-analysiscategoriesto mental events themselves, under the generalheadingof 'strategies'.The statusof the idea of mental strategyhas been discussedby Bialystok (1990) and McDonough (1995), and placed within cognitive theory by O'Malley and Chamot(1990). The productsof suchclassificationsystemscan be counted,and frequencydistributions comparedfor various purposes.Alternatively, or by way of complement,qualitativeanalysismay be performed,in which the verbal report is interpretedto show the flow and developmentof, for example,a student'scomprehensionof a text, or to comparewriting performedfor different purposes or in different genres (narrative versus expository; paraphraseversussummary).A qualitative analysisis usually elaborative: the researcherattemptsto interpretthe learner'sprotocol. 5. Reliability

The reliability of the data analysisis normally checkedby having two or three independentjudges operatethe classification system and compute their levels of agreement.In addition, a 'within-judge' check can be performed in which one judge (usually the researcher)reworks part of the data again and comparesthe original classificationor interpretationwith the secondtime around. 6. Triangulation

Already mentionedoften elsewherein this book, triangulation is a frequent technique in introspective methods too. The first and most

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

LOOKING INSIDE: METHODS FOR INTROSPECTION

199

important triangulation is betweenthe verbal report and the product of the activity (for example,evidenceof the actual comprehensionof the passage, the written composition, residualmemory of the vocabulary items tackled) both to establishthe link-ups referred to under 3 above, but also to act as a first step in interpretationof how the learner performed the task. Another important triangulation is between the verbal report and field notes by the observer,to supplementthe respondent'saccountwith other observations.Furtheropportunitiesfor triangulationmay arise from using other performancedata such as successor failure in tests, independent judgementsof quality of written work, other indicesof task performance such as comprehensionquestions,and so on, dependingon the task being investigated.Verbal report data may be usedin triangulationof other kinds of data, such as lesson observationtranscripts(Cohen and Aphek, 1981) or in studiesof 'uptake'of lessoncontents(Slimani, 1989). This brief six-stageoutline of the methodologyis intended to facilitate undertakingthis kind of researchby any teacherwho hasthe opportunityto investigateindividual learners,or in somecasesgroupsof learners,and find out how they are solving their language-learningand language-usingproblems. The methodologyhas already been used for some time to establish strategiesfor learningand use (seeMcDonough, 1995, for a comprehensive survey) and has beenclosely associatedwith the notion of 'learnertraining': the suggestionthat one function of languageteachingis to give learnerscertain kinds of instruction in how to learn. Thus teacherscan use this kind of researchmethod for a variety of projectsthat may be close to home in their own teaching,and find out more about how their own studentsreact to the languageand to languageinstruction, particularly, perhaps,in remedialsituations. It is noticeablethat a good proportion of the published'think-aloud' literature has in fact been concernedwith remedial situations. A teacher wanting to find out why a particularstudentdoesnot seemto be able to cope in certain circumstances,for examplein expandingvocabulary,can use the method to find out what the studentnormally doesand perhapsmake suggestionson that basis for remedial strategies.The methodologyis suitable for small-scaleindividual studiesand for large-scalesurvey work; for qualitative descriptionand for preciseexperimentalwork involving comparison of groups and treatments.Perhapsthe most tedious aspect,certainly the most time-consuming,is the transcriptionof the reports from the immediate taped record; but, on the other hand, this necessaryprocesscan be used for a preliminary impressionisticanalysisof the kind of information to be found in the data and someindicationsof the most appropriateanalysiscategoriesor analysismethodfor that data.The versatility of the methodallows projectson a whole rangeof problems,for example: • studentsprocessingin readingcomprehension • listening comprehension

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

200

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

• • • • • • • • •

writing compositionsand other tasks the processingof feedback learningthe organizationof talk and pragmaticstrategies vocabularylearningand recall the useof translation encounterswith different kinds of materialsand resources taking tests learnertraining evaluationof learningactivities including learningstrategies.

Someexamples There have beenpublishedstudiesin all of the aboveareas.Cohen'sbook (1992) uses predominantly verbal report data and strategy work and is addressed,interestingly, to three separatesets of people - researchers, teachersand learners- in the belief that talking aboutlanguagelearningin this way and doing this kind of researchis equally interestingto all three groups of participants in the profession.Two mainstreamexamplesof publishedwork will be briefly describedhere.

Writing processes Arndt (1987) used protocol analysis to study six Chinesewriters writing first in their native Chineseand then in English. She performed a full analysis of the verbal report transcripts using an adaptation of Perl's (1981) schemewith full time-basedlinkage to the written products. She was interestedmainly in the comparisonbetweenthe writers' processing in their Ll and L2 (English) in which they were moderatelyproficient. The protocol analysesrevealedthat, perhapssurprisingly, the very different expressionopportunitiesavailableto them in the two languagesdid not swamp theirindividuality. She found evidencethat eachwriter operatedin consistentways in whichever languagethey were writing, and that each writer was very much an individual in their approach,to the extent that she was able to show how the ways in which each writer planned, rehearsed,reviewed, edited, commentedand revised their texts fell into distinct and individual styles.

Readingcomprehension Block (1986; 1992) investigated first and second language readers in college who were at a low level of proficiency compared to the level demandedby their courses.She used a planned reporting method consisting of preparedtexts with a red marker at the points where verbal

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

LOOKING INSIDE: METHODS FOR INTROSPECTION

201

reporting was requested: basically at the end of each sentence or clause.The readersalso gavean immediateverbal summaryor retelling of the text. On the basis of previous work and the categoriesthat the data seemedto suggest,Block set up a categorysystemof readingstrategies, and proceededto code each report for the use of thesestrategies.Thus, comparisonscould be madebetweenpatternsof strategyuseand learning outcomeas indicatedby grade-pointaverage(the crucial measureof academic progressin the USA). However, the most interestingaspectof her analysiswas the identification of two patternsof strategyuse adoptedby the students,which were connectedto their empathywith the meaningof the texts. Shecalled them 'integrators'and 'non-integrators'.The 'integrators' kept their commentsfairly exactly to the actual text, and highlighted the main theme in their retellings, using self-monitoring moves and noticing the internal structure of the text. The 'non-integrators' talked a greatdeal abouttheir own reactionsto the text and their personal associations,reactedless to text structure,monitoredthemselvesless, and reportedmore details and fewer main points in their verbal summaries. Thesetwo modesof respondingto the text are derived from the patterns of strategyuseas revealedby the protocols,and were also,associated with academicprogress.The integratorshad higher grade-pointaveragesat the end of the first semester.This researchdemonstrateddifferentiation among studentsworking in English as a first and as a secondlanguage, but at an unsatisfactorylevel of achievement,on the basisof strategyuse. It also demonstratedthe depthof insight which this methodcan provide.

Conclusion This chapterhastakenthe notion of 'introspection',which was the defocto framework for our earlier review of diariesand diary studies,and hasmore formally defined it as a procedurefor research,particularly but not only in languagelearning. We looked at a numberof the more problematicaspects involved in verbal reporting of internal cognitive processes,including the issue of how comprehensive,accurate and reliable such data can be. Comparisonswere then drawn betweenresearchon others and research on oneselfThe chapteralso discussedthe more pragmaticareasof thinkaloud proceduresand data analysis,noting particularly the appropriacyof interpretive method. This section included a discussionof the ways in which people might be trained to conduct research,and actually 'do' verbal reportsthemselves. A numberof examples'from languagelearning and teachinghave been included, both actual and potential; and we have tried to show that researchbasedon introspectionis appropriateas an approachfor teachers and also viable in methodologicalterms.

202

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Discussionnotes 1. Reflect on someof your own experienceswhile using or learninga foreign language, and how you might investigate your own thoughtswhile performingthesetasks. 2. Perhapsyou havestudentsin your classwho needremedialteaching in some areasof languagelearning: observehow they tackle these different tasks, and try to work out a way of asking them what they are thinking about when they are working at them. 3. Do you think there are recommendedlearning practices in methodologytextbookswhich could bearre-examinationby introspectivemethods?

13

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Studying cases

Introduction A casestudy, it must be said at the outset,is not itself a researchmethodnor the equivalentof one: it employsmethodsand techniquesin the investigation of an objectof interest.Given that casestudiestypically offer instancesof the use of severalmethodsfor one piece of research,they are discussedin this penultimatechapterwhich leadsdirectly to a considerationof multi method projects. As we shall see, the notion of a 'case' is in many ways quintessentially naturalistic, though not exclusively so, notwithstandingCohen and Manion's assertion that 'present antipathy towards the statisticalexperimentalparadigm has createdsomethingof a boom industry in case study research'(1989: 125). Casestudiescan rangefrom large-to small-scale, though it is particularly at the more 'micro' end of the spectrumthat they are arguably most appropriate for teacher-generatedresearch.They have an important role to play in action research,whether for personalor wholeschool development(Elliott, 1991). Closer to our own professionalconcerns here, they are a very suitableformat for studiesof languagelearning. In sum, casestudieshavea good deal of potential,but are also somewhatproblematic in both principle and practice,especiallyfrom the point of view of the single caseand the wider value of sucha study. The most explicit discussionof casestudiesas a researchgenre is to be found in education,and in social sciencemore generally,rather than in languagelearningand TEFL, so we begin this chapterwith a review of the key definitionsof casestudy and its major characteristicsand subdivisions.This is followed by a discussionof its methodologicalunderpinnings,including the natureand analysisof data. The central section of the chapteris concerned with manifestationsof casestudy researchin languagelearningan.d teaching. We concludewith a look at someof the controversysurroundingthe use of casesin research,a debatewhich focusesparticularly on the requirement- or otherwise- for researchto be valid, reliableand generalizable.

What is a case? To most people, the term 'case' probably suggestsmuch the sameas the initial meaning in the Concise OxjiJrd Dictionary - an 'imtance of things

204

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

occurring'.This is generalto say the least,but it doescarry the important implication of a focus on the singular,the individual. From a researchperspective, the study of casesplays a role in many diverse disciplines, for examplethe following. • Clinical uses/healthstudies:medical practitionershold 'caseconferences' to look at the symptoms and treatment of individual patients; thesecasescontribute to broaderetiological study, and conversely,known phenomenaare invoked for specificdiagnoses. • Law: 'caselaw', where individual casesare usedin the interpretation of future cases,is an integral part of many legal systems. • Sociology and anthropology: social scientists study whole cultures, particular subgroups(the elderly, drug addicts ...), or individuals, sometimesby constructingbiographiesand life histories (see,for example,Plummer,1983). • Education: practitionersstudy schoolsor groups of schools(see Cohenand Manion, 1989: 134ft); curricula; the effect of innovations; the implementationof materials;classrooms;teachers;students.Much of this work comesunderthe headingof 'evaluation', putting it firmly within definitions of research.We return to this topic when discussingcasestudiesin EFL. • Languagelearning: researchers often study mothertongueacquisition and L2 developmentby looking at individual learners,and at what they havein common. Yet more examplescould be drawn from psychology,history or business studies.Although there are obviously many differencesin scale, purpose, focus, approachand method in these various orientationsto case study research,we look now at someencompassingdefinitions and sharedcharacteristics,basedprincipally on educationalresearch.

Definitionsand characteristics The studyof casesis not only a qualitativeundertaking,nor doesit presentan either/or perspectivein quantitative/qualitativeterms. Where, for example, researchersneedto study large-scaletrends,caseswill usually be selectedon the basisof random samplingand the data submittedto statisticalanalysis. The individual casesare important in terms of their representativeness and their ability to contributeto making predictionsaboutwhole populations- of patients,consumers,children, and so on: indeed,we shall seelater that both this paradigmand methodologyhave a place in researchon languagelearning. It is neverthelesstrue to say that the weight of currentdiscussionon case study is on interpretiveapproachesand, sinceteachershaveaccessto certain kinds of data, resourcesand timing, naturalistic casestudy is in tune with their reality for reasonsof practicalityas well as principle.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

STUDYING CASES

205

Leaving aside for the moment the considerablecontroversysurrounding the study of individual cases,let us now run through the key features of casestudy.An educationalresearcherwho haswritten a greatdeal about it (especiallyin relation to evaluation)is Stake(1994; 1995), who offers a number of overlapping definitions: 'the epistemologyof the particular', 'the study of the particularityand complexity of a single case'and 'an integrated ... working system'. These are echoed by other writers: Walker (1986) writes of 'an instancein action', and Nunan (1992b: 76-7) quotes six sourceswhosedefinitionsclusterround the sameperspective. From this definitional starting point, there are a number of important corollaries: we should at this point recall that a casecan be one of a number of different entities. First, and most important, a caseis now conventionally referred to as a 'boundedsystem', a term originally coined by Louis Smith, an educationalethnographer(seeStake,1995). Stakehimself representsit with the Greek letter (theta) to indicate that it has both boundariesand working parts. The notion of 'boundedness'is conceptually neat but operationallycomplex: a moment'sreflection will show that actually delimiting a 'case'is no easymatter. If a teacherdecidesto study one of his or her learners,for example,he or she might well puzzle over what aspectsof contextand environmentto include/excludewhen considering influenceson learningbehaviour;a team evaluatinga particularprogrammewill need to decide on the extent to which they should take the institutional and social framework into account.Stake(quotedin Johnson, 1992: 76) proposes'confining the attentionto thoseaspectsthat are relevant to the researchproblemat the time', a useful if somewhatchicken-andegg argument.Hitchcock and Hughes(1995: 319) are rathermore specific, suggestingthat a casecan be looked at in terms of 'key players','key situations' and 'critical incidents'in the life of the case,and going on to refer to Miles and Huberman's(1994) boundarydefinitions relatedto:

e

• temporalcharacteristics • geographicalparameters • inbuilt boundaries • a particularcontextat a point in time • group characteristics • role/function • organizational/institutionalarrangements. Second, a case study in a naturalistic setting will follow the central tenets of qualitative researchby being emic (from within the case) and holistic (the whole systemin its context). Third, although a caseis to be thought of as an 'object', this is not an inert, reified concept,but is crucially concernedwith an understandingof people'sown meaningsand perspectives.Fourth, as data from a casestudy are interpreted,research questionsare 'emergent'rather than fixed a priori. Finally it is, perhaps

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

206

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

surprisingly, still possible to invoke the notion of sampling where it is appropriateto compareor aggregatecases.However, this is done by what Stenhouse(1988) refers·to as 'judgemental'(as opposedto random) sampling, with decisions taken about relevant characteristics:'balance and variety are important; opportunity to learn is of primary importance' (Stake,1995: 6). Before leaving this section, we should note that some researchers, becauseof the elusivenessof the concept, define casesin terms of what they are not. Stake points out that a teacher may be a case, but not teaching,nor policies, nor relationshipsamong schools,becausethey are not sufficiently specific: 'the caseto be studiedprobablyhas problemsand relationships,and the report of the caseis likely to have a theme,but the caseis an entity' (1995: 133). This is, of course,to ignore the more general study of aspectsof teaching(or policies and so on) that might be investigated via casesamplingand aggregation, andwe commenton the criticism levelled at this positionin the final sectionof this chapter.

ClassifYingcases Whateverthe entity chosenfor investigation,casestudiescan be classified in different ways, and the type preferredwill dependon the objective of the researchand probably on the paradigmunderpinningit. A teacheror researchermay be interestedin following the languagedevelopmentof one particular learnerand the contextualinfluenceson that; a public presentation of this one casemay stimulateothers to examineindividuals to compareand contrastwith the original case;an aggregationof such casesmay lead to a fresh perspectiveon learning strategies,or programmeevaluation, or teachingstyles; and this perspectivemay in turn generatefurther casestudies.(This is, it should be stressed,a possiblebut not a necessary progression,and is merely intendedto be illustrative of different orientations to how casesare selected.) A numberof useful taxonomiesare on offer. Yin (quoted in Hitchcock and Hughes,1995) focuseson the outcomeof the study and differentiates betweencasestudiesthat are:

1. exploratory (like a pilot study) 2. descriptive (narrative) 3. explanatory (testingor generatingtheory). The sameauthorsquoteMerriam'sthree-wayschema,divided into:

1. descriptive 2. interpretative (developingconceptualcategories,supporting/chal lengingassumptions) 3. evaluative (addingjudgementto the first two).

STUDYING CASES

207

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Both of theseclassificationsare concernedparticularly with the way the researchis conductedand its outcomein the final report. Stake(1995) looks at caseselectionfrom the point of view of the purposeinforming the initial choice,and distinguishesbetween: 1. the intrinsic casestudy, where the interestis in the casefor its own sake (a teacher looking at a problematic student, or a single course) 2. the instrumentalcasestudy, selectedto help in the understanding of somethingelse (looking at a teacherto seehow he or she marks studentwork, for example) 3. the collective casestudy (coordinatingdata from severalteachers, or severalschools). The broadesttaxonomy is put forward by Lincoln and Guba (1985: 361). They note that casestudies:

1. may be written with different purposes(to chronicle; to 'render'= to describe;to teach;or a combination) 2. may be written at different analytic levels (cf. Merriam andYin) 3. will demanddifferent actionsfrom the enquirer 4. will result in different products. This variety of overlapping categorizationsis set out here becauseit illustratesthat casestudiesoperateon different continua(suchas singularity - aggregation- transferability),all of which can be usefully harnessed by the teacherresearcher.

Methodsin casestudy research Cases,as we and many othershave noted,are objectsto be studiedand are not themselvessynonymouswith any particular techniques.They are for this reasonmethodologicallyeclectic, with a numberof different permutations and possibilities for choice. In this section we do not go into any detail about specific methods becausethey are covered in the relevant chapterselsewherein this book. Instead we look at the main trends in methodsselectionand someof their implications. More detailedexemplification of casestudy researchin languagelearningand teachingfollows in the next section. Where casesare soughtout - sampled- primarily becauseof their typicality and/or their intendedcontribution to some more general pattern or theory, the researchis clearly more likely to use techniquesallowing for numerical analysis of elicited data, particularly questionnairesand structured interview schedules.Other instruments,such as coded observation and factual logs, will makeuseof prespecifiedcategoriesof information.The goal of such researchmay be to examinelarge-scaletrends, for exampleto

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

208

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

make predictions,or to study correlationalrelationshipsbetweendifferent factors. Qualitative methodology,as we know, does not stand in diametric opposition to quantitative, though it is driven by different aims and approachesto data collection, and techniqueswill usually sit at the more 'naturalistic'end of the spectrum.The batteryof possibilitiesfor interpretive casestudy - whetherof single entitiesor groupsof instances- typically includes: • • • • •

naturalisticand descriptiveobservation narrativediaries unstructuredand ethnographicinterviews verbal reports collection of existing information (students'written work or test data,for example:seeJohnson,1992).

Within these parametersthere are priorities and preferences.Cohen and Manion see observationalmethods'at the heart of every casestudy' (1989: 125), whereasStakeclaims that 'qualitativeresearcherstake pride in discoveringand portrayingthe multiple views of the case.The interview is the main road to multiple realities'(1995: 64). Clearly his view is most relevant where the entity is a whole school,or a programme,with a number of different peopleinvolved. Whichevercombinationof techniquesis considered appropriatefor any particular case, it is as always important to rememberthat structuredinterviews, codedobservations,pre-specifieddiary entries are in no way precluded.Just to invent a brief example: Irina Petrov (see Chapter 1) has decidedto study the fate of a new-styleESP syllabus via questionnairesto colleaguesbacked up with selectedethnographicinterviews; her own diary; studentfeedbackand test results;and a structuredinterview with the Head of Faculty. (Cohenand Manion, 1989, have some interesting examples from mainstreameducation of mixed methodcasestudiescombiningnumericaland qualitativetechniques.) The wide range of ways of analysingdata from diese various sources include, as we saw in the chaptersprecedingthis one, correlation,tabulation, tallying, coding, thematic frequency and saliency, quantitativecontent analysis, and so on. In the interpretive mode, Stake (1995) uses a modern idiom and talks of 'surfing through the data', and with specific application to casestudy researchadds the two further categoriesof (1) categoricalaggregationof instanceswithin and acrosscasesand (2) direct interpretation.Furthermore,in the processof analysisand interpretation, triangulation will obviously be an important feature, particularly of data sources(human and documentary)and methods.(We recall that Denzin, 1978, also refers to 'investigator'and 'theory' triangulation.)Other factors to be taken into account,especiallywith a longitudinal casestudy, are permissions ana ethical considerations.This will include the techniqueof 'memberchecking',where researchoutcomesare taken back to the people

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

STUDYING CASES

209

in the contextstudiesto be checkedfor accuracyand plausibility and also for further illumination (seeChapter4 for a longer discussionof research ethics). We leave this sectionwith a further perspectiveoffered by Stake(1995) that particularly addressesthe emergent researchdesign and ways in which it can becomegradually more focused.It is useful to us in the context of the presentbook becauseit is an attemptto distinguishbetweenthe substantivecontentof researchand its methodologicalprocedures(a distinction, however,that Stakeacknowledgesto be a difficult one to explain). For Stake,casestudy researchhastwo centralelements,as follows. • Themes.Subject-matter,information, topical content (e.g. 'What are the typical academicprogrammestaken by these students?'; 'What do the parentswant from the music programme?';or, from our own examples in Chapter 5, 'patterns of participation' or 'How do teachersdeal with latecomers?').To a degreethesequestions are amenableto descriptivedata. • Issues. These are not themes. Rather, they constitute the researcher's'conceptualstructure'.They are the problematicscaffolding on which researchquestionsare built and which move them on to the next stage.(For example:'Is [x] at risk? Why was she identified by teachersas a subject for study, and what is the institutional responsibility?'or 'What are the triggersfor the varying learningstylesexhibitedby this student?') We leavethe final word hereto Stakebecausehe usesthis notion to link different orientationsto research:'In a qualitative researchproject, issues emerge,grow, and die. In quantitativeresearch,as an issuebecomesmore refined or important,a parallel or subsequentstudy is started;the present one keepsits issuesintact' (1995: 21).

Casestudiesin languagelearningand teaching This section commentson how case studies have been used by language learning researchersand by teachers,and on their potential for teacher researchin particular. Both professionalresearchersand teachersare obviously interestedin how learnerslearn, so in principle, and simply intuitively, learner casestudy is one area where their concernsshould dovetail most closely,and indeedwherethereshouldbe fertile groundfor collaboration. Researchinto secondlanguageacquisition The presentbook is in no senseabout SLA (secondlanguageacquisition) researchper se: there are many sound works written by specialistson the subject (for example, Beebe, 1988; Chaudron, 1988; Ellis, 1985; 1990;

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

210

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

SharwoodSmith, 1994), someof which deal with the implications of this researchfor learning in formal classroomsettings.SLA is somethingof a minefield, containing a very large body of often disparatemicrostudies. What is interesting from our point of view is how SLA researchers approachthe notion of a learner as a 'case' in terms of what it reveals aboutresearchmethodology.(Researchers also look at teachers,though the interest is usually in how teachersconstruct learning environmentsfor their students.)There is somemethodologicaldiscussion,though much is implicit. It will come as no surprisethat SLA is underpinnedby the two major paradigmsof hypothetico-deductiveand inductive-interpretiveresearch. Researchers of the first persuasiontypically set up experimentalsituations to test hypothesesderived from theory, so an individual 'case'is a 'subject' serving the goals of the researchdesign. Inductive researchers,on the other hand, first let the casesspeak for themselvesin advanceof theorybuilding. So, for instance,a researchermay deliberatelyconstructa set of interactionsbetweena native and non-nativespeaker,hypothesizingthat the non-nativewill patternthe discoursein certainanticipatedways: alternatively, anotherresearcherwill observenaturally occurringclassroomphenomenaand then seehow they fit into known patterns.Ellis hassomevery useful tabulationsof the techniquesused to investigatea variety of issues in SLA, including tests of different types, recordings of spontaneous speech,oral description of pictures, and written data based on stimuli (1990: 147-9). Whicheverperspectiveis taken - experimental,empirical, naturalisticmost mainstreamSLA researchis intendedto contributeto building, and falsifying, specific hypotheses,suchas: • • • • •

creativeconstruction interlanguage collaborativediscourse input interaction

(and so on: detailscan be found in the referencesgiven earlier). Both dataand theory-first approaches,then, relatecaseresearchto more generalpatterns of how people learn languages.Ellis is unequivocal: 'Classroom researchshould be directed at building a theory of languagelearning -a goal which both types of research can help to meet' (1990: 5). Furthermore,the role of contextual factors tends to be more restricted than in interpretivemodesof educationalor social scienceresearch.At the same time, Ellis writes, almost in passing,about 'the differencesbetween classrooms- which shouldneverbe underestimated'(1990: 90, italics added), but it is not clear how this variability might be fully incorporatedinto the objective of building an overarching theory. He also (p. 48) quotes

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

STUDYING CASES

211

Lightbown, who writes, in relation to a particular set of studies: 'This researchhas beenvery persuasive,but it can be shown to be limited in generalizability. For everyone of the examples ... there are other studies which provide counter-evidence'(italics added). There is, then, some uneaseabout the fit betweenhypothesis-and theory-building,on the one hand, and actual instances- cases- on the other, a point to which we return in the final sectionof this chapter.In this context it is particularly interestingto recall the 'expert' languageleatning diaries (Chapter8) as individual casestudies,becausethey sit firmly at the naturalisticand context-boundend of the spectrum.The sameis true, for example,of the longitudinal case studies cited in Johnson (1992), dealing with second languagewriting, 'cognitive scaffolding' and languagelearning, reading strategies,and adult literacy, or the work of Hakuta (1976) and Wode (1976), and many case-basedstudies of learning strategies(for example, Oxford, 1990).Thesemore individualistic studiesdo not follow the strictly nomotheticobjectivesof much SLA work, and are closer in orientationto the work of teachers. We leave this section with the observationthat SLA offers a great deal of fascinatingand challengingstudy for teachers'practice. Every teacher will find his/her own areasof interest from sifting through the literature: for the record,theseare just a few of ours: role of error correctionand feedback can teachingcauseerrors? enhancementof learningby learners'control of topics delay in uptakeof teachinginput extent to which instruction can affect the 'natural' sequenceof learning • stylesand strategiesof individuals within a group.

• • • • •

We now turn to the question of how teachersmight contribute, via cases,to our understandingof languagelearning, but also of other aspects of practice. Casestudiesand the languageteacher

As far as thematiccontentis concerned,we havealreadycommentedon the obvious congruenceof interestof professionalresearchersand teachersin learnersand learning. Stakedraws a different and rather novel parallel that is particularly relevant to the study and public presentationof qualitative case research by equating the roles of teacher-learnerwith those of researcher-reader. He writes (1995: 92): Teachingis not just lecturing, not just delivering information; ... it is the arrangementof opportunitiesfor learnersto follow a natural human inclination to becomeeducated.... It is important to realize

212

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

that, eventhoughstudentsdo not learn all they are taught,they learn considerablymore than they are taught.... The competentteacher anticipates unanticipatedlearning.... The classroom teacher soon knows eachindividual face and somethingabout the mind behind it, but all too little. This implies, then, a value-ladenview of the caseresearcher'srole, who is enjoined(by Stakeat least)to offer the caseto its readersfor otherkinds of interpretationand perspective,even persuasionabouta point of view. (We will seelater how it relatesthis to 'naturalistic'generalization.)

Teachersand learners Where learnersare concerned,teachersdo not haveto seekout cases:they are there in front of and around them, in daily proximity. Teachersspend their working lives dealing in different ways with individuals, and they needto understandthose'cases',not in the first instanceto build theories and searchfor broaderpatterns,but to understandtheir learners'behaviours, learning styles, languagedevelopment,successes,failures, attitudes, interestsand motivation.As Berthoff puts it: 'We do not neednew information; we need to think about the information we have' (1987: 92). They are, moreover,very well placed to do this over extendedperiods of time, making the longitudinal case study a natural format. Our learners are unique casesalmost by definition, and our knowledgeof them is particularistic and idiographic (Bolster, 1983, discussedin Chapter 2 of this book). Teachersstudy casesto enhancetheir own understanding;to share that understandingwith others who may then carry out parallel work of their own; perhapsto changetheir teachingmethodology;and sometimes to collaboratewith a researcher,becauseteacherand researcherstudies should be complementary,not incompatible(see,for example,the collaborative action researchreportedby Wallat et al., 1981). Teachersare also in a position, as completeparticipants,to be sensitive to context and therefore to have an instinctive senseof how each case might be delineatedas a 'boundedsystem'. It is useful to borrow here from role theory, where a 'focal person' is placed at the centre to show thosefactors that impinge on him/her (seeFig. 13.1). All theseelements in variouscombinationsaffect learnersin classrooms. There are huge numbersof casestudiesin the professionalliterature, although many of them do not call themselvesthat. Betweenthem they use all the available researchmethodology- surveys, interviews, questionnaires,observation,introspectivetechniques,test instruments,written documentsand more. In generalterms their rationale is to offer up instances,or collectionsof instances,for public scrutiny as a contribution to professional knowledge. We will look at two explicit case study

STUDYING CASES

family

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

friends

213

other learners

LEARNER

resources! materials

teachers

setting: institutional! socio-educational

Fig. 13.1 (Adaptedfrom Handy, 1985)

examplesin a little more detail: each of them tells us somethingabout method,and the varying degreesto which boundarydefinitions are taken into account. 1. The caseofQ(McDonoughand McDonough,1993)

Q was an adult learner who neededto do English for his later academic study. He was a studentof one of the presentauthorsover severalmonths. His level on arrival could roughly be describedas intermediate,though his initial reluctanceto participatein classor engagewith English madeit difficult to tell. The starting point for the casestudy was the teacher'sdiary, which expressedconsiderablefrustration at his passivity. The casestudy emergedfrom this, and was not consciouslyplannedin advance.In outline, the study proceededas follows. Timing:

over six months,with subsequentfollow-up

Methods: (triangulated)

Teacher'sdiary including noteson Q:s classroom performance Q:s diary and class work Reportsfrom other teachers Interviewswith Q Interviewswith his future academicsupervisor Languagetest results

Casereport:

Shownto Q in full and checkedby him for accuracy Permissionto usein public report if anonymized

Thematiccontext and outcomes:

A contributionto methodsof studentevaluation Particularisticunderstandingof one learner: affective factors, the process of acculturation and its interaction with language learning; his attitude to learning English;his uptake(or not) of instructional input.

214

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

2. The caseof Marcos Games,1984) Marcos was a Brazilian postgraduateand Jamesthe languagetutor who provided the English language supervision on a 1: 1 basis over many monthsas Marcos was writing his Ph.D. The casestudy offers a fascinating and detailed insight into a student's language developmentas he attemptedto expresscomplex arguments.Through reading,negotiationof meaning,discussionand observation,Jamesfound three categoriesof mistake, shownherein summaryform: • leadingto breakdownin meaning • leadingto blurring of meaning • distractingthe readerfrom the meaningconveyed (eachof which is exemplified in detail in the paper). In some ways the caseof Marcos, focusing as it does on very specific aspectsof languagelearning, is more 'bounded'than that of Q However, readerscome to understandsomeof the factors affecting the way this student writes, including the demandsof the British academicculture in which he has chosento study, his previous languagetraining, his beliefs about mother-tongueinterference,the style of his discipline (sociology), and the Marxist environmentof his Brazilian university. We also learn about theattitudesof the tutor studyingthe case. Both casescited here illustrate how teachers,working from the inside, can confine their attention to aspectsthat are relevant to the research problem, thus making judgementsas to the boundariesof the particular case. Other kinds 0/ case

For a languageteacher,the learner virtually self-presentsas an object of study: however,there are other kinds of case thatteachersare well placed to investigate.One, of course, is themselves(or their peers): readersare referred to the chapteron diaries for someexamplesof how this might be done, although the possibilitiesextend beyond the application of just one technique. It is in fact worth observing that there are relatively few reported case studies of teachers,so perhaps a shift of focus is due. Beyond this, as we have noted, casesdo not have to be human, exceptin the sensethat everythingwe are dealing with is a human construct.There is only space here to enumeratebriefly three categorieswhich fit both within the definition of researchadoptedin this book, and the notion of 'case'.All three areashave an enormousliterature. They are of particular relevanceto teachersbecause,from a casestudy perspective,they are concernedwith the investigationof local meaningsin specific contexts.

STUDYING CASES

215

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

1. Evaluation 'All evaluation studies are case studies' (Stake, 1995: 95). This is the biggest industry of all in education.In our own field, programmeevaluation has becomeincreasingly important: discussionand exemplification can be found in Rea-Dickinsand Germaine(1992). There is a standard distinction betweensummativeandformative modesof evaluation,the former concernedwith the end-product and the latter with the ongoing processof coursedevelopment.Theseintersectwith a range of available quantitative and qualitative techniques, including measurementscales, questionnairesand interviews. Educationalistsfurther distinguishbetween autocratic, bureaucratic and democratic evaluation (Norris, 1990). An example of a 'democratic'evaluation(in the USA) is afforded by Bread and Dreams(MacDonaldand Kushner, 1983), which is called by the authorsa 'casestudy' in which the word 'case'comesto representboth the school and the study itself From a teacher-researcher point of view, there is a natural affinity with a format that gives them someownershipof the study and that can handle the perceptionsof everyoneinvolved as they change and develop over time. It is for this reasonthat Elliott (1991) seesa close link betweenevaluationcasestudiesand the tenetsof action research. 2. Needsanalysis The analysisof learner needshas received most attention in the field of English for Specific Purposes(ESP) becauseof the largely instrumental natureof thesekinds of courses.Needsanalysesmay be carried out by an external consultant, or by a teacher/coursedesigner. Being concerned with specific groups/individualsin specific contexts,they provide a central manifestationof a 'case' in EFL. A representativeexample can be found in Markee(1986), who conducteda needsanalysisin a complex language situation at Khartoum Polytechnic: like James' Marcos, the Sudanesecasealso raisesissuesof more generalinterestas well as being a fascinating 'intrinsic' study in itself Much needsanalysis has an actionresearchethosbecauseit leadsnecessarilyto the constructionof appropriate programmes,though cases can also contribute to broader needsanalysis researchby modifying and testing out procedural models (see Hutchinson and Waters, 1987). The techniquesmost used are questionnairesand interviews, carried out with learnersat various stages,language teachers,specialiststaff, employersand administrators. 3. Programmedesignand development Thif> sits logically betweenneedsanalysisand evaluation,and can be seen most usefully as a casewhen it is part of a whole package,beginningwith fact finding (needsanalysis),through choice of approachand designpara-:meters(design) to reflection on their appropriacy(evaluation).To extend

216

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS-

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Nunan's(1992b) justification of evaluationas research,programmeconstruction therefore contains questions, data and interpretive analysis. Jordan's(1983) Case Studiesin ELT providesa useful survey of this perspective.

The controversyof casestudy At the end of his casestudy of Marcos,Jameswrites: 'It would be tempting to speculatewhat lessonsone might draw ... I shall, however, resist this temptation.For one thing, it is dangerousto generalizefrom one case study' (1984: 112). It is the intrinsic, naturalistic casestudy in particular that has beenaccusedof failing to meet the conventionalresearchcriteria of generalizabilityand external validity. The debateis very wide-ranging, and we can only scratchthe surfacehere. The one-off casestudy, it is said by its critics, dealsby definition with an individual (languagelearner, programme,child, and so on) in context: howeverinterestingin itself, the researchdesignhas failed to build in features such as samplingor experimentaltreatmentthat would allow extrapolation to a wider population. If a researcheris seekingtypicality or is primarily concernedwith theory-building,then the chargehassomeforce. Atkinson and Delamont - in a paper that refers directly to Bread and Dreams (MacDonald and Kushner, 1983),-are particularly vociferous in their criticisms, claiming that much casestudy researchis a 'methodological cop-out'.They are concerned'aboutwhat they seeas imprecisedefinitions and the vaguenessof the unit of analysis as a 'boundedsystem'; about thenumber of definitions that say what a caseis not; and about an over-focus on ethics and political issues at the expenseof theory and method.They are firmly againsta seriesof one-offs because'theory cannot be left to accumulate'(1986: 251). There are a numberof countersto this kind of critique (and, of course, counter-criticismsas well), and the title of one well-known collection Towards a Scienceof the Singular (Simons, 1980) - is a clear indication of the attitudeof proponentsof naturalisticcasestudy research.Someappear to be against generalizationaltogether.Cronbach,talking about changeability and therefore problems with 'fixing' generalizations,puts it like this: 'The trouble, as I seeit, is that we cannotstoreup generalizationsand constructsfor ultimate assemblyinto a network. It is as if we neededa gross of dry cells to power an engine and could only make one a month. The energywould leak out of the first cells before we had half the battery completed. So it is with the potency of our generalizations'(quoted in Eisner and Peshkin, 1990: l78). At the level of principle, many writers arguefor the value of aggregatedcasestudiesso that (rather like caselaw) a body of knowledge is gradually accumulatedthat can inform further

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

STUDYING CASES

217

studies. This is not the strict replication requirementof the positivist paradigm,though Kennedy(1979), proposinga methodologyfor sampling the attributesof single cases,comesvery close.The teacherresearcher,like James,offering his or her findings to a wider professionalaudience,is essentiallyinviting the collection of further such data which might confirm the outcomeof the original case- Lincoln and Guba's(1985) 'transferability'. Stake (1995) proposes a distinction between 'petite' generalizationsfrom a few casesin a particularsituation, and 'grand' generalization, where a case may contribute either by confirmation or conversely by showing dissimilarity with established findings, thus underminingthem. A casestudy may also help to form questions,not just lead to answers.In our own field it may well be that generalizationsfrom applied linguistics contribute to teachers'uneaseabout the relevanceof such research,and their own daily contact with the idiographic, formulated in casestudy terms,could offer an importantalternativeperspective. Stakealso invokesa notion of 'naturalisticgeneralization'(a conceptnot universallyacceptedeven by researchersof the samepersuasion).It is in a senseanothernamefor learningfrom and applying the work of others,and is to be distinguished from the conventional notion of 'explicated' or 'propositional'generalization.'Naturalisticgeneralizationsare conclusions arrived at through personalengagementin life's affairs or by vicarious experienceso well constructedthat the person feels as if it happenedto themselves'(1995: 85). In other words, readersgeneralizeby virtue of interpreting the case as put to them by the researcher.Operationally at least this is a useful notion for the teacherstudyingand then going public with data from a single case:for us, James'analysisof Marcos' languageis illuminating for our own situation, despite his modest claims as to its transferability. Finally, it is useful to set out the advantagesof qualitative case study researchproposedby Adelman et al. (1980: 59-60), becausein our view they summarizethe main reasonswhy this is a 'user-friendly'researchformat for the teacherto undertake: 1. Casestudy datais 'strongin reality'. 2. Casestudiesallow generalizationsabout an instance,or from that to a class. 3. They recognizethe complexity of 'social truths' and alternative interpretations. 4. They can form an archive of descriptive material available for reinterpretationby others. 5. They are a 'step' to action' (for staff/institutional development; for formative evaluation). 6. They presentresearchin an accessibleform.

218

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Conclusion This chapterhas drawn togethera wide rangeof typesof study under the headingof 'case'.Various ways of defining the essentialattributesof casestudy researchhave beenexplored,looking both at what countsas a 'case' and what methodsof 'study' are appropriateand used. Several ways of classifyingcasestudieswere presented,which demonstratehow broad and useful this researchgenre has been.Researchmethodologyhas usedboth quantitativeand qualitativetechniques,and focused,predecidedquestions as well as 'emergent'designs.The central section of the chapter gave a range of examplesof case studies, first in second languageacquisition research, andsecondin researchaboutlanguageteaching,both concerning individual teachersand learnersand broaderissuessuch as needsanalysis, evaluationand programmedevelopment.Finally the chaptercritically discussedsomeof the crucial issuessurroundingcasestudy, but finished on an optimistic note, reviewing the reasonsfor recommendingcase study methodologyin our field.

Discussionnotes 1. The contextualfactors taken into accountdiffer in the casesof Q and Marcos: they are 'bounded'in different ways. If you undertook a casestudy of an individual learner,how would you go about defining that learneras a 'boundedsystem'?What would you leave in/out? 2. Do you think it sufficient if a casestudy is just of intrinsic interest in a local context?If so, what is its value? 3. Selectinga possiblecase(learner,yourself, colleague,programme, set of materials), sketch out a research design to show what methodsand techniqueswould be most appropriate.

14

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Mixing research methods

Introduction In the course of this book, both by implication and from time to time explicitly, we have touched onthe useof researchmethodsin combination with others.By and large, however,the precedingchaptershaveessentially treated these methodsand techniquesas discrete, in order to bring out their main characteristicsand applications (rather like discussingindividual languageskills before considering ways in which they might be integrated,though the parallel should not be pushedtoo far). Casestudy researchis an obviousexception, beingmore likely than not to amalgamate two or more procedures- interviews,diaries,observations,and so on. Following on, then, from that discussion,this final chapterfocuseson the principles and methodology of mixing research procedures.It is patently impossible,and unhelpful, to cover all possiblepermutationsfor linking two or more methods: the chapter is therefore divided into two main sections.First of all, we look in generalterms at someof the issues raised by methodsmixing and at someof its manifestations.Second,the book goes full circle as it were by picking up on a few of the teachercase studiesoutlined earlier in terms of researchpotential, this time, however, convertingthat potential into actual researchundertaken.The purposeis simply to illustrate just a few of the many possibilities.

Principles in mixing methods The different ways in which researchmethodscan be combined have a numberof implications, which are subsumedhere under two main headings: one concernsscale,the other level. We take eachof thesedimensions in turn. Scale

As with any researchproject, a multimethodapJll"oachcan also occur on a continuumfrom, at one end, a microstudycarried out by an individual to

220

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

a large-scaleundertakingat the other. It would be simplistic and misleading to suggestthat teacherresearchnecessarilytakes place on a smaller canvas,and certainly there.isno equationsuchthat: professionalresearcher= large-scalework teacherresearcher=small-scalework. However, the nature of teacherinvolvement in researchis likely to differ dependingon the scale of its conception,and direct teacherinitiation of researchwill probably fit more comfortablyin the localizedcontextof personalprofessionalexperience. . At the macroend of the spectrum,thereare many instancesof researchin educationand in languageteachingand learninginvolving quite large numbers of peoplethat is both multimethodand multisite. In Johnson'sdefinition: 'Multisite, multimethod, large-scale research refers to large studies in which a team of collaboratingresearcherscollects data from a number of sites and employs a variety of both qualitative and quantitative datacollection and analysis strategies' (1992: 192). Moreover, this kind of researchmay also be linked to evaluationstudies,and have implications for educational policy decisions. There may well be considerable funding involved, with sponsoringbodies having quite definite expectationsas to researchoutcomes.To mix metaphorsas well as methods,the holdersof the purse-stringswill probably want a stake in calling the tune, and it is still sometimesthe casethat a quantitative/surveymethodologyis more likely to attract funding becauseits parametersappearto be more clear-cutand the timescalemore predictablethan with an ethnographic/qualitativeproject. There are many examplesof large-scalemulti method researchthat can quite readily be found in the educationaland applied linguistic/language teachingliterature.Here are just a few of them. 1. HumanitiesCurriculum Project(HCP). 2. Ford TeachingProject(FTP). Both of theseprojects took place in British schoolsin the 1970s,and are describedat length in Elliott (1991). They are notable from our point of view because,although initiated by university-basedteacher educators (HCP by Stenhouse),they were informed by a commitmentto an 'emancipatory' principle that centrally involved teachers(and pupils) in large-scale action research. 3. L2 methodscomparisons,such as the well-known if not entirely successfulPennsylvaniaproject comparing grammar-translation and audio-lingualapproaches. 4. Evaluationof bilingual educationprogrammes. Theseare both reportedin Johnson(1992).

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

MIXING RESEARCH METHODS

221

5. The large-scale, nationally funded Australian Migrant English Program(AMEP), which has over the years involved large numbers of teachersin investigating and evaluating curriculum and policy developmentand change(Burns and Hood, 1995; Nunan, 1988). 6. Large-scalemultimethod multi trait researchhas been conducted by languagetestersto investigatesomeof the core theoreticalconcepts of language testing. Bachman and Palmer (1982) investigated the 'Unitary CompetenceHypothesis' and the different testing methodsused to evaluateit. This was partly motivated by the wish to dissociatetesting particular skills from particular test types in order to get a 'purer' measureof the underlying competence. 7. A very interesting project investigating reading test validity was conductedby Andersonet al. (1991), which related three kinds of information: studentperformanceon test items, verbal reportsof studentstrategiesfor finding the answers,and theoreticalclassifications of the questions in terms of their logical status. The study thereforecombined qualitative and quantitative data in an areatraditionally associatedwith quantitativeevaluationonly. In many large-scaleprojects,of course,teachersare part of the research scenery, investigatedrather than themselvesdoing the investigating. In many more, however, teachershave becomeincreasinglyinvolved, not on the whole as initiators but certainly as active participants,and often in collaborativemodes- academicwith teacher,teacherwith teacherunder the aegis of a project director. (SeeBrindley's 'researchagenda',1990, for an extendeddiscussionof collaborativeresearchpossibilities.) In principle, no method is excludedfrom this kind of research,though it is useful to note hereJohnson'slist of the more commondata-collection techniques, applicable particularly but not only to evaluation studies (1992: 196): • • • • • • •

site visits observations interviews questionnaires test data documents quantitativedata.

Teachersare more likely to have scope to generateresearchfurther along the continuum and away from large-scale,often longitudinal projects using full-time professionalresearchers and frequently involving very time-consuming funding applications to government agencies and

222

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

researchcouncils and the like. It is difficult to be specific about central points on continua,but the kind of medium-scaleresearchprobably using more than a singlemethodcan be illustratedby the following. 1. Replicatedor follow-up studies,such as the accumulationof case study research outlined in Chapter 13, where one teacher's researchreportedin a public forum is picked up on by teachersin other institutions. 2. School-based/local collaborativeaction-researchprojects. Yet smaller-scaleresearchwill be carried out by individual practitioners in their own context. We have met a couple of examplesin the courseof this book, and the themewill be exploredagain in the final sectionof this chapter.Here is a brief list of examples. 1. Using questionnairesand selectiveinterviewingin needsanalysis. 2. Combiningobservationand documentaryevidencein programme evaluation. 3. Using test data, experiment and self-reporting in researchon readingcomprehension. 4. Mixing diaries, interviews, documents,all from different sources, in casestudiesof individuals. 5. Studentcourseevaluationby promptedfocus group discussionat the end of the coursewith diaries written during the course. 6. Using lessontranscriptsand test scoresto investigatethe effects of different error-correctionstrategies. 7. Investigating listening strategiesby qualitative analysis and implicationaI scalingto discoverthe hierarchyof strategicchoice. Level

A mixed-methodapproachto researchraisessomekey questionsas to the level at which that mix takes place, or might legitimately do so. At the broaderparadigm level, there has been a considerabledegreeof controversy: is a researcherwho deliberately chooses,say, a survey approach alongsidenarrativeintrospection,with the explicit aim of exploring reality from different perspectives,doing some kind of epistemologicaldamage? Is it even logically impossible as an approachto research?Some people hold the view that quantitativeand qualitative paradigmsdo not mix well: as we noted earlier, Lincoln and Guba (1985), for example,arguestrongly that they virtually representdifferent world views. Others consider that the distinction is overstated,and that 'thereis ... no necessarylink between choiceof methouand logic of enquiry' (Brannen,1992b: 10). Hammersley (1992) is likewise concernedto 'deconstructthe qualitative-quantitative divide' from a conviction that competing paradigms(indeed, the whole

MIXING RESEARCH METHODS

223

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

notion of a 'paradigm')unhelpfully obscurediversity and complexity. He arguesthat the following seven polarizations(which were touched on in Part 1 of this book) are in fact simplistic. 1. 2. 3. 4. 5. 6. 7.

Q!lalitative versusquantitativedata Natural versusartificial settings Meaningsversusbehaviour Adoption versusrejectionof naturalsciencemodel Inductive versusdeductive Cultural patternsversusscientific laws Idealism versusrealism.

He expresseshis position on the appropriacyof linking paradigmsin the following terms(1992: 52): the distinction betweenquantitativeand qualitative approachesdoes not capturethe full range of options that we face, and ... misrepresentsthe basison which decisionsshouldbe made.What is involvedis not a crossroadswhere we have to go left or right. A betteranalogyis a complex maze where we are repeatedly faced with decisions, and where pathswind back on one another.The prevalenceof the distinctioY} betweenqualitative and quantitativeapproachestendsto obscure the complexity of the problemsthat face us and threatensto render our decisionslesseffective than theymight otherwisebe. At the level of technique,the debatehas raged somewhatless fiercely, and it is more likely to be acceptedthat the numerical and interpretive can in principle be used in combination.As we have seen,qualitative data may in fact be counted and quantified (as in content analysis, for example); and reliability is arguablyenhancedby pluralism in methodology. Triangulation is obviously very important when methodsare combined. Brannen makes the common-sensepoint that, in reality, 'the practice of researchis a messy and untidy businesswhich rarely conforms to the models set down in methodology textbooks.... The cart often comes before the horse.... Researchersare often required to conduct balancing acts betweena number of pragmatic considerations'(l992b: 3-4). She then sets out the different possible relationshipsbetween quantitativequalitative in a practical researchundertaking. In summary form, these are as follows. I. Wherequalitative methodsplaya subsidiaryrole

a. They can act as a sourceof 'hunches'which are then tested by quantitativework. b. They can be usedto pilot researchinstruments. c. They may help in datainterpretation.

224

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

2. Wherequantitativemethodsplaya subsidiaryrole

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

a. They provide quantified backgrounddata to contextualizesmallscaleprojects(e.g. official statistics). b. They may test hypothesesderived from qualitativework. c. They can provide a basisfor casesampling.

3. Wherequalitativeand quantitativemethodshaveequalweight a. In separatebut linked studies. b. Integratedin one study. c. Conductedsimultaneouslyor consecutively. From a data-analysisand interpretation point of view, the last group is both the most interesting and the most problematic. Collapsing for the moment the distinction between3a and 3c, there are published studies which use different methodsfrom different traditions to study the same phenomena.In researchon the two issuesmentioned here as examples, work in the two traditions to a certain extent confirmed the conclusions from the other in one case,and in the other the results were quite discrepant.The first exampleis from training in first languagecomposition skills developmentat college level in the USA. The issue was strategies and kinds of revision used by thesetrainee academicwriters. Faigley and Witte (1981) performed an extensive text analysis of the products of writers gradually refining their texts. This process/productstudy confirmed many of the findings of an earlier study by Sommers(1980) which had usedan extensiveand very detailedinterviewing method. The secondexampleis about the issueof the effectivenessof feedback by teacherson written work producedby L2 writers. A think-aloud study of strategiesfor processingfeedbackby Cohenand Cavalcanti(1990) cast considerabledoubt on the effectivenessof feedbackbecauseof the high proportion of teachers'commentsthat the studentsreportedthey had not understoodor did not know how to handle.The quasi-experimentalstudy by Fathmanand Whalley (1990) mentionedin Chapter to, on the other hand, showed that feedback did have measurablebeneficial effects, and gave a much more positive outcome. Such a discrepancy,whether due to researchmethod,contextor theory, is naturally a spur to further research. An exampleof 3b in the areaof testingresearchwas given earlier in this chapter,Andersonet at. (1991). It is worth mentioningagain in this context becausethe data analysis,which was both interpretiveand statistical, requiredcountingrespondents'frequencyof mention of certainstrategies and comparingthe frequencieswith item analysisof the test questions- to find, for example,what strategieswere used most with easy and difficult items, among other things. Another example whereboth qualitative and

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

MIXING RESEARCH METHODS

225

quantitative methods were used is Tonkyn et a/. (1993). This study of English for Academic Purposesteachers'predictionsof academicsuccess of their studentsin a British university used a variety of data sources: mainly questionnairesfrom studentsand tutors, teachers'reports, and interviewswith academictutors and EAP teachers.The methodof analysis was entirely numerical,counting the frequenciesof resultsfrom the questionnaires,and analysingthe reports and the interviews also by counting frequency of mention of a whole range of factors. These two pieces of researchdemonstrateboth the complicationsand the rewardsof attempting to integrate qualitative and quantitative data and subject both to a commonmethodof analysis. In the end, individual researcherswill haveto make up their own minds about the degreeto which the techniquesthey selectrepresenta particular view of that part of the social world they have chosen to investigate. Questionsof principle are most likely to come into play at the initial stage of conceptionof the research,and techniquesto slot into place as detailed planningunfolds. It is neverthelessimportantthat implicationsof particular methodschoice are fully understood,and that researchdoes not happen 'blindfold' just becauseof familiarity with, or preferencefor, certain techniques. In the next and final sectionwe follow up five of the teachersintroduced in Chapters 1 and/or 5, and describe the researchin which they have becomeinvolved followed by a short commentary.

Teachers'research:somecontinuingcasestudies Table 14.1 provides a summary tabulation of the teachers'working situations, the main features affecting researchpossibilities for them, and, whereapplicable,any researchundertakenso far. We now supposethat eachof them carried on (or started)their research in a numberof different ways: Kenji Matsuda

Findings:

The textbook analysis yielded a very small number of techniques,mainly (i) translation of supposedlydifficult lexical items, and (ii) picture descriptions.

Teaching techniques tried:

(i) English-Englishdictionary explanations,(ii) oral explanationsin English, and (iii) blank-filling of 'learnt' items in texts.

Tests:

Showed that (i) blank-filling was useful at all levels, (ii) more elementarystudentsrecalledmore with translation.

Languageschool 25 hrs p/wk Nursing background

School 18 hrs p/wk plus other duties 35 in class AET nsp colleague

Ministry-initiated syllabus Local teachers'group Six months'UK training

Techniquesfor teaching vocabulary

Mail/ characteristics

Cfll/text features

Researchbegun

Contactsmadewith hospital to collect languagedata

Introductionof ESP Membershipof ARELS and teachers'association Time remission

.-1. Barker

K.Matsutia

Table 14.1

Questionnaires Diaries Interviews

Changein statusof English Writes own materials Resourceslimited

University English for Engineering 16--20 in class/mixed levels

I. Petrot'

Learningstyles Data collection Updating

Works in se\'eralschools Accessto parents,other teachers,school records

Specialneeds SomeESL children Peripatetic Works with groupsand individuals

C. Turner

Studentexpectations Questionnaire

Researchas part of contract Conferences

University EAP Postgraduates

F.Jflnes

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

MIXING RESEARCH METHODS

227

Follow-up:

Some colleaguesteaching at other levels tried the same techniques.Translation appearsto be less facilitating as proficiency increases. KM has been askedto give talks on vocabularyteaching to predepartureteachersgoing to UK on training programmes.

Summaryof research methods:

Coursebook analysis. Frequencytabulation. Tests. Comparativedata.

Ann Barker Data collection: Different kinds of written documentsmadeavailable. Spokendata could not be recordedbecauseof confidentiality, but AB was allowed to 'listen in' and take notes. Analysis:

Languageanalysisof written texts. Somemore informal comparisonwith spokendata.

Follow-up:

Data collectedfrom other medicalpersonnelfor comparison/contrastin termsof languageand content. Resourcebank of pedagogicmaterialsset up in the school for useon its medical English courses. AB has beengiven someremissionfrom teachingin order to do the researchand materials development.She has also given reports at ARELS workshopsand written an article for Modern English Teacher.

Summaryof research methods:

Collection of written data from different professional areas. Data comparison(i) betweencontentareas,(ii) between spokenand written language. Various methodsof text and discourseanalysis.

Irina Petrov Researchidea: Needs analysis, materials development,and subsequent evaluation. Procedure:

Examination of the languageneedsof Engineeringstudents in relation to present study and future jobs. Information is sought from current and past students, and from Engineeringstaff.

228

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Analysis of attitudes to English classesand materials in use. Piloting of someUnits of new materialswritten by teaching team,·and subsequentevaluationby learners. Summaryof research methods:

QIestiohnairesto threedifferent groups. Selectedstructuredinterviews: staff and students. Follow-up evaluationvia questionnairesand interviews. Peerobservationusing new pilot materials.

Carol Turner Researchfocus: ESL children's language developmentover the school year using a small numberof pupils. Research instruments:

Children'swork from specialclass. Selectionof work from regularclasses. Personal teaching diary. Interviewswith parentsand other teachers. of progressvia standardizedtests. Regularmeasurement

Main findings: Language development differs widely between individuals dependingon a rangeof factors, such as amount of exposure to English outside class, family support, degreeof participation,and preferredlearningmodes. Follow-up:

More in-depthcasestudies. Reading/attending in-service training on the language developmentof youngerESL learners. By chance, CT's Local Education Authority has been approachedby the local university to participate in a researchproject on second languageacquisition. CT is able to share her own data with the researchteam, and also to assist in the administration of test instruments designedto elicit certainkinds of languagedata.

Frank Jones

Focus:

PostgraduateEAP students'expectationsand attitudesto study over an academicyear period.

Findings:

There are clear changes,and high/low points, in the attitudes, expectationsand experiencesof overseasstudents as the academicyear progresses.This is verified by replicatedstudiesconductedelsewhere.

MIXING RESEARCH METHODS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Follow-up:

229

Semi-structuredinterviewsover the next year with volunteer students.An attemptis madeto find peoplefrom different countriesand studyinga variety of subjects.

Data analysis: Interviews are transcribedand subjectedto both content and language analysis. The questionnairefindings are confirmed to some extent, but there is also quite sharp differentiation betweenpeople'sexperiencesmaking any generalizationdifficult. Summaryof research methods:

Two questionnairesto large studentpopulations(about

300). Use of questionnairein other institutions. Semi-structuredinterviews('member-checked' with intervieweesfor accuracy).

Commentary

Thesefictional, though not entirely imaginary, researchstoriesillustrate a numberof themesthat havebeendiscussedin variouspartsof this book. Scale

These teachersare involved in researchon various levels of scale. KM looked mainly at his own English classes,but, given normal classsize in his country, he obtaineddata from over 100 learnersfrom three classes,and then involved other colleaguesin short-term trials of the teaching techniques he wanted to focus on. CT amasseda large amount of different sorts of data about a small number of her pupils, from them, from their parents,from herself, and from regular classes.FJ looked at a relatively large numberof peopleenteringand subsequentlyworking in a mediumsized university with an annualentry of overseasstudentsof betweenone and two thousand. Scale brings problems of record-keepingand data safety, but mainly of time-managementand administrationof the research project. Collaboration

Someof our group have been ploughing their lonely researchfurrow, but some have beenable to involve colleaguesin the researchin various ways. KM persuadedother teachersin his own and other schoolsto tryout his techniquesand tell him the results. IP was working as a member of a materials writing team, contributing in particular the needsanalysis but also piloting draft materialsin the classroom.CT's researchcame to the

230

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

attentionof someresearchers,in the local university and she discussedit with them while collaboratingwith them in their own project. Research collaborationfor theseteacherscameto mean two things: peerevaluation and mutual contributions,and sharingdata. Mixi11g methods None of theseteachersusedonly one data sourcein their research.All of them contrived to combine several kinds of data in the one project. AB looked at written documentsfrom medical histories and performedcontent and linguistic analysis,made her own notesof caseconferences,and looked at the way medical, nursing and administrativestaff talked about and wrote about their patientsin the hospital. IP used questionnairesand interviews in the needsanalysisand arrangedobservationof each other's classesusing the new materialsby the project team. Communication Severalof our group have already'gonepublic' with their research,in the form of talks at teachers'workshopsand even of a publishedarticle. IP's team is not so far advancedas to be able to publish articles, butsheis hoping to get funding to attendan IATEFL conferenceat which she will give a paper. FJ has attendedconferenceswhere other researchersin the area have talked about their work, and when the comparisonbetweenthe questionnaire and the interview findings is completeit will be written up as an article, perhapseven in the form of a higher degreethesis.AB has already publishedan article in a relevant journal and is hoping to do anotherone including someevaluationof the new materials.In theseways, their findings can be brought to a wider audienceand can be subjectedto critical analysisby their peers. Description. evaluationand measurement None of theseteachershas chosenan explicit experimentalor even quasiexperimentalapproach.The nearestto such a designwas KM, measuring different outcomesfollowing different teaching treatments.For several, their final data analysis involved quantitative methodsas well as verbal data in various forms. All of them were attemptingsomekind of description of aspectsof their own teachingcontexts,either monitoring or evaluating a change in teaching practice; performing a needs analysis and developingnew materials;attemptingto uncoverthe rangeof attitudesand expectationunderlying performancein tertiary education;or investigating languagedevelopmentin young learners.Their researchbenefitedin each casefrom their own privileged knowledgeof their teachingcontexts,but

MIXING RESEARCH METHODS

231

the researchalso taughtthem a greatdeal about which previouslythey had held inaccurateassumptions.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Action research? Only one, CT, was doing anything resemblingfull-scale action research,as representedby the action-researchspiral (Chapter 2). Although she was only following a small numberof pupils, she was doing so over quite a long timespan,a whole year, and looked at different data sourcesas they became relevant. However, someof the other projectswere using action researchin the weaker sense,and one feature in particular, the emergenceof research issues,was a characteristicof severalof them. The researchquestiondeveloped through the courseof the researchparticularly for her, for AB, for IP and for FJ. As the issueat the heartof eachproject becameclearerand they beganto seedifferent ways of looking at it, so the methodsthey choseto follow up the new leadschanged.FJ turned from surveysusing questionnaires to interviews; CT switched to in-depth casestudies;IP moved from needs analysis to evaluation to collaborativeevaluationby peers;AB moved outwards into other areasof medical discourseto illuminate her original problem and widen the scopeof the materialsresourcebank.

Feedforward Theseteachersgot involved in researchfor a numberof reasons:in response to an employmentrequirement,for personalprofessionaldevelopment,to introduceinnovationsin their teachingmethods.AB and FJ had time made available for their researchthrough contract and remission of teaching duties; KM had his interest and motivation stimulatedby a Governmentsponsoredperiod at a university in the UK; IP and CT were respondingin their own ways to a needthey had perceived.Their researchand the results brought both personalbenefits in terms of professionaldevelopmentand increasedinvolvementin the issues,and also tangible 'feedforward'into the teachingcontexts:things beganto change.In three institutions, new teaching materialsor new techniquesfor classroomexploitation of materials,and new workplans, were availableand were gradually being taken up by other colleagues.CT's LEA began to take greater interest in her special-needs work, and a possibility of greaterfunding is on the horizon. FJ's university is reviewing its policies for recruitmentand support of international studentsand he hasbroughtthe researchto their attentionalready.

Conclusion This chapterhas pulled together for discussionand examinationvarious suggestionsmadein other chaptersabout the true natureof the difference

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

232

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

between normative and interpretive, quantitative and qualitative approaches.It has explored, with examples,albeit not comprehensively, practicalways of combiningdifferent datasourcesand methodsof analysis in the interestsof completenessof description,greateraccuracyand sensitivity of interpretation.Some of the difficulties of mixing methodsand triangulation have beenaddressed.Finally, we took a last 10Qk at someof the fictitious teacherresearcherswhoseresearchcareerswe have beenfollowing off and on throughoutthe book, and we haveattemptedto personalize the discussionby narrative.Of course,in the real world, the positive outcomesdescribedmay not always happen;remissionof time off teaching to do researchmay be cancelled becauseof adverseeconomic circumstancesor trading position, and staff shortage;interestingresults may be 'shelved' by authorities and no lasting developmentallowed to occur. However, thesecasehistories show what can happen,and, since they are we know they do happen. basedon real acquaintances,

Discussionnotes Taking a teachingsituationwith which you are directly or indirectly familiar, and using the following headingsjust as guidelines,sketchout a plan for a small-scaleresearchproject that would take it from inception atleast to the dataanalysisstage: • issue,problem,focus of interest • source(s)of data (tests, learners, text, colleagues,other written data,and so on) • overall rationalefor and approachto researchdesign • preferred/appropriate researchinstruments • methodof dataanalysis • timescale • anticipatedfindings • possiblefollow-up • disseminationof findings.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Conclusion

The primary intention of this book has been to layout the range of approaches,methodsand techniquesavailable for researchby teachersof English as a foreign/secondlanguage, to give some indication of the underlying philosophies,to indicate what a rich menuof possibilitiesthere is, to open up what 'doing research'might actually mean, and give some practical proceduresfor starting, generatingresearchquestions,looking at the history of issues,choosing methods, and bringing the work to the attentionof a wider public. We have illustrated this in three ways. First, we have describeda substantial number of examplesof researchby teachers,mostly but not exclusively of English as a foreign language,which have appearedin print, commenting on the designs and methods used, and on the outcomes. Second,we haveU6ed examplesby professionalresearchers in 'appliedlinguistics', which serve to demonstratethe methodsin operation,often in large-scalework, but also the difficulties and shortcomingsof such work from the practisingteacher'spoint of view. Cross-fertilization,as we shall argue below, is, however, preferableto mutual ignorance.Third, we have followed a set of fictional but not imaginary teachersin action from their starting points in their contextsand their often highly constrainedpossibilities for doing research through some of their crucial decisionprocessesto when their researchis nearly completedand they are looking for how to develop the next stage, and how to communicateit to other interestedcolleagues.We intend this narrative strand not to be comprehensivebut to personalizethe more generaland inevitably theoreticaldiscussionof researchphilosophiesand methods.

The role of researchby teachers A very important themein the book has beenthat of researchby teachers being able to contribute to various kinds of practical outcomes,for example both the creation of knowledgeand the development,monitoring

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

234

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

and maintenanceof innovations,in teachingmethodsas well as in classroom organization,curriculum development,materials and institutional cultures.The importanceof this contribution by teacherscan be seenat severalpoints: in personaland professionaldevelopmentof the teacher,in bringing to bearthe teacher'sinside knowledgeon problemsthat arise,and in the communicationof that specialinformation to the wider professional communityof teachersand researchers.By developingthe right questions, by taking up issuesthat arisedirectly from their experienceof the interaction with learners,the teacherresearcherscan expandthe generalknowledge base in relevant ways, both contributing to and challenging the conventionalwisdom.

Appropriatemethodologies Lest it be thought that such considerationsare only relevant for a small group of privileged people in one native-speakerenvironment,we have madea point of illustrating many of our points with examplesfrom countries other than Britain. This raisesan importantissuewhich has receiveda greatdeal of discussion recently, the situation where methodologiesare imported or exported from one educationalenvironmentto another. 'Appropriate methodology' as discussedby Holliday (1994) and by Budd (1995) refers mainly to the reaction against the wholesale adoption of (usually) communicative methodologyin countriesother than Britain, whereit was developedin the early 1980s. Imported methodologies,especially when the traffic is from countries such as Britain and North America ('BANA') into countries whereEnglish is taughtas a foreign languagein tertiary, secondaryand primary state education (,TESEP'), often do not match local conditions in terms of teachertraining, usual levels of proficiency among the teachers, cultural attitudesto certain kinds of activities, acceptableroles for teachers and learners,class sizes, and so forth. Despite the difficulties involved in drawing Holliday's line, it is reasonableto ask how curriculum and methods developmentoccurs in these various contexts.This is not a simple task; Clegg (1995: 133) shows how different the various contexts for language teachingare in just one country (Britain), and how difficult communication is betweenthem, and what a long time coordinateddevelopmentactually takes.There is a role here for researchwhich is home-grownand sensitive to local context, initiated by teachers,conductedty teachersboth working alone and in collaboration with others. Such researchcan aid, accelerate and give direction to the developmentof methodology and institutional forms for languageteachingwhich are relevantto thoselocal contexts.This is as true of Britain as it is of any other country. Holliday makesthis point himself (1994: 9) but does not indicate how teachersin TESEP contexts can be given the skills necessaryto accomplishsuchresearch.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

CONCLUSION

235

Of course, as Holliday (1995) points out, the very idea of teacher researchin the senseof this book, and the associatedphilosophy of the reflective practitioner, may itself be a BANA concept,and may have to fight hard to be acceptedin those contextswhich also find other BANA ideas like the communicativeapproachuncongenial.However, as he also points out, BANA countriesdo not have a monopoly on thoughtful practising teachers,and many teachersin thosecontextshave as much to contribute as BANA teachers(including, of course, expatriatesworking on British Council, Overseas Development Administration, and United StatesInformation Service projects overseas).The opportunity for such contribution, however, dependson the vocabulary and assumptionsof teacherresearchnot being perceivedas either yet anotherimport or imposition, or as yet anotherforeign (or indeedalien) exclusivediscourseto be receivedcourteouslybut not collaboratedwith. An interestinginsight into the processof introducing the methodsof action researchand qualitative researchinto schoolsin EasternEurope(specifically the Czechrepublic) is given by Buresova(1996), who reports the gradualbreaking-downof traditional attitudesto researchboth in the academiccommunity and among the languageteachers,to the benefit of applied research.However, such developmentshave to arise from a local perceptionof relevanceand local accessto the thinking and the methodsof doing suchresearch,ratherthan be 'flown in' by foreign 'experts'.

Developmentof criteria We intend the book to be read as a contribution, not a final comment,to the developmentof appropriatecriteria for researchin English language teachingand applied linguistics. Fields are typically defined by topics and by methodsof enquiry. They developas the professionalproblemsmultiply, as the knowledge gained through researchsuggestsnew leads for research,as methodsfor researchopen up new ways of obtaining goodquality data.As we haveseen,availableresearchmethodologiesrangefrom the precise,interventionist,objective,context-freeexperimentalparadigm right across to the descriptive, interpretive, context-boundnaturalistic paradigm.We have discussedteacherresearchin languageteachingusing elementsfrom the whole range, and mixing methods.As the field develops, so will the practitionerscome to agreeon criteria for designingand acceptingsuch research.The classicalcriteria are set out in Chapter4: but as we saw there, new kinds of researchhave to develop their own argumentsaboutthe importanceof thosecriteria and how they are worked out in actual practice. One dimensionon which there is clearly movementis that referredto by Van Lier (1989) as 'openness',referring to ethnography. As Schecterand Ramirez (1992)note, teacherresearchtendsto downplay

236

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

aspectssuchas literaturesearchand generalizability(notably thosecriteria looking outside the immediateenvironmentexcept for applicability) but requiresopennessin the senseof clarity of decisionprocess,and narrative own involvement in the processof obtainabout the teacher/researcher's ing the knowledge.

'Popularity'of varioustechniques In this contextit is interestingto note how teachersreact when askedwhat kinds of researchthey found most congenialand what they feel they could do in their typical teachingcontext. In a small-scalesurveyof participants in a researchmethodsclass for MA studentson a coursefor experienced teachers,one of the authorsfound that the 'top four' topics were: • • • •

classroomobservation questionnaires casestudies ethnography.

Perhapssurprisingly, action researchfeatured fairly low in rank order, down with experimentalmethods.This is probably becauseit is perceived as very time-consuming and large-scale in nature. Research training coursesclearly have to bear in mind two setsof needs,which do not necessarily overlap: thoseof peoplewho are confrontedperhapsfor the first time with difficult numerically evaluated experimentsand quasi-experiments (becausesuch a large part of the applied linguistics and secondlanguage acquisition literature uses these methods),and also those of the teachers when they return to work and wish to make their contribution to the field and do empirical work to introduceinnovation in their own teachingenvironments.Researchby teachersin situ does not have to be constrainedto use only a narrow range of techniquesfor want of guidelines in how to expandthat range.We have tried to show that a wide spectrumof fieldwork techniquesis available,and a matchingrange of techniquesof analysis,in many caseswith useful off-the-pegcomputationalaids for both quantitative and qualitativeevaluation.

Teacherresearchand higher-degreeresearch At several points in our text we have mentionedthe different demands, and criteria, of teacher researchand researchperformed in pursuit of higher academicdegreessuchas MA, MPhil and PhD. Apart from scale(a PhD thesisis after all a book-lengthtreatise),thereare clearlyother issues at stake in the comparison,which bear upon the notion of compatibility between the two. One is the role of scholarshipand literature review.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

CONCLUSION

237

While we have argued, under the heading of cross-fertilization, that knowledgeof the history of issuesand of previousways of tackling them is beneficial to teacherresearch,it is also evident that 'university' research demandsa particularstyle and format for such a review, and the reasonis not solely derived from the principles of good research;it also has to do with entry to a particularprofessionand demonstrationof a skill of scholarship that is expectedof membersof that profession.The academiccommunity - in our view rather unfortunately - has tended to promote a particular view of research,namely the dominant positivist and experimental paradigm,at the expenseof others.Teachershave to demonstrate other skills in order to be acceptedin their profession. Another issueis the commonplacedislocationfrom context usually suffered by postgraduatestudentsdoing research,especiallyif they are from overseas.Their researchprojects,often despitetheir own first inclinations, cannot be basedwithin their own teachingcontextsbecausethey cannot stay in those contexts while studying for their degree. Consequently researchdegreesare frequently different in aim and conception to the kind of researchconsideredmainly in this book. This is not necessarilya healthy situation; however, there are developmentsin the university system which allow researchby practising teachersto be counted towards degreeawards:part-time study incorporatingaction researchin the teachers' schools,distancelearningwith local supervision,split-site registration for PhD work carriedout in anothercountry. Increasingly,communication betweenuniversity and degreeaspirant usesmodern electronic meansfax and e-mail. Suchinteractionbetweendifferent elementsof the profession can only increaseinterdependencewithin the researchcommunity, and will slowly help to establishthe kind of developmentof appropriate researchmethodology,criteria, meansof disseminationand mutual cooperationenvisagedby this book.

Evaluationsof teacherresearch Earlier in the book we referred to Shecterand Ramirez's(1992) useful article describing a meta-analysisof a number of teachersengagedin action researchof various types. That article, however,did not attemptto evaluatethe effectivenessof the activity. There have beenremarkablyfew suchevaluationsin the languageteachingliterature; there has beena great deal of enthusiasmfor promotingsuch involvementin research,and those involved have often commented,as do Schecterand Ramirez'steachers, very positively on the personaland professionaldevelopmentprospects that they have experiencedas a result of doing the research.A study by Morrow and Schocker(1993) demonstratedjust how powerful the involvement of teacherson a teacherdevelopmentcoursein the evaluationof the

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

238

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

courseby various meansbecame:involvementthrough group discussions, individual interviews and written feedbackpromoted reflection, interest and sheeramountsof time being devotedto the activities and the course, even at four o'clock on a Friday afternoon! But involvementand positive attitudes on a short course are one thing, evaluation of a long-term researchproject another.One paperthat attemptedsuchan evaluationwas Roberts' 'Evaluating the impacts of teacherresearch'(1993; see Chapter 2). This study reports on two teacher-research projects, both involving collaborationwith a university in different ways. In one, teachersin state schoolscollaboratedin classroomresearchas part of the wo.rk they needed to do to qualify for MA degrees;in the other, ELT staff collaboratedon a mixed-ability teaching project doing action researchin a state school in Israel. In both projects, there was considerablesupport and enthusiasm among the participants for doing researchas a means of professional development,but the picture also containedsome difficulties. For example, whereteachersare working towardsdegreeawards,thereare increased and for some intolerable pressuresof workload; there are also difficulties with colleagueswho sometimesresentthe addedstatustheir collaboration might bring not to them but to those engagedin the research;there are frustrationswhere the researchis not seento be taken up and used in the schoolafter it hasbeencompleted. Robertsdrew severalinterestingconclusions,which are worth repeating hereby way of closing the circle: that action researchdoespromotereflectivity, but is not the only way of doing this; that researchalone will not changecontexts,but that opportunitiesfor changenecessarilydependalso on the power of the teacherresearchersto implement the change; that doing researchgives teachersnew ways of respondingto problems; and that, despite the frustrations, most teacherswho engagein this activity developvery positive attitudesand do experience(and report) the senseof personaland professionaldevelopmentthat has been referred to several times in this text. Involvement can itself have strong effects even if the researchis not completed. Naidu et al. (1992) write about an action-researchproject which decisively affected the participatingteachers,although,at the time their article was written, they had not completedthe observationphaseof their plan nor analysedany of the field recordsof the observationsthey had performed.The article documentstheir use of structuredtalking sessions involving the four teachersdiscussingin a plannedand disciplined fashion their experienceof heterogeneityin teaching English in large Classes.Arranging and maintaining the discussionsessionsin the middle of full teachingloads was a non-trivial problem in itself. This led to a new, sharedunderstandingof the issuesin their teaching practices,which in turn led to an observationand analysisplan. However, they were so struck

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

CONCLUSION

239

by the effect on themselvesof the discussionsthey decidedto write that up as an independenttopic. Incidentally, they comment that, although they had some familiarity with work elsewhereon the issue of large classes,they found that consideringpoints arising from the work of others at that juncture clouded their developingperceptionsof their own issues, and thoughtthat the properplace for considering'the literature' would be after, not before, they had formulated and analysedtheir own points of view. This commentis congruentwith, and perhapshelps to explain, the attitudesto the history of issuesin other people'swork which have been mentionedabovein connectionwith researchwork by teachers. Naturally, in evaluatingthe successof researchby teachersin promoting and implementinginnovation one should not forget that the capacity for any kind of researchto do this is limited. Indeed,the pressuresfor conservatism and againstchangein educationare strong.The applicationof any kind of researchis subjectto political pressureswhich may emanatefrom students,colleagues,senior administrators,external agents(such as producersand suppliersof teachingmaterialsand other resources)and wider structureswith power over the decisionsof the particularinstitution.

And finally ... A teacheronce queried,after readingcopiousamountsof researchin language teaching and about language-research methods, whether research can ever 'get to the bottom' of the complicationsin a teachingsituation or whetherteacherswere condemnedto 'shootin the dark' for ever. To claim that researchwill enlighten everything would be too positivistic; to claim that the normal processof questionsthrowing up further questionswas eternal darknesswould be too nihilistic. What is clear is that research, properly conducted,with due regard for context and for the development of theory, using appropriatemethodsfrom the full range available,asking carefully formulated questions,interpretingwith imagination but with an honest judgementof plausibility, will increaseour understandingof the many issuesin languageeducationand enableus to produceprofessional answerswith confidenceto professionalproblemsas they arise.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

This page intentionally left blank

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Appendix

Addressesof organizations British Associationfor Applied Linguistics c/o Multilingual Matters Frankfurt Lodge ClevedonHall Victoria Road Clevedon Avon BS21 7SJ

UK

The British Council Medlock St ManchesterMIS 4PR

UK

Centrefor Information on LanguageTeaching(CILT) 20 Bedfordbury London WC2N 4LB

UK

ClassroomAction ResearchNetwork (CARN) Schoolof Education University of EastAnglia Norwich NR4 7TJ

UK

EducationalResearchInformation Center(ERIC) Clearinghouseon Languagesand Linguistics Centerfor Applied Linguistics 118 22nd StreetNW WashingtonDC 20037 USA

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

242

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

InternationalAssociationof Teachersof English as a ForeignLanguage (IATEFL) 3 KingsdownChambers Kingsdown Park Tankerton Whitstable KentCT52DJ UK TESOL 1600CameronSt, Suite 300 Alexandria,VA 22314-2751 USA

Electronicaddresses Birkbeck CollegeVirtual Library http://www.bbk.ac.uk.Departments/AppliedLinguistics/VirtuaILibrary.html British Council http://www.open.gov.uk/bc/bcchomOl.html ERIC e-mail: [email protected] IATEFL http://www.man.ac.uk/iatefl TESL-EJ [email protected] Type the following message: SUB TESLEJ-L first namesecondname TESL-L [email protected] or [email protected] Type the following message: SUB TESL-L first namesecondname TESOL e-mail: [email protected]

Journals AppliedLinguistics Oxford University Press Walton St Oxford OX2 6DP UK

APPENDIX

EL Gazette

10 Wrights Lane London W8 6TA

UK

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

ELTJournal

Oxford University Press Walton St Oxford OX2 6DP

UK

Englishfor SpecificPurposes

PergamonPresspic I leadingtonHill Hall Oxford OX3 OBW

UK

InternationalJournal ofAppliedLinguistics

Novus Press POBox748 Sentrum NI066 Oslo Norway International ReviewofAppliedLinguistics

Julius Groos Verlag Postfach10 24 23 69014Heidelberg Germany LanguageAwareness

Multilingual Matters Frankfurt Lodge C1evedonHall Victoria Road Clevedon Avon BS21 7SJ

UK

LanguageLearning

178 Henry S FriezeBuilding 105 SouthStateStreet Ann Arbor Michigan 48109-1285 USA LanguageTeaching

CambridgeUniversity Press The EdinburghBuilding ShaftesburyRoad CambridgeCB2 2RU

UK

243

244

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

LanguageTeachingResearch

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Arnold A memberof the HodderHeadlineGroup 338 EustonRoad London NWI 3BH UK

Linguisticsand LanguageBehaviorAbstracts

SociologicalAbstractsInc. 7428TradeSt SanDiego CA 92121-2479 USA

Modern English Teacher Macmillan 25 EcclestonPlace London SWI W9N'F UK Practical English Teacher

Pilgrims LanguageCourses 8 Vernon Place Canterbury KentCT13NG UK

Prospect NCELTR MacquarieUniversity 2109 Sydney Australia RELCJournal

RELC 30 OrangeGrove Road Singapore

SemndLanguageReseanh Arnold A memberof the HodderHeadlineGroup 338 EustonRoad London NWI 3BH UK Studiesin SemndLanguageAcquisition CambridgeUniversity Press The EdinburghBuilding ShaftesburyRoad CambridgeCB2 2RU UK

APPENDIX

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

5jl stem ElsevierScienceLtd The Boulevard Langford Lane Kidlington Oxford OX5 1GB

UK

TESOLQuarterly TESOL 1600CameronSt, Suite 300 Alexandria,VA 22314-2751 USA

The TeacherTrainer

Pilgrims LanguageCourses 8 Vernon Place Canterbury KentCTl3NG

UK

245

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

This page intentionally left blank

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

References

ADELMAN, C. 1991: Talk on action researchgiven at the Centrefor Applied Researchin Education(CARE), University of East Anglia (March). JENKINS, D. and KEMMIS, S. 1980: Rethinking casestudy: notes from the second Cambridgeconference.In Simons(1980: 45-61). ALDERSON,). C. 1991: Bandsand scores.In Aldersonand North (1991: 71-86). and NORTH, B (eds) 1991: Languagetesting in the 1990's: the communicativelegacy. Review of EL T, I (I). London: Macmillan, Modern English Publicationsin association with the British Council. ALL WRIGHT, D. 1980: Turns, topics and tasks: patternsof participation in language learningand teaching. In Larsen-Freeman,D. (ed.), Discourseanalysis in secondlanguageresearch.Rowley, Mass.: Newbury House,165-87. 1988: Observationin the languageclassroom.London: Longman. 1993: Integrating'research'and 'pedagogy':appropriatecriteria and practical possibilities. In Edgeand Richards(1993: 125-35). and BAILEY, K. M. 1991: Focus on the language classroom. Cambridge: Cambridge University Press. ANDERSON, N. )., BACHMAN, L., PERKINS, K. and COHEN, A. 1991: An exploratorystudy into the constructvalidity of a readingcomprehensiontest: triangulation of datasources.LanguageTesting8( I), 41-66. APPEL,J. 1995: Diary ofa languageteacher.Oxford: Heinemann. ARNDT, v. 1987: Six writers in searchof texts: a protocol-basedstudy ofLI and L2 writing. ELTJournal 41(4), 257-67. ATKINSON, P. and DELAMONT, S. 1986: Bread and dreamsor bread andcircuses?A critique of 'casestudy' researchin education.In Hammersley(1986: 238-55). BACHMAN, L. and PALMER, A. 1982: The constructvalidation of the F.S.1.oral interview. LanguageLearning 31(1), 67-86. BAILEY, K. M. 1983: Competitivenessand anxiety in adult second languagelearning: looking at and through the diary studies.In Seliger, H. W. and Long, M. H. (eds), Classroom-orientedresearch in semndlanguage acquisition. Rowley, Mass.: Newbury House,67-103. 1990: The use of diary studiesin teachereducationprograms.In Richardsand Nunan (1990: 215-26). and OCHSNER,R. 1983: A methodologicalreview of the diary studies:windmill tilting or social science?In Bailey, K. M., Long, M. H. and Peck, S. (eds), Secondlanguageacquisitionstudies.Rowley, Mass.: Newbury House;188-98.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

248

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

BASSEY, M. 1986: Does action researchrequire sophisticatedresearchmethods? In Hustleroet.al. (1986: 18-24). BEEBE, L. M. (ed.) 1988: Issuesin secondlanguageacquisition: multiple perspectives.Rowley, Mass.:Newbury House. BELLELI, L. 1993: How we teach and why: the implementationof an action research model for in-servicetraining. In Edgeand Richards(1993: 65-75). BERTHOFF, A. E. 1987: The teacheras REsearcher.In Goswami and Stillman (1987: 28-38). BIALYSTOK, E. 1990: Communicationstrategies.Oxford: Blackwell. BLACKIE, D. 1995: The Internet: what's in it for an EFLer? IATEFL Newsletter129, 10-11. BLOCK, E. 1986: The comprehensionstrategiesof second languagereaders. TESOL Q;larter(j, 20(3), 463-94. 1992: See how they read: comprehensionmonitoring of L1 and L2 readers. TESOL Q;larter(j, 26(2),319-43. BLUE, G. (ed.) 1993: Language, learning and success:studying through English. London: Macmillan. Modern English Publicationsin associationwith the British Council. BOLSTER, A. S. 1983: Towardsa more effective model of researchon teaching.Harvard EducationalReview53(3), 294-308. In Goswamiand Stillman BOOMER, G. 1987: Addressingthe problemof elsewhereness. (1987: 4-12). BRANNEN, J. (ed.) 1992a: Mixing methods: qualitative and quantitative research. Brookfield: A vebury. 1992b: Combining qualitative and quantitative approaches:an overview. In Brannen (1992a:3-37). BRINDLEY, G. 1990: Towards a researchagendafor TESOL. Prospect6(1), National Centre for English Language Teaching and Research, Macquarie University, Sydney,7-26. 1991: Assessingachievementin a learner-centredcurriculum. In Alderson and North (1991: 153-66). BROWN, C. 1984: Two windows on the classroomworld: diary studiesand participant observationdifferences.In Larsen, P., Judd, E. and Messerschmidt,D. (eds), On TESOL '84: a brave new world/or TESOL.WashingtonDC: TESOL, 121-34. BROWN, J. D. 1988: Understandingresearchin secondlanguagelearning: a teacher'sguide to statisticsand researchdesign. Cambridge:CambridgeUniversity Press. BRUMFIT, C. and MITCHELL, R. (eds) 1989: Research in the language classroom. London: Modern English Publicationsin associationwith the British Council. BRYANT, I. 1993: 'These are the facts': some informal ideas about practice. Action Researcher(introductoryissue),3-4. BUDD, R. (ed.) 1995: Appropriate methodology:from classroommethodsto classroom practice. TheJournal ofTESOLFrance, 2(1). BURESOVA, V. 1996: Any takers for classroom research? Teacher Development. Newsletterof the IATEFL TeacherDevelopmentSIG, No. 31,8-10. BURNS, A. 1995: Teacherresearchers:perspectiveson teacheraction researchand curriculum renewal.In Burnsand Hood (1995: 3-20). and HOOD, S. (eds) 1995: Teachers'voices: exploring coursedesign in a changingcurriculum. National Centre for English LanguageTeaching and Research,Macquarie University, Sydney. BURTON, J. and MICKAN, P. 1993: Teachers'classroomresearch:rhetoric and reality. In Edge andRichards(1993: 113-21).

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

REFERENCES

249

CALDERHEAD,J. (ed.) 1987: Exploring teachers'thinking. London: Cassell. CAMPBELL, D. T. and STANLEY, J. C. 1963: Experimentaland quasi-experimental designsfor researchon teaching.In Gage,N. L. (ed.), Handbookofresearchon teaching. Chicago:RandMcNally, 171-246. CANE, B. and SCHROEDER, C. 1970: The teacher and research. Slough: National Foundationfor EducationalResearch(NFER). CARR, W. and KEMMIS, S. 1986: Becoming critical. London and Philadelphia: The FalmerPress. CARRELL, P. L. 1989: Metacognitive awarenessand second language reading. The Modern LanguageJournal 73(2), 121-34. and EISTERHOLD, J. c. H. 1983: Schema theory and ESL reading pedagogy. TESOLQuarter6'17(4), 553-73. PHARIS, B. G. and LIBERTO, J. C. 1989: Metacognitivestrategy training for ESL reading. TESOLQuarter(y23(4), 647-78. CARROLL, B. J. and WEST, R. 1989: ESUframework: performancescalesfor English languageexaminations.Harlow: Longmanand English SpeakingUnion. CASANA VE, C. P. and HUBBARD, P. 1992: The writing assignmentsand wrtting problems of doctoral students: faculty perceptions, pedagogical issues, and needed research.Englishfor SpecificPurposes11(1), 33-49. CHAMOT, A. U. 1995: The teacher's voice: action research in your classroom. ERIC/CLL NewsBulletin 18(2), Washington,DC: Centerfor Applied Linguistics. CHAUDRON, C. 1988: Second language classrooms: research on teaching and learning. Cambridge:CambridgeUniversity Press. CLEGG, J. 1995: Languageeducationin schoolsand the role of British EFL. In Budd (1995: 133-46). COHEN, A. D. 1984: On taking languagetests: what studentsreport. Language Testing Vo!' 1, 7~81.81 1992: Languagelearning. Rowley, Mass.: Newbury House. 1995: Verbal reports as a sourceof insights into second languagelearning strategies. Papergiven at the University of Essex. and APHEK, E. 1981: Easifying secondlanguagelearning. Studiesin SewndLanguage dicti(Jnar), 3(2), 221-36. and CA V ALCANTI, M. L. 1990: Feedbackon compositions: teacher and student verbal reports.In Kroll (1990: 155-77). COHEN, L. and MANION, L. 1989: Researchmeth(Jdsin edufation. London: Routledge (3rd edition). to secondlanguagelearning. In COOK, V.). 1986: The basisfor an experimental approach Cook, V. J. (ed.), Experimental approaches to semnd language learning. Oxford: Pergamon. CORTI, L. 1993: Using diaries in social research. S(Jci,,1 Research Update, Issue 2. Guildford: University of Surrey. CRIPER, C. and DAVIES, A. 1988: Researchrep(Jrt I (i): ELTS validation project report. London and Cambridge:The British Council and University of CambridgeLocal ExaminationsSyndicate. CROOKES,G. 1993: Action researchfor secondlanguageteachers:going beyond teacher research.AppliedLinguistics 12(2), 13~. 44 CRYSTAL, D. 1991: A dicti(Jnar), I!rlinguistin and ph(Jnetics.Oxford: Blackwell (3rd edition). 1995: The Cam!mdge emYc/(Jpaedia(Ir the English language. Cambridge: Cambridge University Press.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

250

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

DADDS, M. 1995: Finding your 'self in action research.Paperpresentedat the second TeachersDevelopTeachersResearchconference,Cambridge,January1995. DAY, R. R. 1990: Teacherobservationin secondlanguageteachereducation.In Richards and Nunan(1990: 43-61). DENZIN, N. K. 1970: The methodologiesof symbolic interactionism:a critical review of researchtechniques.In Stone,G. P. and Faberman,H. A. (eds), Social p~ychology through interaction. Waltham,MA: Ginn-Blaisdell,447-65. sociological 1978: The research act: a theoretical introduction to sociological methods. New Yor:k: McGraw-Hili (2nd edition). and LINCOLN, Y. (eds) 1994: Handbookof qualitative research.ThousandOaks, CA: Sage. DOUGHTY; C. and PICA, T. 1986: Information gap tasks: do they facilitate secondlanguageacquisition?TESOLQuarter(y 20(2), 305-25. EDGE, ). and RICHARDS, K. (eds) 1993: Teachersdevelop teachers research. Oxford: Heinemann. EFL SERVICESLTD., 1992: Studentstalking. Elsworthy: EFL ServicesLtd. EISNER, E. and PESHKIN, A. (cds) 1990: Qualitative enquiry: the continuingdehate.New York: TeachersCollegePress. ELLIOTT, ). 1991: Action research for educational change. Milton Keynes and Philadelphia:Open University Press. and ADELMAN, C. (eds) 1975: Ways of doing research in one's own classroom. Cambridge:CambridgeInstitute of Education. ELLIS, M. 1993: Introducing reflective practice: an experimental project in Silesia, Poland.In Edgeand Richards(1993: 102-12). ELLIS, R. 1985: UnderstandingSUlinti language acquisitilln. Oxford: Oxford University Press. 1990: InstructedsUllnd languagel/t'quisitifm. Oxford: Blackwell. 1994: The stul6' IIfseamdlangullge Ofquisitilln, Oxford: Oxford University Press. ELTON, L. 1995: Researchand teaching in higher education. In Gilpin, A. (cd.) Proceedingsf!(the jilUrth annual seminllr, Institute for English TeacherDevelopment in Higher Education,p. 8. ENRIGHT, L. 1981: The diary ofa classroom.In Nixon (1981: 37-51). ERICKSON, F. 1986: Qualitative methodsin researchon teaching. In Wittrock, M. C. (ed.),A hllndbllllk IIfresearch lin teOfhing. New York: Macmillan, 119-61. and SCHULTZ,). 1981: What is a context?Someissuesand methodsin the analysisof social competence.In Greenand Wallat (1981: 147-60). ERICSSON, K. A. 1988: Concurrentverbal reports on text comprehension.A review. Text 8, 295-325. and SIMON, H. A. 1987: Verbal reports on thinking.. In Faerch,C. and Kasper, G. (cds), Inlrospution in umnd language reulIrt'h. Clevedon: Multilingual Matters, 24--53. and SIMON, H. A. 1993: Protllclil anll(ysis: verbal repllrts lin dala. Cambridge,MA: MIT Press(revisededition). FAIGLEY, L. and WITTE, S. 1981: Analysing revision. Clillege Compllsition and Cllmmunicatilln 32, 4()()-14. FANSELOW,). F. 1977: Beyond Rashomon:conceptualizingand describingthe teaching act. TESOLQuarterly1l(1), 17-39. FATHMAN, A. K. and WHALLEY, E. 1990: Teacherresponseto studentwriting: focus on form v. content.In Kroll (1990: 178-90). FITZPATRICK, F. 1995: Peeringat your peers.The TeacherTrainer 9(2),14-16.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

REFERENCES

251

FLANDERS, N. 1970: Ana(J'singteachinghehaviour. Reading,MA: Addison-Wesley. FREIRE, P. 1972: The pedagogyofthe oppressed.Harmondsworth:Penguin. FROHLICH, M., SPADA, N. and ALLEN,]. P. B. 1985: Differencesin the communicative orientationofL2 classrooms.TESOLf!!Jarter(}! 19(1),27-57. GAGE, N. L. (ed.) 1963: Handhookofresearchon teaching.Chicago:Rand McNally. GAlES, S. and BOWERS, R. 1990: Clinical supervisionof languageteaching: the supervisor as trainer and educator.In Richardsand Nunan(1990: 167-81). GARDNER, R. C. and MACINTYRE, P. 1993: On the measurementof affective variablesin secondLanguagelearning. LanguageLearning43(2), 157-94. GASS, S. M. and SCHACHTER,]. 1989: Linguistic perspectiveson secondlanguageacquisition. Cambridge:CambridgeUniversity Press. GEERTZ, C. 1973: The interpretationofcultures. New York: Basic Books. GIROUX, H. 1988: Teachers 11.1 intellectuals: towards a critiwl pedagogy of learning. Massachusetts: Bergin and Garvey Publishers. GOOD, T. L. and BROPHY,]. E. 1978: Lookin,l!; in classrooms.New York: Harper and Row (2nd edition). GOSWAMI, D. and STILLMAN, P. R. (eds) 1987: Reclaiming the classroom: teacher researchas an agencyfilr change.Portsmouth,NH: Boynton/CookPublishers. GREEN,]. L. and WALLAT, C. (eds) 1981: Ethnograph)!and languagein educationalsettings. Norwood, NJ: Ablex. GREEN, P. S. (ed.) 1975: The languagelahorator)! in schoolpe1;lrmanceand prediction: the York Study.Edinburghand New York: Oliver and Boyd. GUILFORD,]. P. and FRUCHTER, B. P. 1973: Fundllmentalstatisticsinprediction: and education.New York: McGraw-Hill. HAKUTA, K. 1976: A casestudy of a Japanesechild learning English as a secondlanguage.LanguageLearning 26(2),321-51. HAMMERSLEY, M. (ed.) 1986: Controversiesin c1l1ssroom research. Milton Keynes and Philadelphia:Open University Press. 1989: The dilemma of qualitative method: Herhert Blumer and the Chiwgo School. London: Routledge. 1992: Deconstructingthe qualitative-quantitativedivide. In Brannen(1992a:39-55). and ATKINSON, P. 1983: Ethnography: principles in practice. London and New York: Tavistock. HANDAL, G. and LAUV As, P. 1987: Promoting reflective teaching: supervisionin aflion. Milton Keynes: Societyfor Researchinto Higher Education (SRHE) and Open University Press. • HANDY, C. B ..1985: Understandingorganizations.London: PenguinBooks (3rd edition). HATCH, E. and FARHADY, H. 1982: Reseanhdesign and statisticsj;lr applied linguistics. Rowley, Mass.: Newbury House. HITCHCOCK, G. and HUGHES, D. 1995: Researchand the teacher. London and New York: Routledge(2nd edition). HOLLIDAY, A. 1994: The houseofTESEPand the communicativeapproach:the special needsof stateEnglish languageeducation.ELTJournal 48(1), 3-11. 1995: A post-communicativeera? Method versus social context. In Budd (1995: 147-57). HOLLY, M. L. 1984: Keeping a pmonal-prrdessionaljournal. Deakin: Deakin University Press. HOPKINS, D. 1993: A teacher'sguide 10 classroomreseanh.Buckinghamand Philadelphia: Open University Press(2nd edition).

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

252

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

HOSENFELD, C. 1976: Learning about learning: discovering our students'strategies. Foreign LanguageAnnals9(2), 117-29. 1979: Cora'sview oflearninggrammar.CanadianModern Llln[(uage Review35, 602-7. HUSTLER, D., CASSIDY, A., and CUFF, E. C. (eds) 1986: Action researchin classrooms andschools.London: Allen and Unwin. HUTCHINSON, B. and WHITEHOUSE, P. 1986: Action research,professionalcompetenceand schoolorganization.British EducationalResearchJournal 12(1), 85-94. HUTCHINSON, T. and WATERS, A. 1987: En[(lish for specijic purposes: a learnin[(centredapproach.Cambridge:CambridgeUniversity Press. JAMES, K. 1984: The writing of thesesby speakersof English as a foreign language:the results of a casestudy. In Williams, R., Swales,J. and Kirkman, J. (eds), Common [(round: shared interests in ESP and communicationstudies. ELT Documents 117, Oxford: Pergamon,in associationwith the British Council, 99-113. JARVIS, J. 1992: Using diaries for teacherreflection on in-servicecourses.ELT Journal 46(2), 133-43. JOHNSON,D. M. 1992: Approachestil researfhin secondlan[(ua[(e learnin[(. New York and London: Longman. JOHNSON,H. andJOHNSON,K. Forthcoming:Handbookofappliedlin[(uistics. Oxford: Blackwell. JONES,F. R. 1994: The lone languagelearner:a diary study. System22(4),441-54. JORDAN, R. R. (ed.) 1983: Casestudiesin ELT. London and Glasgow:Collins. 1993: Study skills: experienceand expectations.In Blue (1993: 70-9). KEMMIS, S. 1991: Improving educationthrough action research.In Zuber-Skerritt,O. (ed.),Action researchin higher education.Brisbane:Griffith University, 57-75. KENNEDY, M. M. 1979: Generalizingfrom single case studies. Evaluation Quarter6' 3(4),661-78. KROLL, B. (ed.) 1990: Semndlanguagewriting. Cambridge:CambridgeUniversity Press. LINCOLN, Y. S. and GUBA, E. G. 1985: Naturalistic education. BeverleyHills, CA: Sage. LLEWELYN, S. 1995: Topics, text-typesand grammar: making the links. In Burns and Hood (1995: 67-74). LOMAX, P. (ed.) 1990: Managingstaffdevelopmentin schools: an action-researchapproach. Clevedon,Philadelphia:Multilingual Matters. LONG, M. 1983: Does secondlanguageinstruction make a difference?A review of the research.TESOLQuarter(), 17(3): 359-82. LOW, G. 1991: Talking to questionnaires: pragmatic models in questionnairedesign. In Adams, P., Heaton,B. and Howarth, P. (eds), Socioculturalissuesin En[(lish for academicpurposes.London: Macmillan. Modern English Publicationsin associationwith the British Council, 118-34. MACDONALD, B. and KUSHNER, S. (eds) 1983: Bread and dreams: a fasespuijic of bilin[(ual schoolin[( in the USA. Norwich: Centrefor Applied Researchin Education, University of EastAnglia. MACKAY, R. and MOUNTFORD, A. (eds) 1978: En[(lish jilr spuijic purposes.London: Longman. MALAMAH-THOMAS, A. 1987: Classroom interaction. Oxford: Oxford University Press. MALMKJAER, K. 1993: A lin[(uistics entydopaedia.London: Routledge. MARKEE, N. 1986: Towardsan appropriatetechnologymodel of communicativecourse design.En[(lishfilr Spuijic Purposes5(2),161-72. MARSHALL, C. and ROSSMAN, G. B. 1989: Designing qualitative resellrfh. Newbury Park: Sage.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

REFERENCES

253

McDONOUGH,]. 1994: A teacherlooks at teachers'diaries. ELTJournal 48(1), 243-52. and McDONOUGH, S. 1990: What'sthe useof research?ELTJournal 44(2), 102-9. and McDONOUGH, S. 1993: From EAP to chemistry: risking the anecdotal.In Blue (1993: 132-40). and SHAW, C. 1993: Materials and methodsin ELT. Oxford: Blackwell. McDONOUGH, S. H. 1995: Strategy and skill in learning a foreign language. London: Edward Arnold. McLAUGHLIN, B. 1987: Theoriesofsecondlanguagelearning. London: Edward Arnold. McNIFF,]. 1988:Action research:principles lind practice. London: Macmillan. MILES, M. B. and HUBERMAN, A. M. 1994: Qualitative datalanguage.ThousandOaks, CA: Sage(2nd edition). MORROW, K. and SCHOCKER,M. 1993: Processevaluationin an INSET course.ELT Journal 47(1), 47-55. MURPHY, R. and TORRANCE, H. (eds) 1988: Evaluating education: issuesand methods. London: Paul Chapman,in associationwith The OpenUniversity. MURPHY O'DWYER, L. M. 1985: Diary studies as a method for evaluating teacher training. In Alderson,J. c. (ed.), Evaluation. LancasterPractical Papersin English LanguageEducation,Vol. 6. Oxford: PergamonPress,97-128. NAIDU, B., NEERAJA, K., RAMANI, E., SHIVAKUMAR,]. and VISWANATHA, V. 1992: Researchingheterogeneity:an accountof teacher-initiatedresearchinto large classes.ELTJournal 46(3), 252-63. NAIMAN, N., FROHLICH, M. and STERN, H. H. 1975: The good language learner. Ontario Institute for Studiesin Education: Modern LanguageCentre,Department of Curriculum. NISBETT, R. E. and WILSON, T. D. 1977: Telling more than we can know: verbal PsychologicalReview84, 231-59. reportson mental processes. NIXON,]. (ed.) 1981: A teacher'sguide to action research.London: Grant McIntyre. NORRIS, N. 1990: Understandingeducationalevaluation. London: Kogan Pagein association with CARE. NUNAN, D. 1988: The learner-centredcurriculum. Cambridge:CambridgeUniversity Press. 1989: The teacheras researcher.In Brumfit and Mitchell (1989: 16-32). (ed.) 1992a: Collaborative language learning and teaching. Cambridge: Cambridge University Press. 1992b:Researchmethodsin languagelearning. Cambridge:CambridgeUniversity Press. O'MALLEY, ]. M. and CHAMOT, A. U. 1990: Learning strategies in sewnd language acquisition. Cambridge:CambridgeUniversity Press. OPEN UNIVERSITY /SCHOOLSCOUNCIL 1981: Curriculum in action: practical classroom evaluation. Milton Keynes:Open University Press. OXFORD, R. 1990: Languagelearning strategies: what evet:y teachershould know. Rowley, Mass.: Newbury House. PARKINSON, B. and HOWELL-RICHARDSON, C. 1989: Learnerdiaries. In Brumfit and Mitchell (1989: 128-40). PEAL, E. and LAMBERT, W. 1962: The relation of bilingualism to intelligence. Washington: American Psychological Association. Psychological Monographs: Generaland Applied, 76 (546). PECK, A. 1988: Languageteachersat work: a descriptionofmethods.London: PrenticeHall. PENNINGTON, M. C. 1990: A professionaldevelopmentfocus for the languageteaching practicum.In Richardsand Nunan(1990: 132-52). PERL, S. 1981: Coding the composing process: a guide for teachersand researchers. Manuscriptwritten for the National Institute of Education,Washington,DC, USA, ED 240609.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

254

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

PEYTON,}. K. and STANTON, J. (eds) 1991: Writing our lives. EnglewoodChffs, NJ: Prentice-Hall/Regents. PICKETT, G. D. 1978: The fimign language learning process. London: British Council English TeachingInformation Centre. PLUMMER, K. 1983: Documentsoflift· London: Allen and Unwin. PORTER,P. A., GOLDSTEIN, L. M., LEATHERMAN,}. and CONRAD, S. 1990: An ongoing dialogue: learning logs for teacher preparation.In Richards and Nunan (1990: 227-40). POULISSE, N., BONGAERTS, T. and KELLERMAN, E. 1987: The use of verbal reportsin the analysisof compensatorystrategies.In Faerch,C. and Kasper,G (eds) Introspectionin secondlanguageresearch.Clevedon:Multilingual Matters,213-29. POWELL, R. C. 1992: 'More than just a hobby': language teachers' perceptionsof involvementin a curriculum developmentproject. System20(4), 493-506. PROGOFF,I. 1975: At a journal workshop.New York: Dialog HouseLibrary. RAIMES, A. 1983: Techniques in tellChing writing. New York and London: Oxford University Press. RAMANI, E. 1987: Theorisingfrom the classroom.ELTJournal 41(1), 3-11. REA-DICKlNS, P. and GERMAINE, K. 1992: Evaluation.Oxford: Oxford University Press. RICHARDS, }. C. 1985: The context I!F language teaching. Cambridge: Cambridge University Press. and LOCKHART, C. 1994: Reflective teaching in second language classrooms. Cambridge:CambridgeUniversity Press. and NUNAN, D. (eds) 1990: Secondlanguageteachereducation.Cambridge:Cambridge University Press. and RODGERS,T. S. 1986: Approachesand methodsin languageteaching. Cambridge: CambridgeUniversity Press. RICHARDS, K. 1992: Pepysinto a TEFL course.ELTJournal 46(2), 144-52. RIVERS, W. 1964: The Psychologicaland thefilreign languageleacher. Chicago:University of ChicagoPress. ROBERTS,}. R. 1993: Evaluatingthe impactof teacherresearch.System21(1),1-19. RUDDUCK,}. and HOPKINS, D. 1985: Researchas a basisJilr leaching: readingsfrom the work ofLawrenceStenhouse.London: Heinemann. SAVILLE- TROIKE, M. 1989: The ethnograpkvofcommunication.Oxford: Blackwell (2nd edition). SCHECTER, S. R. and RAMIREZ, R. 1992: A teacher-researchgroup in action. In Nunan (l992a: 192-207). SCHMIDT, R. W. and FROTA, S. N. 1986: Developingbasicconversationalability in a secondlanguage:a casestudy of an adult learnerof Portuguese.In Day, R. R. (ed.), Talking to learn: CIInversation in secondlanguageacquisition. Rowley, Mass.: Newbury House,237-326. SCHON, D. A. 1983: The reflectivepractitioner. New York: Basic Books. SCHUMANN, F. E. and SCHUMANN, }. H. 1977: Diary of a languagelearner: an introspectivestudy of secondlanguagelearning. In Brown, H. D., Crymes, R. H., and Yorio, C. A. (eds), On TESOL '77: teachingand learning English as a secondlanguage- trendsin researchandpractice. Washington,DC: TESOL, 241-9. SCHUMANN, ). H. 1993: Some problems with falsification: an illustration from SLA research.AppliedLinguistics 14(3): 295-306. SEEDHOUSE, P. 1995: Using the Internet for research purposes. Research News. NewsletterofIATEFL ResearchSIG No.6, 10-11. SELIGER, H. W. and SHOHAMY, E. 1989: Secondlanguageresearch methods.Oxford: Oxford University Press.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

REFERENCES

255

SEVIGNY, M.]. 1981: Triangulatedenquiry -a methodologyfor the analysisof classroom interaction.In Greenand Wallat (1981: 65-86). SHARWOOD SMITH, M. 1994: Swmd language learning: theoretical /iJUndations. Harlow: Longman. SIMONS, H. (ed.) 1980: Towardsa scienceofthe singular. OccasionalPapers19, Centrefor Applied Researchin Education:University of East Anglia. SKEHAN, P. 1989: Individual differencesin secondlanguagelearning. London: EdwardArnold. SLIMANI, A. 1989: The role of topicalization in classroomlanguagelearning. System 17(2),223-34. SMITH, K. 1995: Guided and self-directed reflections in teachertraining. Paper presentedat the secondTeachersDevelop TeachersResearchconference,Cambridge, January1995. SOMEKH, B. 1993: Quality in educational research- the contribution of classroom teachers.In Edgeand Richards(1993: 2~38).38 SOMMERS,A. 1980: Revision strategiesof studentwriters and experiencedadult writers. College Compositionand Communication31, 378-88. SPOLKSY, B. 1989: Conditions/ilr secondlanguage learning. Oxford: Oxford University Press. STAKE, R. 1994: Casestudies.In Denzin and Lincoln (1994: 23~7).47 1995: The art ofcasestudyresearch.ThousandOaks,CA: Sage. STENHOUSE,L. 1975: An introduction to curriculum research and development.London: Heinemann. 1988: The conduct,.analysisand reporting of case study in educationalresearchand evaluation.In Murphy and Torrance(1988: 212-24). STEVICK, E. W. 1980: Teaching languages: a way and ways. Rowley, Mass.: Newbury House. STRICKLAND, D. S. 1988: The teacheras researcher:towards the extendedprofessional. LanguageArts Vol. 65, 754-64. SWAN,]. 1993: Metaphorin action: the observationschedulein a reflective approachto teachereducation.ELTJournal 47(3), 242-9. TAYLOR, S.]. and BOGDAN, R. 1984: Introduction to qualitative researchmethllds. New York: Wiley. THORNBURY, S. 1991: Watching the whites of their eyes: the use of teachingpractice logs. ELTJournal 45(2), 140--6. TONKYN, A., LOCKE, c., ROBINSON, P. and FURNEAUX, C. 1993: The EAP teacher:prophetof doom or eternaloptimist? EAP teachers'predictionsof student success.In Blue (1993: 37-48). TRANTER, D.l986: Changingschools.In Hustler et al. (1986: 105-22). TRIPP, D. 1993: Critical incidentsin teaching. London: Routledge. VAN LIER, L. 1988: The classroom and the language learner. London and New York: Longman. 1989: Ethnography: bandaid, bandwagon,or contraband?In Brumfit and Mitchell (1989: 33-53). WALKER, R. 1985: Doing research: a handbookjilr teachers.London: Methuen. 1986: The conduct of educational case studies: ethics, theory and procedures. In Hammersley(1986: 187-219). and ADELMAN, C. 1976: Strawberries.In Stubbs,M. and Delamont, S. (eds) 1976: Explorationsin classroomobservation.Chichester:John Wiley, 133-50. WALLACE, M. ]. 1991: Training jimign language teachers: a rejlative approach. Cambridge:CambridgeUniversity Press.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

256

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

WALLAT, c., GREEN,]. L., CONLIN, S. M. and HARAMIS, M. 1981: Issuesrelated to action researchin the classroom- the teacherand researcheras a team. In Green and Wallat (1981: 87-113). WATSON-GEGEO, K.A. 1988: Ethnographyin ESL: defining the essentials.TESOL Quarter(j, 22(4), 575-92. WEIR, C. 1988: The specification,realizationand validation of an English languageproficiency test. In Hughes, A. (ed.), TestinK English Jorqualitative study. London: Macmillan. Modern English Publicationsin associationwith the British Council, 45-110. and ROBERTS,].R. 1994: Evaluationin ELT. Oxford: Blackwell. WEITZMAN, E. A. and MILES, M. B. 1995: Computerprogramsfor qualitative data ana(j,sis.ThousandOaks,CA: Sage. WENDEN, A. 1987: How to be a successfullanguagelearner.In Wenden,A. and Rubin, ]. (eds), Learner strategiesin language learning. Hemel Hempstead:Prentice Hall, 103-18. WHITE, R. V. and ARNDT, V. 1991: Processwriting. Harlow: Longman. WIDDOWSON, H. G. 1978: Teat"hinK language as CIImmunication. Oxford: Oxford University Press. WODE, H. 1976: Developmentalsequencesin naturalisticL2 acquisition. Working Papers in Bilingualism Vol. ll, 1-31. WOODS, A., FLETCHER, P. and HUGHES, A. 1986: Statistics in language study. Cambridge:CambridgeUniversity Press. WRIGHT, T. 1987: Rolesofteachersand learners. Oxford: Oxford University Press.

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Index

Accountability 3 Action 7,8,21,22-5,35 Adelman,C 26, 27 etal. 217 Adult Migrant English ProgramAMEP 32, 132, 180,221 Alderson,C 139 Allwright, D 14,61, 112 and Bailey, K 78,80,95,125, 130 Anderson,NJ et al. 221,224 Anxiety and competitiveness 130, 149 Appel,] 132 Applied linguistics 6, 28, 66, 85, 220, 235 British Associationfor Applied Linguistics (BAAL) 97,241 Applied ScienceseeTechnicalrationality Applicability, Application 3,65-7,seeals() Replication,Utility, Generalizability, Transferability AppraisalseeEvaluation Appropriatemethodology 234 Aptitude 149, 150 Arndt, V 200 Association 145, seeals() Chi-square Atkinson, P and Delamont,S 216 Bachman,L and Palmer,A 221 Bailey, K 122, 126, 129, 130, 13l, 134 seeals() Allwright and Bailey and Ochsner 126 BarkerAnn (AB) lO-ll, 76, 227, 230,231 Bassey,M 69 Beebe,L 67,209 Berthoff, A.E 79,212 Bilingualism 65,220 Bialystok, E 198 Block, E 200,201 Blumer, H 46

Bolster, A 22,23,29,212 Boomer,D.S 2, 16 Boundedsystem 205,212,218 Brannen,] 222 Breadand Dreams 215,216 Brindley, G 32,65,82,180,221 Brown, C 51,93,95 Brown,J.D 155,156 Brumfit, C and Mitchell, R 2, 22 Bryant, I 8 8 Budd, T 234 Buresova,V 235 Burns,A 132 and Hood, S 32,221 Burton,] and Mickan, P 61,62 Calderhead,] 46 Campbelland Stanley 156 Cane,B and Schroeder,C 25 Carr, Wand Kemmis, S 2fr-7 Carrell, P 181 et al. 161,166,167,168 Carroll, B and West, R 139 Casestudy 47,66,86,95,136, 182, 203-18,222 Definitions 204 Classification 20fr-7 Methods 207-9 Themeand issue 209 Casanave,c.p and Hubbard,P 180 Categorysystemsseecodingsystems Cause 48,50,147,155,157-8 Centraltendency 140-1,seeals() mean Centrefor Information on Language Teaching(CIL T) 98,241 Chamot,A-U 32, 82 Changeseeinnovation Chaudron,C 209

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

258

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Chi-square 109, 145,151 Classroom interaction 9,114,162-4 events 25 Action ResearchNetwork 27,97,243 research 101,155,181,190 Classsize 8 Clegg,] 234 Codingsystems 94,101,105-11,115, 192, 198,207 examples 105-6 advantagesand disadvantages of 1O~7 7 Cohen,A.D 193,194,200 and Aphek, E 199 Cohen,L and Manion, L 42,48,65, 69, 95, 177, 182, 183, 186, 187,203,204, 208 COLT (Communicationorientationof languageteaching) 106 Communicative competence 85 languageteaching 10, 14,29,93-4,234 Computationalaids 125,152,178 Confidentiality 68,123,185,191 Contentanalysis 94,125,187,192,223 Context 8-9,27,45-7,51,52,65,114, 172, 184 Control 42,47,121 group 41,45,162,163,168 and counterbalance 45, ISS, 160 Correlation 144-6 Corti, I 122, 125, 134 Counselling 186 Credibility seevalidity Criteria 69,235 Critical incidents 80 Criper, C and Davies,A 148 Cronbach,L 216 Crookes,G 33 Cross-tabulation 179 Curriculum development 25 in action 27, 105 Dadds,M 62,196 Data 41,53,54,116 collection 221 diary 123, 124 display 118 - first 79, 125, 130,210 protection 68, 191 reduction 118 thick vs thin 124, 136 Day,R 117

Decontextualization 165-6 Denzin, N.K 46,71,208 DependabilityseeReliability Description,Descriptive 4,166,192,230 elaborative 112, 198 vs intervention 44-7,54 thick vs thin 116,118 Development,institutional and personal 15,17,101,203 Deviation, standard 48, 141-3 Diaries 30,47,49,64,93, 94, 138, 208 and diary studies 121-36, 12~7 7 expert 129-30,136,211 keepinga diary 134-6 languagelearners' 127-8, 136 teachers' 131-2, 136, 190, 194 Discrepantcases 53,54,65 Dispersion 141-4 Doughty,C and Pica, T 163 Edge,] and Richards,K 31 Education 25-8,81,121,203,204,220 ERIC (EducationResearchInformation Center) 98,99 Egocentricerror 194 Elliott,] 24,26,28-9,40,121,203,215, 220 Ellis, R 65,67, 190,209,210 Electronicinformation 98 Elton, L 64 Emergent 80-1,83,117,205,231 emic principle 51,52, 114,205 English for specific purposes 11, 87, 111, 148,180,208,215 for academicpurposes 77, 164, 180, 224 English languageteaching 28-34,35,121 lind learning 220 seea/s(J secondlanguageacquisition Erickson,F 52,53,66,94,168 and Schultz,] 46 Ericsson,K.A 194 and Simon, H 192 Error treatment 127, 130 Ethics 39,53-4,67-8,123, 167-8, 191, 208 Ethnography,Ethnographic 50,51, 116, 181,208,220,235 'smashand grab' 156 Ethnomethodology 46, 50 Evaluation 15,17,25,87,181,182,190, 204,206,215,220,221,230 of teacherresearch 34,237-8

INDEX

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

Experiment,Experimental 4,41,47,49, 95,137,155-69,230,237 quasi-experiment 47,49,156,158-61 Explanationvs prediction 50 Exploratory teaching 61 Factoranalysis 149 Falsification 54, ~5, 5 159 Fanselow,j 105 Faigley,L and Witte, S 224 Fathman,A and Whalley, P 164, 166,224 Feedforward 231 seealso Changeand innovation Field notes 112,192,197 Fitzpatrick, F 118 Flanders,N 106, III FlandersInteractionAnalysis Categories FIAC 106, 107, III FOCUS 105 Focusgroup 71, 185 Ford TeachingProject 27,96,220 French,David (DF) 12,16 Frequency 138, 140 Gaies,R and Bowers,R 118 Garcia,Anna (AG) 76, 86 Gardner,R and MacIntyre, P 149, 179 Gatekeeper 68, 166 Geertz,C 116 Generalizability,Generalization 21,42, 27,29,52,65,136,158, 165,203, 216-7,236 petite and grand 217 naturalistic 217 Generation 3,9,58-61,75-89,125 Giroux, H 23 Good, T.L and Brophy,].E 104 Goswami,D and Stillman, P.R 27,32 Green,].Land Wallat, C 32 Green,P 45 Guilford, j and Fruchter,B 155 Hakuta,K 211 Hammersley,M 46, 222 and Atkinson, P 23,115,116 Handal,G and Lauvas,P 30, 34 Handy, C. B 9,213 Hatch, E. M and Farhady,M 137, 155 Heededprocesses 192 Hitchcock, G and Hughes,D 131,156, 205,206 Holistic principle 51, 114,205 Holliday, A 234, 235 Holly, M.L 122,126,131

259

Homogeneityseedispersion Hopkins, D 3,26,69,80,84,95,101, 118, 122, 181 Hosenfeld,C 190, 193 HumanitiesCurriculum Project 27,220 Hutchinson,T and Waters,A 215 Hycner 186 Hypothesis 3,42, 78, 80 hypothetico-deductive 78, 101,210 null 159 Idiographic 22,24, 104,212 Inductive 79,80,101,210 Initiation seegeneration Innovation 15, 17, 18,27,33,67,104 Instances 7, 10-4,76-8,86--8,205, 225-31 Instructedlanguagelearning 65 Interest 58, 82 Interlanguage 85 seealsl! secondlanguageacquisition IATEFL 31,97,99,230 Interview 47,71,93,94,95,172,181-7, 190,224 group 185 qualitativeanalysisof 186 recordingof 186 semi-structured 182, 183-4, 187 structured 182, 207 unstructured 182, 184,208 Introspection 124, 189-201 seealsl! Protocol,Think-aloud james,K 214,216,217 johnson,D 82,149,205,208,211,220, 221 jones,F.R 128,129,130,135 jones,Frank (FR) 77,228-9,230,231 jordan, R 180,216 journals 97, 122, 133, 134 Kemmis, S 168 Knowledge 15,18,25,104 craft 24,29,35,133 insider 61, 114 knowing-in-action 23 L2 methods 220 seealsl! applied linguistics, second languageacquisition Learnertraining 199,200 Lewin, K 26 Lightbown, P 211

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

260

RESEARCH METHODS FOR ENGLISH LANGUAGE TEACHERS

Lincoln, Y.S and Guba,E.G 63,64, 102, 114, 135, 182,207,217,222 Listeningcomprehension 199 Literaturesearch 70,236,237,239 Llewellyn, S 132 Local meanings 52, 93 Logs 122,133,207 seealso diaries Long, M 65 Lopez, Antonio (AL) 12,17 Low, G 172,173 Malamah-T homas,A 8 8 Management 27 Marcos 214,215,216,217,218 Markee,N 215 Marshall, C and Rossman,G 85, 94 McDonald, B and Kushner,S 215,216 McDonough,] 132 and Shaw,C 9,31, 128 and McDonough,S 41,81,128,132, 213 McDonough,S 190,198,199 McLaughlin, B 49,65,78, 149 McNiff,] 26 Matsuda,Kenji (KM) 10, 15,33,76,225, 229,230 Mean 48,141 Measurement 47,138-9,230 interval 138 nominal 138 ordinal 134,141 ratio 139 Memberchecking 186,208 Merriam 206 Meta-analysis 65, 70 Miles, M.B and Huberman,A.M 118, 205 Monitor model 85 Morrow, K and Schocker,M 237 Motivation 149, 161 Murphy O'Dwyer, L 133 Naidu, B el al. 238 Needsanalysis 41, 138, 180, 182,215 Nin, A 121 Nisbett, R.E and Wilson, T.D 192 Nixon,] 95 Norris, N 215 Numericalanalysis 47-8,95,137-53,167, 207,225 of qualitativedata 125 Nunan,D 2, 16,32,82,83,95,III, 112, 114,125,145, 181,20~216,221216,221

Objective,Objectivity 3, 42, 48, 62 vs subjective 124, 135 Observation 93,95, 101-19,138, 190 naturalistic 101, 102, 114-8, 208 participant 50,51-2,66,101, 116 reductionist 112, 114 systematic 47, 101,105-11 observer'sparadox 115' O'Malley,J.Mand Chamot,A-U 198 Originality 38,59,68 Ownership 18,54,68,215 Oxford, R 211 Paivio and Begg 155 Paradigm 3,14,24,30,79,206,222 seealso research- qualitative, quantitative,interpretive, normative Parkinson,B and Howell-Richardson,C 128 Patternanalysis 117, 186 Peal,E and Lambert,W 65 Peck,A 112-4 Pennington,M 24,28 Petrov,Irina (IP) 11-2,208,226-7,229, 230,231 Perl, S 200 Personalconstruct 30, 79 Peyton,J.Kand Stanton,] 126 Phenomenology 46 Pickett, G 129 Plausibility 53,61, 187 Plummer,K 204 Porter,P.A el al. 133 Positivism 50, 168,237 Poulisse,N el al. 112 Powell, R. C 176,180 Power 33,34, 132, 156 Practice 1, 25 Probability 47,48, 142 Programmedesign 215 Progoff, I 135 Proof 3 Protocol 48, 190, 193 analysis 193,195,200 see"lWl introspection,think-aloud Publication 60-1,68,191 and dissemination 96-9,230

Q 213,218

Questionnaire 41,93,94,138,171, 172-81,192,207,225 analysis 178 as conversation 172-3

INDEX

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

as tests 173-4 construction 177 factual questions 174 multiple-choice 175 open-ended 176, 180 rankedquestions 17~, 6 189 Ramani,E 85--6, 133 Randomill'.ation 49, 160, 161, 166-7 Rank order 138, 141 Rea-Dickins,P and Germaine,K 215 Readingskills 12,29,43,76,87,199 comprehension 200-1 skim reading 195 Recording audio 110-1, 191, 197 video 110--1,112,197 Regression 149 Reflection 8,21,22-5,27,29,30,131 -in-action 23, 30, 34 Reflectivepractitioner 23, 30 Reflectivequestions 126 Reflexivity 23,70, 115 Relevance 3, 18, 19,21,24 Reliability 63,68,173,198,203 ofcodings 106,108 Replicability, Replication 3, 65--6, 68, 158,222 Research 70, action 2,3,24,26-8,34,35,62,67, 128, 166, 181,203,215,220,222, 231,238 cycle, spiral 26, 190, 231 attitudes 15, 16,41 basicvs applied 32, 42-3, 54 collaboration 16,27,33,34,84,212, 221, 222,229 condition-seeking 65, 149-50 critil:al 168 design 61-5,70,157 first, 2nd, 3rd person 62, 168, 196 higherdegree 12, 17, 40, 60, 236 individual 15,16 institutional 15, 16 interpretive 14,47,5H, 80,168 market 38-9, 182 method,technique 94--5 multi-method 71,208,219-25 multi-site 220 normative 47,47-50,54,80 outsider 14,15,16,51,137,156,196 qualitative/quantitative 53, 54, 60, 190,198,205,208,220,224

261

researcher/university 2, 14,84,237, 190 scale 14,32,164,167, 187, 190,203, 219,229 stance 1,2, 7, 75 teacher 1,3,15,21-36,60,110,190, 203,233,235 training 17, 85--6 two senses 37-41 Retrospective,Retrospection 129, 191 Richards,J 8, 133 and Lockhart,C 7,9,30,46,82,126, 131,134 and Rodgers,T 94 Roberts,J 34,238 Rogers,C 186 Role theory 9,212,213 Sample judgmental 206 random 160, 206 size 160 Saville-Troike,M 124,184 Scaling Likert 176,180 Guttman 177 Scattergram 146 Schecter,S.R and Ramirez,R 70,71,88, 235 Schematheory 43, 85 Schmidt,Rand Frota, S 129 Schon,D 23,24,29,34,79,131 Schumann,J 65 and Schumann,F 129 Schwartzand Ogilvy 115 SecondLanguageAcquisition 42,49, 181,190,204,209-11 Self observation 193 report 173, 193 revelation 173,193 Seliger,Hand Shohamy,E 78,81,95, 101, 118 Sensitivity 61,68,173 Sevigny,M.J 46 Sharwood-Smith,M 42,78,210 Significance 48,49, 150--1 interpretationof 151-2 Simons,H 216 Skehan,P 61,66,149,150 Skill learning 189 Slimani, A 199 Smith, K 30 Sociology 204

Downloaded by [University of California, San Diego] at 04:00 17 March 2017

262

RESEARCH METHODS FOR ENGLISH LANGU\GE TEACHERS

Somekh,B 33 Sommers,A 224 Specialneeds 13 Specificity and the methodof detail 59-60, 157-8 contextual 60, 68 Spolsky,B 66 Standardscoreseedeviation Stake,R 3, 184, 186, 205, 206, 208, 209, 211,215,217 Statistics descriptive 137 inferential 150 multivariate 148-9 Stenhouse,L 3,25,69,104, 206, 220 Stimulatedrecall 112 Strategies 190, 198,206 metacognitive 181 Strawberries 115Strickland,D 32 Subjects 18,156,160 subjectivity 115,135 Supervision 30 clinical 30, 118 Survey 171, 181, 199,220 Swan,H 31 Symbolic interactionism 46, 50 Taylor, S.] and Bogdan,R 80,94 Teachersin action 7-19, seealsl! instances Teacher development 31, 196, seealso personal development training 28-30,35,121, 132 questions 157 Technicalrationality 23,29, 35 TESOL 97 Testing 138,180,181,193 ESU framework 139 TOEIC 13 TOEFL 13, 147, 180 IELTS 13,139, 147 proficiency 63 questionnairesas tests 173 research 221 Text analysis 112, 224 Theory 24, 94 first 78-9,130,210 groundedtheory 46, 51

Think-aloud 95, 181, 190, 191-4 level 1 and 2 reporting 192, 197 method 196-9 seealsl! introspection,protocol Thornbury,S 133 Time 166 Tonkyn, A et al. 225 Transferability 45-7,217' seealsl!generalizability Triangulation 51,71,93,198-9,208, 223 Tripp,D 80 Turner, Carol (CT) 13,77,228,229,231 Universalgrammar 79,85 Utility 40,66,67 Validity 21,38,43,62-3,68,lll, 147, 158, 160-1, 173, 187,203 Van Lier, L 47,48,51,96,114,121,148, 157, 185,235 Variables confounding 44,49, 52, 159 dependent 44,49,159 independent 44,49,159 moderator 159, 192 Verbal reportseeintrospection,thinkaloud, protocol Vocabularylearning 76, 85 Wallace,M 29, 101 Wallat, C et al. 212 Walker, R 16,94,95,102,117,186,205 and Adelman,C 114 Watson-Gegeo,K 51 Weitzmann,E.A and Miles, M.B 125 Weir, C 180 and Roberts,J 181 Wenden,A 187 Wode, H 211 Wolcott 116 Woods,A et al. 137 Wright, T 9 Writing skills 87, 164, 180 feedback 164,166,224 process 200 revision 224 Yin

206