Assessing L2 Students with Learning and Other Disabilities [1 ed.] 9781443852845, 9781443851367

Citation preview

Assessing L2 Students with Learning and Other Disabilities

Assessing L2 Students with Learning and Other Disabilities

Edited by

Dina Tsagari and George Spanoudis

Assessing L2 Students with Learning and Other Disabilities, Edited by Dina Tsagari and George Spanoudis This book first published 2013 Cambridge Scholars Publishing 12 Back Chapman Street, Newcastle upon Tyne, NE6 2XX, UK British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library Copyright © 2013 by Dina Tsagari, George Spanoudis and contributors All rights for this book reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without the prior permission of the copyright owner. ISBN (10): 1-4438-5136-1, ISBN (13): 978-1-4438-5136-7

TABLE OF CONTENTS

INTRODUCTION ............................................................................................ ix Dina Tsagari and George Spanoudis EDITORIAL ................................................................................................. xv Judit Kormos PART I: DIAGNOSING SLL’S LEARNING AND OTHER DISABILITIES CHAPTER ONE .............................................................................................. 3 Disproportional Representation of English Learners among Students Identified with Disabilities: Equity and the Federal Accountability System Keira Gebbie Ballantyne CHAPTER TWO ........................................................................................... 27 Assessment of Literacy Difficulties in Second Language and Bilingual Learners John Everatt, Amir Sedaghi, Louisa Grech, Mohamed Elshikh, Shaimaa Abdel-Sabour, Nasser Al-Menaye, Brigid McNeill and Gad Elbeheri CHAPTER THREE ........................................................................................ 45 Students Classified as Learning Disabled in L2 Courses: The ‘Special’ Case of the United States Richard L. Sparks CHAPTER FOUR .......................................................................................... 69 Assessing D/Deaf Students as Visual L2 Learners: From Theory to Practice Katherine M. Groves, Maria Tagarelli De Monte and Franca Orletti CHAPTER FIVE............................................................................................ 93 Using Curriculum-Based Assessment to Identify Young Second-Language Learners’ Risk for Delayed Second-Language Reading Progress Melody Kung, Jill Fitzgerald and Steven J. Amendum

vi

Table of Contents

CHAPTER SIX............................................................................................ 111 Screening for Specific Learning Disabilities in Second Language Learners: The Role of Teachers’ Estimates Faye Antoniou and Suzana Padeliadu PART II: TRAINING NEEDS AND ASSESSMENT PROCEDURES CHAPTER SEVEN ...................................................................................... 133 Investigation of Trainee-Teacher Awareness of At-Risk and Dyslexic Students in the EFL Classroom in Germany Lana Loumbourdi and Yvonne Karacic CHAPTER EIGHT ....................................................................................... 151 Developing “Cognitive Assessments for Multilingual Learners” Anne Margaret Smith CHAPTER NINE ......................................................................................... 169 Fairness and Validity in Testing Students with SpLDs: A Case Study from Italy Claudia D’ Este and Geraldine Ludbrook CHAPTER TEN .......................................................................................... 189 Assessment Accommodations in FL Reading Competence for Slovene FL Students with Specific Reading Differences Florina Erbeli and Karmen Pižorn CHAPTER ELEVEN .................................................................................... 207 L2 Teaching and Assessment of University Students with Disabilities Kathleen Brannen and Martyna Kozlowska PART III: VOICES FROM L2 EXAMINATION BOARDS CHAPTER TWELVE.................................................................................... 229 Assessing Students with Disabilities: Voices from the Stakeholder Community Lynda Taylor and Hanan Khalifa CHAPTER THIRTEEN ................................................................................. 253 Special Needs Test Forms: Leveling the Playing Field for Test Takers with Disabilities Jayanti Banerjee, Natalie Nordby Chen and Barbara Dobson

Assessing L2 Students with Learning and Other Disabilities

vii

CHAPTER FOURTEEN ................................................................................ 271 On Equal Footing? Accommodations for Disabled Candidates in the TestDAF Ulrike Arras, Anika Müller and Sonja Zimmermann CONTRIBUTORS ........................................................................................ 287

INTRODUCTION DINA TSAGARI AND GEORGE SPANOUDIS

One of the recent tendencies in the field of education nowadays is that the population of students is becoming increasingly diverse, both culturally and linguistically. As a result, the numbers of children diagnosed with Specific Learning Differences (SpLD), e.g. dyslexia, specific language impairment, attention deficits, as well as those with other disabilities, e.g. visual, hearing or physical impairments, are steadily growing. So is the number of students enrolled in special education. This situation, combined with greater awareness of individual human rights, has led to an increased demand for appropriate testing and assessment provision. This is of particular concern to Second/Foreign Language (L2/FL) test providers (Taylor 2012) and teachers (Kormos and Smith 2012), who are very often faced with the challenge of having to offer special arrangements (accommodations) to Second Language Learners (SLLs) with disabilities. Within this framework, the present volume seeks to discuss the theoretical, ethical and practical considerations involved in assessing SLLs with disabilities. More specifically, it explores theoretical models and research findings that identify the special needs of SLLs with SpLD and other disabilities and evaluates the effectiveness of accommodation practices employed. Studies of both high-stakes tests and classroom-based assessments related to the special needs of SLLs are presented by professionals and researchers working in the area of psychology, special education and L2/FL testing and assessment. Cross-sectional and longitudinal studies are included, as well as studies conducted among young and adult SLLs with SpLD and other disabilities. Related issues are examined through multidisciplinary and multifaceted approaches. As such the volume explores recent thinking and research in the fields of special education, psychology and language testing and assessment and critically expands work already done in these fields by presenting new, exciting and uncharted avenues and territories where these fields meet in a dialectic and informative relationship. This volume is a compilation of fourteen chapters, both theoretical and research-oriented, addressing the fair assessment of this special population of SLLs. The volume consists of three parts. Part I contains six chapters

x

Introduction

focusing on issues related to diagnosing SLLs with SpLD and other learning disabilities. Part II consists of five chapters that discuss training needs and assessment procedures. Part III includes three chapters involving the perspective of L2/FL examination boards. In chapter one, Keira Ballantyne makes detailed reference to the federal educational accountability system in the United States which, as the author explains, does not systematically collect data necessary to measure whether non-native English students are disproportionately represented in special education at the school, district, or state level. The chapter alerts us to the lack of disproportionality data and the inequities caused by misclassification in the US educational system. In the second chapter, Everatt et al. discuss research investigating cognitive-linguistic predictors of literacy, i.e. predictors of reading ability among Arabic, Maltese and Persian speaking school children. Findings of their study support the view that phonological skills predict variability at the word level, and measures of language understanding and word decoding predict variability in text comprehension. In the next chapter, Richard Sparks reports on the status of students classified as SpLD students in L2 courses in the U.S. educational context. His comprehensive review discusses problems of definition and diagnostic criteria for Learning Disabled (LD) students and studies that reveal long-term relationships between students’ L1 (First Language) skills and their L2 proficiency as well as studies conducted with LD and non-LD students in L2 courses. The author concludes with a discussion of the practical problems for L2 assessment in the U.S. In chapter four, Groves et al. examine the factors affecting language acquisition of deaf people. The chapter discusses theoretical and practical approaches from a deaf education perspective. Special attention is paid to the potential use of the Common European Framework of Reference (CEFR) for languages in the development of language education curricula for deaf learners. In the next chapter, Kung et al. provide empirical evidence on the value of curriculum-based reading assessments in identifying young SLLs who can benefit from accommodated reading instruction. In chapter six, Antoniou and Padeliadu explore whether SLLs are identified as SpLD students in unequal proportions compared to native speakers. The authors discuss defining factors of SLL and SpLD, highlight the overrepresentation of SLL students in the SpLD category and emphasise the importance of measuring not only oral language and literacy, but also writing skills, mathematical ability and reasoning capacity when screening for SpLD in SLLs.

Assessing L2 Students with Learning and Other Disabilities

xi

In the opening chapter of Part II, Loumbourdi and Kracic argue that foreign language teachers should be better prepared to cope with students’ literacy problems in English. The chapter presents data from teacher trainees’ interviews at the beginning and end of their studies exploring their knowledge of issues related to dyslexia and at-risk students in the EFL classroom, as well as their familiarity with various methods of assessment. In chapter eight, Ann Margaret Smith considers the complexity of providing appropriate exam arrangements to SLLs in linguistically “super-diverse” communities. The chapter describes the process of developing and trialing task design and considers practical constraints when assessing SLLs in contexts where resources and funding are limited. In chapter nine, D’Este and Ludbrook examine issues of validity arising from the assessment of the English language proficiency of students with SpLDs in the Italian university system. The chapter focuses on a case study in order to describe measures that have been developed and adopted to allow dyslexic students at Venice University access to the CEFR B1 level English entrance test. In their chapter, Erbeli and Pižorn examine the latent structures of Slovene EFL students with Specific Reading Differences (SRDs) and students with no SRDs. Their findings indicate that well-developed fluency and orthography skills in EFL are important for efficient EFL reading competence. The authors propose assessment accommodations and modifications for the group of students with SRDs. In the last chapter of Part II, Brannen and Kozlowska address issues related to students with hearing and visual impairments aiming at increasing awareness and helping teachers adapt to the emerging L2 teaching context in ESL courses at the Université du Québec à Montréal. The chapter identifies common as well as divergent accommodations needed for handicapped populations of students. Part III comprises three chapters that present the perspective of examination boards. In the first chapter of this section, Taylor and Khalifa consider some of the theoretical and practical aspects associated with test accommodations offered by Cambridge English Language Assessment to SLLs with disabilities. The chapter explores current issues and challenges in this area by examining the perspectives of four different stakeholder groups involved in accommodated language tests for test takers with disabilities. Based on qualitative and quantitative data, the authors investigate the match between policy and practice, and identify areas that merit further attention. In the next chapter, Banerjee et al. present a case study of the process by which CaMLA, a large-scale test provider, prepares modified test forms for test takers with special needs. The chapter considers the production of

xii

Introduction

Braille versions of two high-stakes language tests and addresses the challenges of providing test takers with modified test forms to appropriately accommodate their disabilities. In the final chapter of the volume, Arras et al. address current accommodation practices used in the Test of German as a Foreign Language (TestDaF) for blind test takers and discuss issues of validity and fairness of the test. Overall, the chapters of the volume raise important questions and demonstrate the beginning of a new era of conscious epistemological traffic between various disciplines. We remain hopeful that this volume will contribute to recent discussions about the assessment of SLL with learning and other disabilities, and will offer an effective answer to the needs of these special groups of SLLs in our increasingly globalised multicultural world. For this, we would like to sincerely thank all authors for sharing their expertise and experience with us. We would like to propose a few directions that could be followed in the future based on what we learnt by putting these chapters together. Other than overcoming practicalities involved in meeting the assessment needs of this special group of SLLs, much more research is needed to provide the basis of clearer definitions, classifications and identification of SpLD and other disabilities in the SLL population that expand on our current classification systems. We would also like to recommend that future researchers should replicate SpLD studies that have been conducted among monolingual students. Data from both cross-sectional and longitudinal studies should be used in order to develop a classification system that can provide developmental language and cognitive benchmarks and simplify the identification procedures of SpLD children. It is also important to develop identification strategies that can improve understanding of comorbid conditions such as attention deficits and intellectual disabilities. Furthermore, assessment tools should be developed in order to accurately and validly measure student behaviors, and interactions in the contexts of school, community, and home and help teachers identify language, literacy, and academic competencies in SLLs with SpLDs. In tandem with designing appropriate accommodations for standardized accountability assessments, research should also provide empirical evidence that assessment practices for SLLs with learning and other disabilities are appropriate and work well (Abedi et al. 2007). We hope to see more developments and research in the field in the years to come that can follow up on the work presented in this volume.

Assessing L2 Students with Learning and Other Disabilities

xiii

Works cited Abedi, Jamal, Hofstetter Carolyn, and Lord Carol. 2004. “Assessment Accommodations for English Language Learners: Implications for Policy-Based Empirical Research.” Review of Education Research 74:1–28. Kormos, Judit, and Smith Anne Margaret. 2012. Teaching Languages to Students with Specific Learning Differences. Clevedon Bristol: Multilingual Matters. Taylor Lynda. 2012. “Accommodation in Language Testing.” In Cambridge Guide to Second Language Assessment, edited by Christine Coombe, Peter Davidson, Barry O’Sullivan and Stephen Stoynoff, 307–315. Cambridge: Cambridge University Press.

EDITORIAL JUDIT KORMOS

Second-language users and learners constitute a diverse group. Some use two or more languages routinely in their everyday and professional lives, and need to be competent in multiple languages to complete even simple routine tasks. In contrast, other students encounter another language only in a classroom context, and the target language is nothing more than an object of study for them. Notwithstanding the variety of language learning contexts and situations for bi- and multilingualism, learners also show variations in their personality, learning style, family background, motivation to learn and cognitive and physical abilities. While a search in Google Scholar using the term individual differences and second-language learning results in over two million possible hits, indicating how widely individual differences factors in the field of second language learning are researched, the term disabilities and second language learning produces considerably fewer results. It is only in recent years that specific learning differences and perceptual and physical impairments have been focused on, and volume-length publications have been published (e.g. Kormos and Kontra 2008; Kormos and Smith 2012; Martin 2009). Although the notion of test-fairness has long been central to assessment, little research has been done on—and often insufficient consideration has been given to—the needs of disabled students in second language testing. The lack of attention to students who have some kind of impairment that prevents them from fully participating in social activities is disheartening, as disabled people constitute approximately 20 percent of the population (see, e.g., United States Census Bureau 2010). These are students for whom their impairment causes a social barrier and who are often deprived of equal chances in education, despite adequate legislative stipulations being in place. In the context of multilingualism, assessment and disability, we need to consider two important issues: how to identify specific learning differences in two or more languages and how language assessment practices can be made more inclusive. The variety of language learning and multilingual settings makes the assessment of literacy-related learning

xvi

Editorial

differences a daunting task. The assessment of learning differences needs to take into account cognitive, behavioural and educational aspects, and present a comprehensive description of the nature of the difficulties a learner might experience. The cognitive aspects of learning differences are inherently linked to the characteristics of the language(s) to be acquired. Learners from different language backgrounds might have different strategies for processing spoken and written language. If one works with students who constitute a fairly large group and whose language is shared by the assessor, it is advisable to develop tests that are based on the learners’ first language. There are contexts, however, in which this might not be possible. In these cases one has no choice but to find a battery of tests that are not language specific and that focus on what is shared by all languages and the cognitive processes that are common to different language backgrounds. Although the importance of validated and standardized tests of learning differences is unquestionable, the assessment of learning differences might not always take place via formal tests administered by specialists. Continuous assessment performed by teachers via teachers' observations also provides invaluable insights into students’ difficulties and rates of progress, and reveals how these are influenced by their linguistic and cultural background. Learning differences are also relative to the educational context. Some students do not experience any difficulties in inclusive educational contexts, whereas others face particular challenges because of the nature of their instructional setting. Therefore, both the developers of assessment tools and those who apply them should thoroughly consider the learners’ cultural background and educational context. Consequently, the assessment of learning differences in a bi- and multilingual setting cannot consist of a single battery of cognitive tasks; rather, it needs to provide room for an in-depth and comprehensive understanding of individual learners in their own contexts. Another significant issue is how fairness in the assessment of the second-language competence of students with specific learning differences and physical disabilities can be ensured. Language examinations often involve very high stakes, as they serve as gatekeepers to school and university admissions, and job recruitment procedures. Although one often thinks of classroom assessment as involving low stakes, this might also have important consequences for learners, such as progression to another grade or to the next stage of education. Achievement in tests can have substantial influence on students’ selfesteem, self-confidence and self-worth, and might impact on motivation.

Assessing L2 Students with Learning and Other Disabilities

xvii

Tests also impinge on the teaching process through influencing what will be taught and how (Alderson and Wall 1993; Wall 2000). Therefore, it is of great importance that assessment procedures are: valid, i.e. they should give accurate information about learners’ competence; and fair, i.e. they should provide adequate opportunities for learners to display what they know. However, striking a balance between fair and valid assessment procedures is not easy. Many of the skills that language tests seek to measure, such as reading, writing and spelling, are precisely those that are problematic for students with Specific Learning Differences (e.g. dyslexia). Therefore the results may not fully reflect the learners’ language competence. Researching fairness and validity in the context of special educational needs is a difficult task. In high-stakes contexts, one needs to demonstrate that adjustments offered to students do not compromise the validity of the test, in other words they do not affect the construct the test aims to measure. Research in the cognitive validity of tests that investigate the underlying mental processes in test performance offers a promising new direction in this regard. At the same time, however, we also have to consider that any accommodations should offer meaningful help to testtakers and, also, be viable for institutions that administer the assessment. Hence we would need a far larger number of impact studies that elucidate the attitudes and views of various stakeholder groups, such as those of the test-takers, test designers and test administrators. Research in the field of dyslexia and second-language learning is often conducted within the psychometric paradigm, using a quantitative design, and aims at findings that are generalizable across a wide range of settings. Consequently, many studies apply biological and medical models of disability, and many of them adopt an etic perspective, in which the researcher remains an outsider in the research context. Studies conducted within this paradigm mainly apply survey instruments and language tests that are administered to dyslexic and non-dyslexic students to compare their disposition to learning. Evidence gained from studies that compare the performance of students with special needs to the performance of those who have not been identified as requiring assistance is indispensable for making decisions in high-stakes assessment situations. However, the field would also benefit from more qualitative research in students’ testpreparation and test-taking experiences and strategies. Research that aims to present students’ and other stakeholders’ perspectives, and which adopts an insider’s perspective, is rare in this area. This field would greatly benefit from studies that view language learners with special needs as a diverse group interdependent with the

xviii

Editorial

social and instructional context. This could help us better understand the barriers that are present in current assessment practices and educational policies. This volume, which represents a wide range of countries, language backgrounds, educational settings and learners with different types of disability, takes an important first step in this direction.

Works cited Alderson, Charles J., and Dianne Wall. 1993. “Does Washback Exist?” Applied Linguistics 14:115–129. Kormos, Judit, and Anne M. Smith. 2012. Teaching Languages to Students with Specific Learning Differences. Clevedon: Multilingual Matters. Kormos, Judit, and Edit H. Kontra. 2008. Language Learners with Special needs: An International Perspective. Clevedon: Multilingual Matters. Martin, Deidre. 2009. Language Disabilities in Cultural and Linguistic Diversity. Clevedon: Multilingual Matters. United States Census Bureau. 2010. Americans with Disabilities: 2010. Report. http://www.census.gov/prod/2012pubs/p70-131.pdf. Wall, Dianne. 2000. “The Impact of High Stakes Testing on Teaching and Learning: Can This Be Predicted or Controlled?” System 28:499–509.

PART I: DIAGNOSING SLL’S LEARNING AND OTHER DISABILITIES

CHAPTER ONE DISPROPORTIONAL REPRESENTATION OF ENGLISH LEARNERS AMONG STUDENTS IDENTIFIED WITH DISABILITIES: EQUITY AND THE FEDERAL ACCOUNTABILITY SYSTEM KEIRA GEBBIE BALLANTYNE

Almost four and a half million students in the United States are identified as English learner students—non-native English-speaking students who are not able to meet academic content standards in classes where English is the language of instruction. By law, these students are provided with services to enhance their English language proficiency. Almost half a million of these students are also identified as students with disabilities. Historically, students from marginalized populations have been found to be disproportionately identified as having disabilities—both over- and underrepresented. Educators have been particularly challenged by English learner students. They often work with these students without sufficient training in second language acquisition processes, or appropriate tools to adequately assess whether these students have disabilities. English learner students thus run the risk of being misclassified as students with disabilities. The current chapter finds that despite this risk, the federal educational accountability system in the United States does not systematically collect the data necessary to measure whether or not English learner students are disproportionately represented in special education at the school, district, or state level. Because disproportionality data are not systematically collected, the inequities created by misclassification become invisible. There is no impetus to provide resources such as more accurate assessments, increased staffing, or additional professional development for teachers.

4

Chapter One

1. Introduction There are approximately 50 million students served by US public schools. Of these students, 4.4 million are identified as being English learner (EL) students, and 5.8 million are students with disabilities (SWDs). The categories of English learners and students with disabilities intersect, and approximately 480,000 students are identified as being EL SWDs (Ed Data Express, IDEA Data Accountability Center). Providing an appropriate education for the population of English learners with disabilities has proved to be a particularly challenging problem for schools and districts. This challenge extends to ensuring that English learner children are not misclassified as having language or learning disabilities when no such disability exists. Conversely, it ensures that when an EL child does have a disability, the disability is accurately identified by educators. Large-scale patterns of such misclassification, when they are specific to a particular demographic group, are referred to in the literature as disproportionalities. Disproportionalities can be measured via a calculation of the relative risk ratio for a particular demographic group. The risk ratio is a measure of the relative risk that membership of a particular demographic group increases the likelihood of being identified as a student with a disability. If demographic group membership has no correlation with the risk of being identified as a SWD, then no disproportionality exists. In the United States, education is managed by a patchwork of jurisdictions at the federal, state, and district level. At the federal level, the Elementary and Secondary Education Act (ESEA)—reauthorized in 2001 as No Child Left Behind (NCLB)—aims, in part, to reduce academic achievement gaps between relatively disadvantaged groups of students. The accountability system laid out in the legislation is the mechanism by which this civil rights aim is measured. A key innovation of the 2001 reauthorization of the legislation was to require states and districts to “disaggregate” data by demographic categories including ethnicity, socioeconomic status, and English language proficiency. The law is currently due for reauthorization by the US Congress. NCLB has been criticized widely for its reliance on standardized test scores as accountability measures and also for punitive consequences when test score targets are not met. Johnson and Avelar (2010) argue that accountability systems which rely too heavily on outcome measures like test scores can miss other inequities which are made visible by other types of data. They term this the wallpaper effect—inequities hidden because of

Disproportional Representation of English Learners among Students

5

gaps in data collections or questions not asked within accountability systems. These gaps matter because large-scale data collections have influence on resource allocations. They affect decisions about when to create new or more accurate assessments, when to increase staffing in schools and districts, and when to offer increased teacher education and professional development. Federal data collection efforts, in particular, because of their nationwide influence, have broad implications not only at a national level but also at the school and classroom levels, where data are collected. In this paper, I aim to demonstrate that the needs of English learner students with disabilities—and those who are misclassified—are in danger of being wallpapered because data gaps mean that equity issues are made invisible. Specifically, I examine exhaustively current national requirements for the collection of demographic data on this population of students. Also, I discuss best practices for uncovering disproportionalities. Finally, I conclude that given the current nature of the data collection system, these best practices cannot be applied. Despite this, there is a means to compare the proportion of EL students to the population of EL students with disabilities which might uncover evidence of disproportionality. I find that, at a national level, there generally appears to be no disproportionality in the assignment of EL students to special education services. However, there is a greater degree of variation at the state level. Earlier findings from the literature indicate that disproportionalities which are not apparent on a macro scale can be uncovered at finer levels of granularity. State level data from the current analysis should be viewed with some caution. Furthermore, more detailed analyses at the district, school, grade, or program type level may be required in order to truly understand the exact manifestations of disproportionality for this population of students. Top-down pressure from required federal data collections is one method to ensure that the data required to carry out such analyses are collected. Without accurate and comparable counts of the number of EL students, the number of students with disabilities, and the number of students who fall into both of these categories, disproportionalities can remain hidden. The key finding of the current study, therefore, is not the numerical findings which emerge from the method of analysis employed herein. It is rather that the commonly accepted metric for assessing disproportionality cannot be applied given the available data. Accountability for any inequities due to the disproportional representation of English learner students in special education is hence not a structural component of the accountability system.

6

Chapter One

I begin by presenting a brief synopsis of the definition of disproportionality and of key assumptions underpinning the concept. I then move to an overview of the literature on disproportionalities and English learner students. This is followed by a discussion of the role of the accountability system in detecting inequities generally. I next provide a technical explanation of the risk ratio formula, and then a review of publicly available federal data on EL and SWDs. These data are analyzed by state to reveal percentage point differences in the population of English learner students and the population of students with disabilities who are English learners. I conclude with a discussion of the limitations of these data and particularly the limitations incurred when the risk ratio formula cannot be employed.

2. What is disproportionality? The documented history of the intersection of ethnic and racial categories and disability in education extends at least to the onset of the last century. Historians have documented that Black and Latino students were categorized as having disabilities at proportions greater than White students (Artiles et al. 2011). Multiple commentators within the special education literature have noted that students are not proportionally represented within the population of students with disabilities (see, e.g., Artiles et al. 2001; Harry and Klingner 2006; Artiles et al. 2011). Instead, students from particular demographic backgrounds tend to be more or less at-risk as being identified as having a disability. This is referred to in the literature as disproportionality: the number of children from a particular demographic background who are diagnosed as having a disability is out of proportion to their share of the general population of children. Disproportionalities in special education continue to this day. For instance, African American male students constitute approximately 9% of the total student population, but close to 20% of the total population of students who are identified with an intellectual disability1. Fig. 1 illustrates this disproportionality.

1

Note that until October 2010 the term mental retardation rather than intellectual disability was used in the Individuals with Disabilities Education Act, and the former term is retained in some data collections (National Dissemination Center for Children with Disabilities 2011).

Disproportional Representation of English Learners among Students

7

Fig. 1. Example of disproportionality

The definition of disproportionality relies on the assumption that, all other factors being equal, disabilities ought to be equally distributed among the population. As Samson and Lesaux put it, “[t]here is no empirical reason to expect that disabilities should occur in some subgroups more than others” (2009, 149). If a particular group of students is found to be more or less heavily represented in the population of students with disabilities than they are in the general student population, the assumption is that something is amiss. Either some of these students have disabilities which are not being identified (underrepresentation), or students who do not have a disability have been misclassified as having one (overrepresentation). This paper will take the assumption of baseline proportionality to be axiomatic for the purposes of analysis. Artiles, Sullivan et al. (2010) note three potential causes of disproportionality that have been identified in the literature. First, biases may exist in the professional practices of educators, which may result in overrepresentations, combined with the deficit perspectives of students from diverse backgrounds. The second type of analysis—sociohistorical—traces the roots of the problem to complex intersections of historical factors, structural inequities, race, class, and ethnicity. As Artiles, Kozleski et al. put it, [w]e assume disproportionality is a symptom of larger cultural and historical processes that shape the educational experiences and opportunities of students from historically underserved groups.2

2

Artiles, Kozleski et al. 2010, 296.

8

Chapter One

Finally, sociodemographic accounts attend to individual or community characteristics which might lead to disproportionate incidences of disability, particularly in communities in poverty, such as low birth weight, or prematurity (see also Artiles, Kozleski et al. (2010) on the problem of essentializing such characteristics as inherent to populations rather than emerging from environmental conditions). To these, high levels of lead could be added, as well as other pollutants which may engender negative health effects (Federal Interagency Forum on Child and Family Statistics 2008). While causative factors may emerge from a combination of the origins put forth by each of these types of analyses, sociodemographic factors would in fact result in actual disabilities being present at a greater degree in some populations over others. In other words, if sociodemographic factors are truly resulting in disproportionalities, then it is not the case that students are being misclassified. However, if these contributory factors to poor health and learning outcomes are preventable in middle class communities, then they ought to be preventable in all communities too, given the political will. So such a finding would mean that efforts to stop the inequities, which result from disproportionality, should involve the health of the wider community outside of the sphere of just the school and the education system.

2.1 EL and disproportionality In general, there has been greater attention to disproportionalities across racial and ethnic groups than to those across linguistic groups. The literature on disproportionalities among EL students is more scant, although with some uptick in recent years. A number of causes for misidentification of ELs have been put forward. These include a lack of understanding of typical second language acquisition trajectories on the part of education professionals, misinterpretation of difficulties with highly demanding language tasks, inadequate instruction, and inappropriate assessments for identification. Artiles, Kozleski et al. (2010) remind us that diagnosis of highincidence learning disabilities has a heavy subjective component. Educators may not have had sufficient specialised training so as to distinguish typical second language acquisition processes from language impairments. Furthermore, they may misunderstand the educational needs of ELs, and students may not have access to linguistically proficient educators. Educators who do not have sufficient experience in or training for working with students who are in the process of second language

Disproportional Representation of English Learners among Students

9

acquisition may employ mistaken assumptions about causes of language difficulties in the second language. They may attribute the lack of full comprehension and fluency to a learning disability (Klingner et al. 2006; Samson and Lesaux 2009). Because both students with learning disabilities and EL students can have difficulty with highly demanding English language tasks, educators are required to have specific knowledge, experience, and tools to tease out the causes of such difficulty. Lacking such tools, they may overidentify or fail to identify disabilities in EL students. Furthermore, EL students, with or without disabilities, may not receive adequate instruction as they progress through schooling, which can lead to additional problems in identification (Sullivan 2011). Finally, assessments—both for identification and for other purposes— may not be appropriate when English is the medium of assessment. Moreover, appropriate assessment instruments may not be available in the native languages of culturally and linguistically diverse students. A number of commentators recommend bilingual assessments for bilingual children, particularly in the case of assessments to detect language disabilities (see, e.g., Peña et al. 2011; Thibeault 2009). In general, measured at a national level, EL students are no more or no less likely to be identified as having disabilities than are White3 students. This macro level proportionality, however, masks disproportionalities that exist at other units of analysis. Descriptions in the literature uncover disproportionalities in two directions. English learner students may be overrepresented in the special education category when they are identified as having disabilities at rates higher than the White population. At this point, there is a concomitant assumption that a subset of these students is identified as having a disability when none is present. Conversely, EL students with disabilities may not be identified, resulting in underrepresentation and a situation where students are not provided with the services they need in order to access an optimal learning environment. Sullivan cautions that in order to truly uncover the inequities that disproportionality indicates, data should be explored at multiple levels— school, district, and state. Her 2011 study finds that disproportionality occurs for EL students in some disability categories (overrepresentations in the categories of learning disabilities and intellectual disabilities). A number of additional studies which examine fine-grained data have

3

I follow the convention of using White students as the comparison group, as explained by Artiles et al. (2005).

10

Chapter One

brought the open disparities into play, which would have been missed if data had been analyzed only at the macro level. Disproportionalities may manifest differently at different grade levels. Rueda et al. (2002) find that identification of ELs who require special education services increases around the fifth grade. They hypothesize that this may be due to decreasing first language support in the later grades. In an analysis of data from the Early Childhood Longitudinal Study— Kindergarten Cohort (ECLS-K)—Samson and Lesaux (2009) find that linguistic minority children are underrepresented in special education services in kindergarten and first grade, but overrepresented by third grade. They also note that linguistic minority children who have communication difficulties are often identified as having speech impairments in the early childhood grades. In later grades, the same children are more often identified as having a learning disability. They speculate that teachers may be reluctant to refer ELs to special education services until they develop English proficiency. Furthermore, they support that educators may hold the mistaken belief that students are not eligible to access services for language instruction and for special education simultaneously. The type of instructional program may also impact disproportionality. Students in US schools may be instructed in programs that use the students’ home language in addition to English, or they may be in programs that use English alone. The specific type of program in place is dependent upon multiple factors. These include state regulations and policy, the availability of resources for a particular language group, the demographics of the student population, and specific policies at the school and district level. Artiles et al. (2005) find that the type of instructional program makes a difference. EL students in programs where the home language was not used as a language of instruction were more likely to be referred to special programs than such students in programs where instruction was in two languages. Artiles et al. (2011) raise a number of additional equity questions regarding EL students with disabilities. They note Zehler et al.’s (2003) finding that EL students with disabilities are less likely than other EL students to be educated in settings with home language support. They consider the literature on charter schools, and question whether the market-driven forces of charter schools represent a structural disincentive to provide services to ELs, students with disabilities, and of course ELs with disabilities. Finally, Albus et al. (2009) study on state assessment data found that, nationwide, 30 states did not disaggregate assessment data for EL students with disabilities.

Disproportional Representation of English Learners among Students

11

The valuable work which has been done to uncover these complex inequities has relied upon individuals or small research teams working with individual schools, districts, or states to collect the data required. To date, however, there is no systematic collection of national data which allows for the critical review of disproportionalities, affecting English learner students in special education programs.

3. Accountability and educational equity For the past several decades, a key concept in education policy making in the United States has been that of accountability. In a nutshell, it is the notion that taxpayer-funded schools, districts, and state offices of education ought to have metrics in place to ensure that public education indeed provides that which it sets out to do. While the specific purpose or purposes of the educational enterprise remain contested, since the 2001 reauthorization of the Elementary and Secondary Education Act (ESEA), known as No Child Left Behind (NCLB), there has been intensified focus on disaggregating accountability data. The aim is to uncover disparate educational outcomes across ethnic and linguistic groups (National Academy of Education 2009). Under NCLB, [states must complete annual] report cards [which provide] [...] information, in the aggregate, on student achievement at each proficiency level on the State academic assessments […] (disaggregated by race, ethnicity, gender, disability status, migrant status, English proficiency, and status as economically disadvantaged).4

The federal policy imperative to report these data has resulted in increased efforts nationwide to enhance data collection. Data do not include only students’ academic achievement, but also population counts of students and of students in specific subgroups, including English learners. At the same time, data collections required under the Individuals with Disabilities Education Act (IDEA) have made data collections on SWDs more comprehensive and accurate. More sophisticated data collections on behalf of state education agencies are paired with the launch of user-friendly online data tools which make these data available to the general public. This means that accountability data are now available for public examination in ways that have not been possible before. So, researchers, advocates, policy-makers and educational stakeholders now have available data which more than ever before can provide compelling 4

ESEA §1111(h)(2).

12

Chapter One

arguments for educational equity. Armed with a detailed picture of the disparities in outcomes for linguistic minorities, advocates have the capacity not only to change hearts and minds but also to provide clear evidence of educational inequities to legislators and the legal system. They can pinpoint the particular needs and requirements of their local educational systems. They can also push legislators and funding sources to meet the teaching, assessment, and professional development and instructional needs of those students who require the most help. New examinations of available data are a key point of leverage for education policy advocates. ǹ frank appraisal of what is not explicitly measured is just as critical as an examination of the types of inequities the available data might uncover. If the purpose of metrics within the educational accountability system is ultimately to effect change within that system (either by explicitly requiring change at the legislative level, or by providing, by extension, data which give rise to policy pressure), then an analysis of educational policy must also consider what is not measured and reported. Artiles, Sullivan et al. (2010) note that while states are required to report and examine disproportionalities according to race and ethnicity categories, they are not required to do so for the category of English learner students. I argue below that not only are they not required to report and examine disproportionalities, but also disproportionality cannot be calculated in a systematic fashion, given the data reported.

4. Calculating disproportionality: The risk ratio Assessing numerically whether students from particular groups are disproportionately assigned to special education categories rests on the availability of data establishing the proportion of students of a particular group in the special education population. Such data also allow for the comparison of the previously mentioned proportion to the proportion of a control group in the special education population. The most commonly used method is the risk ratio. It is a ratio which numerically represents disproportionate assignments of students in specific demographic groups to special education (Skiba et al. 2005; Sullivan 2011). To calculate the risk ratio for English learner students identified as students with disabilities, the percentage of total EL students identified as SWDs is compared with the percentage of White students identified as SWDs (see Fig. 2).

Disproportional Representation of English Learners among Students EL students with disabilities Total population of EL students

13

White students with disabilities Total population of White students

Fig. 2. Risk ratio calculation for EL students with disabilities

ǹ risk ratio greater than one indicates an overrepresentation whereas a ratio less than one indicates underrepresentation. There is a certain amount of variation in the literature in terms of cut-offs for determining when a risk ratio is in fact an indicator of disproportionality. This variation ranges from 1.2 through 4.0 interpreted as evidence of overrepresentation (Sullivan 2011, 323). The calculation of the risk ratio, however, depends critically on the ability to establish the proportion of EL students that are identified as students with disabilities. As I will show below, however, data on EL students with disabilities and data on the overall EL population are collected separately, making this proportion impossible to establish at the state and national level.

5. Federal data on EL students with disabilities Data collected by the US Department of Education (2012) for the 50 states, the District of Columbia, and the Commonwealth of Puerto Rico indicate that a total of 49,957,916 students were served in K-12 public schools in the school year 2010–2011. Of these, 4,665,488 were classified as English learners (ED Data Express). A total of 5,811,669 students nationwide were identified as students with disabilities, and there were 481,129 students with disabilities who were also English learners (IDEA Data Accountability Center). English learner students are defined in federal law as “limited English proficient” students. Their abilities in English are not sufficient to achieve success in classrooms where the language of instruction is English. These students are not able to pass the state assessment of English language proficiency (ESEA §9101(25)). The establishment of English language proficiency assessments and the cut scores which indicate proficiency are put in place by each state. Similarly, students with disabilities are also defined within the federal legislation. IDEA defines a child with a disability as a child evaluated [...] as having mental retardation, a hearing impairment (including deafness), a speech or language impairment, a visual impairment (including blindness), a serious emotional disturbance (referred to in this part as emotional disturbance), an orthopedic impairment, autism,

14

Chapter One traumatic brain injury, any other health impairment, a specific learning disability, deaf-blindness, or multiple disabilities, and who, by reason thereof, needs special education and related services.5

5.1 Population data A proportionality calculation assessing the ratio of ELs in the population of student with disabilities can, in fact, be applied using available population data. It is not, however, the standard risk ratio calculation used in other literature on disproportionality, so its applicability is limited, and no comparisons can be made to non EL populations. Two sources of data are drawn on for this calculation. Data on characteristics of students with disabilities under the Individuals with Disabilities Education Act (IDEA) are collected. These data include multiple measures of characteristics of this population of students, including demographics, type of educational environment, staffing, assessment data, data on students exiting the category, discipline rates and dispute resolution. Data collected under IDEA Part B (for children aged 3– 21) and Part C (for children 0–3) are available through the IDEA Data Accountability Center, funded by the US Department of Education Office of Special Education Programs (OSEP). Pertinent to the current question, this data set identifies (i) the total number of students with disabilities, per state, and (ii) the total number of students with disabilities who are English learners, per state. These data can be used to calculate the percentage of students with disabilities, per state, who are English learners. Note that there is no baseline axiom that English learner students are equally distributed throughout the population. English learners constitute almost 23% of the total student population in California but less than one percent in West Virginia. Social, historical, and geographical factors influence the distribution of EL students across the United States, and this is not necessarily a prima facie indication of inequity at the state level. There is thus no concomitant expectation that English learners should be equally distributed throughout the population of students with disabilities. If we assume, however, that disabilities are equally distributed among the population, then the proportion of EL students in any given state ought to remain equal across both the total population and the population of students with disabilities. A disproportionality calculation can thus be conducted by comparing the proportion of ELs in the total population of a state with the proportion of ELs in the population of students with disabilities in that state. If the percentage point difference between these 5

IDEA §300.8, and see note 1 on the use of the term mental retardation.

Disproportional Representation of English Learners among Students

15

figures is small, then there is no evidence of disproportionality. If the percentage point difference is large, however, English learner students are likely to be either over- or under-represented. Publicly available federal data involve the total number of students, the number of students who have been identified as English learners, and the proportion who are English learners, by state, and for the US as a whole. The total number of students is drawn from the Institute of Education Sciences National Center for Education Statistics Common Core of Data (CCD). Data for the CCD are gathered from surveys sent annually to state departments of education. The number of students who are identified as English learners comes from the Consolidated State Performance Reports (CSPR) of the states, also known as EdFacts. This is a data collection submitted annually by states to the US Department of Education. The data were accessed via the Ed Data Express of the US Department of Education.6 These data pertain to K12 students and are for school year 2010–2011. These two sets of data can be used to calculate the proportion of students who are English learners, by state, as well as the national proportion. See Table 1 in the Appendix for a breakdown of these data for the 2010–2011 school year. The proportion of total students in each state who are English learners can be compared with the proportion of students with disabilities who are English learners. Numbers of students with disabilities and subgroup characteristics of these students are collected under the authority of IDEA. These data are made publicly available via the IDEA Data Accountability Center. For data for each state and national totals see Table 2, Appendix. Available data, then, include (i) the percentage of the total student population who are English learners, and (ii) the percentage of the population of students with disabilities who are English learners. It should be noted that the key piece of data required to calculate the risk ratio for English learners students in special education populations, namely the percentage of English learner students who have been identified as students with disabilities, is missing.7 Nonetheless, if one assumes that the proportion of English learner students in any given state ought to remain constant across the total population and the population of SWD, the percentage point difference between these two proportions can illustrate disproportionalities. 6

CCD data can be accessed directly from http://nces.ed.gov/ccd/. CSPR data are archived online on the US Department of Education website at http://www2.ed.gov/admins/lead/account/consolidated/index.html. 7 Because CSPR and IDEA are distinct data collections, it is inadvisable to calculate the proportion of ELs with disabilities by using the number of ELs with disabilities from the IDEA data collection as the numerator and the total count of ELs from the CSPR data collection as the denominator.

16

Chapter One

A negative percentage point difference indicates that the proportion of ELs with disabilities is smaller than their share of the general population (suggesting underrepresentation), while a positive percentage point difference is likely to be indicative of an overrepresentation (but see the section on limitations of these data, below). At a national level, the percentage point difference is -1.1 (note that, as Wyoming did not report IDEA data, it is excluded entirely from the national percentage point calculation).

Fig. 3. Percentage point difference between SWDs who are ELs and total population of students who are ELs8 8

Note that Wisconsin did not provide IDEA data for SY 2010–2011.

Disproportional Representation of English Learners among Students

17

A total of 41 states had a negative percentage point difference, with only 10 states showing a positive percentage point difference. The biggest positive and negative differences were in New Mexico (7.3 percentage points) and Arizona (-4.6 percentage points). Fig. 3, above, provides a graphic overview of the percentage point differences, indicating those states with positive and negative percentage point differences as well as the relative magnitudes of the differences. The percentage point data are presented in full in Table 3 in the Appendix.

5.2 Limitations of these data At first glance, it may appear that the data needed to calculate the risk ratio are, in fact, available. The number of English learners with disabilities is provided in Table 2, while the total number of English learners is available in Table 1. The percentage of ELs who have been identified as students with disabilities, then, ought to be a simple ratio of these two figures. Limitations in the comparability of the two data collections mean that such a calculation is inappropriate. The former figure is drawn from IDEA data collections and the latter from CSPR/EdFacts data. Because these data are collected at different times and with different criteria—IDEA data for children aged 6–22 and CSPR for K-12 grades—a calculation which mixes data from the two collections is problematic. In fact, these limitations extend to comparisons of the percentage point difference between the populations of ELs and that of ELs with disabilities. This limitation in turn means that the risk ratio calculation, which is generally accepted in the literature as the best practice for calculating disproportionality, cannot be applied to these data. The fact that the risk ratio calculation is widely used means that (i) there are established cut scores for the point at which a positive or negative risk ratio warrants action rather than just indicating noise in data, and (ii) risk ratios are comparable across different demographic groups. Differences between states in the way that English learner students are identified also contribute to difficulties in interpreting these data. A student who is identified in state A as an English learner student may be identified in state B as a student who does not require language instruction services. This happens because assessments and cut scores used to identify EL students differ from state to state. Finally, as noted above, analyzing data at the state level can mask disproportionalities that might exist at the district, school or grade level, or might be tied to the specific type of instructional program in place for EL students.

18

Chapter One

6. Conclusion Close to half a million children in the United States are identified as English learners with disabilities. Prior findings in the literature indicate that specific racial and ethnic groups often experience inequitable treatment in being identified as students with disabilities. More specifically, there are historical and current disproportionalities in the proportion of non-White students in the general population and their proportion in special education populations. English learner students can be misclassified due to the bias of educators, the educators’ lack of sufficient training on and understanding of second language acquisition processes, inappropriate assessments or lack of opportunities for students to learn. The educational accountability system is based on the premise that publicly funded education ought to be accountable to the public. Accountability for disproportionalities in the identification of English learner students as students with disabilities is not, however, a structural component of this system. This gap in the accountability system means that schools and districts are not required to systematically collect data which might uncover any inequities in special education services as these are provided to linguistic minority students. An analysis of the available national data on English learner students with disabilities indicates that, overall, there appears to be approximate proportionality for EL students, but that states vary in whether their proportion of EL students with disabilities is congruent with the proportion of EL students in the total state student population. The current data, however, suffer from a number of limitations. Chief among them is the fact that the commonly accepted calculation of disproportionality, the risk ratio, cannot be applied to data as they are currently collected. Due to these crucial limitations in the data, educational stakeholders, advocates, and policy-makers lack the data required to push for equitable resources for English learners with disabilities. Disproportional assignment of English learner students to special education is wallpapered. It is hidden by systematic data collection structures which mean that accurate evaluation of inequities is not possible at a national scale, and hence also not available for local advocates at the state, district, or school level. Given the literature on the challenges faced by educators of this population of students, the data gap also means that it is difficult to advocate for resources such as more accurate identification tools (for example, native language assessments for learning disabilities), and for additional classroom, staffing, and professional development resources.

Disproportional Representation of English Learners among Students

19

Works cited Albus, Debra, Martha Thurlow, and Kristi Liu. 2009. “State Reports on the Participation and Performance of English Learners with Disabilities in 2006–2007.” Technical Report 54. Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes. Artiles, Alfredo J., Elizabeth B. Kozleski, Stanley C. Trent, David Osher, and Alba Ortiz. 2010. “Justifying and Explaining Disproportionality, 1968–2008: A Critique of Underlying Views of Culture.” In Exceptional Children 76 (3):279–299. Artiles, Alfredo J., Rueda Robert, Jesus Salazar, and Ignacio Higareda. 2005. “Within-group Diversity in Minority Disproportionate Representation: English Language Learners in Urban School Districts.” In Exceptional Children 71:283–300. Artiles, Alfredo J., Amanda L. Sullivan, Federico R. Waitoller, and Rebecca R. Neal. 2010. “Latinos in Special Education: Equity issues at the Intersections of Language and Culture.” In Handbook of Latinos and Education: Theory, Research, and Practice, edited by Enrique G. Murillo Jr., Sofia Villenas, Ruth Trinidad Galván, Juan Sánchez Muñoz, Corinne Martínez and Margarita Machado-Casas, 361–381. New York: Routledge. Artiles, Alfredo J., Beth Harry, Daniel J. Reschly, and Philip C. Chinn. 2001. Over-identification of Students of Color in Special Education: A Critical Overview. Chicago: Monarch Center. Artiles, Alfredo J., Federico R. Waitoller, and Rebecca R. Neal. 2011. “Grappling with the Intersection of Language and Ability Differences: Equity Issues for Chicano/Latino Students in Special Education. In Chicano School Failure and Success: Past, Present, and Future, edited by Richard R. Valencia, 213–234. New York, NY: Routledge Falmer. Ed Data Express. “Data on English Learner Students, by State, School Year 2010–2011.” Accessed October 22, 2012. http://www. eddataexpress.ed.gov/. Elementary and Secondary Education Act of 1965, Pub. L. No. 89-10, 79 Stat. 27 (codified at 20 U.S.C. § 6301). Federal Interagency Forum on Child and Family Statistics. 2008. “America's Children in Brief: Key National Indicators of Well-being, 2008.” In Federal Interagency Forum on Child and Family Statistics. Washington, DC: U.S. Government Printing Office. Harry, Beth, and Janette Klingner. 2006. Why Are So Many Minority Students in Special Education? Understanding Race and Disability in Schools. New York: Teachers College Press.

20

Chapter One

Individuals with Disabilities Education Act (IDEA) Data Accountability Center. “Data on Students with Disabilities, Aged 6–21, School Year 2010–2011.” Accessed October 16, 2012. https://www.ideadata.org/. Johnson, Ruth S., and Robin Avelar La Salle. 2010. The Wallpaper Effect: Data Strategies to Uncover and Eliminate Hidden Inequities. Thousand Oaks, CA: Corwin. Klingner, Janette K., Alfredo J. Artiles, and Laura Mendez Barletta. 2006. “English Language Learners Who Struggle with Reading: Language Acquisition or LD?” In Journal of Learning Disabilities 39 (2):108– 128. National Academy of Education. 2009. “Standards, Assessments, and Accountability.” In Education Policy White Paper. Washington, DC: Author. Accessed November 27, 2012. http://www.naeducation.org/ Standards_Assessments_Accountability_White_Paper.pdf. National Dissemination Center for Children with Disabilities. 2011. “Intellectual Disability. In NICHCY Disability Fact Sheet 8.” Washington, DC: Author. Accessed November 24, 2012. http://nichcy.org/disability/specific/intellectual#idea. Peña, Elizabeth D., Lisa M. Bedore, and Ronald B. Gillam. 2011. “Two to Untangle; Language Impairment and Language Differences in Bilinguals.” In AccELLerate! 3 (3):7–9. Rueda, Robert, Alfredo J. Artiles, Jesus Salazar, and Ignacio Higareda. 2002. “An Analysis of Special Education as a Response to the Diminished Academic Achievement of Chicano/Latino Students: An Update.” In Chicano School Failure and Success: Past, Present, and Future, edited by Richard R. Valencia, 2nd edition, 310–332. London: Routledge Farmer. Samson, Jennifer F., and Nonie K. Lesaux. 2009. “Language-minority Learners in Special Education: Rates and Predictors of Identification for Services.” In Journal of Learning Disabilities 42 (2):148–162. Skiba, Russell J., Lori Poloni-Staudinger, Ada B. Simmons, Renae L. Feggins-Azziz, and Choong-Geun Chung. 2005. “Unproven links: Can Poverty Explain Ethnic Disproportionality in Special Education?” In Journal of Special Education, 39:130–144. Sullivan, Amanda L. 2011. “Disproportionality in Special Education Identification and Placement of English Language Learners.” In Exceptional Children 77 (3):317–334. Thibeault, Connie H. 2009. “The Dual Language Assessment: First Language Insights into Second Language Acquisition.” In AccELLerate! 1(3):4–5.

Disproportional Representation of English Learners among Students

21

US Department of Education, Office for Civil Rights. 2012. “Civil Rights Data Collection, 2006 Projections.” Accessed October 17. http://ocrdata.ed.gov/. Zehler, Annette M., Howard L. Fleischman, Paul J. Hopstock, Todd G. Stephenson, Michelle L. Pendzick, and Saloni Sapru. 2003. Descriptive Study of Services to LEP Children and LEP Children with Disabilities: Summary of Findings related to LEP and SpEd-LEP children (Policy Report). Arlington, VA: Development Associates, Inc.

Appendix State Name Alabama

Total Number of Students

Number of Students Identified as EL

Percentage of Students Identified as EL

755,552

20,124

2.7%

Alaska

132,104

16,313

12.3%

Arizona

1,071,751

100,683

9.4%

Arkansas

482,114

32,743

6.8%

California

6,289,578

1,441,643

22.9%

Colorado

843,316

110,377

13.1%

Connecticut

560,546

31,121

5.6%

Delaware

129,403

6,864

5.3%

District of Columbia

71,284

6,238

8.8%

Florida

2,643,347

264,183

10.0%

Georgia

1,677,067

88,144

5.3%

Hawaii

179,601

19,709

11.0%

Idaho

275,859

16,280

5.9%

Illinois

2,091,654

179,824

8.6%

Indiana

1,047,232

49,191

4.7%

495,775

21,415

4.3%

Kansas

483,701

43,454

9.0%

Kentucky

673,128

15,743

2.3%

Louisiana

696,558

13,042

1.9%

Maine

189,077

5,183

2.7%

Maryland

852,211

51,911

6.1%

Massachusetts

955,563

70,459

7.4%

Iowa

22

Chapter One

Michigan

1,587,067

73,881

4.7%

Minnesota

838,037

69,681

8.3%

Mississippi

490,526

6,710

1.4%

Missouri

918,710

22,712

2.5%

Montana

141,693

3,300

2.3%

Nebraska

298,500

20,548

6.9%

Nevada

437,149

87,286

20.0%

New Hampshire

194,711

4,697

2.4%

1,402,548

56,140

4.0%

New Jersey New Mexico

338,122

54,284

16.1%

New York

2,734,955

238,792

8.7%

North Carolina

1,490,605

110,086

7.4%

North Dakota Ohio Oklahoma Oregon

96,323

3,687

3.8%

1,754,191

38,312

2.2%

659,911

39,648

6.0%

570,720

62,403

10.9%

Pennsylvania

1,793,284

47,091

2.6%

Puerto Rico

473,735

2,994

0.6%

Rhode Island

143,793

7,399

5.1%

South Carolina

725,838

36,385

5.0%

South Dakota

126,128

4,921

3.9%

Tennessee

987,422

32,142

3.3%

Texas

4,935,715

743,810

15.1%

Utah

585,552

44,845

7.7%

Vermont

96,858

1,676

1.7%

Virginia

1,251,440

97,033

7.8%

Washington

1,043,788

98,467

9.4%

West Virginia

282,879

1,727

0.6%

Wisconsin

872,286

48,205

5.5%

Wyoming

89,009

1,982

2.2%

49,957,916

4,665,488

9.3%

United States Total

Table 1. Ed Data Express: Students who are ELs, K-12, 2010–2011

Disproportional Representation of English Learners among Students

Total Number of SWDs

Number of those students Identified as EL

Alabama

74,794

1,546

2.1%

Alaska

15,944

2,377

14.9%

Arizona

111,060

5,366

4.8%

State Name

23

Percentage of those Students Identified as EL

Arkansas

51,847

2,799

5.4%

California

599,770

176,256

29.4%

Colorado

72,913

8,675

11.9%

Connecticut

60,234

4,307

7.2%

Delaware

16,485

1,093

6.6%

District of Columbia

10,990

691

6.3%

Florida

332,781

23,231

7.0%

Georgia

161,633

6,537

4.0%

Hawaii

17,318

1,364

7.9%

Idaho

23,462

823

3.5%

Illinois

266,589

11,864

4.5%

Indiana

147,348

5,010

3.4%

Iowa

61,123

2,555

4.2%

Kansas

56,269

2,888

5.1%

Kentucky

84,407

1,494

1.8%

Louisiana

72,516

745

1.0%

Maine

28,437

520

1.8%

Maryland

90,615

3,462

3.8%

Massachusetts

150,864

9,467

6.3%

Michigan

195,774

6,247

3.2%

Minnesota

107,774

7,002

6.5%

Mississippi

53,847

406

0.8%

Missouri

111,273

1,801

1.6%

Montana

15,105

339

2.2%

Nebraska

39,249

1,633

4.2%

Nevada

41,131

7,947

19.3%

New Hampshire

26,785

554

2.1%

24

Chapter One

New Jersey

214,929

4,816

2.2%

New Mexico

41,390

9,649

23.3%

New York

389,619

36,136

9.3%

North Carolina

166,674

10,380

6.2%

North Dakota

11,456

404

3.5%

Ohio

237,000

3,511

1.5%

Oklahoma

88,952

2,893

3.3%

Oregon

71,658

7,651

10.7%

Pennsylvania

264,008

5,962

2.3%

Puerto Rico

112,608

239

0.2%

Rhode Island

22,387

885

4.0%

South Carolina

89,206

3,008

3.4%

South Dakota

15,288

457

3.0%

Tennessee

107,167

1,825

1.7%

Texas

400,525

58,122

14.5%

Utah

61,288

6,316

10.3%

Vermont

12,174

131

1.1%

Virginia

145,257

12,011

8.3%

Washington

113,703

11,137

9.8%

West Virginia

39,400

193

0.5%

Wisconsin

108,643

6,404

5.9%

Wyoming

no data

no data

no data

5,811,669

481,129

8.3%

United States Total

Table 2. Proportion of SWDs who are ELs, students aged 6–21, 2010–20119

9

Source: IDEA Data Accountability Center.

Disproportional Representation of English Learners among Students

State

PPD

Alabama

-0.6

Alaska

2.6

Arizona

-4.6

State

25

PPD

State

PPD

Kentucky

-0.6

North Dakota

-0.3

Louisiana

-0.8

Ohio

-0.7

Maine

-0.9

Oklahoma

-2.8

Arkansas

-1.4

Maryland

-2.3

Oregon

-0.3

California

6.5

Massachusetts

-1.1

Pennsylvania

-0.4

Colorado

-1.2

Michigan

-1.5

Puerto Rico

-0.4

Connecticut

1.6

Minnesota

-1.8

Rhode Island

-1.2

Delaware

1.3

Mississippi

-0.6

S. Carolina

-1.6

Missouri

-0.9

South Dakota

-0.9

Dist. of Columbia

-2.5

Montana

-0.1

Tennessee

-1.6

Florida

-3

Nebraska

-2.7

Texas

-0.6

Georgia

-1.2

Nevada

-0.6

Utah

2.6

Hawaii

-3.1

New

Vermont

-0.7

Idaho

-2.4

Hampshire

-0.3

Virginia

0.5

Illinois

-4.1

New Jersey

-1.8

Washington

0.4

Indiana

-1.3

New Mexico

7.3

W. Virginia

-0.1

Iowa

-0.1

New York

0.5

Wisconsin

0.4

Kansas

-3.9

North Carolina

-1.2

Wyoming

no data

10

National

-1.1

Table 3. Percentage point differences (PPD) between the proportion of ELs in the total population and the proportion in the population of SWDs

10

Note that the national calculation excludes Wyoming, which did not provide data for the school year 2010–2011. A negative number is suggestive of possible underrepresentation of ELs in the SWD population, a positive number suggestive of overrepresentation.

CHAPTER TWO ASSESSMENT OF LITERACY DIFFICULTIES IN SECOND LANGUAGE AND BILINGUAL LEARNERS JOHN EVERATT, AMIR SADEGHI, LOUISA GRECH, MOHAMED ELSHIKH, SHAIMAA ABDEL-SABOUR, NASSER AL-MENAYE, BRIGID MCNEILL AND GAD ELBEHERI

This chapter presents a discussion of research investigating cognitivelinguistic predictors of literacy. The primary aim was to inform literacy assessment procedures to identify learning difficulties of children from different language contexts, including those acquiring reading and writing skills in more than one language. Following a discussion of research targeting word-level literacy and phonological processing, additional cross-country studies are examined. Their focus is on predictors of text reading among Arabic, Maltese and Persian speaking school children. Differences in orthography/language features make these cross-language comparisons potentially informative about commonalities and variations in literacy predictors. Furthermore, English is a language of learning for many children in their own context. Children completed measures of word and text reading, and were assessed on language competence, non-verbal skills, phonological ability, orthographic processing and memory. Overall, the findings from this work were consistent with a relatively simple model of reading. Phonological skills predicted variability at the word level and measures of language understanding and word decoding predicted variability in text comprehension. However, in addition to the need for more appropriately normed measures for bilingual/second language contexts, research findings show specific orthographic and cross-language influences that argue for the development of assessments tools to take account of these language/orthography features.

28

Chapter Two

1. Introduction The present chapter focuses on research aimed at informing the development of assessment procedures that can be used to identify individuals with literacy learning difficulties. Appropriate assessment tools are essential to the educational practitioner for both the initial identification of those at risk and the formation of an individual education plan (see Brooks and Everatt 2009; Frederickson and Cline 2002). Additionally, work related to literacy learning difficulties, particularly dyslexia, has argued that early assessment is important for successful support and intervention. Early identification has been found to lead to more effective outcomes, especially when reading and writing are the focus (Torgesen 2005). In contrast, failure to recognize difficulties can lead to emotional/behavioural problems, such as a child becoming anxious or depressed and suffering serious losses in self-esteem, confidence and motivation. These problems form barriers to learning and can lead to a lack of engagement (Everatt and Reid 2010; Miles 2004). Despite the potential importance of such procedures, few have been developed for use within multilingual contexts (Cline and Shamsi 2000), which may mean delays in support and/or mis-identification of problems. The lack of assessment procedures has occurred for many reasons, some of them practical and/or economic. However, the lack of a theoretical basis for the development of tools, and concerns over the effects that inefficient second language learning may have on literacy skills independent of a learning difficulty have both stood in the way of work in this area (see Cline and Reason 1993; Peer and Reid 2000; Smythe and Everatt 2004). The importance of objective assessment procedures can be seen most clearly in cases where an observed behavioural problem may be caused by a number of underlying deficits or experiential factors. Optimal effective intervention may depend on distinguishing between these causes. In the area of literacy assessment, a problem with learning to read English could be due to an underlying English language processing deficit. Such a deficit can be a phonological weakness that leads to poor decoding or a comprehension difficulty that leads to inappropriate meaning interpretation/inference. However, such reading problems could equally be due to poor educational experience (lack of appropriate schooling, for example) or inefficient English-as-a-second language acquisition. Each of these different causes may require specific interventions, and assessment procedures able to distinguish between them should support the identification of the most effective support plans.

Assessment of Literacy Difficulties in L2 and Bilingual Learners

29

2. Language and literacy In most cases, an assessment procedure targeted at identifying literacy learning difficulties would include a determination of the child’s (or adult’s) language skills. Those children (or adults) with weak language skills would be expected to struggle with literacy learning. Efficient phonological skills involved in the processing of basic sounds within the language (e.g., recognising that /dog/ begins with a /d/ sound) should support the individual’s recognition of the association between the written and spoken form. These skills should be useful for the identification or decoding of familiar or unfamiliar words. Evidence of such importance is offered by a large body of research indicating that phonological processing skills are a key component in effective literacy learning. It also indicates that phonological deficits are related to the literacy problems faced by children with dyslexia (see reviews Gillon 2004; Snowling 2000). Poor scores in measures of phonological processing are good predictors of early problems with literacy learning (see, e.g., Puolakanaho et al. 2008). They continue to be associated with literacy weaknesses, and dyslexia, throughout development and into adulthood (Beaton et al. 1997). In comparison with peers, the child with phonological processing deficits would be predicted to show an increased likelihood of accuracy errors and/or poor fluency in production. They would also be expected to show overall slower acquisition of reading and writing. Another expectation relates to the potential for lower levels of independence in literacy learning and an over-reliance on less efficient word identification strategies. This link between phonological skills and literacy has mainly been derived from English-language data. However, similar findings have been found in a number of cross-language comparisons (Smythe et al. 2008; Ziegler et al. 2010) and research on non-Latin-based scripts, such as Arabic (Elbeheri and Everatt 2007) and Chinese (Ho and Bryant 1997). The potential usefulness of such commonalities in skills across languages can be seen in a research by Everatt et al. (2002). In this work, assessments of underlying phonological skills provided a distinction between individuals with dyslexia and those who read in a second language, despite equally poor literacy skills being presented by both groups. However, phonological assessment procedures do need to take into account the context/background of learning (see Everatt et al. 2010). For example, orthographies vary in relative simplicity in which letters and sounds are associated. In English, there is a relatively complex relationship – one letter or grapheme can stand for several sounds or phonemes, and a sound can be represented by several letters or letter

30

Chapter Two

combinations. Other orthographies are more transparent in that they have a much simpler, sometimes one-to-one, correspondence between graphemes and phonemes. This variation in transparency of the script is a factor that affects the appropriateness of particular assessment procedures. Whereas assessments of accuracy in letter-string decoding tasks have been found to be appropriate to assess literacy skills in English, such tools may be less effective in more transparent orthographies (see Smythe and Everatt 2004). Instead, decoding skills may be better determined by the speed at which a child can produce a certain level of decoding accuracy (fluency of performance). When there is a near one-to-one association between letters and sounds, decoding may be highly accurate. However, children with phonological weaknesses may still be slow at performing this decoding process, which will be indicative of finding such processing effortful. Despite good word reading accuracy levels, these children may struggle in literacy tasks, particularly when text understanding is required (see Everatt et al. 2010). Clearly, when text understanding is part of the literacy assessment, more than phonological processing leading to word identification is required. Although the ability to recognise a familiar word, or decode an unfamiliar one, is necessary for good reading levels, excellent word decoding in itself does not produce high reading comprehension levels (Cain and Oakhill 2007). One way of conceptualising these two areas of literacy is via the simple model of reading (Gough and Tunmer 1986; Hoover and Gough 1990). Despite being derived mainly from Englishlanguage data, this framework has the potential to allow comparisons across languages/orthographies, as well as across types of literacy problems. The simple model separates processes related to printed word recognition (decoding) and those where reading is combined with spoken language (linguistic comprehension). Decoding comprises skills needed to connect print representations to appropriate pronunciations and word meanings; and is most clearly associated with the phonological skills discussed above. Word identification can be considered to be initiated by the graphemic encoding of the word, and access to word meaning is then acquired either directly with this graphemic code or via a phonological code. Linguistic comprehension comprises processes that are more often associated with spoken language, such as vocabulary, syntax, semantics, and pragmatics. However, these processes may have some differences between written and spoken forms of language. To achieve full understanding of text, these skills need to function efficiently. Research by a number of authors (see, e.g., Nation and Snowling 1998; Yuill and Oakhill 1991) has investigated cohorts of children who read accurately but

Assessment of Literacy Difficulties in L2 and Bilingual Learners

31

show specific difficulty in understanding what they read. Much of the evidence suggests that these children have more general language deficits than the phonological deficits associated with dyslexia. Based on the above, the model provides a basis on which to investigate similarities and differences in reading difficulties across languages/orthographies. This is achieved through a distinction between comprehension and word-level decoding processes. Nevertheless, more research is required to specify further this simple model framework. Such research can determine differences in comprehension between written text and spoken discourse, and identify interactions between word level decoding and text comprehension processes. For a discussion of the need to include an independent speed component in the simple model, see Joshi and Aaron (2000). Based on the simple model framework outlined above, assessments aimed at identifying reading problems will need to consider word-level, phonological-related decoding problems as separate from text-level comprehension weaknesses. The former problems may be described as consistent with our current understanding of dyslexia. The latter may be better considered a separate type of literacy learning problems requiring different intervention procedures (see examples in Bower-Crane et al. 2008; Clarke et al. 2010). Commonalities across languages/orthographies in terms of phonological-related decoding problems suggest that, with the appropriate development, phonological assessment tools can support the identification of word-level literacy problems in multilingual contexts (Everatt et al. 2010). The identification of specific reading comprehension difficulties/disabilities, though, may present even more problems in multilingual contexts. Poor second language skills will lead to poor understanding of second language text. This will make it difficult for second language literacy assessments to distinguish between reading comprehension problems due to insufficient second language learning and an underlying cognitive-linguistic processing problem more consistent with a specific learning difficulty. So, more research should support the development of assessment tools that can be used in multilingual contexts. Such research could involve determining commonalities across languages in the processes of understanding, needed to support reading comprehension. The comparative research discussed in this chapter aims to inform such investigations.

32

Chapter Two

3. Second language assessments In Malta, both Maltese and English are official languages (Camilleri 1996), and the acquisition of both languages is considered important for social and educational purposes. Students are required to be educated and instructed in both official languages (Sciriha 2001). It is expected that, by the end of compulsory schooling, students are able to converse, write and read in both Maltese and English. However, this general policy conceals quite large variations across groups within the country. Maltese is the more widely spoken language and, particularly for state schools children, it is experienced as the preferred language of communication and the one usually spoken at home. Despite this, the lack of Maltese assessments tools means that English standardized tests are employed for the screening and diagnosis of students who are experiencing literacy difficulties. Whether the content of such English-background assessment tools is suitable for most state children for whom Maltese is the dominant language is questionable (Figueroa 1989). Scores gained by these students on English measures may not reflect their true literacy levels, particularly if compared to standardization norms derived from children whose first language is English. For example, the Neale Analysis of Reading Ability (NARA; Neale 1989) is often used as an assessment tool to determine literacy levels of Maltese children (Firman 1994). In research investigating the performance of a reasonably large cohort (N = 135) of Maltese language-dominant children, Grech (2011) reached an interesting result. If these children’s reading scores were compared to the NARA norms, then they would have reading comprehension scores approximately two years below what would be expected of them based on their chronological age. The English language data from which the test norms were derived were not appropriate for comparison with the data of the Maltese children. Such conclusions concur with those obtained by Hutchinson et al. (2003). These researchers found that bilingual students within the UK were one year behind monolingual English norms on the NARA. Similarly, Frederickson and Frith (1998) compared students whose second language was English with a group of students assessed as having specific learning difficulties. The former group scored lower on comprehension levels, but not on phonological decoding, than the children with learning difficulties. In each of these studies, lower scores on measures of reading comprehension seemed characteristic of second language children. Such poor comprehension levels may be due to weak semantic and syntactic processing and/or delayed vocabulary development. Alternatively, it may be caused by limited background understanding of the information

Assessment of Literacy Difficulties in L2 and Bilingual Learners

33

presented in the passages because of different cultural experiences. Indeed, the differences between the Maltese and UK second language children in their divergence from test norms seems to argue for some level of influence of cultural experience. Practitioners/assessors, therefore, need to be aware of such variations that occur between different population groups. Similar issues, possibly related to experiential differences, have been found in assessments of phonological processing (see discussions in Everatt et al. 2010). However, this does not necessarily mean that assessments of skills in one language cannot be informative of those same skills in another. The determination of the level of performance on a task needs to be sensitive to the background of the individual, particularly when contrasting with expected levels of performance based on different background cohorts. Nevertheless, similar tools may still be appropriate if the same underlying skills form the basis of acquisition. In contrast, if it is found that the processes in one language are different from those involved in a second, or that learning difficulties in one orthography are due to different underlying causes compared to another, then assessment tools across languages may need to be radically altered (see discussions in Smythe and Everatt 2004). In the Maltese context, for example, findings argue for the NARA norms to be treated with caution when determining levels of performance of Maltese-dominant children. However, the development of a Maltese reading comprehension measure consistent with the NARA allowed research to investigate underlying predictors of Maltese reading comprehension. Consistent with the simple model perspective outlined in the previous sub-section, both decoding processes and language understanding skills predicted variability in scores on this Maltese reading comprehension measure. The results indicated that the youngest children tested (40 children, approximately 9 to 10 years old, in their fifth year of school) showed evidence of Maltese non-word decoding fluency being the dominant influence on Maltese reading comprehension. For older children (86 children, 11 to 12 years old, in higher school grades), listening comprehension was the better predictor. A group of children with an assessment of dyslexia (N = 26) was also tested. They were approximately two years older than the children in their fifth year of school, but their reading age was equivalent to these younger typical readers. Their results showed stronger influences of decoding skills on reading comprehension. These Maltese language data suggest that, for younger and less able readers, text comprehension was somewhat restricted by decoding skills. For the older groups, variability in verbal and text understanding co-varied. Taken together, these data were consistent

34

Chapter Two

with a simple model viewpoint within a context of developing literacy skills.

4. Cross-language comparisons The Maltese findings suggest that a similar framework for the identification of different types of literacy learning problems (word-level decoding weaknesses associated with dyslexia versus comprehension-level problems) may be applicable across first and second language contexts; at least for children growing up in a dual-language environment. However, it is possible that it is this dual-language/bilingual environment that leads to commonalities in predictors. This means that the same conclusions may not be derived from other second language contexts. Furthermore, although Maltese comes from a Semitic language background, it has many borrowed words from Italian and English. It also uses the Latin orthography as the basis of its written form, as does English. Hence, commonalities in orthographic form may lead to similarities in literacyrelated processes. In contrast, Arabic and Persian use the same Arabicbased orthography that varies in several ways from the Latin-based script used in English and Maltese. Comparisons between Latin-based and Arabic-based scripts should inform conclusions about commonalities of predictors across different orthographies. In contrast to Latin-based scripts, Arabic and Persian are written from right to left. Arabic-orthography written text is cursive (i.e., a letter connects to other letters around it within a word) and many letters change their shape when written in text compared to when they are presented in isolation. Combinations of dots and marks are also used to distinguish letters and determine pronunciation, as well as to represent syntactic rules and morphological forms. These features have been argued to make Arabic a visually complex orthography (Elbeheri et al. 2011). Further, although long vowel sounds have their own written letter, several diacritic marks are used to represent short vowel sounds. In text used to support early learning, these short vowel markers are included, but they are rarely evident in passages targeted at more experienced readers. This elimination of short vowel markers leads to a reasonably large number of homographs in written Arabic and Persian. The Arab children, included in the research discussed in this chapter, will experience the elimination of short vowel marks, typically, after grade three. For Persian (Farsi-) speaking children following the Iranian education curriculum, short vowel diacritics are rarely used after first grade. Hence, relatively young Arab and Persian children will need to learn to infer short vowel sounds from the context

Assessment of Literacy Difficulties in L2 and Bilingual Learners

35

within which a word is written. The aim is to obtain the correct pronunciation and meaning of a homograph. Therefore, most Persian and Arab children may need to learn skills related to text comprehension even to support word identification at a very early age. These specific features of the Arabic orthography may lead to skills developing differently from those predicted by current models of reading derived from English. The wide geographic distribution of Arabs has led to the necessity for most Arab children to learn two varieties of the Arabic language (a form of diglossia). The vernacular or spoken Arabic is used in everyday speech, and can vary considerably across the Arabic-speaking world. However, the Arabic orthography represents Modern Standard Arabic (MSA), which is spoken only in specific contexts. MSA can vary considerably from the vernacular spoken by a child. This has led some to parallel Arab children’s literacy acquisition to second language acquisition (Ibrahim and AharonPeretz 2005). Potential confusions between sounds used in a local dialect versus those in MSA have led to the argument that phonological processing strategies may not support Arabic literacy acquisition as well as in some other orthographies (see Saiegh-Haddad 2005). This form of diglossia may be evident also among Persian speakers. In Iran, Persian is called Farsi, whereas in Afghanistan it is called Dari, and the accent used in the two versions varies. The written form, however, is based on a standard form of Persian. So, an Afghan child learns to read and write using a different form of their spoken language. This range of differences between Arabic, Persian and English make contrasts potentially informative as to the benefit from a common framework for assessment development. In the work discussed in this chapter, a range of education and language contexts was considered. (Detailed information about the research can be found in: Al-Menaye 2010; Elshikh 2012; Sadeghi 2013). Children were tested in Arabic or Persian first language/learning contexts: 260 Arabic-speaking children in primary schools in Kuwait and 232 Persian-speaking children in primary schools in Tehran, Iran. In addition to these primarily monolingual cohorts, children learning literacy skills in both Persian and English or Arabic and English were also tested. These comprised a cohort of children (N = 146) in English-medium schools in Australia and New Zealand who were from Persian-language backgrounds (about two-thirds from Afghanistan) and learning to be literate in Persian during weekend classes. A second bi-scriptal cohort of Kuwaiti Arabic first language children was tested (N = 138). These children were learning to be literate in English in schools where the teaching of English language skills was an important component of successful educational attainment.

36

Chapter Two

In all samples, children completed measures of word-level reading and text comprehension. Measures of word-level reading focussed on word reading accuracy/fluency in context to allow assessment of non-vowelized items. They also focused on pseudo-word reading to assess processing of novel letter strings. Pseudo-words were vowelized to reduce the number of alternative pronunciations possible with each non-word. Measures of reading comprehension comprised passage reading followed by question answering (similar to NARA). Questions required children to recall details about the text but also to make inferences across the text. In addition, cloze sentence completion procedures were used, which required the child to choose a word to complete a sentence in a meaningful way. Language/verbal skills were assessed by listening comprehension (a simple task of saying a sentence or two, then asking a question about the sentences) and receptive vocabulary (the child points to a picture that represents the meaning of a word or short phrase). Syntactic awareness was also assessed. This typically involved identifying incorrect word order, e.g. ‘He after ran the bus’, or errors in rules, e.g. ‘Two cat were asleep on the rug’. Measures of phonological processing focused on tests of sound deletion (deleting a sound from a spoken word: ‘cat without the /c/ sound’), which seemed most appropriate for the primary grade levels targeted by the research. Simpler measures showed ceiling effects with the more transparent orthographies. Orthographic measures focused on word chains (spaces between words were removed, e.g. ‘thehelptimeafterman’, and the child had to indicate the word boundaries), and orthographic matching (saying whether pairs of letter strings were the same or different, where the different pairs differed by one letter/grapheme: ‘sand - send’). Also, in cases where several letters represented the same sound (i.e., English and Persian), orthographic measures focused on orthographic choice. This required the child to indicate a correct word in pairs of letter strings that produced the same pronunciation if translated using spellingsound conversion rules: ‘monk - munk’. Rapid naming of familiar objects, letters and colours was also used to assess speeded naming. Measures of memory considered simple recall of sequences of digits and repetition of pseudo-words. With older children, working memory skills were also assessed by requiring the child to process information to meaning while retaining other information in short-term memory.

5. Linguistic/orthographic general and specific effects Despite differences between the tested cohorts of children, and consistent with English data (and the Maltese context discussed above),

Assessment of Literacy Difficulties in L2 and Bilingual Learners

37

measures of linguistic comprehension and word recognition/decoding were both found to be predictors of reading comprehension. Language skills of listening comprehension, vocabulary and syntactic awareness showed evidence of influencing reading comprehension levels. Phonological processing skills and orthographic awareness influenced reading levels via word identification/letter string decoding—with speeded naming times also predicting word-level variability. Therefore, overall, the simple model framework discussed in the previous sub-sections allowed for the identification of variability in word reading and reading comprehension levels in all testing contexts (see further discussion in Sadeghi et al. in preparation). Contexts also included those where the Arabic orthography represented the child’s dominant or second language of literacy (i.e., the Kuwait and Australasian contexts). Although a common framework was appropriate in all the contexts, there were specific variations across groups. In the Persian data, word reading and pseudo-word decoding measures achieved high accuracy levels in young readers. This means that fluency (items read correctly per unit of time) was a more appropriate way of assessing word-level variability. This finding was consistent with the view that more transparent orthographies require measures of speeded accuracy to assess variability (see Smythe and Everatt 2004). However, there are irregularities in the relationship between letters and sounds in Persian. A number of letters represents a single sound due to the use of the Arabic orthography to represent Persian. There are differences between the two languages in the number of phonemes. Furthermore, de-vowelization means that young Persian children will encounter many words in written text that can be pronounced in different ways. Moreover, in the Arabic and Maltese contexts studied, there was less evidence of ceiling levels in word and non-word accuracy than in Persian children, despite both Arab and Maltese children possessing higher levels of transparency than Persian. One explanation is that the Persian educational context lays more emphasis on phonological training early in literacy learning than the Arabic or Maltese contexts tested in the present work. This suggests that explicit teaching of the links between phonology and orthography will influence decoding skill levels and, potentially, the measures required for appropriate literacy assessment procedures. However, accuracy and rate of word reading were also influenced by orthographic factors. For example, vowelized text showed higher accuracy levels and faster reading speeds than non-vowelized text in the Arabic cohorts. The effects of vowelization were much clearer in Arabic than Persian data, probably due to the older age at which the Arab children

38

Chapter Two

experienced de-vowelization. However, reading comprehension levels showed evidence of interaction between orthographic factors and experience (Al-Menaye et al. in preparation). For grade 3 Arab children, comprehension levels were higher in vowelized text than non-vowelized text. On the contrary, amongst grade 4 Arab children, who had more experience of de-vowelized text, comprehension levels were higher in non-vowelized than vowelized text. Additionally, phonological awareness measures were found to be predictive of reading levels in grade 3 children. Working memory measures were found to be more predictive of reading levels amongst grade 4 children when processing non-vowelized text. These findings were consistent with more complex skills required to support text processing when short vowel markers are removed, potentially due to comprehension level skills supporting word-level processing. Furthermore, in the Persian data, orthographic knowledge was directly related to reading comprehension from an early grade, independent of word decoding. This effect of orthography could be due to the need to process non-vowelized text from an early age. Hence, children with good reading comprehension skills would be able to use these skills to support orthographic processing. This leads to a reciprocal relationship between orthographic processing and reading comprehension skills. Similar reliable influences of orthographic processing on variability in Arabic reading comprehension have also been identified (Elbeheri et al. 2011). These specific effects of orthography diverge somewhat from the simple model framework in that orthographic processing may not simply influence reading via word level processes. Such influences will also need to be considered in the development of procedures for assessing comprehension skills in children learning the Arabic orthography. A final point to consider in second language assessments is that skills in one language may support relative weaknesses in a second (see discussions in Bialystok et al. 2005). In research investigating writing production of Arab adolescents learning English as a second language (Elshikh et al. in preparation), text coherence was used as a measure of the production of meaningful English text. This measure showed small correlations with English vocabulary levels, but a closer relationship with syntactic awareness. The correlation with syntactic awareness was higher than expected based on data from first language English students. This may be due to such syntactic awareness skills being useful in producing the appropriate written form of Arabic. These skills may then transfer to English writing tasks. Hence, skills learnt in one language/orthography may support development in another, just as a weakness in one area of processing may be supported by skills in another (see also Everatt et al.

Assessment of Literacy Difficulties in L2 and Bilingual Learners

39

2010). Therefore, the features of literacy that we observe in an assessment of a bilingual/second-language learner may be as much a result of the transference of these cross-language skills as a feature of a problem with literacy learning. Assessment procedures that are sensitive to such influences should lead to better conclusions about literacy skills.

6. Conclusions and implications The chapter argues for the need for further research in different languages/orthographies. Such research will inform the development of appropriate literacy and literacy-related measurement tools, and assessment procedures, for use with children from bilingual/second language backgrounds. It is argued that an appreciation of language influences (including cross-language transfer effects) will lead to a better understanding of the child’s performance in both language and literacy tests. In spite of this argument, the majority of the evidence presented supports the view that phonological deficits can be identified as an underlying cause of literacy learning problems across languages and amongst bilingual/second language populations. It also suggests that a simple model framework can support the development of assessment procedures at the word level and also text comprehension level, and that common underlying skills form a core of literacy advancement across languages. However, at both word and comprehension levels there will be specific language/orthography effects that will influence acquisition (a view akin to the conclusions of Geva and Siegel 2000). Also, the specific assessment of these influences will better inform or refine conclusions from assessments. Additionally, a relationship between literacy development and language skills indicates cross-language/orthography transfer, which may lead to positive outcomes when skills learnt in one language support development in a second. Hence, bilingualism, or second language learning, need not be a barrier to literacy acquisition. The features of literacy learning (and language skills) presented may show influences of the languages used by the child. This is not to argue that assessment practices should stop while bilingual procedures are developed. The evidence still argues that assessments in one language can be informative of difficulties in another, which should lead to appropriate support being identified. Rather, the argument is that further research in different languages/orthographies is needed to identify the main features of a language/orthography that influence performance in language/literacy tasks. It should also indicate how these features might interact in

40

Chapter Two

bilingual/second-language contexts. Through such practitioner-focused research, better assessment measures should be developed. Assessment procedures should be more likely to determine the reason for the level/type of performance shown by a child/adult.

Works cited Al-Menaye, Nasser. 2010. “Memory and Executive Functioning in Arabic Literacy Skills.” PhD diss., University of Surrey, UK. Al-Menaye, Nasser, Gad Elbeheri, and John Everatt. (In preparation). Reading Comprehension in Vowelized and Non-Vowelized Arabic Text. Beaton, Alan, Siné McDougall, and Chris Singleton, eds. 1997. “Special Issue: Dyslexia in Literate Adults.” Journal of Research in Reading 20 (1). Bialystok, Ellen, Catherine McBride-Chang, and Gigi Luk. 2005. “Bilingualism, Language Proficiency, and Learning to Read in Two Writing Systems.” Journal of Educational Psychology 97 (4):580–590. Bowyer-Crane, Claudine, Margaret J. Snowling, Fiona J. Duff, Elizabeth Fieldsend, Julia M. Carroll, Jeremy Miles, Kristina Götz, and Charles Hulme. 2008. “Improving Early Language and Literacy Skills: Differential Effects of an Oral Language Versus a Phonology with Reading Intervention.” Journal of Child Psychology and Psychiatry 49 (4):422–432. Brooks, Peter, and John Everatt. 2009. “Phonology, Discrepancy, Instruction and Dyslexia: Adversaries or Allies?” In Educational Psychology: Cognition and Learning, Individual Differences and Motivation, edited by Jonathon E. Larson. New York: Nova Science Publishers. Cain, Kate, and Jane Oakhill. 2007. Children's Comprehension Problems in Oral and Written Language: A Cognitive Perspective. New York: Guilford Press. Camilleri, Antoinette. 1996. “Language Values and Identities: Code Switching in Secondary Classrooms in Malta.” Linguistics and Education 8 (1):85–103. Clarke, Paula J., Margaret J. Snowling, Emma Truelove, and Charles Hulme. 2010. “Ameliorating Children’s Reading-Comprehension Difficulties: A Randomized Controlled Trial.” Psychological Science 21 (8):1106–1116. Cline, Tony, and Rea Reason. 1993. “Specific Learning Difficulties (Dyslexia): Equal Opportunities Issues.” British Journal of Special Education 20 (1):30–34.

Assessment of Literacy Difficulties in L2 and Bilingual Learners

41

Cline, Tony, and Tatheer Shamsi. 2000. Language Needs or Special Needs? The Assessment of Learning Difficulties in Literacy among Learning English as an Additional Language: A Literature Review. London: DfEE Publications. Elbeheri, Gad, and John Everatt. 2007. “Literacy Ability and Phonological Processing Skills amongst Dyslexic and Non-dyslexic Speakers of Arabic.” Reading and Writing 20 (3):273–294. Elbeheri, Gad, John Everatt, Abdessatar Mahfoudhi, Mosaad Abu AlDiyar, and Nadia Taibah. 2011. “Orthographic Processing and Reading Comprehension among Arabic Speaking Mainstream and LD Children.” Dyslexia 17 (2):123–142. Elshikh, Mohamed. 2012. “Psycho-linguistic Predictors of L1-Arabic and L2-English Reading and Writing Skills for Arabic Speaking Children”. PhD diss., University of Surrey, UK. Elshikh, Mohamed, John Everatt, and Gad Elbeheri. (In preparation). Predictors of Arabic and English Writing Skills. Everatt, John, Dina Ocampo, Kazuvire Veii, Styliani Nenopoulou, Ian Smythe, Haya Al-Mannai, and Gad Elbeheri. 2010. “Dyslexia in Biscriptal Readers.” In Reading and Dyslexia in Different Orthographies, edited by Nicola Brunswick, Sine McDougall and Paul de Mornay Davies. Hoboken: Taylor and Francis. Everatt, John, and Gavin Reid. 2010. “Motivating Children with Dyslexia.” In Motivating Literacy Learners in Today's World, edited by Jo Jo Fletcher, Faye Parkhill and Gail T. Gillon. Wellington: NZCER Press. Everatt, John, Ian Smythe, Dina Ocampo, and Kazuvire Veii. 2002. “Dyslexia Assessment of the Bi-scriptal Reader.” Topics in Language Disorders 22 (5):32–45. Figueroa, Richard A. 1989. “Psychological Testing of Linguistic-Minority Students: Knowledge Gaps and Regulations.” Exceptional Children 56 (2):145–152. Firman, Christine. 1994. “Dyslexia in Malta.” In The International Book of Dyslexia, edited by Robin Salter and Ian Smythe. London: World Dyslexia Network Foundation. Frederickson, Norah, and Tony Cline. 2002. Special Educational Needs, Inclusion and Diversity, a Textbook. Buckingham: Open University Press. Frederickson, Norah, and Uta Frith. 1998. “Identifying Dyslexia in Bilingual Children: A Phonological Approach with Inner London Sylheti Speakers.” Dyslexia 4 (3):119–131.

42

Chapter Two

Geva, Esther, and Siegel, Linda, S. (2000). “Orthographic and Cognitive Factors in the Concurrent Development of Basic Reading Skills in Two Languages.” Reading and Writing: An Interdisciplinary Journal 12:1– 30. Gillon, Gail. 2004. Phonological Awareness: From Research to Practice, Challenges in Language and Literacy. New York: Guilford Press. Gough, Philip B., and William E. Tunmer. 1986. “Decoding, Reading, and Reading Disability.” Remedial and Special Education 7 (1):6–10. Grech, Louisa. 2011. “Reading Comprehension in Maltese-English Bilinguals”. PhD diss., University of Surrey, UK. Ho, Connie Suk-Han, and Peter Bryant. 1997. “Phonological Skills Are Important in Learning to Read Chinese.” Developmental Psychology 33 (6):946–951. Hoover, Wesley A., and Philip B. Gough. 1990. “The Simple View of Reading.” Reading and Writing 2 (2):127–160. Hutchinson, Jane M., Helen E. Whiteley, Chris D. Smith, and Liz Connors. 2003. “The Developmental Progression of ComprehensionRelated Skills in Children Learning EAL.” Journal of Research in Reading 26 (1):19–32. Ibrahim, Raphiq, and Judith Aharon-Peretz. 2005. “Is Literary Arabic a Second Language for Native Arab Speakers?: Evidence from Semantic Priming Study.” Journal of Psycholinguistic Research 34 (1):51–70. Joshi, R. Malatesha, and P. G. Aaron. 2000. “The Component Model of Reading: Simple View of Reading Made a Little More Complex.” Reading Psychology 21 (2):85–97 Miles, Tim R., ed. 2004. Dyslexia and Stress. 2nd ed. London: Whurr. Nation, Kate, and Margaret J. Snowling. 1998. “Semantic Processing and the Development of Word-Recognition Skills: Evidence from Children with Reading Comprehension Difficulties.” Journal of Memory and Language 39 (1):85–101. Neale, Marie D. 1989. Neale Analysis of Reading Ability. UK: NFERNELSON. Peer, Lindsay, and Gavin Reid, eds. 2000. Multilingualism, Literacy and Dyslexia. London: David Fulton Publishers. Puolakanaho, Anne, Timo Ahonen, Mikko Aro, Kenneth Eklund, Paavo H. T. Leppänen, Anna-Maija Poikkeus, Asko Tolvanen, Minna Torppa, and Heikki Lyytinen. 2008. “Developmental Links of Very Early Phonological and Language Skills to Second Grade Reading Outcomes: Strong to Accuracy but Only Minor to Fluency.” Journal of Learning Disabilities 41 (4):353–370.

Assessment of Literacy Difficulties in L2 and Bilingual Learners

43

Sadeghi, Amir. (2013). “Towards a Universal Model of Reading: Investigations into Persian Monolinguals and Persian-English Bilinguals”. PhD diss., University of Canterbury, NZ. Sadeghi, Amir, John Everatt, and Brigid McNeill. (In preparation). Predictors of Persian Reading Comprehension. Saiegh-Haddad, Elinor. 2005. “Correlates of Reading Fluency in Arabic: Diglossic and Orthographic Factors.” Reading and Writing 18 (6):559– 82. Sciriha, Lydia. 2001. “Trilingualism in Malta: Social and Educational Perspectives.” International Journal of Bilingual Education and Bilingualism 4 (1):23–37. Smythe, Ian, and John Everatt. 2004. “Dyslexia: A Cross Linguistic Framework.” In The International Book of Dyslexia: A Guide to Practice and Resources, edited by Ian Smythe, John Everatt and Robin Salter. London: Wiley. Smythe, Ian, John Everatt, Nasser Al-Menaye, He Xianyou, Simone Capellini, Eva Gyarmathy, and Linda S. Siegel. 2008. “Predictors of Word-Level Literacy amongst Grade 3 Children in Five Diverse Languages.” Dyslexia 14 (3):170–187. Snowling, Margaret J. 2000. Dyslexia. 2nd ed. Oxford: Blackwell. Torgesen, Joseph K. 2005. “Recent Discoveries on Remedial Interventions for Children with Dyslexia.” In The Science of Reading: A Handbook, edited by Margaret J. Snowling and Charles Hulme. Malden, MA: Blackwell Publishing. Yuill, Nicola, and Jane Oakhill. 1991. Children's Problems in Text Comprehension: An Experimental Investigation, Cambridge Monographs and Texts in Applied Psycholinguistics. Cambridge: Cambridge University Press. Ziegler, Johannes C., Daisy Bertrand, Dénes Tóth, Valéria Csépe, Alexandra Reis, Luís Faísca, Nina Saine, Heikki Lyytinen, Anniek Vaessen, and Leo Blomert. 2010. “Orthographic Depth and Its Impact on Universal Predictors of Reading: A Cross-Language Investigation.” Psychological Science 21 (4):551–559.

CHAPTER THREE STUDENTS CLASSIFIED AS LEARNING DISABLED IN L2 COURSES: THE ‘SPECIAL’ CASE OF THE UNITED STATES RICHARD L. SPARKS

In the United States (U.S.), students generally do not begin second language (L2) study until high school. In fact, they study language as a subject, rather than to become fluent or literate. As a result, there are no standardized testing measures or assessment procedures that can determine how well U.S. students learn L2s. Students with disabilities, including learning disabled (LD) students, have access to “reasonable accommodations” in their schooling. However, the legislative mandate for accommodations has highlighted longstanding problems in the U.S. with the definition and diagnostic criteria for LD. These problems have resulted in misdiagnoses of LD and confusion for L2 educators about which students require mandated accommodations. This chapter reports on the status of students classified as LD in L2 courses in a U.S. context by reviewing problems with definition and diagnostic criteria for LD, studies that reveal long-term relationships between students’ L1 skills and their L2 proficiency, and studies conducted with LD and non-LD students in L2 courses. Unique assessment procedures developed to measure U.S. students’ L2 proficiency and accommodation practices for students classified as LD are described. The chapter concludes with a discussion of the practical problems for L2 assessment in the U.S.

1. Introduction When compared to many other countries, the United States (U.S.) is a ‘special’ case for second language (L2) learning and L2 learning problems. Unlike other countries, U.S. students do not begin the formal study of a L2 until high school (9th grade), and generally study the L2 as a subject, e.g., math, science, but not to become fluent or literate in the language. There are few programs in U.S. elementary or middle schools in

46

Chapter Three

which students engage in L2 study. A major problem for L2 assessment, resulting from this practice, is that there are no standardized measures or norms by which U.S. students can be compared to native speakers of the target language or to other U.S. students learning the language. Also, not all secondary or postsecondary students are enrolled in L2 courses because passing L2 courses or achieving a certain level of proficiency in the L2 are not graduation requirements in most states (U.S. Department of Education 2010). However, some secondary and postsecondary institutions have their own L2 requirements. Moreover, the large majority of U.S. students have little or no exposure to a L2, and only 18% of Americans report speaking a language other than English (U.S. Department of Education 2010). On the one hand, this situation is problematic because U.S. students have very limited opportunity to become proficient in a L2. On the other hand, the lack of engagement in L2 study until many years after students have mastered their oral and written L1 provides unique opportunities to investigate long-term relationships between L1 skills and L2 learning. It also allows for the development of unique assessment tools for measuring L2 proficiency. Students classified as learning disabled (LD) have always been enrolled in L2 courses (see, e.g., Dinklage 1971). With the passing of disability legislation for elementary and secondary students in 1975, i.e. Individuals with Disabilities Education Act (IDEA) (1997), and for adults in 1990, i.e. Americans with Disabilities Act (ADA) (1990), there has been a notable increase in the number of secondary and postsecondary students classified as LD enrolled in L2 courses. If students have a valid diagnosis of LD, the aforementioned legislation requires that they be provided with “reasonable accommodations” in their schooling, including L2 courses. Unfortunately, the legislative mandate for accommodations has highlighted longstanding problems in the U.S. with the definition and diagnostic criteria for LD students. These problems have resulted not only in misdiagnoses of LD students but also failure to diagnose students with legitimate learning problems. The problems with the definition and diagnostic criteria for LD students mask larger problems for L2 educators. These problems include confusion over which students with L2 learning problems in their classes have a learning disability in their native language (L1). They also involve failure to develop effective teaching methodologies and accommodations practices for students with L2 learning problems, learning disabilities or otherwise. This chapter reports on the status and outcomes of students classified as LD in L2 courses in a U.S. context. First, ongoing problems related to the definition and diagnostic criteria for LD students are reviewed.

Students Classified as Learning Disabled in L2 Courses

47

Second, empirical studies demonstrating strong, long-term relationships between students’ LI skills and their L2 proficiency and achievement are described. Third, findings from studies conducted with secondary and postsecondary LD students enrolled in L2 courses are reviewed. Fourth, assessment procedures developed to measure U.S. students’ oral and written proficiency, used in the aforementioned studies are described. Fifth assessment and accommodations practices used with students classified as LD in the U.S. are reviewed. Lastly, practical problems for L2 instruction and assessment for all learners, including LD students, are summarized.

2. Learning disabilities in the United States Since first designated as a disability condition in 1968, the learning disability concept has been contentious. In the U.S., the learning disability concept evolved as a way to understand individual differences and as a result of advocacy (Moats and Lyon 1993). Although LD students now account for over half of all students receiving special education services in public schools, the concept is both confusing and misunderstood. The longstanding failure of the learning disability field to develop an empirically-based definition and valid diagnostic criteria for LD students has caused this situation (Fletcher et al. 2007). In the U.S., research in learning disability has been ongoing for over 50 years. Yet, the conceptual definition of learning disability used today is the same definition introduced into federal law in 1968: Specific learning disability means a disorder in one or more of the basic psychological processes involved in understanding or using language, spoken or written, which may manifest itself in an imperfect ability to listen, think, speak, read, write, spell, or to do mathematical calculations. The term includes such conditions as perceptual handicaps, brain injury, minimal brain dysfunction, dyslexia, and developmental aphasia. The term does not include children who have learning problems which are primarily the result of visual, hearing, or motor handicaps, of mental retardation, of emotional disturbance, or of environmental, cultural, or economic disadvantage.1

This broad definition was a compromise between those who viewed learning disability as uneven growth between academic skills, a neurological impairment, difficulty mastering academic skills, discrepancy between “potential” for learning and achievement, and as excluding other 1

U.S. Office of Education 1968, 34.

48

Chapter Three

disabilities, e.g., intellectual and sensory deficits. Because the definition was so ambiguous, there was a pressing need to operationalize LD so that students could be identified and provided with services in educational settings. By 1978, when IDEA was first implemented, the U.S. Office of Education proposed the use of a “severe” discrepancy between a student’s “potential” to learn—as measured by performance on a standardized measure of intelligence (IQ)—and his/her scores on standardized measures of academic achievement, i.e., reading, math, written language. However, the size of the aptitude-achievement discrepancy was left to each of the 50 states. Unfortunately, states adopted different levels of discrepancy to identify LD students, e.g., 1.0 standard deviation (SD), 1.50 SD, 2.0 SD. The diagnosis of learning disability became more confusing because each state was required to adopt an “override procedure” by which one could qualify for learning disability services if the required discrepancy was not met. As a result, researchers found that over 50 percent of students receiving LD services in the states did not meet a state’s criteria for LD students. Also, they were often indistinguishable from low-achieving students (see, e.g., Ysseldyke 1983). Recent research with postsecondary students has found that classification as LD is dependent on the diagnostic criteria used by a diagnostician, and those criteria may differ from clinician to clinician (see, e.g., Sparks and Lovett 2009). By 1991, researchers had discredited the aptitude-achievement (and achievement-achievement) discrepancy model for diagnosing students as LD. In numerous studies, researchers showed that using IQ as a benchmark for learning “potential” was mistaken (Stanovich 1991). For example, Siegel (1989) found that IQ scores measured learned knowledge, short-term memory, and language ability, i.e., tasks on which children with learning problems exhibited deficits, and that some children with low IQ scores were good readers. Other researchers have found that IQ is not a good predictor of early reading achievement and that there are no differences in the pattern of reading-related cognitive deficits between low and high IQ poor readers (see, e.g., Stuebing et al. 2002). Despite the robust evidence that has discredited the discrepancy concept, and criticism of the LD field for its use (see, e.g., Stanovich 2005), many diagnosticians continue to use aptitude-achievement discrepancy to classify students as LD in educational settings. This practice creates problems for the identification, assessment, and treatment of students with L2 learning problems.

Students Classified as Learning Disabled in L2 Courses

49

3. Relationships between L1 skills and L2 learning Prior to 1990, little was known about why students, LD or otherwise, exhibited L2 learning problems. In the U.S., the most productive line of inquiry into L2 difficulties focused on cognitive variables such as intelligence and language aptitude. By the early 1970s, researchers reported that intelligence (IQ) was not a good predictor of L2 achievement (Wallace and Lambert 1972), and found L2 aptitude was a more valid predictor. Both Paul Pimsleur, who authored the Language Aptitude Battery (Pimsleur 1966), and John Carroll, who authored the Modern Language Aptitude Test (MLAT; Carroll and Sapon 1959), developed L2 aptitude batteries for secondary and postsecondary level students. The batteries, which consisted of language-related variables, i.e., phonetic coding, grammatical sensitivity, inductive language learning ability, were found to be predictive of L2 achievement (see Carroll 1962, 1990). Pimsleur’s and Carroll’s research also revealed links between students’ L1 skills and their language aptitude for L2 learning. For example, Carroll (1973) hypothesized that L2 aptitude may be a “residue” of L1 skills. Pimsleur (1968) found that “auditory ability”, measured with speechsound discrimination and sound-symbol association tasks, best discriminated good and poor L2 learners. In 1991, Richard Sparks and Leonore Ganschow hypothesized links between L1 and L2 learning, and that students’ L2 learning problems were related to problems in their L1 (Sparks and Ganschow 1991). In 1989, they completed a pilot study showing that LD students exhibited deficits in their L1 skills and L2 aptitude on the MLAT. These findings led them to propose the Linguistic Coding Differences Hypothesis (LCDH), in which they hypothesized that: a) Students’ L1 skills serve as the foundation for their L2 aptitude and achievement; b) Both L1 and L2 learning depend on basic language learning mechanisms common to both languages; c) L2 aptitude is componential, and problems with one language component, e.g., phonological processing, have a negative effect on both L1 and L2; and d) L2 learning runs along a continuum of learners with strong to weak language skills (Sparks 1995; Sparks et al. 1989). The LCDH led to a series of studies over 20 years that investigated the following questions: 1) Are there differences in L1 skills, L2 aptitude, and L2 outcomes between high-and low-achieving L2 learners? 2) Are there long-term relationships between students’ early L1 skills and later L2 aptitude and L2 proficiency? When investigating the first question, Sparks and Ganschow found significant differences in L1 reading, spelling, writing, and vocabulary between secondary and postsecondary high-achieving and low-achieving

50

Chapter Three

L2 learners, even after controlling for IQ (Ganschow and Sparks 2001). These findings led them to infer that students’ levels of L1 skill are related to their levels of L2 aptitude and L2 proficiency. The studies also found that the L1 differences between high- and low-achieving L2 learners appeared primarily in the phonological (sound and sound-symbol) and syntactic (grammar) components of language (see Sparks 1995). They measured students’ L1 skills at the time they began L2 study in high school. It is an approach with limitations, though. They could not determine whether L2 learners who exhibited L1 difficulties in 9th grade exhibited these same L1 differences in elementary school or whether early differences in L1 skills may be related to L2 aptitude and L2 proficiency in secondary school. To answer the second question, they conducted retrospective and prospective longitudinal studies that examined individual differences in early L1 skills and investigated whether those differences were related to L2 aptitude and L2 proficiency in high school. In a retrospective study, high-and low-achieving high school L2 learners were followed over two years of L2 study and compared on 4th and 6th grade L1 literacy and language tests, the MLAT, and measures of L2 literacy (Sparks et al. 2008). After controlling for IQ differences, results revealed significant differences between the high- and low-achieving groups on all early L1 literacy and language measures favoring the high-achieving group as early as fourth grade. The high-achieving group outscored the low-achieving group by over one standard deviation in L1 literacy, L2 aptitude, and L2 proficiency. In a prospective study, they followed 54 students over ten years from first grade through two years of L2 courses in high school. In 1st–5th grades, students were administered measures of L1 literacy and language and an IQ test. In 9th grade, the MLAT was administered, and, at the end of 10th grade, L2 tests assessed students’ L2 proficiency. Several investigations were conducted using this dataset including comparison, prediction, and factor analysis studies (see review by Sparks 2012). All studies revealed significant, long-term relationships among L1 skills in early elementary school and L2 aptitude and L2 proficiency in high school. In one study, the students were divided into high-, average-, and low-proficiency groups and compared on the L1 measures administered in elementary school, the MLAT, and L2 grades (Sparks, Patton et al. 2009). Findings showed significant overall differences among the three proficiency groups on all L1 skill measures as early as 2nd grade and in L2 aptitude favoring the high-proficiency L2 learners. The results of these studies revealed that L1 skill differences emerged early in elementary

Students Classified as Learning Disabled in L2 Courses

51

school and were strongly related to L2 aptitude and L2 proficiency several years later in high school.

4. Students classified as LD and low-achieving L2 learners In the 1960s, Pimsleur and his colleagues studied “underachievers” who exhibited problems with L2 learning (Pimsleur et al. 1964). They found that these students exhibited deficits in “auditory ability” on speech sound discrimination and sound-symbol association tasks. Dinklage (1971) observed that some college students with L2 learning problems exhibited L2 learning difficulties similar to dyslexic students. In the 1980s, anecdotal reports about students with LD who exhibited L2 learning problems appeared in the literature (Pompian and Thum 1988). In an empirical study, Gajar (1987) found that LD students in L2 classes performed significantly more poorly on all five MLAT subtests than nonLD students. In these studies, the authors described links between students’ L1 difficulties and their L2 learning. In a series of studies, Sparks et al. found that LD students exhibited significantly lower L1 skills, especially in phonological processing, and weaker L2 aptitude than students without LD who reported similar findings (see, e.g., Downey and Snyder 2001). In their studies, Sparks et al. found that many low-achieving L2 learners did not meet diagnostic criteria for LD; instead, they were low-achieving learners in both L1 and L2. These findings led to a series of studies with LD students in L2 courses in which they examined three questions: 1) Are there differences in L1 skills, cognitive ability, and L2 aptitude between students classified and not classified as LD in L2 courses? 2) Are there differences between students classified as LD with and without IQachievement discrepancies? 3) Are there differences between students classified as LD who pass L2 courses or receive waivers/course substitutions for the L2 requirement? In studies involving the first question, findings showed that LD students did not exhibit IQ and L1 differences in reading, spelling, writing, and vocabulary when compared to low-achieving L2 learners (see reviews by Sparks 2001, 2006). In one study, results showed that high school L2 learners with LD and low-achieving L2 learners did not exhibit significant differences on measures of L1 achievement (reading, vocabulary, spelling) and L2 aptitude (Sparks, Artzer, Javorsky et al. 1998). In a recent study, results showed that high school L2 learners with LD and low-achieving L2 students exhibited no differences on measures of L1 literacy and language

52

Chapter Three

in 4th grade, L2 aptitude, L2 proficiency, and L2 grades (Sparks et al. 2008). In studies investigating the second question, findings revealed no differences on cognitive and L1 achievement measures between L2 learners with LD and varying degrees of IQ-achievement discrepancies (i.e., 1.0–1.49 SD, 1.5–1.99 SD, > 2.0 SD) and LD students with and without IQ-achievement discrepancies. In one study, LD college students with different levels of IQ-achievement discrepancies did not exhibit significant differences in college entrance scores, L2 aptitude, college L2 GPA, and graduating college GPA (Sparks et al. 1999a). These findings were replicated with high school students and different populations of college students with LD in L2 courses (Sparks, Artzer, Javorsky et al. 1998; Sparks, Philips and Javorsky 2002). The results of studies examining the third question found no cognitive, L1 skill, or demographic differences between LD students who received course waivers/substitutions and students with LD who fulfilled the L2 requirement by passing L2 courses. In one study, LD students in these two groups did not exhibit significant differences in IQ, L1 skills, graduating college GPA, and college entrance scores (Sparks et al. 1999b). In a replication study with a different population of LD students, results revealed similar findings (Sparks, Philips and Javorsky 2003). In sum, empirical studies with LD students in L2 courses have shown that there are no L1 skill, L2 aptitude, and L2 achievement differences between: a) low-achieving and LD students in L2 courses; b) LD students with and without IQ-achievement discrepancies; and c) LD students granted waivers/substitutions and LD students who fulfilled the L2 requirement. In part, these findings may reflect problems in the U.S. with the definition and diagnostic criteria for LD described earlier. Findings aside, the L2 achievement and proficiency of all students, including lowachieving and LD students, must be assessed in a thorough and fair manner, a topic that is addressed in the next section of the chapter.

5. Assessment procedures for L2 learners In the U.S., there are no standardized L2 measures for U.S. students engaged in L2 learning and no norms by which U.S. students can be compared to native speakers of another language or to other U.S. students studying a L2. Most U.S. researchers who study L2 learning develop their own assessments. In several studies, Sparks and Ganschow used authordeveloped measures based on the American Council on the Teaching

Students Classified as Learning Disabled in L2 Courses

53

Foreign Languages (ACTFL) Proficiency Guidelines to assess the proficiency of L2 learners. The ACTFL Proficiency Guidelines describe what individuals can do in reading, writing, listening, and speaking L2s in a spontaneous, i.e., nonrehearsed, context (ACTFL 2012). The Guidelines were published in 1986 for the U.S. government’s Interagency Language Roundtable Skill Level descriptions. They were revised in 1999, 2001, and 2012. For each oral and written language skill, the Guidelines identify five levels of proficiency: Distinguished, Superior, Advanced, Intermediate, and Novice. The levels from Advanced to Novice are further subdivided into High, Mid, and Low sublevels. The Guidelines describe what an individual can/cannot do at each level and sublevel regardless of when, how, or where the L2 was acquired. The Guidelines do not adhere to a specific curriculum or to any particular teaching method or learning theory. ACTFL provides training in several L2s for educators and other professionals in developing and administering proficiency tests and applying the Guidelines. The ACTFL Guidelines are not standardized measures of L2 proficiency; rather, an individual, school, or researcher develops oral and written L2 measures that are situation- or contextspecific, and the Guidelines are used to assess the learner’s proficiency in the L2. For their studies, Sparks and Ganschow teamed with L2 educators in Spanish, French, and German who were formally trained by ACTFL to develop and administer measures of oral and written proficiency. A brief description of the measures is included here.

6. L2 word decoding The L2 word decoding task consists of a word list of 20 real words and 20 pseudowords. The real words and pseudowords in the three L2s measure a specific decoding skill that involves use of vowel sounds in each L2 that are not consistent with the vowel sounds in English. The vowel sounds in the L2s use different letter combinations, and the list contains multisyllabic words and words with diacritical markings. For a response to be correct, the student has to produce a natural reading (pronunciation) of the word or pseudoword. The word lists in the three L2s are presented in Appendix A.

7. L2 reading comprehension On this test, students read a one-page letter written in the L2 and answer ten multiple-choice questions in English (conforming to ACTFL

54

Chapter Three

guidelines) about the letter. The letter is designed using criteria descriptive of the intermediate-high level of the ACTFL Guidelines, which are considered appropriate for high school L2 learners completing a second year in the language. Next, the students read a passage from Reader’s Digest in Spanish, French, or German and answer ten multiple-choice questions about the passage. Students are given 15 minutes to read the letter and answer the questions, and 15 minutes to read the passage and answer the questions. The letter written for the Spanish students to read and comprehend, and the Reader’s Digest in Spanish are included in Appendices B and C.

8. L2 writing On this test, students write a response in the L2 to the letter for the reading comprehension task. The letter contains five questions to which the students are asked to respond. The students are given 15 minutes to write the letter. Each student’s writing sample is scored for five criteria: vocabulary, cultural appropriateness, structure, comprehensibility, and spelling. The ACTFL Guidelines for determining proficiency levels are used in assigning the scores (0–5) on each of the five criteria. Each student is evaluated in the five areas using the five-point scale: 0 = no production, 1 = novice-low, 2 = novice-mid, 3 = novice-high, 4 = intermediate-low, and 5 = intermediate-high and above (maximum score = 25). The scoring criteria are presented in Appendix A.

9. L2 spelling The spelling task for each of the three languages consists of 20 words designed to measure specific decoding skills. This measure includes vowel sounds in the L2 that are not consistent with the vowel sounds in English, and contains multisyllabic words and words with diacritical markings. The student writes the words spoken by the examiner. A word can be repeated or used in a sentence for the student. A response is marked as either right or wrong. The spelling lists for the three L2s are presented in Appendix D.

10. L2 listening/speaking Students’ oral proficiency is assessed through a 10–15-minute individual interview. Interviewers have no previous information about the participants, including their L1 skills and L2 aptitude scores or their L2 classroom achievement. Interviewers use randomly selected topics about

Students Classified as Learning Disabled in L2 Courses

55

which the students converse. Each student’s interview is taped for later scoring. The oral interview is scored for five criteria: pronunciation, vocabulary, grammar, comprehensibility, and listening comprehension. Each student is evaluated in the five areas using the five-point scale: 0 = no production, 1 = novice-low, 2 = novice-mid, 3 = novice-high, 4 = intermediate-low, and 5 = intermediate-mid and above. The scoring criteria are presented in Appendix E. A total L2 proficiency score was obtained by combining the number of points on the L2 reading comprehension, writing, listening/speaking, word decoding, and spelling tests. Local norms were then developed for comparison purposes.

11. Assessment and accommodation of LD students In the U.S., there are dozens of standardized tests to determine how well English language learners (ELL) are mastering oral and written English. However, there is not an established method, standardized test(s), or assessment protocol to determine how well students whose L1 is English are mastering a L2. In most cases, each secondary school and university, or individual L2 teachers, develop their own measures of L2 proficiency. Thus, it is unsurprising that LD students are limited primarily to instructional accommodations for learning a L2, and testing accommodations for assessing their L2 proficiency. As noted earlier, LD students are permitted by IDEA or ADA to have access to “reasonable accommodations” in their schooling, including L2 courses. Reasonable accommodations generally take two forms: instructional/testing accommodations and course waivers/substitutions. In a small number of cases, LD students and other low-achieving students may receive L2 instruction in special, self-contained classes. Instructional accommodations may include but are not limited to differentiated assignments, reduced homework, alternative modes of instruction (e.g. computers), written examples of oral instruction and content, peer tutoring, preferential seating in the classroom, and/or outside tutorial support. Testing accommodations may include but are not limited to extended time on tests and exams, alternative testing formats (oral instead of written tests), readers and/or scribes for quizzes and exams, word banks of potential correct responses, vocabulary lists for a specific test, ignoring spelling and grammar errors, and testing in an alternative environment (private room). As yet, there are no empirical studies that indicate which testing (or instructional) accommodations are most beneficial to LD students.

56

Chapter Three

The second form of accommodations for LD students is waiver of the L2 requirement or course substitutions, e.g., a French history course instead of French. Each high school or college has its own process for determining whether a student meets its requirements for a waiver/substitution. Waivers and substitutions are generally granted by assuming that students will not pass the L2 course(s) and/or master any L2 proficiency requirements. Colleges and universities are not required by law to allow waivers and substitutions (see Sparks and Javorsky 1999), but their practice has increased. As a result, many students without a history of L1 learning problems or a previous diagnosis of LD have sought this diagnosis when they encounter the L2 requirement. In their studies with postsecondary LD students, Sparks et al. found that it is relatively easy to obtain a diagnosis of LD for this purpose (see Sparks 2006). In a small number of secondary and postsecondary schools, a third accommodation involves small, self-contained L2 classes for LD students and other low-achieving L2 learners. In one college program, Amend et al. (2009) described a two-semester modified Spanish sequence for students with language-based LDs. Downey, Snyder and Hill (2000) described a modified, two-semester sequence of Latin for LD students. In these Latin classes, they used a teaching method, the Orton-Gillingham (O-G) approach (Gillingham and Stillman 1960), traditionally used to teach students with dyslexia and language-based LDs to read, spell, and write their L1. The O-G approach directly and explicitly teaches the phonology (sound-symbol relationships), grammar, and vocabulary of the L2 by introducing a small amount of information at one time and teaching that material until it is mastered through practice and review. Sparks et al. (1991) have described the O-G methodology for teaching Spanish, and Schneider and Evers (2009) reviewed the methodology for teaching German and Hebrew. In the 1990s, Sparks et al. conducted studies in which the O-G methodology was used to teach Spanish to high school students classified as LD. In their initial studies, they found after one year that students made significant gains in L1 word decoding, L1 vocabulary, and L2 aptitude (MLAT) when compared to LD students enrolled in traditional L2 classes. They also found that LD students taught Spanish by this method made significantly greater gains on L1 word decoding measures than non-LD students in traditional Spanish classes (Ganschow and Sparks 1995). In a larger comparison study, Sparks, Artzer, Patton et al. (1998) followed four groups of high school Spanish students over two years of L2 instruction, including: a) LD students in a self-contained class receiving O-G instruction, b) LD students in a self-contained class with regular Spanish

Students Classified as Learning Disabled in L2 Courses

57

instruction and instructional accommodations, c) LD students in regular Spanish classes with supportive tutoring, and d) non-LD students in regular Spanish classes. After two years, findings showed that the O-G group and the non-LD students made significantly greater gains in L1 word decoding, L1 reading comprehension, and L2 aptitude (MLAT) than the other LD groups. When the four groups’ L2 proficiency was assessed after two years of Spanish, results showed that the O-G and non-LD groups achieved significantly higher L2 proficiency than the other LD groups. Results also showed that there were no differences in L2 proficiency between the O-G and non-LD groups. These findings suggest that addressing LD students’ weaknesses in their L1 skills by using direct and explicit instruction in the L2 may be beneficial in helping them become more successful in a L2.

12. Summary and conclusions The state of affairs concerning L2 study and assessment for all U.S. students may pertain specifically to this country. Despite evidence that learning another language is beneficial, the U.S. does not promote L2 learning in its schools. Generally, students enroll in L2 courses in high school long after they have mastered their oral and written L1, and study the target language as a subject, not to become fluent or literate. In many countries, there is strong motivation to learn another language because of its economic or social importance to the individual and society. In these countries, students receive strong messages that learning a L2 is important because of inclusion of L2s in a national curriculum, strong curriculum standards, parental reinforcement, and adolescent social experiences. Although cultural conditions vary from country to country, these cultural conditions are largely absent in the U.S. Another factor impacting U.S. practices is that millions of students around the world are learning English as a L2, a reality that reduces further the motivation for U.S. students to learn a L2. The impact of U.S. policies means that passing L2 courses and achieving L2 proficiency are not required, even for high-achieving students. Although many students complete two years of L2 courses in high school, very few become literate or fluent in a L2. While many colleges and universities require that students complete at least two years of L2 study in high school, many postsecondary institutions have dropped L2 courses from their core curricula and graduation requirements. Many LD and low-achieving students do not enroll in L2 courses at any level of education and are not advised to do so. There are cases where some LD

58

Chapter Three

students are required to enroll in L2 courses because of a high school’s policy or because they are encouraged to enroll by their parents in preparation for college. These students are provided with instructional and testing accommodations that may allow them to pass the L2 course but do not ensure that they achieve proficiency in the L2. In other cases, especially in college, students classified as LD are granted waivers and course substitutions, a practice that typifies current U.S policies about the importance of language learning.

Works cited American Council on the Teaching of Foreign Languages (ACTFL). 2002. American Council on the Teaching of Foreign Languages Proficiency Guidelines. Hastings-on-Hudson, NY: Author. Amend, Amanda E., Carolyn A. Whitney, Antonia T. Messuri, and Hideko Furukawa. 2009. “A Modified Spanish Sequence for Students with LanguageǦbased Learning Disabilities.” Foreign Language Annals 42:27–41. Americans with Disabilities Act. 1990. (42 U.S.C. 12101-12213); 49 U.S.C. 322. Carroll, John. 1962. “The Prediction of Success in Intensive Foreign Language Training.” In Training and Research in Education, edited by Robert Glaser, 87–136. Englewood Cliffs, NJ: Prentice-Hall. —. 1973. “Implications of Aptitude Test Research and Psycholinguistic Theory for Foreign-Language Teaching.” Linguistics 11:5–14. —. 1990. “Cognitive Abilities in Foreign Language Aptitude: Then and Now.” In Language Aptitude Reconsidered, edited by Thomas Parry, and Charles Stansfield, 11–29. Englewood Cliffs, NJ: Prentice-Hall. Carroll, John, and Stanley Sapon. 1959. Modern Language Aptitude Test. San Antonio, TX: Psychological Corp. Dinklage, Kenneth. 1971. “Inability to Learn a Foreign Language.” In Emotional Problems of the Student, edited by Graham Blaine, and Charles MacArthur, 185–206. New York: Appleton-Century-Crofts. Downey, Doris M., and Lynn E. Snyder. 2001. “Curricular Accommodations for College Students with Language Learning Disabilities.” Topics in Language Disorders 21:55–67. Downey, Doris M., Lynn E. Snyder, and Barbara Hill. 2000. “College Students with Dyslexia: Persistent Linguistic Deficits and Foreign Language Learning.” Dyslexia 6:101–111.

Students Classified as Learning Disabled in L2 Courses

59

Fletcher, Jack, G. Reid Lyon, Lynn Fuchs, and Marcia Barnes. 2007. Learning Disabilities: From Identification to Intervention. New York: Guilford. Gajar, Anna H. 1987. “Foreign Language Learning Disabilities: The Identification of Predictive and Diagnostic Variables.” Journal of Learning Disabilities 20:327–330. Ganschow, Leonore, and Richard Sparks. 1995. “Effects of Direct Instruction in Spanish Phonology on the Native-Language Skills and Foreign-Language Aptitude of At-Risk Foreign-Language Learners.” Journal of Learning Disabilities: 107–120. Ganschow, Leonore, and Richard Sparks. 2001. “Learning Difficulties and Foreign Language Learning: A Review of Research and Instruction.” Language Teaching 34:79–98. Gillingham, Anna, and Bessie Stillman. 1960. Remedial Training for Children with Specific Disability in Reading, Writing, and Penmanship. Cambridge, MA: Educators’ Publishing Service. Moats, Louisa Cook, and G. Reid Lyon. 1993. “Learning Disabilities in the United States: Advocacy, Science, and the Future of the Field.” Journal of Learning Disabilities 26:282–294. Pimsleur, Paul. 1966. Pimsleur Language Aptitude Battery. New York: Harcourt Brace. —. 1968. “Language Aptitude Testing.” In Language Testing Symposium: A Linguistic Approach, edited by Alan Davies, 98–106. New York: Oxford University Press. Pimsleur, Paul, Donald M. Sundland, and Ruth D. McIntyre 1964. “Underachievement in Foreign Language Learning.” IRALInternational Review of Applied Linguistics in Language Teaching 2:113–150. Pompian, Nancy W., and Carl P. Thum. 1988. “Dyslexic/Learning Disabled Students at Dartmouth College.” Annals of Dyslexia 38:276– 284. Schneider, Elke, and Tsila Evers. 2009. “Linguistic Intervention Techniques for AtǦRisk English Language Learners.” Foreign Language Annals 42:55–76. Siegel, Linda S. 1989. “IQ is Irrelevant to the Definition of Learning Disabilities.” Journal of Learning Disabilities 22:469–478. Sparks, Richard L. 1995. “Examining the Linguistic Coding Differences Hypothesis to Explain Individual Differences in Foreign Language Learning.” Annals of Dyslexia 45:187–214.

60

Chapter Three

—. 2001. “Foreign Language Learning Problems of Students Classified as Learning Disabled and Non-Learning Disabled: Is There a Difference?” Topics in Language Disorders 21:38. —. 2006. “Is There a ‘Disability’ for Learning a Foreign Language?” Journal of Learning Disabilities 39:544–557. —. 2012. “Individual Differences in L2 Learning and Long-Term L1-L2 Relationships.” Language Learning 62, Supp. 2:5–27. Sparks, Richard L., Marjorie Artzer, James Javorsky, Jon Patton, LeonoreGanschow, Karen Miller, and Dottie Hordubay. 1998. “Students Classified As Learning Disabled and NonǦLearning Disabled: Two Comparison Studies of Native Language Skill, Foreign Language Aptitude, and Foreign Language Proficiency.” Foreign Language Annals 31:535–551. Sparks, Richard L., Marjorie Artzer, Jon Patton, Leonore Ganschow, Karen Miller, Dorothy J. Hordubay, and Geri Walsh. 1998. “Benefits of Multisensory Structured Language Instruction for At-Risk Foreign Language Learners: A Comparison Study of High School Spanish Students.” Annals of Dyslexia 48:239–270. Sparks, Richard L., and Leonore Ganschow. 1991. “Foreign Language Learning Differences: Affective or Native Language Aptitude Differences?” The Modern Language Journal 75:3–16. Sparks, Richard L., Leonore Ganschow, Silvia Kenneweg, and Karen Miller. 1991. “Use of an Orton-Gillingham Approach to Teach a Foreign Language to Dyslexic/Learning-Disabled Students: Explicit Teaching of Phonology in a Second Language.” Annals of Dyslexia 41:96–118. Sparks, Richard, Leonore Ganschow, and Jane Pohlman. 1989. “Linguistic Coding Deficits in Foreign Language Learners.” Annals of Dyslexia 39:177–195. Sparks, Richard L., Nancy Humbach, and James Javorsky. 2008. “Individual and Longitudinal Differences among High and LowAchieving, LD, and ADHD L2 Learners.” Learning and Individual Differences 18:29–43. Sparks, R., and James Javorsky. 1999. “Section 504 and the Americans with Disabilities Act: Accommodating the Learning Disabled Student in the Foreign Language Curriculum: An Update.” Association of Departments of Foreign Languages (ADFL) Bulletin 30:36–44. Sparks, Richard L., and Benjamin J. Lovett. 2009. “Objective Criteria for Classification of Postsecondary Students as Learning Disabled: Effects on Prevalence Rates and Group Characteristics.” Journal of Learning Disabilities 42:230–239.

Students Classified as Learning Disabled in L2 Courses

61

Sparks, Richard, Jon Patton, Leonore Ganschow, and Nancy Humbach. 2009. “LongǦTerm Crosslinguistic Transfer of Skills From L1 to L2.” Language Learning 59:203–243. Sparks, Richard L., Lois Philips, Leonore Ganschow, and James Javorsky. 1999a. “Students Classified as LD and the College Foreign Language Requirement: A Quantitative Analysis.” Journal of Learning Disabilities 32:566–580. Sparks, Richard L., Lois Philips, Leonore Ganschow, and James Javorsky. 1999b. “Comparison of Students Classified as LD who Petitioned for or Fulfilled the College Foreign Language Requirement.” Journal of Learning Disabilities 32:553–565. Sparks, Richard L., Lois G. Philips, and James Javorsky. 2002. “Students Classified as LD Who Received Course Substitutions for the College Foreign Language Requirement: A Replication Study.” Journal of Learning Disabilities 35:482–500. Sparks, Richard L., Lois Philips, and James Javorsky. 2003. “Students Classified as LD Who Petitioned for or Fulfilled the College Foreign Language Requirement—Are They Different? A replication study.” Journal of Learning Disabilities 36:348–362. Stanovich, Keith E. 1991. “Discrepancy Definitions of Reading Disability: Has Intelligence Led Us Astray?” Reading Research Quarterly: 7–29. —. 2005. “The Future of a Mistake: Will Discrepancy Measurement Continue to Make the Learning Disabilities Field a Pseudoscience?” Learning Disability Quarterly: 103–106. Stuebing, Karla K., Jack M. Fletcher, Josette M. LeDoux, G. Reid Lyon, Sally E. Shaywitz, and Bennett A. Shaywitz. 2002. “Validity of IQDiscrepancy Classifications of Reading Disabilities: A MetaAnalysis.” American Educational Research Journal 39:469–518. U. S. Office of Education. 1968. First Annual Report of the National Advisory Committee on Handicapped Children. Washington, D.C.: U. S. Department of Health, Education, and Welfare. U.S. Congress. (1997). Individuals with Disabilities Education Act (First Session ed.): One Hundred and Fifth Congress of the United States of America. U. S. Department of Education. 2010. “Education and the Language Gap.” Accessed October 15, 2012. http://www.ed.gov/newspeeches/ education-and-the-language-gap-secretary-arne-duncans-remarksforeign-language-summit. Wallace, Charles, and Wallace E. Lambert. 1972. Attitudes and Motivation in Second Language Learning. Rowley, MA: Newbury.

Chapter Three

62

Ysseldyke, James E. 1983. “Current Practices in Making Psychoeducational Decisions about Learning Disabled Students.” Journal of Learning Disabilities 16:226–233.

Appendix A Lists of real words and pseudowords for Spanish, French, and German Spanish

French

Real words

Pseudowords

Real words

Pseudowords

anoche enero isla orilla usted mesa seĔora jefa entretenamiento salón inventado mural la amada agencia filólogo repentinamente antena corriente alto preocupacíon

loche regua traceo placeta sucrete popeta porrosca asurge hastilla movadiza vestuto cantón calahoria meradario zebajo pantaora cebaduría grallanado llenosidad yagüe

adorable midi robe chaise difficile croix ouest soeur magnifique aéroport soleil joël longtemps pluie médicament rançon hier décidé nationalité réveillon

elars trousante etez pentants tonseur pateau gaissons pretons els lace desentrez tadelle ponte exploter d’assoint soulangerie trençons boignon ramposé touvertes

Students Classified as Learning Disabled in L2 Courses German Real words

Pseudowords

frei schier nachbar bitte bücher einwand todesurteile pflicht jawohl deshalb ausgezeichnet fussboden leute flugzeug besonders walstatt geschlecht räuber rauchen unbezähmbar

zogen angesammen biedel leiner nochtbar auwinnern sotürlich zeiben speulen möffel besuchgekammen kössen weschen hilgescher hols wangst vertaunen mautern leisling dömpferbehn

Appendix B Reading comprehension letter for L2 proficiency test2 Please read this letter and answer the ten questions found on the next page. You will have 15 minutes to read the letter and answer the questions. You may refer to the letter when answering the questions. Madrid, 5 de marzo, 2012 Querido John, Hace un mes que estoy en España. ¡Cómo vuela el tiempo! Tal vez es porque me acostumbré rápidamente la vida de Madrid. Desde mi llegada no he descansado ni un minuto. Cuando yo vuelva a los Estados Unidos, la primera cosa que haré es relajara. Pero en este momento no tengo tiempo. 2

There were separate letters for males and females.

63

Chapter Three

64

Hay tantas cosas que hacer y ver. Las clases del colegio todavía me van muy bien. Nunca he trabajado tanto en toda mivida. De todos mis profesores pienso que el profesor de literatura es el mejor. Nos da mucha tarea pero aprendemos tantas cosas interesantes. He aprendido jugar al fútbol. Si me vierasb jugar, reirías. Pero es tan divertido. También viajo un poco. Ya he ido a Toledo y a Segovia con dos de mis compañeros declase. El mes que viene tenemos vacaciones de una semana. Visitaremos la Costa del Sol. Pienso que tomaremos el tren porque está a una distancia larga de aquí. Prometo mandarte una bella tarjeta postal de allí para tu colección. Antes de terminar esta carta quiero darte las gracias por la carta bonita que me mandaste el mes pasado. Lo siento de no tener tiempo para escribir más. ¿Pudierasc escribirme otra carta para contestarme estas preguntas? 1. 2. 3. 4. 5.

¿Qué pasa en la escuela? ¿Qué pasa los fines de semana? ¿Conociste a un amigo nuevo? ¿Qué hiciste durante las vacaciones de la Navidad? ¿Qué harás durante las vacaciones del verano?

Escríbeme con todas las noticias tan pronto como tengas el tiempo. Hasta pronto, Peter a

relajar – to relax vieras – from the verb ver c Pudieras – from the verb poder b

Students Classified as Learning Disabled in L2 Courses

Appendix C Reader’s digest passage for L2 proficiency test Los Pelos de Punta MoniqueLanders, estudiante de enseñanza media, de 16 años, tiene un negocio de trenzadoa del cabellob al estilo africano. Mejor dicho: lo tenía, hasta que un consejoc gubernamental del Estado de Kansas la clausuród porque la chica no cuenta con una licencia de cosmetóloga. Monique se enteróe de que debía renunciar a su actividad cuando regresó de Nueva York, donde fue reconocida, junto con otros cuatro jóvenes, como una destacadaf empresaria de edadpreuniversitaria. Un funcionariog del Consejo de Cosmetología del Estado de Kansas leyó la noticia del premio, y le exigió a Monique que abandonara el negocio, pues de lo contraririo tendría que pagar una multah o ir a la cárcel o ambas cosas. El director del consejo dice que Monique debe asistir a una escuela de cosmetología para dedicarse a trenzar el cabello. Y la chica replica que no la admitirán en ninguna escuela hasta que cumpla 17 años. “Primero necesito terminar la enseñanza media”, agregai. “El consejo nome permite ganar mi propio dinero, ni deja que los jóvenes como yo salgamos adelante por nosotros mismos”. a

trenzar – to braid cabello – hair c consejo – council d clausurar – to close e enterarse – to find out f destacada – outstanding g funcionario – bureaucrat h multa – fine i agregar – to add b

65

Chapter Three

66

Appendix D List of spelling words for Spanish, French and German Spanish

French

German

bien arpa dulce gafas litro maiz pago después secreto paisaje placita tranquilo mirando debajo horarios tagarote periódico felicidad zanahoria abecedario

faire comme maison cher vraiment voudrais etoile pendant visiter poivre déjeuner choisir pourrais adorable printemps chanteuse raconteur magnifique chaise ouvrir

zeit handschuh natürlich lederhosen kuchen bücher verbinden hoffentlich wechsel bahnhof bienenstich pferderei gefallen schmeckt frucht gewöhnlich besuchen schule obst mittagessen

Students Classified as Learning Disabled in L2 Courses

67

Appendix E Scoring criteria for L2 writing and oral proficiency (listening/speaking) tasks Skills I. Writing Vocabulary Cultural appropriateness Structures Comprehensibility Spelling II. Speaking/Listening Pronunciation Vocabulary Grammar Comprehensibility Listening comprehension 0 = No production 1 = Novice-Low 2 = Novice-Mid 3 = Novice-High 4 = Intermediate-Low 5 = Intermediate Mid/High

No production

Novice Low

Mid

Intermediate High

Low

Med/ High

CHAPTER FOUR ASSESSING D/DEAF STUDENTS AS VISUAL L2 LEARNERS: FROM THEORY TO PRACTICE KATHERINE M. GROVES, MARIA TAGARELLI DE MONTE AND FRANCA ORLETTI

Language learners who are D/deaf encounter major difficulties in developing mastery in a written/spoken language. As an international phenomenon, the factors affecting language acquisition (including modality of acquisition) are combined with cultural and linguistic features of the local educational environment to create a unique situation for each D/deaf language learner in their language acquisition process. In this chapter we discuss these factors and how second language acquisition studies can contribute towards the development of educational methodologies and assessment practices that provide a productive language learning situation for D/deaf language learners. Theoretical and practical approaches will be analyzed from the deaf education perspective, with attention to the potential use of the Common European Framework of Reference for languages (CEFR) in the development of language education curricula for D/deaf learners. This evaluation from both theoretical and practical perspective aims to stimulate reflection for future work in this area.

70

Chapter Four

1. Introduction It is an international phenomenon that signers and oral deaf people (henceforth, D/deaf) 1 approach language acquisition and learning differently from the majority of their non-deaf peers. It is also the case that a large number of them experience reading/writing literacy problems regardless of the language modality they use (speech with residual hearing or sign language). The literacy problems faced by a large majority of D/deaf people are largely discussed by Leeson (2006), who provides a clear description of the existing situation in several European countries; Grushkin (1998), who outlines the importance of literacy in the lives of D/deaf people; and Fabbretti and Tomasuolo (2006), who depict the Italian situation. As we will see in this chapter, a variety of independent factors plays a role in affecting the quality of language acquisition. These factors include the level of hearing loss, age of first language acquisition, local educational policies, quality of access to education and of education itself (Marschark and Spencer 2003). The varying impacts of these factors result in the differing competencies and learning needs among D/deaf language learners (DLLs). In this chapter we will outline issues regarding language acquisition in DLLs and the evaluation of linguistic abilities without focusing on physiological abilities. In this direction, we will provide an examination of second language (L2) acquisition research as it pertains to all DLLs focusing on theories and approaches including those of the Processability Theory (Pienemann 1998), error analysis (Corder 1967), interlanguage (Selinker 1972) and the Common Underlying Proficiency model (Cummins 1984). This examination of L2 research, normally applied to hearing language learners, will consider varying viewpoints supporting the application of L2 research to DLLs (Berent 2009; Berent and Kelly 2008; Schönström 2010). We will discuss adaptation efforts for a Common European Framework of Reference for Language (CEFR) for D/deaf written/spoken language users. We will also refer to ways in which linguistic aspects of the spoken language need to be preserved when represented by the written language. An outline of suggestions for carrying out the adaptation of existing evaluation tools will conclude our contribution, in an attempt towards a

1

In this chapter D/deaf refers to all deaf and hard of hearing people regardless of whether they use sign language or spoken language as their first language.

Assessing D/Deaf Students as Visual L2 Learners

71

better understanding of written language development and evaluation in D/deaf learners.

2. To be a D/deaf language learner Literature on deafness has widely shown that children who are born deaf or become deaf before the third year of age tend to encounter serious challenges in the acquisition of the national spoken language. These challenges involve not only the vocal and hearing rehabilitation (through the use of hearing support), but also the mastery of the written language (see the edited volumes of Marschark and Spencer 2003; Marschark et al. 2010 and 2011). The literacy level in prelingual D/deaf people (including adults) often establishes itself to a level comparable to that of hearing children aged 10– 13 (Leeson 2006). Difficulties are experienced in both the grammatical and lexical dimensions of language (for the Italian situation, see Caselli et al. 2006; Fabbretti and Tomasuolo 2006). While some of the difficulties are culture and language specific (Leeson 2006), literacy development in the written/spoken language remains a challenge for D/deaf people. It indicates that particular aspects of literacy learning are deafness-related. From the deafness perspective, defining a DLL means taking into account various factors (see Konigsmark 1972; Braden 1994; Gregory 1996; Marschark et al. 2002; Marschark et al. 2009) that play a role in language development. These factors include: -

level of hearing loss age of onset nature and quality of mother-child communication age of first language acquisition quality of access to education educational/rehabilitation program (Individualized Education Plan) quality of the educational environment local/national educational policies

These factors are not always considered as a whole throughout the course of addressing the DLL's needs. From the diagnosis perspective, doctors do not often think of such factors as local educational policies as much as teachers without prior experience with deafness might not think of the quality of the DLL's access to education. These factors, however, all play their own respective roles in determining the overall language

72

Chapter Four

competence that the DLL will acquire, and how this competence can be measured and built upon throughout the learning process. Many D/deaf individuals learn spoken language as their first language at a later age than their hearing peers, normally upon diagnosis of deafness. Meaningful exposure to language thus begins at the same time as speech therapy (Militano 2002) and formal education (Berent 2009). This leads to delays in meaningful exposure to language that is often difficult to recover. It results in a gap between the language development in D/deaf children and that of their hearing peers (in absence of other conditions that may impact language development). This delay in meaningful exposure does not appear in the case of D/deaf children born into signing D/deaf families. In a study conducted by Freel et al. (2011), these children showed rare delays in language development, scoring higher on literacy tests than D/deaf children born into non-Deaf signing families. The study of Freel et al. presents strong support for sign language not only as a natural first language for D/deaf children, but also as a supporting tool towards the development of written language proficiency (see also Rathmann et al. 2007; Schönström 2010). In these cases, the role of bilingualism constitutes a factor in the evaluation of the DLL's language skills (Marschark and Hauser 2008).

Fig. 1. Classification of Deaf Language Learners2

One of the latest trends in research on D/deaf literacy development compares the quality of input to the written/spoken language to that of 2

Based on Berent and Kelly 2008.

Assessing D/Deaf Students as Visual L2 Learners

73

prototypical second language learners. Second Language Acquisition (SLA) theories have been tested for applicability to D/deaf language learners. Coming from this perspective, Berent and Kelly (2008) have divided DLLs into the categories described in Fig. 1. Following Berent and Kelly's classification, the next section will focus on the first two categories of DLLs (DL1Sig and DL1i) and provide reflections on theories coming from cognitive linguistics and language education.

3. Second language acquisition in D/deaf learners From the perspective of L1 sign language users and oral deaf students still developing their L1 in the written/spoken language, we will briefly examine interlanguage (Selinker 1972), error analysis (Corder 1967) and Common Underlying Proficiency (Cummins 1984). The aim is to gain insight into language development and error analysis in DLLs. We will also analyze an attempt to apply the Processability Theory (Pienemann 1998) to the development of the D/deaf written/spoken language competence. An understanding of the way in which L1 and L2 are acquired by DLLs is a prerequisite to understanding how language is conveyed and thus how to effectively evaluate the acquisition by the learner at a given moment.

3.1 Bridging concepts and creating links to language Deaf language learners with a strong L1 (we refer to this as DL1Sig and DL1Sp in section 2) have a basic foundation from which concepts can be transferred from one language to another. This phenomenon is explained in Cummins’ “Common Underlying Proficiency” model (Cummins 1984 and 2000). L1 learners need to focus on developing the entire package of language competence—including syntax, semantic meaning, lexicon and phonetics. On the other hand, the L2 learner draws upon their knowledge of semantic meaning to learn the features (phonetics, lexicon, and syntax) of the new language (see Fig. 2). Semantic meaning might not even be expressed in any particular language, but rather as concepts that are clear to the language user.3 In the case of DL1i, semantic meaning might not be

3

See Courtin et al. (2008) for an analysis of the emergence of Theory of Mind in D/deaf children of Deaf vs. hearing parents.

74

Chapter Four

as rich as the common underlying proficiency of L2 with a full L1. The concept of interlanguage can help us better understand the DL1i.

Fig. 2. Common Underlying Proficiency model4

Interlanguage is a phenomenon first defined by Selinker (1972) in which learners of a new language “negotiate” existing language knowledge with concepts from the new language. Theoretically, without the presence of the phenomenon of fossilization, this personal pidginized language zone would shrink as the competency in the new language increases. Within the interlanguage zone, as we see in Fig.3, is the error zone: the types of errors the language learner makes will depend on their origins within the interlanguage zone and the amount of influence of L1 or L2 on the error. Analysis of these errors (Corder 1967) is a valuable strategy for the language teacher to understand where the learners’ difficulties are found, and to be able to address them in the classroom environment5. In contrast to the DL1Sig and DL1Sp outlined previously, oral deaf learners with weak L1 skills (DL1i) can be considered to be L2 learners without an L1 due to the severely limited access to their first language input (Berent and Kelly 2008). The facilitation of the acquisition of the spoken language through the visual channel of communication provides only limited input to the spoken language (Berent and Kelly 2008). The visual channel of communication involves “lip reading, facial expressions, 4

Based on Cummins 1984. It is worth noting that there is often a lack of connection between research and instruction: error analysis provides a powerful tool for personalizing language instruction but teachers lack appropriate training in this area. This occurs particularly in the understanding of how modalities (signed versus spoken) and individual languages may have an impact on the types of errors a DLL might make.

5

Assessing D/Deaf Students as Visual L2 Learners

75

signed English, cued speech, and (once a foundation is established) reading and writing” (Berent 2009, 524). Such facilitation is, thus, less effective for a DLL than it will be for a prototypical hearing learner without such limitations. It is crucial to consider how to effectively facilitate language acquisition in order for the DLL to build a semantic base from which the rest of linguistic and literacy competencies is constructed.

Fig. 3. Error zone within the interlanguage zone in L2 learners6

In the case of DL1i, who has not a fully developed first language to fall back on, capitalizing on error analysis is difficult because it is unclear where and what the constituents of the language competence showed by the learner are. Berent (2009) suggests that the errors produced by a DL1i will come from attempting to directly access a universal grammar instead of relying on their knowledge in L1 to build their competence in L2 like hearing L2 learners do (540). This observation suggests the need to determine the differences in errors in L1 Æ L2 learners and late L1 learners. It is necessary to study whether or not there is a possibility that certain concepts are obtained from direct access to universal grammar rather than transfer from an L1. It is not clear, however, how learners would have access to universal grammar and the way in which the linguistic competence is influenced by it. More research needs to be conducted in this area to understand if universal grammar indeed plays a role here. This is beyond the scope of this chapter. The position held by Berent and other SLA researchers to explain the language acquisition in D/deaf learners provides potential stimulation for the advancement of research in this area. However, it also serves as confirmation that further appropriate research in the area of oral language development of D/deaf learners is needed. This is particularly important when considering the difficulties that L1 teachers meet with regards to oral deaf learners. Oral deaf learners display difficulties toward the spoken L1 6

Based on Selinker 1972 and Corder 1967.

76

Chapter Four

without any other language to aid them in constructing concepts and understanding (see Marschark et al. 2009; Nuccetelli 2010; Spano et al. 2011).

3.2 Processability Theory As a theory which predicts and describes the grammar development of L2 learners, the Processability Theory (PT), first developed by Pienemann (1998) defines a universal framework of second language development. This framework can serve a better understanding of the stages of an incremental process, with each stage developing as a consequence of the skills acquired in the previous stage.7 The “universality” of PT and its applicability to DLLs were some of the core questions of the doctoral project by Swedish linguist Schönström (2010). He studied the written production of Swedish Deaf learners with a sign language as an L1 (in this case Svenskt teckenspråk, STS), learning a second language in a different modality (written Swedish). For his study he focused on the following stages of development (based on Schönström 2010, 196): 1. The identification and acquisition of the words of the target language; 2. The acquisition of the lexical morphology of the target language (e.g. verb and noun suffixes); 3. The acquisition of phrasal morphology (e.g. the rule of noun phrase agreement); 4. The extension of processing to include feature unification between phrases (e.g. predicate agreement plus the rule of subject-verb inversion); 5. The acquisition of clause boundaries (e.g. different word order rules for main and subordinate clauses).

Schönström (2010) divided Swedish texts produced by Swedish Deaf pupils from grades 5 to 10 into analyzable groups following the “macro syntagm analysis,” (Loman and Jörgensen 1971). He also applied the acquisition criterion of PT towards the analysis of the texts from two perspectives: 1. Emergence criterion, focusing on the learner’s first systematic use of a structure; 7

PT is formally modeled using Lexical Functional Grammar (see Kaplan and Bresnan 1982; Bresnan 2000), focusing on syntactical structures through its relationship to morphology and semantics.

Assessing D/Deaf Students as Visual L2 Learners

77

2. Mastery criteria, using a scale to measure a learner's mastery of a structure.

Following the emergence criterion, Schönström (2010) found that: […] the results show clearly that the informants follow a developmental manner as predicted by PT. Their development is implicational and therefore supports PT as a L2 development theory that is applicable to Deaf L2 learners.8

Following the mastery criteria, however, showed that some of the learners in his study showed weaker skills at the third stage (acquisition of phrasal morphology) than the subsequent stages. Furthermore, some of the learners seemed to develop structures at the fourth stage (extension of processing) before developing mastery of the second (acquisition of lexical morphology) and third (acquisition of phrasal morphology) stages. The variation in competency among different levels might come from its ease of comprehension on the part of the learner that was different from expected. It is also possible that morphological rules (stages 2 and 3) were easier to emerge than to master for the Deaf learners in his study in comparison to syntactic rules (stages 4 and 5). 9 It remains unclear, however, which factors affect mastery between level. This is worth further study to understand this aspect of the learning process in Deaf L2 learners.

4. Evaluating language input and output: quantity and quality Reading alone cannot serve as a primary visual conduit of spoken language input because reading presupposes linguistic knowledge of the spoken language symbolized through the writing system .10

Another point that deserves further exploration, bringing us closer to the practical experience of teachers of D/deaf learners, is the one related to the quantity and quality of written/spoken language input for DLLs, as well as the existing relation between these and the output.

8

Schönström 2010, 198. Krister Schönström, email message to author, November 30, 2012. 10 Berent and Kelly 2008, 81. 9

78

Chapter Four

4.1 Language input Regardless of the preferred mode of communication of the DLL, Grushkin's state of the art on deafness and literacy (1998) urges the need to recognize D/deaf people as visual learners. From this perspective, Grushkin argues that anything primarily based on sound will be the weakest point in literacy development for D/deaf students. Since hearing children often learn the written language starting with connecting sounds to print, D/deaf children are at a disadvantage due to the poor quality of spoken language input. In the absence of auditory input, perceptions of the world for D/deaf people occur through the other senses. In the area of cognitive studies, research has recently shown that a disruption of auditory stimuli at a young age causes a cross-modal reorganization in areas of the brain that process information from different modalities (Dye et al. 2008). The study by Dye's research team also showed that the D/deaf individual’s rely on vision as the primary source of language input and visual information. This reliance causes an overload of information and a redistribution of attention resources, affecting behavior connected to the division of attention to the environment around him. This knowledge could help the development of more appropriate educational methodologies adjusted to this particular type of learner, although it is not yet fully understood how it can be applied. Berent (2009) pointed out the fact that input for D/deaf learners through the visual domain is not as comprehensive in terms of creating “equal opportunities” as is available for hearing L1 learners. In this case, when considering language instruction adapted for D/deaf learners, the quantity and quality of input must be consciously considered. By quality, we mean the ability of the D/deaf learner to absorb the input as a natural source of language, meaning the input is visual and meaningful. This definition also considers the adoption of specific strategies by the speaker/teacher to capitalize on the attention resources available to the learner (all senses except hearing) to convey meaning. A simple example, often overlooked, is to allow the learner to read a visual aid (such as a PowerPoint slide) before discussing its contents, since they cannot simultaneously follow the visual aids and the lecture. By quantity we refer to the inundation of the learner's environment with language input such as full texts, subtitles, transcripts, notes, words, phrases to which the learner can return to continuously when needed. In the case of bilingual learners who approach the written language through sign language (DL2), capitalizing on the learner’s semantic base (vis à vis Cummins’ Common Underlying Proficiency) is crucial for bridging concepts from one language to another. However, it is just as crucial to

Assessing D/Deaf Students as Visual L2 Learners

79

provide as much access and experience with the target language to allow the learner to develop connections to both languages based on actual language use.

4.2 Language output The assessment phase focuses on the learner's language output, and it is important to remember the possible influences on language input by all the variables we have discussed thus far in this chapter. When assessing the DLL’s language competence, it is important to consider the DLL’s access to the evaluation materials. If the language of the evaluation materials is too difficult, the evaluator will be measuring the DLL’s ability to access the measurement tool rather than obtaining an accurate picture of the DLL’s overall language abilities. Grushkin (1998) writes that reading must be viewed as a “visually and cognitively rather than phonologically based process” (181). Thus, Grushkin continues, educators’ attention should be turned to […] visually based and cognitive strategies for reading instruction, since these are two of the areas in which deaf and hard-of-hearing readers possess no disadvantage in comparison to their hearing peers.11

We must ask ourselves how this might be brought from theory into practice. Thus far, Italian research on the literacy development of D/deaf learners have focused on grammar, syntax and morphology, leading to the identification of typical errors made by Italian DLLs. Only recently has the nature of their difficulties been investigated through the lens of developmental psychology. These studies have confirmed the influence of sign language on the acquisition of the spoken language (Tomasuolo 2006) and the influence of deafness on the development of semantic (Russo Cardona and Volterra 2007) and syntactic (Fabbretti 1997) structures of language competence.

5. Looking for standards: the role of the CEFR As we have seen, the goal of assessing literacy skills in DLLs becomes complicated by many variables. One such issue that we have yet to address is a standard level of literacy to use as a reference when evaluating DLLs. Such a standard includes the levels described by the Common European 11

Grushkin 1998, 191.

80

Chapter Four

Framework of Reference for Languages (CEFR). It is used to measure the communicative competences of foreign language learners—primarily hearing—in Europe. Since its inception in 1996,12 the CEFR has become one of the major frameworks for professionals and students. It covers the areas of language development, teaching and evaluation not only in Europe but increasingly as a reference document outside the EU area. The CEFR was originally designed as a generic document to allow for standardization of assessment throughout Europe. This means it requires adaptation when applied to individual languages, generically referred to as a language profile. Despite its limitations as a definitive resource (see Weir 2005; Muñoz et al. 2006; de Jong 2011; van der Haagen and de Haan 2011), the CEFR holds potential as a guideline for language education and evaluation of DLLs with consideration of their particular language usages and learning needs. Thus far, the CEFR has already been targeted in adaptation efforts for signed language instruction for second language learners and spoken language evaluation for DLLs.

5.1 Existing work in connection to sign languages Many countries in Europe with formal education for sign language have begun to adapt their teaching curricula through the development of specific profiles for sign language, aiming at the communicative levels described by the CEFR. However, the existing differences 13 in the interpretation of non-manual components of sign language, such as facial expressions, body postures and mouthings, as well as the lack of a formally adopted written version of sign language has led to the production of several variants of CEFR for Sign language profiles. In these profiles, these two elements are treated differently. In 2011, the European Science Foundation Exploratory Workshop on Development of Theoretical and Practical Guidelines for the Adaptation of 12

The CEFR was put together by the Council of Europe as the main part of the project Language Learning for European Citizenship between 1989 and 1996. Its main aim is to provide a method of learning, teaching and assessing which applies to all languages in Europe. 13 For example, while Italy is beginning to adapt sign language classes to the levels depicted by the CEFR, the Netherlands already issues certificates of CEFR competence. France and Spain provide a full profile for French (LSF) and Spanish (LSE) sign language. While France considers video recording in place of writing, Spain considers LSE to pertain only to the oral dimension and does not provide any equivalent to the CEFR writing abilities.

Assessing D/Deaf Students as Visual L2 Learners

81

the Common European Framework of Reference (CEFR) to Sign Languages was held in Zurich, Switzerland. It was convened with the aim of discussing a CEFR for SL, and the adaptation of SL curricula and assessment (Haug and Keller 2012). One of the topics under discussion was the classification of the body skills required to sign—considered an important grammatical component of sign language competency—and the evaluation of these skills according to the communicative abilities expressed by the CEFR. It was suggested that an A0 level be added to the CEFR, created to address the rigidity that adult sign language learners face when approaching sign language for the first time. However, the issue of an A0 level is still under debate among European sign language linguists. They take into account the fact that nonverbal cues and facial expressions are considered the grammatical spine of sign language, thus falling into a separate category within the CEFR.14 A discussion was conducted within the SignLEF project 15 based on informal observations by sign language teachers of adult learners in Rome (and a few other cities in Italy). Findings agreed that novice signers tend to be more rigid and experience difficulties in the use of their body, while expert signers tend to feel comfortable in the use of facial expressions and body postures for communication.16 Another observation was that many novice Italian signers tend to confuse natural gestures with signs, using gestures in place of signs when having to produce certain meanings normally articulated by speech and gestures in everyday spoken communication. Concerning the “written form” of sign language, recently, some experts (see di Renzo et al. 2011) within the field of sign language linguistics have looked at the potential use of SignWriting. It is an adapted form of what Sutton (1974) invented for dancing, and is now spreading as a written form of signed languages. On the other hand, two factors have led a few countries, such as France, to adopt video as a recorded (“written”) form of sign language (Conseil de l'Europe 2002; Leeson and Grehan 2010). These factors are the extensive use of video communication and video chats by signers and the fact that the use of video is actually influencing sign language in a similar way to

14

Franz Dotter, email message to author, January 16, 2012; see also Haug and Keller 2012. 15 See www.signlef.aau.at. 16 Katiuscia Andò, Massimo Paletta and Roberto Zuccari (Italian Sign Language Instructors/Researchers, Istituto Statale per Sordi di Roma), in discussion with the authors, May 2012.

82

Chapter Four

the impact of writing on spoken language (see Ong 1982 and Halliday 1985/1989 for the influence of writing on oral languages17). In a situation like the one described above, any evaluation system is far from being considered final and effective. The situation for Italy, at the moment, is a general adoption of what is called Metodo Vista18 and/or the curriculum distributed by the National Association for the Deaf (Ente Nazionale Sordi). The assessment is planned according to the topics followed by the teachers in the classroom. It is normally based on testing both the practical and theoretical competence acquired by the student during the course. Practical competences in production and comprehension are tested through an interview in sign language and, according to the level of the learner, questions on the contents of videos in sign language showed during the testing phase. Theoretical competences are tested through interviews in spoken Italian.

5.2 Existing work in connection to D/deaf learners: Written/spoken languages An attempt to adapt the CEFR to the experience of oral deaf learners with spoken language (the third group of DLLs referred to by Berent, referred here as DLsp) was made by the research team of the DeafPort project. Its main objective was to develop a multilingual European Language Portfolio for the Deaf and Hearing-Impaired (ages 16 and above) to develop, extend and diversify their language skills. The DeafPort project took into consideration the communicative domains of reading, writing, speaking and listening, and included a consideration of D/deaf specific strategies for navigating and communicating in an oral world. An analysis done by Deaf researchers in the field of education and language19 has revealed the weaknesses of the tool. One of the strongest limitations of this tool is the lack of a full consideration of the D/deaf experience towards the spoken language, and the level of difficulty of the self-evaluation tool. These require a higher level of linguistic competence 17

It would be interesting to study the concepts of the relationships between oralityliteracy and computer-mediated communication in connection to sign languages. However these issues will not be addressed in this chapter. 18 Published by the Collana Sorda of the Edizioni Kappa, Metodo Vista is adapted from the Signing Naturally American Sign Language curriculum series published by DawnSignPress. 19 Gabriele Gianfreda and Kate Groves, in discussion with the authors, March 2012.

Assessing D/Deaf Students as Visual L2 Learners

83

than the one being tested, as seen above. Another limitation consists in the evaluation of comprehension of auditory input, and the individual ability to lip read various types of people. Lip reading skills are not directly tied to language abilities, since only a percentage of sounds are produced, and are therefore visible, on the lips. Thus, despite the value of the CEFR to make language educators aware of specific listening and speaking styles of D/deaf learners, the risk is to overemphasize the physiological difficulties in accessing a spoken domain. In the construction of educational activities for language learning programs, the research unit of the Istituto Statale dei Sordi di Roma worked on the adaptation of learning behaviors within the CEFR. Its aim was to fit the goals and characteristics of its end users. When examining the CEFR for teaching written Italian they realized that the evaluation of listening or speaking competences needed to be adapted to the particular way in which D/deaf people access auditory content. Some of the existing learning behaviors have thus been adapted to consider the approaches D/deaf people take to various communication situations. An example of a modification of learning behaviors, taken from the A2 level, concerns audiovisual reception skills from an original CEFR behavior: Original: Can identify the main point of TV news items reporting events, accidents etc. where the visual supports the commentary.20 Modified: Can identify the main point of TV news items reporting events, accidents etc. where subtitles and the visual support the commentary.

The process of work with the CEFR for its use with DLLs, however, requires more intensive study and analysis. Various issues need to be taken into account, such as the evaluation of the skill being developed when listening (attending21) to a spoken text or reading its transcript. As we have outlined in section two, the evaluation of the capacity of understanding a spoken text depends on the quality of both educational and rehabilitation factors, with the second focusing on physiological abilities and thus inapplicable to this analysis. On the other hand, how far can the written form of spoken dialogue truly represent the spoken language and when does it become a reading 20

Council of Europe 2001, 71. We refer to attending as the behavior of receiving information in its auditory form, since D/deaf people may or may not have the ability to utilize their residual hearing to comprehend speech. 21

84

Chapter Four

competence? For example, in a State of the Union address given by U.S. President Barack Obama in 2012, the video alone is not accessible to a majority of D/deaf viewers. Access to the speech via a separate transcript22 allows the D/deaf viewer to read President Obama's speech. However it is at times difficult to comprehend the text without the accompanying video due to particular speech effects he utilized to interact with his audience. The difficulty, here, is in the interpretation of a spoken style while reading a written text (see Fig.4).

Fig. 4. Transcript of audio information23

The use of subtitles directly on video (Obama 2012a) allows the reader to follow the pace of the speaker. Yet the reader must be able to follow the dialogue instantaneously and to hold together the threads of the speaker's dialogue (see Fig. 5).

22 23

Obama 2012b. Courtesy of www.whitehouse.gov.

Assessing D/Deaf D Students as Visual L2 L Learners

85

Fig. 5. Video with audio capptions24

Another example of visual v access is a video off the same sp peech that was created with subtitless and graphicss (Obama 20122b) (see Fig. 6). 6

Fig. 6. Enhannced video25

The grapphics served to support th he content off the speech,, and the subtitles alsso included sound effects, such as aapplause, pau uses, and interrupted w words. The use of subtitles following f the natural spee d of speech, however, would helpp in evaluatiing the leveel of compettence in thee learner, considering the similaritiies of the requ uired abilitiess to those of listening: natural speeed, one-shot exposure, e diffiiculties given by backgrou und noise. In all the exxamples givenn, the quality and quantity of the visual language input changges the overalll access of th he video for thhe viewer wh ho cannot

24 25

Courtesy off www.whitehoouse.gov. Courtesy off www.whitehoouse.gov.

86

Chapter Four

access the auditory information. However, the question regarding the competence these require—listening or reading?—remains.26

6. Concluding comments As we have seen, two things remain constant in the DLLs’ approach to language learning: 1. Language learning is strongly influenced by the degrees of its visibility (such as lip reading and printed text) and accessibility (availability of text alternatives to speech, meaningful texts, etc.); 2. From the perspective of the language learning process towards the written/spoken language of the majority culture in which the learner lives, many D/deaf first language (L1) learners seem to function similarly to hearing second language (L2) learners. The similarity involves the limited input to the target language and the influence of the first language (or lack of) on the language being learned. In this chapter we have discussed how second language acquisition theory, and the related evaluation methodologies, can be applied to D/deaf learners. A main concern is the lower quality and quantity of language input available to them in comparison to their peer L1 learners. However, DLLs and L2 learners are not exactly the same. By outlining their differences, we get a better understanding of how teaching methodologies and assessment methods can be adapted to fit the realities of learning approaches used by D/deaf learners. The goal is to maximize their chances for learning a written language, from the perspective of making quality input available. The studies that have been outlined in this chapter, have given us evidence that second language acquisition theory can be applied to D/deaf learners (i.e. Schönström’s study, 2010, of the Processability Theory with signing Deaf learners). By applying the same route of analysis of language structures to the development of CEFR guidelines, we understand that the CEFR descriptions do not need to change at the level of content for D/deaf language users.

26 See Sacks et al. 1974 for a discussion of the types of spoken structures that remain preserved while presented in the written form.

Assessing D/Deaf Students as Visual L2 Learners

87

As useful guidelines for measuring language competencies of language learners and as a tool for guiding learners to higher levels of abilities through self-monitoring and scalable learning materials, the CEFR provides a valuable standard. It constitutes a tool in an area where there is none, for developing useful guidelines adapted to DLLs' existing levels, and ways of bringing those learners to a higher level of proficiency. However, an initial study of the CEFR renders half of it—the listening and speaking domains—useless for DLLs. The reason is that it is converted from its original purpose of measuring communicative abilities to measuring physiological abilities of D/deaf people who will never regain their hearing nor attain perfect speech and listening abilities. Given the unique communication needs of D/deaf language learners, new or adapted language evaluation guidelines need to focus on other abilities. Such abilities should be built on constructively rather than focusing on what the learner can or cannot do because he or she is deaf. These guidelines also need to elaborate on communicative strategies independent of speech and listening abilities. They should pay attention to realistic communication situations that D/deaf language users find themselves in. They should also involve the communicative strategies they use to communicate with non signers and those who are unfamiliar with D/deaf communication needs. The information laid out in this chapter shows us how complex language learning can be for D/deaf children. It also highlights the need to consider the unique set of experiences and perceptions of the world that D/deaf people have from the very beginning. When considering the written literacy development of D/deaf people, we need to be cautious in comparing it to that of hearing people.

Acknowledgments Writing a chapter means putting a number of readings, conversations, debates and professional exchanges into a sequential order that belongs not only to the authors, but also to all those people nurturing their thoughts and ideas on a daily basis. We thus thank our collaborators and the people with whom we have had the chance to exchange special conversations on Deaf literacy. We would also like to thank the institutions providing funding to our research, without which none of this would be possible: the Italian Ministry of Education and the European Commission Lifelong Learning Programmes.

88

Chapter Four

With regards to this chapter, we would like to thank Professor Elisabetta Bonvino of the University of Roma Tre for her moral support and an initial review of our abstract. We would also like to thank Cynthia Tjhie for creating the graphics used in this chapter.

Works cited Berent, Gerald P. 2009. “Chapter 22: The Interlanguage Development of Deaf and Hearing Learners of L2 English: Parallelism via Minimalism.” In The New Handbook of Second Language Acquisition, edited by William C. Ritchie and Tej K. Bhatia, 523–543. Bingly, UK: Emerald Group Publishing. Berent, Gerald P., and Ronald R. Kelly. 2008. “The Efficacy of Visual Input Enhancement in Teaching Deaf Learners of L2 English.” In Understanding Second Language Process, edited by ZhaoHong Han, 80–105. Clevedon, UK: Multilingual Matters. Bresnan, Joan. 2000. Lexical-Functional Syntax. Vol. 16 of Blackwell Textbooks in Linguistics. Oxford: Blackwell Publishers. Braden, Jeffery P. 1994. Deafness, Deprivation and I.Q. London: Plenum Press. Caselli, M. Cristina, Simonetta Maragna, and Virginia Volterra. 2006. Linguaggio e Sordità: Gesti, Segni e Parole nello Sviluppo e nell'Educazione. Bologna: Il Mulino. Conseil de l’Europe. 2002. “Cadre Européen Commun de Référence pour les Langues: Le Premier Outil Pédagogique Pour la Langue des Signes.” Accessed 13 March, 2013. www.alfeda.net/pdf/langue%20des% 20signes.pdf. Corder, Stephen P. 1967. “The Significance of Learner's Errors.” International Review of Applied Linguistics in Language Teaching 5:161–170. Council of Europe. 2001. Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge: Cambridge University Press. Courtin, Cyril, Anne-Marie Melot, and Denis Corroyer. 2008. “Achieving Efficient Learning: Why Understanding Theory of Mind is Essential for Deaf Children.” In Deaf Cognition: Foundations and Outcomes, edited by Marc Marschark and Peter C. Hauser, 102–130. New York: Oxford University Press. Cummins, Jim. 1984. Bilingual Education and Special Education: Issues in Assessment and Pedagogy. San Diego: College Hill.

Assessing D/Deaf Students as Visual L2 Learners

89

—. 2000. Language, Power and Pedagogy: Bilingual Children in the Crossfire. Clevedon: Multilingual Matters. Deaf Port project partnership. 2010. “Deaf Port: European Language Portfolio for Deaf and Hard of Hearing People (2008–2010).” Accessed 13 March, 2013. www.deafport.eu. de Jong, John H.A.L. 2011. “Measurement Issues in Aligning Test Scores to CEF.” Plenary speech presented at the Writing Assessment in Higher Education: Making the Framework Work, Amsterdam, The Netherlands, October 26–28. Di Renzo, Alessio, Luca Lamano, Tommaso Lucioli, Barbara Pennacchi, Gabriele Gianfreda, Giulia Petitta, Claudia S. Bianchini, Paolo Rosssini, and Elena Antinoro Pizzuto. 2011. Scrivere la LIS con il SignWriting: Manuale Introduttivo. Rome: ISC-CNR. Dye, Matthew W.G., Peter C. Hauser, and Daphne Bavelier. 2008. “Visual Skills and Cross-Modal Plasticity in Deaf Readers: Possible Implications for Acquiring Meaning from Print.” Annals of the New York Academy of Sciences 1145:71–82. Fabbretti, Daniela, and Elena Tomasuolo. 2006. Scrittura e Sordità. Rome: Carocci Editore S.p.A. Fabbretti, Daniela. 1997. “Scrivere e Segnare: La Costruzione del Discorso nell'Italiano Scritto e nella Lingua Italiana dei Segni delle Persone Sorde.” PhD diss., Università degli studi di Roma, La Sapienza. Freel, Brittany L., M. Diane Clark, Melissa L. Anderson, Gizelle L. Gilbert, Millicent M. Musyoka, and Peter C. Hauser. 2011. “Deaf Individuals’ Bilingual Abilities: American Sign Language Proficiency, Reading Skills, and Family Characteristics.” Psychology 2(1):18–23. Gregory, Susan. 1996. “Bilingualism and the Education of Deaf Children.” In Bilingualism and the Education of Deaf Children: Advances in Practice: Conference Proceedings, edited by Pamela Knight and Ruth Swanwick, Leeds: University of Leeds, School of Education. Gruppo SILIS, and Mason Perkins Deafness Fund. 2004. Metodo Vista per l'Insegnamento della Lingua dei Segni Italiana. Roma: Edizioni Kappa. Grushkin, Donald A. 1998. “Why Shouldn’t Sam Read? Toward a New Paradigm for Literacy and the Deaf.” Journal of Deaf Studies and Deaf Education 3(3):179–204. Halliday, Michael A.K. 1989. Spoken and Written Language. Victoria: Oxford University Press. Haug, Tobias, and Jörg Keller. 2012. “ESF Exploratory Workshop on Development of Theoretical and Practical Guidelines for the Adaptation of the Common European Framework of Reference (CEFR)

90

Chapter Four

to Sign Languages: Scientific Report.” Summary of the European Science Foundation Exploratory Workshop, Zurich, Switzerland, 16– September 18, 2011. Kaplan, Ronald M., and Joan Bresnan. 1982. “Lexical-Functional Grammar: A Formal System for Grammatical Representation.” In The Mental Representation of Grammatical Relations, edited by Joan Bresnan, 173–281. Cambridge, MA: MIT Press. Konigsmark, Bruce W. 1972. “Genetic Hearing Loss with No Associated Abnormalities: A Review.” Journal of Speech and Hearing Disorders 37:89–99. Leeson, Lorraine, and Carmel Grehan. 2010. “A Common European Framework for Sign Language Curricula? D-Sign(ing) a Curriculum aligned to the Common European Framework of Reference.” In Papers from the 1st Symposium in Applied Sign Linguistics, edited by Maria Mertzani, 21–34, Centre for Deaf Studies, University of Bristol, September 24–26, 2009. Leeson, Lorraine. 2006. “Sign Languages in Education in Europe: A Preliminary Exploration.” Paper presented at the Intergovernmental Conference: Languages of Schooling: towards a Framework for Europe, Strasbourg, Austria, October, 16–18. Loman, Bengt, and Nils Jörgensen. 1971. Manual för Analys och Beskrivning av Makrosyntagmer. Lundastudier i Nordisk Språkvetenskap. Lund: Studentlitteratur. Marschark, Marc, Patricia Elizabeth Spencer, and Peter E. Nathan, ed. 2011. Oxford Handbook of Deaf Studies, Language, and Education. Vol. 1 of Oxford Library of Psychology. New York: Oxford University Press. Marschark, Marc, Patricia Elizabeth Spencer, and Peter E. Nathan, ed. 2010. Oxford Handbook of Deaf Studies, Language, and Education. Vol. 1 of Oxford Library of Psychology. New York: Oxford University Press. Marschark, Marc, Patricia Sapere, Carol M. Convertino, Connie Mayer, Loes Wauters, and Thomastine Sarchet. 2009. “Are Deaf Students’ Reading Challenges Really about Reading?” American Annals of the Deaf 154(4):357–370. Marschark, Marc, and Peter C. Hauser. 2008. “Cognitive Underpinnings of Learning by Deaf and Hard-of-Hearing Students: Differences, Diversity and Directions.” In Deaf Cognition: Foundations and Outcomes, edited by Marschark, Marc and Peter C. Hauser, 3–23. New York: Oxford University Press.

Assessing D/Deaf Students as Visual L2 Learners

91

Marschark Marc, and Patricia Elizabeth Spencer, ed. 2003. Oxford Handbook of Deaf Studies, Language, and Education (Psychology). New York: Oxford University Press. Marschark, Marc, Harry G. Lang, and John A. Albertini. 2002. Educating Deaf Students: From Research to Practice. Oxford: Oxford University Press. Militano, Maria Giuseppina. 2002. “Strategie di Lettura in un Bambino Sordo Figlio di Sordi e in un Bambino Sordo Figlio di Udenti.” Thesis, Università degli Studi di Roma, La Sapienza. Muñoz, Ana, Jonathan Mueller, Martha Álvarez, and Sandra Gaviria. 2006. “Developing a Coherent System for the Assessment of Writing Abilities: Tasks and Tools." Íkala, Revista de Lenguaje y Cultura 11(17):265–307. Nuccetelli, Giuseppe. 2010. “L'E-Learning Come Strumento di Promozione del Plurilinguismo nella Sordità.” In L'apprendimento a Distanza dell'Italiano. Modelli Teorici e Proposte Didattiche, edited by Andrea Villarini, 124–140. Milan: Mondadori Università. Obama, Barack. 2012a. “2012 State of the Union Address.” Accessed 13 March, 2013. www.whitehouse.gov/photos-and-video/video/2012/01/ 24/2012-state-union-address-0. —. 2012b. “2012 State of the Union Address Enhanced Version.” Accessed 13 March, 2013. www.whitehouse.gov/photos-andvideo/video/2012/01/25/2012-state-union-address-enhanced-version. Ong, Walter J. 1982. Orality and Literacy: The Technologizing of the Word. London: Routledge. Pienemann, Manfred. 1998. Language Processing and Second Language Development: Processability Theory. Amsterdam: John Benjamins. Rathmann, Christian, Wolfgang Mann, and Gary Morgan. 2007. “Narrative Structure and Narrative Development in Deaf Children.” Deafness and Education International 9(4):187–196. Russo Cardona, Tommaso, and Virginia Volterra. 2007. Le Lingue dei Segni. Storia e Semiotica. Rome: Carocci Editore. Sacks, Harvey, Emanuel A. Schegloff, and Gail Jefferson. 1974. “A Simplest Systematics for the Organization of Turn-Taking for Conversation.” Language 50:696–735. Schönström, Krister. 2010. “Tvåspråkighet hos Döva Skolelever. Processbarhet i Svenska och Narrativ Struktur i Svenska och Svenskt Teckenspråk. (Bilingualism in School-Aged Deaf Pupils. Processability in Swedish and Narrative Structure in Swedish and Swedish Sign Language.).” PhD diss., Stockholms Universitet.

92

Chapter Four

Selinker, Larry. 1972. “Interlanguage.” International Review of Applied Linguistics 10:209–241. SignLEF project partnership. 2013. “SignLEF.” Accessed 14 March, 2013. www.signlef.aau.at. Spano, Ivano, Giuseppe Nuccetelli, and Maria Tagarelli De Monte. 2011. Inclusione Linguistica dei Sordi e Social Network: Costruire la Conoscenza nella Rete. Tecnologie Didattiche 52:28–35. Sutton, Valerie. 1974. “SignWriting.” Accessed 13 March, 2013. www. signwriting.org. Tomasuolo, Elena. 2006. “La Valutazione della Abilità Linguistiche in Bambini e Ragazzi Sordi.” PhD diss., Università degli Studi di Roma, La Sapienza. van der Haagen, Monique, and Pieter de Haan. 2011. “The Use of Sophisticated Language in Advanced Dutch EFL Writing: A Longitudinal Study.” Paper presented at the Writing Assessment in Higher Education: Making the Framework Work, Amsterdam, The Netherlands, October 26–28. Weir, Cyril J. 2005. Limitations of the Common European Framework for Developing Comparable Examinations and Tests. Language Testing 22:281–300.

CHAPTER FIVE USING CURRICULUM-BASED ASSESSMENT TO IDENTIFY YOUNG SECOND-LANGUAGE LEARNERS’ RISK FOR DELAYED SECOND LANGUAGE READING PROCESS MELODY KUNG, JILL FITZGERALD AND STEVEN J. AMENDUM

In the present chapter we report a United States study that addressed the value of curriculum-based reading assessments in identifying young Second-Language Learners (SLLs) who would benefit from accommodated reading instruction. The curriculum-based assessments departed from standardized testing procedures. We investigated whether SLLs who began first and second grades with different English-as-a-Second Language (ESL) word-reading abilities demonstrated different Instructional Reading Level (IRL) trajectories. Forty-two SLL first- and second-grade children were followed for two years. At the start, Oral-English Ability (the only standardized test in the study) and Word-Reading Ability assessments were employed. Instructional Reading Level (IRL) was assessed at the beginning, middle, and end of each of the two years. A two-level hierarchical linear model was employed, with repeated measures on IRL nested within students, initial Word-Reading Ability as a predictor, and initial Oral-English Ability as a control variable. IRL trajectories differed according to initial levels of Word-Reading Ability. The trajectory gap between initially lower and higher word-readers increased over time. The findings indicated that proficiency in young SLLs’ early word-reading is important for their reading development. Also, reliable and valid curriculum-based measures may be important early identification tools for teachers to use with SLLs with differential needs.

94

Chapter Five

1. Introduction Over the past three decades, the number of school-aged children in the United States who speak a language other than English at home increased approximately threefold from 4.7 to 11.2 million (U.S. Department of Education 2011). This increase presents a significant challenge to teachers. A widely accepted tenet is that reading is fundamental to all students’ academic progress and general well-being. However, on the most recent National Assessment of Educational Progress (2011), 70% of fourth-grade and 71% of eighth-grade Second-Language Learners (SLLs) scored below the basic level on English reading. Understanding SLLs’ English-reading abilities and documenting key factors involved in their reading growth are essential for enhancing classroom instruction and assessment. Most recently, a few large-scale United States studies using standardized assessments have revealed that oral language proficiency and socioeconomic status moderated SLLs’ overall growth in reading-achievement, oral reading fluency, and wordreading (Al Otaiba et al. 2009; Kieffer 2008, 2011, 2012; MancillaMartinez and Lesaux 2010; Nakamoto et al. 2008; Yesil-Dagli 2011). For instance, Kieffer (2011) found that SLLs who were initially fluent in oral English were able to catch up with their monolingual peers in reading by the end of first grade. Also, their ensuing trajectories through eighth grade were consistent with their peers’. The SLLs who were not initially fluent in oral English experienced a steeper initial rate of growth as compared to their monolingual peers’. However, on the whole, they did not reach their peers’ English-reading level by the end of eighth grade. Also, socioeconomic status (SES) was correlated with initial oral-English proficiency, and when SES was taken into account statistically, it reduced the impact of initial oral-English ability on SLLs’ reading progress. While there are significant educational implications of the large-scale SLL trajectory-tracking studies, little is known about SLLs’ progress on the kinds of everyday reading activities required of young SLLs. Furthermore, there is a dearth of research on the developmental trajectories of young SLLs who start schooling with differing secondlanguage reading abilities. At the same time, much is known about significant early contributing factors involved in young monolingual children’s reading progress. One key factor that has been identified in monolingual research is early word-reading ability. It is an ability that develops through a series of phases during which children learn to attach sounds to letters, identify rules for ways in which word patterns influence sounds in words, and learn written words as sight words (Ehri 1999).

Young Second-Language Learners’ Risk for Delayed L2 Reading Progress

95

Importantly, variation in early word-reading ability significantly explains monolingual-English children’s reading achievement from kindergarten through third grade (Pianta et al. 2008; Snow et al. 1998). If the same significance of initial word-reading pertains to SLLs, it would be important for early elementary teachers to identify SLLs at risk for delayed reading progress in order to intensify early word-reading instruction. An overlooked means of understanding SLLs’ early reading is examination of students’ progress using curriculum-based assessment. Curriculum-based measures are routinely used in United States primarygrade classrooms to facilitate enhanced instructional planning and monitor student progress (Flynn et al. 2011; Kim et al. 2010). Typically standardized in the way they are administered, examples include informal reading inventories designed to provide individual student instructional reading levels, oral reading fluency indicators, and ability to read words in isolation. Some have argued that widely-used standardized norm-referenced measures of reading may not be adequate to assess SLLs’ new-languagereading proficiency (e.g., García 1991). On the other hand, assessments more directly connected to their normal, day-to-day reading activity might better capture their abilities and progress, and represent a valid, efficient method of gauging SLLs’ abilities and progress (De Ramírez and Shapiro 2006). The study we undertook represented a modest effort to discern whether curriculum-based assessment might usefully inform us about the relationship between SLLs’ early word-reading ability and their reading progress. The research question was: is Initial Word-Reading Ability significantly related to Instructional Reading Level growth for young SLLs, using curriculum-based assessment?

2. Method 2.1 Design The current study draws on an existing database of 42 students from two cohorts in two schools from 2001–2003. The design of the study was a two-year cohort-sequential design. Students who started the study as first graders were followed into second grade, and students who started the study as second graders were followed into third grade. Upon entry to school, school officials administered an Oral-English assessment, yielding an Initial Oral-English Ability score for each student. At the beginning of the study, students’ Initial Word-Reading Ability was assessed. Students

96

Chapter Five

were assessed on Instructional Reading Level at the beginning, middle, and end of each of the two years. Analysis was conducted using a hierarchical linear modeling method.

2.2 Participants Forty-two SLLs from two schools in a southeastern state in the United States started the study as first or second graders. SLLs were students who first learned a language other than English in their home and community (and/or other country) and then were learning English as a new language (cf. Genesee et al. 2006). Twenty-five students started the study as first graders and 17 students started the study as second graders. Twenty-one students were male, and twenty-one were female. Thirty-four (81.0%) students were Latino; the ethnicities of the remaining eight (19.0%) students were unspecified. All participants in the study were enrolled in English-immersion classrooms with pullout English-as-a-second-language classes approximately twice a week for approximately 30 minutes each session. This is a common scenario in the United States for languageminority students.

2.3 Data Sources Two curriculum-based reading assessments and one standardized oralEnglish test were administered. On one reading assessment, students read increasingly difficult passages out loud from the Bader Reading and Language Inventory (Bader and Weisendanger 1994) while miscues were recorded (Clay 1993). On another reading assessment commonly used in primary-grade classrooms in the United States, students were given progressively more difficult sight word lists to read out loud (Barr et al. 1995). The standardized, norm-referenced, oral-language assessment, the IDEA Proficiency Tests (Dalton et al. 1991), was administered upon entry to school. Students were given stimuli such as pictures and simple questions to answer. The content, criterion, and construct validity of the reading assessments have not been evaluated psychometrically. However, they have face, ecological, curricular, and population validity. The assessments were similar to early reading assessments used in elementary classrooms, and assessed similar reading sub-skills, were aligned with curriculum objectives, and were tested on a sample that is representative of the population of SLLs in the United States.

Young Second-Language Learners’ Risk for Delayed L2 Reading Progress

97

2.4 Variables From the three data sources, two reading variables were created, Initial Word-Reading Ability and Instructional Reading Level, as well as an Initial Oral-English variable. Initial Word-Reading Ability was created by first obtaining the raw score sum of the number of words read correctly. Raw scores could potentially range from 0 to 220 and were then converted to percent correct. The interrater reliability estimate for ratings within five percentage points was .93. Instructional Reading Level was the highest level of passage that the student read with at least 90% word-reading accuracy. For example, a score of 0 meant that the student was unable to pass the lowest level, and a score of 1 meant that the student was at approximately the end-of-first-grade level. The interrater reliability estimate was .86 for complete agreement and .95 within one level. Initial Oral English Ability scores could range from 1 (non-English speaker) to 6 (fluent). Test authors reported split-half and test-retest reliability estimates were .82 and .83, respectively.

2.5 Analyses Analysis was two-level Hierarchical Linear Modeling (HLM) with repeated measures (time) (Level 1) nested within students (Level 2). The dependent variable was Instructional Reading Level growth across six time points. The independent variable was Initial Word-Reading score. The control variable was Initial Oral-English Ability. The analysis was a three-step procedure. First, we estimated an unconditional model, where repeated measures were included at Level 1 and the random intercept and slope were included at Level 2. The purpose of the unconditional model was to determine if initial word-reading levels and reading-growth rate varied among the students. Once we confirmed that there was student-level (Level 2) variance, we conducted a conditional model that included the Level-2 control variable, Initial Oral-English Ability, to determine the amount of variance explained by that variable. Finally, we conducted the full conditional model that included the Level-2 predictor, Initial Word-Reading Status, and the Level-2 control variable, Initial Oral-English Ability. Both the predictor and control variables were grand-mean centered for interpretation.

Chapter Five

98

3. Results 3.1 Preliminary analyses Preliminary analyses suggested that, as shown in Table 1, on average, students initially read correctly only about 20% of the words on the Initial Word-Reading Ability measure, and their Oral-English Ability was modest: 3.71. At each of the six time points for Instructional Reading Level, there was an increase over previous time points. On average, students started at an Instructional Reading Level of .26. This indicates that, taken as a whole, the beginning-first- and beginning-second-grade SLLs were reading on the pre-primer level typically for the beginning of first grade. At the end of the two years, the average Instructional Reading Level was high: 5.24. It is important to keep in mind that the Instructional Reading Level variable was based principally on oral reading of words in the context of the passages. Students could progress to higher passages based on their oral reading pronunciation. The rate of mean growth of Instructional Reading Level across the two years was distinctive. The growth across Year 1 was gradual, from .26, on average, to 1.21. Then during Year 2, Instructional Reading Level growth increased more rapidly, from an average of 1.79 to 5.24. Standard deviations at each time point increased, indicating a greater spread of students’ scores as time wore on. Instructional Reading Level Parameters M

Word Reading 20.2

OralEnglish 3.71

Beg Yr 1 .26

Mid Yr 1 .60

End Yr 1 1.21

Beg Yr 2 1.79

Mid Yr 2 3.55

End Yr 2 5.24

SD

30.44

1.70

.75

1.21

1.71

2.04

3.69

4.41

N

.42

.42

.41

.41

.36

.28

.27

.28

Table 1. Initial Word-Reading Ability, Oral-English Ability, Instructional Reading Level - Mean (Standard Derivation, N)1

1

An Instructional Reading Level score of “0” indicates that a student was unable to pass the lowest reading passage; a score of .50 indicates that the student achieved a primer level usually achieved around the middle of first grade; a score of “1.00” indicates that the student was at approximately the first grade level; etc. The Ns varied across the time points because not all students in the sample had data on all variables.

Young Second-Language Learners’ Risk for Delayed L2 Reading Progress

99

Table 2 shows the correlations between Initial Word-Reading, Initial Oral-English Ability, and the Instructional Reading Levels at all six time points. Initial Word-Reading and Initial Oral-English Ability had a moderate, positive correlation of .57. Initial Word-Reading and Instructional Reading Level were strongly correlated at the middle and end of Year 1. Though the correlations are still strong through Year 2, the strength of the relationship steadily decreased. This could be due to the type of texts the students were reading at the later time points. Early in the study, knowing sight words may have been sufficient support for students reading simpler texts. However, as the texts became more complex, knowing sight words may not have been enough; there could be other variables besides Initial Word-Reading that impacted their Instructional Reading Levels. The pattern of correlations for Initial Oral-English Ability and Instructional Reading Level appears similar to the pattern of correlations for Initial Word-Reading and Instructional Reading Level. Again, the correlations at all time points were moderate, positive relationships, but they decreased over the two years. Perhaps oracy played less of a role in reading as students achieved higher Instructional Reading Levels, and/or other variables may have impacted the Instructional Reading Level.

3.2 Instructional Reading Level growth From the unconditional model, the individual mean Initial WordReading (intercept; chi square value = 126.36, p < .01) and growth (slope; chi square value = 12.73, p < .01) varied significantly among the students. Therefore, conditional models were run to explain the significant variance. In the first conditional model, Initial Oral-English Ability was not significantly related to students’ initial Instructional Reading Level (intercept) or growth slope. The overall intercept was -.85, which was the predicted mean Instructional Reading Level score at the beginning of Year 1 for a student with average Initial Oral-English Ability, t(40) = -4.13, p < .01. The overall slope was 0.76, which was the predicted mean Instructional Reading Level growth slope for a student with average Initial Oral-English Ability, t(40) = 6.42, p < .01.

Chapter Five

100

English

.57 (.00, 42)

IRL Beg Yr 1

IRL Beg Yr 2

.76 (.00, 41)

.41 (.01, 41)

IRL Mid Yr 1

IRL End Yr 1

.92 (.00, 41)

.55 (.00, 52)

.75 (.00, 40)

IRL End Yr 1

IRL Mid Yr 1

.91 (.00, 36)

.56 (.00, 44)

.66 (.00, 36)

.89 (.00, 42)

IRL Beg Yr 2

IRL Beg Yr 1

.88 (.00, 24)

.58 (.00, 29)

.75 (.00, 24)

.80 (.00, 27)

.89 (.00, 28)

IRL Mid Yr 2

English

.85 (.00, 22)

.42 (.03, 27)

.78 (.00, 22)

.84 (.00, 25)

.69 (.00, 26)

.84 (.00, 27)

IRL End Yr 2

Word Read

.71 (.00, 24)

.29 (.13, 28)

.57 (.00, 24)

.68 (.00, 27)

.67 (.00, 27)

.77 (.00, 28)

IRL Mid Yr 2

.89 (.00, 26)

Table 2. Correlations with p level & n (in parentheses) between Initial Word-Reading, Initial Oral-English, and Instructional Reading Level (IRL)2

2

All correlations are significant at the .001 level (2-tailed) except correlations between English and IRL- Mid Yr 2 (significant at the .05 level) and English and IRL-End Yr 2 (insignificant at the .05 level).

Young Second-Language Learners’ Risk for Delayed L2 Reading Progress 101

For the final model, the outcome was Instructional Reading Level growth, the Level-2 predictor variable was Initial Word-Reading Ability, and the control variable was Oral-English Ability, with randomly varying intercepts and slopes for the predictor. As shown in Table 3, neither Initial Oral-English Ability nor Initial Word-Reading Ability significantly predicted Instructional Reading Level intercept. Also, Initial Oral-English Ability was not significantly related to the Instructional Reading Level growth slope. The overall intercept was -0.79, which was the mean predicted Instructional Reading Level score at the beginning of Year 1 for a student with average Initial Oral-English Ability and an average Initial Word-Reading Ability, t(39) = -3.52, p < .01. Coefficient

Standard Error

t-ratio

df

p-value

Intercept Initial Oral-English Initial Word-Reading

-0.79 0.09 0.00

0.22 0.16 0.01

-3.52 0.59 0.29

39 39 39

0.00 0.56 0.78

Slope Initial Oral-English Initial Word-Reading

0.71 -0.05 0.01

0.19 0.07 0.00

7.16 -0.74 4.00

39 39 39

.10). Secondary education teachers were not as competent as primary education teachers, to differentiate between NSLD and TSLLs, and they attributed them similar academic characteristics. A possible explanation, however, may lie on the fact that students’ differences may not be that evident in

Chapter Six

118

secondary education, due to reasons related to the time educators spend with their students, socioemotional factors, etc.

Listening Speaking Reading Writing Reasoning Math

Groups NSLD TSLL NSLD TSLL NSLD TSLL NSLD TSLL NSLD TSLL NSLD

Mean 5.62 6.42 4.88 6.13 4.25 5.86 3.53 5.28 4.06 5.80 3.84

SD 1.70 1.86 1.75 1.94 1.64 2.18 1.46 1.93 1.57 2.15 1.89

TSLL

5.72

2.21

t

Df

P

-1.693

864

.091

-2.543

864

.011

-3.486

863

.001

-4.255

863

.000

-3.950

864

.000

-3.533

860

.000

Table 2. Mean comparisons between the two groups in primary education

Listening Speaking Reading Writing Reasoning Math

Groups NSLD TSLL NSLD TSLL NSLD TSLL NSLD TSLL NSLD TSLL NSLD TSLL

Mean 5.49 5.18 4.75 4.57 4.45 4.06 3.67 3.56 4.17 4.29 4.24 3.98

SD 1.88 1.64 1.87 1.78 1.80 1.96 1.67 1.67 1.75 1.91 1.87 2.36

t

Df

P

.590

774

.555

.350

772

.727

.784

770

.433

.221

771

.825

-.263

762

.793

.329

752

.742

Table 3. Mean comparisons between two groups in secondary education

Screening for Specific Learning Disabilities in L2 Learners

119

3.3 Students’ characteristics differentiating NSLD from TSLL across educational levels The purpose of this section is to identify individual predictors of SpLD identification across educational levels. In order to achieve that, we implemented the Receiver Operating Characteristic Curves (ROC). ROC curve analysis provides indices of classification accuracy. By analysing the data from primary education teachers, results indicated that listeningrelated predictors of SpLD classification were related to word and sentence/question comprehension and comprehension of instructions. The former produced poor to fair classification rates ranging from 66% to 71%. The latter, related to instruction comprehension items, were poor predictors (i.e. correctly classified 65% of the cases). Regarding speaking, the items that contributed to correct classification of SpLD were related to word -repetition, -retrieval and -knowledge. These items were poor to fair predictors of SpLD identification (ranging from 68% to 72%). Furthermore, items related to spontaneous speaking and storytelling were poor to fair predictors of SpLD identification (ranging from 66% to 73%). At last, fair predictors were also items related to difficulty to recall common word/phrase sequences (correctly identified 74% of the student cases) and familiar information (correctly identified 70% of the participants). Reading and writing items were significant predictors of SpLD identification. Specifically, reading items related to decoding, fluency, reading comprehension and strategic reading were poor to fair predictors of SpLD group membership, ranging from 65% to 77%. The items that did not prove to be significant predictors of SpLD identification were awkward writing and little interest in writing (p > .10). Interestingly, motivational indicators did not emerge as significant predictors (i.e., interest). The sub-scales of reasoning and mathematics also were significant predictors of SpLD identification. All items of both reasoning and mathematics were fair to good predictors of SpLD group membership, ranging from 65% to 80%. This finding suggests that central to cognition variables such as reasoning and mathematics are core defining features of the disorder.

120

Chapter Six

3.4 Students’ characteristics differentiating TSLLs from SLLLD across educational levels ROC curves were estimated in order to distinguish those characteristics that predict SpLD identification among SLLs, at each educational level. Table 4 shows the items that were significant predictors of the SpLD classification in the SLL population for both primary and secondary education. The results are presented with regard to oral and written language and mathematics, since reasoning did not yield any significant results. Regarding listening, the items that significantly predicted SpLD identification across both educational levels were related to comprehension of oral language. The competence to comprehend words, directions, sentences or commands (as estimated by the educators) was a significant predictor of SpLD identification across both educational levels. All items were fair predictors of SpLD referrals (ranging from 70% to 72%). With regard to the ability of speaking, the skill to repeat a longer statement and the capacity to correctly provide information emerged as significant predictors of teacher referrals for LD. Furthermore, regarding reading, results showed that significant predictors of SpLD membership among SSLs for both primary and secondary educators were mostly related to reading comprehension. Specifically, the items that correctly identified 68% to 72% (fair predictors) of the participants, respectively, were related to students’ ability to organize and distinguish important information within a text. Results also showed that the ability of writing was more correctly evaluated by the teachers, than the other abilities, since five items were significant predictors of SpLD identification across both educational levels. Regarding spelling, the skill of intonation emerged as a significant predictor of SpLD identification (correctly identifying 74% of the participants). Good predictors were also items related to written expression. Particularly, teachers’ estimations of students’ ability to recall and organize ideas while writing and to edit their texts correctly identified 82% to 85% of the participants (salient predictors). Finally, regarding mathematics and reasoning, only one math item significantly predicted SpLD identification. This finding may relate to the fact that the item “the student faces difficulties with adjusting strategic thinking to the problem’s demands”, which correctly identified 87% of the SpLD cases, is closely related to the skill of reasoning (similar in content to mathematics).

Screening for Specific Learning Disabilities in L2 Learners

121

Secondary Education

Primary Education Difficulty with:

Listening word and direction comprehension sentence comprehension (in rapid rate) execution of multiple commands Speaking long sentence/phrase repetition providing information in the correct order Reading distinguishing important text information organizing the text’s information Writing intonation organizing ideas during expressive writing recalling ideas during expressive writing argumentative writing text editing Mathematics adjusting strategic thinking to the problem’s demands

AUC

SE

p

AUC

SE

p

.707

.088

.028

.735

.093

.030

.715

.086

.023

.729

.094

.035

.700

.085

.034

.758

.087

.017

AUC

SE

p

AUC

SE

p

.763

.079

.005

.751

.089

.020

.756

.077

.007

.751

.089

.020

AUC

SE

p

AUC

SE

p

.684

.105

.051

.736

.093

.032

.717

.092

.022

.767

.091

.015

AUC .744

SE .090

p .010

AUC .760

SE .091

p .018

.819

.071

.001

.774

.087

.012

.822

.064

.001

.800

.087

.006

.834 .851 AUC

.062 .068 SE

.000 .000 p

.716 .772 AUC

.098 .088 SE

.048 .013 p

.872

.066

.000

.833

.098

.023

Table 4. AUC for variables across educational levels for the TSLLs & SLLLD

3.5 Reading differences between NSLD, TSLL and SLLLD groups In order to examine whether (a) NSLDs are distinguishable from their TSLL peers and (b) TSLLs have a different reading status from the SLLLD, we analyzed the mean reading proficiency levels of these groups. The following Table 5 presents the mean scores and standard deviations of

Chapter Six

122

each group in the reading constructs: decoding words and pseudowords, fluency and comprehension.

NSLD TSLL SLLLD

M SD M SD M SD

Decoding words .67 .26 .75 .19 .66 .21

Decoding pseudowords .70 .31 .81 .18 .68 .26

Fluency 92.49 47.97 105.70 45.24 88.14 48.98

Reading Comprehension .47 .30 .77 .20 .49 .30

Table 5. Descriptive characteristics of each group for all reading variables Results revealed that the TSLL group outperformed both the NSLD and SLLLD groups in decoding words and reading comprehension. Specifically, TSLL group was significantly better than the NSLD group in word decoding (t = -2.501, p < .05) and in reading comprehension (t = 2.501, p < .001). The TSLL group also significantly outperformed the SLLLD group in word decoding (t = 3.163, p < .05) and in reading comprehension (t = 5.913, p < .001).

4. Discussion The aims of the current study were to investigate the proportional representation of SLLs in the SpLD diagnostic category. In particular, the study aims to highlight the students’ referral characteristics across educational levels, and to observe the differentiation parameters that lead teachers to different classification decisions (i.e., SpLD classification between TSLLs and NSLD as well as between the SLL groups), in primary and secondary education. Furthermore, a principal goal was to detect differences between all groups in reading. SLLs appeared to be unequally represented in SpLD diagnosis, a finding that extends the disproportional representation that Samson and Lesaux (2009) found by the third grade of education. In the current study, third to ninth grade SLLs’ skills were rated by their educators and the results revealed that significantly more SLL were classified as SpLD than their native speaking peers. Similar statistics apply in the study of Artiles and Trent (1994, 414), who also noted that “the larger the minority student population is in the

Screening for Specific Learning Disabilities in L2 Learners

123

school district, the greater the representation of students in special education classes”. Since Specific Learning Disabilities reflect an intrinsic to the person condition, we expected that they will exist in similar numbers to all learners, first or second language ones, although they may manifest themselves with different profiles. These potentially different manifestations make the identification of SpLD more difficult in the case of SLLs. Nevertheless, the need for identification is the same and pressing, in order to avoid discrimination and provide appropriate education. There is also need to protect the students’ self-esteem, which is often hampered when they do not know why they fail in learning (Schwarz 2000). Diagnosis of SpLD in SLLs using standardized testing has been quite controversial and ineffective, and, at the same time, screening for SpLD is made within the school context and by educators. So the issue of educators’ capability and means to correctly identify SLLs at risk for SpLD becomes of high significance (Sideridis et al. 2008). Are there any specific student behaviors that could confuse teachers and make them mistakenly identify SLLs for students with LD? Barrera (2006, 144) reported that “SLLs may appear to exhibit learning characteristics very similar to those of their peers with LD”. Are there any specific students’ behaviors that could lead to correct screening of SLLs at risk for SpLD by their teachers? Answers to these questions were the primary focus of the present chapter. Based on our analysis, it appears that teachers in primary and secondary education rely, in general, on the same student behaviors when referring a NS or a SLL, as being at risk for SpLD. This is especially true for the oral language skill and reading behaviors, suggesting that low oral language and reading skills of SLLs should not be treated as a sign of LD, as it could lead to over-identification of the former as having SpLD. Geva (2000, 6) also reported that professionals attribute difficulties in reading comprehension to poor oral language. Nevertheless, a closer examination of the data yielded by primary education teachers revealed SLLs were estimated to be more competent than the NSLD at skills related to instruction comprehension and reading comprehension. The low scores in auditory processing of SLLs that mislead teachers’ ratings may result from low proficiency in the language spoken at school and not from SpLD existence (Klingner et al. 2006, 111). It is obvious that a student with SpLD will face significant difficulties in areas related to reading, but will not show an overall language delay (Flanagan et al. 2006). It seems that an SLL profile of low language, and reading skills but of good writing, reasoning and math, leads us away from a SpLD referral. Everatt et al.

124

Chapter Six

(2000, 43) suggested that we should rely on age- and not language-related tasks in order to assess for SpLD. Could these skills be mathematics and reasoning? Further research is needed in order to answer this question. Thus, although SLLs may appear as at risk for SpLD a more analytic examination of their learning behavior profile reveals that TSLLs are much better in writing in comparison to the NSLD group. It seems that although TSLLs make many mistakes in writing, these mistakes are typical for someone who learns a second language and do not qualitatively resemble the mistakes of students with SpLD (i.e. Confusion with verb tenses, plurals, possessives, subject/verb agreement or wrong use of articles, Ferris 2002; Harper and de Jong 2004). Actually, Barrera (2006, 153) reported that SpLD new or second language learners were better in note-taking compared to their NS peers. Another study noted that older SLLs may perform better in written than oral language (Klingner et al. 2008). On the other hand, it has been reported that in Greek language wherein decoding is not especially difficult, students with SpLD face severe problems not in decoding but mostly in spelling and written production (Mouzaki 2010). Furthermore, the current study aims to draw attention on listening and reading comprehension difficulties, speaking and writing expression difficulties and intonation issues, since these behaviors discriminated effectively between SLLs and SLLLD. This pattern of classification does not seem to be different from that of the NS population. It is in line with the findings in the NS literature and the SpLD classification characteristics. Durgunoglu (2002, 189) highlighted those cognitive and linguistic processes that are cross-linguistically transferred. This issue raises the question of how these students are going to be fairly assessed and accurately classified. Geva (2000) suggested measuring and comparing oral language and written language proficiency in order to distinguish SLLLD. How can we implement Geva’s discrepancy model in order to identify SpLD? There seems to be a discrepancy between the psychologist’s assessments and school testing practices (Milnes and Cheng 2008, 50). An interesting finding of the current study was that primary education teachers were more capable of recognizing accurately SpLD characteristics than their secondary education colleagues. This may be due to the fact that they spend more time with their students and they are aware of their linguistic capacity and cognitive behaviors. Parallel research findings (Milnes and Cheng 2008, 50) suggest that secondary educators rate SLL students’ competence by the socioemotional characteristics they exhibit (e.g. motivation, effort, participation, etc.) rather than their actual

Screening for Specific Learning Disabilities in L2 Learners

125

cognitive skills. On the other hand, elementary school teachers rely on observations and informal measures in order to estimate their students’ abilities and progress. Nevertheless, emphasizing the usage of typical and standardized measures may lead to more accurate results. A finding of the current study that could contribute to a more accurate SpLD classification was that NSLD, when assessed by a reading test, were characterized by significant phonological and comprehension deficits manifested in reading. On the other hand, typical SLLs did not hold the same profile. Actually, the use of phonological skills in diagnosing dyslexia was founded by Frederickson and Frith (1998). It was clearly documented in that study that, at least the bilingual phonological-dyslexics can be identified through phonological tests, even in the second language. Second language learners, on the other hand, have experienced two different phonological systems. Thus, as they improve in literacy, they exhibit good phonological awareness and decoding skills (Geva 2000, 17). Most second language learners do not experience significant decoding problems, so in case these difficulties are documented, they can be viewed as signs of a more specific problem (Milton Keynes Council, 2004). Furthermore, the reading of words was less related to oral language proficiency (Durgunoglu et al. 1993). Geva (2000) found that phonological processing skills contributed more than language proficiency to decoding while the opposite was true for reading comprehension. Based on an integrative view of our data, we can conclude that in terms of reading comprehension competence, SLLs may be lower than the native speakers but significantly better than the NSLD. Therefore, teachers may be prone to refer them or simply think that these low readers are actually students with SpLD. It seems that the comprehension of texts in the second language is related to oral language proficiency, and word decoding skills. However, limited reading comprehension does not reside in poor vocabulary or grammatical difficulties (Geva 2000, 18). To conclude, a mere description of the proportion of SpLD non-native speaking students does not help educators early identify students with SpLD and intervene on a timely fashion (Samson and Lesaux 2009, 149). The questions that lead educators to accurately monitor students’ progress and decide whether or not to refer them for special educational services should be the ultimate goal of research in that field. Teachers need to avoid rushing towards referral of SLLs as students with SpLD, and become aware of the specific profiles of typical and SpLD SLLs. We suggest that teachers should remember that SLLs may have lower oral and reading behavior skills but they are often better in writing, reasoning and math, and this profile is what differentiates them from the SpLD

126

Chapter Six

population. We also suggest that fewer phonological errors in reading and text comprehension are also indicative of typical SLL development, and can assist with correct screening and appropriate SpLD referral rates for second language learners, especially within secondary education.

Works cited Artiles, Alfredo J., and S. C. Trent. 2000. “Representation of Culturally/Linguistically Diverse Students.” In Encyclopedia of Special Education, edited by C. R. Reynolds and E. Fletcher-Jantzen. Vol. 1, 2nd ed., 513–517. New York: John Wiley & Sons. August, Diane, Maria Carlo, Cheryl Dressler, and Catherine Snow. 2005. “The Critical Role of Vocabulary Development for English Language Learners.” Learning Disabilities Research & Practice 20:50–57 Barrera, Manuel. 2006. “Roles of Definitional and Assessment Models in the Identification of New or Second Language Learners of English for Special Education.” Journal of Learning Disabilities 39:142–156. Collins, R., and Lanthin D. Camblin. 1983. “The Politics and Science of Learning Disability Classification: Implications for Black Children.” Contemporary Education 54:113–118. Donovan, Suzanne, and Christopher T. Cross. 2002. Minority Students in Special and Gifted Education. Washington, DC: National Academy Press. Durgunoglu, Aydin Y. 2002. “Cross-Linguistic Transfer in Literacy Development and Implications for Language Learners.” Annals of Dyslexia 52:189–204. Durgunoglu, Aydin Y., William E. Nagy, and Barbara J. Hancin-Bhatt. 1993. “Cross Language Transfer of Phonological Awareness.” Journal of Educational Psychology 85:453–465. Everatt, John, Ian Smythe, Ewan Adams, and Dina Ocampo. 2000. “Dyslexia Screening Measures and Bilingualism.” Dyslexia 6:42–56. Everatt, John, Ian Smythe, Dina Ocampo, and Eva Gyarmathy. 2004. “Issues in the Assessment of Literacy-Related Difficulties across Language Backgrounds: A Cross-Linguistic Comparison.” Journal of Research in Reading 27:141–151. Ferris, Dana R. 2002. Treatment of Error in Second Language Student Writing. Ann Arbor: University of Michigan Press. Figueroa, Richard A., and Patricia Newsome. 2006. “The Diagnosis of LD in English Learners: Is it Nondiscriminatory?” Journal of Learning Disabilities 39:206–214.

Screening for Specific Learning Disabilities in L2 Learners

127

Flanagan, Dawn P., Samuel O. Ortiz, Vincent C. Alfonso, and Jennifer T. Mascolo. 2006. The Achievement Test Desk Reference: A Guide to Learning Disability Identification. NJ: Wiley. Frederickson, Nora, and Uta Frith. 1998. “Identifying Dyslexia in Bilingual Children: A Phonological Approach with Inner London Sylheti Speakers.” Dyslexia 4:119–131. Geva, Esther. 2000. “Issues in the Assessment of Reading Disabilities in L2 Children-Beliefs and Research Evidence.” Dyslexia 6:13–28. Guron, Louise Miller, and Ingvar Lundberg. 2003. “Identifying Dyslexia in Multilingual Students: Can Phonological Awareness Be Assessed in the Majority Language?” Journal of Research in Reading 26:69–82. Hammill, Don. 1995. The Learning Disability Diagnostic Inventory. Austin, ȉX: Pro-Ed. Hammill, Don D., and Brian R. Bryant, 1998. Learning Disabilities Diagnostic Inventory Examiner’s Manual. Austin, TX: ProEd. Harper, Candace, and Ester de Jong. 2004. “Misconceptions about Teaching English Language Learners.” Journal of Adolescent & Adult Literacy 48:152–162. Hoover, John J. 2008. “Data-driven Decision Making in a Multi-Tiered Model.” In Why Do English Language Learners Struggle with Reading: Language Acquisition or Learning Disabilities?, edited by J. K. Klingner, J. J. Hoover, and L. Baca, 75–91. Thousand Oaks, CA: Corwin Press. Kahn-Horwitz, Janina, Joseph Shimron, and Richard Sparks. 2006. “Weak and Strong Novice Readers of English as a Foreign Language: Effects of First Language and Socioeconomic Status.” Annals of Dyslexia 56: 161–185. Klingner, Janette K., Alfredo J. Artiles, and Laura Méndez Barletta. 2006. “English Language Learners Who Struggle with Reading: Language Acquisition or LD?” Journal of Learning Disabilities 39:108–128. Klingner, Janette K., Estella Almanza de Schonewise, Carmen de Onis, and Laura Méndez Barletta. 2008. Misconceptions about the Second Language Acquisition Process. Thousand Oaks, CA: Corwin. Liljequist, Laura, and Kimberly Renk. 2007. “The Relationship among Teachers’ Perceptions of Student Behaviour, Teachers’ Characteristics, and Ratings of Students’ Emotional and Behavioral Problems.” Educational Psychology 27:557–571. Limbos, Marjolaine M. and Esther Geva. 1991. “Accuracy of Teacher Assessments of Second-language Students at Risk for Reading Disability.” Journal of Learning Disabilities 31:136–151.

128

Chapter Six

Macnamara, John. 1966. Bilingualism and Primary Education. Edinburgh: Edinburgh University Press. Milnes, Terry, and Liying Cheng. 2008. “Teachers’ Assessment of ESL Students in Mainstream Classes: Challenges, Strategies, and Decision Making.” TESL Canada Journal 25:49–65. Milton Keynes Council. 2004. “Guidance on the Assessment of Pupils with English as an Additional Language Who May Have Special Educational Needs.” http://www.milton-keynes.gov.uk/emass/docu ments/guidance-eal-sen-28-09-2011.pdf. Mouzaki, Angeliki. 2010. “Spelling Disorders” (in Greek). In Spelling: Learning and Disorders, edited by Angeliki Mouzaki and Athanassios Protopapas, 223–236. Athens: Gutenberg. Ortiz, Alba. 1997. “Learning Disabilities Occurring Concomitantly with Linguistic Differences.” Journal of Learning Disabilities 30:321–332. Oswald, Donald P., Martha J. Coutinho, Al M. Best, and Nirbahay N. Singh. 1999. “Ethnic Representation in Special Education.” The Journal of Special Education 4: 194–206. Padeliadu, S., and Georgios D. Sideridis. 2008. Learning Disabilities Screening for Teachers. YPEPTH, EPEAEK. Padeliadu, Susana, Faye Antoniou, and Georgios D. Sideridis. 2008. “LD Screening by the Educators: An Application of Rasch Model for the Test’s Psychometric Analysis in Greek.” Psychology 15:329–266. Padeliadu, Susana, Georgios D. Sideridis, and Faye Antoniou (In press). Reading Test. Samson, Jennifer F., and Nonie K. Lesaux. 2009. “Language-Minority Learners in Special Education: Rates and Predictors of Identification for Services.” Journal of Learning Disabilities 42:148–162. Sideridis, Georgios D., Faye Antoniou, and Susana Padeliadu. 2008. “Teacher Biases in the Identification of Learning Disabilities: An Application of the Logistic Multilevel Model.” Learning Disability Quarterly 31:199–209. Sideridis, Georgios D., Paul L. Morgan, Georgios Botsas, Susana Padeliadu, and David Fuchs. 2006. “Predicting LD on the Basis of Motivation, Metacognition and Psychopathology: An ROC Analysis.” Journal of Learning Disabilities 39:215–229. Simos, Panagiotis G., Georgios D. Sideridis, Angeliki Mouzaki, Aspasia Chatzidaki, and Maria Tzevelekou. 2012. “Vocabulary Growth in Second Language among Immigrant School-Aged Children in Greece.” Applied Psycholinguistics 1:1–27. Schwarz, Robin. 2000. “Identifying and Helping Learning Disabled English as a Second Language (ESL) Students in a College Intensive

Screening for Specific Learning Disabilities in L2 Learners

129

English Programme.” In Multilingualism, Literacy and Dyslexia: A Challenge for Educators, edited by L. Peer and G. Reid. London: David Fulton Publishers, Inc. Wood, Jamie G., and Stephen L. Benton. 2005. “Attributional Responses to Students with Attention-Deficit-Hyperactivity Disorder Who Fail.” Teacher Education and Special Education 28:153–162.

PART II: TRAINING NEEDS AND ASSESSMENT PROCEDURES IN LANGUAGE TEACHING

CHAPTER SEVEN INVESTIGATION OF TRAINEE-TEACHER AWARENESS OF AT-RISK AND DYSLEXIC STUDENTS IN THE EFL CLASSROOM IN GERMANY LANA LOUMBOURDI AND YVONNE KARACIC

A number of studies have confirmed that 5 to 25% of students in German schools have difficulties with reading, writing and spelling—including dyslexia. Recent DESI and PISA1 studies have shown that students in the Hauptschule 2 display relatively poor reading and writing skills in both German and English; in all too many cases, these students are often barely capable of writing complete English sentences even after six years of English. These findings call for significant changes in teacher training. Foreign language teachers should be better prepared to cope with students’ literacy problems in English, including the knowledge of specific criteria and tasks involved in testing and grading these students. In our investigation, we interviewed teacher trainees at the beginning and the end of their studies in order to examine their knowledge of the issues surrounding dyslexia and at-risk students in the EFL classroom, as well as their familiarity with various methods of assessment. Finally, we looked at the courses offered for English teacher trainees in a number of German universities to determine the degree to which these issues are addressed in the university training programs.

1

DESI (International German English Student Assessment) is a longitudinal study that investigates language competences both in German and English in the German school system (ninth grade). PISA (Programme for International Student Assessment) is a world-wide study that aims at testing literacy. 2 The Hauptschule is a form of secondary school which offers lower secondary education and begins after four years of elementary school (Grundschule). Students in the Hauptschule graduate at the age of 15 or 16.

Chapter Seven

134

1. Introduction Our research was motivated by the concern expressed by both students and staff that English Foreign Language (EFL) teacher training programs in German universities largely overlook dyslexia and other language related difficulties (Christie 2012, 12). This leads to failure to prepare EFL teachers to adequately identify and diagnose reading, writing and orthographic difficulties. Neglecting the abovementioned issues also makes it nearly impossible to properly assess these students’ performance in the classroom and provide them with the support needed. Our claim is that foreign language teachers in Hesse—a state (Bundesstaat) in Western Germany—are trained to teach ideal pupils rather than those who struggle with foreign languages. We also believe that teacher trainees are unaware of the special performance criteria that must be applied when assessing such students. We chose to focus our research on teacher awareness not only of dyslexic students, but also of the so-called at-risk students, who likewise have reading, writing and orthographic difficulties, but are not officially diagnosed as dyslexic. There are three reasons for this decision: -

several studies have shown the difficulties involved in diagnosing and differentiating between these two groups of students; proper teaching methods can help all students with language learning difficulties; according to Hessian educational regulations, 3 the term reading and orthographic weakness or disorder (Lese- und Rechtschreibschwierigkeiten— LRS) is used to characterize all students with language-related difficulties who are entitled to additional support.

The Conference of the Ministers of Education and Cultural Affairs in all German states (KMK) has explicitly mandated additional support for low-achieving students. This includes educating and instructing teacher trainees to cope with these issues in all three school forms (Hauptschule, Realschule, Gymnasium—see explanation below). Moreover, teacher trainees must be competent in diagnosing language difficulties, promoting language skills and providing individual support in the integrative and inclusive classroom (Konferenz der Kultusminister der Länder in der Bundesrepublik Deutschland 2010, 9). Foreign language teachers, thus, have a special obligation to provide support to their students. The reasons 3

Hessian educational regulation: Verordnung zur Gestaltung des Schulverhältnisses Vom 19. August 2011. Gült Verz. Nr. 721.

Investigation of Trainee-Teacher Awareness

135

are that English is a compulsory subject beginning in the 3rd grade, and it is a basic requirement in the main written exams upon graduation from all three school forms, and consequently in most vocational training and higher education institutions. Teachers have a responsibility to diagnose and provide support to atrisk and dyslexic students. Therefore, preparatory instruction must be provided to teacher trainees during their university studies in order to equip them with the necessary skills to meet these challenges. Taking teacher trainees at the Goethe University in Frankfurt as our example, our study aims to determine whether these future teachers feel they have received an adequate education. Also, it aims to determine whether Hessian universities offer courses on assessing at-risk and dyslexic students as well as on the related topic of inclusion. A brief overview of recent developments in the research on at-risk and dyslexic students is presented next, before the analysis of the theoretical framework.

2. At-risk and dyslexic students in the EFL classroom Experts estimate that approximately 5 to 25 % of students have various forms of reading, writing and orthographic difficulties, including dyslexia (Gerlach 2010, 11). The fact that the percentages vary so widely is due to the number of conflicting methods of categorizing students with learning difficulties in general and dyslexia in particular (Schulte-Körne 2001, 1). As a result, many students who are diagnosed by some institutions as being dyslexic are not identified as having any acknowledged reading, writing or orthographic difficulty elsewhere. This is despite the fact that they might have similar problems with German and/or a foreign language. These problems can range from mild to severe, with reading and orthographic difficulties or disorders (LRS) often being defined as mild, and dyslexia (Legasthenie) as severe. In our chapter we will deal with two categories of low-proficiency students: 1. at-risk (LRS) students who show foreign language-learning difficulties, and 2. dyslexic students (who are officially diagnosed and receive special support in the official language, while showing difficulties in English).

2.1 At-risk students in the EFL classroom The primary reason for including at-risk foreign language learners in our study was to investigate teacher trainees’ awareness of specific language-related difficulties in English that are not classified as instances of dyslexia. Teacher trainees in Hesse are required to have a general

136

Chapter Seven

awareness of how to diagnose and deal with reading, writing and orthographic difficulties. The Ministers of Education and Cultural Affairs define these difficulties as “LRS”, rather than using the more narrow term dyslexia (German: Legasthenie). The term “LRS” encompasses social and familial factors, while Legasthenie represents the traditional medical explanation (Valtin 2006, 48). A comprehensive approach to dealing with reading, writing and orthographic difficulties can also be justified in view of the research on low-proficiency foreign language learners. Studies conducted by Ganschow and Sparks over the last ten years have shown that poorly performing foreign language learners usually also have similar deficits in their respective native language (Sparks, Ganschow et al. 1989; Ganschow, Sparks et al. 1998; Sparks, Humbach et al. 2008). In several empirical studies, they verified their earlier claim that L1 and L2 learning crucially depends on basic language learning mechanisms. They also found that problems with one language skill, e.g., phonological processing, will have a negative effect on both language (Sparks, Humbach et al. 2008, 30). Cummins makes the same point in his Linguistic Interdependence Hypothesis (1979) by indicating that L1 and L2 are interdependent and their learning relies on a common underlying proficiency. Moreover, the results of the DESI (International German English Student Assessment) study for the school form Hauptschule reveal that ninth graders (15–16 years of age) have low literacy skills in both German and English. According to DESI, this is due to low reading skills in German. It is not surprising, therefore, that a high percentage of German ninth-graders have reading problems in foreign languages and are unable to meet minimal curricular demands (DESI-Konsortium 2008, 137). Low literacy skills in turn affect their performance in a number of different subjects, causing particular difficulties when it comes to foreign languages. The fact that these poor results are concentrated in a single specific school form—Hauptschule—is connected to the tripartite school system in Germany. Here, children are taught together in elementary school; once they reach secondary school (at the age of 10–11), they are divided up according to their proficiency level into three different school forms: the Hauptschule, Realschule or Gymnasium. The Hauptschule and Realschule provide basic education, including a foreign language and preparation for vocational training. The Gymnasium includes secondary levels I and II and offers a more thorough education. After the Gymnasium students obtain a general certificate required in order to gain access to tertiary education,

Investigation of Trainee-Teacher Awareness

137

either in the university or the German Fachhochschulen (university of applied sciences). According to Bos (2004), the lack of adequate literacy skills constitutes the decisive hurdle for making the transition from the 4th grade to the next educational level. Therefore, most at-risk students can be found in the Hauptschule or in the lower-proficiency levels of comprehensive schools. Various empirical studies such as PISA (2001 and 2006), the Ländervergleichsstudie (Köller, Knigge et al. 2010) and DESI (DESIKonsortium, 2008) confirm that the problem of low literacy in the Hauptschule continues to exist.

2.2 Dyslexic students in the EFL classroom There has been a number of studies in the fields of medicine, psychology and education on the issue of dyslexia. Together they have produced a variety of different concepts and methods for identifying dyslexia. The research confirms that dyslexia is not a disorder with clearcut dimensions that can easily be demarcated. Here we will briefly mention two explanations in order to highlight the complexity of the issue: 1. The phonological deficit hypothesis theory of dyslexia is the leading description used by organisations such as the WHO (World Health Organisation 2010) and the International Dyslexia Association. The difficulties are associated with phonological deficits: Dyslexia is a specific learning disability that is neurobiological in origin. It is characterized by difficulties with accurate and/or fluent word recognition and by poor spelling and decoding abilities. These difficulties typically result from a deficit in the phonological component of language that is often unexpected in relation to other cognitive abilities and the provision of effective classroom instruction.4

2. An exclusionary definition is the intelligence discrepancy theory, which has its origins in the Isle of Wight study (Rutter and Yule, 1975). It is used as a diagnostic tool for differentiating between dyslexia and other reading and spelling difficulties (unlike Hesse, Bavaria differentiates between LRS and dyslexia based on the child’s IQ achievement score). However, empirical research has proven the intelligence discrepancy theory inaccurate (Stanovich 1994; Vellutino et al. 2004). According to the results of comparative research (Sparks, 4

Lyon, Shaywitz et al. 2003, 2.

138

Chapter Seven

Artzer and Javorsky 1998), there is no significant difference between learning-disabled students and at-risk foreign language learners not classified as learning-disabled. Reading and spelling difficulties typically encountered with dyslexic students are found across the whole IQ spectrum. Nevertheless, IQ tests are still employed in Germany to differentiate between students who are dyslexic and those who are merely poor readers. Elliot and Gibbs (2008, 488) criticize the use of the term dyslexia, and state that proper treatment has been hindered by the false dichotomy between dyslexic and non-dyslexic learners. This is made particularly clear by the fact that the same remedial methods used to support dyslexic students are also used to support low-proficiency foreign language learners. Several articles have stressed the benefits of multisensory structured language learning for at-risk and dyslexic students (Sparks and Artzer 1998; Sparks and Ganschow 1993). Crombie states in her paper that “for children who have language learning problems, dyslexic or not, many of the issues and possible solutions seem likely to be the same” (Crombie 2000, 113). So, it should be ensured that all children with literacy difficulties receive the help they need.

3. Hessian university course offerings for EFL teacher trainees Research studies have pointed to a high correlation between teacher awareness and student achievement (Kormos and Kontra 2008), making the quality of teacher training a crucial factor in improving language learning performance among low-proficiency students. To that end we reviewed teacher training programs at various Hessian universities, all of which are based upon the German tripartite school system: L3 teacher trainees study to become teachers for the Gymnasium, L2 for Hauptschule and Realschule, L1 for primary school and L5 for Förderschulen (schools for students with special need). There is an online review of the courses offered by Hessian universities to foreign language teachers. We took the winter semesters in 2012 and 2013 as our example. This review offers an obvious indication of whether the topics of inclusion, supporting and assessing dyslexia or at-risk students have in fact been incorporated into teacher training. Marburg’s teacher trainees receive foreign language teacher education for the Gymnasium with a special focus on teaching literature. Additional topics include a general introduction into foreign language teaching,

Investigation of Trainee-Teacher Awareness

139

language acquisition, teaching with new media, teaching skills and subskills, as well as different methods and approaches to teaching foreign languages. Very similar course descriptions can be found in Kassel, Giessen and Frankfurt. Even in Frankfurt and Giessen, the only universities that prepare teacher trainees in English for students with special needs, there are hardly any seminars that specifically address the issue of language-related learning difficulties or other learning disorders. The only seminar that covers the issue of at-risk students in the foreign language classroom is in Frankfurt and is offered by one of the authors of this chapter. We can conclude from the university curricula on teaching English as a foreign language that there are no specific training programs for teacher trainees on inclusion or on assessing students with dyslexia and other language related difficulties. However, these findings cannot be easily generalized, as we only took a relatively small sample size of Hessian university courses and did not include all such programs throughout Germany.

4. Research background The purpose of this study was to investigate the degree to which the English department at the Goethe University prepares its teacher trainees to properly teach and assess dyslexic and at-risk students. In the wake of the decision to apply the UN resolution on inclusion, much concern was expressed by trainee teachers regarding their readiness to fulfil the demands placed upon them. There was no concrete plan to include such courses in the curriculum or pre-service. Neither was there any plan to accommodate the currently matriculated students, nor in-service to help already employed teachers. Still, there were discussions within the student community, as future teachers would either have to conduct their own research and educate themselves, or disregard dyslexic and at-risk students’ needs altogether. This study will show that teachers finishing their studies feel entirely unprepared to cope with the demands of the new status quo. It also suggests that steps must be taken to make the corresponding training of English teachers for dealing with the problems of dyslexic and at-risk students mandatory. Student discussion of these problems led to the idea of conducting teacher interviews in order to investigate the matter in a more thorough and structured manner, following the suggestions of King and Horrocks (2010, 25):

Chapter Seven

140 -

Frame the research questions Choose the type of interview Define the sample and find participants Develop an interview guide

The study addressed the following research questions: Question 1: To what degree are teachers aware of the issues raised by inclusion, and of the implications of teaching and assessing students with reading and writing difficulties? Question 2: Does pre-service training meet teachers’ needs in terms of assessing students in the English language classroom? Do teachers feel ready to meet such a challenge?

We claim that teachers are largely unaware of what is involved in teaching and assessing dyslexic and at-risk students, and that they feel unprepared to undertake the challenge of inclusion.

5. The interviews After the interview protocol was designed (see Appendix 1), it was piloted on a group of teachers and redrafted according to their feedback and the framework of the research. Individual, face-to-face interviews using open-ended questions were conducted with 30 teachers, 25 at the end of their studies, and 5 at the beginning. Sampling was not difficult as both authors are currently employed at the university. To enhance diversity, teachers from all types of studies were recruited (elementary school, Gymnasium, Real/Hauptschule and Förderschule). The sample was otherwise random. The interviews lasted 10-15 minutes, were conducted in English, were audio-taped and later transcribed and analysed by both researchers and a third expert colleague to enhance validity and reliability. The interviewees participated voluntarily, having been informed first about the research focus and their involvement (King and Horrocks 2010). They were also informed about all the ethical considerations that ensure ethical practice was fulfilled, namely informed consent, no deception, right to withdraw, debriefing and confidentiality (Willig 2001). After the interviews were transcribed, they were manually analyzed, following the system of thematic analysis suggested by Langbridge (2004) and adapted by King and Horrocks (2010). According to this system, the first stage involved the descriptive coding of data. That is, the researchers read through the manuscripts and highlighted the material that connects the participants’ views with the questions of the

Investigation of Trainee-Teacher Awareness

141

research, by adding brief comments. The next stage was to use these comments to define descriptive codes, thus creating categories of different material related to the research questions. Upon re-reading the data, the researchers merged some of these codes together to create more general, inclusive categories of data (King and Horrocks 2010). Next, the interpretative coding followed, where researchers grouped participants’ comments that shared common meaning and created interpretative codes that seemed to capture that meaning. Finally, overarching themes were defined, by grouping together the interpretative codings, and matching them with key concepts in the research. The final overarching themes were the following: -

Bio-data. General overview of dyslexia/reading and writing difficulties (symptoms/implications). Inclusion in Hesse/consequences/problems. Knowledge on dyslexia, reading and spelling difficulties, and implications for teaching. Awareness of at-risk and dyslexic students, and assessment in the English language classroom. In-service training on dyslexia/reading and spelling difficulties and assessment.

6. Presentation and analysis of the results In this section, each of the overarching themes will be described and discussed “[...] referring to examples from the data and using direct quotes to help characterize the theme for readers” (King and Horrocks 2010, 165).

6.1 Bio-data Out of the 30 teachers, 23 were in their final semester, 2 had two more semesters to finish, 4 were in their first semester and 1 was in her second. The majority were Gymnasium teachers (18), followed by L2 teachers (8), L1 teachers (3) and finally, L5 teachers (1). The numbers are equivalent to the actual numbers of teacher studies. The majority of them were studying English and German (12), and the rest were studying English and Maths (5), or History (5), or Physical Education (4), or French (3), or Spanish (1). Only 6 had some actual teaching experience, excluding internships. Three are still currently employed as substitute teachers, and three worked

142

Chapter Seven

in the past as assistant teachers. None of them has worked full time, or more than a year.

6.2 General overview: dyslexia/reading and spelling difficulties Only five of the students were able to provide a correct and comprehensive definition of dyslexia and its symptoms. Only three were able to comment briefly on the implications that it has for the language classroom, and these were the experienced teachers. As T3 said: I know how dyslexia affects the classroom because I have seen it in two classes I teach. I cannot work on reading strategies the same way I do with other students, and writing tasks need to be organized differently for dyslexic students. My experienced colleagues helped me with this, especially the main teacher of the class. She told me she took a seminar in England on how to deal with dyslexia. Otherwise, I wouldn’t know what to do.

The rest of the teachers (25) had a false or misguided idea about what dyslexia is. Some thought it has to do with mispronouncing words and several knew that it involved difficulty in reading, but could not specify it more. Finally, 20 of the teachers admitted that although they heard about it, they did not know what it was. All beginner students had no idea what it was, and as T16 explained: I have heard the word before, but I have no idea what it is. I should though, shouldn’t I?

Asking those who knew more or less what dyslexia is, about the sources of their knowledge, one explained that it was briefly mentioned in their Erziehungswissenschaftliche und Psychologische Grundlagen von Bildung, Unterricht und Erziehung module (a module in psychology in teaching and education, in their main studies for all teachers). Others mentioned school internships, other students or the internet. Finally, a student mentioned briefly examining it on a seminar regarding errors and mistakes. No one could answer how to support at-risk and dyslexic students in English and what the possible reasons for the problems could be.

6.3 Inclusion in Hesse/consequences/problems With regards to the inclusion decision in Hesse, 25 of the 30 students have heard about it, but when asked to elaborate, only 10 had the right

Investigation of Trainee-Teacher Awareness

143

information. The rest claimed that they heard about the decision, but were unable to discuss the implications. Only 4 out of 10 pointed out correctly the problems arising in Hesse, concerning the placement of students with learning difficulties and disabilities. As T24 claims: The number of teachers was supposed to increase, so that students with difficulties can get additional help. What will happen now is that we will have larger classes of mixed abilities, including students with dyslexia and the same or smaller number of teachers, so that the costs are controlled. I wouldn’t know how to begin helping these different students on my own.

6.4 Knowledge on dyslexia, reading and spelling difficulties and implications for teaching All 30 students claimed that they do not feel ready to teach a classroom that includes dyslexic and at-risk students to different extents. Twentythree claimed that they felt totally unprepared, anxious, overwhelmed and bewildered. Seven, including the experienced teachers, claimed that with the support of their colleagues and proper in-service training, they would be able to manage, though they would feel very unsure in the beginning. Twenty felt that the assessment process would be much more demanding than the instruction process, since grading and evaluation is involved, which includes issues such as fairness or transparency. Finally, twenty eight did not feel ready to assess dyslexic and at-risk students, stating that they lack basic knowledge on dyslexia and reading and spelling difficulties, let alone suitable assessment techniques and tasks.

6.5 Awareness of at-risk and dyslexic students and assessment in the English Language classroom Twenty-six students mentioned that they would definitely give dyslexic and at-risk students more time to complete assessment tasks. Four students claimed that they would do nothing differently compared with the other students, either because they wanted to be fair, or because they had no idea how to approach assessment of dyslexic and at-risk students. Eleven teachers claimed that they would try to support students more during an assessment task or test. Two mentioned that they would try to separate different activities with different colours, a technique other colleagues advised them of. None of the students could be more specific on mentioning concrete tasks, form of instructions, working with computers, or how to differentiate assessment criteria. They all expressed their concern over the topic, with T4 and T13 stressing that:

Chapter Seven

144

[...] I really am at a loss here. I have no idea how to assess them. Maybe with multiple choice? But what if they mix up the choices and write b instead of a? If I am alone in the classroom I can’t give them more time, who will take care of the rest of the students? Maybe I can use dictation to assess them. [I will assess them the same way] because I don’t think it helps being pampered a lot.

Teacher 6 adds: [...] I was a substitute for three months and I thought this student, he or she has problems, but because of the time pressure I couldn’t really work on it and I had too little feedback from the other teachers [...] I tried to .. but I think they had a really bad teacher before.

Finally, several of the teachers mentioned that they would be able to deal with assessing dyslexic and at-risk students differently only if they had a teaching assistant in class. For question 12, the majority of the teachers (25) picked the red pen to correct the sentence, which should be avoided in general, with dyslexic students (Nijakowska, 2011). Only 5 picked a different colour. When asked afterwards, three of them mentioned that they were aware that red should be avoided. Twenty-four of the students crossed out the word and wrote it correctly, as Nijakowska (2011) suggests, and six, wrote either part of the words correct, or just mentioned that there were spelling mistakes.

6.6 In-service training on dyslexia/reading and spelling difficulties and assessment The students commented that they were generally more or less satisfied with the content of the courses offered. However, several felt that there was little connection with the classroom reality, as some of them are conducted by lecturers who have never taught and cannot provide accounts of what really goes on in the classroom. Some students also mentioned that several courses are designed for the perfect classroom, which is far from reality. L1, L2 and L5 students were also of the opinion that studies should be more differentiated and adjusted to the school context. None of the students ever took a course on dyslexia or other learning difficulties, or ever heard of a course being offered. Twenty-seven students stated that there is direct and imminent need for a seminar focusing on teaching and assessing students with dyslexia and reading and spelling

Investigation of Trainee-Teacher Awareness

145

difficulties. Furthermore, they said that they would all make an effort to definitely put it on their seminar schedule, and as T7 proposed: I think it should be obligatory, [to take such a seminar], otherwise you will not know if you are doing the right thing.

T27 added: University just deals only with the normal, the typical students, but we have all kinds of students in our class.

Concerning the content of such a course and its contribution to their future development, the majority of the teachers (26) claimed that it would definitely help them a lot in understanding their students better, and in developing correct and reliable ways to assess them in order to be fair to them and give them all the opportunities they deserve. They all understood that inclusion will bring about major changes. As for the content, students asked for a general overview of dyslexia and reading and spelling difficulties, and their causes and symptoms. Another aspect to be covered is how we can adapt our assessment in practical ways that can be applied in the reality of the schools. As T7 explains: I would also like to find out what is possible in schools and what is not, because, a lot of things that we learn at the University, they are not manageable in school.

7. Conclusions and implications The findings presented in this chapter indicate that there is almost no pre-service training on supporting and assessing dyslexic and at-risk students. The results of the interviews were not very encouraging. Teachers close to the end of their studies expressed inability to appropriately evaluate dyslexic and at-risk students in English, choose assessment tasks and organise an evaluation system that would allow them to keep the balance among students of all levels in the same class. They ask for more pre-service training to become available, but also for in-service modules to be established as a system of support throughout their career. They find the educational system not ready to adapt to the demands of the new situation and they claim that it will be the teachers who will ultimately pay the price. Finally, they believe that they would have to do research themselves and invest a lot of their personal time in order to feel more

146

Chapter Seven

prepared to assess dyslexic and at-risk students. However, they would still be unsure as to what will be the final assessment criteria that would determine students’ success in the classroom, or even university entrance. Another concern is whether the final assessment criteria should be the same with the rest of the students. In our view, the paramount focus of foreign language educators and researchers in universities in Germany should be on preparing teacher trainees for diagnosing, supporting and also assessing low-achieving students, or students with learning difficulties. Only then would inclusion become a reality.

Works cited Baumert, Jürgen, ed. 2001. PISA 2000. Basiskompetenzen von Schülerinnen und Schülern im Internationalen Vergleich. Opladen: Leske und Budrich. Bos, Wilfried, ed. 2004. IGLU. Einige Länder der Bundesrepublik Deutschland im Nationalen und Internationalen Vergleich. New York: Waxmann. Christie, Heike. 2012. Teaching Learners with Dyslexia in the EFL Classroom. Flensburg: Flensburg University Press. Crombie, Margaret A. 2000. “Dyslexia and the Learning of a Foreign Language in School: Where Are We Going?” Dyslexia 6/2:112–123. Cummins, Jim. 1979. “Linguistic Interdependence and Educational Development of Bilingual Children.” Review of Educational Research 49:222–251. Ganschow, Leonore, Richard L. Sparks, and James Javorsky. 1998. “Foreign Language Learning Difficulties: An Historical Perspective.” Journal of Learning Disabilities 31/3:248–258. Gerlach, David. 2010. Legasthenie und LRS im Englischunterricht: Theoretische Befunde und Praktische Einsichten. Münster: Waxmann Verlag. Hessisches Kultusministerium. 2011. Hessian educational regulation: Verordnung zur Gestaltung des Schulverhältnisses Vom 19. August 2011. Gült Verz. Nr. 721. Accessed August 2, 2012. http://verwaltung. hessen.de/irj/HKM_Internet King, Nigel, and Christine C. Horrocks. 2010. Interviews in Qualitative Research. London: SAGE. Klieme, Eckhard, Wolfgang Eichler, Andreas Helmke, Rainer H. Lehmann, Gunter Nold, Hans-Gunter Rolff, Konrad Schroder, Gunther Thome, and Heiner Willenberg (DESI-Konsortium). 2008. Unterricht

Investigation of Trainee-Teacher Awareness

147

und Kompetenzerwerb in Deutsch und Englisch: Ergebnisse der DESIStudie. Weinheim und Basel: Beltz Verlag. Köller, Olaf, Michel Knigge, and Tesch Bernd. 2010. Sprachliche Kompetenzen im Ländervergleich. Münster: Waxmann. Kormos, Judit, and Edit H. Kontra. 2008. “Hungarian Teachers’ Perception of Dyslexic Language Learners.” In Language Learners with Special Needs: An International Perspective, edited by Judit Kormos and Edit H. Kontra, 189–213. Clevedon: Multilingual Matters. Langridge, Daren. 2004. Introduction to Research Methods and Data Analysis in Psychology. Harlow: Prentice Hall. Lyon, Reid G., Sally E. Shaywitz, and Bennett A. Shaywitz. 2003. “A Definition of Dyslexia.” Annals of Dyslexia 53:1–14. Nijakowska, Joanna. 2011. “Correcting Dyslexic Spelling.” Accessed August 2, 2012. http://oupeltglobalblog.com/2011/03/22/correctingdyslexic-spelling/ Prenzel, Manfred, ed. 2008. PISA 2006. Die Kompetenzen der Jugendlichen im Dritten Ländervergleich. Münster: Waxmann Verlag. Rutter, Michael, and William Yule. 1975. “The Concept of Specific Reading Retardation.” Journal of Child Psychology and Psychiatry 16:181–197. Schulte-Körne, Gerd, ed. 2001. Lese-Rechtschreibstörung und Sprachwahrnehmung: Psychometrische und Neurophysiologische Untersuchungen zur Legasthenie. Münster: Waxmann. Sekretariat der Ständigen Konferenz der Kultusminister der Länder in der Bundesrepublik Deutschland ed. 2010. Förderstrategie für Leistungsschwächere Schülerinnen und Schüler. Köln: Carl Link. Sparks, Richard L., Leonore Ganschow, and Jane Pohlman. 1989. “Linguistic Coding Deficits in Foreign Language Learners.” Annals of Dyslexia 39:1–19. Sparks, Richard L., and Leonore Ganschow. 1993. “The Effects of Multisensory Structured Language Instruction on Native Language and Foreign Language Aptitude Skills of at-Risk High School Foreign Language Learners: a Replication and Follow-Up Study.” Annals of Dyslexia 43:194–216. Sparks, Richard L., Marjorie Artzer, and James Javorsky. 1998. “Students Classified as Learning Disabled and Non-Learning Disabled: Two Comparison Studies of Native Language Skill, Foreign Language Aptitude, and Foreign Language Proficiency.” Foreign Language Annals 31:1–17. Sparks, Richard L., and Marjorie Artzer. 1998. “Benefits of Multisensory Structured Language Instruction for at-Risk Foreign Language

148

Chapter Seven

Learners: a Comparison Study of High School Spanish Students.” Annals of Dyslexia 48:239–270. Sparks, Richard L., Nancy Humbach, and James Javorsky. 2008. “Individual and Longitudinal Differences Among High and LowAchieving, LD, and ADHD L2 Learners.” Learning and Individual Differences 18/1:29–43. Stanovich, Keith E. 1994. “Annotation: Does Dyslexia Exist?” Journal of Child Psychology and Psychiatry 35/4:579–595. Valtin, Renate. 2006. “Der Medizinische Ansatz der Legasthenie und Seine Problematik.” In Legasthenie: Lese- Rechtschreibstörungen oder Lese-Rechtschreibschwierigkeiten? Theoretische Konzepte und Praktische Erfahrungen mit Förderprogrammen, edited by B. Hoffmann and A. Sasse, 44–59. Berlin: Deutsche Gesellschaft für Lesen und Schreiben. Vellutino, Frank R., Jack Fletcher, Margaret J. Snowling, and Donna M. Scanlon. 2004. “Specific Reading Disability (Dyslexia): What Have We Learned in the Past Four Decades?” Journal of Child Psychology and Psychiatry 45/1:2–40. Willig, C. (2001). Introducing Qualitative Research in Psychology: Adventures in Theory and Method. London: Open University Press. World Health Organization. “International Statistical Classification of Diseases and Related Health Problems 10th Revision (ICD-10) Version for 2010.” Accessed November 12, 2012. http://apps.who.int/ classifications/icd10/browse/2010/en#/F81

Investigation of Trainee-Teacher Awareness

149

Appendix Dyslexia Interview Protocol 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17.

Which semester are you in? (Is this your last semester?) What are you studying? Have you ever worked as a teacher? Where, and for how long? Are you happy with the content of your studies and the courses offered by the department so far? Is there a special topic you would like to see in a course? Do you know what dyslexia is? (definition, special characteristics, diagnosis, problems in instruction, etc)? What are the implications in second language teaching? Where have you heard about it? What are your sources? Are you familiar with the inclusion decision in Hessen? What do you know about it? What is your opinion as a teacher? Do you feel ready to teach in an inclusion language classroom? Do you think you have the necessary knowledge? Do you feel ready to test and assess dyslexic students and students with reading and spelling difficulties in the foreign language classroom? How would you assess dyslexic students (in terms of assessment tasks, time management, assessment criteria, grading)? Would the above be different than the ones applied to the other students? Have you ever dealt with dyslexic students? How did you assess/grade them? Please correct the following writing sample of a dyslexic student. (Teachers are provided with a pencil case with several pens and a card that reads ‘we have to fait craim’). Have you ever had a course on dyslexia and/or other learning difficulties in the department? To your knowledge, were similar course offered? Do you think there is need for more courses on this topic? Would you take a course on instruction and assessment of dyslexic students if it was available? How important do you think that would be for your future development? Would you make an effort to put it in your studying schedule, or only if it fit this schedule? What would you like to hear about in such a course?

CHAPTER EIGHT DEVELOPING “COGNITIVE ASSESSMENTS FOR MULTILINGUAL LEARNERS” ANNE MARGARET SMITH

Providing appropriate exam access arrangements to Second Language Learners (SLLs) is increasingly complex in linguistically “super-diverse” communities (Vertovec 2007). In the case of Specific Learning Differences (SpLDs) such as dyslexia, it is not always apparent when accommodations in exams are required or justified. Generally, existing tools for identifying SpLDs are standardised on the performance of relatively linguistically and culturally homogeneous populations, rendering them invalid when used with SLLs. “Cognitive Assessments for Multilingual Learners” (CAML) is an assessment suite which avoids conflating language proficiency with cognitive function by removing the second language element from the assessment task, as far as possible. It is informed by theoretical and ethical constructs from studies of second language acquisition, language test design and the nature of SpLDs. This chapter describes the process of development, considering task design and trialling as well as practical constraints on assessing SLLs in contexts where resources and funding are limited. The inevitable tensions between reliability and validity, and in removing cultural bias from assessment are discussed. Through further trialling, CAML continues to develop, and comparison with existing standardised assessment materials leads to applicability to a wider SLL population. .

1. Introduction Around the world, formerly monolingual, homogeneous communities are becoming more diverse as migration patterns change and people become more mobile. Previously multilingual and multicultural communities are now becoming “superdiverse” (Vertovec 2007) with complex and fluid patterns of population movement, and alongside that, of linguistic change (Blommaert and Rampton 2011). Any individual’s

152

Chapter Eight

linguistic repertoire may consist of languages used in various domains with varying degrees of proficiency. No assumptions should be made as to the degree to which a person is comfortable or proficient in communicating in any given language, based solely on his/her ethnicity, nationality or current geographic location. This greater degree of diversity is reflected in language classrooms around the world. Experienced teachers know that learners progress at different rates, and find ways to facilitate differentiated language development. Learners from different educational and linguistic backgrounds may have disadvantages in one area of the target language or another, and this can be taken into account once these differences are understood. Occasionally, though, a student does not progress as expected, even taking into account these known factors. One of the challenges facing teachers of SLLs is that of determining the cause of the observed difficulties. In some cases, emotional or motivational issues may be identified that could account for the lack of expected progress. Otherwise, individual cognitive differences (such as dyslexia or other SpLDs) may need to be investigated. However, this is not straightforward, because of the many similarities between the manifestations of SpLDs and the kind of difficulties commonly resulting from the typical language learning process (Kormos and Smith 2012). The European Dyslexia Association (2013) defines dyslexia as: [...] a difference in acquiring reading, spelling and writing skills, that is neurological in origin. The cognitive difficulties that cause these differences can also affect organisational skills, calculation abilities etc. It may be caused by a combination of difficulties in phonological processing, working memory, rapid naming, sequencing and the automaticity of basic skills.1

The overlaps between first language development (in people with SpLDs) and second language development (in most people) may easily be seen in the development of literacy practices. For example, dyslexic learners and SLLs alike may have difficulty decoding phonetically irregular words, spelling accurately and inferring meaning from metaphorical language. Other characteristics of SpLDs will be discussed in Section 3 in relation to the tasks analysed there. In many countries, tests are available which are designed to identify unusual cognitive profiles and thus diagnose dyslexia. These tests focus on certain aspects of cognitive function, typically memory, phonological 1

European Dyslexia Association 2013.

Developing “Cognitive Assessments for Multilingual Learners”

153

awareness, speed of processing, literacy development and oral language use. Tests of general ability are usually also administered. These tests are standardised on the local population and therefore reflect typical developmental patterns in the majority language. However, Elbro et al. (2012) note that in multilingual contexts, the standardised tests tend to favour speakers of one language over others. For learners who do not have the majority language as a first language, these tests norms are therefore not valid, and, moreover, are likely to yield positive identifications of SpLDs. Some of these results will inevitably be false positives, which is, at best, unhelpful for the learners concerned and extremely upsetting. In practice, at least in the UK and some other countries where English is an official language, SLLs are often not formally assessed, as assessors (perhaps rightly) feel that they do not have the appropriate tools to do so (Geva 2000). This means, however, that their needs are not properly identified and addressed, perpetuating their lack of progress. This clear lack of equality in the way that SLLs in the UK (and elsewhere) are treated is surely unacceptable. The UK Equality Act (2010) should prevent such injustices from recurring. However, in practical terms it is ineffective because appropriate alternative assessment materials for a diverse and multilingual population are missing from the resources available to most dyslexia assessors. These anomalies in the UK system led to the development of the assessment suite: Cognitive Assessments for Multilingual Learners (CAML). It was first developed within a college of Further Education in the north of England (an institute offering courses at post-compulsory but pre-university level), which had a cohort of English language learners around 300 strong each year, from approximately 40 different language backgrounds.

2. Principles underpinning CAML During the development process, three main principles were established, which underpin the assessment suite. Firstly, for validity to be upheld, the assessment tasks should be equally accessible to all learners, regardless of their level of English language proficiency. The tasks should investigate different areas of cognitive function, not linguistic ability. Furthermore, areas of strength should be identified as well as areas of difficulty. The purpose of the assessment should be not only to confirm or disprove the presence of SpLDs, but also to inform recommendations for appropriate teaching strategies that would benefit the learner. Secondly, a holistic, qualitative approach was preferred over a purely quantitative, statistical model. For one thing, the CAML tasks are designed

154

Chapter Eight

to uncover coping and working strategies that quantitative assessments cannot reveal (Klein 2000). Moreover, the diversity of the population under consideration precludes standardisation of the scores. Even if sufficient numbers of learners representing speakers of all the world’s languages could be gathered and assessed, other factors still need to be taken into account. Such factors are (lack of) educational opportunity, and most importantly, linguistic repertoire. Being bilingual from birth is a different experience from becoming bilingual later in childhood (de Houwer 2009) whether through migration or education. Even bilinguals from childhood present differences ranging from growing up in a multilingual environment to growing up in a multilingual household within a largely monolingual environment. A difference in phonological development, which may manifest as reduced accuracy in differentiating between phonemes and/or as a slower speed of phonological processing, is often considered one of the key indicators of dyslexia. Therefore, tests that evaluate this aspect form a core component of dyslexia assessments in the UK. The effects that different linguistic environments may produce on the phonological development of learners (whether or not they have SpLDs) have not been well documented. Deponio et al. (2000) suggest that becoming bilingual may have a positive effect on the phonological awareness and processing skills of an individual. However, it is not clear if this would mitigate or indeed override any underlying phonological weakness, such as is characteristic of people with dyslexia. Tests normed on a monolingual population do not take this into account. The third underpinning principle is that CAML should be accessible to language teachers as well as dyslexia assessors. Classroom teachers are best-placed to observe and gather important background data that can inform the eventual conclusion of the assessment. Learners often develop good trusting relationships with their teachers, and may be more willing to disclose vital pieces of information to them than to an unknown assessor. Furthermore, language teachers often have at their disposal a wide range of strategies to allow them to communicate effectively with people with whom they do not share a first language. Equally, dyslexia assessors and specialist teachers are adept at interpreting data relating to cognitive function, and can offer insights as to whether a particular observed behaviour may be significant or not in relation to possible SpLDs. CAML therefore advocates a collaborative assessment process which draws on the expertise of both language teachers and dyslexia assessors (acknowledging that the two groups are not mutually exclusive!).

Developing “Cognitive Assessments for Multilingual Learners”

155

3. The development process 3.1 An overview of CAML CAML consists of four main areas of investigation in line with the requirements of UK assessment guidelines (Great Britain, Department for Education and Skills 2005). Such areas are a holistic overview, literacy practices, short term and working memory, and other areas of cognitive function, such as phonological awareness and speed of processing (see Appendix A: Overview of CAML activities). The first of these is perhaps the most important. Without a thorough knowledge of the background of the person being assessed, it is impossible to rule out other causes of difficulty with studying, and thus determine that SpLDs are the most probable cause (Waine and Kime 2005). In order to gain a holistic overview it is recommended that teachers and assessors start by observing the learner carefully for a short time. To bring a degree of objectivity into the observations, CAML offers an observation schedule which can be kept for a few weeks, or a couple of months, on which any unusual behaviours that may be indicative of SpLD can be recorded (see Appendix B: Example observation schedule). Teachers may notice a student having difficulty with co-ordination or orientation (e.g. managing his/her workspace, navigating the classroom) or issues with concentration or focus, personal organisation (bringing the right equipment to class, at the right time) or speed of processing (perhaps needing a lot longer than peers to complete relatively straightforward tasks, like copying from the board). All staff members who come into contact with the learner can contribute to this. At the end of the agreed observation period, this can also form a starting point for a conversation with the learner regarding any indicative behaviour, in order to establish how self-aware the learner is about the difficulties s/he experiences. This is also the time to find out as much as possible about the learner’s previous educational background, family situation, health and general coping strategies that may already be in place (for example, to support a weak memory). After these two steps, a decision needs to be made as to whether the difficulties observed can be accounted for in any other way, or whether further investigation of possible SpLDs is warranted. If further assessment is required, then exploring the learner’s literacy practices is recommended. This should be done using the learner’s first language wherever possible. The scanning task in the CAML suite (described below) does not require the learner to have developed any literacy practices at all, but assesses the underpinning

156

Chapter Eight

sub-skills of reading (tracking, visual discrimination, pattern matching etc.). Since memory is implicated in many SpLDs in different ways (Grant 2010), it is important to assess both short-term memory (the ability to hold a small amount of information in mind for a short time) and working memory (the ability to hold a small amount of information in mind while manipulating that information in some way). CAML offers tasks that assess both auditory and visual memory, since it is often the case that while a learner may experience difficulties in one of these aspects, the other may be an area of strength. As noted above, assessment in an educational context ought to be about determining possible fruitful ways of studying, as well as establishing where the weaknesses are. The final section of CAML seeks to evaluate the learner’s speed of processing and ability to perceive and manipulate phonological material, as difficulties with this are also typically associated with dyslexia (Nijakowska 2010), and other SpLDs. This is perhaps the most problematic area of assessing SLLs because assessors try to evaluate a phonological system which they do not share. However, a number of tasks are suggested in the CAML suite that assessors can choose from, to suit the students they are assessing. Some allow the student to work in his/her first language, thereby possibly compromising the rater reliability but certainly increasing the construct validity by assessing phonological development in the first language. Other tasks ask the student to work in English, either with a subset of words that s/he determines, or on tasks in which the meaning is not important and accuracy of production is not assessed. In total there are currently 23 tasks suggested in the CAML suite but, in this chapter, the process of developing the three tasks that were the first to be used will be discussed in detail: writing in the first language (First Language Writing), the non-language-specific reading task (Scanning Task), and a test of processing speed (Rapid Automatic Naming).

3.2 First Language Writing Most professionals, when assessing a learner for SpLDs, will ask him/her to write a sample of free writing within a given time (typically 10 minutes). This enables the assessment of several key aspects of cognitive development, assuming that the person being assessed has followed a typical educational path in his/her first language. The amount s/he can produce in the given time offers an insight into the speed of processing (how quickly s/he can think of ideas and convert them into text). The

Developing “Cognitive Assessments for Multilingual Learners”

157

legibility and regularity of the handwriting gives information about the degree of fine motor control that the writer can exert. The accuracy of spelling and sentence structure is indicative of the writer’s phonological awareness, short term memory and sequencing skills (i.e. the ability to remember a sequence and to sort data according to a known sequence, e.g. the alphabet). There are norms available for how much an Englishspeaking learner at different stages of education might be expected to produce (e.g. 16 words per minute at the end of compulsory education). No doubt, they exist for other languages as well, but they are not currently available in all languages. Asking SLLs to write in English is not a valid task for assessing these aspects of cognitive function. They are likely to write a lot less than a person with English as a first language at a similar stage of education, and accuracy is likely to be a lot lower. If they also write in a second script, features are likely to emerge which may suggest lack of fine motor control, but may actually indicate interference from the first language script (Sassoon 1995). Therefore, it seemed sensible to adapt the task and ask SLLs who are being assessed for SpLDs to write in their first language. In evaluating the writing, there are a number of challenges for the assessor, of course, who may well not be able to read all of the languages that are presented in this way. However, there is still a lot that can be deduced from the script. The response to the task can be judged as enthusiastic or reluctant. The assessor can look at how much has been written in the time allowed, and whether the character or letter formation seems regular throughout with regard to size, orientation and spacing. Most importantly, the assessor can evaluate how the writer approached the task in terms of thinking or planning time compared to actual writing time, and self-correction or editing strategies. These qualitative data are crucial in developing the overall picture of the person being assessed. Gauging the accuracy and complexity of ideas is something that most assessors would not be able to do for all scripts. One option may be to find a competent person who does read the language of the person being assessed and ask him/her to evaluate it. On the other hand, this may be problematic, as s/he may not be trained in identifying the key features of SpLDs in that language, and the assessor may not have any insights into what to direct them to look for (i.e. the errors typically made by dyslexic writers of that language). To allow a comparison of length of scripts within language groups, a large sample was required to determine what might be a typical amount for adult writers using different languages. Therefore, for 3 consecutive years, all SLLs in the college where CAML originated were asked to

158

Chapter Eight

complete this task and this enabled a database to be established. Students who attended the college for more than one year were not asked to contribute a second script. As well as SLLs in this college, SLLs from local universities, neighbouring colleges and private language schools contributed samples to the database, ensuring that the full range of educational experience and general ability levels are represented. The database continues to grow and other institutions contribute their data in return for the opportunity to compare their students to others from the same language background. Quantifying the amount of text can raise challenges for reliability. Even the apparently simple task of counting words is not straightforward. Counting lines might seem more reliable, but because the size of script can vary considerably, this also needs to be taken into account. Guidelines were produced for tutors to follow, when they collect samples for this database, to ensure rater reliability. The guidelines indicate what would be considered good practice in administering the task, and set out procedures for counting the words or lines produced (e.g. counting hyphenated words as one item). Tutors are advised to ask for further guidance, if desired. Samples are compared only to other samples written in the same language. Where there are not enough samples of a language to provide a meaningful comparison, related languages are considered, as well. The figures thus gained are not precisely standardised, but rather offer parameters to assessors within which they can compare the performance of the people they assess. In addition to these benchmark figures, the qualitative data gathered by observing the student writing are important in enhancing the validity of the task. They are extremely useful in contributing to a professional judgment about whether a student has SpLDs or not. Language and cultural differences are built into the quantification process, and when interpreting the data each learner’s individual educational background is also taken into account.

3.3 The Scanning Task As well as assessing writing, SpLD assessments seek to ascertain how well learners can read (in their first language). Tests typically include tasks investigating decoding strategies, the range of sight vocabulary and comprehension. To do this with a multilingual cohort without simultaneously assessing their English language proficiency seemed problematic. Finding equivalent texts in all the possible languages of the world would clearly be impractical. Instead, it was decided to devise a task, which would not be language specific, to assess some of the

Developing “Cognitive Assessments for Multilingual Learners”

159

underpinning sub-skills that are needed for reading. The sub-skills to be focussed on were determined to be: the ability to track lines of text; visual discrimination of symbols; pattern recognition (including sequencing); short-term visual memory; speed of visual processing. Higher order skills such as comprehension and inference were not included, as they seem to depend more on linguistic and cultural proficiency (see, for example, Erten and Razi 2009). The task was conceived as a word search activity, such that students would be asked to find words within a text by scanning it as quickly as they could. This echoes the kind of strategy needed to tackle the type of comprehension questions that SLLs may well be familiar with. A simple text was written and 10 target words were identified that appeared only once, and ranged from 2 to 9 letters long. They were evenly distributed throughout the text, appearing at either end and in the middle of lines. A shorter practice text was also devised which contained 3 example target words, from 1 to 4 letters long (see Appendix C: Practice phase of the Scanning Task). The texts were then converted into the Wingdings font, so that the letters became symbols that were not specific to any script. In administering the task, students are presented with the practice text and are asked to find the 3 example words. This is an opportunity to clarify that a word is separated from other words by space and each target word appears only once in the text. Once this is understood, the students are given the longer text with 10 target words to find. They are timed and observed as they complete the task, providing both quantitative and qualitative data. The time taken to find the 10 items is recorded, and any errors made (or items not found) incur a 10 second penalty. The total time (in seconds) is then divided by the number of items correctly identified, giving a final score. The task was trialled on volunteers at the college, who included SLLs, other English-speaking students, and colleagues of the task designer as well as people outside the college environment. In all, 98 samples were collected at this trial stage: 51 from people known or considered to have a SpLD and 47 from people thought not to have a SpLD. An overall average range was generated by calculating the mean of the 98 sample performances, and allowing 1 standard deviation either side (mean = 33.5; standard deviation = 18.4; average range = 15–52). This average range is quite wide, as the population that performed the task is extremely diverse in terms of age (range from 16–50+), language background (19 different first languages) and educational background (from those with no formal education to holders of higher degrees). The average range is not an absolute limit, however, and should be considered in the light of what is

160

Chapter Eight

known about the person who has completed the task. For example, a 25year old graduate who scores well below the mean may be considered more at risk of having difficulties in the cognitive functions assessed in this task than a 50-year old person who had little formal education and whose score is slightly lower. Observation reveals a lot about the strategies a learner has available for coping with text. Some students will track along lines with a pen while marking the side of the line with a finger. Others are systematic in marking off the items found, or the items left for later. By discussing strategies with the learner afterwards, it is possible to gain even more insights into the approaches taken, and thereby the student’s own perceptions of his/her strengths and weaknesses. For example, some students need to assign verbal labels to the symbols to help them remember them, aware that their visual memories alone will not be reliable enough. Some students report holding strings of three or more symbols in mind as they search. Others say that one or two was all they could remember, and that they had to check frequently to make sure they were searching for the right string of symbols. Close examination of any errors made can shed light on the possible reasons for them, for example, whether they stem from misidentifying symbols or forgetting sequences. No cultural or linguistic bias was detected in the trial data. Both the top ten and bottom ten scores in the sample data included men and women from at least five widely varying language backgrounds (Asian and European). One factor contributing to this is that scanning along the lines can be done left to right or right to left, or a combination of both, as candidates feel most comfortable. It has been noted, however, that those who complete the task most quickly do not scan systematically line by line. They take a more holistic approach, and this suggests that visual processing is a significant strength in these people. In the top ten performers on this trial, 2 had been previously identified with SpLDs, and of the bottom ten, 6. Thus, in itself it is not a definitive test of SpLDs—no single task can be, as the characteristics of SpLDs are so diverse. It does, however, provide a very helpful piece of the jigsaw when putting together a person’s cognitive profile. Since the task draws on several areas of cognitive function, it needs to be used in conjunction with other tasks to determine what the specific cause of any difficulty might be. Thus, the construct validity may be thought to be somewhat diluted, as it seeks to evaluate several different things at once. However, because it mimics the kind of tasks that SLLs often have to undertake, it can be considered to have a high degree of face validity. Marking can be done using a transparent answer sheet that fits

Developing “Cognitive Assessments for Multilingual Learners”

161

over the test paper and highlights the correct items; this contributes to good rater reliability. The overall reliability of the test depends on such factors as how good the assessor is at timing the student, and external factors such as the environment, and the degree to which the student is motivated or experienced in doing similar tasks. However, in practice it has proved to be a very useful task as it clearly distinguishes students who would benefit from extra time in exams from those who would not.

3.4 Rapid Automatic Naming (RAN) A task requiring candidates to retrieve words quickly is common to many assessment suites, aiming to assess speed of processing. Learners are typically asked to look at 6 or 8 pictures of familiar objects and identify what they would call them. Then they are presented with a page of the images repeated several times in a random order. They are asked to name them as fast as they can. They are timed and observed while doing so. The RAN tools available in other commonly-used, standardised assessment suites were problematic in a number of ways, so it was necessary to adapt them. Firstly, sometimes learners would not know the appropriate English word for the items shown. So it became a vocabulary learning task, rather than an evaluation of rapid retrieval. In some versions, too, the images provided (typically black and white line drawings, simplified for a younger age group than the college students being assessed) were stylised to the extent that they became unclear. So the learners also had the cognitive challenge of remembering that what to them looked like a boat was actually supposed to be labelled as leaf. To look at a boat and say leaf is counter-intuitive, and to do that in a second language is an additional cognitive step that probably had an impact on the overall performance time. Instead, the CAML version of this task uses colour photos of everyday objects (reproduced here in black and white only—see Appendix D: Practice phase of the Rapid Automatic Naming) which the learners are likely to recognise. Learners are asked to name them in their first language. The assessor may not necessarily understand the language being used, but is able to pick up on the patterns that recur, and to monitor if the learner is hesitating or self-correcting. Some concerns have been raised that learners may not be naming the items, but simply making random noises as fast as possible. However, this seems unlikely, as it would surely be more difficult to think of different noises to make, than to just name an item illustrated. If this is thought to be a concern, the accuracy of the performance can be checked by recording it and playing it back later,

162

Chapter Eight

either to the learner to self-check, or to another person who speaks the same language. This task was trialled amongst learners at the college and some guidelines were established as to how long it might take to name all the items. Because of the difference in the length of words in different languages, it was felt that some learners may be at a disadvantage in terms of speed of performance. However, when an initial sample was scrutinised using software to measure the relative time spent uttering compared to the time spent silent (i.e. thinking between items) it emerged that it was the silences that were significant in contributing to a slower overall performance, rather than the length of utterances. The task is quick to administer, and can be considered to be highly valid, as it allows the evaluation of rapid retrieval in the candidate’s first language, rather than through the medium of a second language. The reliability of the marking may be considered lower, as accuracy is harder to gauge, unless the performance is recorded and checked later. However, the qualitative aspects of the task are extremely revealing, as the assessor can observe whether the learner needs to track the items with a finger, or self-correct or backtrack. The level of anxiety in performing the task can also be significant, with some students reporting afterwards that they felt under pressure and that their minds went blank. This suggests awareness, perhaps from previous experience, that processing visual material and retrieving common words rapidly is not easy for them.

4. Conclusion The development of these first three tasks showed that it was possible to adapt and create assessment tools that would be appropriate and effective in identifying cognitive differences in SLLs that may be indicative of SpLDs. From those beginnings, the CAML suite has grown to cover further elements of literacy, auditory and visual memory (short term and working memory) and aspects of phonological awareness and speed of processing. In 2012/13, more data are being collected for these later additions, in order to refine the benchmark figures provided. Feedback from professionals who are using the suite allows the continuous improvement of the tasks to make them as user-friendly as possible. Ongoing research being conducted in 2012/13 compares the CAML suite with standardised tools, to see how multilingual learners perform on them. One aim is to see if it is possible to establish an English language proficiency threshold at which it makes no difference whether the tasks are conducted in English or the student’s first language. Any such threshold

Developing “Cognitive Assessments for Multilingual Learners”

163

will probably be the product of a combination of factors, including the length of time the learner has been in the English speaking environment, the amount of exposure to English and the existing linguistic repertoire of the individual. At the time of writing, this has not yet been established. Once this has been established for the adult population, it would be interesting to gather data on the multilingual school-age population and establish similar benchmarks and thresholds. There are one million children in UK schools who have English as an Additional Language, according to the National Association for Language Development in the Curriculum (NALDIC 2011). These younger learners are still at risk of having their cognitive differences neglected, thus making learning much more challenging for them. Assessing for the presence of SpLDs in multilingual and multicultural SLLs is a very complex process, and one which needs to be handled very sensitively. There are many variables that need to be considered, and it is hard to ensure that the balance between reliability and validity is managed effectively. Where tasks are successful at tapping into the particular cognitive functions that need to be investigated, often it is at the expense of easy or reliable rating for an assessor who does not share the SLL’s first language. In addition, it is impossible to remove all cultural aspects from an assessment process such as this. The very act of assessing is culturally motivated, and SLLs will respond in the way that their culture dictates is appropriate. Although there are tests of visual ability, visual processing and memory which do not require the use of language, very often the images which are utilised are very much situated within the host culture. These are issues that still need to be addressed, if full cognitive assessments of SLLs outside of their own countries are to become possible. There can be little argument that they do need to become possible, as superdiversity becomes the norm across the world.

Works cited Blommaert, Jan, and Ben Rampton. 2011. “Language and Superdiversity.” Diversities. 13/2 Accessed November 15, 2012. www.unesco.org/shs/ diversities/vol13/issue2/art1. de Houwer, Annick. 2009. Bilingual First Language Acquisition. Clevedon: Multilingual Matters. Deponio, Pamela, John Landon, Kirstine Mullin, and Gavin Reid. 2000. “An Audit of the Processes Involved in Identifying and Assessing Bilingual Learners Suspected of Being Dyslexic: a Scottish Study.” Dyslexia. 6:29–41.

164

Chapter Eight

Elbro, Carsten, Hanne Trebbien Daugaard, and Anna S. Gellert, 2012. “Dyslexia in a Second Language?: A Dynamic Test of Reading Acquisition May Provide a Fair Answer.” Annals of Dyslexia. 62/3:172–185. ELT well. 2012. “Cognitive Assessments for Multilingual Learners.” www.ELTwell.co.uk. Erten, øsmail HakkÕ, and Salim RazÕ. 2009. “The Effects of Cultural Familiarity on Reading Comprehension.” Reading in a Foreign Language. 21/1:60–77. European Dyslexia Association. 2013. “What is Dyslexia?” Accessed March 14, 2013. http://www.eda-info.eu/en/about-dyslexia.html. Geva, Esther. 2000. “Issues in the Assessment of Reading Disabilities in L2 Children: Beliefs and Research Evidence.” Dyslexia. 6:13–28. Grant, David. 2010. That’s the Way I Think: Dyslexia, Dyspraxia and ADHD Explained (2nd Edition). Abingdon: Routledge. United Kingdom, Parliament. 2010. “Equality Act 2010 (Act of Parliament).” London: HMSO. Great Britain, Department for Education and Skills. 2005. “SpLD Working Group 2005/DfES Guidelines”. Accessed February 27, 2013. http://www.sasc.org.uk/(S(ge1tm555uilvjs45omb1ns45))/SASCDocum ents/SpLD_Working_Group_2005-DfES_Guidelines.pdf. Klein, Cynthia. 2000. Diagnosing Dyslexia. London: The Basic Skills Agency. Kormos, Judit, and Anne Margaret Smith. 2012. Teaching Languages to Students with Specific Learning Differences. Clevedon: Multilingual Matters. National Association for Language Development in the Curriculum (NALDIC). 2011. “EAL Statistics”. Accessed August 3, 2012. http:// www.naldic.org.uk/research-and-information/eal-statistics. Nijakowska, Joanna. 2010. Dyslexia in the Foreign Language Classroom. Clevedon: Multilingual Matters. Sassoon, Rosemary. 1995. The Acquisition of a Second Writing System. Bristol: Intellect Books. Vertovec, Steven. 2007. “Super-Diversity and Its Implications”. Ethnic and Racial Studies. 30(6):1024–1054. Waine, Liz, and Sue Kime. 2005. “Practical Aspects of Assessment”. In Dyslexia?: Assessing and Reporting, edited by Gill Backhouse and Kath Morris. 29–43. London: Hodder Education.

Developing “Cognitive Assessments for Multilingual Learners”

Appendix A Overview of CAML activities Activity

Observation Interview

Focus Holistic Overview - patterns of behaviour: general areas of strengths and weaknesses and approaches to learning. - educational, health and family background

Literacy - educational background, Writing in the first language (=L1) L1 literacy development Writing in English - L2 literacy development, or other target language (= L2) transfer of L1 skills Reading in the first language (= - educational background, L1) L1 literacy development - reading strategies, visual discrimination, Scanning short term memory, processing speed a) repeating names b) ordering names c) sounds forwards d) sounds backwards e) phonemes forwards f) phonemes backwards a) remembering shapes b) ordering shapes c) remembering sequences d) reversing sequences a) months of the year b) picture naming

Auditory memory - short term auditory memory - working auditory memory - short term auditory memory - working auditory memory - short term auditory memory - working auditory memory Visual memory - short term visual memory - working visual memory - short term visual memory - working visual memory Rapid Automatic Naming processing speed & sequencing & phonological retrieval

Phonological Awareness a) perceiving syllables - segmentation of words b) same sounds – L1 - phonological discrimination c) same or different –English - phonological discrimination d) taking away sounds - isolation of phonemes e) spoonerisms - isolation and manipulation of phonemes

165

Chapter Eight

166

Appendix B Example observation schedule Name of learner: Behaviour observed:

Date of first observation (+ tutor’s initials)

Dates of subsequent observations (+ tutor’s initials)

Bumping into furniture Forgetting peers’ names Doing wrong task in class Arriving at wrong time Often mispronouncing common, previously mastered words

Appendix C Practice phase of the Scanning Task

Devveloping “Cognnitive Assessmeents for Multilinngual Learners””

Append dix D Prractice phase of the Rapid d Automatic N Naming task

167

CHAPTER NINE FAIRNESS AND VALIDITY IN TESTING STUDENTS WITH SPLDS: A CASE STUDY FROM ITALY CLAUDIA D’ESTE AND GERALDINE LUDBROOK This article sets out to examine some of the issues of validity that arise when considering the assessment of the English language proficiency of university students with Specific Learning Difficulties (SpLDs) in the context of the Italian university system. National guidelines established by the Italian Ministry of Education guarantee access to learning and assessment for students with documented SpLDs. As a result, increasing numbers of dyslexic students continue their studies at secondary school and university level. Italian universities now require mandatory certification of general English proficiency on enrolment. The growing numbers of dyslexic students applying for university entrance therefore raise the question of providing fair and valid tests of English language, in the framework of the national guidelines imposed by the Italian law. We focus on a case study to describe the measures that have been developed and adopted to allow dyslexic students enrolling at Venice University access to the CEFR level B1 English entrance test. In particular, we illustrate the different phases of the test administration to a student with severe SpLDs by pointing to all the difficulties involved. In the light of this experience, we discuss the issues of fairness and validity that emerge, and how they have been addressed.

1. Introduction A policy of integration for disabled students has been applied to mainstream education since the 1970s. However, only recently has Italy established national guidelines for learning and assessment for students who present official documentation of Specific Learning Disabilities (SpLDs) to guarantee their access to all levels of education. The guidelines

170

Chapter Nine

are mostly intended for students in secondary education. University education is not considered a separate context, although learning and assessment procedures differ greatly from the school setting. Italian universities now require mandatory certification of general English proficiency at the CEFR B1 level on enrolment. Students who do not possess certification from an internationally-recognised examinations board are required to sit in-house tests developed by each individual university. Venice University has developed a computer-based B1 English test to be administered to first-year students to meet this requirement. The multiple-choice test assesses reading and listening skills, and knowledge of grammar and lexis. This article sets out to examine some of the issues of validity that arise when considering the assessment of the English language proficiency of university students with dyslexia and related SpLDs. We begin with a review of the legislative background in Italy to identify the provisions made for students with SpLDs. We then give a brief definition of these SpLDs and the implications they have for Italian students learning English as a second or foreign language. We continue with an outline of concepts of validity in language testing, and a discussion of how a judicious application of the Italian guidelines is needed to reduce the threat to test validity and fairness. In the second part of the paper, we focus on the case study of a student with severe multiple SpLDs, and the manner in which access to the B1 English test was arranged. We present his test results and the difficulties encountered in making principled decisions regarding their use so as to open up discussion on their validity as a consequence of the compensatory measures applied. We conclude with a reflection on the difficulties of establishing standardized accommodations for test takers with severe multiple SpLDs. We also refer to the need for considering individual cases when deciding on accommodations so as to guarantee both fair access and valid testing.

2. Disabilities legislation in the Italian education system Italy was one of the first OECD countries to apply a policy of integration of disabled students in education. Currently about 99% of disabled students are enrolled in mainstream education. The remaining 1%—mainly students with severe visual, hearing or cognitive difficulties —is educated in separate institutions. Estimates from 2011 data suggest that students with disabilities represent about 2.3% of the school population (Associazione Trelle et al. 2011).

Fairness and Validity in Testing Students with SpLDs

171

A variety of forms of assistance to learning are available to disabled school students under Italian law. They range from the presence of support teachers in the classroom to adaptive technologies to enable learners with disabilities to be educated in mainstream classrooms to the greatest extent possible. Such technologies are speech-to-text conversion or voice recognition, magnification devices, Braillers, and visual and graphic organisers. Specific legislation also governs the assessment of students with disabilities. In the school system, each student has an individual educational plan that takes into account their learning potential and their starting level. Assessment is then calibrated to each student’s individual potential. Access to the final state exam is assisted by additional time, adaptive technologies and the presence of support teachers. However, students may also sit “equivalent” exams, designed by a commission of teachers within each individual school (Associazione Trelle et al. 2011). Access to university education was granted in 19921 Further legislation regarding services for disabled students intending to continue their studies at the tertiary level was introduced in 1999. 2 The services include assistance from personal tutors and the provision of adaptive technologies to facilitate learning and assessment. Most universities now have specific offices with trained staff to assist students with disabilities in their university studies and provide co-ordination with teachers regarding access to examinations. The legislation cited above has been integrated in recent years by regional laws, regulations, and administrative decisions. Nevertheless, it remains a rather generic background of guidelines. Especially the guidelines regarding university examinations, despite the assistance of disability offices, are often dealt with on the basis of individual cases with no or little co-ordination between teachers of different disciplines.

2.1 Legislation on SpLDs Specific legislation on SpLDs has been drafted only in the last decade. Even though students with SpLDs represent around 4–5% of the Italian school population, in the early 2000s legislation was limited to local regulations, regional laws and ministerial letters. Recognition of students with SpLDs, adoption of learning and teaching strategies to meet their needs, and use of compensative instruments and technologies were applied 1 2

See law n. 104/1992. See law n. 17/1999.

Chapter Nine

172

depending on local legislation with differences between regional educational contexts. Best practices were, thus, not shared on a national level, but limited to a small number of teachers and education professionals. Specific learning difficulties were fully recognised by Law 170/2010. The key words of this law are early diagnosis and certification, teaching and learning flexibility, and the introduction of compensatory measures or exemptions. The law defines dyslexia, dysgraphia, dysorthographia, and dyscalculia as specific learning difficulties, and sanctions the right to education for students with these learning difficulties. It first delineates the need for an official diagnosis and certification of cases of SpLDs. Once official diagnosis has been obtained and documentation presented to the school, students may obtain an individual educational plan (PEI, or Piano Educativo Individualizzato). Individual educational plans are a combination of what is called individual learning (didattica individualizzata) and personalised learning (didattica personalizzata). Individual learning refers to the student’s individual activities and method of study to develop and improve weaker skills and competences. Personalised learning involves the use of different teaching methodologies and strategies, in order to meet the student’s needs and promote his/his potential. However, individual educational plans refer only to the school context, and the law makes no similar provisions for university education. Conversely, the compensatory measures or exemptions established by law can be applied at all levels of education, in the classroom but also during examinations. University courses are for the first time included in the analysis and investigation of students with SpLDs. The indications provided by the law must also be applied to entrance tests and curricular examinations at university level. Further guidelines implement the law by focusing on new educational and assessment methodologies and on the training of support teachers. They basically stipulate two kinds of provisions: compensatory measures and exemptions. The decree and guidelines also contemplate the use of educational and technological compensatory instruments to facilitate study, such as: -

tables or mind maps, voice synthesizers to convert language text into speech, recorders instead of note-taking, PC word processors with spelling and grammar checks, calculators.

Exemptions refer to:

Fairness and Validity in Testing Students with SpLDs -

173

exoneration from activities students with SpLDs find particularly hard to perform such as written tasks, which can be replaced by oral activities, up to 30% extra time allotment during tests, reduction of the curricular programme to be studied.

Only in the case of serious SpLD diagnosis can students be exempted from written examinations and given an oral interview instead. The 2011 guidelines refer specifically to university students with SpLDs, and confirm their right to the same compensatory measures and exemptions provided for primary and secondary schools. In addition, they also introduce a networking of tutoring services in order to mediate with teachers and monitor the application of the law.

2.2 Legislation on foreign language learning and testing In addition to the national guidelines for learning and assessment for students with dyslexia and SpLDs, further legislation has provided additional guidelines with specific reference to the teaching and testing of foreign languages.3 Recommendations are provided as to the criteria for use in assessment. In relation to receptive skills (reading and listening), the legislation suggests that teachers and testers should focus on general understanding rather than on accuracy and detailed comprehension. As far as productive skills are concerned, teachers are invited to apply the same principles and evaluate communicative effectiveness. Task achievement is of the utmost importance. However, it is suggested that, when assessing students with dyslexia and SpLDs, grammar range and accuracy together with the use of a wide range of lexical resources are not to be considered as essential features. In addition to guidelines for assessment criteria, the legislation provides for the use of technological compensatory instruments in testing settings. These include voice synthesizers to assist reading skills, and PC word processors with spelling and grammar checks and online dictionaries to assist writing skills. What instruments are used in examinations is at the discretion of the examining commission. Exemptions are represented by a reduction in the curricular programme and, in the case of certified severe SpLDs, students might be exonerated from written assessment and be provided with equivalent oral assessment. Exemption from all forms of written assessment of a foreign language was 3

DM 12/07/2011 (Legge Gelmini).

174

Chapter Nine

initially permitted, but this measure was modified in later legislation. It is now recommended that dyslexic students be given additional time for written exams in foreign language state exams rather than exemption to ensure the legal validity of the final diploma. The critical aspect of the legislation is the discretional choice of compensatory measures and exemptions. In the secondary school context, teachers may observe the performance of students over a period of time following the guidelines for their individual learning plans, and make informed decisions about their application. However, at university level, students work in relative isolation, and co-ordination between individual teachers and the disabilities offices is at best tenuous. This makes decisions regarding what measures to apply problematic. We will return to this issue in the next sections.

3. Dyslexia and related SpLDs The national guidelines for learning and assessment for students who are certified with SpLDs refer explicitly to the definition and recognition of dyslexia, dysgraphia, dysorthographia, and dyscalculia. It is beyond the scope of this article to propose a detailed definition of dyslexia and related learning difficulties. However, the literature points to four main features of dyslexia. It will be useful to review them here to better understand the difficulties dyslexics face in processing language. Firstly, dyslexia is a learning difficulty that primarily affects the skills involved in accurate and fluent wording, reading and spelling. Dyslexics have difficulty in mapping phonemes to graphemes, which can lead to slow or inaccurate word-recognition (Kormos and Smith 2012). Secondly, characteristic features of dyslexia are difficulties in phonological awareness, verbal memory and verbal processing speed. It appears that dyslexics have difficulties in retaining spoken information within their short-term memory systems, in accessing spoken information from long-term memory, and in reflecting on the units of sound within words (Kormos and Smith 2012, 30–31). Thirdly, dyslexia is neurobiological in origin and occurs across the range of intellectual abilities; in other words, it is not related to measured IQ. In typical readers, cognition and reading/spelling develop together while in dyslexic readers they appear to develop differently (Gabrieli 2009). Fourthly, dyslexia is usually conceived as on a continuum ranging from mild to severe difficulties. It is not therefore a distinct category, with clear cut-off points, but is often a series of related linguistic difficulties,

Fairness and Validity in Testing Students with SpLDs

175

such as dysgraphia, dysorthographia and dysnomia (Payne and Turner 1999). In addition, there are some common cognitive problems connected with dyslexia. These are dyscalculia, dyspraxia, short-term memory deficit, and Attention Deficit Disorder (ADD), often associated with hyperactivity (ADHD). Dyslexics thus may have difficulties in retaining information in the working memory, which hinders reading and listening to longer oral or written texts. At the same time a limited attention span adds fatigue to learning and requires repeated input of new knowledge. Difficulties with attention also lead to problems in managing time and personal organization (Kormos and Smith 2012, 32–33). Not only do these problems occur comorbidly, they also manifest themselves in varying degrees of severity. It is therefore problematic making inferences from one individual to another as no individuals have the same linguistic and cognitive profile. See Ferrer et al. (2010) and Lyon et al. (2003) for further information on defining dyslexia and related SpLDs. The frequency of dyslexia also varies between languages. In fact, dyslexia is common in languages with deep orthography: writing systems with relatively irregular correspondence between sounds and letters. Dyslexia is less common in languages with more transparent orthographies: writing systems with more consistent mappings between sounds and letters (Lindgren 2012, 19). In their study of English- and Italian-speaking dyslexics, Job et al. (2006) came to the conclusion that it is the orthographic irregularity of a language that may alter the degree of success in learning to read. It can also affect the actual manifestation of dyslexic symptoms. From this brief overview of potential language problems for dyslexics, it is clear that, in general, the process of second language acquisition brings with it an additional load for learners with dyslexia and related SpLDs. Such learners have considerable difficulty in reading and using a second language (see Sparks and Ganschow 1991; Sparks et al. 2008). In addition to this generalised difficulty in acquiring a second language, the learning of English for Italian dyslexic students is additionally problematic if we take into consideration the contrast in orthographies between the two languages. English has a highly inconsistent match between the 26 graphemes and the 44 phonemes of its writing and sound systems. Indeed the British Dyslexia Association estimates that 10% of the British population are dyslexic, 4% severely so.4 4

http://www.bdadyslexia.org.uk/about-us.html.

176

Chapter Nine

This compares with the much lower numbers in countries with more transparent orthographies such as 4 to 5% in Italy, which has a shallow orthography and a close matching between letters and sounds (Lindgren et al. 1985; Lindgren 2012; Associazione Trelle et al. 2011). Additional problems therefore arise for Italian learners of English as a second language. We now turn to the issues of validity and fairness in language testing. We examine how the guidelines for the treatment of dyslexic students set out in Italian law represent a possible threat to the validity and, more generally, the fairness of high-stakes English language tests.

4. Issues of validity and fairness in language testing The issue of validity in language testing concerns principally the use and interpretation of test results. Messick (1989, 13) defines test validity as “an integrated evaluative judgement of the degree to which empirical evidence and theoretical rationales support the adequacy and appropriateness of inferences and actions based on test scores”. The validity of a test therefore lies in the degree to which inferences about the test taker’s language performance can be justified. How a test taker performs on a high-stakes test of English for university entrance in English-medium programmes, for example, is a prediction of how they will be able to perform in an English-medium educational context. Messick (1989, 34–35) identifies two major threats to validity in language tests. The first is construct-irrelevant variance. This occurs when a test measures variables that are unrelated to the construct, i.e. what the test claims to measure. This may result in construct-irrelevant easiness, when the test tasks may provide clues that allow some test takers to respond correctly in ways that are irrelevant to the test construct, and result in higher scores. This may also result in construct-irrelevant difficulty, when the test is more difficult for some test takers, for irrelevant reasons, and results in lower scores. Construct-irrelevant variance is often perceived as a threat to the validity of score interpretations for students with SpLDs. This is because such students often have difficulties which can affect their performance on educational tests. For example, a student may be unable write test answers with decipherable handwriting, making it difficult for the rater to read the written product. The effects of the SpLD are construct-irrelevant if the skills (handwriting, in this case) are irrelevant to the construct being assessed. Messick’s second threat to validity is construct under-representation.

Fairness and Validity in Testing Students with SpLDs

177

This occurs when the tasks measured in the assessment fail to include important dimensions or facets of the construct. As a result, the test scores do not reveal the test taker’s true abilities within the construct, as this is defined for the purpose of the test (Messick 1989, 35). If, for example, students with SpLDs are exonerated from a part of a test, the test scores no longer fully interpret the student’s abilities. Closely related to Messick’s concepts of construct-irrelevant variance and construct under-representation, and of particular importance when assessing the performance of students with SpLDs, is the notion of cognitive validity. According to O’Sullivan and Weir (2011), one of the most important assumptions made when designing test items and tasks is that responding to them relies on a correct activation of certain underlying cognitive processes. The circumstances of a real-world communicative event cannot be reproduced in the artificial environment of a test. However, cognitive validity requires discovering whether the mental processes that a test elicits from a candidate resemble the processes that they would employ in non-test conditions (Glaser 1991). Characteristic features of dyslexia are difficulties in phonological awareness, verbal memory and verbal processing speed. These traits suggest that the cognitive processes used by dyslexic test takers may be different from those used by students without SpLDs. Field (2006) proposes three questions that may be explored to investigate cognitive validity in language testing for students with SpLDs: 1. To what extent are the cognitive processes elicited by a test comparable to those that would be employed in a real-world setting? 2. Is the range of processes elicited by a test comprehensive enough to be representative of behaviour in a real-world setting? 3. Are the cognitive demands imposed by a test sufficiently finely calibrated to reflect the level of the test?

Research on L2 learning of students with SpDLs is limited, and fails to provide an adequate basis for definitive conclusions about the mental processes elicited by language tests. Cognitive validity therefore remains an almost unexplored feature of the overall test validity. In addition, learning difficulties are not homogeneous, and similar disabilities may affect different people in different ways. Therefore, it is extremely difficult to identify a possible standard mental process. Nonetheless, investigation on cognitive validity may be decisive. The understanding of mental processes in the language learning of students with SpDLs is potentially the basis for the identification of the test construct on which to design and build a valid test.

178

Chapter Nine

Another essential aspect of test validity is fairness, and “anything that weakens fairness compromises the validity of a test” (Xi 2010, 147). There are three generally-accepted descriptions of test fairness: lack of bias, equitable treatment of all test takers in the testing process, and equity in access to learning the materials covered in a test. See Kunan (2000) for a more detailed discussion of fairness in language testing. Fairness with regard to test takers with disabilities is generally understood as providing access to a test applying appropriate accommodations. Accommodations are pre-approved alterations to the standard administration conditions designed to ensure accessibility to a test for test takers with disabilities. They do not alter the construct of the test being measured or substitute for knowledge or abilities that the student has not attained. Nor do they provide an unfair advantage for students with disabilities over students taking tests under standard conditions. Frequent forms of testing accommodations are flexibility in scheduling/timing; flexibility in the setting used for the administration of assessments; changes in the method of presentation; and changes in the method of response (see Stretch and Osborne 2005). Modifications, on the other hand, are changes made to the testing process or the content of the assessment itself, or provision of certain adaptive technologies or services, which affect the constructs being tested. Examples of testing modifications that affect the construct of the test are simplification or explanation of test questions; reading of items designed to test the student’s reading skills; use of spell and/or grammar-check devices on a test of the student’s writing skills; and use of a calculator on a test of the student’s computational skills. Test fairness requires ensuring that irrelevant factors do not give rise to differences in test performance across subgroups and disabilities, i.e. construct-irrelevant personal characteristics of test takers have no appreciable effect on test results or their interpretation (Xi 2010). Yet, all test takers must be treated equally in the testing process. In the next section, we will discuss the guidelines, set out under Italian law, for students with medically diagnosed SpLDs in both learning and testing contexts in the light of the concepts of validity and fairness mentioned above.

5. Issues of validity and fairness in the Italian context A close analysis of the compensatory measures set out in the Italian legislation suggests that their injudicious use might create a serious threat to the validity of tests, i.e. the validity of the score interpretations, if used

Fairness and Validity in Testing Students with SpLDs

179

in the testing context. Indeed, most of the compensatory measures appear to be modifications—changing what is tested—rather than accommodations merely providing increased access to the test. The assumption that students can be allowed to use the same adaptive technologies on tests that they are accustomed to using in the classroom, such as a voice synthesizer for a reading comprehension test, may alter the nature of the ability being tested. This would seem to be a case of one of Messick’s two serious threats to validity in language tests—construct under-representation—in which important features of the construct are omitted from the test. If the test claims to measure the test taker’s ability to read a written text in the foreign language, a voice synthesizer that converts text to speech may measure instead the test taker’s ability to understand spoken text, thus excluding reading skills. It is for this reason that international examination boards certifying English second/foreign language proficiency only permit measures that are clearly accommodations, such as extra time (usually 25% of the normal time) and supervised breaks to compensate for fatigue. Use of a computer is allowed, but without use of the spell check, grammar check or thesaurus functions. Measures that modify the nature of the ability being tested, such as voice recognition software or programmes that convert speech to text, and screen-reading software, such as voice synthesizers, are not permitted.5 Test accommodations are designed to promote fairness in testing and to lead to more accurate interpretation of students’ test scores. However, if the accommodation leads to an unfair advantage for the students for whom they are applied then the scores from accommodated exams may be invalidly inflated. This would be unfair to students who do not receive them. The benefits provided to test takers with disabilities through the use of electronic dictionaries or the consultation of tables and mind maps would represent an unfair treatment to students taking tests under standard conditions. A further challenge arises from the generic nature of the guidelines set out in Italian legislation and the consequent difficulties that teachers and testers have in making principled decisions regarding their use. This is particularly true in the case of foreign language—and especially English— learning and testing. Distinguishing between errors due to dyslexia or related SpLDs and errors due to lower levels of proficiency or to the normal language learning process is problematic. Italian legislation leaves the choice of which measures to adopt for 5

See http://www.cambridgeesol.org/exams/special-circumstances/index.html.

180

Chapter Nine

students with SpLDs to the discretion of the teachers and examiners in charge of the test being administered. In the school context, where teachers have had extended contact with students and are aware of their abilities and special needs, this would seem to be a useful procedure. It allows a personalised assessment of the needs of each individual student. This is particularly important when test takers have multiple disabilities requiring a variety of accommodations that go beyond the standard requests for extended time and additional breaks. However, in the Italian university context this is not an easy process to carry out when there has been no previous contact with the individual students. Test administrators are required to make decisions based on very little background information. Students must hold documentation of their SpLDs in order for compensatory measures to be applied. Often this documentation is a brief medical certificate noting the disability or disabilities (dyslexia, dyscalculia etc.), and the degree of severity (mild, severe). Even when more detailed information is provided, it is usually beyond the professional competences of the teachers and test administrators to be able to interpret such information in terms of appropriate compensatory measures. We will now turn to the case study of a severely dyslexic student to describe how the considerations above were applied in practice. We will illustrate some of the difficulties test developers and administrators have to resolve when testing the English language proficiency of students with such SpLDs.

6. Marco: A case study In the increasingly international world of education, certification of English proficiency has growing importance. It is used to gain access to university education or European exchange programmes for which English language at B1 or B2 level of the CEFR is required. The national requirement for evidence of CEFR B1 level of English proficiency for students enrolling at Italian universities has been met by Venice University. The university developed an in-house computer-based test for students who do not hold certification of this level from international certification boards. The test format is multiple-choice, and ninety minutes are given to complete the test. The test assesses the receptive skills of reading and listening, and knowledge of grammar and lexis at the B1 level. The B1 test is a high-stakes test as first-year students cannot continue their studies unless they have passed it. Students are assumed to have

Fairness and Validity in Testing Students with SpLDs

181

reached a B1 level of English proficiency on leaving secondary school, after studying the language for an average of 8 years. Nevertheless, about 35% of students do not pass the test at their first attempt. Test administrators at the Venice University Language Centre apply a policy of inclusion. They attempt—as far as possible—to avoid exemptions or alternative forms of testing for students with disabilities, even though the law allows students to request these measures. Deaf students are exonerated from the listening test, otherwise a variety of compensatory measures are applied to enable students with disabilities fair access to the test. Visually-impaired students, for example, are allowed screen magnification or enlarged font and bigger line spacing, or readaloud accommodations in which a reader reads the test. Following the 1999 legislation which guarantees access to universities for students with learning difficulties, most Italian universities have set up disability offices to help co-ordinate assistance for these students. Some of the services students may request by law are: supply of teaching material in electronic form, sharing of lecture notes from other students, and personalised examination conditions with extra time and the use of a computer with voice synthesizer. The law also makes provision for individual tutors to assist these students with their studies. In 2010 the Venice University Language Centre was contacted by the disabilities office and asked to help with the case of Marco, who had recently enrolled at Venice University and needed to pass the B1 exam. Marco was diagnosed with multiple comorbid SpLDs. They included severe dyslexia combined with dysorthography and dyscalculia. In addition, he had short-term memory and attention deficit. Marco had requested the services of a tutor to prepare for the English test. In addition he had initially asked to replace the computer-based test with an oral exam, which is a compensatory measure available to students with SpLDs. The inclusion policy of the Language Centre aims to remove unfair disadvantages for disabled students. However, it also aims to prevent unfair advantages to the same students through the application of compensatory measures in an attempt to maintain the validity of the B1 English test as far as possible, while applying the measures set out under Italian law. The testing team, made up of language teachers and test administrators, technicians and a representative of the Disabilities Office, thus met with Marco to discuss his needs and to agree on suitable compensatory measures for the B1 English test. During the meeting it was explained that the computer-based test had a multiple-choice format and was not a written exam. As a result, Marco

182

Chapter Nine

agreed to attempt the test instead of replacing it with an oral exam. Consequently, a first series of decisions were made concerning the way the test would be administered. Marco would sit the test in an individual session with 25% extra time and using a voice synthesizer. In addition, the test would be delivered in separate sections, i.e. listening and grammar, reading and lexis, on two separate days. Marco sat the test twice, in January and May 2011, without reaching a satisfactory result in all four sections at the same time. His scores were not consistent between different skills. For example, in one session his listening reached a pass result, in the next it did not; his reading was below the pass score the first time, but improved the second time. A second meeting was held to discuss further measures to be applied in the administration of the test. The 25% extra time and individual administration was confirmed. In addition, a better quality voice synthesizer was made available. Individual language tutoring was arranged with an experienced English mother-tongue tutor, and it was decided to focus on the individual sections of the test separately. Marco would sit each section at four-week intervals to allow him time to prepare for each section. Marco asked to use an electronic dictionary and to consult grammar tables during the test. Marco sat the test in three separate sessions between January and May 2012, choosing to take the reading and listening sections separately, but to attempt the lexis and grammar sections on the same day. It was observed that, even though it was available, Marco did not use the dictionary during the test, but briefly consulted grammar tables. Marco’s results on the test with the new revised delivery were surprising as, despite the individual tutoring, he performed worse on all of the sections of the test than in previous attempts. The testing team now felt that all possible efforts had been made to meet the legal requirements for testing students with SpLDs and to guarantee fairness in Marco’s test administration. At this point, and in the light of his most recent results, it was decided to interpret his scores. This was not an easy task as his scores on the separate sections had remained inconsistent between different skills. The language teacher who had tutored Marco was interviewed in an attempt to understand the level he had reached in the classroom setting. Besides providing useful insights into Marco’s learning difficulties and the strategies used to overcome them, the tutor assured us that Marco had shown improvement over the three-month period, and emphasised his high motivation. She was of the opinion that Marco had reached his highest possible level of English proficiency at this stage. The tutor identified

Fairness and Validity in Testing Students with SpLDs

183

Marco’s short-term memory as being the greatest obstacle in his learning as it affected his ability to retain grammar rules and lexical resources. The final decision was made to use Marco’s best scores in each section of the test, whenever it had been taken. In addition, even though his results on the grammar section had always been below the cut score, it was decided to accept this result. The rationale behind the decision was that the grammar section is the longest part of the test, possibly leading to anxiety and fatigue due to Marco’s difficulties with attention deficit. Grammar is also one of the most difficult language features for dyslexic students in their L1 and is generally transferred to their L2; this was confirmed by Marco’s tutor (see Brinckerhoff and Banerjee 2007, for further discussion of decisions regarding accommodations in high-stakes tests). The report of this small case study has served to highlight some of the difficulties encountered by test administrators when dealing with students with SpLDs to ensure both fair access to the test and the validity of the test itself. The complexity of this student’s SpLDs made identifying his needs particularly problematic. However, an interview with the student and agreement on the compensatory measures to be applied, together with the information provided by his language tutor, ensured that the validity of the test was maintained to the highest degree possible. We will develop the discussion of these issues in the final section below.

7. Discussion and future research The case study described above provided an interesting exploration of the issues of validity and fairness when testing the English language proficiency of students with SpLDs within the framework of Italian law on access to university education. Building a validity argument for the score interpretation of a test is a complex and ongoing process. Evidence of test validity derives from different sources of information and is never judged in absolute terms, but, generally, in degrees (Messick 1989). In their discussion on test validity, Alderson et al. (1995, 171) make an important distinction between “internal” validity, which involves the perceived content of the test and its perceived effect, and “external” validity, which involves comparing test scores with external measures. In this small case study, our considerations of the validity of the B1 test delivered with the chosen compensatory measures focused on the aspects of internal validity: face validity, content validity, and construct validity. Face validity, or the degree to which non-experts judge that the test results are valid, appeared to be maintained as interviews with non-testing

184

Chapter Nine

experts, such as Marco’s language tutor, representatives from the disability office, and Marco himself, revealed. Content validity, or the degree to which the test covers the content domain, also appeared to be maintained. The test content was not modified or reduced in any way, altering neither the B1 syllabus of reference nor the test specifications. Construct validity, or the degree to which a test successfully measures what it sets out to test, avoiding Messick’s threats of construct-irrelevant variance and construct under-representation (see Section 4 above), is harder to establish. The English test discussed here serves to ascertain that students enrolling at Venice University have the minimum language proficiency required for undergraduate study. The test results therefore theoretically reflect the test takers’ possession of a CEFR B1 level of English proficiency. The validity of the construct when testing the language proficiency of students with SpLDS remains an open question. On the one hand, the construct of the test administered to Marco appears to be unvaried: Marco sat the same test as other students with no exemptions, thus maintaining face and content validity, and apparently avoiding construct under-representation. On the other hand, the use of certain compensatory measures permitted under Italian law may lead to precisely this threat. The use of a voice synthesizer on the reading test, in particular, would seem to modify the language skills actually being assessed. The issue of fairness as a feature of test validity has also been examined. The arrangement of individual language tutoring and the organization of the test delivery into separate sections were decided on to give Marco the opportunity to focus on the individual sections of the test. He was also allowed 25% extra time to complete the test. However, these measures would certainly have provided additional advantages for test takers without disabilities. It is agreed that equitable treatment is necessary in test administration. The question remains as to whether these measures represented equitable treatment, or whether they gave Marco an unfair advantage. There are no definitive solutions to the issues raised in this discussion. However, it is apparent that some degree of balance is required between the needs of test takers with SpLDs and the responsibility of teachers and test administrators to guarantee equal access to learning and assessment. Considerations of the principles of fairness and validity, and the awareness that legislation is intended to guarantee equal opportunities to all, not equal outcomes for all, are important. They influence greatly the decisions as to what measures can be applied in the testing context. Flexibility and

Fairness and Validity in Testing Students with SpLDs

185

sensitivity in applying guidelines on a case-by-case basis and the principled application of legislation are necessary to meet the needs of all the stakeholders involved in the testing process: test takers, test administrators and educational institutions.

7.1 Directions for future research The case study discussed here provides useful indications for future research projects related to the investigation of the special needs of students with SpLDs in foreign language testing. The student observed had severe multiple comorbid SpLDs, which aggravated the issues to be dealt with. Nevertheless, two main areas for future investigation emerged from this study. The first area is to explore the issue of reading skills and the use of a voice synthesizer in testing contexts. The use of voice synthesis has been shown to enhance reading rate and comprehension and to sustain longer reading for students with SpLDs (see, for example, Elkind et al. 1996). Nevertheless, the use of voice synthesizers in international examinations of English as a second/other language is generally not allowed as it is perceived as modifying the construct of the test, substituting reading skills with listening skills. There appears to have been little investigation of whether students with SpLDs using synthesizers to enhance reading are merely listening to the voice generated or whether some reading skills are also involved. Verbal reports and semi-structured interviews may capture test takers’ perceptions of how they use reading and listening skills. Evidence of some reading may, to a degree, counterbalance the argument against the use of voice synthesis in language tests. The second area is to examine the question of cognitive processing of Italian students with SpLDs when dealing with English language testing. The validity of a test relates to the interpretation of the correct responses to items. So, what matters is not what the test constructors believe an item to be testing, but which responses are considered correct, and what process underlies them. In other words, to clearly establish the trait that has been measured we need to investigate the processing necessary for task completion (Alderson 2000, 97). In the case of the language testing of students with SpLDs, a threat to cognitive validity appears to derive from an incomplete awareness of what the real mental processes produced by these students are. As a result, it is extremely difficult to distinguish their real and effective proficiency unless we identify the cognitive features of a ‘good’ SpLD language learner and isolate all the elements in the test which do not activate these features

186

Chapter Nine

correctly. Following Weir’s (2005) suggestions, the cognitive validity of test takers with SpLDs might be investigated in two ways: modelling the skill of an expert language learner with SpLDs, and studying candidate behaviour (verbal report) in order to identify the features of the test that affect cognitive validity. The outcomes of such a study might be essential in ensuring that test results and their interpretation are not affected. The factors potentially affecting the results are construct-irrelevant elements of the test or personal characteristics strongly influenced by disability and relevant to the mental processes of students with SpLDs. Further work with students with dyslexia and related SpLDs will produce detailed guidelines for good professional practice in testing the foreign language proficiency of students with language and learning disabilities in the context of the Italian education system. The guidelines will serve as a protocol for appropriate accommodations in the delivery of computer-based foreign language tests. In these guidelines, the issues of validity and fairness for all language learners are suitably addressed.

Works cited Alderson, J. Charles. 2000. Assessing Reading. Cambridge: Cambridge University Press. Alderson, J. Charles, Caroline Clapham, and Dianne Wall. 1995. Language Test Construction and Evaluation. Cambridge: Cambridge University Press. Associazione Trelle, Caritas Italiana, and Fondazione Agnelli. 2011. Gli Alunni con Disabilità nella Scuola Italiana: Bilancio e Proposte. Trento: Edizioni Erickson. Brinckerhoff, Loring C., and Manju Banerjee. 2007. “Misconceptions Regarding Accommodations on High-Stakes Tests: Recommendations for Preparing Disability Documentation for Test Takers with Learning Disabilities.” Learning Disabilities Research and Practice 22(4):246– 255. Elkind, Jerome, Molly Sandperl Black, and Carol Murray. 1996. “Computer-based Compensation of Adult Reading Disabilities.” Annals of Dyslexia 46(1):159–186. Ferrer, Emilio, Bennett A. Shaywitz, John M. Holahan, Karen Marchione, and Sally E. Shaywitz. 2010. “Uncoupling of Reading and IQ over Time: Empirical Evidence for a Definition of Dyslexia.” Psychological Science 21:93–101. Field, John. 2006. Cognitive Validity. http://www.beds.ac.uk/__data/assets

Fairness and Validity in Testing Students with SpLDs

187

/pdf_file/0003/185034/Cognitive-valdity-springseminar-CompatibilityMode.pdf Gabrieli, John D.E. 2009. “Dyslexia: A New Synergy between Education and Cognitive Neuroscience.” Science 325:280–283. Glaser, Robert. 1991. “Expertise and Assessment.” In Testing and Cognition, edited by Merlin C. Wittrock and Eva L. Baker, 17–30. Prentice Hall, Englewood Cliffs. Job, Remo, Francesca Peressotti, and Claudio Mulatti. 2006. “The Acquisition of Literacy in Italian.” In Handbook of Orthography and Literacy, edited by R. Malatesha Joshi, P.G. Aaron, 105–119. Lawrence Erlbaum Associates: NJ. Kormos, Judit and Anne Margaret Smith. 2012. Teaching Languages to Students with Specific Learning Differences. Bristol, Buffalo, Toronto: Multilingual Matters. Kunan, Antony J. 2000. Fairness and Validation in Language Assessment. Cambridge: Cambridge University Press. Lindgren, Sott D., Ennio De Renzi, and Lynn C. Richman. 1985. “Crossnational Comparisons of Developmental Dyslexia in Italy and the United States.” Child Development 56(6):1404–1417. Lindgren, Signe-Anita. 2012. “Mild Developmental Dyslexia in University Students: Diagnosis and Performance Features in L1, L2, and L3.” English Department Publications 6. Turku: Åbo Akademi University. Lyon, G., Reid, Sally E. Shaywitz, and Bennett A. Shaywitz. 2003. “Defining Dyslexia, Comorbidity, Teachers’ Knowledge of Language and Reading: A Definition of Dyslexia.” Annals of Dyslexia 53(1):1– 14. Messick, Samuel. 1989. “Validity.” In Educational Measurement (3rd ed.), edited by Richard L. Linn, 13–103. New York: Macmillan. O’Sullivan, Barry and Cyril J. Weir. 2011. “Test Development and Validation.” In Language Testing: Theories and Practices, edited by Barry O’Sullivan, 13–33. Basingstoke, England: Palgrave Macmillan. Payne, Trevor, and Elisabeth Turner. 1999. Dyslexia: A Parents’ and Teachers’ Guide. Great Britain, WBC Book Manufacturers Ltd. Sparks, Richard L. and Leonore Ganschow. 1991. “Foreign Language Learning Difficulties: Affective or Native Language Aptitude Differences?” Modern Language Journal 75:3–16. Sparks, Richard L., Leonore Ganschow, and James Patton. 2008. “L1 and L2 Literacy, Aptitude and Affective Variables as Discriminators among High- and Low-achieving L2 Learners with Special Needs.” In Language Learners with Special Needs: An International Perspective,

188

Chapter Nine

edited by Judit Kormos and Edit H. Kontra, 1–10. Clevedon: Multilingual Matters. Stretch, LoriAnn S. and Jason W. Osborne. 2005. “Extended Time Test Accommodation: Directions for Future Research and Practice.” Practical Assessment, Research & Evaluation 10(8). Weir, Cyril. 2005. Language testing and validation: an evidence-based approach. Basingstoke: Palgrave Macmillan. Xi, Xiaoming. 2010. “How Do We Go about Investigating Test Fairness?” Language Testing 27(2):147–170.

CHAPTER TEN ASSESSMENT ACCOMMODATIONS IN EFL READING COMPETENCE FOR SLOVENE EFL STUDENTS WITH SPECIFIC READING DIFFERENCES FLORINA ERBELI AND KARMEN PIŽORN

The purpose of the present study was to examine the latent structures1 of the English as a foreign language (EFL) reading competence of 283 Slovene EFL students with specific reading differences (SRDs) and 292 students with no SRDs. A battery of cognitive, language and literacy tests in EFL was administered to both groups to gain a perspective on the EFL reading competence structure of this population. On the basis of exploratory factor analysis, it was found that the latent structures vary between the two groups. The fluency-orthography factor formed the first factor in the group of students with no SRDs, whereas in the group of students with SRDs the auditory-vocabulary-spelling factor was the first factor. This finding might indicate that well-developed fluency and orthography skills in EFL are important for efficient EFL reading competence, and that the group of students with SRDs lacks these skills. Thus, assessment accommodations and modifications for the group of students with SRDs need to be addressed in EFL fluency and EFL orthography skills.

1. Introduction Assessing English as a Foreign Language (EFL) reading competence is challenging. It is a complex task and reading in an EFL context can be influenced by numerous linguistic, cultural and educational factors. When 1

Latent structures are defined as correlations among observed variables determined by making assumptions about the hidden (latent) causes of those variables.

190

Chapter Ten

assessing EFL reading competence, one has to bear in mind that students’ attainment of EFL reading competence may be disrupted by difficulties with any linguistic factor. Such factors are problems with EFL reading fluency or a lack of EFL vocabulary knowledge. Distinguishing between different sources of impediments in EFL reading competence is crucial if we are to tailor instruction, intervention and assessment appropriately (Francis et al. 2006, 302). In assessing EFL reading competence of (Slovene) EFL students with Specific Reading Differences (SRDs), it is particularly important to pinpoint sources of difficulty (Erbeli and Pižorn 2013, 46). Students with SRDs can have extremely uneven profiles of skills. They may have good EFL vocabulary knowledge but very limited word reading fluency skills (Erbeli and Pižorn 2012, 135). The most widely used assessment tasks (such as a cloze task) fail to distinguish the various sources of poor reading competence. These tasks do not provide information on the components (sources) of EFL reading competence with which EFL students with SRDs might need further substantial help. Thus, we do not know the components of EFL reading competence on which assessment accommodations and modifications should be performed.

2. The nature of EFL reading competence One line of reading research is the so-called component skills approach or the componential approach. The component skills approach was first proposed by Carr and Levy (1990) and is widely used by L1 reading researchers (Koda 2010, 19). Research in this domain attempts to identify the components of reading. Another goal of this approach is to account for the individual and developmental differences in reading performance in terms of the differences in specific component processes or knowledge sources (Shiotsu 2011, 2). Component skills are closely interrelated, and a deficiency in one skill can create problems in others. Sources of reading impediments can therefore be identified by means of a clear understanding of the multilayered relationships among component skills (Koda 2010, 20). The component skills approach enables us, firstly, to identify the cognitive, linguistic and reading components forming the (latent) structure of the EFL reading competence of EFL students with SRDs and with no SRDs; secondly, to classify common deficiencies and limitations characterizing students with SRDs; thirdly, to estimate the likely interplay of component skills between different L1s and EFLs; finally, to analyze cross-linguistic interactions between EFL input and L1 skills (Koda 2010, 20).

Assessment Accommodations in FL Reading Competence

191

L1 componential analyses suggest that individual differences in reading may be accounted for by individual differences in various components. Such components are vocabulary knowledge, word or subword recognition efficiency, phonological awareness, and working memory, all of which overlap with general verbal comprehension skills (Koda 2010; Noort et al. 2006, 289). However, the question as to whether these same components explain individual differences in EFL reading competence remains unanswered. The following section focuses on the efforts of EFL researchers to identify and characterize possible components of EFL reading competence. An early theoretical attempt to model English as a EFL reading was Coady’s tripartite model (1979, 5), in which he linked many EFL reading problems to limited L1 ability. For example, he suggested that poor L1 reading competence affects EFL reading mastery. Exploring the question of whether EFL reading is a reading problem or a language problem, Alderson (1984, 14) identified Coady (1979, 5) as supporting the view that it is a reading problem and Yorio (1971, 109) as claiming that it is a language problem. Alderson (1984, 19) himself concluded that it is both. The evidence, however, is in favor of the view that for EFL readers with a low level of EFL proficiency it is more of a language problem. Clarke’s short-circuit hypothesis (1980, 206) has led to a number of research activities focusing on the issue (Brisbois 1995). Research shows that 30–40% of EFL reading variance is explained by EFL knowledge (Bernhardt and Kamil 1995, 15). This line of research is componential in nature, as it tests whether L1 reading competence and EFL linguistic competence are significant sources of individual differences in EFL reading. The construct of EFL linguistic proficiency is, nonetheless, operationalized unidimensionally (Koda 2010, 24), including knowledge of vocabulary and/or grammar. However, analyzing the structure of EFL reading competence necessitates more than just testing for the relative contribution of the vocabulary or grammar variable, i.e., the linguistic component. In analyzing the structure of EFL reading competence, the explaining variance of the reading and cognitive component must not be diminished. Decoding (Koda 2010, 29), phonological knowledge (van der Leij and Morfidi 2006, 74), orthographic knowledge and orthographic distance (Koda 2010, 43), vocabulary (Grabe 2009), visual perception (Bosse and Valdois 2009, 230), executive functioning (Andersson 2010, 458), listening comprehension (Vandergrift 2007, 191), and reading fluency (Kato 2009, 471) all contribute to reading performance in the EFL. It is therefore essential to identify all of the possible components underlying the structure of EFL reading competence in Slovene EFL

192

Chapter Ten

students with and without SRDs, and to determine differences between the latent structures of both groups.

3. Assessment accommodations and modifications Definitions of test accommodations vary, but the focus is on their function. Their main goal is to eliminate sources of difficulty that are irrelevant to the intent of the measurement, and thereby to affect solely the measurement of the construct in students with SRDs, without giving them an advantage over students who are not receiving accommodated assessments (Abedi et al. 2004, 6). In addition to accommodations, however, modifications may also be implemented in classroom-based assessment, in order to meet the needs of students with specific learning differences (Kormos and Smith 2012, 159). “Modifications result in a change in the test […] and work across the board for all students with equal effects” (Hollenbeck et al. 1998, 176). Hence, assessment tasks and techniques can be designed and modified to suit the individual profiles of learners with specific learning differences in classroom assessment (Kormos and Smith 2012, 159–163). In order for accommodations not to have an impact on test validity, however, another question needs to be addressed. Reviews of the effects of accommodations on the performance of students with disabilities in accountability and other high-stakes tests caution against a one-size-fits-all approach (Thompson et al. 2002). The most consistent finding reported across the reviews is the observation that students are heterogeneous. Simply defining students as learning disabled or as “students with a disability” without considering the area of disability may dilute the effect of an accommodation. The accommodation should be specific to the type of disability. In a summary of the research on testing accommodations, Thompson et al. (2002) reported that 31% of studies defined the sample as “learning disabled”, 17% as having “cognitive disability”, 16% as “students with disabilities”, and only 9% as “reading or math disabled”. This research summary suggests that many studies provide accommodations without considering the relation of the accommodation to the area of difficulty. This provides little information on the specificity of the accommodation to the disability. Seen as a whole, these findings indicate that accommodations and modifications need to be adjusted to specific areas of difficulties. They also indicate that students with SRDs may need continuous and relevant feedback, as well as further help, in order to successfully overcome such difficulties. However, no such research with the structure of EFL reading

Assessment Accommodations in FL Reading Competence

193

competence as a construct of central interest has thus far been conducted in this field of inquiry on EFL Slovene students with or without SRDs.

4. The study Empirical work focusing on the relative significance of EFL proficiency for EFL reading competence has shown that EFL proficiency has a consistently larger effect than L1 reading competence (Bernhardt and Kamil 1995, 15). Therefore, a dedicated investigation of the former would prove illuminating. The components and their relationships underlying the EFL structure of reading competence of Slovene students have not yet been covered in the literature concerning EFL reading. In the present study, we therefore confine ourselves to investigating whether there is a different factor structure underlying EFL reading competence of Slovene students with SRDs compared to students with no SRDs. To recapitulate, the purpose of this study was threefold: 1. to identify the number of factors comprising EFL reading competence among Slovene students with SRDs and students with no SRDs, 2. to examine the ways in which these factors of EFL reading competence differ between the two groups, 3. to determine how accommodation and modification procedures in assessing EFL reading competence can be enhanced, taking into consideration the differences among these factors between the two groups. Specifically, the study was designed to answer the following research questions: 1. How does the factor structure underlying EFL reading competence differ between Slovene EFL students with SRDs and Slovene EFL students with no SRDs? 2. In assessing EFL reading competence, what accommodation and modification procedures can be used in instruction, considering the difference of the factor structure between the two groups?

Chapter Ten

194

5. Method 5.1 Participants The sample comprised 575 Slovene-speaking 7th grade students aged between 12 years and 13 years (mean age 12 years 11 months). The sample was divided between students with no SRDs (n = 292) and students with SRDs (n = 283). The students attended 21 different primary schools in Slovenia, all with Slovene as the language of instruction. All students learned English as a foreign language (EFL is a compulsory school subject from Grade 4 to Grade 9). The English program includes listening, speaking, use of language and reading activities. English instruction is based on a communicative teaching approach, while reading instruction includes a balanced combination of decoding-oriented and meaning-based methods. The Slovene school system is a public, unitary system, based on the ideology of inclusion. The aim of the school system is to include all students with SRDs in the (language) classroom. In Slovenia, students with SRDs are assessed according to a Statement of Special Educational Needs by a panel of qualified professionals (doctors, psychologists, specialist teachers, etc.). These professionals then diagnose the kind of disability present in the individual and decide what provision needs to be made in terms of additional support by a specialist teacher, so that the learner can be included in a mainstream class. Specialist teachers provide adaptive materials, and assistive technology is available if needed. The syllabus is specially adapted to each individual with SRDs according to an individualized education program (IEP).

5.2 Instruments A battery of cognitive, language and literacy tests in English were administered to the students. The battery was designed to measure key components related to EFL reading competence: executive functioning, visual perception, reading fluency, listening comprehension, vocabulary knowledge, and orthographic skills. 5.2.1 Executive functioning Previous studies on children have provided evidence that executive control processes play an important role in foreign language processing (Andersson 2010, 460). In order to measure multiple aspects of executive

Assessment Accommodations in FL Reading Competence

195

functions, the standardized Test of Verbal Conceptualization and Fluency (TVCF; Reynolds and Horton 2006) was used. Three subtests were used in the current study: 1. Categorical Fluency, which measures the student’s ability to retrieve words that fit within a conceptual category (e.g., animals), 2. Letter Naming, which measures word retrieval by initial sound, 3. Trails C, which measures the ability to coordinate high attentional demands, sequencing, and the ability to shift rapidly between Arabic numerals and the linguistic representation of numbers.

The content, construct and criterion-related validity can be inferred from the test manual. We investigated the test-retest reliability using a group of 52 students in a pilot study. The test-retest reliability coefficients were r = .81, r = .83 and r = .70 for each subtest respectively. 5.2.2 Visual perception Previous studies (Martin 2006) have examined the relationship between reading ability and visual perceptual processes. These studies have indicated that students with dyslexia experience more perceptual problems than proficient readers. These students may have poor figureground ability or poor visual closure skills. The figure-ground ability is the ability to see an object or form distinct from other objects or forms or from the background. This ability is important when we are presented with a lot of visual information at one time. Students with figure-ground weakness may have difficulty focusing attention on a detail of an object from surrounding objects, for example, identifying a particular word in a paragraph. The visual closure skill helps students to quickly process information because the visual system does not have to analyze every detail to recognize what the person is seeing. It is a foundation skill for fluency and speed in reading and spelling. Efficient readers rely on this skill because with each fixation of the eye only part of the letters of a word is actually perceived. Two abilities (figure-ground and visual closure) related to the underlying processes of visual perception were measured with two subtests of the standardized Test of Visual Perceptual Skills (TVPS; Martin 2006). The Figure-Ground subtest required students to find one design among many within a complex background. In the Visual Closure subtest, the student is shown a completed design on the page and is asked to match it to one of the incomplete patterns shown on the page. Content validity is built into the test through its design specifications and through the procedures followed in item selection. Construct validity is

Chapter Ten

196

examined using the Principal Component Analysis method. The threefactor solution of the TVPS explains 86% of the variance. The reliability (Cronbach’s Į) was .79. 5.2.3 Reading fluency Reading fluency ability was indexed by the standardized Test of Silent Word Reading Fluency (TOSWRF; Mather et al. 2004) and by the standardized Test of Silent Contextual Reading Fluency (TOSCRF; Hammill et al. 2006). The TOSWRF and the TOSCRF measure word identification and speed, but they also measure word comprehension. Due to the fact that TOSWRF and TOSCRF scores reflect competence in so many aspects of reading, test results can be used with confidence to identify both good and poor readers. Evidence for this claim is found in the chapters regarding test validity in the manuals for the TOSWRF and the TOSCRF. The TOSWRF is designed to measure single word identification, while the TOSCRF is designed to measure the speed with which students can recognize individual words in a series of printed passages. In the TOSWRF, students were presented with a row of words (e.g., dimhowfigblue) and were given three minutes to draw lines between the boundaries of as many words as possible (e.g., dim/how/fig/blue/). In the TOSCRF, students were presented with passages without punctuation or spaces between words and were given three minutes to draw lines between as many words as possible. For example: AYELLOWBIRDSATONMOTHERSPRETTYHAT A|YELLOW|BIRD|SAT|ON|MOTHERS|PRETTY|HAT.

In a pilot study conducted in June 2011, both the TOSWRF and the TOSCRF were modified to the English proficiency level of Slovene 7th grade students. The test-retest reliability was .86 and .87 for each test respectively. 5.2.4 Listening comprehension One of the factors that contributes to variance in EFL reading is EFL listening comprehension (Vandergrift 2007, 205). EFL listening comprehension was measured by the standardized Test of Auditory Comprehension of Language (TACL; Carrow-Woolfolk 1999). This test measures the student’s ability to understand the structure (syntax) of spoken language. Three subtests were used in the present study:

Assessment Accommodations in FL Reading Competence

197

Vocabulary, Grammatical Morphemes, and Elaborated Phrases and Sentences. The teacher read the item aloud, and the student was directed to circle the picture that he or she believed best represented the meaning of the word, phrase or sentence uttered by the teacher. The TACL was adapted to the English proficiency level of Slovene 7th grade students in a pilot study conducted in June 2011. The content, construct and criterionrelated validity of the TACL can be inferred from the TACL manual. The internal consistency reliability was Į = .83. 5.2.5 Vocabulary knowledge In EFL settings, researchers (Grabe 2009, 266) have shown that vocabulary knowledge is the strongest of the component skills associated with EFL reading. Vocabulary size was therefore also measured in the present study, using the Vocabulary Size Test (VST; Nation and Beglar 2007). As a standardized vocabulary size test, the VST involves retrieval of word meanings from the mental lexicon. The test consists of 140 items. Students were instructed to circle the letter in front of the definition of the word that best described the item. The VST was adapted to the English proficiency level of Slovene 7th grade students in a pilot study conducted in June 2011. The reliability was Į = .87. 5.2.6 Orthographic skills L1-EFL orthographic distance constitutes another significant factor explaining quantitative variations in EFL word recognition, and therefore also in EFL reading (Koda 2010, 42). The Test of Orthographic Competence (TOC; Mather et al. 2008) is designed as an efficient, reliable and valid measure of orthography in school-age students. Three subtests were used in the present study: 1. Grapheme Matching, in which students are shown a series of rows, each containing five figures, and are instructed to identify two identical figures in each row by making a slash through them (with a time limit of 45 seconds); 2. Letter Choice, in which students are shown rows of words. Each row has one of four letters (p, d, b, q) missing (e.g., __etter, b is missing). Students are then given two minutes to insert the missing letters in order to correctly complete as many words as possible; and 3. Sight Spelling, in which the teacher says a word while students look at part of the word with one or more of the letters missing (e.g., know, students see ___ow). Students are then asked to fill in the missing letter

198

Chapter Ten or letters (which include an irregular or unusual orthographic element) in order to complete the spelling of the word.

The TOC was adapted to the English proficiency level of Slovene 7th grade students in a pilot study conducted in June 2011. The test-retest reliability coefficients for the first two subtests were r = .77 and r = .89, while the internal consistency reliability of the Sight Spelling subtest was Į = .86.

5.3 Procedures Students received instructions in L1 Slovene at the beginning of each test. The tests were group-administered and conducted in three sessions, each with 45-minute duration. Data collection took place in April, May and June 2012. Parent or guardian consent was obtained for the participation of each student prior to testing.

6. Results and discussion 6.1 Exploratory factor analysis In order to address the first research question, Exploratory Factor Analysis (EFA) was conducted on a battery of tests that included EFL reading fluency, EFL orthographic skills, EFL vocabulary, EFL auditory comprehension, executive functioning and visual perception measures. According to previous research (Shiotsu 2011, 2), these are some of the most fundamental measures with regard to the structure of EFL reading competence. Prior to conducting EFA, a number of criteria for assessing whether the variables entered have factorability were met: Pearson’s correlation coefficients, Kaiser-Meyer-Olkin measures, residual values and Bartlett’s test of sphericity. The methods for selecting the number of factors to retain were as follows: all factors with eigenvalues greater than 1.00 and the scree test. Following principal components extraction, factors were rotated with the oblimin oblique algorithm. This algorithm was selected over orthogonal rotation because the orthogonal algorithm is based on the assumption that the factors are mutually completely independent.

Assessment Accommodations in FL Reading Competence

Measure 3 Measure 4

WRF

.82

AC Morphemes

.79

CRF

.79

AC Vocabulary

.77

OS Sight Spelling OS Letter Choice

.69 .64

Vocabulary Knowledge AC Elaborated Sentences OS Sight Spelling

.74 .73

Measure 1

AC Vocabulary

.78

WRF

.74

.75

OS Letter Choice

.74

.66

CRF

.62

OS Grapheme Matching

Measure 3 Measure 4 Measure 1 Measure 2 Measure 3

EF Categorical Fluency OS Grapheme Matching EF Letter Naming

.78

EF Trail C VP Visual Closure VP FigureGround -

.73

EF Letter Naming EF Categorical Fluency

.69

-

.52

-

.74 .65

VP Visual Closure VP FigureGround EF Trail C

.73 .53 .86

13 %

Measure 2

Vocabulary Knowledge AC Elaborated Sentences

.82 .83

12 %

Measure 3

AC Morphemes

15 %

Measure 2

.61

19 %

-

17 %

-

Measure 1

Factor 3

66 %

Measure 5

Measure 4

Factor 4

62 %

10 %

Factor 2

Factor 1

Measure 2

Students with SRDs

22 %

Measure 1

Students with no SRDs

20 %

Total Variance after Rotation

199

.79 .41

Table 1. Names of measures, explained variance (in percentage) and oblique oblimin factor loadings: students with no SRDs (n = 292) and students with SRDs (n = 283)2 EFA was completed in two groups: students with no SRDs and students with SRDs (see Table 1). Four factors were extracted in both groups, each consisting of similar skills but accounting for a different amount of variance within each group. 2

WRF = Word Reading Fluency; CRF = Contextual Reading Fluency; OS = Orthographic Skills; AC = Auditory Comprehension; EF = Executive Functioning; VP = Visual Perception. Measures 1–5 refer to a particular test.

200

Chapter Ten

Within the group of students with no SRDs, F1 (the fluency–orthography factor) accounted for 20% of the variance. F1 included all of the reading fluency measures and two measures of orthographic skills. This could mean that when the students are relatively fluent in EFL reading, they also have well-developed orthographic skills (the measures correlate positively). Loadings (representing correlations between measures and a factor) are high (all above .64), which means that in the group of students with no SRDs these measures define F1 well. Within the group of students with SRDs, F1 (the auditory-vocabularyspelling factor) accounted for 22% of the variance. This F1 included all auditory comprehension, the sight spelling and the vocabulary measures. This could mean that for students with SRDs the same kind of knowledge is necessary for listening to an EFL text as for vocabulary and spelling assignments. This knowledge, however, is not as important for EFL reading as the knowledge of reading fluency and EFL orthography. This finding can be surmised from F1 in the group of students with no SRDs, i.e., the skilled readers group. Loadings are high (all above .61). The auditory-vocabulary factor emerged as F2 within the group of students with no SRDs. This F2 accounted for 17% of the variance. It included all auditory comprehension measures and the vocabulary measure. Within the group of students with SRDs, F2 (the fluency-orthography factor) included all reading fluency and two orthographic skills measures. It accounted for 19% of the variance. By means of EFA, it was possible to examine which kinds of knowledge come into play in which measurements. Furthermore, it can be inferred that fluency and orthography skills, on the one hand, and auditory skills and vocabulary size, on the other hand, acquire different kinds of knowledge, as they load separately on different factors. In a cross-group analysis, we can observe that the first two factors exchanged positions. Factor 1 was significantly different between students with no SRDs and those with SRDs. This finding could have an impact on EFL reading competence within the group of students with SRDs. The most powerful factor (F1) that emerged within the group of students with no SRDs had a pattern (fluency-orthography pattern) that is associated with fluent and accurate reading. However, the most powerful factor (F1) within the group of students with SRDs did not have this pattern. This might indicate that well-developed fluency and orthography skills in EFL are important for efficient EFL reading competence, and that the group of students with SRDs lacks these skills. Difficulties in reading fluency and orthographic skills could affect the acquisition of EFL reading in the group

Assessment Accommodations in FL Reading Competence

201

of students with SRDs. Thus, these students would have difficulties with EFL reading, resulting from deficient general reading fluency skills and orthographic skills. This is also evident from the result of a t test (t = 10.01, p < 0.01) comparing fluency–orthography factor scores of the cohort of all students. The t test showed that, on average, students with no SRDs achieved higher attainment on all measures compared to students with SRDs. These results confirm those of other research on Slovene EFL students (Erbeli and Pižorn 2012, 135), showing that skilled EFL readers were faster and more accurate in EFL reading and were better at spelling words than less-skilled EFL readers. Table 1 also shows that measures of reading fluency (decoding and accuracy, word and contextual) and measures of orthographic skills (letter choice and sight spelling) loaded onto one factor. In the group of students with no SRDs, this factor was the first factor, the most important factor. However, in the group of students with SRDs, this factor was only the second factor. The finding that fluency and orthography measures loaded onto one factor is in line with previous research results. Sparks et al. (2011, 267) have shown that “processing speech sounds and learning sound–symbol relationships may be a modular skill.” This finding may have direct implications for understanding the nature of the slowness of students with SRDs in EFL reading and in learning to recognize and spell new words. In students with no SRDs, repeated exposure to words leads to the development of interconnections between the visual and the verbal modalities. However, in students with SRDs, when visual stimuli were presented for recognition or recall, such as in the measure letter choice, they were less likely to evoke an interdependent network of visual and verbal associations. It is, therefore, with this key factor of EFL reading identified, i.e., the fluency-orthography factor, where further substantial help for students with SRDs should be pursued. It is in the position of this factor that the two groups differ. Thus, accommodations and modifications in assessment within this factor should be pursued for students with SRDs, in order to successfully overcome their difficulties.

6.2 Assessment accommodations and modifications In order to answer the second research question, appropriate modifications and accommodations in teaching and assessing EFL reading competence for students with SRDs are proposed. EFA revealed that one distinct factor underlying EFL reading competence structure differs between both groups of students, i.e., the fluency-orthography factor.

202

Chapter Ten

Prior to conducting appropriate modifications and accommodations in assessing EFL reading competence for students with SRDs, three distinctive features of assessment for students with SRDs must be acknowledged: 1. Accommodations and modifications are matched to the students’ needs. The outcome of our study implies that Slovene EFL students with SRDs need modifications in the areas of decoding, fluency and orthography skills. 2. In an effort to obtain the most accurate evaluation of students, more than one type of assessment should be used. 3. If assessment is to provide information on the extent to which students are making progress, it is critical that there be a close match between instructional and assessment accommodations. IEP teams make decisions about both kinds of accommodations, and should do so based on need (Kormos and Smith, 2012). In line with these features, the following modifications are proposed: 1. As is evident from the EFA results, students with SRDs lag behind their peers with no SRDs in fluency skills. Decoding and fluency skills can be developed through extensive reading and revised repeated readings of relatively short passages (50–250 words). Poetry is short, highly patterned, and predictable, and it contains letter patterns that can be adapted for building students’ fluency. Assessment tasks should be modified for students with SRDs such that these students are asked to produce short poetry presentations, dramatic performances, letter groups and words on lists for beginning readers. Fluency assessments should not include long unknown texts for students with SRDs. 2. In order to develop fluency through extensive reading, one of the most widely recommended exercises is one in which a student searches as quickly as possible for the target word from among distracters. According to Crawford, the majority of word recognition exercises in textbooks related to EFL fluency reading resemble this exercise (Crawford 2005, 36). Assessment modifications in timing could be implemented in this exercise. Rather than being assessed on fluency at intervals (once a month), students with SRDs should be assessed on the spot (every day). This sort of assessment, i.e., continuous assessment, gives students

Assessment Accommodations in FL Reading Competence

3.

4.

5.

6.

203

with SRDs an opportunity to obtain continuous, corrective feedback and appropriate support on a daily basis. In order to develop fluency, teachers can experiment with supplements to text reading, such as word and sub-word study and word lists, as well as with the proportion of time devoted to textlevel and word-level practice. Again, the timing of assessment tasks can be modified. Students with SRDs can be asked to show their effort at daily vocabulary quizzes. Depending on the individual profile, students with SRDs might sometimes even need modifications in learning objectives. One such modification in learning objectives in EFL reading allows students with SRDs to read only shorter texts. Textual modifications are also performed, i.e., modifying written EFL input, which is quite a common practice in many pedagogical settings. Textual modifications include simplified written input, textual input enhancement, and glossing to reduce the processing demands on EFL students by making the input more comprehensible (enabling students to read more quickly and accurately). Research evidence supports the effectiveness of multi-sensory structured learning with dyslexic students (an approach that promotes direct and explicit teaching of the EFL linguistic system) (Kormos and Smith 2012, 163). Therefore, when students with SRDs are assessed on grammar, for instance, they should be given exercises that support multi-sensory structured learning, such as modified grammatical exercises including the use of drills. As is evident from the EFA results, students with SRDs lag behind in orthographic skills. A familiarity with intra-word orthographic regularities is essential for fluent EFL reading. Strategies to develop and reinforce orthographic skills in class can include word searches, anagrams and peer proofreading. The kind of tasks used in class should also be reflected in assessments. If spelling problems nevertheless remain, assessment modifications could include disregarding spelling errors, using mnemonic aids for spelling or using spellcheckers (Kormos and Smith 2012).

Research on the relationship between EFL reading competence in groups of students with SRDs and targeted accommodations and modifications is worth expanding. One way to pursue it is by considering the limitations of the present study. We need to further examine the extent to which the validity of assessments is enhanced when specific types of

204

Chapter Ten

accommodations are matched to specific sub-skills of EFL reading competence. The findings of this study show that reading fluency, orthographic skills and other EFL linguistic and cognitive factors underlie the structure of EFL reading competence in Slovene 7th grade students. However, these results should be viewed as preliminary and requiring replication. Other factors (EFL exposure, metalinguistic awareness) should be included in future studies in order to examine the validity of this structure of EFL reading competence more fully and to explain individual differences more precisely.

7. Conclusion Despite the limitations of the present study, overall the results are meaningful both theoretically and practically. The results show that Slovene students with SRDs have a different EFL reading competence structure than students with no SRDs, and that they are characterized by poorer fluency and orthography-related processes. Poorer results for these processes may be indicative of the students’ worse overall EFL reading performance compared to students with no SRDs. In view of these results, researchers and educators might consider implementing assessment accommodations and modifications in specific problem areas, as well as designing instructional programs carefully.

Works cited Abedi, Jamal, Carolyn Huie Hofstetter, and Carol Lord. 2004. “Assessment Accommodations for English Language Learners: Implications for Policy-Based Empirical Research.” Review of Educational Research 74:1–28. Alderson, Charles J. 1984. “Reading in a Foreign Language: A Reading Problem or a Language Problem?” In Reading in a Foreign Language, edited by Charles J. Alderson and A. H. Urquhart, 1–24. Harlow, UK: Longman. Andersson, Ulf. 2010. “The Contribution of Working Memory Capacity to Foreign Language Comprehension in Children.” Memory 18:458–72. Bernhardt, Elisabeth B., and Michael L. Kamil. 1995. “Interpreting Relationships between L1 and L2 Reading: Consolidating the Linguistic Threshold and the Linguistic Interdependence Hypotheses.” Applied Linguistics 16:15–34.

Assessment Accommodations in FL Reading Competence

205

Bosse, Marie-Line, and Sylviane Valdois. 2009. “Influence of the Visual Attention Span on Child Reading Performance: A Cross-Sectional Study.” Journal of Research in Reading 32:230–53. Brisbois, Judith E. 1995. “Connections between First- and SecondLanguage Reading.” Journal of Reading Behavior 27:565–84. Carr, Thomas H., and Betty A. Levy, eds. 1990. Reading and Its Development: Component Skills Approaches. San Diego: Academic Press. Carrow-Woolfolk, Elisabeth. 1999. Test for Auditory Comprehension of Language. Austin, TX: Pro-Ed. Clarke, Mark A. 1980. “The Short Circuit Hypothesis of ESL Reading: Or When Language Competence Interferes with Reading Performance.” In Interactive Approaches to Second Language Reading, edited by Patricia L. Carrell, Joanne Devine and David E. Eskey, 114–124. New York: Cambridge University Press. Coady, James. 1979. “A Psycholinguistic Model of the ESL Reader.” In Reading in a Second Language: Hypotheses, Organization, and Practice, edited by Ronald Mackay, Bruce Barkman and R. R. Jordan, 5–12. Rowley, MA: Newbury House. Crawford, Michael. 2005. “Adding Variety to Word Recognition Exercises.” English Teaching Forum 43:36–41. Erbeli, Florina, and Karmen Pižorn. 2012. “Reading Ability, Reading Fluency and Orthographic Skills: The Case of L1 Slovene English as a Foreign Language Students.” CEPS Journal 2:119–39. Erbeli, Florina and Karmen Pižorn. 2013. “The Structure of Reading Competence in English as a Foreign Language.” Journal of Contemporary Educational Studies 1:44–58. Francis, David J., Catherine E. Snow, Diane August, Coleen D. Carlson, Jon Miller, and Aquiles Iglesias. 2006. “Measures of Reading Comprehension: A Latent Variable Analysis of the Diagnostic Assessment of Reading Comprehension.” Scientific Studies of Reading 10:301–22. Grabe, William. 2009. Reading in a Second Language: Moving from Theory to Practice. Cambridge, UK: Cambridge University Press. Hammill, Donald D., Lee J. Wiederholt, and Elisabeth A. Allen. 2006. Test of Silent Contextual Reading Fluency. Austin, TX: Pro-Ed. Hollenbeck, Keith, Gerald Tindal, and Patricia Almond. 1998. “Teachers’ Knowledge of Accommodations As a Validity Issue in High-Stakes Testing”. The Journal of Special Education 32:175–83.

206

Chapter Ten

Kato, Shigeo. 2009. “Suppressing Inner Speech in ESL Reading: Implications for Developmental Changes in Second Language Word Recognition Processes.” The Modern Language Journal 93:471–88. Koda, Keiko. 2010. Insights into Second Language Reading: A CrossLinguistic Approach. New York: Cambridge University Press. Kormos, Judit, and Anne Margaret Smith. 2012. Teaching Languages to Students with Specific Learning Differences. Bristol, UK: Multilingual Matters. Martin, Nancy A. 2006. Test of Visual Perceptual Skills. Novato, CA: Academic Therapy Publications. Mather, Nancy, Donald D. Hammill, Elisabeth A. Allen, and Rhia Roberts. 2004. Test of Silent Word Reading Fluency. Austin, TX: ProEd. Mather, Nancy, Rhia Roberts, Donald D. Hammill, and Elisabeth A. Allen. 2008. Test of Orthographic Competence. Austin, TX: Pro-Ed. Nation, Paul, and David Beglar. 2007. “A vocabulary size test”. The Language Teacher 31:9–13. Noort, van den Maurits W., Peggy Bosch, and Kenneth Hugdahl. 2006. “Foreign Language Proficiency and Working Memory Capacity.” European Psychologist 11:289–96. Reynolds, Cecil R., and Arthur M. Horton, Jr. 2006. Test of Verbal Conceptualization and Fluency. TX: Pro-Ed. Shiotsu, Toshihiko. 2011. Components of L2 Reading: Linguistic and Processing Factors in the Reading Test Performances of Japanese EFL Learners. Cambridge, UK: Cambridge University Press. Sparks, Richard L., Jon Patton, Leonore Ganschow, and Nancy Humbach. 2011. “Subcomponents of Second-Language Aptitude and SecondLanguage Proficiency.” The Modern Language Journal 95:253–73. Thompson, Sandra, Amanda Blount, and Martha Thurlow. 2002. “A Summary of Research on the Effects of Test Accommodations: 1999 through 2001 (Tech. Rep. 34).” Last modified February 27. http://www .cehd.umn.edu/nceo/onlinepubs/Technical34.htm. Van der Leij, Aryan, and Eleni Morfidi. 2006. “Core Deficits and Variable Differences in Dutch Poor Readers Learning English.” Journal of Learning Disabilities 39:74–90. Vandergrift, Larry. 2007. “Recent Developments in Second and Foreign Language Listening and Comprehension Research.” Language Teaching 40:191–210. Yorio, Carlos A. 1971. “Some Sources of Reading Problems for Foreign Language Learners.” Language Learning 21:107–15.

CHAPTER ELEVEN L2 TEACHING AND ASSESSMENT OF UNIVERSITY STUDENTS WITH DISABILITIES KATHLEEN BRANNEN AND MARTYNA KOZLOWSKA

The population of students with disabilities enrolled in ESL courses at the Université du Québec à Montréal (UQAM) has been increasing yearly. However, teachers are often unfamiliar with the challenges these students face and with the challenges they as teachers will encounter with a heterogeneous group of students in terms of physical capacities. This chapter presents work we have done to increase awareness and help teachers adapt to this newly emerging L2 teaching context. Our work focuses on students with hearing and visual impairments. After a series of consultations with students, counsellors, and teachers, we have formulated two guidebooks for university-level L2 teachers. The guidebooks provide information on the nature of students' disabilities, university protocol regarding handicapped students, support services, adaptive technologies, classroom management, and adaptation of course content. This chapter expands upon these issues as they impact placement tests and teaching and assessment in L2 single skills classes. Focusing on single skills underscores the importance of content accessibility, validity, and uniformity of instruction. We identify common as well as divergent accommodations needed for these distinct populations of students. We also examine current legislation and recommendations within the UQAM protocol, and suggest how they need to be adapted to the L2 context.

1. Introduction In the last decade there has been a dramatic rise in the number of students with disabilities entering postsecondary education in the province

208

Chapter Eleven

of Québec, Canada. 1 Between the years 2005 and 2012 the number of students with disabilities enrolled in Québec universities has more than doubled, growing from 2360 to 5194 (AQICESH 2012). For the same period, at the Université du Québec à Montréal (UQAM) the number has gone from 218 to 714. This important increase seems to be triggered mainly by the emergence of impairments such as learning disabilities, attention deficit disorders, and mental or medical conditions which characterize emerging student populations. Interestingly, the number of students with traditional disabilities has also been steadily rising. For example, the population of university students with visual and hearing impairments in Québec has gone from 231 to 335 in this same period. At the École de Langues 2 at UQAM, the number of students with disabilities who enrol in our programs has also been growing on a yearly basis. This population has been increasingly represented by students with emergent disabilities. However, aside from dyslexia, no other disabilities have been as challenging for second language (L2) teachers as the traditional impairments of blindness and deafness. These two disabilities directly affect a student’s capacity to interact through traditional means of communication and therefore to acquire an L2 through traditional instruction and classroom practice. Even if in a university setting there are fewer students with visual impairment and hearing loss when compared to other types of handicaps, their desire for success in a classroom is no less important and must be given equal consideration. In addition, in an L2 program, the necessity for practical or procedural accommodations for such students is even more obvious than for students with some other disabilities.

2. The English program at UQAM Our English program prides itself on its versatile course selection and unique program design that distinguishes oral/aural versus written domains and incorporates two contrastive approaches to L2 teaching. On 1

Québec's education system has four levels: elementary (which also includes preschool), secondary, college and university. The colleges in Québec are known by the term CÉGEP, which is an acronym for Collège d'Enseignement Général et Professionnel. The language of instruction in the Québec school system is French. However, children with serious learning disabilities may, at the request of their parents, receive instruction in English or be exempted from ESL classes if required to facilitate the learning process (MELS 2013). 2 Throughout the paper we will be using the French title of the school which corresponds to the English term School of Languages.

L2 Teaching and Assessment of University Students with Disabilities

209

the one hand, the program makes use of a formal approach, where instruction draws upon the theoretical underpinnings of language competence and highlights relations among structural components of the language. On the other hand, a functional approach promotes instruction that targets communication skills and more fluent functioning in the L2. Four fundamental competencies are taught as single language skills: Speech Perception, Pronunciation, Critical Reading, and Syntax are given in the formal approach, while Listening, Speaking, Reading, and Writing in the functional one. Combined skills are taught in six different courses: Sounds (oral/aural), Script (written), and Grammar, all three given within the formal approach, and Conversation (oral/aural), Text (written), and Skills, given within the functional approach. The overall program structure is presented below in Table 1. Functional Approach

Formal Approach

Skills Conversation Listening Speech Perception

Text

Speaking

Reading

Writing

Pronunciation

Critical Reading

Syntax

Sounds

Script Grammar

Table 1. The English program at the École de Langues All our courses are offered at three proficiency levels: elementary, intermediate, and advanced. To obtain the English Certificate, students are required to take a specific number of courses (10 in total) at indicated proficiency levels; otherwise there are few restrictions on their course selection. This model is flexible enough to allow students to take those courses that best correspond to their learning needs, styles, and capabilities. In this sense, our program more easily accommodates students with disabilities. However, despite this relatively universal and accessible system, students with disabilities still face numerous difficulties. One step in the admission process is an obligatory placement test, which presents a particular challenge to students with visual and hearing impairment, and, consequently, to program administrators.

210

Chapter Eleven

3. The placement test The English Placement test at the École de Langues was created in 1998. Its design corresponds directly to the structure of our program and has proven a reliable tool for placing students in courses. The test is divided into nine sections, each dedicated to a separate linguistic competence: listening, reading, writing, speaking, speech perception, pronunciation, critical reading, syntax, and vocabulary. The test is computerized, and aside from the writing, speaking, and pronunciation sections, in which students’ responses are recorded as text files and sound files respectively, all other sections involve multiple choice questions. The test is timed and administered during designated testing sessions, for all new applicants to the program. Many of the test features, such as the computerized format, the independent targeting of language skills, and time allocation for each test section have contributed to the validity and efficiency of the test. However, these same features have turned out to be problematic for the assessment of students with disabilities, particularly students with visual or hearing impairments. 3 Aspects like timed responses, oral recordings, aural targets, and the visual format of questions made the test, to a certain extent, inaccessible to those students. Faced with this reality, we embarked on a gradual and still ongoing process of adjusting the test for students with disabilities. The test has been translated into Braille and transferred into Word format; the timer, which can now be set at desired time intervals, functions in audible and visual modes to indicate the remaining time; the test questions have been revised to ensure that they can be answered without bias; and, finally, special test administration conditions have been set up to accommodate students who require them. A number of other testing accommodations remain to be implemented. However, for reasons discussed in the following sections, these should be further evaluated in terms of the impact they may have on the integrity and validity of the test.

3.1 Extended test time Providing a student with extended time on a test is one of the most common and widely accepted accommodations at the university level. It is also likely that this type of accommodation does what it is meant to 3

This is true, as well, for students with dyslexia, but this will not be the focus of this chapter.

L2 Teaching and Assessment of University Students with Disabilities

211

accomplish, i.e. it changes how the constructs are measured without affecting the nature of the constructs measured. It allows for assessment that better reflects a student’s proficiency level. The effectiveness of this accommodation remains to be empirically established, for example, through the measure of differential boost, as discussed in Cook et al. (2006). Nevertheless, it does lead to assessment that better reflects a student's proficiency level. Timed testing conditions result mainly from practical considerations. Another way to level the playing field between students with and without disabilities would be to follow suggestions from Stretch and Osborne (2005). They suggested exploring possible test designs that would facilitate uniform assessment of second language skills among the entire student population, thus reducing the need for special accommodations.

3.2 Accommodation vs. modification Typically, students who come to our program from another postsecondary setting have already been accorded some type of accommodation measures, be it at the entry stage or in course-internal evaluations. These accommodations are allocated to them in accordance with the university accommodation policy as outlined in Section 4 below. Accommodations can be providing students with additional time or conducting the test in a separate location. They can also be modifications of the actual test by changing the format to Word or Braille, reading the test questions out loud, eliminating parts of the test, and allowing for the use of tools such as a dictionary or spell-checking software. However, many of these accommodations or modifications become very problematic. This happens when applied in the context of a placement test whose purpose is to evaluate the proficiency of a student in a foreign language, where language is both the object and the means of assessment. Koretz and Barton (2003) address this issue in their review of research on the inclusion of students with disabilities in large-scale assessment. One of the issues they raise is the problem of disabilities that are related to measured constructs. Another issue is how accommodations that offset poorer performance due to these disabilities may inadvertently undermine the validity of test scores by changing the nature of the proficiencies measured by the test. For example, this question arises each time students with dyslexia who, according to the university policy, are entitled to the use a dictionary or a spell-checking tool during the writing evaluation component on the test. It is also raised in the case of blind or severely visually impaired students whose Braille reading skills are inadequate for

212

Chapter Eleven

a testing context. For those students, a reading comprehension or a critical reading component of the test comes closer to a listening task if text-tospeech assistive technology is used. This issue has also been pointed out by Cahalan-Laitusis and Cook (2008) with respect to audio presentation of test content in reading assessment. More specifically, they point out the lack of agreement among researchers on whether read-aloud versions of tests constitute a mere accommodation in the test format or whether they constitute a test modification and hence give scores representing a measure of listening rather than reading comprehension. The investigations further carried out by Cook et al. (2006) among elementary school students showed no significant impact of a read-aloud test change on the underlying constructs measured by the comprehension part of the reading test they used. However, they do caution that their results are far from conclusive in the light of contradictory evidence coming from other studies. Finally, test score validity becomes an issue with respect to students with hearing impairment. Their scores on listening comprehension, speech perception, speaking or pronunciation may be negatively affected by their disability. They may also be a reflection of varying degrees of severity of impairment rather than variation in the students' competence. The question is: at what degree of impairment should evaluators judge the student to be exempt from a given test section, or to what extent do the scores need to take into account that the student’s performance may be a result of their impairment? Although some of the necessary background information is typically furnished by university counsellors, the crucial decision about how to interpret such a student’s scores remains within the domain and responsibility of the placement test administrators. In as far as the placement test plays a crucial role in the future course of a student’s studies, it remains no more than a snapshot of the student’s linguistic competence in his or her second language. However, it should be administered with sufficient attention to the type of accommodations/modifications allowed. A further investigation into effective accommodation measures will allow us to better adjust our lenses to capture the true state of the student’s knowledge. An entirely different problematic emerges when we turn to the context of the second language classroom and the accommodations necessary for students with disabilities. These students should learn, participate, and be evaluated through methodology and measures equivalent to those used for students without disabilities. As with the testing context, there are two primary sources of concern for an L2 teacher when working with students with disabilities. The first is to make sure that the material and instructions are

L2 Teaching and Assessment of University Students with Disabilities

213

equally accessible to all students, and the second is to evaluate the extent to which the student’s disability is related to the constructs at hand. The latter issue is strongly manifested for students with visual and hearing disabilities.

4. UQAM policy By adopting the Politique d’Accueil et de Soutien des Étudiantes, Étudiants en Situation de Handicap,4 the administrative council of UQAM adheres to the policies of the Government of Québec. Through this policy, UQAM recognizes the right of handicapped students to benefit from all resources on campus and in the university community in order to ensure success in their studies, and to do so without discrimination and without privilege. The office that serves as a link between students with disabilities and teachers at UQAM is the Service d’Accueil et Soutien aux Étudiants en Situation de Handicap (ASESH). On the one hand, it informs students of measures intended to promote their academic success, such as the interpreting service, alternate assessment formats, and accessibility. On the other hand, it raises awareness amongst teachers and support staff as to the realities of students living with impairments. For their part, handicapped students have the responsibility to meet with intermediaries (ASESH counsellors, program directors, etc.) in order to facilitate their integration into the university community. These students should support the intermediaries to find solutions to specific problems linked to limitations posed by their impairments. In observing UQAM’s policy on the integration and support of handicapped students, it is important to offer all students an equal opportunity to study ESL. However, because students with disabilities constitute an emerging population, L2 university instructors are often unaware of related issues. They are not usually aware of these students’ needs, the challenges faced by both students and teachers in the classroom, the available resources, the adaptive measures that can be applied, and various teaching strategies at their disposal.

5. Guidebooks To facilitate the necessary adjustment of the program to this newly emerging context, we have formulated a set of guiding principles and 4 This translates as Policy Regarding the Integration and Support of Students Living with a Handicap.

214

Chapter Eleven

recommendations for our L2 teachers of students with disabilities. We focused on two populations of students that were most perceptibly at a disadvantage compared to their classmates: those with hearing impairments and those with visual impairments. Given the distinct needs and special arrangements required for each population, our recommendations have been formulated in two separate guidebooks (Brannen et al. 2011, Brannen et al. 2012a). Throughout the process of collecting information for the guidebooks, we discovered that none of what was available addressed specifically the context of a second language classroom. A second language classroom is different from other types of university classrooms in many practical aspects and, most crucially, in that the subject taught is also the medium through which instruction is provided. We therefore used the project of the guidebooks as an opportunity to examine ways in which L2 instruction can be revised to accommodate L2 students with disabilities. We conducted a series of consultations with former and current handicapped students and with L2 teachers who have had experience teaching such students. In addition, we consulted advisors working in Québec universities whose role is to assist students with disabilities. We tried to get the best picture of the practical, pedagogical, and conceptual implications of visual and hearing impairments in the process of L2 acquisition. Our goal is to offer quality education to all students in our program, while respecting the minimal content and course objectives as described in the master course plans.5 Through the guidebooks, our instructors are informed of University policy regarding handicapped students as well as the rights and responsibilities of both the student and the educator. The guidebooks further provide information about different types and degrees of visual and auditory impairment and what challenges students living with these types of handicaps encounter. Large portions are dedicated to providing tips on classroom management and suggesting particular teaching strategies, adaptive measures, and exam accommodations. Teachers are provided with clarification as to their role as educators, the role of the handicapped students, as well as that of any resource persons involved in the integration of the student into the L2 classroom, e.g. a sign-language interpreter. These guides help instructors understand how the handicapped student perceives the world, and how they as teachers can adopt attitudes that foster a healthy and functional relationship with the student. 5

Each of our courses has a master course plan (plans de cours cadre), the objective of which is to inform students and teaching personnel as to the general goals, content, and evaluation methods of that course.

L2 Teaching and Assessment of University Students with Disabilities

215

Although the two guides point to several common means and procedures for accommodating students with impairments—visual or auditory—it became apparent to us that these two populations differ in the nature of the accommodations they require. In Brannen et al. (2012b), we present similarities and highlight differences in the type of challenges and accommodations. Teachers need to consider these issues when preparing for classes that include students with hearing versus visual impairments. In both instances, course planning should take place well in advance. The material should be available to the student, note-taker, or interpreter long enough before the class. The course content should be structured in a way to avoid improvised or spontaneous activities. Evaluations need to allow for additional time, and, above all, the teacher should create an inclusive atmosphere between him- or herself, the handicapped student, and the rest of the class. Regarding differences between visual and hearing impaired students in a L2 classroom, the most evident contrast is between the linguistic modalities in which they operate, i.e. oral versus gestural. This fundamental difference implies numerous others, as outlined in Table 2. Type of accommodation

Students with visual impairment

Visual support in class: Technical support in class:

To be avoided

To be promoted

Used for direct access to course material

Used for communication with the teacher and classmates

Material adaptation:

Text-to-voice

Voice-to-text

Assistive technology

Adaptation and modification in methodology of instruction

Principal modality of accommodation:

Students with hearing impairment

Table 2. Differences in types of accommodation

6. Classroom context Much of the information provided in our guidebooks can be applied to any university course. However, as mentioned above, the L2 classroom usually differs in several respects compared to classes in other domains of study. Firstly, the L2 classroom involves much group work, discussion, conversation, and other types of spontaneous interaction. This dynamic atmosphere is often not found in other types of university courses, especially those of the traditional lecture format. This presents some real challenges to the teacher and the student.

216

Chapter Eleven

6.1 Classroom dynamics One of these challenges is how best to integrate the handicapped student into the classroom culture. This may mean overcoming prejudice on the part of other students and sometimes even on the part of the handicapped student. For example, some deaf students who closely identify themselves with the Deaf culture may have an affective block when it comes to interacting with other students. This may often be due to previous experiences that may have negatively affected their perception of hearing people. Therefore, the teacher should find ways to overcome any reticence, through education and with activities that encourage familiarization. Spontaneity will be reduced in the L2 classroom with a visually or hearing impaired student. Since course materials should be provided in advance, teachers have less freedom to improvise during class. The classroom with a hearing or visually impaired student will necessarily be more structured than it otherwise might be. The teacher may find that pair work is more manageable than larger group work in such situations. In the case of a visually impaired student, communication through body language and other non-verbal cues must be conveyed through the auditory channel. Any comings or goings must be signalled. During discussions and group work, turn-taking must be controlled: in the case of the hearing impaired student, this is particularly important, especially if an interpreter is present. To promote fuller engagement of the deaf student in classroom discussion, comments from other students should be made available in written form or through sign language. The latter option highlights the extra demands put upon the sign-language interpreter in the L2 classroom, something which the teacher needs to take into consideration.6 Visually and hearing impaired persons have restricted access to certain sources of information, a situation that is particularly acute for the Deaf. The availability of information through channels such as mass media and ambient conversation is severely limited. This means that deaf students have less exposure to a variety L2 vocabulary items, idiomatic expressions, and syntactic structures, a point to which we will return later. It also means that they may feel they are less knowledgeable than hearing people, which in turn makes them more hesitant to ask questions and express their opinion during class. The L2 teacher needs to realize this, 6

Another burden for interpreters in the UQAM context is that they must first translate from English into French for themselves before interpreting in sign language.

L2 Teaching and Assessment of University Students with Disabilities

217

and hopefully find a means in which to help the student overcome any shyness. Another aspect that is an integral part of L2 classroom dynamics is the language of instruction. Translation into the first language (L1) is usually avoided. The reasons for this are twofold. First, standard L2 teaching methods explicitly discourage the use of the L1 in the classroom. Although this approach has been questioned and is now gradually changing, it remains a tenet of many L2 teachers today (Cook 2008). The second reason for the avoidance of the L1 in the L2 classroom is of a practical nature, and one that applies to ESL courses at UQAM. Our student population is quite heterogeneous in terms of native language background. Although we are within a French university, the percentage of L1 French students in our classrooms is relatively low. This means that using L1 translation as a teaching tool is largely impracticable in our classrooms.7 However in a case of a deaf student, who relies upon an LSQ interpreter, translation is unavoidable. Implications of this aspect of classroom dynamics will be discussed below.8

6.2 Classroom assessment Learning and assessment in second language courses differ from learning and assessment in most other academic fields in how they draw on metalinguistic knowledge. This creates particular challenges with respect to the assessment of students with disabilities in the SL classroom. Our goal should be to ensure that tests are fair for all students. This means offering accommodations to students with disabilities. However, we should strike a balance between offering accommodations and ensuring fairness in assessment. The ETS Standards for Fairness and Quality states that ‘fairness requires that construct-irrelevant personal characteristics of test takers have no appreciable effect on test results or their interpretation’ (Xi 2010, 150). This entails that instructors should take care to construct tests that are sensitive to the students' backgrounds, avoiding elements that could prejudice the results. This leads to the question of how to interpret construct-irrelevant with respect to students with disabilities. How does the fact that students belong to Blind culture or Deaf culture affect their performance? How does lack of experience with the visual or auditory 7 Even though students are required to have achieved a functional level of French before they enrol in the ESL program at UQAM, translation into French does not have the same impact as it does for students whose L1 is French. 8 LSQ is the abbreviation for Langue des Signes Québécoise, Québec French Sign Language.

218

Chapter Eleven

world impact a student's ability to perform on a second language test? These questions should be kept in mind in an effort to ensure that tests do not discriminate against any student from any cultural background. When language is the object of assessment, the components include vocabulary, syntax, and pronunciation. Grammatical structures, lexical specificity and complexity, and spelling are all components that are considered important in writing assessment (Bae and Bachman 2010) as well as in reading. In some classroom tests, teachers may require that students memorize vocabulary items, and thus forbid recourse to dictionaries. This might present difficulty, especially for deaf students. Because of their limited exposure to spoken language and reliance on LSQ interpretation, these students often have a poorer lexicon in their L2 (French) and in their third language (English). As such, they are at a disadvantage compared to non-hearing-impaired ESL students. Syntax is also a problem for the deaf student. The linear order of spoken and written English is different from the spatial configuration of LSQ. This presents a particular challenge in learning the correct usage of items such as function words. A non-impaired student can usually make reference to similar structures in their L1, but the student with LSQ as their L1 cannot. In contrast, deaf students tend to be better spellers than hearing students because they are not influenced by pronunciation spelling.9 Also, the fact that the deaf student has access to L1 translation may put him/her at an advantage over other students. To what extent should we consider these factors to be relevant or irrelevant to the constructs we measure? The issue may be larger than this however. The distance between English and LSQ may be seen simply as a difference in degree rather than in kind. For example, Chinese does not share a Latin-based vocabulary as French and English do, and the pronunciation, syntax, and morphology of Chinese is likewise quite far removed. The orthographic system is ideographic, without reference to pronunciation, and as such, must be memorized (Caplan 1987). Memorization perhaps takes place in the same way as the deaf student must memorize lexical items in English. The question of whether and how to offer equivalent instruction and fair assessment to all students is obviously one that needs more study. Recent developments in the concept of universal instructional design, which aims to provide accessibility for all, may help resolve many of these problems by reducing the need for adaptive measures.

9

Suzanne Villeneuve (Groupe de recherche sur la LSQ et bilinguisme sourd, UQAM), in an interview with the authors, March 2012.

L2 Teaching and Assessment of University Students with Disabilities

219

In considering the special context of integrating and accommodating students with disabilities into the L2 classroom, the unique structure of the ESL program at UQAM presents a further challenge. Unlike some other L2 programs, many of our courses focus on single skills, an issue to which we turn in the next section.

7. Single skills The English program of the École de Langues is based upon a pedagogical model that offers mixed and single skills courses (see Table 1). Regarding students with disabilities, the focus on single skills underscores the importance of issues such as content accessibility and validity of class assessment.

7.1 Reading and writing classes For students with visual impairment, our single skill reading classes obviously pose a problem. Most students use text-to-speech software to circumnavigate this problem, while some students may use Braille. Braille is extraordinarily difficult to master, particularly as an adult (Wong et al. 2011). This is why few visually impaired students use this option. But for those students who do use Braille, it is not clear how closely this resembles reading as is carried out by students without visual impairment. This is further complicated by the fact there are two types of Braille: abridged and complete. In contrast to the abridged form, which incorporates abbreviations, complete Braille resembles sighted writing in the sense that it is presented linearly and letter by letter. Readers move the fingers from left to right and from top to bottom on the brailled page, as in sighted reading. Tactile graphics can help readers to find information in a text or read graphs. Furthermore, research has shown that Braille activates the visual cortex similarly to sighted reading (Büchel et al. 1998). However, English Braille uses phonograms, i.e. it is sound-based, and this makes it different from traditional English orthography in that the spelling-sound correspondence is closer. In conclusion, the technique of reading Braille does bear similarities to sighted reading, even if the orthographic systems differ. Text-to-speech software, on the other hand, clearly implies a change of modality, and as such, a reading class actually becomes a listening class for the visually impaired student using this technology. This has implications for assessment. In terms of content validity, it is questionable whether we measure reading skills in these cases. In terms of predictive

220

Chapter Eleven

validity, it remains to be determined if practice with listening leads to better reading skills. There is some evidence for a relationship between reading and listening. Song (2008) investigated whether listening and reading involve the same competencies. It was found that listening and reading are made of the same sub-skills: both involve similar comprehension processes. However, decoding was different in each modality. Research by Lund (1991) found that readers retained more information than listeners, but that listeners retained more main ideas, while readers retained more details. Park (2004) confirmed these findings that listeners were better at inference and synthesis, while readers were better with factual and local information. These studies were done with sighted participants, so it is unknown whether the results can be applied to visually impaired population. However, they do indicate some similarities and differences between the two modalities. Technology may also serve to make a listening task more similar to reading. One strategy that sighted readers are taught to use is skimming a text for keywords and information. Some new applications available to the visually impaired, such as the Victor Reader combined with Daisy Reader, incorporate hyperlinks that allow listeners to return to a certain part in the audio to retrieve information. The ability to navigate an audio text certainly appears to bring the listening task closer to a reading task. While a reading class is an obvious challenge for the blind student, as mentioned in section 6.1 it also poses a problem for the deaf student. Many deaf students are not as proficient in reading as the hearing population simply because they have not had as much exposure to reading in their educational experience. Many deaf children of hearing parents have not been exposed to LSQ before they begin school at 6 years old. Given the importance of these first 6 years of life in the development of language, these individuals lack a fully developed L1 (Marschark 2009) This puts them at a disadvantage when learning French as an L2 and of course English as an L3. In Québec, LSQ-French bilingual education has only been freely available in the past decade or so, and, up until recently, ESL courses have not been encouraged for deaf students through the preuniversity years. It has only been within the last decade that deaf students are required to attend ESL classes at primary, secondary, and CÉGEP levels.10 10

Lyse, Lapointe and Stéphanie Campeau (Ministère de l'Éducation, du Loisir et du Sport) e-mail messages to Suzanne Villeneuve (Groupe de recherche sur la LSQ et bilinguisme sourd, UQAM), November 27, 2012.

L2 Teaching and Assessment of University Students with Disabilities

221

Reading in the hearing population involves knowledge of pronunciation and phonological decoding, which is thought to help develop good reading skills (Tallal 2012). Research has found that deaf people also make use of phonological decoding in reading. They can access phonological information through speech-reading, and kinesthetic feedback through finger spelling and speech movements (Morere 2011). However, the effectiveness of phonological decoding is not uncontroversial (Bélanger et al. 2011, Morere 2011). Nonetheless, the possibility of phonological decoding for deaf students, and text-to-speech software and Braille for blind students are ways in which the written code may be accessed. But more research is required to determine predictive validity in the L2 context; in other words, the extent to which these techniques contribute to better reading skills for visually and hearing impaired students.

7.2 Listening and speaking classes The listening courses at UQAM aim to improve the student's range of vocabulary and idiomatic expressions. They also aim to familiarize them with different registers of speech, and with different dialects of English, to identify main and secondary ideas in oral discourse, and to develop listening strategies. Listening courses no doubt pose problems for deaf students.11 We have not yet had the experience of a deaf student in one of our listening courses. Nevertheless, if a deaf student were to participate in such a course, he or she might need the support of sub-titles and transcripts for video and audio materials. An oralist might be able to participate with recourse to an interpreter who uses transliteration provided the videos clearly show the actors' articulators.12 Nonetheless, for the deaf student taking a listening course, there will probably be a change in modality from aural to visual, either through LSQ, lip reading, gestures, or reading. This presents problems with respect to transmitting idiomatic expressions, distinguishing formal and informal speech, and of course different dialects of the target language. However, identifying main and secondary ideas would still be a possible skill to develop, as well as certain listening strategies, such as predicting upcoming content based on key words. 11 The Speech Perception courses at UQAM focus on improving the student's discrimination of speech sounds and stress patterns, and as such would be inaccessible to most deaf students. 12 Recourse to transliteration raises the issue of translation, as discussed in Section 6.1.

222

Chapter Eleven

Speaking and pronunciation will pose a problem, especially for students who are deaf from birth. However, there is likely to be more demand for these types of courses than for listening classes, as deaf students seek to improve their articulation. Some techniques used in articulatory training could be used by a deaf student since they have access to proprioceptive feedback. Training on aspects of articulation that are less visible or perceptible may prove more difficult, as is the case for vowels, intonation, and stress; however, recourse to speech analysis software proves helpful in these cases. Deaf students with cochlear implants or hearing aids may benefit to some extent from aural feedback. Nonetheless, the question remains as to how we evaluate a deaf person's pronunciation. For a teacher who has little or no experience teaching deaf students, it may be difficult to know what standard to apply. This discussion has shown that the particularities of single skills classes in L2 programs pose challenges to students with disabilities and teachers conducting such classes. It is clear that more research is needed on the impact that different modalities may have on L2 acquisition and how these modalities are related. We may find that some modalities are simply not equivalent, and therefore we may have to accept that the input provided for students with disabilities will be impoverished and teaching objectives may need to be adjusted. However, we should also keep in mind that the purpose of the classes is to enable students to function in their L2 (or L3) in the real world. Therefore, it is important that teachers be flexible in their teaching and assessment methods. In this way, they will provide students with the means to make the most of their capabilities and to enable them to communicate through the different channels of reading, writing, listening, and speaking.

8. Conclusion The overall issue of instruction and assessment of skills that are directly influenced by disabilities is very complex. In this chapter, we have discussed issues that apply mainly to the L2 classroom, and are not considered in general university protocols. We feel it is important that language schools fill this gap by providing appendices to the protocol, which address concerns specific to their programs. We have outlined some of the considerations that need to be taken into account with respect to placement and proficiency testing for students with disabilities. These suggest that test designers need to seek a balance between adjusting assessment tools to make them accessible to a general university

L2 Teaching and Assessment of University Students with Disabilities

223

population, including handicapped students, and maintaining rigorous application of testing measures that reliably assess students’ knowledge. We have also highlighted some of the challenges that students with impairments face in a second language classroom and the need for teachers to adapt to the needs and capabilities of these students. It is important to stress, however, that as much as we generalize about potential problems and suggest solutions, each student needs to be considered individually. All students come into the L2 learning process with their own learning styles and levels of metalinguistic knowledge, and in the case of students with disabilities, different types and degrees of impairment. Teachers should be supported and guided in their adaptation to these emerging student populations. They should be encouraged to find teaching and assessment methods versatile enough. In this way, they will meet the needs of groups that include handicapped students and enable them to have confidence in the validity and efficacy of their instruction.

Works cited Association Québécoise Interuniversitaire des Conseillers aux Étudiants en Situation de Handicap (AQICESH). 2012. “Statistic Reports.” http:// aqicesh.ca/association-aqicesh/statistiques-et-rapports. Bae, Jungok, and Lyle F. Bachman. 2010. “An Investigation of Four Writing Traits and Two Tasks across Two Languages.” Language Testing 27:213–234. Bélanger, Nathalie N., Shari R. Baum, and Rachel I. Mayberry. 2011. “Reading Difficulties in Adult Deaf Readers of French: Phonological Codes, Not Guilty!” Scientific Studies of Reading 16:263–285. Brannen, Kathleen, Jaime Demperio, Martyna Kozlowska, and Jane Nicholls. 2011. “Manuel à l’intention des enseignants d’un étudiant avec une déficience visuelle.” http://www.langues.uqam.ca/ang/ enseignant.htm Brannen, Kathleen, Martyna Kozlowska, and Beau Zuercher. 2012. “Manuel à l’intention des enseignants d’un étudiant avec une déficience auditive.” http://www.langues.uqam.ca/ang/enseignant.htm Brannen, Kathleen, Martyna Kozlowska, and Beau Zuercher. 2012. “Manuel à l’Intention des Professeurs d’Étudiants en Situation de Handicap dans un Cours de Langues Seconde.” Paper presented at the annual conference of AQICESH, Montréal, Québec, June 6–8. Büchel, Christian, Cathy Price, R.S.J. Frackowiak, and Karl Friston. 1998. “Different Activation Patterns in the Visual Cortex of Late and Congenitally Blind Subjects.” Brain 121:409–419.

224

Chapter Eleven

Cahalan-Laitusis, Cara, and Linda L. Cook. 2008. “Reading Aloud as an Accommodation for a Test of Reading Comprehension.” http://www. ets.org/Media/Research/pdf/SPOTLIGHT1.pdf. Caplan, David. 1987. Neurolinguistics and Linguistic Aphasiology. Cambridge: Cambridge University Press. Cook, Linda, Daniel Eignor, Jonathan Steinberg, Yasuyo Sawaki, and Frederick Cline. 2006. “Using Factor Analysis to Investigate the Impact of Accommodations on the Scores of Students with Disabilities on English Language Arts Assessment.” Journal of Applied Testing Technology 10:187–208. Cook, Vivian. 2008. Second Language Learning and Language Teaching. London: Hodder Education. Koretz, Daniel M., and Karen Barton. 2003. Assessing Students with Disabilities: Issues and Evidence. CSE Technical Report 587. University of California. Los Angeles, CA. Lund, Randall J. 1991. “A Comparison of Second Language Listening and Reading Comprehension.” The Modern Language Journal. 75:106– 204. Marschark, Marc. 2009. “Are Deaf Student’s Reading Challenges Really about Reading?” American Annals of the Deaf 154:357–370. Ministère de l'éducation, loisir, et du sport (MELS). 2013 “Régime pédagogique de l'éducation préscolaire, de l'enseignement primaire et de l'enseignement secondaire”, article 23.2 . http://www2.publications duquebec.gouv.qc.ca/dynamicSearch/telecharge.php?type=2&file=//I_ 13_3/I13_3R8.htm Morere, Donna A. 2011. “Reading Research and Deaf Children.” NSF Science of Learning Center on Visual Language and Visual Learning Research Brief 4. Park, Gi-Pyo. 2004. “Comparison of L2 Listening and Reading Comprehension by University Students Learning English in Korea.” Foreign Language Annals 37:448–458. Song, Min-Young. 2008. “Do Divisible Subskills Exist in Second Language (L2) Comprehension? A structural equation modeling approach.” Language Testing 25:435–464. Stretch, LoriAnn S., and Jason W. Osborne. 2005. “Extended Time Test Accommodation: Directions for Future Research and Practice.” Practical Assessment, Research and Evaluation 10:1–8. Tallal, Paula. “Neuroscience, Phonology and Reading: The Oral to Written Language Continuum.” Accessed November 26, 2012. http://www.childrenofthecode.org/pvid/tallal/part1.htm.

L2 Teaching and Assessment of University Students with Disabilities

225

Wong, Michael, Vishi Gnanakumaran, and Daniel Goldreich. 2011. “Tactile Spatial Acuity Enhancement in Blindness: Evidence for Experience-Dependent Mechanisms.” Journal of Neuroscience 31:7028–7037. Xi, Xiaoming. 2010. “How Do We Go about Investigating Test Fairness?” Language Testing 27:147–170.

PART III: VOICES FROM L2 EXAMINATION BOARDS

CHAPTER TWELVE ASSESSING STUDENTS WITH DISABILITIES: VOICES FROM THE STAKEHOLDER COMMUNITY LYNDA TAYLOR AND HANAN KHALIFA

This chapter considers some of the theoretical and practical issues associated with special arrangements (or test accommodations) offered by one international language test provider to Second Language Learners (SLLs) with disabilities. The innovative contribution of this paper will be to explore current issues and challenges in this area by examining the perspectives of four different stakeholder groups involved in accommodated language tests for test takers with disabilities. Such groups include not only the test takers themselves, but also the different individuals and agencies who are directly or indirectly involved. Their involvement relates to responding to an initial request for accommodation (e.g. exam board personnel), preparing suitable test materials (e.g. test writers), setting up the test venue appropriately (e.g. test centre administrators) and conducting the modified test and processing the results (e.g. examiners). The chapter draws on both qualitative and quantitative data recently gathered and analysed by the examination board, using documentary review, questionnaires and semi-structured interviews. The aim was to investigate the match between policy and practice, and to identify any areas meriting further attention. The chapter contributes to the literature within the relatively under-researched area of test impact as it concerns language assessment for students with disabilities.

1. Background and context Thanks to improved access to education and widening participation at all levels, recent years have seen significant growth in the number and diversity of students with disabilities who are learning a second or additional language. This includes both learners with physical disabilities,

Chapter Twelve

230

such as hearing or visual impairment, and those with cognitive or learning difficulties, such as Down’s syndrome or dyslexia. As a result, demand has increased for appropriate language testing and assessment provision, particularly from large-scale providers of nationally and internationally certificated examinations. Making appropriate assessment provision for learners with disabilities typically entails some degree of modification to standard testing material and administration—commonly referred to as test accommodations. The general principle underlying the provision of test accommodations is that standard test content, format or administration should be modified to meet the specified needs of an individual test taker, or group of test takers, in the interests of fairness and equity. Abedi (2008) AERA/APA/NCME (1999), Fulcher and Davidson (2007) and Taylor (2012) provide helpful discussion of the ethics, principles and practice of accommodations in relation to both tests of content knowledge and tests of language proficiency. By providing accommodations, test providers seek to make appropriate adjustments or arrangements for a test taker to take the test on an equal footing or level playing field with other candidates, as far as possible. Cambridge English Language Assessment (formerly known as University of Cambridge ESOL Examinations, or Cambridge ESOL) is one such test provider. It offers English as a second language tests across the proficiency continuum, as well as across differing domains, e.g. general, business English, and differing age-ranges, e.g. young learners, adults. In its provision for test takers with disabilities, the overarching aim is to do everything possible to meet the special requirements of such candidates whilst ensuring that the necessary assessment objectives are still met: As an examination board, Cambridge ESOL is committed to ensuring that, as far as practicable, candidates are not excluded from taking an examination because of a disability or learning difficulty, whether temporary or permanent.1

Cambridge English Language Assessment uses the term Special Arrangements (SA) to refer to the modified processes and procedures which are intended to facilitate attainment in the skill being assessed. At the same time, it ensures that candidates receiving such provision are not given unfair advantage over other candidates and that the user of the test certificate is not misled about the nature of the candidate’s attainment. As with most other responsible examination providers, typical categories of 1

Khalifa and Weir 2009, 2.

Assessing Students with Disabilities

231

test accommodations or SA offered by Cambridge English Language Assessment include: -

test forms for test takers with hearing/speaking difficulties (e.g. deafness, speech impediment), test forms for test takers with visual impairments (e.g. blindness), test forms for test takers with specific learning difficulties (e.g. dyslexia, attention deficit disorder), test forms for test takers with other kinds of physical disability (e.g. broken wrist in plaster, wheelchair user).

Some examples of the most commonly requested Special Arrangements are mentioned later in this chapter (see Table 1). However, details of the many different types of provision offered by Cambridge English Language Assessment can be found on the examination board’s website. The wide variety of accommodations offered by Cambridge English Language Assessment is also fully listed and discussed in a series of four published volumes on its writing, reading, speaking and listening tests (Shaw and Weir 2007; Khalifa and Weir 2009; Taylor 2011; Geranpayeh and Taylor 2013). Each of these skill-focused volumes contains a chapter examining the key test taker characteristics that should be considered when designing a language test to assess that particular skill. Any factors associated with a permanent or temporary disability are included. In general, decisions about the nature and extent of modifications to standard language test content and delivery tend to be based upon the judgment of qualified and experienced test development staff. The staff is informed by insights from the educational and health professional bodies working in a given area of disability. Taylor (2012) highlighted the significant ethical, logistical and measurement challenges involved in undertaking empirical research into accommodated tests or conducting test validation studies among groups of test takers with disabilities. As a result, it is relatively rare for research findings to feed systematically into the development and validation of accommodated language tests. Given the inherent difficulty of conducting large-scale experimental studies involving quantitative methodologies, there have been calls for a greater focus on carrying out more qualitative or mixed-methods research in this area (see for example, Weir and Taylor 2011; Taylor 2012; Taylor and Geranpayeh 2013). Designing and conducting empirical studies on test accommodations within a smaller-scale, qualitative paradigm may lead to a more manageable research agenda. Such research outcomes can still

232

Chapter Twelve

generate fruitful insights for the future development and validation of accommodated tests. An example of this approach can be found in Shaw and Weir (2007, 20–27), where a series of small-scale studies investigating writing assessment provision for candidates with dyslexia is provided. Another way of improving our understanding of the nature and effectiveness of test accommodations might be from the perspective of studying test impact. Specifically, the investigation of impact concerns particular stakeholders associated with modified tests for test takers with disabilities. Interest in the study of test impact first emerged in the early 1990s (Alderson and Wall 1993). Since then, an important strand of impact research has been to identify the many different stakeholders involved in, or influenced by, a test (Taylor 2000, 2; Hawkey 2006, 15). In the case of test accommodations, key stakeholders include not only those who take the test (i.e. the test takers themselves), but also those who may be involved directly or indirectly in responding to an initial request for accommodation (e.g. exam board personnel), in preparing suitably modified test materials (e.g. test writers), in setting up the test venue appropriately (e.g. test centre administrators), and in conducting the modified test and processing its results (e.g. examiners). Although the body of research investigating test impact has grown substantially since the early 1990s, studies reporting the investigation of impact specifically in relation to test accommodations seem hard to find. Few language testing volumes or journals appear to report such studies. This suggests that this dimension of impact remains relatively neglected as far as research is concerned and is worthy of further consideration. As a significant provider of internationally certificated tests of English language proficiency, Cambridge English Language Assessment strives to provide a flexible range of assessment tools that can be used as essential components of language learning, teaching and assessment worldwide. Ensuring that the impact on learners and society is measurable and can be shown to be positive is stated as a key component of the examination board’s mission (University of Cambridge ESOL Examinations 2011a, 2011b). The examination board employs the concept of ‘impact by design’ to plan and monitor the use of its assessments in educational contexts. This approach encourages the scrutiny and monitoring at all stages of the effects and consequences of the board’s tests, whether at the macro level in society or at the micro level in specific teaching and learning contexts. Cambridge English Language Assessment is thus committed to a longterm, ongoing programme of research to investigate various dimensions associated with the impact of its tests (see Saville 2009). The study described in this chapter focuses on the board’s assessment provision for

Assessing Students with Disabilities

233

learners with disabilities. It constitutes an important element within that wider research agenda. The remainder of this chapter reports an analysis of empirical data recently gathered by Cambridge English Language Assessment as part of its systematic investigation of impact on a broad range of stakeholders for its tests. A documentary review of current examination board policy and practice was triangulated with survey and interview data from exam administrators, test writers, speaking examiners and test takers themselves. The aim of the review is to investigate the match between policy and practice, and identify any issues meriting attention and further development.

2. Methodology Following the emergence of test impact during the 1990s as a field of research in its own right, a considerable body of literature has developed approaches to studying it. There now exists a wealth of guidance regarding different methodologies and instruments that can be used to investigate different dimensions of impact (see for example, Cheng and Watanabe 2004; Hawkey 2006; Cheng 2008). A mixed-methods approach, combining both quantitative and qualitative tools, is generally acknowledged to provide rich and varied insights into the questions of interest. This was the approach selected for the study reported here, bringing together the use of survey questionnaire instruments and semistructured interviews.

2.1 Survey questionnaires A short and user-friendly questionnaire was developed (using SurveyMonkey) for circulation to all authorised Cambridge English Language Assessment examination centres that requested Special Arrangements (SA) during 2012. The 10 to 15-minute survey was designed to elicit the perspective of local exam managers and administrators on specific questions concerning: -

the extent of applications for Cambridge English Language Assessment SA of the testing centre, the accessibility of information provided by the board on SA provision, the user-friendliness of the SA application process operated by the board,

Chapter Twelve

234

-

the nature of SA support offered to the centre by the board, the impact of the SA on the local testing centre, including both positive impact and any difficulties experienced.

The survey invited respondents to agree or disagree with a set of statements using a four-point scale, or a category labelled “No Experience” if they had not encountered a particular item in their centre. Free text boxes also invited respondents to offer further explanatory comments as appropriate. The questionnaire ended with the opportunity for respondents to make additional comments if they wished. Questionnaire results are reported below. The perspective of all test takers who received an SA in the December 2012 test sessions was sought via a short email survey administered after taking the examination. Candidates were invited to comment on whether the SA had met their needs, whether they felt any improvements to the SA were needed, and whether they had felt at a disadvantage at any time.

2.2 Semi-structured interviews Sets of interviews were arranged with representatives of two other stakeholder groups: -

those responsible for developing modified test materials (an internal examination board staff member and the external consultant with specific responsibility in this area), those responsible for delivering the face-to-face speaking test2 and providing test scores (i.e. speaking examiners).

A 1-hour face-to-face interview was set up with the test development team members. An interview protocol with questions was prepared and sent to the participants beforehand to guide the semi-structured interview. The interview was recorded (with permission) and later transcribed. Field notes were also taken. A 1-hour telephone interview was set up with each of the two of Cambridge English Language Assessment speaking examiner cadre. Both were invited from a pool of UK-based speaking examiners whose examining record showed that they had direct experience of testing 2

Cambridge English tests normally include a paired format face-to-face speaking test, i.e. two candidates and two examiners (an Interlocutor and an Assessor). This format may be modified in various ways for candidates with a hearing, speaking or other disability (see O’Sullivan and Green 2011, 46–50).

Assessing Students with Disabilities

235

candidates with disabilities for the Cambridge English tests. An interview protocol with questions was prepared and sent to the participants beforehand to guide the semi-structured interview which was recorded with the participants’ permission. Field notes were also taken.

3. Results and discussion This section presents some of the outcomes of the research methods described above. Outcomes are discussed in terms of observations and insights offered by the four key stakeholder groups: exam administrators, test material developers, speaking test examiners and test takers.

3.1 Perspective of test administrators The perspective of test administrators at the local level was reflected in responses to the survey questionnaire circulated to all authorised Cambridge English Language Assessment examination centres that requested Special Arrangements during 2012. Responses were received from 144 out of 249 centres based in 34 countries in Europe, Asia and the Americas (see Appendix A for a full list of centre locations). The roles of respondents varied from Centre Exams Manager (72%) to other staff members (28%) with the following job titles: Exam Administrators, Exam Assistants, Exam Officers, Exam Coordinator. All respondents had experience of dealing with SA management, with 78% of the respondents having more than two years of experience (see Appendix B for further details). SA requests in the 144 centres were made for all Cambridge English exams including IELTS. The highest percentage of requests made and granted was for Cambridge English: First (25%), followed by Cambridge English: Preliminary (19%), Cambridge English: Advanced (17%), Cambridge English: Key (14%) (see Appendix C). Table 1 lists the type of applications made and the provisions given.

Chapter Twelve

236

Special Arrangements Extra time

86.0%

Separate invigilation

62.5%

Use of word processor/copier/typewriter/assistive technology

46.3%

Special listening test

44.9%

A3 enlarged print papers

40.4%

Single candidate versions of the speaking test

37.5%

Use of amanuensis/reader

30.9%

A4 modified enlarged print papers

25.7%

Braille papers

26.5%

Supervised breaks as necessary

26.5%

Lip reading version of the listening test

24.3%

Braille and enlarged print versions of the speaking test

22.8%

Exemptions3 from listening/speaking test

19.9%

Electronic version of paper–PDF

2.2%

Other

6.6%

Table 1. Special Arrangements requested4 Table 2 provides an overview of responses to statements about the accessibility of information regarding SA. (The “strongly agree” and “agree” categories have been collapsed into a single category, as have those for “disagree” and “strongly disagree”.)

3

Exemption applies when a candidate opts not to take one component of an exam, e.g. a deaf candidate chooses not to take the Listening subtest, in which case the test certificate carries the following endorsement: “The candidate was exempt from satisfying the full range of assessment objectives in the exam”. 4 Note that an individual test taker is likely to request and receive more than one accommodation, e.g. extra time and separate invigilation as well as a Braille paper; for this reason the percentage column does not sum to 100.

Assessing Students with Disabilities

- Information on how to apply is easily accessible - The application procedure is explained clearly - The Special Requirements Booklet is helpful and easy to read - The application procedure is straightforward - There is plenty of information on the ESOL website - Using Cambridge ESOL CentreNet5 to find information and download forms is straightforward - The application form is easy to complete

237

Agree

Disagree

No experience

94%

5%

1%

89%

9%

2%

89%

9%

2%

87%

11%

2%

68%

23%

9%

73%

24%

3%

92%

6%

2%

Table 2. Accessibility of information on SA provision With regard to the service Cambridge English Language Assessment delivers to its centres, the following was reported: -

88% agreed that the Exams Processing Unit (Special Requirements) was efficient and helpful, 89% thought the Cambridge English helpdesk was helpful, 80% felt that Cambridge English staff explained procedures clearly, 12% had no experience and only 8% disagreed.

When asked to reflect on their own practice, 96% felt that candidates sitting for exams at their centres had found their Special Arrangements helpful. On the other hand, 24% of the respondents stated that they had encountered difficulties (e.g. arranging extra rooms, extra invigilators, amanuensis, reader) when organising the SA in their centre. The recurrent reason reported for the perceived difficulties was that organising an SA is “time consuming and expensive”. One centre commented that organization is not difficult but is resource intensive. The respondent explains:

5 CentreNet is an online service exclusively for authorised examination centres providing technical support and resources.

238

Chapter Twelve […] When students need extra time and, therefore, separate invigilation, that means arranging for a special room just for them, an invigilator just for them, and set up extra speakers just for them. All of this, beside more hassle, is more expense. We completely agree with the special arrangement system but it does put extra strain on the centre.

Another centre referred to the process of issuing final results. The respondent shared the following experience: […] Although we had no particular problems with the administration of the exams, this summer we encountered problems with results issuing. We had quite a few cases where our special needs candidates were issued X results. To resolve each pending result, we had to review our records and supply evidence that candidates were indeed present. This caused delay in the issuing of their results.

Respondents were invited to share their experience of any unusual or challenging SA requests they had received. The five cases reported below were the most striking in their individual reports: […] A partially sighted candidate requiring magnifying equipment to read papers on-screen. The Spanish Association for the Blind provided the equipment and I insisted on having a specialised technician on site in case of breakdown. A blind candidate who is not familiar with braille and requested specialised computer software for all the four skills. A YLE Starters student with Down’s Syndrome. We thought it would be a problem because of the age (Starters is for 5–7 while the candidate was 17) and because we weren’t sure what Special Arrangements were needed. In the end, it was a very positive experience, for us, for the school she came from and for the candidate. Once we enrolled a candidate who suffered from muscular dystrophy. He took the PET exam and the following year FCE. He really appreciated the fact that he had the possibility to do these exams and that we could make special arrangements for him. We had a case where the candidate has limited motor skills and it takes an extremely long time for the candidate to type the answers onto the computer. The 4 parts of the test on one day is extremely taxing and we accordingly requested that the candidate take the examination over 2 days. The candidate must lie down while working.

Assessing Students with Disabilities

239

Other comments from the test centre perspective focused on: the need for timely despatch to the centres of test materials and certificates; modernising the provision for visually impaired candidates (computer packages rather than just Braille); allowing speaking examiners to see the adapted materials before the live test; the training of speaking examiners; a potential mismatch between practices in the UK (the overarching jurisdiction) and elsewhere—for example, under Dutch Law, medical practitioners are not permitted to provide medical statements and in Italy all dyslexic students are exempted from taking writing examinations.

3.2 Perspective of test material developers This section reports outcomes from the semi-structured interview conducted with members of the Cambridge English Language Assessment team responsible for preparing accommodated test materials. The test developers commented on significant changes which have taken place since the early 1990s in Cambridge English Language Assessment’s SA provision. There has been a move increasingly away from modifying a test shortly before the test date in response to a specific candidate or centre request and towards a more systematic policy of premodifying tests. This policy can be held on-the-shelf longer term, for use as need arises. This was necessitated partly by considerable expansion in the range of tests offered by Cambridge English Language Assessment, as well as by the time and specialist expertise required to prepare highquality accommodated versions. Furthermore, it has proved important to develop an extensive range of accompanying documentation, e.g. guides/handbooks for test centres, candidates and examiners. Some specific challenges in providing test accommodations were noted: -

Providing modified tests for lower level test takers:

[…T]he lower level tests such as Young Learners Tests and KET and the ‘for schools’ tests […] there are far more issues arising with them, simply because the visual content is there […] it’s got lots of colour in it […] the candidates themselves may have limited braille skills […] it’s much easier with the higher level tests such as ICFE [International Certificate in Financial English], ILEC [International Legal English Certificate] where you’ve got content with very little visual material in it […] it’s the lower level ones with the very limited level vocabulary which are the difficult ones to modify.

Chapter Twelve

240

-

Providing modified listening and speaking tests:

[…T]hose tests are more challenging to modify papers for than straightforward reading or writing papers […] quite a lot of the Cambridge English [speaking] tests are very visual […] the difficulty then is providing a stimulus which is the equivalent of the visual input.

-

Meeting needs of test takers with specific learning difficulties:

[…W]e’ve had recurring issues to do with candidates with specific learning difficulties such as dyslexia […] one of the reasons is that candidates around the world are used to having differing provisions in their own countries for particular exams and sometimes find what we will allow doesn’t actually match up with the provisions that they’re used to. […I]t is difficult to reassure them that we’re giving them as many things as they could get from other awarding bodies, particularly in their own country: extra time, use of a computer, final work copied out, enlarged print papers, colour overlays.

-

Complex demands of specific assessment criteria or test tasks:

[…O]ne challenge for examiners is conducting an oral exam with a candidate who has a bad stutter or stammer and one of the assessment criteria is ‘fluency’ […]. […T]he other thing is for the higher level reading exams where you have lengthy texts to read […] we separate the questions from the text (as advised by the RNIB [Royal National Institute for the Blind]) so you have a question booklet and a text booklet and that goes into Braille for a blind candidate so that means they have to work from 2 booklets and find where they are.

The importance of adequate training and support for test centre staff and examiners was highlighted, especially regarding the speaking test: […W]e’ve produced various documents over the years for our examiners and our centres […] if I was a speaking examiner and I had a candidate in front of me who was blind and using Braille, I would be quite worried about how I was going to approach the test, so we need to give them as much support as we can and simple questions are answered in the document: Do I need to be able to read Braille to examine a blind candidate? […T]he answer is of course ‘no’. [… I]f you have a deaf candidate in front of you, you need to make sure that the candidate is looking at you before you start speaking […] so these sorts of tips are

Assessing Students with Disabilities

241

really important for anybody involved in exams because they’re not always obvious.

The important role played by other examining bodies and by professional bodies in the field of disability 6 was stressed. It relates to sharing good practice when designing test accommodations and in terms of accessing specialist expertise regarding a given disability. At the same time, however, the potential for some inconsistency in practices across examining bodies was acknowledged, highlighting the need to maintain an ongoing network of collaborative partnerships. The impact of advances in technology on SA provision was noted as a significant recent development, including the increased use of screen readers and other forms of access or assistive technology. This can raise fresh questions for the examination board and local testing centre over matters of equipment compatibility and technical support as well as test security. Skill levels in using Braille can vary among blind candidates, with younger learners not necessarily learning Braille nowadays but instead using a computer and screen reading program for their reading and writing. Areas for possible future attention were highlighted as follows: -

Improving listening tests for the deaf and hard of hearing:

“[…W]e currently have the supervisor or a reader in the centre reading out texts to candidates […] they are often worried about doing this because the candidate will be lip-reading and they want to do as well as possible for the candidates–one option we could look at is actually video recording someone reading the texts beforehand with somebody who is perhaps a professional reader so that the centre can play the texts to candidates.

-

Increased public awareness-raising of the SA provision available, with access to more online exemplar materials:

[…] I don’t think it’s a question of candidates in a particular culture being less prepared to take tests, I think it’s more a question that they just don’t know they can take [accommodated] tests […] and centres are sometimes unaware that we have special requirements speaking and listening versions […] it’s a question of publicity and sample materials will help.

6

In the UK, for example, the RNIB and Action on Hearing Loss (formerly RNID).

Chapter Twelve

242

[…Y]ou do have to give candidates the opportunity to prepare for exams just like any other candidate […] nowadays I don’t think there’s any reason why they can’t access materials online or on the various websites we run. […T]o actually see a candidate with a disability taking a test can be an inspiration to other candidates […] and it might help with promotion of provision that is available.

-

Research into the effectiveness of specific provisions and test comparability, including observational studies of test takers completing a modified task:

[…M]aking sure that what we’re producing is of equal difficulty and as challenging as a standard paper is important, and that perhaps is a subject for research.

3.3 Perspective of speaking test examiners This section reports on insights gathered from the semi-structured interviews conducted independently with two speaking examiners who had direct experience of examining test takers with disabilities for the Cambridge English tests. Both examiners reported that opportunities for examining test takers receiving Special Arrangements for the Cambridge English Speaking tests were relatively infrequent. They also stated that the types of disability encountered could be very varied and often unpredictable. As a result, they felt it was difficult for a speaking examiner to build up comprehensive and sustained expertise in the area of assessing test takers with disabilities. Interestingly, both examiners commented that they had personal experience of disability within their own families which they felt assisted them in their examining role with test takers. The most commonly encountered SA tended to be conducting speaking tests with a wheelchair user, with a blind or visually impaired candidate and with a test taker who had impaired hearing, e.g. deafness, or speech difficulty, e.g. a stutter. Though both examiners reported having assessed relatively small numbers of candidates with disabilities over a period of several years, they had the impression that numbers were slowly increasing. They speculated that this might be due to a higher profile and greater public awareness of educational opportunities for learners with disabilities. Some specific challenges facing the speaking test examiner when assessing candidates were highlighted:

Assessing Students with Disabilities

243

-

The pressure (for the Interlocutor) due to additional demands involved in managing the interaction with a candidate receiving an SA. For example, handling the modified materials is sometimes complex due to multiple sheets; monitoring timing is often nonstandard and involves extra time; maintaining the discourse because discourse is adjusted to match the test taker’s disability; the non-standard and demanding nature of the whole exercise can make it difficult for the examiners to “fine-tune” their own global assessment as the Interlocutor.

-

The “intense concentration” needed (on the part of the Assessor) to evaluate the linguistic production of a test taker with a speech impediment, i.e. disentangling lack of fluency due to a stammer from the rest of the test taker’s skill in speaking English.

-

The need to sometimes “think on your feet” in unexpected and demanding situations, perhaps improvising as Interlocutor or being flexible with timing in response to a need or situation as it was encountered. This scenario may arise because the examiner has received limited information or advance warning of the actual special circumstances of the candidate and the SA applied.

-

A conflicting sense of wanting to be “kind and supportive” but at the same time being “fair”, of “getting warmth across and making candidates feel at ease” while still conducting a reliable and valid assessment exercise.

-

A sense of feeling less prepared or well-equipped to examine test takers with specific learning difficulties, e.g. attention deficit hyperactivity disorder, than with other types of physical disability. They do not always know what sort of behaviour to expect from test takers and how best to respond.

Reflecting on their experience, the examiners commented on several specific challenges which they believed the test taker faced when receiving an accommodated speaking test: -

some variability among blind candidates in their own Braille skills, memory demands for some blind candidates resulting in difficulty in retaining test instructions,

Chapter Twelve

244

-

a risk that a speech difficulty, e.g. stutter, becomes more pronounced under the pressure of a speaking test, the potentially unhelpful (interfering?) presence of a relative/friend who is accompanying the test taker for support (requiring firm but sensitive handling by the test centre).

The relationship between speaking examiner and test centre seems to be key in ensuring that examiners are adequately briefed in advance and appropriately supported on the day of the test. Both examiners reported not always receiving as much detailed information or advance warning about the candidate and their SA as they might find helpful before arriving at the test centre to conduct a speaking test. This situation may be exacerbated because examiners do not normally have prior access to the modified speaking test materials (which are held securely by the test centre) until the day of the examination. Both examiners observed that if they cannot see materials until the day of the test itself they would welcome easy access to a range of exemplar recordings. Such recordings could show test takers receiving different test accommodations for the Cambridge English Speaking tests, accompanied by in-depth guidance for speaking examiners. Cambridge English Language Assessment recently compiled a set of frequently asked questions (FAQs) on SA Speaking tests. This is designed to be read in conjunction with the standard Instructions to Speaking Examiners. However, the examiners reported not being as familiar with these as they should be, possibly because they now receive so much documentation associated with their examining roles and responsibilities. The flagging up of new/key information was suggested within the routine documentation examiners now receive. Also, increased access to DVD or online material offering specialised training for certain scenarios, particularly for speaking examiners with limited experience of testing candidates with disabilities, were suggested. Both examiners communicated an awareness of their own limitations, especially when facing unknown or unpredictable situations, but they also conveyed a strong professional sense of the need to “get on with it and do your best”.

4. Perspective of test takers In general, comments invited from test takers were positive with few complaints or suggestions for improvement. The following quotes are representative of the views expressed.

Assessing Students with Disabilities

245

[...] I am really glad to have done the exam with the special arrangements provided. I have Writer’s Camp (a kind of Focal Dystonia), a disease which affects my handwriting and causes cramps or involuntary spasms affecting certain muscles of my hand and fingers. As I am no longer able to write, I need to use the computer as a means of written communication. [...] In my opinion, the SAs which I received were right, for example, the time was exact. [...] Personally, I consider the administration was excellent. One of the points you could improve is to translate into Braille even the paper which you give us with our web-key. [...] I’m afraid the listening paper arrangements were not much of help, as I had someone sitting very close to me. I did not feel comfortable. I would suggest you to provide the recording with longer pauses or the possibility of listening three times. [...] I felt well because I could sit for the exam just like my classmates.

Although the individual circumstances and disabilities of each informant are not explained in full here, these five test taker comments illustrate a wide range of test accommodations. These accommodations include: the option to do the writing test on computer, rather than by hand; the allocation of additional time; access to a Brailled test version via a web key; and a special arrangement for a listening test (possibly involving headphones). It is also likely that some of these test takers received additional special arrangements according to their particular needs. For example, a separate room in which to take their test, a dedicated supervisor to assist them, the provision of specialised equipment together with technical support, or frequent breaks to reduce fatigue/discomfort. For the most part, these informants seem to have appreciated the special arrangements that were made for them by the exam board and the local test centre. They also found the test accommodations appropriate to their specific needs, i.e. access to a computer, increased time allocation. Nevertheless, one informant clearly felt that whatever modifications had been made to their listening test and its delivery were not as helpful as they had hoped or expected. This highlights the challenge for the exam board and the test centre of ensuring the optimum match between the test accommodation provided and the specific needs and expectations of the test taker who receives it. In this instance, it is not clear whether a change to the test materials (e.g. more and longer pause times) or to the physical

246

Chapter Twelve

arrangements for the test administration (e.g. a separate room) might have improved the situation for this particular candidate. Two further points are worth noting from this test taker feedback. Firstly, there seems to be a genuine sense in the comments that students with disabilities welcome the chance to sit for the test just like everyone else (“[J]ust like my classmates”). This should be a source of encouragement for test providers as well as a motivation to continue to strive for better access to assessment opportunities for all language learners with disabilities. Secondly, such test takers are clearly well placed to provide valuable advice regarding potential improvements to the provision offered. In this case, they suggested extending the use of Braille to the paper instructions accompanying the computer web key that enables online access to the accommodated test version. This confirms the usefulness of small-scale impact studies such as this one as a means of gathering systematic feedback that can inform continuous monitoring and improvement of test accommodations.

5. Conclusions and recommendations A number of recurring and persistent themes can be discerned in the data gathered from the four stakeholder groups in this study. These threads offer useful insights into some important issues associated with the design and delivery of test accommodations, and highlight areas for potential attention and improvement. Such areas can be the extent and nature of SA information provided to stakeholders; the importance of sustained and comprehensive training; the complex logistics of implementation; the challenge of achieving a consistent and professional approach; and possible future avenues for research. The results of the questionnaire suggest that most centres are generally happy with the SA application process and the availability of information from Cambridge English Language Assessment. Centres offered some constructive ideas for improvement, for example, an additional Braille practice paper would be helpful to candidates. Similarly, the test developers advocate increasing the range of sample test materials available, ideally online via the Cambridge English Language Assessment website. This will provide candidates with encouragement and opportunities for test preparation. The test developers noted the importance of comprehensive training for test centre staff and administrators. Centres also highlighted a need for more training/helpful advice for speaking examiners when assessing candidates with a severe speech impediment. Speaking examiners would

Assessing Students with Disabilities

247

welcome training materials showing different types of modified speaking test, as well as targeted guidance for assessing test takers with specific learning difficulties. Centre views varied over whether all candidates who receive extra time do actually need it or use it and this suggests an area for potential research, perhaps through observational studies. Both test developers and speaking examiners appear to suggest that task comparability across differing accommodated formats is also worthy of investigation. As advances continue in the area of assistive technology, this is likely to become a priority area for validation research. Decisions about best practice will continue to be informed through professional advice from disability organisations and other examining bodies. Nevertheless, this may sometimes be complex given inconsistencies in national and international disability legislation. Special Arrangements can entail considerable extra work, inconvenience and extra expense for centres, as well as present test developers and examiners with significant challenges. However, it is encouraging to hear that all of these stakeholder groups confirm the positive impact of test accommodations in terms of benefits to learners, their teachers and families. This was echoed in test takers’ testimonials on the SA provision. The challenge of meeting the often unique configuration of assessment needs of each test taker with a disability, in a manner that is reliable, valid and equitable, should not be underestimated. It is therefore encouraging for centres and test developers to hear the comments made by the test takers on the suitability and the appropriate match between the SA provision and their individual requirements. The particular value of this chapter lies in the way it explores the impact of assessing students with disabilities through the voices of four different stakeholder groups involved in the process. This chapter will hopefully deepen our understanding of the roles and needs of such groups and thus contribute to the body of literature on test impact in the relatively under-researched area of language assessment for students with disabilities. There are, of course, other key stakeholders who could (and should) be canvassed for their views, including disability organisations who offer advice to examination boards, and test users who have to interpret scores from modified tests. Our hope is that this chapter will inspire further studies in this field so that the assessment provision for language learners with disabilities can be constantly reviewed and refined.

248

Chapter Twelve

Works Cited Abedi, Jamal. 2008. “Utilizing Accommodations in Assessment.” In Encyclopedia of Language and Education (2nd ed.), Vol. 7 of Language Testing and Assessment, edited by Elana Shohamy and Nancy Hornberger, 331–347. New York: Springer Science+Business Media LLC. AERA/APA/NCME. 1999. Standards for Educational and Psychological Testing. Washington, DC: Author. Alderson, John Charles, and Dianne Wall. 1993. “Does Washback Exist?” Applied Linguistics 14:115–129. Cheng, Liying. 2008. “Washback, Impact and Consequences.” In Encyclopedia of Language and Education (2nd ed.), Vol. 7 of Language Testing and Assessment, edited by Elana Shohamy and Nancy Hornberger, 349–364. New York: Springer Science+Business Media LLC. Cheng, Liying, and Yoshinori Watanabe with Andy Curtis. 2004. Washback in Language Testing: Research Contexts and Methods. Mahwah, NJ: Lawrence Erlbaum Associates. Fulcher, Glenn, and Fred Davidson. 2007. Language Testing and Assessment: An Advanced Resource Book. London: Routledge. Geranpayeh, Ardeshir, and Lynda Taylor, eds. 2013. Examining Listening: Research and Practice in Assessing Second Language Listening. Cambridge: UCLES/Cambridge University Press. Hawkey, Roger. 2006. Impact Theory and Practice: Studies of the IELTS Test and Progetto Lingue 2000. Cambridge: UCLES/Cambridge University Press. Khalifa, Hanan, and Cyril J. Weir. 2009. Examining Reading: Research and Practice in Assessing Second Language Reading. Cambridge: UCLES/Cambridge University Press. O’Sullivan, Barry, and Anthony Green. 2011. “Test Taker Characteristics.” In Examining Speaking: Research and Practice in Assessing Second Language Speaking, edited by Lynda Taylor, 36–64. Cambridge: UCLES/Cambridge University Press. Saville, Nick. 2009. “Developing a Model for Investigating the Impact of Language Assessment within Educational Contexts by a Public Examination Provider.” PhD diss., University of Bedfordshire. Shaw, Stuart D., and Cyril J. Weir. 2007. Examining Writing: Research and Practice in Assessing Second Language Writing. Cambridge: UCLES/Cambridge University Press.

Assessing Students with Disabilities

249

Taylor, Lynda. 2000. “Stakeholders in Language Testing.” Research Notes 2:2–3. Taylor, Lynda, ed. 2011. Examining Speaking: Research and Practice in Assessing Second Language Speaking. Cambridge: UCLES/Cambridge University Press. Taylor, Lynda. 2012. “Accommodation in Language Testing.” In The Cambridge Guide to Second Language Assessment, edited by Christine Coombe, Peter Davidson, Barry O’Sullivan and Stephen Stoynoff, 307–315. New York: Cambridge University Press. Taylor, Lynda, and Ardeshir Geranpayeh, eds. 2013. “Conclusions and Recommendations.” In Examining Listening: Research and Practice in Assessing Second Language Listening, edited by Ardeshir Geranpayeh and Lynda Taylor, 322–334. Cambridge: UCLES/Cambridge University Press. University of Cambridge ESOL Examinations. 2011a. Principles of Good Practice: Quality Management and Validation in Language Assessment. Cambridge: UCLES. University of Cambridge ESOL Examinations. 2011b. Making an Impact. Cambridge: UCLES. Weir, Cyril, and Lynda Taylor. 2011. “Conclusions and Recommendations.” In Examining Speaking: Research and Practice in Assessing Second Language Speaking, edited by Lynda Taylor, 293– 313. Cambridge: UCLES/Cambridge University Press.

Chapter Twelve

250

Appendix A Centre locations Country

No & % of respondents

Italy Spain Poland United Kingdom Switzerland Netherlands Germany Czech Republic Sweden Brazil Greece Portugal Romania Bulgaria France Russian Federation Uruguay Argentina Austria Canada Colombia Croatia Cyprus Denmark Hong Kong Ireland Luxembourg Malta Mexico New Zealand Norway Slovakia Ukraine United States of America

24 23 17 16 8 6 5 4 4 3 3 3 3 2 2 2 2 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

TOTAL

144

16.7% 16.0% 11.8% 11.1% 5.6% 4.2% 3.5% 2.8% 2.8% 2.1% 2.1% 2.1% 2.1% 1.4% 1.4% 1.4% 1.4% 0.7% 0.7% 0.7% 0.7% 0.7% 0.7% 0.7% 0.7% 0.7% 0.7% 0.7% 0.7% 0.7% 0.7% 0.7% 0.7% 0.7%

Assessing Students with Disabilities

Appendix B Years of experience in dealing with Special Arrangements Range

No & % of respondents

0-1 2-5 6-10

31 48 32

21.5% 33.3% 22.2%

11 or more

33

22.9%

Appendix C Exams for which SA was requested Exams BEC CAE CPE FCE ICFE IELTS ILEC KET PET Skills for Life TKT YLE Other

No & % of SA requests 17 71 36 106 3 9 4 57 81 1 1 28 6

4% 17% 9% 25% 1% 2% 1% 14% 19% 0% 0% 7% 1%

251

CHAPTER THIRTEEN SPECIAL NEEDS TEST FORMS: LEVELLING THE PLAYING FIELD FOR TEST TAKERS WITH DISABILITIES JAYANTI BANERJEE, NATALIE NORDBY CHEN AND BARBARA DOBSON

In the context of assessing language learners with special needs, there is no longer a debate as to whether accommodations should be provided. Rather, the focus is now on how to fairly meet the needs of those learners. This chapter will present a case study of the process by which CaMLA1, a large-scale test provider, prepares modified test forms for test takers with special needs. Considering the production of Braille versions of a test, questions such as the treatment of pictures and other images will be discussed. Other important issues to be explored are the adjustments needed in paired-format speaking tests when only one test taker has a visual disability. Also, an issue of concern is the qualifications that should be demanded of an amanuensis for the writing section. The chapter will address the challenge of providing test takers with modified test forms that appropriately and reasonably accommodate their disabilities. At the same time, such modifications ensure that the construct measured by the test remains unaltered and that the interpretation of the test score is the same whether the test is delivered in the standard format or in a modified format.

1. Introduction When assessing language learners with special needs, most large-scale test providers offer a variety of standard accommodations, including modifications to the test, accessible facilities, additional personnel or 1

Based in Ann Arbor, Michigan, CaMLA is a not-for-profit organization jointly owned by the University of Michigan and the University of Cambridge.

254

Chapter Thirteen

equipment, or additional time in which to complete the test (Taylor 2012). In the United States of America, policies are informed by the Americans with Disabilities Act2. Students with disabilities must be “provided with appropriate accommodations when necessary to enable participation in … assessments”3. The aim is to place test takers with special needs “on an equal footing” with candidates who take the standard version of an exam (Taylor and Gutteridge 2003, 2). This is accomplished by removing the construct-irrelevant factors that could prevent test takers with special needs from demonstrating their true language knowledge and competence. However, concerns remain with the extent to which accommodations achieve the aim of leveling the playing field without unwittingly tipping the balance in favor of the test taker who seeks the accommodation. As Sireci et al. posit: […] If the accommodation leads to an unfair advantage for the students who get them, for example, if everyone would benefit from the accommodation, then the scores from accommodated exams may be invalidly inflated, which would be unfair to students who do not receive accommodations.4

They review 28 empirical studies of the scores achieved by test takers on standard and modified versions of educational tests, looking in particular at the two most popular accommodations: extended time and oral presentation. With respect to extended time, Sireci et al. (2005) find that test takers who receive extra time benefit significantly in relation to test takers who take the exam under standard conditions. Here the authors note that educational tests tend to be unintentionally speeded (that is, the time allowed is not sufficient to complete all items without time pressure). In light of this, they conclude that the accommodation is still justified but suggest that if the tests were not speeded there would be less need for time extensions. With respect to oral presentation, the results were mixed. The oral presentation of math problems led to positive effects while the oral presentation of problems in other subject areas led either to no gains for students with disabilities or similar gains for both the students who received accommodations and those who did not. It is important to note, however, that many studies have been conducted in the context of educational assessments. They looked 2

http://www.ada.gov/pubs/ada.htm. http://www2.ed.gov/policy/elsec/guid/standardsassessment/guidance_pg4.html# disabilities3. 4 Sireci et al. 2005, 458. 3

Special Needs Test Forms

255

primarily at test accommodations in K-12 education and particularly in content subjects such as mathematics (Abedi 2012; Sireci et al. 2005; Thurlow et al. 2006). Studies into the effect of accommodations in the context of English language assessments, such as those reported in Shaw and Weir (2007, 23–27), are relatively uncommon and there are no published studies of braille use in English language assessments. This is perhaps understandable: experimental studies in the context of language assessment would be challenging to institute. The number of test takers who request accommodations is a small proportion of the test taking population. Indeed, over the last five years, only 0.8% of the total number of CaMLA’s test takers requested a testing accommodation. Test takers with special needs are also varied, with concomitantly varied accommodations provided – ranging from special seating and large-print test books to the employment of amanuenses, to name a few. As a result, the N-size of any proposed study would be too small and the independent variables too numerous to draw helpful conclusions. More than this, however, test takers request accommodations for language assessments because they are unable to process language under standard conditions such as time restrictions. Yet, some typical accommodations—such as the provision of additional time to complete the test—could tremendously benefit test takers who do not have special needs. Consequently, a control group is theoretically difficult to achieve. The difficulty of conducting such empirical research does not, though, absolve test developers from their responsibility to address the core concern in any discussion about test modification. The aspects of that concern are that accommodations should not alter the construct measured by the test, and that the interpretation of the test score should be the same either the test is delivered in the standard format or in a modified format (see Koenig and Bachman 2004). As Messick notes: […] tests are imperfect measures of constructs because they either leave out something that should be included […] or else include something that should be left out, or both.5

The worry, then, is that when modifying a test or making an accommodation, the test developer inadvertently introduces a constructirrelevant factor (something that should be left out) or excises a constructrelevant factor (something that should be included), rendering the test even more “imperfect”. This chapter will discuss how CaMLA addresses these concerns when preparing Braille versions of exams. It provides a detailed 5

Messick 1989, 34.

256

Chapter Thirteen

description of the modifications made for the Braille versions of the Examination for the Certificate of Competency in English (ECCE) and the Examination for the Certificate of Proficiency in English (ECPE). The chapter will reflect upon the ways in which CaMLA accounts for test takers’ needs while safeguarding the underlying construct of the exams.

2. CaMLA, the ECCE, and the ECPE CaMLA has a long history in English language assessment. Originally a part of the University of Michigan’s English Language Institute (ELI), it launched the ECPE in April 1953. One of the creators of the test, Robert Lado, is widely credited with writing the seminal text of the field (Lado 1961). In the seventy years since that first ECPE administration, the ELI (now CaMLA) has introduced a variety of additional language examinations, including both single- and multi-level proficiency tests, placement tests, and tests of speaking for international teaching assistants. The ECCE and ECPE are standardized, single-level proficiency tests that are taken by approximately 100,000 test takers annually, worldwide. The ECCE is a high-intermediate examination aimed at the B2 level on the Common European Framework of Reference for Languages: Learning, Teaching, Assessing (CEFR; Council of Europe 2001). The ECPE is an advanced-level examination aimed at the C2 level on the CEFR. Both examinations assess test takers’ language proficiency in all four skill areas, i.e. reading, writing, speaking, and listening. They cover topics in a range of contexts, including public, educational, and occupational language use situations. Tables summarizing the content of each test can be found in the appendices (Appendix A and B). Full descriptions of both examinations are available on the CaMLA website6 along with item samples. The exams contain a variety of item types that together assess language in ways that approximate authentic language use as closely as possible. The layout of the exams also reflects this aim, using columns, images, and other formatting to reflect the different ways in which text types are presented. The challenge, when modifying the exams for test takers who request accommodations, is to understand their needs well enough to make appropriate test modifications. Such modifications should preserve the construct of the exam but present text in a manner that is authentic within the context of the test takers’ language use.

6

http://www.cambridgemichigan.org/.

Special Needs Test Forms

257

3. Braille test versions: issues and solutions Braille has its origins in French military strategy and was first developed into a reading and writing system for the blind in 1821. The system is based on cells of six dots. These cells allow for 63 different patterns of dots representing letters of the alphabet, numbers, symbols for punctuation marks and other formatting marks (ICEB 2010). Providers of English Braille text are encouraged to adopt Unified English Braille (ICEB; International Council on English Braille 2010). CaMLA offers test takers the choice of reading either uncontracted (grade 1) or contracted (grade 2) Braille. In order to prepare a test for conversion into Braille, test providers need to carefully consider what is entailed in making the test accessible to blind test takers. They should ensure that the Braille version preserves item types and content as much as possible, while also capturing the same construct as the sighted version: Test items should be deleted or substituted only if the item cannot be provided in Braille, tactile graphics, large print, or audio format without significantly changing the item and the intent of the question […]. Substituted items should assess the same skill and have equal value and validity. Substituted items must maintain the correct answer in the same position as that of the original test item.7

Decisions about item types, test content, and the construct of the Braille version should take into account a number of key issues: 1. Braille reading speeds, 2. How Braille text is read, 3. The provision of visuals.

3.1 Braille reading speeds Though Braille reading speeds can be as high as 200 words per minute, the average speed is approximately 125 words per minute. The average reading speed of sighted readers is 200–250 words per minute. This suggests that a test taker reading Braille would need approximately twice as much time as a sighted test taker. Admittedly, this estimation is far from scientific; published research varies in its time extension recommendations from 1.5 times (Spungin 2002) to as much as 2.5 times (Morris 1974). It 7

Allman 2004, 7–8.

258

Chapter Thirteen

also appears that the age and experience of the Braille reader is important when determining a reasonable accommodation (see, for example, Wetzel and Knowlton 2000). As Sireci et al. (2005, 481–483) point out, test takers seeking accommodations (in this case a Braille version) are sufficiently diverse to pose problems in deciding upon the most appropriate amount of extra time to offer.

3.2 How Braille is read Typically, a person reading Braille will use both hands; the right hand will read a line while the left hand moves to the next line to make a smooth transition from one line to the next. If the hands are lifted from the text, Braille readers will take time to recover their position, more time than sighted readers would need if they lifted their eyes from the text and then needed to find their place again. This presents a challenge when converting long reading exercises into Braille. Both the ECCE and the ECPE reading sections require test takers to read an extended piece of writing and then answer comprehension questions. For sighted readers, the text and questions are presented side by side either on facing pages or on the same page. This means that sighted test takers can view the stimulus and the questions simultaneously. In contrast, the Braille version of the text runs over a number of pages. Since Braille text needs to be read sequentially, if the questions were presented at the end of the reading passage, test takers would need to remember key information from the text so that they could identify the correct answer. This presentation of the questions would place a greater demand on test takers’ memory (a construct irrelevant variable) and would also deny them the option that sighted test takers have of easily returning to the text at will. This approach also constrains the kinds of questions that can be asked. In particular, referent questions—which entail being able to quickly scan short stretches of text—would be impracticable in a Braille version of a test, thus narrowing the construct being measured. An alternative approach would be to place the questions on a pullout sheet that can be laid beside the passage, so that the test takers could move between the two pages. However, this would require the test takers to lift their hands from the page and they could lose their place both in the passage and in the questions. This suggests that a more productive approach would be the one taken in the studying and scanning tasks described in Knowlton and Wetzel (1996) and Wetzel and Knowlton (2000). In this approach, the test takers are given the reading passage in Braille and are told that they will be asked some questions. When the test

Special Needs Test Forms

259

takers are ready, these questions are read aloud by the examiner. In this approach, the test takers can keep their hands on the page and can reread sections if necessary to refresh their memory when they answer the questions.

3.3 Visuals Where visuals are not central to the item design (perhaps constituting additional formatting rather than encoding key information) they can either be eliminated in the Braille version or briefly described. The latter approach is feasible for the ECCE speaking test, where the task information is presented in a combination of text and images (Fig. 1). The images on the task sheet are intended to support the verbal instructions given by the examiner and could be replaced with brief text in Braille without measurably changing the task for the blind test taker. In some cases, however, the visuals are central to the input. For instance, the ECCE listening section contains 30 items where the answer choices are shown as pictures (Fig. 2). The aim is to minimize the amount of reading tested in the listening section by limiting the amount of text that needs to be processed. For the Braille version of the exam, these pictures would need to be made accessible to a reader of Braille. One approach could be to provide short descriptions of each picture. Test takers could then choose the description that answers the question. This has two drawbacks. The first is that test takers who take the Braille version of the exam are required to process more text than test takers who take the unmodified version. This raises questions about the comparability of the challenge being presented to the different groups of test takers. The second drawback is that the short description might inadvertently key the item by providing helpful vocabulary or orienting information. An alternative approach could be to provide tactile graphics where the image is designed to be touched. A tactile graphic is not a faithful reproduction of the original image but represents the content of the image with a set of tactile symbols. Crucial for the readability of a tactile graphic, however, is title or subject line that orients the reader to the subject and the view. This approach presents very similar drawbacks to the picture descriptions approach: it adds to the reading load of the Braille reader, and the orienting text might key the item.

260

Chapter Thirteen

Fig. 1. Exampple of an ECCE E speaking prom mpt

Special Needs Test Forms

261

Fig. 2. Example of an ECCE listening: The answer-choices are pictures

This suggests that the picture items are not suitable for the Braille version of the ECCE since they cannot be provided “without significantly changing the item[s or] the intent of the question” (Allman 2004, 7). Consequently, they should be deleted or substituted. If the items were deleted, the ECCE listening section would be reduced to 20 items, a more than 50% reduction in the total number of items. This would have an adverse effect upon the reliability of the section and, since all dialogic language would be eliminated from the section, would also markedly change the construct of the listening section. In order to conserve the construct of the ECCE listening section, the picture items would need to be replaced with items that tap the same construct but that are designed to present the answer choices as text (see also, Allman 2004, 8).

4. Preparing the ECCE and ECPE Braille versions The majority of the ECCE and ECPE items convert readily into Braille. However, some adjustments need to be made so that the exams are fully accessible for readers of Braille. The Braille version of each exam is inspected to ensure that none of the items assume knowledge of visual

262

Chapter Thirteen

information (such as the variation in two artists’ color palettes). Test takers to whom the Braille version of the test is administered take the exam in a separate room with a dedicated proctor who also performs the role of an amanuensis (or scribe). Proctors for the Braille version (indeed, for any version of the test that requires the proctor to read test material aloud) must be able to read aloud clearly, at a normal pace, and with good pronunciation. They must also be familiar with the words, terms, symbols, or signs that are specific to the test content. Proctors receive step-by-step instructions for administering the Braille version of the exam (CaMLA, 2012), including a script to follow. This script ensures a consistent testing experience for all test takers receiving these accommodations. The subsections that follow briefly describe the other adjustments that are made.

4.1 Listening section As already discussed, the ECCE listening section contains 30 items in which the answer choices are shown as pictures. Since the pictures cannot be converted into Braille while also conserving the construct of the ECCE listening section, the picture items are replaced with items that tap the same construct but which present the answer choices as text. No such modification is required for the other ECCE and ECPE listening items. These questions and answer choices are converted into Braille. Test takers receive two times the amount of time provided for nonaccommodated test takers. The audio for the listening section provides 24 seconds pause time between questions, twice the length of the pause in the unmodified exam. All other pause times are also doubled. This ensures that each test taker has enough time to read the answer choices (which are provided in Braille) as well as to work with the amanuensis to complete the answer sheet.

4.2 Reading section The ECCE and ECPE grammar and vocabulary items are converted into Braille without modifications. However, the item types that demand extended reading are modified for the Braille version of the exams. The reading passages are presented in Braille and the questions are also available in Braille. However, the test takers have the option of asking the proctor to read the questions and answer choices aloud, and can ask that they be repeated. This enables test takers to keep their hands on the

Special Needs Test Forms

263

reading passages so that they can review the material easily instead of relying upon their memory of the text. The ECPE cloze section is also presented in Braille, using standardized formatting to indicate where the blanks occur. When the test takers reach a blank, the proctor reads the answer choices aloud; answer choices can be read more than once. If the item is testing a discourse-level feature and the test takers need to read more of the text in order to finalize an answer, they can do so easily because their hands are still on the text. In this way, the Braille version of the cloze passage can fulfil the aims of the cloze item type, and the construct of the ECPE reading section can be maintained. As in the case of the listening section, test takers receive twice the amount of time provided for non-accommodated test takers. This accommodates for the time required to read Braille text and also gives the test takers time to dictate answers to the amanuensis.

4.3 Writing section Test takers may choose to dictate their essay to the proctor, who also functions as the amanuensis. As they dictate, test takers must spell each new content word and indicate where punctuation is needed. If they wish to review as they write, test takers can ask proctors to read back sections of the text. Alternatively, test takers may type their essay on a computer or on a Braille typewriter. If the test takers use a Braille typewriter, after they have finished the essay they will read the text aloud to an amanuensis. When the essay is complete, the proctor/amanuensis will read back the whole text and make any other requested changes. Because proctors have to work so closely with the test takers to complete the writing task, proctors should be patient and willing to read text back to the test takers more than once. They should also be able to work with the test takers supportively, without creating pressure, but also without providing the test takers with answers, hints, or content-based suggestions. Proctors receive detailed guidelines for their role as a scribe (CaMLA Examiners’ Manual for Nonstandard Test Administrations). Additionally, to ensure that the proctor is able to write exactly what the test taker dictates, the proctor’s language proficiency needs to be at or beyond the proficiency level of the exam. Test takers receive 1.5 times the amount of time provided for nonaccommodated test takers.

264

Chapter Thirteen

4.4 Speaking test The ECCE speaking test is not modified. The test taker still engages with the examiner in a problem-solving task. However, the prompt visuals are replaced with brief text in Braille. The ECPE speaking test is a paired task in which both test takers are provided printed information. In order to conserve the construct of the ECPE speaking test and to ensure that the test takers can demonstrate their ability to manage power relations in communication, those taking the accommodated version of the test are paired with a partner who is at the ECPE level in language proficiency. The prompt material is converted to Braille for the blind test taker and provided in print for the partner. The test is conducted following the same protocols as the non-accommodated test with two minor differences. The first is that the test runs longer because the blind test taker will need more time to read the task sheet. The second is that the partner’s performance is not scored. Because the blind test taker will need more time to read the task sheet, this would amount to giving a sighted test taker additional time for the test and could provide a benefit to the sighted test taker.

5. Reflections This chapter has focused upon the conversion of two exams into Braille versions. This is of particular interest since accommodations for the blind are not discussed as often as accommodations for test takers with special learning needs (such as test takers with dyslexia and dysorthographia). There are, of course, many other accommodations issues that need to be addressed – each bringing with it other conundrums. For instance, in the field of general education, oral presentation of test material is used as an accommodation for test takers with reading difficulties such as dyslexia. This accommodation is uncontroversial in subject areas (such as mathematics or science) where the ability to read is not part of the construct being tested. However, the use of read-aloud in tests of reading comprehension is debated. This is largely because of differences in opinion over the construct being tested. Some test providers do not consider text decoding to be part of the construct of reading comprehension, while others do (Laitusis 2010, 154). We remain sceptical of the effect of oral presentation on the construct of a test of English as a second language reading proficiency. Even if text decoding is relatively unimportant in a test of reading comprehension, when text is presented orally, test takers’ listening ability presents a confounding factor to the

Special Needs Test Forms

265

measurement of reading comprehension. It also suggests that certain reading skills, such as the ability to compare and contrast the presentation of detailed information across multiple texts, might not be testable. Though it is important to make tests accessible for test takers with special needs, it is equally important to ensure that the accommodations are reasonable within the construct being assessed. It is also important the final score awarded to have the same meaning as a similar score on the non-accommodated version of the exam. Readers will no doubt have noted that CaMLA’s Braille exam versions are standardized even though we have acknowledged (see Sireci et al. 2005) that test takers who request accommodations are varied in their needs. Ideally, those needs would be individually diagnosed, identified, and accommodated (Wiig 2000, 185–fig. 2). However, such bespoke approaches are impractical in the context of large-scale testing. We have opted instead to seek a proper understanding of the general needs of test takers with the identified disability (in this case blindness). Based on our understanding, we have formulated standard procedures that accommodate those needs and are designed to preserve the construct of the exam. Nevertheless, the accommodations that we have described are not set in stone. They represent our current understanding of the needs of test takers who request Braille versions of our exams. Indeed, the needs of test takers, as well as the accommodations available, are moving targets and the accommodations policies and practices of test providers should evolve. For example, there are indications in the literature that only a very small number of people with visual disabilities learn to read Braille. While some of this might be due to limited access to education and facilities, it is also possible that the advent of screen readers and voice recognition software is changing the way that the blind interact with text. So, by extension, it is changing what “real-life language use” means for a person who is blind. This, in turn, will have definite implications for how the language proficiency of the blind should be assessed and the inferences that can be drawn based on their test performance. As a result, CaMLA, like other large-scale test providers (see, for example, Khalifa and Weir 2009, 24), regularly reviews the organization’s test accommodations policies and practices. In closing, we return to the empirical research that has been conducted so far in this area and note the absence of research that captures and compares the language skills and processes engaged by test takers of either the accommodated or standard version of an exam. Such studies would be extremely informative, revealing the extent to which test providers’ efforts have, in fact, preserved the construct-relevant factors while accommodating

266

Chapter Thirteen

for construct-irrelevant constraints. They would directly address questions about whether the accommodations provided are suitable and also result in valid test scores.

Works cited Abedi, Jamal. 2012. “Validity Issues in Designing Accommodations for English Language Learners.” In The Routledge Handbook of Language Testing, edited by Glenn Fulcher and Fred Davidson, 48–61. Abingdon, UK: Routledge. Allman, Carol B. 2004. Making Tests Accessible for Students with Visual Impairments: A Guide for Test Publishers, Test Developers, and State Assessment Personnel (2nd edition). Louisville, KY: American Printing House for the Blind. CaMLA. 2012. “Examiner’s Manual for Nonstandard Test Administrations to Individuals with Disabilities.” Accessed November 2, 2012. http://www.cambridgemichigan.org/sites/default/files/resources/ Manual_NonStandard.pdf. Council of Europe. 2001. Common European Framework of Reference for Languages: Learning, Teaching, Assessing. Cambridge: Cambridge University Press. International Council on English Braille. 2010. “The Rules of Unified English Braille.” Accessed November 13, 2012. http://www. brailleauthority .org/. Khalifa, Hanan, and Cyril J. Weir. 2009. Examining Reading: Research and Practice in Assessing Second Language Reading. Cambridge: Cambridge University Press. Knowlton, Marie, and Robin Wetzel. 1996. “Braille Reading Rates as a Function of Reading Tasks.” Journal of Visual Impairment and Blindness 90:227–236. Koenig, Judith A., and Lyle F. Bachman. 2004. Keeping Score for All: The Effects of Inclusion and Accommodation Policies on Large-Scale Educational Assessments. Washington D.C.: The National Academies Press. Lado, Robert. 1961. Language Testing. London: Longman. Laitusis, Cara C. 2010. “Examining the Impact of Audio Presentation on Tests of Reading Comprehension.” Applied Measurement in Education 23:153–167. Messick, Samuel. 1989. “Validity.” In Educational Measurement (3rd Edition), edited by Robert Linn, 13–103. Washington D.C.: American Council on Education.

Special Needs Test Forms

267

Morris, June E. 1974. “The 1973 Stanford Achievement Test Series as Adapted for Use by the Visually Handicapped.” Education of the Visually Handicapped 6:33–46. Shaw, Stuart D., and Cyril W. Weir. 2007. Examining Writing: Research and Practice in Assessing Second Language Writing. Cambridge: University of Cambridge ESOL Examinations and Cambridge University Press. Sireci, Stephen G., Stanley E. Scarpati, and Shuhong Li. 2005. “Test Accommodations for Students with Disabilities: An Analysis of the Interaction Hypothesis.” Review of Educational Research 75:457–490. Spungin, Susan J., ed. 2002. When You Have a Visually Impaired Student in Your Classroom: A Guide for Teachers. New York: AFB Press. Taylor, Lynda. 2012. “Accommodation in Language Testing.” In The Cambridge Guide to Second Language Assessment, edited by Christine Coombe, Peter Davidson, Barry O’Sullivan and Stephen Stoynoff, 307–315. Cambridge: Cambridge University Press. Taylor, Lynda, and Mike Gutteridge. 2003. “Responding to Diversity: Providing Tests for Language Learners with Disabilities.” Research Notes 11:2–4. Thurlow, Martha L., Sandra J. Thompson, and Sheryl S. Lazarus. 2006. “Considerations for the Administration of Tests to Special Needs Students: Accommodations, Modifications, and More.” In Handbook of Test Development, edited by Steven M. Downing and Thomas M. Haladyna, 653–673. Mahwah, NJ: Lawrence Erlbaum Associates. United States Department of Education. no date. “Elementary and Secondary Education: Guidance on Standards, Assessments, and Accountability”, Accessed July 22, 2013. http://www2.ed.gov/policy/ elsec/guid/standardsassessment/guidance_pg4.html# disabilities3. United States Department of Justice. 1990. “Americans with Disabilities Act of 1990”, Accessed July 22, 2013. http://www.ada.gov/pubs/ ada.htm. Wetzel, Robin, and Marie Knowlton. 2000. “A Comparison of Print and Braille Reading Rates on Three Reading Tasks.” Journal of Visual Impairment & Blindness 94:146–154. Wiig, E. H. (2000). “Authentic and Other Assessments of Language Disabilities: When is Fair Fair?”, Reading and Writing Quarterly 16:179–210.

Chapter Thirteen

268

Appendix A Format and Content of the ECCE Section

Time (min.)

Speaking

15

Listening

30

Grammar Vocabulary Reading

Writing

80

30

Description Test takers participate in a structured, multi-stage task with one examiner. Part 1 (multiple choice) A short recorded conversation is followed by a question. Answer choices are shown as pictures. Part 2 (multiple choice) Short talks delivered by single speakers on different topics, followed by 4 to 6 questions each. Grammar (multiple choice) An incomplete sentence is followed by a choice of words or phrases to complete it. Only one choice is grammatically correct. Vocabulary (multiple choice) An incomplete sentence is followed by a choice of words to complete it. Test takers must choose the option that best completes the sentence in terms of meaning. Reading (multiple choice) Part 1: A short reading passage is followed by comprehension questions. Part 2: A set of four short texts related to each other by topic are followed by 10 questions each. Test takers write either a letter or essay in response to a short newspaper article.

Number of Items 4 stages

50

35

35

30

1 task

Special Needs Test Forms

269

Appendix B Format and Content of the ECPE Section

Time (min.)

Speaking

30–35

Writing

30

Listening

35–40

Grammar Vocabulary Reading

75

Description Test takers participate in a semistructured, multi-stage task involving two test takers and two examiners. Test takers write an essay based upon one of two topic choices. Part 1 (multiple choice) A short recorded conversation is accompanied by three printed statements. Test takers choose the statement that conveys the same meaning as what was heard, or that is true based upon the conversation. Part 2 (multiple choice) A recorded question is accompanied by three printed responses. Test takers choose the appropriate response to the question. Part 3 (multiple choice) A recorded talk, such as those that might be heard on the radio, is followed by recorded comprehension questions. The questions and the answer choices are printed in the test booklet. Grammar (multiple choice) An incomplete sentence is followed by a choice of words or phrases to complete it. Only one choice is grammatically correct. Cloze (multiple choice) After reading a passage from which words have been removed, test takers must choose one of four words that best fills a missing word slot in terms of grammar and meaning. Vocabulary (multiple choice) An incomplete sentence is followed by a choice of words to complete it. Test takers must choose the option that best completes the sentence in terms of meaning. Reading (multiple choice) A reading passage is followed by comprehension questions.

Number of Items

5 stages 1 task

50

120

CHAPTER FOURTEEN ON EQUAL FOOTING? ACCOMMODATIONS FOR DISABLED CANDIDATES IN THE TESTDAF ULRIKE ARRAS, ANIKA MÜLLER-KARABIL AND SONJA ZIMMERMANN

The chapter addresses current accommodation practices for foreign language learners with specific learning or other disabilities in use with the Test of German as a Foreign Language (TestDaF). After a brief introduction to the test construct and assessment procedures, possible modifications for test takers with special needs are outlined. They are discussed in relation to issues of validity and fairness. The focus is on special modifications offered to blind test takers. Specifically, these participants can choose between two test versions: one in Braille, the tactile writing system used by blind people, and one electronic version that is processed via PC, using a screen reader. Both versions are described and closely examined according to the following questions: In what ways are test design and test administration accommodated? What impact does the use of technology have? What challenges do the changes made pose for appropriate, standardised test development and administration? The discussion centres on the main goal of high stakes-testing. The objective is to provide fair and valid assessment instruments for all test takers, taking into consideration the heterogeneous population of candidates with special needs.

1. Introduction Over the last decades there have been considerable developments to allow persons with disabilities and specific needs, physical as well as psychological, temporary as well as permanent, free access to education and learning. Consequently, this leads to questions concerning testing of language skills of persons with special needs.

272

Chapter Fourteen

Not only national governments but also the European Union encourages institutions of teaching, learning and testing, to offer special facilities and accommodations for persons with disabilities (e.g. European Parliament 2011). Since the 1990s, we can observe important improvements in the field of language testing and assessment. Free access to language testing for everybody is understood as a central criterion of quality assurance. It is defined in various codes and guidelines of associations that aim to establish high quality in language testing (e.g. ALTE 1994; EALTA 2006). The ethical context is striving for fairness. Consequently, institutions that develop, administer and assess language exams endeavour to offer exams that are modified according to the needs of examinees with disabilities. The research interest in the modification1 of test content, format, or administration procedures is growing. A lot of research has been conducted in the field of educational testing, especially in the context of school-based learning and assessment in the U.S. (see e.g. Abedi 2008). However, most of the studies concentrate on isolated and special problems, e.g. the use of Braille or the effect of extended testing time (see Khalifa and Weir 2009, 21–22). Questions regarding test validity are addressed in these studies. However, we still know little about the effects that accommodations have on test takers’ performances. This is due to the fact that research in this field is difficult to design, especially in the case of empirical studies. The empirical basis for selecting and randomising subjects is limited (Taylor 2012, 308; Thurlow et al. 2006, 669) because of a very heterogeneous target group and hence highly diverse accommodations. Nevertheless, this question is crucial in respect to fairness and thus has to be dealt with by language testers. There are two basic concerns that need to be looked at more closely. First of all, do candidates benefit from the accommodations in a way that they can show their real competence without biases provoked by their disability? Empirical investigations regarding this aspect have to consider all types of possible accommodations and compare their extent as well as their effect. Secondly, we have to investigate whether they have an influence on the test construct. In other words, does a candidate’s performance under

1

In this chapter, we use the terms accommodation and modification synonymously for all forms of changes in test material or administration of tests. Terms like special arrangements or adaptions can also be found in the literature (cf. Thurlow et al. 2006, 660).

On Equal Footing?

273

modified testing conditions allow us to make reliable inferences about his/her language proficiency in real life settings? In this article, we will give a short overview of the focus and scope of the Test of German as a Foreign Language (TestDaF). Then, we will present the assessment procedures and possible modifications of this exam available for test takers with special needs. The focus will be on the modifications offered for blind test takers. The differences in comparison to the standard format are discussed in relation to key issues of test quality.

2. Background to the examination The TestDaF is a standardised high stakes language test, set at the levels B2–C1 of the CEFR (cf. Kecker 2011). It is developed by the TestDaF Institute in Germany and administered in licensed test centres all over the world to assure access as well as controlled testing conditions. It is developed for students of all nationalities who plan to study at an institution of higher education in Germany and thus functions as an entry exam. The TestDaF therefore assesses language proficiency within an academic context at an intermediate to advanced level. Examinees have to show that they can understand and produce written and spoken texts that are relevant to everyday life and the academic realities at German institutions of higher education. All settings, tasks and topics are related to academic and study-related needs, and illustrate the required linguistic demands. Reading, listening, writing and speaking are assessed in separate sections. The subtest Reading Comprehension consists of three parts which assess the examinees’ ability to read texts of different style, length and level of abstraction. Also, the item format differs according to the cognitive operations which are to be measured. There are matching tasks as well as multiple-choice items and those in which the candidates have to choose one of three options in a selected response task. The subtest Listening Comprehension also consists of three parts, measuring the candidates’ ability to understand detailed as well as overall information. This component uses both the item format of choosing one of two options and demands note-taking while listening to a lecture or dialogue, assessed by short-answer questions. The Writing Component is designed to measure the examinees’ ability to produce a coherent and well-structured text on a given topic appropriate in an academic context. Candidates have to synthesise and evaluate

274

Chapter Fourteen

information from several given sources, such as a short introductory text providing background information, graph(s) or table(s) and given statements on the topic. The candidates have to develop an argument based on the data presented and experience, and comment on the central problem of the task in an appropriate way. Speaking Ability is measured using a semi-direct format based on the Simulated Oral Proficiency Interview (SOPI; Kenyon 2000). Instead of a face-to-face-interview between an interlocutor and the candidate, examinees are asked to react to a tape- or CD-mediated stimulus. The seven tasks are situated in different academic settings such as taking part in a seminar’s discussion or talking to another student. Candidates are asked to react to the presented stimulus in an appropriate way according to situation, task and register. The oral response is then recorded and sent back to the TestDaF Institute. All candidates’ performances are evaluated centrally in Germany. Written and spoken performances are rated by trained and monitored raters using well defined standardised criteria. Many-facet Rasch measurement is used to establish a fair result. It compensates differences between raters by adjusting the awarded TestDaF level according to rater severity or leniency and thus awarding a so called fair average (see Eckes 2008). The results of each component are assigned to one of three TestDaFlevels: TestDaF-Niveaustufe 3 (TDN 3), 4 (TDN 4) and 5 (TDN 5). Below TDN 3, proficiency levels are not differentiated. The certificate shows four separate results for each subtest, providing information on the strengths and weaknesses in each skill.

3. General accommodation policy of the TestDaF Institute As a member of ALTE the TestDaF Institute is committed to the Code of Practice which safeguards the rights of test takers amongst others in the area of fairness. Therefore the TestDaF Institute makes provisions for candidates with special needs. Such candidates range from test takers with a short-term disability, e.g. a broken arm, to test takers with permanent disabilities such as learning difficulties, visual, hearing or speech impairments. These very individual arrangements should guarantee foreign language learners access to the test, and eliminate construct-irrelevant variables that would influence their assessment. The main goal is to enable them to demonstrate their language skills to the best of their ability.

On Equal Footing?

275

Current accommodation practices for permanent disabilities offered by the TestDaF Institute (see Table 1) comprise generally accepted procedures which can be categorised as follows (Thurlow et al. 2006, 662–663): -

changes in the presentation of test material, such as Braille papers, or the use of an amanuensis2; modification of the way examinees respond, for example, by using word processors; accommodation of time, that is, extending the time for examinees according to their special needs; accommodation of setting, e.g. a separate room to reduce distractions, or a more accessible location for candidates with physical disabilities. Type of disability Learning difficulties

(e.g. Dyslexia, Attention Deficit Hyperactivity Disorder)

Visual impairments

Accommodations - additional time to read and write the answers - additional time - enlarged examination papers - modified preparation and test material in Braille - use of screen-reading software - use of specific writing equipment (e.g. Braille keyboard)

Hearing impairments

Speech impairments Other kinds of physical disability

- assistance from an amanuensis - lip-reading version of test material - use of special headphones or playing audio material at an increased level - omission of the Listening and Speaking Component - additional time in regard to speaking (not preparation) - omission of the Speaking Component - additional time to write the answers - use of a computer to write the answers - assistance from an amanuensis

Table 1. Current accommodation practices for permanent disabilities and learning difficulties

2

An amanuensis is someone who supports the test taker by reading out the test questions or by helping to write down answers.

Chapter Fourteen

276

Linguistic changes of the testing material such as using a simplified language or the alteration of content—also mentioned in the literature as possible accommodation practices (e.g. Thurlow et al. 2006, 663)—are commonly used in the context of content-based testing. However, we do not consider them appropriate in the context of language testing. The TestDaF covers relevant language tasks in an academic setting which candidates—and thus future students—will be facing during their studies, e.g. developing an argument systematically by taking into account different sources. Changes of content or simplifications of the language have an impact on difficulty, which leads to validity problems. The same concern regarding validity and, furthermore, equity of all test takers applies to marking. The policy of the TestDaF Institute is not to grant separate marking to candidates that have been offered special accommodations. This policy is consistent with that of other large-scale test providers. Shaw and Weir (2007) report on the policy change in regard to the Cambridge ESOL writing exams. The provision of separate marking in 2004 for dyslexic candidates was abolished as “not an appropriate Special Arrangement in language-focused assessment” (Shaw and Weir 2007, 27). Accommodations are not explicitly stated on the certificate, except if parts of the test were omitted due to the disability of the candidate. Similar to what other test institutions report on (e.g. Taylor and Gutteridge, 2003 for the Cambridge ESOL exams), the numbers of test takers who are granted accommodations (see Table 2) seem very small compared to the overall number of TestDaF participants 3 . Also, the number of test takers did not rise significantly over the years. Learning difficulties

Visual impairments

Hearing impairments

Speech impairments

Other disabilities

66

22

27

25

17

Table 2. Candidates’ number granted special arrangements (TestDaF: 2004–to present) Nevertheless, we would still like to draw attention to those numbers since they require intensive counselling. Each agreement on modifications is tailored to the individual needs of a candidate. Both test developers and the administration staff at the test centres are eagerly working on the best 3

In 2012, over 24,000 candidates took the TestDaF. For more information see www.testdaf.de.

On Equal Footing?

277

solution possible for each candidate. Those individual solutions cannot necessarily be transferred to other cases.4 The kind of accommodation granted depends on three different aspects: First of all, the medical perspective is crucial. Handing in a medical report is compulsory for all candidates with special needs who apply for accommodations. This report should include the type and degree of disability as well as medical recommendations on how to modify exam procedures. Secondly, the TestDaF Institute applies its own guidelines for those purposes. These are based on an inquiry among ALTE members and other test providers. They are regularly modified by evaluating the appropriateness of the accommodations granted. The guidelines deliberately bear flexibility and can be adopted individually, which brings in the third perspective: the test takers. Because their situation and needs are highly heterogeneous, it is necessary to ensure the appropriateness of the accommodations by communicating with the candidates. Balancing these different —sometimes contradictory—concerns and suggestions is essential to come to a professional judgement about the best possible practice. This is, therefore, the main goal of our policy.

4. Accommodations for blind candidates Amongst visually impaired individuals only a small percentage is blind and depends entirely on tactile (Braille) or auditory (screen reader) methods to read. Accommodations offered for blind candidates are broad and thus suitable for outlining the wide range of generally possible alterations. Due to this, the present chapter focuses on this small group to exemplify how the TestDaF is modified in order to meet the needs of test takers with disabilities. In discussing the accommodations for blind test takers, we will differentiate between modifications of test material on the one hand, and modifications of the test administration on the other hand. The latter comprises timing, response and setting. Test materials for blind candidates are modified and provided in two different versions: - as Braille question papers, using uncontracted Braille5, 4

Taylor points to the fact that decisions about modifications not only depend on the individual, but also on the cultural context (Taylor 2011, 53). 5 In uncontracted Braille (also known as Grade 1), every letter of every word is represented by an individual Braille symbol. In contracted Braille (also known as Grade 2), the symbols do not represent single letters, but stand for common letter combinations or even entire words. This version was developed to reduce the size of Braille texts and to increase reading speed. However, the meanings of those

Chapter Fourteen

278

-

as electronic question papers, processed via screen reading6.

The tests for blind candidates are redesigned versions of existing TestDaF exams that are altered according to the needs of the target group, but still cover the same assessment objectives. At the test pre-administration stage, blind candidates can familiarise themselves with the two options by working through available preparation tests and thereafter decide which version is more suitable for them. Irrespective of the version selected, some general administration arrangements are made for all blind candidates. The most common procedure is that blind candidates receive extra time to read the test material and to write the answers. According to individual needs, the extension can be up to 100%. This decision is based on the fact that the average reading speed of Braille readers is much slower compared to print readers (see Khalifa and Weir 2009, 22) and corresponds to the procedures of other large-scale test providers. Since reading is involved in every part of the TestDaF, the accommodation of timing affects the whole test. The same applies to the area of setting: blind candidates always take the test in a separate room. This is to ensure, on the one hand, that they are not disturbed by other test takers with a different time schedule. On the other hand, they themselves do not disturb others in case they make use of special equipment, e.g. a computer, or if they are assisted by an amanuensis. The use of technical equipment is only granted if it is guaranteed that disabled candidates using assistive technology are not advantaged by the use of functions that are not available to other candidates (e.g. thesaurus, spell check). To ensure that the administration of the TestDaF—even though it might be altered in some ways—remains as standardised as possible, test administrators are provided with a set of information before carrying out the exam with blind candidates. Apart from detailed guidelines on the administration itself, they are supplied with a review of the original test as well as the alternate format of the exam (e.g. standard print of the Braille paper). The above issues lead us to the following questions: What do the changes made to material and administration look like in practice? Upon shortcuts vary from language to language. The decision on presenting the test papers in uncontracted Braille was made upon the fact that not all blind students are familiar with contracted Braille, especially when reading in a second language. 6 Test takers deal with the test papers on a PC by means of a speech output software, which converts screen text to speech. In most cases, that software can also produce Braille output via Braille typewriter or display.

On Equal Footing?

279

what consideration have the changes been made and are they appropriate? Do changes also affect cognitive processes or even change content, and thus, potentially, requirements? Exemplified by parts of the TestDaF Reading and Writing Component, we will first describe significant changes and, then, discuss the questions posed.

4.1 Reading component As briefly outlined above, the TestDaF Reading Component consists of three tasks. Those vary in terms of length and density of text, item format, and difficulty. What changes have been made to the standardised format in order to present the reading input according to the needs of blind test takers? Time extension is applied to the Reading Component on a scale of 100% for both the Braille and the screen reading version. The changes made to the presentation of text and items are especially significant in the Braille version. A closer look at the second reading task helps to illustrate this. The task is composed of a text (450–550 words) and 10 multiplechoice items. In the regular test, both parts are presented together on one double page, the text on the left and the items on the right side. Hence, test takers can individually decide how to work through the task—they can first read the text and then the items, or vice versa. In both cases, they have to identify independently those passages of the text that relate to the answers to the items. This standardised layout is maintained in the Braille version, but supplemented by an alternative presentation mode. After text and items are displayed as described for the standard version, the text is presented again paragraph by paragraph. The items are integrated in the text, always placed in front of the paragraph that includes the answer to the question. One could argue that this change in presentation makes it easier for blind candidates to match text passages and items, hence alters the requirements and changes difficulty. However, two arguments can be brought forward against this. Firstly, the change in layout does not lead to a decrease in difficulty, as the items in the regular test also follow the development of the text and are therefore easy to match as well. Secondly, uncontracted Braille letters are much bigger than normal print. In the regular TestDaF, candidates have all the necessary information (here: text and items) on one double-page. Braille readers, however, find the same information spread over seven pages and are hence forced to continuously page backward and forward, while keeping all the information in mind. This might put a greater workload on their short-term memory and thus

280

Chapter Fourteen

possibly threaten validity. Because of this very reason, one—fairly subtle—change in material arrangements is also made to the first reading task, a matching task consisting of eight short texts and 10 items. Whereas the regular format presents the whole task printed on one double page, the Braille version spreads texts and items over a couple of pages. In order to equalise requirements and to grant Braille readers the same flexibility as their non-blind counterparts, they are provided with a loose, separate piece of paper on which the items are displayed. With this, they can align texts with items flexibly. For the screen reader version, all tasks are stored in electronic documents. Special formatting guidelines ensure that the software in use is able to transfer the written text into comprehensible spoken output. In contrast to the Braille version, the electronically processed reading paper has not been changed in terms of arrangement of the tasks. All tasks are presented in the order of the standardised format. One major difference between the screen reading version and Braille or black print is that in the audio format all content has to be presented in a linear way and at predetermined speed. The candidate thus cannot decide as independently as in printed versions how to read the texts and the items. This also implies that reading techniques such as quick scanning, rereading or reviewing can only be applied to a very limited extent. This might lead to a greater workload on the short-term memory, since all the information has to be stored temporarily (see Educational Testing Service 2010, 34–35). We try to reduce this impact by clearly marking crucial points in the document, e.g. endings of paragraphs, the beginning of a new task or the numbers of the items, to make the navigation of texts and items as easy and flexible as possible. The most important question concerning the use of screen readers for the Reading Component that is addressed in many papers is, however, of a different and more fundamental nature. If all the reading texts are presented via auditory input, is it still reading ability that we actually measure? According to the changes made to the presentation of test material, we also provide blind test takers with different response formats. Whereas in the standard version participants have to transfer their answers to separate marking sheets, blind candidates can choose between two alternative answering opportunities: they can note them with their Braille typewriter (computer respectively) or dictate them to an amanuensis. Consequently, the standardised directions on the answering behaviour have been rephrased in both versions.7 7

For the importance of rephrasing answering directions see Allman 2009, 12.

On Equal Footing?

281

4.2 Writing component The presentation of input material for blind candidates in the Writing Component differs in format in much the same way as does the Reading Component from the standardised test format. Usually the whole task is presented on a double page, with the introductory text on the left side, and graph(s) and table(s) as well as given statements and instructions on the right. But since this clearly arranged presentation is not possible in Braille and in the screen reader version, all textual and visual material is presented successively. For Braille readers this requires the turning of pages back and forth to capture all the necessary information provided, and hence leads to time consuming activities before candidates can even start to actually write their text. The most serious adaptation though is the different presentation of visual input: On the one hand, one can argue that the comprising of statistical data is essential in relation to the test construct and therefore should not be excluded from the test “even if they present accessibility barriers” (Educational Testing Service 2010, 7). One solution would be to verbalise the information, i.e. to provide candidates with relevant parameters of the input without using specific vocabulary of a graph description. This is a challenge in test development.8 An example of how this is achieved in the TestDaF Writing Component is provided in Fig. 1. In this particular case, the line graph and bar chart are not very complex, and therefore the statistical information can be described, i.e. converted into words and numbers without greater difficulties. This could be more problematic in the case of more complex graphs and tables, especially if different degrees of shading are included. Typically different colours or shadings are used in this context to help test takers capture all relevant information included in the graphical input without great efforts. This is done for reasons of clarity and therefore minimises the cognitive processing in relation to handling all the material provided. In the case of blind test takers, this clarity cannot be provided. Sometimes, even more words are needed to describe different parameters of a graphical input. This might be a disadvantage for these particular test takers.

8

The use of tactile graphs for the Braille version was considered, but finally rejected as not every Braille reader is familiar with them. Also not every graph can be shown as a tactile graph, e.g. if it is complex and/or has many details.

282

Chapter Fourteen

Fig. 1. Graphical Input for the TestDaF Writing Component: original graph and verbal description for blind test takers.

Another aspect in regard to fairness that needs to be mentioned is the fact that the presentation of data might influence the rating of the writing performance. One aspect in assessing the candidate’s performance in the TestDaF is how the information held in the graph or table(s) is presented by the candidate. Is it done by simply enumerating data, or is the information described clearly and logically? In the given example, data in the Braille and screen reader version are arranged in a special way which might lead to enumerating. Comparing and contrasting data might be hindered by this kind of presentation and therefore result in a weaker scoring of the performance. Besides modifications in the presentation of test material, blind candidates are offered different options to respond by using access technology. Depending on what they are more familiar with, they can either write their texts on a computer or use a Braille typewriter.9 What might be an accommodation in favour of the candidate, raises concerns: The fact that word processed texts seem shorter in comparison to handwritten performances, and errors are discernible more easily, might have an impact on the rating. It is also questionable if the test construct 9

The option that candidates could dictate their response was not considered as a valid accommodation (see Abedi 2008, 334).

On Equal Footing?

283

alters, while changing from the medium of handwriting in a paper-pencil test to the medium of computer administration.10

5. Discussion In this chapter we outlined the accommodation practices in use with the TestDaF, focusing on the special arrangements which are offered for blind candidates. The departing from the standardised test format and administration inevitably has consequences for test validity and fairness. Concerning validity, we argued that especially the medium of presentation—influenced by new means of accessed technology—raises the question to what extent the underlying test construct is altered, or even violated. First of all, a concern could be if a reading test which is presented through another channel—in the case of blind test takers through audio presentation—is still a test of reading (Fulcher 2010, 236). Though little empirical research has been conducted on this question11, it is discussed in many papers. Most researchers agree on the fact that reading in Braille entails, to a great extend, similar decoding processes as print reading and therefore consider it to be a valid accommodation in reading tests. However, screen readers are seen more critically (see Allman 2009, Appendix A). Some argue that read-aloud accommodations skip the decoding processes and therefore generally do not consider them to be a valid measurement in reading tests (see Thurlow et al. 2009). Others deal with screen readers in a more differentiated way, pointing out that the appropriateness of its use depends on the construct assessed by particular items. If decoding skills are measured, tactile material needs to be provided to ensure fair assessment. If, however, comprehension skills are to be assessed, auditory input is considered as a valid way to do so. In such cases, test developers have to ensure that the presented tasks do measure reading and not listening (Allman 2009, 56). Despite such unanswered questions of comparability and possible limitations to validity, we argue that, in terms of fairness and free 10

For the impact of response format on scores, see Shaw and Weir 2007, 176–180. See Laitusis 2010 for one of the rare empirical studies on the impact of readaloud accommodations on standardised test scores of reading comprehension. Laitusis compared the performance of students with and without reading-based learning disabilities in both a standard administration and a read-aloud administration of a reading comprehension test. She found that the mean score on the audio version was higher than scores on the standard version for both groups of students. Results also proved that students with reading-based learning disabilities benefited differentially more than students with no disability.

11

284

Chapter Fourteen

accessibility of the TestDaF, it is important to offer both versions. Not all blind students are efficient Braille readers and not all methods work for all students equally. Furthermore, to assure valid assessment, the simplification of language or the omission of input material, e.g. the graphical input in the Writing Component, does not seem appropriate in the context of language testing. If essential features of the test construct are excluded from the test, interpretation of test scores is problematic.12 The impact on test reliability has not been discussed explicitly in this chapter, but will hopefully receive more attention in future research. One aspect that would be of interest is that of additional time. This is generally perceived as one of the most effective and appropriate accommodations for disabled candidates. Available time extensions range from 50 up to 150%. However, relevant studies come to different conclusions on the effect of additional time. The question of how much extra time is appropriate for which situation remains yet unanswered. A different approach to the question of reliability in this field might be to see if the performance of blind candidates is influenced by the mode of presentation, i.e. if Braille papers and electronic question papers (processed via screen reading) pose the same requirements and thus test the same ability. In conclusion, we consider striving for fairness as the basis for all accommodation policies. Most professional judgements on appropriate modifications for disabled candidates are based on little empirical research findings, either relying on medical expertise or focussing on test quality issues. More research is needed in both fields. Findings in these fields together with the candidates themselves who are aware of their specific needs will hopefully lead to the best possible practice.

Works cited Abedi, Jamal. 2008. “Utilizing Accommodations in Assessment.” In Encyclopedia of Language and Education (2nd edition). Language Testing and Assessment. Vol.7, edited by Elena Shohamy and Nancy Hornberger, 331–347. New York: Springer Science+Business Media LLC. Allman, Carol B. 2009. Making Tests Accessible for Students with Visual Impairments: A Guide for Test Publishers, Test Developers, and State 12

See Taylor 2012 for the problematic issue of interpreting test scores when the exam has been modified.

On Equal Footing?

285

Assessment Personnel (4th edition.) Louisville, KY: American Printing House for the Blind. Association of Language Testers in Europe (ALTE). 1994. “Code of Practice”. Accessed July 15, 2012. http://www.alte.org/setting_ standards/code_of_practice. Eckes, Thomas. 2008. “Assuring the Quality of TestDaF Examinations: A Psychometric Modeling Approach.” In Multilingualism and Assessment: Achieving Transparency, Assuring Quality, Sustaining Diversity - Proceedings of the ALTE Berlin Conference May 2005, edited by Lynda Taylor and Cyril J. Weir, 157–178. Cambridge: Cambridge University Press. Educational Testing Service. 2010. “How ETS Works to Improve Test Accessibility.” Accessed November 14, 2012 http://www.ets.org/s/ about/pdf/how_ets_works_improve_accessibility.pdf. European Association for Language Testing and Assessment (EALTA). 2006. “Guidelines for Good Practice in Language Testing and Assessment.” Accessed October 16, 2012. http://www.ealta.eu.org/ guidelines.htm. European Parliament. 2011. “Report on Mobility and Inclusion of People with Disabilities and the European Disability Strategy 2010–2020“. Accessed November 15, 2012. http://www.europarl.europa.eu/sides/ getDoc.do?pubRef=-//EP//TEXT+REPORT+A7-2011-0263+0+DOC+ XML+V0//EN. Fulcher, Glenn. 2010. Practical Language Testing. London: Hodder Education. Kecker, Gabriele. 2011. Validierung von Sprachprüfungen. Die Zuordnung des TestDaF zum Gemeinsamen Europäischen Referenzrahmen für Sprachen. Frankfurt: Lang. Kenyon, Dorry M. 2000. “Tape-Mediated Oral Proficiency Testing: Considerations in Developing Simulated Oral Proficiency Interviews (SOPIs).” In TestDaF: Grundlagen für die Entwicklung eines neuen Sprachtests. Beiträge aus einem Expertenseminar, edited by Sibylle Bolton, 87–106. München: Goethe-Institut. Khalifa, Hanan, and Cyril J. Weir. 2009. Examining Reading. Research and Practice in Assessing Second Language Reading. Cambridge: Cambrigde University Press. Laitusis, Cara Cahalan. 2010. “Examining the Impact of Audio Presentation on Tests of Reading Comprehension.” In Applied Measurement in Education (Special Issue: Testing Students with Disabilities) 23:153–167.

286

Chapter Fourteen

Shaw, Stuart D., and Cyril J. Weir. 2007. Examining Writing. Research and Practice in Assessing Second Language Writing. Cambridge: Cambrigde University Press. Taylor, Lynda, ed. 2011. Examining Speaking. Research and Practice in Assessing Second Language Speaking. Cambridge: Cambrigde University Press. Taylor, Lynda. 2012. “Accomodation in Language Testing.” In The Cambridge Guide to Second Language Assessment, edited by Christine Coombe, Peter Davidson, Barry O’Sullivan and Stephen Stoynoff, 307–315. Cambridge: Cambridge University Press. Taylor, Lynda, and Mike Gutteridge. 2003. “Responding to Diversity: Providing Tests for Language Learners with Disabilities.” In Research Notes 11, 2–4. Accessed September 30, 2012. http://www. cambridgeesol.org/rs_notes/rs_nts11.pdf. Thurlow, Martha L., Sandra J. Thompson, and Sheryl S. Lazarus. 2006. “Considerations for the Administration of Tests to Special Needs Students: Accommodations, Modifications, and More.” In Handbook of Test Development, edited by Downing, Steven M. and Haladyna, Thomas M., 653–673. Mahwah, NJ: Lawrence Erlbaum, Inc. Thurlow, Martha L., Ross E. Moen, Kristin K. Liu, Sarah Scullin, Kristin E. Hausmann, and Vitaliy Shyyan. 2009. Disabilities and Reading: Understanding the Effects of Disabilities and Their Relationship to Reading Instruction and Assessment. Minneapolis, MN: University of Minnesota, Partnership for Accessible Reading Assessment.

CONTRIBUTORS (in alphabetical order)

Abdel-Sabour, Shaimaa has been a teacher of English as a foreign language for more than 18 years; though she has also taught Arabic as a foreign language and English for specific purposes primarily for the banking and business sectors. She has worked with 6 years old to adults, and is trained as a dyslexia therapeutic tutor. Currently, she is conducting research on Arabic/English Bilingual Reading Acquisition for her PhD via the University of Canterbury, New Zealand. Al-Menaye, Nasser is Assistant Professor in Psychology at the University of Kuwait. He has a Masters in Cognitive Neuropsychology and obtained his PhD on the relationship between working memory and Arabic reading at the University of Surrey, UK. He teaches courses in a range of areas related to psychology and education but with a specialization in cognitive neuropsychology. His interests focus mainly on the relationship between memory and learning, particularly among speakers of Arabic. Amendum, Steve J. PhD, is Assistant Professor of Literacy Education at North Carolina State University in Raleigh, North Carolina, USA. His current research and teaching interests focus on early literacy intervention, literacy development for multilingual learners, and classroom-based literacy instruction reform efforts. His research has been published in premier literacy journals, and in 2009 he won the Promising Researcher Award from the National Council of Teachers of English (USA). Antoniou, Faye PhD, is Lecturer at the Department of Philosophy, Pedagogy and Psychology in the National and Kapodistrian University of Athens, Greece. She is also a member of the teaching staff at the Department of English at the Hellenic Open University. Her research interests focus on enhancing the reading comprehension and written expression of students with special learning disabilities. She is also interested in the in-service training of primary and secondary educators.

288

Contributors

Arras, Ulrike works as Test Development Officer at the TestDaF Institute since 2001. She is responsible for the training of item writers and raters. She has worked for several years as teacher for German as foreign language and as teacher trainer in different cultural and linguistic contexts. She has published in the field of language exams quality assurance. In her dissertation she analyzed assessment strategies using introspective methods. Ballantyne, Keira Gebbie is Research Scientist in the Graduate School of Education and Human Development at The George Washington University. Her work in educational linguistics includes publications on early childhood education for English learner students and on professional development for content area teachers of English learner students. She has also contributed to reports to the Congress of the United States on accountability and English learner education. Banerjee, Jayanti is CaMLA’s Research Director and oversees research and development activities for all of CaMLA’s international testing programs. Her previous research activities include a study to document the linguistic markers of different levels of language proficiency and an investigation of differential item functioning on the Michigan English Test. She has published in the areas of language testing and English for academic purposes and is on EALTA’s membership committee. Brannen, Kathleen holds a PhD in linguistics from McGill University in Montreal. She is Lecturer at the École de langues at the Université du Québec à Montreal (UQAM) where she mainly teaches English pronunciation. She also taught courses in linguistics at the University of Ottawa and McGill. Since 2009 she has been involved in the School’s projects on ESL teaching to students with visual and hearing impairments. D’ Este, Claudia is Language Testing Professional at the Ca’ Foscari University of Venice, Italy. She holds an MA in Language Testing from the University of Lancaster, UK. Her research interests include language testing and the testing of students with SpLDS. De Monte Tagarelli, Maria is a PhD candidate in Applied Linguistics at the Università degli Studi Roma Tre and Junior Researcher at the Istituto Statale per Sordi di Roma. She holds a BA in Communication Sciences, University of Siena, with a major in Cognitive Science and Interaction

Assessing L2 Students with Learning and Other Disabilities

289

Design. Her research interests concern bilingualism in D/deaf learners and their children, with a special focus on cognitive and pragmatic linguistics. Dobson, Barbara is Director of Professional Partnerships and leads CaMLA in establishing and maintaining relationships between CaMLA and external organizations. As a language assessment professional, she has developed and overseen the administration of tests for a wide variety of levels and purposes, conducted and presented research on CaMLA exams, and served on executive committees and review panels for ILTA and MwALT. Elbeheri, Gad is Associate Dean of Foundation & Academic Support at the Australian College of Kuwait and a consultant at the Centre for Child Evaluation and Teaching, Kuwait. He is also Technical Director of the Early Learning Challenges and Disability Project of the United Nations Development Programme. An applied linguist who obtained his PhD from the University of Durham, UK, his research focuses on cross-linguistic studies of dyslexia and other specific learning difficulties and their manifestations in Arabic. Elshikh, Mohamed completed his PhD on Predictors of Literacy amongst Arabic-English Bilingual Children at the University of Surrey, UK, and now works in Kuwait. His background as a teacher of English as a foreign language (EFL) has led to an interest in factors in EFL teacher education that may influence children’s achievement. His main research interests are in second language literacy difficulties and intervention programs that may help improve literacy skills for low-achievers. Erbeli, Florina is a PhD student at the University of Ljubljana, Faculty of Education, Slovenia. Her research interests include developing strategies for improving reading fluency and reading comprehension skills with students learning English as a foreign language, in particular students with specific learning differences. She is the author of a student’s book for English for special education students and of articles on reading fluency and reading competence in foreign language instruction. Everatt, John is Professor of Education at the College of Education, University of Canterbury, New Zealand. He received his PhD from the University of Nottingham and, before moving to NZ, he lectured in Psychology at the Universities of Wales and Surrey, UK. His research focuses on literacy acquisition and literacy learning difficulties, as well as

290

Contributors

additional language learning and cross-language/orthography comparisons, and has involved research projects in a large number of countries around the world. Fitzgerald, Jill is Distinguished Research Scientist at MetaMetrics, Durham, North Carolina, USA, and Professor Emerita and Research Professor Adjunct at The University of North Carolina at Chapel Hill. Her current research interests center on multilingual literacy, text complexity, and composing. A member of the Reading Hall of Fame, she has published over 100 works and won research awards from the American Educational Research Association and the International Reading Association. Grech, Louisa is Senior Visiting Lecturer at the University of Malta. She has a Master’s degree from the University of Southampton, UK, in Specific Learning Difficulties, and completed her PhD on understanding reading comprehension levels of Maltese-English bilinguals at the University of Surrey, UK. Her research interests include dyslexia, inclusion, literacy and bilingual development. She currently forms part of the Maltese national committee for language and policy making in Education. Groves, Katherine M. is a PhD candidate in Applied Linguistics at the Università degli Studi Roma Tre. She collaborates with the Istituto Statale Sordi Roma as a Deaf researcher and specialist in bilingual Deaf education and English as a foreign language for Deaf learners. Her research interests include impact of language attitudes on language learning, adaptation of language assessment, foreign language teacher training, and literacy programs for Deaf youth. Karacic, Yvonne is a secondary school teacher and Lecturer in TEFL, Department of English Studies, Goethe University in Frankfurt, Germany. She has participated in research projects in the field of teaching and learning foreign languages and at-risk students. She is also the coordinator of the language town project which is designed and prepared with teacher trainees and English teachers at a school for at-risk and special needs students in downtown Frankfurt. Khalifa, Hanan is Head of Research and Publications and Research and Validation at the University of Cambridge ESOL Examinations. She has wide experience of educational assessment, holding a PhD in Language

Assessing L2 Students with Learning and Other Disabilities

291

Testing from the University of Reading (UK). Hanan has extensively presented, published and consulted on educational reform and impact, bilingual education, school effectiveness, capacity building in test development and issues related to the CEFR. Her 2009 book, Examining Reading (co-author Cyril Weir), is used for ALTE training and UK Masters programmes. Hanan’s 2010 research on the effectiveness of bilingual teaching and learning in UAE has been endorsed by ADEC and reported in the UAE National newspaper. Kormos, Judit is Reader in Second Language Acquisition at Lancaster University. She is the co-author of the book Teaching Languages to Students with Specific Learning Differences and a co-investigator in the Dyslexia for Teachers of English as a Foreign Language project sponsored by the European Commission. Her research interests include the psycholinguistics of second language acquisition and second language speech production. Kozlowska, Martyna holds a PhD in Linguistics from McGill University in Montreal. She works as a Maître de langues at the École de langues at the Université du Québec à Montreal (UQAM) where she teaches on the relationship between syntax and interpretation of English sentences. She has been involved in School projects on placement testing and the integration of students with disabilities in ESL classes. Kung, Melody is currently a PhD candidate in the Early Childhood, Special Education, and Literacy Program in the School of Education at The University of North Carolina at Chapel Hill, USA. She is also a Research Intern at MetaMetrics in Durham, North Carolina, USA. Her current research interests include second-language learners and bilingual literacy. Loumbourdi, Lana is Lecturer and Research Assistant in TEFL/TESL, Department of English Studies, Goethe University in Frankfurt, Germany. She has taught, published and researched in the area of language assessment and evaluation and task-based teaching, and has participated in projects investigating the impact of language testing and task based learning. Ludbrook, Geraldine is Researcher in English Language and Linguistics at the Ca’ Foscari University of Venice, Italy. She holds a PhD in Applied Linguistics from the University of Lancaster, UK. Her research interests

292

Contributors

include language test development and the language testing of students with SpLDS. McNeill, Brigid is Senior Lecturer in Literacy Education at the College of Education, University of Canterbury, New Zealand. A former speech and language therapist, she has written many papers on the relationship between language and literacy. Her research focuses primarily on understanding the nature and educational consequences of developmental speech disorders (particularly developmental verbal dyspraxia) and the effectiveness of interventions aimed at supporting children, including those from different language backgrounds, in their acquisition of literacy. Müller, Anika works as Test Development Officer at the TestDaF Institute since 2011. She is responsible for both the reading comprehension section of the TestDaF and the unit providing special arrangements for test takers with disabilities. She has been a member of academic staff at the Department for German as a Foreign Language at Saarland University and has worked as a teacher in the same field in Turkey. Nordby Chen, Natalie serves as CaMLA’s Director of Assessment, guiding activities that create, support, and strengthen CaMLA testing programs. Her professional assessment experience includes developing a wide range of testing products; conducting item writing and rater training workshops; serving on professional review panels; mapping test specifications to governmental standards; and participating on research teams for both organizational and grant-funded projects. She has taught ESL/EFL in the United States and Taiwan. Orletti, Franca is Professor of Linguistics and Historical Change at the Univeristà degli Studi Roma Tre and a member of the Doctoral Committee of the Linguistics Department. She is head of numerous research groups and projects with a focus on language learning, identity and multimodality in communication. She is also interested in the role of bureaucratic texts and functional writing. Padeliadu, Susana is Professor in Special Education/Learning Disabilities, Department of Philosophy and Pedagogy in Education, Aristotle University of Thessaloniki in Greece. She has coordinated a large number of research and training projects in the field of special education and she has published her work in many national and international

Assessing L2 Students with Learning and Other Disabilities

293

academic journals. Her major research interests involve the identification of learning disabilities, the inclusion of students with LD and teacher preparation. Pižorn, Karmen is Assistant Professor of English Language Teaching Methodology at the University of Ljubljana, Faculty of Education, Slovenia. She has taught various courses at BA, MA and PhD levels related to English language teaching methodology, language assessment and general English. Her research interests include all aspects of language testing, teaching foreign languages to young learners, and other issues involved in learning and teaching foreign languages in general. Sadeghi, Amir is Post-doctoral Researcher in the College of Education, University of Canterbury, New Zealand, and Lecturer at the Islamic Azad University, Damavand Branch, Iran. He obtained his PhD in education (language and literacy) at the University of Canterbury, NZ. He teaches courses in second language acquisition (SLA) and language assessment, in particular TESOL. His research interests focus mainly on language acquisition and inter-language influences particularly among speakers of Persian and second language learners of English. Smith, Anne Margaret is the Director of ELT well and Learning Development Adviser at the University of Cumbria, UK. Through her work she seeks to bring professionals from the two worlds of ELT and SpLD closer together, in order to share good practice and support multilingual learners who have learning differences. She offers training and inclusive materials to both communities, as well as appropriate assessments to identify SpLDs. Sparks, Richard L. is Professor in the Department of Education at the College of Mt. St. Joseph in Cincinnati, Ohio USA, where he teaches courses in reading and dyslexia, learning disabilities, assessment, and research. He has conducted numerous studies and published extensively in both the L2 and learning disabilities literature. He has a private practice in which he conducts psychoeducational evaluations of children, adolescents, and adults, and consults with professional organizations. Taylor, Lynda holds an MPhil and a PhD in Language Testing and Assessment, both from the University of Cambridge, UK. For over 30 years she has accumulated extensive knowledge and experience of the theoretical and practical issues in language teaching, learning and

294

Contributors

assessment. With a background in modern languages and English language education, she has worked closely with Cambridge ESOL since the mid 1980s, both as an external consultant and as a fulltime member of staff. Zimmermann, Sonja is Test Development Officer at the TestDaF Institute. She has many years of experience in providing operational and quality assurance support for the TestDaF writing component and has published in this field. Her current research interests include assessing productive skills with specific reference to the testing of speaking, and the impact of technology on testing.