Trends in E-Tools and Resources for Translators and Interpreters 9004351795, 9789004351790

Citation preview

Trends in E-Tools and Resources for Translators and Interpreters

Approaches to Translation Studies Founded by James S. Holmes Edited by Henri Bloemen Leo Tak-hung Chan Cees Koster Ton Naaijkens Şebnem Susam-Saraeva

VOLUME 45

The titles published in this series are listed at brill.com/atts

Trends in E-Tools and Resources for Translators and Interpreters Edited by

Gloria Corpas Pastor Isabel Durán-Muñoz

LEIDEN | BOSTON

Cover Illustration: The Rosetta Stone, British Museum Creative Commons Library of Congress Cataloging-in-Publication Data Names: Corpas Pastor, Gloria, editor. | Duran-Munoz, Isabel, editor. Title: Trends in e-tools and resources for translators and interpreters / edited by Gloria Corpas Pastor, Isabel Duran-Munoz. Description: Leiden : Brill/Rodopi, [2018] | Series: Approaches to translation studies ; Volume 45 Identifiers: LCCN 2017048043 (print) | LCCN 2017050074 (ebook) | ISBN 9789004351790 (E-book) | ISBN 9789004351783 (hardback : alk. paper) Subjects: LCSH: Translating and interpreting--Technological innovations. | Translating and interpreting--Computer network resources. | Translating and interpreting--Data processing. Classification: LCC P308 (ebook) | LCC P308 .T74 2018 (print) | DDC 418/.020285--dc23 LC record available at https://lccn.loc.gov/2017048043

Typeface for the Latin, Greek, and Cyrillic scripts: “Brill”. See and download: brill.com/brill-typeface. issn 0169-0523 isbn 978-90-04-35178-3 (hardback) isbn 978-90-04-35179-0 (e-book) Copyright 2018 by Koninklijke Brill nv, Leiden, The Netherlands. Koninklijke Brill nv incorporates the imprints Brill, Brill Hes & De Graaf, Brill Nijhoff, Brill Rodopi, Brill Sense and Hotei Publishing. All rights reserved. No part of this publication may be reproduced, translated, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without prior written permission from the publisher. Authorization to photocopy items for internal or personal use is granted by Koninklijke Brill nv provided that the appropriate fees are paid directly to The Copyright Clearance Center, 222 Rosewood Drive, Suite 910, Danvers, ma 01923, usa. Fees are subject to change. This book is printed on acid-free paper and produced in a sustainable manner.

Contents Foreword vii Acknowledgements ix List of Illustrations x Introduction 1 Gloria Corpas Pastor and Isabel Durán-Muñoz

part 1 Electronic Tools for Translators 1 Investigating the Use of Resources in the Translation Process 9 Joanna Gough 2 User Perspective on Translation Tools: Findings of a User Survey 37 Anna Zaretskaya, Gloria Corpas Pastor and Míriam Seghiri 3 Assessing Terminology Management Systems for Interpreters 57 Hernani Costa, Gloria Corpas Pastor and Isabel Durán-Muñoz 4 Human Translation Technologies and Natural Language Processing Applications in Meaning-based Translation Learning Activities 85 Éric Poirier

part 2 cat and cai Tools 5 Monitoring the Use of newly Integrated Resources into cat Tools: A Prototype 109 Aurélie Picton, Emmanuel Planas and Amélie Josselin-Leray 6 Can User Activity Data in cat Tools help us measure and improve Translator Productivity? 137 John Moran, David Lewis and Christian Saam

vi 7

contents

Computer-assisted Interpreting: Challenges and Future Perspectives 153 Claudio Fantinuoli

part 3 Machine Translation 8 The ACCEPT Academic Portal: A Pre-editing and Post-editing Teaching Platform 177 Pierrette Bouillon, Johanna Gerlach, Asheesh Gulati, Victoria Porro and Violeta Seretan 9

The Challenge of Machine Translation Post-editing: An Academic Perspective 203 Celia Rico, Pilar Sánchez-Gijón and Olga Torres-Hostench

10 scate Taxonomy and Corpus of Machine Translation Errors 219 Arda Tezcan, Véronique Hoste and Lieve Macken

Appendix 1 245



Index 249

Foreword We witnessed the birth of the modern computer between 1943 and 1946; it was not long after that when Warren Weaver wrote his famous memorandum in 1949 suggesting that translation by machine might be possible. Weaver’s dream did not quite come true: while automatic translation went on to work reasonably in some scenarios and to do well for gisting purposes, even today, against the background of the latest promising results delivered by statistical Machine Translation systems such as Google Translate, automatic translation is not good enough for professional translation. There was a pressing need for a new generation of tools to assist and speed up the translation process more reliably and in 1971 Krollman put forward the reuse of existing human translations. In 1979 Arthern went further and proposed the retrieval and reuse not only of identical text fragments (exact matches) but also of similar source sentences and their translations (fuzzy matches). It took another decade before the ideas sketched by Krollman and Arthern were commercialised as a result of the development of various computer-aided translation (cat) tools such as Translation Memory systems in the early 1990s. These e-tools revolutionised the work of translators and the last two decades saw dramatic changes in the translation workflow. In the 1990s another development established itself as an important trend and has had an increasing impact on the future work of translators and, later, on interpreters. Computers not only made it possible for experts to develop purpose-specific translation tools; they also made it possible to collect and exploit electronic data conveniently. This gave rise to the collection of growing amounts of monolingual, parallel and comparable corpora, which were to emerge as an invaluable resource for translation (and now for interpretation). We are describing a scenario where the translator (or the interpreter) is the centre of attention. While machines and data in general (and tools and resources in particular) are important assistants in the translation process, it is human users, such as translators, who are the main characters who take the decisions and who decide how and to when to use machines or data. This applies to the case of translation tools that assist the translation process by proposing solutions which can be overridden by translators or to cases where the output of a Machine Translation program is post-edited or analysed by humans. In this scenario the translator and the interpreter, or the human user in general, is the main character. This main character is the focus of the current volume and the editors deserve credit for choosing this scenario.

viii

Foreword

This volume is a collection of contributions from key players in the field who discuss various aspects related to the tools and resources for translators and interpreters. While the range of tools and resources for translators has been growing in recent years, their counterparts for interpreters are still very scarce and their development is in its infancy. In both cases, however, recent advances have not been sufficiently documented and this volume fills this gap in the literature in a convincing manner. In this volume, the latest tools and resources for translators and interpreters are presented, methodological issues related to the teaching and training are discussed and the needs of translators and interpreters are analysed and reported. One particular strength of this volume is the balance and complementarity: there are contributions which can serve both scholars and practitioners. In addition, this volume is accessible and comprehensive enough to be of benefit to both categories of readers. Last but not least, the editors are to be commended not only for attracting these excellent contributions, but also for providing a wider picture and looking not only at the world of technology for translators but also at the hitherto overlooked world of technology for interpreters. In doing so, they pave the way for the research and development that will address the needs of interpreters. Ruslan Mitkov

Acknowledgements This edited volume has been supported in part by the Spanish Ministry of Economy and Competitiveness (grant nos. FFI2012-38881 and FFI2016-75831P); the Andalusian Regional Government (grant no. HUM2754); and the European Union’s FP7 (grant no. 317471). We would also like to thank the following reviewers for their advice, which brought the quality of the manuscript to the highest level (in alphabetical order): Juan José Arevalillo (Hermes Servicios Lingüísticos, s.l., Spain) Silvia Bernardini (University of Bologna, Italy) Bart Defranq (University of Ghent, Belgium) Joao Esteves-Ferreira (Tradulex, Switzerland) Purificación Fernández Nistal (Universidad de Soria, Spain) Koen Kerremans (Vrije Universiteit Brussel, Belgium) Luis Meneses-Lerín (University of Artois, France) Joanna Monti (Università degli Studi di Sassari, Italy) Ruslan Mitkov (University of Wolverhampton, uk) Pedro Mogorrón (University of Alicante, Spain) Michael Oakes (University of Wolverhampton, uk) Constantin Orasan (University of Wolverhampton, uk)

List of Illustrations Figures 1.1 The structure of a research action 18 1.2 The ttrs grid showing the initial positioning of individual translators according to their primary categories weight scores 28 1.3  t trs grids showing repositioning of participants due to the influence of secondary categories 29 2.1 Employment of the survey participants 44 2.2 Respondents’ familiarity with different types of tools 45 2.3 Usefulness of features of a cat tool 48 4.1 Sentence modifier meaning representation 98 4.2 Phrasal modifier meaning representation 98 4.3 Complex sentence meaning representation 99 4.4 Robert and Collins (2004) entry provided to learners 102 5.1 Contextual needs as a continuum of contextual, pragmatic and encyclopaedic information (Varantola, 1998: 182) 112 5.2 Example of present-day tws (2015): a MemoQ screenshot 116 5.3 Example of Wordfast Pro 4 screen for keyboard shortcuts management 117 5.4 Extract from ‘L’Acier dans les industries chimiques’, European Commission, 1968 118 5.5 Argos interface 121 5.6 Examples of krcs in French and English 123 5.7 Extract from a focus group interview 124 5.8 Example of similarity between a frequently chosen context and the source text 125 5.9 Extract from an interview concerning the need for identifiable source 125 5.10 Extract from the online questionnaire concerning the need for identifiable source 125 5.11 Example of frequently chosen context quoting a source 125 5.12 Extract from an interview on the relevance of providing contexts including several types of information 126 5.13 Extract from an interview on the ideal length of contexts 126 5.14 Extract from an interview on the need for highlighting the search word 127 5.15 Extract from the online questionnaire on the need for highlighting the search word 127 5.16 Extract from the online questionnaire on working pattern b 128

List of Illustrations

xi

5.17 Extract from logs 128 5.18 Extract from a video interview on the need for switching from one resource to another during the translation process 129 5.19 Extract from an interview on the need for referring to source krcs 129 5.20 Extract from an interview on the need for referring to source and target krcs 129 5.21 Extract from an interview on personal habits of translators 130 5.22 Extract from the questionnaire: interface ergonomics 130 5.23 Extract from an interview 131 6.1 A shortened example of a segment editing session 143 6.2  h t versus mt translation throughput ratios for Autodesk data from 2012 147 6.3  h t versus mt translation throughput ratios for Dell data from 2012 147 6.4 Typing speed in words per second on autodesk 149 6.5 Typing speed in words per second on dell 149 6.6 A wireframe mockup to illustrate a privacy model in a proprietary cat tool 151 8.1 Screen capture of the portal start page 184 8.2 Screen capture of the Pre-editing module 184 8.3 List of automatic pre-editing rules for French 188 8.4 Screen capture of the Translation module 189 8.5 Screen capture of the Post-editing module 190 8.6 Screen capture of the revision interface in the Post-editing module 193 8.7 Screen capture of the Evaluation module 193 8.8 Screen capture of the Statistics page 195 8.9 Usefulness of the exercise with the ACCEPT academic portal 197 8.10 User-friendliness of the ACCEPT academic portal 198 10.1  s cate mt error taxonomy 226 10.2 An example source sentence and its machine translation, which contains a number of ‘fluency’ errors 227 10.3 An example source sentence and the corresponding mt output with ‘accuracy’ errors 228 10.4 ‘Extra word’ errors, which can be detected in the target text alone, do not necessarily amount to ‘addition’ errors 229 10.5 ‘Capitalization’ and ‘punctuation’ errors are not always visible in the target text. In this example, two such errors are annotated as accuracy errors 229 10.6 A source segment in English and its mt output in Dutch containing annotations of ‘mistranslation’ errors provided by two different annotators 231 10.7 Annotations coming from annotator 1 (dark blue and dark red) and annotator 2 (light blue and light red) 232

xii

List of Illustrations

10.8 Illustrations of (a) text span similarity threshold determination and (b) the annotation alignment procedure, which uses the text span similarity threshold to align annotations with similar spans 232 10.9 Example error annotations coming from two annotators indicated as red (and pink) and grey, respectively 234 10.10 Number of annotations from each annotator, in three annotation groups (isolated, overlaps – aligned and overlaps – not aligned), when different alignment steps are applied 234 10.11 Proportions of ‘accuracy’ errors per mt system 239 10.12 Proportions of ‘mistranslation’ errors per mt system 240 10.13 Proportions of ‘fluency’ errors per mt system 240 10.14 Proportions of ‘grammar’ errors per mt system 241

Tables 1.1 Overview of primary categories of translator research styles 21 1.2 Classification of participants according to the number of research units 22 1.3 Characterising behaviours for typology of translator research styles 23 1.4 Overview of secondary categories of translator research styles 24 1.5 Weight system for assigning secondary categories to translators’ profiles 25 1.6 Assigning the presence of deep searches to participants’ profiles 25 1.7 Score calculation for the volume and time axis of the translator research style (ttrs) grid 27 2.1 Participants’ education and training in translation 44 2.2 Participants’ education and training in it 44 2.3 Popularity of different types of software among the survey participants 47 2.4 Education and training in translation and use of electronic tools 49 2.5 Computer competence and use of electronic tools 49 3.1 Comparing standalone tms: Intragloss, InterpretBank, Intraplex, sdl MultiTerm and AnyLexic (Part 1/2) 68 3.2 Comparing standalone tms: Lingo, UniLex TermX and Terminus (Part 1/2) 69 3.3 Comparing web-based tms: Interpreters’ Help, WebTerm, Acrolinx, Termflow and flashterm 74 3.4 Comparing mobile tms: Glossary Assistant and The Interpreter’s Wizard 77 4.1 Find the odd one out! Answers provided from iate terminology database 94 4.2 Bracket representations of grammatical meanings 97 4.3 Bracket representation of a complex sentence 99 4.4 Sequence of events to paraphrase 100

List of Illustrations

xiii

4.5 Example of a correspondent selection question and response options 103 5.1 Interface elements of different software 115 5.2 Opinion of participants on the length of the displayed contexts 126 5.3 Participants’ points of view about krcs’ complementarity with other traditional resources 128 5.4 Extract from the questionnaire: interface ergonomics 131 5.5 Extract from the questionnaire: interface ergonomics 131 5.6 Extract from the questionnaire: interface ergonomics 131 6.1 Translator feedback on iOmegaT/OmegaT 145 8.1 Examples of pre-editing in French and impact on machine translation into English 186 8.2 Examples of pre-editing in English and impact on machine translation into French 187 8.3 Examples of post-editing of English mt output 191 8.4 Examples of post-editing of French mt output 192 8.5 Use case for teaching – Instructions handed out to students 195 10.1 Comparison of the characteristics of different error taxonomies. We name the existing taxonomies based on the names of the authors 225 10.2 Number of sentences, number of words and average sentence length (in number of words), listed for the different data sets used for error annotation 230 10.3  i aa results based on Cohen’s kappa coefficient 235 10.4 Percentage of annotations for the aligned annotation pairs from the two annotators on taxonomy level 1. Error categories in the first row represent the annotations from annotator 1 and the categories in the first column from annotator 2. Observed agreement in this annotation set is 89% 235 10.5 Percentage of annotations for the aligned error annotations from the two annotators on taxonomy level 2, given the agreement on taxonomy level 1. Error categories in the first row represent the annotations from annotator 1 and the categories in the first column those from annotator 2. Observed agreement in this annotation set is 93.3% 236 10.6 Number of errors per category, per data set 238

Introduction Gloria Corpas Pastor and Isabel Durán-Muñoz The Digital Age has shaped the world of translators and interpreters to such an extent that ‘tech-savviness’ is not just a desirable feature, but a pressing need. However, there is a gap in the literature with regard to an updated account of technologies currently available or under development, as well as to trends that are rapidly shaping the future. So far, most research has focused on computer-assisted translation (cat) and machine translation (mt) tools, Web-based resources and applications, such as glossaries, dictionaries, corpora, concordancers, terminology management systems, knowledge based tools, cross-linguistic information retrieval (clir) systems, etc., and their degree of adoption by translators (cf. Bowker and Corpas, 2015). Much less attention has been devoted specifically to interpreting tools and resources, which include (but are not limited to) Over-the-Phone, Remote-Video and Web-based interpreting. The papers selected for this volume provide updated information on the field, (i) by presenting cutting-edge tools and resources for translators and interpreters, (ii) by promoting fresh approaches to teaching using translation and interpreting technology, and (iii) by dealing with the needs and expectations of professional translators and interpreters as well as trainees. Chapters have been grouped thematically into three different parts, each of them dealing with remarkable and promising technology and resources in the field of translation and/or interpretation. The first part includes contributions that are related to two necessary skills of any professional translator or interpreter: informational and technological competence (emt Expert Group, 2009; pacte, 2009; Hurtado Albir, 2017). The first competence refers to the ability that professional translators and/or interpreters need to develop when looking for any kind of information that is required for their task, and the second one refers to the ability of using technology in their tasks (cat tools, mt, online resources, etc.). A high level of skill or knowledge in those areas makes the difference between professional and non-professional translators and interpreters. The contributions in this part deal with this matter from different perspectives, but they all intend to underline the importance of technology and online/Web-based resources within the context of translation and interpretation. The first chapter by Joanna Gough (‘Investigating the Use of Resources in the Translation Process’) deals with online translation resources and the © koninklijke brill nv, leiden, ���8 | doi 10.1163/9789004351790_002

2

Pastor and Durán-Muñoz

­challenges of carrying out research into the use of these resources. While cat tools and machine translation have received much attention from the communities of researchers and developers, online resources and their uses have not been adequately addressed despite the fact that they are multiplying and diversifying at an exponential rate. In her study, based on a mixed methodology comprising both questionnaire-based surveys and observational research, she aims to fill this gap and shed some light on how professional translators interact with online resources during the translation process. Anna Zareskaya, Gloria Corpas Pastor and Míriam Seghiri offer an in-depth exploration on the topic of electronic tools and resources among professional translators. In their paper, ‘User Perspective on Translation Tools: Findings of a User Survey’, the authors present the results of a user survey on translation technology focusing on different factors that influence translators’ adoption of tools, such as their education and computer competence. They also discuss translators’ preferences regarding features and characteristics of cat tools. The findings of the survey show that translators do not only expect their cat tools to have a full set of features, but also to be easy to use and intuitive. Usability of translation tools is closely related to the users’ productivity, which has to be taken into account when investigating translators’ needs regarding electronic tools. Terminology tools and resources are widely used by translators, as evidenced by various surveys on translators’ needs and habits. In the next paper (‘Assessing Terminology Management Systems for Interpreters’), Costa, Corpas Pastor and Durán-Muñoz propose to spread the use of Terminology Management Systems (tms) – generally limited to the translation field – to interpretation also. As put forward by the authors, the efficient use and management of terminology will enhance the quality of the interpretations, and tms are a key tool for that purpose. Throughout the chapter, they carry out a thorough analysis of current tms of different natures and analyse them according to a set of relevant features related to interpreters’ needs and requirements in order to establish a ranked list of the most convenient tms for interpreters. The last contribution in this part presents a novel approach to designing meaning-based translation learning activities for a professional translation training course by means of human translation technology and Natural Language Processing (nlp) applications. In ‘Human Translation Technologies and Natural Language Processing Applications in Meaning-based Translation Learning Activities’, Éric Poirier discusses innovative ways to enhance translation learning in online as well as onsite environments by helping to operationalise and organise the learning process of translation tasks as well as to open new horizons on the multiple uses of nlp for carrying out translation tasks and providing consulting services.

Introduction

3

Users’ needs and requirements in theory, practice and training are also at the foreground of the second part, that is devoted to computer-assisted translation/interpreting (cat/cai) tools. It includes three insightful contributions, which approach these technologies from different (yet complementary) angles, but all of them with the aim of improving and analysing translators’ and interpreters’ working environment and specific uses. Picton, Planas and Josselin-Leray (‘Monitoring the Use of Newly Integrated Resources into cat Tools: a Prototype’) bring some debate over the development of new cat tools that integrate further functions related to fine-grained analyses of corpora. They study the relevance of context and co-text in the translation process and propose the integration of Knowledge Rich Contexts (krcs) into the more frequent cat tools employed by translators. Subsequently, they present a novel application called Argos, which interacts with cat tools providing krc, as well as a testing protocol to assess both the relevance of integrating krcs in a cat tool and the integration chosen in the Argos interface. In the same fashion, Moran, Lewis and Saam (‘Can User Activity Data in cat Tools help us Measure and Improve Translator Productivity?’) present an application developed within the iOmegaT project. Its main aim is to analyse the User Activity Data (uad) in a cat tool (OmegaT, a well-known free open-source cat tool). According to the authors, a better understanding of how translators work and interact with various computational linguistic technologies can be reached by analysing translators’ productivity and interaction with a cat tool and that can lead to an optimisation of translation speed and quality. The following chapter deals with the consequences and changes brought about by technology for interpreters in the Digital Market. In ‘Computer-­ assisted Interpreting: Challenges and Future Perspectives’, Claudio Fantinuoli highlights the impact that information technology has on interpretation and discusses the way this technology is changing the interpreting profession and the challenges that lie ahead. Even though Computer-assisted interpreting (cai) tools are still under development and need more research, especially if we compare them to cat tools in the translation field, this paper emphasises the importance of new technological advances in the current context and the challenges that interpreting studies need to address to bridge the emerging gap between the developing profession and the kind of research which is being carried out in the field. Finally, Part 3 rounds up this volume devoted to translation and interpreting technology by including three contributions related to machine translation (mt), an ever growing resource in the translating context. mt is becoming an increasingly useful resource as well as an emerging market niche for many translators and, possibly in the future, also for interpreters. These papers tackle

4

Pastor and Durán-Muñoz

mt from two viewpoints: on the one hand, the pre-editing and post-editing phases of mt and, on the other, the analysis and categorisation of errors in different languages and textual genres. All in all, the three papers highlight the importance of mt in the translation field and the industry efforts to improve the resulting output. Bouillon, Gerlach, Gulati, Porro and Seretan (‘The ACCEPT Academic Portal: A Pre-Editing and Post-Editing Teaching Platform’) introduce a freely available platform developed in the framework of a European project and devoted to improving the automatic translation of user-generated content. The accept technology integrates different modules combining pre-editing, mt, and postediting, as well as evaluation in a single workflow, which, to the best of their knowledge, have never been combined into a single platform. In this chapter, the authors provide a detailed description of the platform and its functionalities, describe possible uses for teaching and, finally, provide the results of the usability test and of the qualitative evaluation with students and teachers obtained by means of an empirical study. In ‘The Challenge of Machine Translation Post-editing: An Academic Perspective’, Rico Sánchez-Gijón and Torres-Hostench discuss the emergence of mt within the translation industry and the need to answer unresolved questions, such as how translation quality is defined or how post-editors are trained. In this context, the authors aim at shedding some light on mt in professional and academic domains by promoting a fresh approach to teaching using translation technology, and to dealing with the needs and expectations of translators regarding mt and post-editing. An outstanding example of the advantages offered by mt and the improvements still needed is the next contribution. In ‘SCATE Taxonomy and Corpus of Machine Translation Errors’, Arda, Hoste and Macken provide a detailed description of the error taxonomy, the annotation task and inter-annotator agreement results and perform a fine-grained analysis of machine translation errors for English-Dutch covering three domains and two different mt systems: a statistical mt system (Google Translate) and a rule-based mt system (Systran). With a specific focus on technology, the contributions included in this volume present novel applications, fresh approaches and cutting-edge research, as well as a wide range of electronic tools and e-resources that represent an asset for translators and interpreters. All papers included have been written by renowned authors and peer-reviewed by experts in the field, which ensures their overall quality and their important contribution to the field. We sincerely hope that this edited book becomes a valuable source of information for scholars, trainees and professionals. Our ultimate goal is to raise awareness of the importance of becoming a ‘tech-savvy’ translator and/or interpreter, to give a

Introduction

5

glimpse on the wide array of electronic tools and resources currently available and to stress their relevance for translation and interpreting practice, research and training … now and in the future! References Bowker, L. & Corpas Pastor, G. (2015). Translation Technology. In Mitkov R. (ed.). Handbook of Computational Linguistics. 2nd edition. Oxford: Oxford University Press. EMT Expert Group (2009). Competences for Professional Translators, Expert in Multilingual and Multimedia Communication. EMT. Brussels. Retrieved from http://ec.europa .eu/dgs/translation/programmes/emt/key_documents/emt_competences _translators_en.pdf (consulted on 3/03/2017). Hurtado Albir, A, (ed.). (2017). Researching Translation Competence by PACTE Group. (Benjamins Translation Library, 127) Amsterdam/Philadelphia: John Benjamins. PACTE (2009). ‘Results of the Validation of the PACTE Translation Competence Model: Acceptability and Decision Making’. Across Languages and Cultures, 10, pp. 207–230.

part 1 Electronic Tools for Translators

∵

chapter 1

Investigating the Use of Resources in the Translation Process Joanna Gough Abstract The use of resources plays an important role in the translation process. Despite the increased adoption of translation technologies, professional translators can spend on average as much as one third of their actual translation time on various external consultations of resources. From a methodological point of view, investigating the use of resources in the translation process has proven to be a difficult task, even in the relatively uniform working environments of the pre-Internet era. Now that most external resources have moved from paper to online and have started to merge into the complex, heavily technologised translation environments, these investigations have become even more demanding. The present chapter explores the various challenges of conducting research into the use of external resources over the last few decades and presents a multi-method approach suited to the complex translation environment of today. It draws on findings from a recent study into the use of resources by professional translators showing how the adoption of this approach enabled a multi-dimensional exploration of translators’ research behaviours in their natural working environment and facilitated the subsequent classification of these behaviours into a Typology of Translator Research Styles (ttrs).

Keywords process-oriented research – translator research styles – online resources – external resources – translation technology – information behaviour – information seeking

1 Introduction The rapidly growing number and diversity of resources available online in the last decade or so can be seen as a reflection of the importance of external resources in the translation process. However, this importance has not always © koninklijke brill nv, leiden, ���8 | doi 10.1163/9789004351790_003

10

Gough

been recognised in Translation Process Research (tpr), especially in its early, pre-Internet era. Furthermore, the recent growth in the number of resources has not been matched by a corresponding increase in the amount of research into their use. Rather, the focus has been almost entirely on productivity gains from the implementation of MT-related developments. This lack of interest can be explained by the fact that in the currently predominant per-word price model freelance translators are not remunerated for the time they spend on research activities (including the use of external resources), obviating the monetary incentive at higher levels of the translation supply chain to streamline those activities. Translators’ research activities can therefore be seen as a type of ‘shadow work’ (Illich, 1981; Marcu, 2015), overlooked by clients and not always acknowledged as part of the translation process. Yet empirical research on which this chapter draws (Gough, 2016)1 shows that, on average, freelance translators can spend as much as 30% of their translating time on interactions with external resources, a relatively high figure in the current realities of the market. This is further exacerbated by the fact that despite the drive towards integration of the various technology components into one ecosystem such as tm, mt, voice recognition, predictive typing, built in resources etc., translators face the difficult task of reconciling a growing array of technology offerings with their own practical translation needs, including the various types of external consultations. Moreover, with translators’ increased use of content-­leveraging technologies resulting in higher throughput expectations, these research needs can potentially be more demanding, and therefore time-consuming. This situation calls for the examination of the interaction between translators and the various elements that affect their performance, including their use of resources. With the increased pressure on translators to achieve productivity gains through the use of newly available technologies, it is necessary to begin to understand what happens in the 30% of the time they are interacting with external resources. This is not only to provide better support for translators through streamlining research activities and reducing unpaid research time, but also for the benefit of buyers of translation services in view of the future potential for competing pricing models such as per hour or per project. The present Chapter aims to present a framework developed to ­capture complex interactions between translators and online resources in a technology-mediated translation environment with a view to exploring these interactions and their possible impact on the translator. Selected previous s­ tudies in this area will be examined in the light of these questions, with a p ­ articular 1 Based on two samples and two types of data (a survey of 540 respondents – self reported data and an observational study of 16 participants – self reported and observed data).

Investigating the Use of Resources in the Translation Process

11

focus on the changes in methodological approaches concurrent with the changes brought about by the professionalisation of translation, the spread of the Internet and the technological transformation of the translators’ working environments. 2

The Changing Nature of Research into the Use of Resources in the Translation Process

Early tpr studies examining the use of external resources before the Internet focussed predominantly on paper sources. Translation Studies as a discipline dedicated to the scholarly study of translation was only two decades old and the ‘Applied’ branch of Translation Studies proposed by Holmes in 1972, which incorporated ‘Translation Aids’, was underdeveloped. Studies related to the use of these aids were few and far between and mostly carried out as part of process-oriented studies which were classified as a separate, ‘pure’ branch of Translation Studies in Holmes’ Map. As such, they focussed on the so-called ‘black box’ and the examination of the norms governing translators’ mental processes during translation activities. Little attention was paid to describing how this process is affected by external factors, including the use of resources. This is reflected in the early tpr literature where studies were conducted predominantly without access to translation aids, leading to the examination of the translation process per se, i.e. in isolation from other integral activities such as interactions with people or the physical environment. Whenever the (often partial) object of study included the use of resources, it was common practice for the researcher conducting the study to supply their own paper dictionaries, encyclopaedias or grammar books, such as in the studies into the use of paper resources carried out by Jääskeläinen (1989) and House (2000). This meant that the participants had no real choice. In other studies participants were asked to bring their own paper resources, but this strategy was not always successful either. Given that the participants were often students or novices, their collections of resources might not have been established and their knowledge about what resources beyond d­ ictionaries could potentially be useful was most likely underdeveloped. For example, in Krings’ much cited study (1986) students brought only a dictionary despite ­being asked to bring any reference books that could help them with the task. Furthermore, with some notable exceptions (e.g. Nord, 1997/2009), early process-oriented studies were mostly carried out under laboratory conditions using language learners, translation students and novice translators. ­Experienced translators featured in those studies only occasionally. Although

12

Gough

current debates also frequently revolve around definitions of competence, professionalism and expertise, the alleged focus of the pre-Internet studies on professional translators seems stretched by today’s standards. Luukkainen (1996) and Tirkkonen-Condit (1989), for example, considered final year students as experienced or ‘professional’ translators. Similarly, Varantola (1998) also used students under laboratory conditions even though her study focussed on the needs of professional translators. In view of this it could be argued that the results of early process-oriented studies examining ‘professional’ translators should be treated with caution, especially when compared with those carried out in authentic translation situations in today’s, completely different translation environment. Although pedagogically motivated lab or classroom research is still very much practiced, interest in studying expert behaviour in natural settings has been growing. These early studies constituted important stepping stones in the evolution of tpr. Arguably, however, their design meant that they often did not reflect real practices in the translation profession, even at that time. This discrepancy between lab experiments and real-life scenarios is particularly evident when looking at Nord’s pioneering study (1997/2009) in which she observed the translation process of professionals in ‘authentic translation ­situations, […] t­ ranslating texts of their own choice arising in their daily routine, using aids normally available to them’ (2009: 203). The list of resources used by those translators appears more varied and includes different kinds of dictionaries, encyclopaedias, personal databases, atlases/chronicles, auxiliary texts and ‘persons’. The following excerpt from a translation forum discussion provides a first-hand account of the real-life scenario of resources use in the pre-Internet era: I had a ton of paper dictionaries (over 300) and I would sit at a table with all of the appropriate dictionaries open (business, legal, etc.). If the word or phrase wasn’t in any of your paper dictionaries, you were in big trouble. You had to spend hours and hours looking things up. […] I also had a book full of paper glossaries, clippings of word lists from magazines, the ata Chronicle, terms on index cards. […] I would also have to make frequent trips to the university library and make telephone calls to experts in order to ask questions. Legal Transform, 2015

Another aspect of process-oriented studies that needs to be considered is the impact of the use of resources on the translator and the translation process. It could be said that from the early studies a rather negative perception of this

Investigating the Use of Resources in the Translation Process

13

impact emerges, where the use of resources is seen as hindering the translation process. For example, House argues that ‘dictionary searches disrupt the flow of thought’ (2000: 156) and Luukkainen concludes from a similar study that the use of reference material negatively impacts on the time spent on translation by slowing it down and limiting translators’ creativity (1996: 71). It could be argued that the very different assortment of resources available to translators today and the different mode of accessing them (online) create a more symbiotic relationship, resulting in the more positive attitudes towards them that now thrive amongst translators (Gough, 2016). These resources are also arguably more responsive to the needs of translators and therefore their role in the translation process might be perceived more favourably than that of their paper counterparts, regardless of any disruptive effects they may have on this process. Thus, it can be said that thanks to the speed of access and the diversity of available resources on the Internet, online resources supporting the translation process have now become the norm in everyday translation practice. This, in turn, has markedly shifted the research perspective by considering the presence of technology in the translation process as a given rather than an option. Accordingly, more recent process-oriented research has begun to investigate a broader range of resources within the translator’s ‘tool box’, extending the scope far beyond dictionaries, encyclopaedias and grammars to reflect current practices where translators also use resources such as term banks, glossaries, corpora, thesauri, concordancers, knowledge-based resources, discussion fora, cross-language retrieval tools (CLIR), search engines, parallel texts or machine translation used as a reference. However, the exponentially growing and diversifying range of resources, coupled with the fast adoption of assistive, content-leveraging translation technologies by translators has brought about new challenges for researchers. The increased technologisation of the working environment and the complexity of translators’ online activities mean that the task of researching the use of resources has never been so multifarious and so riddled with methodological challenges. Firstly, the boundaries between tools and resources have started to blur and the terminology around translation technology has become rather confusing (Gough, 2016) making it difficult for researchers to draw the line between tools and resources in their investigations. Secondly, since most translators now use translation technology in one form or another (tm/mt/tm+tm), investigating the use of resources in those environments has become more problematic. Unlike the rather uniform translation environment of the pre-Internet era, today’s environment is increasingly complex and diverse. Translators use different combinations of tools and resources and they translate in different modes (e.g.

14

Gough

from scratch, post-editing or using adaptive mt). This makes it more challenging to carry out ecologically valid research across these various technologies and processes, but this difficulty is precisely the reason why such ‘transverse’ research should be pursued. The research tools currently available do not facilitate the conduct of research across different platforms and the observation of resources use in different translation modes. This leads to a situation where a more comprehensive evaluation of how translators use online resources in these varied environments becomes fraught with difficulties. As a result, some studies are conducted using one tool, e.g. casmacat2 (Daems et al., 2015; Zapata, 2015) in connection with Inputlog3 (Daems et al., 2015), or single resources e.g. Bi-conc built into casmacat (Zapata, 2015)4 or the eu concordancing tool Euramis5 (Valli, 2012). Such approaches can reduce the manual involvement in data processing and help increase the sample size, but at the same time the representativeness of authentic practices is diminished as translators seldom confine their research activities to interactions with just one resource. Furthermore, as Daems et al. (2015: 121) observe, ‘it is currently impossible to automatically map external resources to the correct segment’ which makes it difficult to relate the research activities to the relevant item in the source text (st). To circumvent this shortcoming, screen recording techniques are used (Lauffer, 2002; Asadi & Séguinot, 2005; Enríquez Raído, 2011; Mutta et al., 2014; Zapata, 2015) to manually map the researched items to the respective resources used to investigate them. This requires a substantial amount of manual work and is not particularly conducive to examining large amounts of data. Therefore, studies using this method tend to have smaller samples. This also goes for other methods such as contextual enquiry (Désilets et al., 2009), focus groups (Domas et al., 2008) or interviews (Lagarde, 2009) where the focus tends to be mostly qualitative due to smaller samples, which results in these studies suffering from a lack of external validity. Surveys, although suitable for gathering larger amounts of data and thus affording a greater scope for generalisations, yield only subjective, self-reported data (e.g. Durán-Muñoz, 2010, 2012; Gough, 2011) that would ideally need triangulating with more objective, observed data. Due to these shortcomings, multi-method studies which combine various logging techniques, screen recordings, eye tracking, interviews and 2 http://www.casmacat.eu/. 3 http://www.inputlog.net/. 4 Although Bi-conc was the main focus of the study, the use of other resources was also captured. 5 http://www.mt-archive.info/EAMT-1997-Theologitis.pdf.

Investigating the Use of Resources in the Translation Process

15

­questionnaires have become popular (Enríquez Raído, 2011; Massey & Ehrensberger-Dow, 2011a, 2011b, 2011c; Volanen, 2015). They permit a more multi-­ dimensional analysis of a given phenomenon, whilst still making it possible to research the translation process in a scientific way and allowing researchers to capture the various aspects of the translation process as it unfolds in real time. Compromises still need to be made along the authentic/laboratory, student/ professional, small/large sample continua because analysing expert behaviour in varied authentic scenarios and including large samples is currently beyond reach. But studies such as those mentioned above do lay the foundations for future scaled-up research designs. The study on which this chapter draws is an example of such a methodological compromise in which the acquisition of detailed and multi-dimensional data and ecological validity were prioritised over large sample and external validity. Research into the use of resources is also driven by the broadening of the scope and goals of studies in this area. Whereas early research was mostly motivated by the pedagogical insights gained from introspection into the translator’s mind, the scope of recent studies has broadened to encompass wider contexts in which translators operate (Muñoz Martin, 2015), and has more pragmatic goals such as productivity gains or user-experience enhancement. More emphasis is being put on studying professionals in authentic situations guided by the view that ‘investigating translation processes becomes truly relevant to translation competence and practice when the processes reflect actual practices of working translators, not artefacts of experimental settings and tasks’ (Ehrensberger-Dow & Massey, 2015: 11). It could be argued that with the changing demands of the globalized market influencing the perceived translation quality continuum and the technologisation of the translation process, a more functional approach to tpr has evolved from the earlier consideration purely of its form. Pedagogical aims are still very much pursued in tpr as part of developing translator competence models. The use of resources is especially examined in order to develop/update technology skills as well as information literacy and research skills. Thus, in an attempt to capture the use of resources and research practices in the post-Internet era, studies have acquired a new perspective, that of information behaviour. Various studies have emerged recently in this area, including Valli (2012), who examined search strategies, types of interactions and recurrent search patterns in the use of concordancers by eu translators, Enríquez Raído (2011) and Volanen (2015), who investigated the Web searching behaviour of student and professional translators respectively, Massey & Ehrensberger-Dow (2011a, 2011b), who focussed on the translation-centred information behaviour of professional translators, and Pinto, Sales and their

16

Gough

co-workers, who concentrated on developing information literacy instruction for translation students, teachers, academics and professionals (Pinto & Sales, 2008; Pinto et al., 2010 & 2014; Sales & Pinto, 2011). These studies highlight the pressing need to investigate the use of external resources in the new context of online search behaviour, translator-computer interaction (O’Brien, 2012) or even translator-information interaction (Zapata, 2015). Furthermore, in an attempt to keep up with the demands of the market for ever faster translations in ever growing volumes, researchers have made progress in capturing aspects of resource use that could feed into the development of new tools and resources or contribute to improving existing ones. Various studies have examined the use of resources in MT-unassisted human translation processes (e.g. Domas White et al., 2008; Désilets et al., 2009; Massey & Ehrensberger-Dow, 2010; Durán-Muñoz, 2012) and in post-editing (e.g. Daems et al., 2015; Zapata, 2015). Thus it could be said that much of tpr has moved on from the descriptive phase to a more pragmatic one, where findings are geared towards problemsolving (developing new technologies), predicting how new norms evolve ­under new conditions such as changing market demands or new social or technological realities, and looking at how the findings of tpr can contribute to the ‘predictive modelling of human translation processes’ (Carl et al. 2016). In contrast to the earlier tpr studies, research agendas have now entered the realm of ergonomics, embodied cognition and techno-social cognition (Risku, 2002; Heyes, 2012; Ehrensberger-Dow & Massey, 2014) thus broadening the scope of investigations to include wider aspects of human cognition activities including ‘various actors and factors’ such as physical and organizational ones (Massey & Ehrensberger-Dow, 2015). This considerably expands the spectrum of investigated phenomena, adding layers of complexity to the process of data collection and analysis. The methodology presented in the following section reflects the new trends in tpr research in relation to the use of resources. 3

Investigating the Use of Resources by Professional Translators in a Quasi-Experimental Setup

As mentioned before, the study of how translators use online resources is important from a productivity point of view, given the amount of time which can be spent on research activities. However, it is important to go beyond partial analyses of selected technologies and processes towards more comprehensive, across-processes and across-technologies studies of expert behaviour. The methodology outlined here attempts to achieve this by collating various aspects of such behaviour into a comprehensive framework from which a ­typology of

Investigating the Use of Resources in the Translation Process

17

translator research styles (ttrs) emerges. This typology can be considered a starting point for further investigations into the patterns of translator research behaviour, especially in the fast-changing landscape of the translator’s working environment. Translation Process as Two Types of Interactions – Developing an Analytical Framework In order to develop an analytical framework that would support a multi-­ dimensional analysis of translator research activities, the study on which this discussion draws (Gough, 2016) made certain assumptions about the nature of the translation process and, consequently, the translator’s research process. The translation process for the purpose of this study was conceived from the perspective of what is observable in screen recordings (see Section 3.2. below). Thus, translation process is operationalised as two6 types of interactions: interactions with texts which take place during the translation activities and interactions with external resources7 which take place during the translation-oriented research activities. This allowed the translation process to be conceptualized as being composed of translation episodes and research episodes, and to concentrate only on the latter. Consequently, the research episode became the central unit of analysis. It was tied to other concepts such as research need, research unit, research session, and research step (see Figure 1.1) which are encompassed in a research action. These concepts will be briefly defined below, starting from research step and working up towards research need. A research step occurs whenever a particular resource is accessed in response to a research need, i.e. the need for information, further defined below. ‘Accessing’ is defined by the act of ‘clicking’, whether directly from a taskbar, through a Web browser or from within a particular resource by means of hyperlinks. Each instance of ‘clicking’ into another resource or its sub-section constitutes a ‘step’. For example, in research episode 1 (see Figure 1.1 above), accessing Wikipedia is considered as step 2 and accessing a language version of Wikipedia is considered as step 3. 3.1

6 The interactions with ‘internal’ resources, which form an integral part of the translation process, were not examined in the study the present chapter draws on. 7 A distinction was made between technologies used for the whole translation task e.g. tm or mt used in post-editing mode, which were considered as tools, and technologies used adhoc to solve terminology-related problems e.g. online dictionaries or mt used in search and discovery mode (Van Der Meer & Ruopp, 2014), which were considered to be resources. The study focussed on the latter.

18

Gough Research Need (The concept of Bitcoin) Research Unit (ru) (Bitcoin)

Research Sessi0n

Research Episode 1

Research Step 1 Google

Research Step 2 Wikipedia 1

Research Step 3 Wikipedia 2

Research Episode 2

Research Episode 3

Research Step 1 Linguee Research Step 1 Google

Research Step 2 Webpage

Figure 1.1 The structure of a research action.

Research episode refers to a series of steps taken to address a particular research need and is tied to a particular lexical item in the source text, i.e. a research unit. A research episode can have multiple steps (consultations), including accessing new resources, modifications to research queries and submitting new queries within the same resource. Research session refers to a series of research episodes referring to the same research unit, separated by more than one translation episode. For example, whenever a research unit is explored at different times during the translation process, such as during the orientation phase and then again during the drafting phase (Dragsted & Carl, 2013), these two research episodes form a research session. Enríquez Raído (2014: 127) refers to a research session as a ‘temporal series of online actions aimed at satisfying a specific information need’, but it is not clear whether it encompasses searches that were triggered by the same research need but happening at different times during the task. Therefore, the concept of research session was introduced as a higher-level unit of analysis to reflect the fact that research pertaining to the same item in the text can happen at different times during the process.

Investigating the Use of Resources in the Translation Process

19

Research unit refers to a lexical item (a single word, a part of a word, or a chain of words acting as a unit of meaning) in the source text which is investigated in response to a research need. It is important to acknowledge that whilst research needs refer to the kind of information required at a particular point in the source text (e.g. target language equivalent, spelling, capitalisation etc.), these needs are triggered by particular words or phrases in the source text, in this work called research units. Research need in this study corresponds to Enríquez Raído’s search need, i.e. ‘the recognition of an information need as perceived within the context of translation problem solving’ (2011: 152). Research need refers to the nature of the information required, such as understanding a concept, finding an equivalent, checking spelling etc., and implies that the identification of such a need is followed by research in external resources. When the flow of translation activity is interrupted and the translator accesses external resources8 to solve a translation problem encountered in the source text, a switch from a text to a resource occurs, indicating a research need and launching a research episode. 3.2 Research Design The research design of this quasi-naturalistic, Internet-mediated study (Mellinger, 2015) reflects the methodological constraints mentioned earlier; however, it attempts to reflect as much as possible the actual research behaviour of freelance professional translators in authentic settings. Its complex design aims to capture the nature of the research activities from as many angles as possible, including the combination of various methods of data collection and the integration of quantitative and qualitative analyses. Therefore, screen recordings were used as the main vehicle for observing the actual use of resources, but were supplemented by other methods such as audio-commentaries and questionnaires to afford a greater level of detail and complement the observational records with the self-reported data. A sample of 16 translators with five or more years of professional e­ xperience9 took part in the main part of the study described here, carried out in 2013/2014. The study was language independent and no quality assessment was carried out. Although ecological validity was sought, a degree of control was exercised by providing the participants with the text. This allowed their research ­activities to be assessed in relation to one another and the differences between various research styles to be established. The research design incorporated 8 It is assumed that this happens when no satisfactory solution can be found in the translator’s internal (i.e. mental) resources. 9 Some of the participants worked part-time at the time of contributing to the present study.

20

Gough

profile questionnaires, a translation task with audio-commentary and posttask questionnaires. Profile questionnaires were used to gather background information about the participants and specific information about their selfdeclared use of resources, time spent on research activities, attitude towards technology etc. For the translation task, a naturally-occurring text in English providing ample information-seeking opportunities was sought. A 412-word journalistic text about Bitcoin and the relatively new (at the time of the data collection) concept of digital currency was chosen as it contained interesting vocabulary in a new context, double-duty words, several named entities and phrases requiring adaptation. As one of the crucial tenets of this study was that translators should work in their natural environment, the participating translators were instructed to use any tools and resources they would normally use in a translation assignment. To capture their work for this study, they were instructed to use the online screen recording software ScreencastOMatic10 and to upload the files to a server on completion. Although this increased the technical difficulty for the participants, it was crucial for obtaining naturalistic data from translators located in various parts of the world. The participants were also asked to comment on specific aspects of their research process during the task. The final phase involved a follow-up questionnaire (via e-mail) in order to achieve a greater depth of understanding of each individual participant’s case and to evaluate the overall experience of this study. Once the data was collected, it was processed by a series of manual and semi-automatic calculations, annotated and coded. Three types of activities (translation, research and commenting) were identified and each instance of these activities was logged, allowing the calculation of the distribution of translation, research and commentary for each participant. During the annotation process, all instances of resource use were noted and flowcharts including the research sessions, research episodes and research steps (as defined above) taken by each of the participants were created. Furthermore, all 16 screen recordings, interviews and questionnaires were manually coded with qualitative observations of the research process. Data was then analysed by means of descriptive statistics, code clustering and triangulation (where possible) of the self-declared and the observed data. 3.3 Developing Categories of Analysis The analytical framework outlined in Section 3.1. enabled the development of two sets of analytical categories of translator research behaviour, primary and secondary, allowing a comprehensive analysis of the data and interpretation of the findings. The primary categories (see Table 1.1) originate from numerical 10

https://screencast-o-matic.com/home.

Investigating the Use of Resources in the Translation Process

21

data and were calculated for all the participants based on their screen recordings and represent ranges of online behaviour. Some of the primary categories are composed of two or more sub-categories. The six primary categories are divided into two groups representing volume-related and time-related features. The light pink end of the spectrum refers to the activities on the lower side of the quantitative scale indicating low quantities and short timings while at the other end of the spectrum, the dark pink colour indicates high quantities and long timings. The only exception is the research pace category, where low (short) duration of research steps and episodes equals high research pace. As an example, Table 1.2 below shows how individual translators were classified into having a High, Medium or Low number of research units (rus). Each sub-category is divided into High, Medium and Low values, capturing the lower end, the middle range and the high end of the spectrum. These values Table 1.1

Primary category

Overview of primary categories of translator research styles.

Sub-category 1

Sub-category 2

Sub-category 3

VOLUME-RELATED FEATURES 1 Research ru Volume Unit Volume Low Medium High 2 Resource Volume

Number of specific resources

Frequency of use of specific resources

Low Medium High Low Medium High 3 Research Intensity

Number of steps per Number of steps per ru (mean) ru (weighted mean)

Number of steps per ru (weighted median)

Low Medium High Low Medium High Low Medium High 4 Research Diversity

Number of types of resources

Number of specific resources

Number of specific resources per ru

Low Medium High Low Medium High Low Medium High TIME-RELATED FEATURES 5 Research Time

Research time (hrs:min:sec)

Research time (%)

Low Medium High Low Medium High 6 Research Pace

Average duration of research step

Average duration of research episode

High Medium Low High Medium Low

22 Table 1.2

Gough Classification of participants according to the number of research units.

ru Volume Translator

Number of rus

Classification

T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11 T12 T13 T14 T15 T16

35 0 43 11 8 13 11 18 23 25 31 10 20 18 36 8

High Low High Low Low Low Low Medium Medium Medium High Low Medium Medium High Low

Range: 43 (0–43) 0–14 Low 15–29 Medium 30–43 High

were given corresponding descriptive labels: Generous, Moderate and Frugal for the volume-related categories and Relaxed, Moderate and Rapid for the timerelated categories, thus affording a qualitative description of corresponding characterising behaviours for each of the categories as indicated in Table 1.3. Data for the secondary categories were gathered mostly from the screen recordings, but, recognising the importance of multi-method approaches in tpr, data from profile questionnaires and post-task questionnaires were also used. This enriched individual research profiles with qualitative attributes and helped to uncover the more nuanced dimensions of the research styles that could not be captured by means of observation alone. The secondary categories were coded on an ad hoc basis and were only applied to those participants

Investigating the Use of Resources in the Translation Process Table 1.3

23

Characterising behaviours for typology of translator research styles.

The range of characterising behaviours

Volume 1 2 3 4 5 Time 6

Primary Categories

Low

Medium

High

ru Volume Resource Volume Research Intensity Research Diversity Research Time Research Pace

Frugal Frugal Frugal Frugal Rapid Rapid

Moderate Moderate Moderate Moderate Moderate Moderate

Generous Generous Generous Generous Relaxed Relaxed

who exhibited behaviours described in those categories. The following 14 secondary categories in Table 1.4 were identified from the data. Unlike many of the primary categories, the secondary categories represent only one aspect of translators’ online research behaviour.11 In order to create a more nuanced analysis of the data, weights were assigned to the observed behaviour phenomena in a graded fashion according to the strength of the presence of a particular secondary category in that translator’s research. As shown in Table 1.5 below, these weight scores created ranges which were then split into three ‘gradients of presence’ – weakly present (wp), moderately present (mp) and strongly present (sp). The weak presence received a weight of 1, the moderate a weight of 2 and the strong a weight of 3. For example, Table 1.6 below shows how the range of observed behaviour of deep searches (in this case 9%–38%) was split into the three gradients, resulting in the following classification: eight, three and two translators were classified as having weak, moderate and strong presence of deep searches respectively. The development of the above categories represents an original approach as far as research into the use of resources is concerned, both in terms of creating new categories and combining the quantitative and the qualitative data into one framework. Furthermore, the treatment of the quantitative data as various volume-related and time-related aspects of research activities is a novel approach providing a systematic and yet nuanced representation of translator research behaviour.

11

The two categories based on self-declared data (information retention and attitude towards technology) do not represent research behaviour as such, but are related to it.

24

Gough

Table 1.4

Overview of secondary categories of translator research styles.

Secondary categories Short description 1

Deep searches

2

Shallow searches

3

Meandering research path Parallel research sequence

4 5

Planning

6 7

Squirrelling Snippet viewing

8 9

Using advanced queries Repetitive behaviour

10

Strategic behaviour

11

Drive for perfection

12

Affective assessment

13

High information retention Attitude towards technology

14

Deep engagement with resources characterised by longer periods of consultation or performing a chain of online interactions, often modifying queries and changing the type of resource Shallow engagement with resources characterised by one-step consultations, quick checks in termino-lexical resources or a rapid Google search with snippet viewing Diverging from a straight path by following internal links and hyperlinks of a resource Consulting more than one resource at the same time by opening several windows in a parallel fashion Considering what research activities will be required for the task Saving information for future retrieval Consulting the search engine results without following the links Using advanced procedural knowledge of Web searching techniques Repeating an initial search action consistently using the same resources in the majority of research episodes The presence of a clear and consistent ‘tactic’ for finding the required information with regard to word and world knowledge Exerting substantial amount of effort to meet one’s research need Engaging feelings or attitudes as part of the evaluation of a solution Self-declared level of retaining information Self-declared classification with regard to adopting new technologies

25

Investigating the Use of Resources in the Translation Process Table 1.5

Weight system for assigning secondary categories to translators’ profiles.

Behaviour attribute

Absent

Weakly ­present Moderately (wp) present (mp)

Strongly ­present (sp)

Weight assigned

0

1

3

Table 1.6

2

Assigning the presence of deep searches to participants’ profiles.

T1

T2 T3

T4

T5

T6

Translator T7 T8 T9 T10 T11 No of deep searches

T12

T13

T14

T15

T16

10

n/a 4

2

1

5

3

2

2

2

5

0

2

2

6

11

% of deep searches in all research episodes 29% 0% 9% 18% 13% 38% 27% 11% 9% 24% 34% 20% 10% 11% 14% 0% Attribute grading label mp

A wp wp

wp

sp

mp

wp wp mp

sp

mp

wp

wp

wp

A

3

2

1

1

1

0

Weight attached 2

0

1

1

1

Range 29% Range span 9%-38% Attribute grading Absent (A) 0%

0

Weakly ­Present (wp) 9%-18%

1

Moderately Present (mp) 19%-29% 2 Strongly ­Present (sp) 30%-38% 3

3

2

1

1

2

26

Gough

Organising Data in a ttrs (Typology of Translator Research Styles) Grid The methodological approach discussed above provided a framework for a multi-dimensional analysis of translator’s research activities. These activities proved more diverse than one might think. They varied considerably in almost all respects, from the number of investigated research units or the number of resources used in those investigations to the diversity of those resources. Moreover, the participants displayed a variety of additional behaviours, some of which could be attributed to more than one participant. Through the creation of the ttrs grid, the present methodology enabled a consolidation of these differences and similarities into a multidimensional representation of translator research behaviour. This was done in two steps. Firstly, the labels attached to the primary categories were assigned arbitrary weight scores: Frugal and Rapid – a score of 2, Moderate – a score of 4 and Generous and Relaxed – a score of 6. This created a range of final weight scores for the volume- and the time-related axes of the ttrs grid (see Table 1.7 below), allowing translators to be positioned on the grid (see Figure 1.2). Based on the score table above, the individual translators were placed on the ttrs grid according to their final weight scores. The grid contains five areas corresponding to the five different ways of conducting research (prolific, explorative, methodical economical and understated) which emerged from the data. The translators who fitted into one of the delineated areas were associated with the corresponding research behaviour. For example, T1’s final Volume score was 22 and her final Time score was 12. She was therefore placed on the ttrs grid in the bottom right corner. Five translators (T3, T4, T11, T15 & T16) were initially plotted on the border positions and their final positions were established based on the secondary categories (see Figure 1.3). Secondly, based on the above ttrs grid where the initial position of individual translators was established, the secondary categories were clustered according to the most commonly displayed secondary categories of the translators located in each of the five segments of the grid. For example, if most translators in the Economical segment of the ttrs grid displayed shallow searches, then it was assumed that shallow searches are one of the attributes of the Economical style. The presence of secondary categories was assessed for each individual translator, resulting in either a strengthening of the existing position or the shifting of the borderline translators to a neighbouring area. Based on the primary and secondary categories, the final position of each individual translator was established. Figure 1.3 below illustrates how each of the participants’ position was either strengthened or altered according to the combination of both primary and secondary categories.

3.4

27

Investigating the Use of Resources in the Translation Process Table 1.7

Score calculation for the volume and time axis of the translator research style (ttrs) grid. VOLUME Frugal T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11

ru Volume Resource Volume Research Diversity Research Intensity Frugal weight scores

2 2 2 2 0 8

2 2 2 2 4

4

2 2 2 2 8

2 2 2 2 8

2 2

4

2 2 2 2 8

T12 T13 T14 T15 T16 2 2

0

0

0

4

2 2 2 6

2 2 2 2 8

2 2 2 2

4

Moderate T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11 ru Volume Resource Volume Research Diversity Research Intensity Moderate weight scores

4 4 4 4 0

4

4

0

0

4 4 8

4 4 4

4 4 4 4 0 12 16

T12 T13 T14 T15 T16 4 4

0

4 4 8

4 4

0

4

4

Generous T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11 ru Volume 6 Resource Volume 6 Research Diversity 6 Research Intensity Generous weight scores 18 0

6

6

6 6

0

0

0

0

6 6

0

6 6 6 6 24

T12 T13 T14 T15 T16 6 6 0

0

0

12

6 6

12 F

10 F

8 F

18 M

14 M

VOLUME AXIS RANGE Final Weight Scores Label

22 8 G F F M Gc

14 14 8 8 12 8 18 16 M M F F F F M M Frugal Moderate Generous

24 G

28

Gough

Table 1.7

Score calculation for the volume and time axis of the translator research style (cont.) TIME T1 T2 T3 T4 T5 T6 T7 T8 T9 T10 T11

Research Time Research Pace

6 2 6 2

4 2

4 6

2 4

4 6

4 6

2 4

4 4

4 4

T12 T13 T14 T15 T16

4 4

2 4

2 4

2 4

6 4

2 4

6 RD

6 RD

6 RD

10 RX

6 RD

TIME AXIS RANGE Final Weight Scores

12 4 6 10 6 10 10 6 8 8 RX RD RD RX RD RX RX RD M M

Label

rd M

Rapid Moderate

rx

Relaxed

FRUGAL

MODERATE

2

13

12

16

3

6

c

8

if i

5

5

ol

14

4

l

ica

m

no

o Ec

GENEROUS

Pr

RAPID

TIME

VOLUME

8 M

MODERATE

Methodical 10

7 9

11

8 9

7

4

15

10

or

a rst de

Ex

pl

Un

RELAXED

er

6

d te

11 1

8

10

12

14

16

18

20

22

12 24

Figure 1.2 The ttrs grid showing the initial positioning of individual translators according to their primary categories weight scores.

29

Investigating the Use of Resources in the Translation Process Initial position after primary categories

Adjusted position after secondary categories

lo Ex p

a rst ted

8

10

Figure 1.3

12

14

16

18

20

11 1 12 22 24

4

8

10

12

r

15

re

7

11

lo

10

Methodical 10 9

d ate rst de Un

re

r

15

6

Ex p

4

RELAXED MODERATE RAPID TIME

9

5

ic

11 8

4

l ica om 16 14 n o 5 8 13 Ec 12 3

lif

ic

7

6 7

2

o Pr

lif

o Pr

5

Methodical 10 9 6

VOLUME FRUGAL MODERATE GENEROUS 4

l ica 2 m o n 14 co 5 8 13 E 12 3 16

de Un

RELAXED MODERATE RAPID TIME

VOLUME FRUGAL MODERATE GENEROUS

6 14

18

20

22 24

ttrs grids showing repositioning of participants due to the influence of secondary categories.

The methodological approach outlined here provides the basis for a new way of conceptualizing translation-related research activities as a comparison of individual translators’ performances in relation to each other. More importantly, it highlights how the different ways of using resources can reflect one’s research style. It is important to note that differences in translator research behaviour were observed in previous studies. For example, in the data obtained by Nord in 1997 in her study of the use of resources by professional translators in the pre-­ Internet era (Nord, 1997/2009) or in a more recent study by Enríquez Raído (2011), where two Web searching styles, ‘shallow’ and ‘interactionistic’ were indentified. The ‘shallow’ style was associated with ‘horizontal, checking and comparing’ behaviour and the ‘interactionistic’ style with deeper and wider searches (Enríquez Raído, 2014: 139–140). The methodological approach presented here also recognises these characteristics, but unlike in Enríquez Raído’s study, they are considered to be attributable not just to experience, but also to the translators’ individual ways of approaching research activities. Building on the above findings, the present methodology attempts to provide a systematic approach to analysing translators’ research behaviour during the translation process. It is beyond the scope of this Chapter to discuss the details of the five research styles – or indeed any other findings of this study.12 12

For detailed findings of this study the reader can consult Gough (2016).

8 9 10 11

1 16

7

12

30

Gough

The important fact to mention here is that although five research styles were identified, the participants were not evenly distributed within the ttrs grid and that half of the sample was associated with the Economical research behaviour linked to low research volumes and low research timings, with an average research time of 24.6 % for this particular group.13 As no previous studies of this type are available, it is not possible to establish whether this clustering around the Economical style is associated with growing productivity demands, where a compromise needs to be struck between translators’ natural working patterns and market demands. However, such influence of the productivity constraints on the resources used and the time spent on consulting them cannot be precluded and further research is needed to confirm this. One of the Economical Translators (T16) reported that she feels the need to adapt to the way the industry works by dealing with the text in a faster and more efficient way, for example, by skipping the orientation phase. Interestingly, T16 was originally placed on the border with the Methodical style associated with more careful planning of research activities as well as being close to the border with the Prolific translator style due to the volume-related aspects of her translation. This would support the hypothesis that market demands might interfere with translators’ natural predispositions and working styles. This hypothesis could be linked to research into the so-called Google generation (Rowlands et al., 2008) where shallow, horizontal, ‘flicking’ search behaviour has been attributed not only to younger generations but to all Web users. Rowlands et al. argue that ‘a fundamental shift in the way people seek and read information has already occurred and that the impact of the shift has yet to be understood by information providers and educators’ (2008: 308). Perhaps the clustering of the professional translators in the Economical section of the ttrs grid supports their observations. 4 Conclusion The proposed research design to study translators’ use of resources goes beyond previous studies in this area in its recognition that translators might have different translation-oriented research styles. Identifying how translators interact with external resources and how they differ in their research practices is very much at the intersection of the cognitive and physical attributes of 13

One translator in this group, T2 was identified as an outlier and therefore excluded from this calculation.

Investigating the Use of Resources in the Translation Process

31

translators’ work environments. By examining these interactions we can understand the trends and requirements for future designs and functionalities of translation technologies. However, what we also need to understand is how these future designs will, in turn, affect future performance of translators. The idea that professional translators can ‘attune’ their performance according to market demands should not obfuscate the fact that, in ‘ideal’ conditions, translators are likely to display their natural styles in accordance with their inherent predispositions because information behaviour, as Spink & Heinström (2011) suggest, is an innate feature of human behaviour. As such, all types of research behaviours need to be valued and encouraged to thrive, unless we resign ourselves to becoming more machine-like in our working patterns, succumbing to the pressures and demands of the market. If the human way of doing things is to be preserved, our efforts need to be put into creating technologies that support human behaviour rather than coercing humans to adapt to technology without considering the cognitive and ergonomic impact it might have on them. As Heinström accurately puts it: no matter how fast information technology evolves or how sophisticated search systems we learn to master, our basic human reactions remain as they have been through centuries. Our behavior, even in seemingly rational activities such as information seeking, is influenced by our holistic being as a creature of physiological, cognitive, and affective processes. This characteristic has important implications for the development of user-centered information services. The user can learn to adapt to search systems, but more importantly search systems should be adapted to users’ natural ways of seeking information. The traditional approach in library and information science has been to support users to overcome their possible weaknesses in search ability. Equally important is to recognize particular strengths in the users’ habitual ways of information seeking and adjust systems to support these tendencies. heinström, 2006a: 1440

The need for a solution that would streamline research activities within the translation process through the integration of online resources and/or the provision of more sophisticated search facilities is real (Zetzsche, 2015). From the methodological point of view, research designs and approaches that can capture and uncover the translators’ research behaviour in these new translation environments are required as they can enable us to discover what these new

32

Gough

ergonomic interfaces should look like. The results of the present study strongly suggest that such developments must take translators’ research behaviour into account by providing solutions adaptable to their particular working styles. As Zapata concludes, ‘there is a tangible need to design and develop ergonomic and flexible interfaces that take the human factor into consideration and that are adapted to the translator’s workflow and needs’ (2015: 142). However, if we only accept that most translators now follow fast, cursory and shallow research patterns (most likely combined with interactive/adaptive mt and predictive typing) we might miss out on potential developments that would foster deeper, explorative tendencies geared towards creativity rather than productivity. The methodology presented in this Chapter could potentially contribute to better technological designs in the area of translation technology in future by taking into account the variety of translator research styles. References Asadi, P., & Séguinot, C. (2005). Shortcuts, Strategies and General Patterns in a Process Study of Nine Professionals. Meta: Journal Des Traducteurs, 50(2), 522. http://doi .org/10.7202/010998ar. Carl, M., Bangalore, S., & Schaeffer, M. (Eds.). (2016). New Directions in Empirical Translation Process. Springer. Daems, J., Carl, M., Vandepitte, S., Hartsuiker, R., & Macken, L. (2015). The Effectiveness of Consulting External Resources During Translation and Post-editing of General Text Types. In Carl M., Bangalore S., & Schaeffer M. (Eds.), New Directions in Empirical Translation Process Research: Exploring the CRITT TPR-DB (pp. 113–138). Springer. Désilets, A., Melançon, C., Patenaude, G., & Brunette, L. (2009). How Translators Use Tools and Resources to Resolve Translation Problems: an Ethnographic Study. In Beyond Translation Memories: New Tools for Translators. MT-Summit Workshop, Ottawa, le 29 août 2009. Retrieved from http://www.mt-archive.info/MTS-2009-Desi lets-2.pdf. Domas White, M., Matteson, M., & Abels, E.G. (2008). Beyond dictionaries: Understanding information behavior of professional translators. Journal of Documentation, 64(4), 576–601. http://doi.org/10.1108/00220410810884084. Dragsted, B., & Carl, M. (2013). Towards a Classification of Translation Styles based on Eye-tracking and Keylogging Data. Journal of Writing Research, 5(1), 133–158. Durán-Muñoz, I. (2010). Translators’ Needs into Account: A Survey on Specialised Lexicographical Resources. In Granger S. & Paquot M. (Eds.), eLexicography in

Investigating the Use of Resources in the Translation Process

33

the 21st century: New Challenges, New Applications. Proceedings of ELEX 2009 Cahiers du Cental (Vol. 7, pp. 55–66). Lovaina-La-Nueva: Presses Universitaires de Louvain. Durán-Muñoz, I. (2012). Meeting translators’ needs: translation-oriented terminological management and applications. Journal of Specialised Translation, (18), 77–92. Ehrensberger-Dow, M., & Massey, G. (2014). Cognitive Ergonomic Issues in Professional Translation. In Schwieter J.W. & Ferreira A. (Eds.), The Development of Translation Competence: Theories and Methodologies from Psycholinguistics and Cognitive Science (pp. 58–86). Newcaste upon Tyne: Cambridge Scholars Publishing. Ehrensberger-Dow, M., & Massey, G. (2015). Translation process research in the workplace. EST Newsletter, 46, 11–12. Enríquez Raído, V. (2011). Investigating the Web search behaviours of translation students: An exploratory and multiple-case study (Doctoral dissertation), Retrieved from http://www.tdx.cat/bitstream/handle/10803/21793/Enriquez_PhD_Thesis_Fi nal.pdf;jsessionid=EC5BCCAEB65F6BD737CAADFB65EB3CD7.tdx2?sequence=1. Enríquez Raído, V. (2014). Translation and Web Searching. New York and London: Routledge. Gough, J. (2011). An empirical study of professional translators’ attitudes, use and awareness of Web 2.0 technologies, and implications for the adoption of emerging technologies and trends. Linguistica Antverpiensia, New Series – Themes in Translation Studies, (10), 195–217. Gough, J. (2016). The patterns of interaction between professional translators and online resources (Doctoral dissertation). Retrieved from http://epubs.surrey.ac.uk/ 813254/. Heinström, J. (2006). Broad exploration or precise specificity: Two basic information seeking patterns among students. Journal of the American Society for Information Science and Technology, 57(11), 1440–1450. Heyes, C. (2012). New thinking: the evolution of human cognition. JSTOR, 367(1599), 2091–2096. http://doi.org/10.1098/rstb.2012.0111. Holmes, J.S. (1972). The name and nature of translation studies. Third International Congress of Applied Linguistics, August 21026. Copenhagen. House, J. (2000). Consciousness and the Strategic Use of Aids in Translation. In ­Tirkkonen-Condit S. & Jääskeläinen R. (Eds.), Tapping and Mapping the Processes of Translation and Interpreting: Outlooks on Empirical research (pp. 149–162). Amsterdam and Philadelphia: John Benjamins. Illich, I. (1981). Shadow Work. New Hampshire and London: Marion Boyars Inc. Jääskeläinen, R. (1989). Translation Assignment in Professional vs. Non-professional Translation: A Think-aloud Protocol Study. In Séguinot C. (Ed.), The Translation Process (pp. 87–98). Toronto: H.G. Publications.

34

Gough

Krings, H.P. (1986). Translation problems and translation strategies of advanced German learners of French (L2). In House J. & Blum-Kulka S. (Eds.), Interlingual and Intercultural Communication: Discourse and Cognition in Translation and Second Language Acquisition Studies (pp. 263–276). Tübingen: Gunter Narr. Lagarde, L. (2009). Le traducteur professionnel face aux textes techniques et á la recherche documentaire (Unpublished Doctoral dissertation), École Supérieure D’interprétes et detraducteurs, Université Paris III – Sorbonne Nouvelle. Lauffer, S. (2002). The translation process: An analysis of observational methodology. Cadernos de Tradução, 2(10), 61–74. Legal Transform. (2015, July 8). Re: On the Internet as a research tool for translators. [Online forum comment]. Retrieved from http://www.proz.com/forum/transla tion_theory_and_practice/287897-on_the_internet_as_a_research_tool_for_trans lators.html#2446780. Luukkainen, T. (1996). Comparison of Translations Made With and Without Reference Material: A Think-aloud Protocol Study (Unpublished Master’s thesis), School of Translation Studies, University of Joensuu, Savolinna. Marcu, D. (2015). Emerging Technologies as a Cause of Shadow Work. LinkedIn. Retrieved from https://www.linkedin.com/pulse/emerging-technologies-cause-shad ow-work-daniel-marcu (Consulted on 13/02/2017). Massey, G., & Ehrensberger-Dow, M. (2010). Investigating demands on language professionals: methodological challenges in exploring translation competence. Bulletin Suisse de Linguistique Appliquée, (1), 127–141. Massey, G., & Ehrensberger-Dow, M. (2011a). Commenting on translation: implications for translator training. Journal of Specialised Translation, (16), 26–41. Massey, G., & Ehrensberger-Dow, M. (2011b). Investigating information literacy: A growing priority in translation studies. Across Languages and Cultures, 12(2), ­193–211. http://doi.org/10.1556/Acr.12.2011.2.4. Massey, G., & Ehrensberger-Dow, M. (2011c). Technical and Instrumental Competence in the Translator’s Workplace: Using Process Research to Identify Educational and Ergonomic Needs. Ilcea, 14, 1–12. Massey, G., & Ehrensberger-Dow, M. (2015). The actors and factors behind translation quality: Exploring processes, products and environments. In Points of View on Translator’ Competence and Translation Quality, 27 November 2015. Cracow. Mellinger, C. (2015). On the applicability of Internet-mediated research methods to investigate translators’ cognitive behaviour. Translation & Interpreting, 7(1), 59–71. Muñoz Martin, R. (2015). From process studies to cognitive translatology. EST Newsletter, 46, 10–11. Mutta, M., Pelttari, S., Salmi, L., Chevalier, A., & Johannson, M. (2014). Digital literacy in academic language learning contexts: Developing information-seeking

Investigating the Use of Resources in the Translation Process

35

­competence. In Guikema J.P. & Williams L. (Eds.), Digital Literacies in Foreign and Second Language Education, (Vol. 12, pp. 240–258). CALICO Monograph Series. Nord, B. (2009). In the year 1 BG (before Google): Revisiting a 1997 study Concerning the Use of Translation Aids. In Wojtak G. (Ed.), Translatione via facienda. Festschrift für Christiane Nord zum 65. Geburtstag (pp. 203–217). Frankfurt am Main: Lang. O’Brien, S. (2012). Translation as human-computer interaction. Translation Spaces, 1, 101–122. http://doi.org/http://dx.doi.org/10.1075/ts.1.05obr. Pinto, M., García-Marco, J., Granell, X., & Sales, D. (2014). Assessing information competences of translation and interpreting trainees: A study of proficiency at Spanish universities using the InfoliTrans Test. Aslib Journal of Information Management, 66, 77–95. http://doi.org/10.1108/AJIM-05-2013-0047. Pinto, M., García-Marco, J., Sales, D., & Cordón, J.A. (2010). Interactive Self-assessment Test for Improving and Evaluating Information Competence. Journal of Academic Librarianship, 36(6), 526–538. http://doi.org/10.1016/j.acalib.2010.08.009. Pinto, M., & Sales, D. (2008). INFOLITRANS: a model for the development of information competence for translators. Journal of Documentation, 64(3), 413–437. http:// doi.org/10.1108/00220410810867614. Risku, H. (2002). Situatedness in translation studies. Cognitive Systems Research, 3(3), 523–533. http://doi.org/10.1016/S1389-0417(02)00055-4. Rowlands, I., Nicholas, D., Williams, P., Huntington, P., Fieldhouse, M., Gunter, B., Tenopir, C. (2008). The Google generation: the information behaviour of the researcher of the future. Aslib Proceedings: New Information Perspectives, 60(4), 290– 310. http://doi.org/10.1108/00012530810887953. Sales, D., & Pinto, M. (2011). The professional translator and information literacy: Perceptions and needs. Journal of Librarianship and Information Science, 43(4), 246– 260. http://doi.org/10.1177/0961000611418816. Spink, A., & Heinström, J. (Eds.). (2011). New Directions in Information Behaviour. Emerald Insight. Tirkkonen-Condit, S. (1989). Professional versus non-professional translation: a think aloud protocol study. In Séguinot C. (Ed.), The Translation Process (pp. 73–85). ­Toronto: H.G. Publications. Valli, P. (2012). How long is a piece of string? Concordance searches and user behavior investigated. Proceedings of Aslib Conference Translating and the Computer 34. London: 29–30 November 2012 (pp. 29–30). Van Der Meer, J., & Ruopp, A. (2014). MT Market Report 2014. Retrieved from: https:// www.taus.net/think-tank/reports/translate-reports/mt-market-report-2014. Varantola, K. (1998). Translators and Their Use of Dictionaries: User Needs and User Habits. In Atkins B.T.S. (Ed.), Using Dictionaries: Studies of Dictionaary Use by Language Learners and Translators (pp. 179–192). Tübingen: Max Niemeyer.

36

Gough

Volanen, S. (2015). Translating with the Web. Professional translators’ information-­ seeking behaviour in translation with online resources (Unpublished Masters thesis), University of Turku. Zapata, J. (2015). Investigating Translator-Information Interaction: A Case Study on the Use of the Prototype Biconcordancer Tool Integrated in CASMACAT. In Carl M., Bangalore S., & Schaeffer M. (Eds.), New Directions in Empirical Translation Process Research: Exploring the CRITT TPR-DB (pp. 139–155). Springer. Zetzsche, J. (2015). Tool Box Journal 247. A Computer Journal for Translation Professionals, (247).

chapter 2

User Perspective on Translation Tools: Findings of a User Survey Anna Zaretskaya, Gloria Corpas Pastor and Míriam Seghiri Abstract Electronic tools have become an important part of a translator’s work. However, professional translators are not always satisfied with the tools they have at their disposal. In addition, many translators are not aware of all the existing types of tools they can use. In this way, it is necessary to investigate translators’ needs regarding electronic tools, as well as to provide them with the necessary training to help adopt them. In this article we discuss different methods that can be applied to investigate user requirements in the context of translation tools. User surveys are one of the most popular methods. We present the process of implementation and the results of a user survey on translation technologies focusing on different factors that influence translators’ ­adoption of tools, such as their education and computer competence. We also discuss translators’ preferences regarding features and characteristics of computer-assisted translation (cat) tools. The findings of the survey show that translators do not only expect their cat tools to have a full set of features, but also to be easy to use and intuitive. We suggest that usability of translation tools is closely related to the users’ productivity, which has to be taken into account when investigating translators’ needs regarding electronic tools.

Keywords translation technologies – user requirements – user survey – translation training – cat tools – machine translation

1 Introduction Electronic tools are an indispensable part of the translator profession today. A translator starting a career in the industry is expected, as a minimum

© koninklijke brill nv, leiden, ���8 | doi 10.1163/9789004351790_004

38

Zaretskaya, Pastor and Seghiri

­requirement, to be able to work with translation memory (tm) software. Most of these programs, apart from providing the tm functionality, offer other features created to help translators save time by automatising or facilitating ­various tasks that are part of the translation process, such as grammar check, quality check, automatic handling of formatting tags, among many others (Bowker & Corpas-Pastor, 2015). Translators who wish to keep pace with the industry developments have to face complex software interfaces as well as a large variety of tools designed for different purposes. Translation training, in its turn, tries to handle the challenge of preparing beginner translators for technology-assisted workflow by introducing technology in the teaching programmes (Bowker et al., 2008). Thus, universities offer courses on computerassisted translation (cat), corpus-based translation, and tools for working with terminology (Bowker & Marshman, 2009). Furthermore, the task of preparing students for their future work with technologies does not only consist in teaching them how to work with the tools, but also how to choose among all the available tools on the market (Starlander & Morado Vázquez, 2013). Translation technologies, and in particular tools created for professional translators such as tm tools, are supposed to increase their productivity and income. However, in reality they often evoke mixed feelings among professionals. There are various reasons for this: they are considered too expensive, too time-consuming to learn, full of unnecessary features, and sometimes too slow and unstable. For some they even present a threat for the profession, as they believe that the technological advancements we are observing today will lead to computers fully replacing human translators in the future. We argue that, in order to reduce the gap between translators and technologies, it is necessary to investigate their needs as users of these tools, and educate them properly about their benefits. In this article we will first present different methods that can be applied in order to investigate translators’ requirements regarding translation technologies (Part 2). We will show that user surveys are a very important, although not the only method to obtain information on translators’ preferences regarding the tools they use. In Part 3 we present a user survey on translation tools and discuss the results. This section will cover issues including popularity of certain types of tools, how it depends on translators’ education and computer competence, the most useful features of cat tools, and translators’ desires on how those tools can be improved. We will conclude by summarizing the results and discussing ­whether translation training can better prepare translators for technological challenges of the industry, how translators see a useful electronic tool and what further steps can be undertaken in order to better understand their needs.

User Perspective on Translation Tools

2

39

Current Research on Translation Technologies and User Needs

The research on translation technologies mainly lies in the field of Natural Language Processing (nlp) with machine translation (mt) being the main research area for several decades. But apart from mt, nlp techniques are applied for enhancing tm leverage, collecting and cleaning parallel data, automatic post-editing of machine-translated texts, terminology extraction, among others. All these are areas of nlp research that aim at improving the performance of the tools, automatise as many repetitive tasks as possible in order to make professional translators’ life easier. There is considerably less research on the user aspect of these technologies. Namely, how users interact with them, what exactly they like and dislike about the tools, their favourite functionalities, and what new types of technologies they would like to have that do not exist yet. One approach to investigating needs of software users is to ask them directly, mainly with the help of user questionnaires. The user surveys that were previously conducted in translation industry (Fulford & Granell-Zafra, 2005; Dillon & Fraser, 2007; DePalma & Kelly, 2009; Torrez Domínguez, 2012) aimed at investigating different technology-related aspects of translation. Some of them studied a specific type of translation technologies. By a way of example, Doherty et al. (2013) focused on machine translation, how it was used by translation companies, and which quality assessment practices were adopted to measure or compare their proprietary mt engines. In addition, among other things it investigated what specific systems respondents were using; whether they were based on statistical, rule-based or hybrid methods; for what purposes companies used mt; and how much of the mt-translated text they normally corrected. The survey by Lagoudaki (2006) focused specifically on translation memory systems and how they can be improved from a translator’s point of view. The survey concerned different aspects of working with tm, such as the reasons why translators used them, what specific tools they used, which text domains were the most common to use tm with, and what functionalities were the most useful for translators. The survey described by Gornostay (2010) and Blancafort et al. (2011) studied tools and practices in translation industry first of all related to terminology, but also to mt tools and nlp applications such as corpora and concordancers. Some surveys, instead of focusing on a specific type of tools, study tendencies or developments in the translation industry in general. One of such surveys was conducted by TradOnline1 (2011) and focused on the changes in the industry caused by arising of new technologies, including appearance of 1 http://www.tradonline.fr/.

40

Zaretskaya, Pastor and Seghiri

new ways of working (such as, for instance, crowdsourcing), translators’ ­attitudes and expectations regarding these changes, as well as evolution of technology as a whole (translator tools and the Internet more generally). Gough (2011, this volume) conducted a study with a yet another perspective on the impact of new technologies on translation practices, namely the changes caused by the emergence of the Web 2.0 technology and issues these changes present to translators. The issue of translation interoperability was covered by the survey distributed by taus (Translation Automation User Society) and lisa (Localization Industry Standards Association); the results were presented in 2011.2 It focused on the problems that arise when translators have to work with multiple translation agencies and when they use a variety of different tools and file formats. Another approach to collecting user feedback in order to investigate translators’ requirements consists in studying the process of translation based on the quantitative data obtained in a natural translation setting. In particular, one of the ideas behind this approach is that translators need tools that make them work faster. Indeed, productivity and speed are crucial in translators’ work nowadays: the faster you are, the more translation throughput you have, and, consequently, the more income you get. Thus, it has been studied how the use of different tools affect translators’ productivity. The quantitative measures used in this type of studies are normally related to translation time, number of keystrokes, number of pauses while typing, time of eye fixations, among others. One of the most popular directions of research in this regard consists in assessing productivity gains with post-editing of mt. Post-editing (pe) is usually understood as ‘a human being (normally a translator) comparing a source text with the mt and making changes to it to make it acceptable for its intended purpose’ (Koby, 2001: 1). As the quality of mt improves, researchers and companies become interested in benefits mt can bring in a professional translation setting, in particular by means of post-editing. These studies normally compare translators’ productivity and sometimes also final quality when translating a text from scratch and when modifying translations produced by mt systems (Plitt & Masselot, 2010; Federico et al., 2012; Läubli et al., 2013; Zhechev, 2014; Parra Escartín & Arcedillo 2015). All of the above mentioned works reported some productivity gain when applying pe compared to translation from scratch. 2 Video of the presentation: http://videolectures.net/w3cworkshop2011_vandermeer_prspec tives/. Slides of the presentation: http://videolectures.net/site/normal_dl/tag=563177/w3c workshop2011_vandermeer_perspectives_01.pdf.

User Perspective on Translation Tools

41

Interactive Machine Translation (imt) is another branch of research that tries to improve translators’ experience with technology, which brings together some of the mt functionalities in an interactive environment of a cat tool (Koehn & Haddow, 2009; Ortiz-Martínez & Casacuberta, 2014). imt tools offer assistance to human translators in form of translation suggestions of whole segments or parts of segments that are generated by an mt engine. Often they function as autosuggest, i.e. while generating the suggestion, the system takes into account what the user types. Thus, Koehn and Haddow (2009) report improvements in both speed and accuracy when using their imt system. Finally, some works studied the effectiveness of the sub-segment leverage feature, which consists in retrieving parts of segments from tm instead of only looking for a match for the entire segments (Flanagan, 2015). Often these parts are reassembled to produce new matches, thus significantly improving the tm system’s recall, i.e. increases the amount of retrieved useful segments ­(Parra Escartín, 2015). 3

Translators’ Perspective on Electronic Tools: User Survey

As we pointed out above, the survey method is one of the most efficient when it comes to investigating user needs. It allows to reach a wide population of users with minimal costs, as well as obtain large amounts of both quantitative and qualitative data. In this section we present results of the survey we conducted in order to investigate professional translators’ attitudes, practices and requirements towards various types of electronic tools. We start with reporting on how the survey was designed and implemented, and describe the p ­ opulation of respondents. The results discussed here cover such issues as respondents’ use and familiarity with electronic tools, how it is related to their education and computer competence, what the most useful features in cat tools are for translators, and what are the general characteristics that are ­crucial for translators in any type of translation tools. 3.1 Survey Design and Implementation The questionnaire3 was designed using SurveyMonkey, an online questionnaire building tool. It was composed of separate sections, where the first section concerned the user profile, the second section included general questions on the use of technologies, and the rest of the sections were focused on ­specific 3 https://www.surveymonkey.com/s/FQ7HHZV (last accessed on 25 February 2016).

42

Zaretskaya, Pastor and Seghiri

types of tools: Machine Translation, Translation Memory, Textual Corpora, ­Terminology Management, Terminology Extraction, Web Resources and Quality Assurance. This structure was chosen in order to be able to use the ‘skip logic’: if respondents were not familiar with tools of a particular type or were not using them in their work, most of the questions in the corresponding section were irrelevant to them, so they could be skipped automatically and the respondents were directed to the next section of the questionnaire. ‘Skip logic’ makes the survey navigation much easier and allows saving respondents’ time and increasing the response and completion rates. Each section contained from two to twenty-one questions. One of the main difficulties one encounters when analysing information on user requirements is the high subjectivity of obtained data. Often users are not certain about their own needs or do not know how to present them in a clear straightforward way. In addition, questionnaire method of collecting user feedback is prone to ambiguities and misunderstanding. In order to prevent this kind of issues, we carried out various preparation and testing steps prior to launching the survey. The first step consisted in analysing publicly available information, such as translators’ blogs, forums, social networks and web sites that could throw light upon the most discussed topics related to translators’ use of translation technologies and identify potential issues and problems that needed to be tackled. This analysis helped us identify, among other things, the most popular cat tools; what features they had; some of the problematic features, such as handling of tags and terminology management. Based on this information together with various user surveys previously conducted in this field, the first draft of the questionnaire was designed. Subsequently, we proceeded to carrying out cognitive interviews with two potential respondents who worked as freelance translators. Cognitive interview is a common survey testing technique where the respondents read the questions and have to speak aloud commenting their reasoning during question answering. This way the interviewer can detect difficulties that participants might encounter while completing the survey and make sure that participants do not misinterpret any question and that the procedure of completing the survey is clear (Willis, 2005). Concurrently, we asked several employees of language service provider companies to complete the survey and provide feedback in terms of the questionnaire content, structure, design, and question wording. After the feedback was collected both from the interviewees and the domain experts, the appropriate changes were made, and we proceeded to the pilot study, which consisted in collecting a small sample of responses (in our case 12) and analysing the results to identify possible defects and redundancies.

User Perspective on Translation Tools

43

The testing of the questionnaire proved to be very helpful, especially for identifying additional answer choices that were not included in the first ­version, and for finding clearer and more specific question wording. Other methods for avoiding ambiguity, redundancy and other problems were applied, such as (1) using as less technical jargon and very specific terms as ­possible; (2) using check-boxes question types, where respondents are able to select multiple options; (3) providing ‘I don’t know’ and ‘Other’ options for cases when the r­ espondent does not find the most suitable answer among the ones available; (4) providing comment fields and open-ended question, where participants could answer questions in a free manner and use wording of their own choice. The questionnaire link was distributed through translation companies, mailing lists and social media groups for translators, translation blogs and translation associations. The final version of the questionnaire took from 15 to 30 minutes to reply. The participants responded actively and many provided feedback and comments. 3.2 Participants’ Profile We received 736 completed responses from 88 different countries. The majority of respondents were experienced translators and almost a half of them had more than 11 years in the industry. The vast majority of translators worked freelance. Figure 2.1 shows the distribution of various occupation types within the sample. From the 720 respondents to this question, the two largest groups were freelancers who worked with an agency but also worked independently apart, and freelancers who only worked independently. Only 86 translators just worked with an agency. Other respondents worked as in-house translators in a translation company (21) or in a non-translation company (23). Finally, 8 translators worked in a government or public institution and 10 were students. (Figure 2.1). Another important characteristic of user profile, in our opinion, is the education. As we can observe in Tables 2.1 and 2.2, most respondents had at least some education in both translation and information technologies (it). Almost a half of the sample had a university degree in translation, and 44% attended specialised courses or seminars. Surprisingly, about a quarter of respondents did not have any training in translation. Another significant finding is that 43% of respondents had attended courses and seminars on it, which shows that translators have great interest in technologies and are motivated to learn how to leverage the variety of available tools and incorporate them in their workflow in a beneficial manner. Moreover, almost all participants were computer users and evaluated their it competence level as advanced (47%) or experienced (42%), while only 10% considered their level as average and 1% as poor.

44

Zaretskaya, Pastor and Seghiri 3%

1%

2%

Student Freelancer working both with an agency and independently In-house translator in a translation company

34%

45%

Freelancer working with an agency Independent freelancer (without an agency) In-house translator in a non-translation company

12% 3%

Translator in a public institution

Figure 2.1 Employment of the survey participants. Table 2.1

Participants’ education and training in translation.

ba in translation ma in translation PhD in translation Specialised courses, seminars, workshops, etc. Specialised courses on cat tools None

21.4% 25.0% 2.7% 44.2% 29.4% 24.6%

Table 2.2 Participants’ education and training in it.

bsc in it msc in it PhD in it Specialised courses, seminars, workshops on it None

3.8% 3.6% 0.7% 42.9% 38.9%

3.3 Translators’ Knowledge and Use of Electronic Tools As we pointed out in the Introduction, there is a wide range of tools available for translators today to help them automatise different tasks involved in the translation process. Despite this rich diversity and the technological and

45

User Perspective on Translation Tools

s­cientific advancements in the field of translation technologies in the last years, translators keep using only the ‘traditional’ translation memory tools on a regular basis. In fact, many of them are not even aware of all the existing ways in which they can employ the technologies to make their tasks easier. Thus, tm systems appear to be the only type of tools that was used regularly by the majority of respondents (417 out of 718), while much less translators had not incorporated tm in their daily routine, but used it sometimes (113). However, there were still a considerable number of respondents who had never heard of such technology (70) and who were familiar with it but still remained reluctant to use it (100). In comparison, other tools were much less popular. ­Consider Figure 2.2, which illustrates the degree of familiarity of the respondents with selected electronic tools, namely translation memories (tm), terminology management systems, term extractors, concordancers, tools for building and managing textual corpora, localisation software, quality assurance (qa) tools, and purchase order (po) software. Except for tm software, other tools, even though familiar to the ­respondents, were not part of their usual workflow. In addition, one type of technologies, concordance systems, was completely unknown to the a great number of translators who replied to this question. Tools for compiling or managing corpora were the least common to use on a regular basis (only 32 participants), even though 272 respondents reported that they had heard about them before. In total, this result indicates a low rate of familiarity with less popular types 0

100

200

300

400

tm terminology management term extractor concordancer corpora tool localisation tool qa software po software Never heard of

Have heard of, but do not use

Sometimes use

Figure 2.2 Respondents’ familiarity with different types of tools.

Use regularly

500

46

Zaretskaya, Pastor and Seghiri

of technologies, as from the 718 who answered this question none of the tools except for tm was familiar even to a half. As for mt systems, they were used by 36% of respondents, while almost the same number (38%) were not using any mt and were not planning to use it in the future. A smaller percentage (15%) did not reject this technology completely claiming that they were planning to use it in the future, and 11% used it before but abandoned it afterwards. This is a strikingly different result compared with tm systems, which were used by 76% of respondents. The majority of participants (85%) have never worked with textual corpora. About a half of all the respondents who had worked with corpora (111) also reported that they compiled their own corpora. About a half (52%) of the translators who participated in the survey used a terminology management tool integrated in their translation software, a slightly smaller number of translators did not use any (42%), and only 6% used a standalone terminology management tool. Terminology extraction (te) tools, on the other hand, were much less adopted among professional translators, as 75% did not use them at all, 21% had a te system integrated within a cat tool, and only 4% had a standalone te system. Finally, the last type of tools investigated in the questionnaire was the qa tools. They were reported to be used by 60% of translators, of which 35% had this feature integrated in their translation software, 11% had a standalone qa tool, and 14% used both types of systems. Table 2.3 puts together all the statistics collected regarding the usage rates of different types of technologies. In relation to Figure 2.3, the number of users was counted by joining together the answers ‘Sometimes use’ and ‘Use regularly’ from the figure. It is not a big surprise that tm systems are the most widely used type, and it is rather remarkable that less participants used mt than ­terminology management and qa tools. One of the reasons is possibly that some of the technologies mentioned in the questionnaire are now often integrated in cat tools (as, for instance, terminology management and quality assurance). This is also confirmed by some of the comments to this question, which say that ‘translation software includes most of the above’. Other technologies are quite specific and probably translators do not consider them appropriate for everyday use (such as tools for compiling and managing corpora). Apart from the mentioned tools respondents were asked to indicate any additional types of software they used in their work, which were sentence aligners (tools for building translation memories from parallel bilingual texts), optical character recognition software, subtitle translation tools and speech recognition tools. Further data analysis can give us some explanations as to why some tools are more popular than others. In particular, in case of free web-based a­ utomatic

47

User Perspective on Translation Tools Table 2.3 Popularity of different types of software among the survey participants.

Type of tool

Percentage of users

Percentage of non-users

tm mt

76% 36%

Standalone mt Integrated mt Corpora Corpora tools Terminology management tools Terminology extraction tools Quality assurance

13% (of mt users) 35.5% (of mt users) 15% 17% (of all corpora users) 58%

24% 64% (including 11% past users and 15% future users) 87% (of mt users) 35.5% (of mt users) 85% 83% (of all corpora users) 42%

25%

75%

60%

40%

translation services, translators mostly pointed out bad quality that prevents them from using it (67%), and some of them simply did not find it useful (35%). In addition, a number of respondents commented that they were not allowed to use mt due to the client’s information security requirements. Translators who do not compile corpora mostly refuse to do so because it requires too much effort, as was reported by 57%, 40% are satisfied with the publicly available corpora, and 34% do not know about any technique for compiling corpora. A rather positive finding is that only 15% thought that it was not a useful thing to do, which means that most of the translators would create their own corpora if they had better methods and tools available. Education and training can be an important factor in translators’ adoption of different tools. In particular, studies in translation are supposed to provide knowledge on the types of electronic tools translators have at their disposal, on the benefits they bring, and to teach how to use them. In this relation, we performed a cross-tabulation analysis of translators’ education and their use of different tools. Consider Table 2.4, which shows the percentage of translators with certain level of education in translation that used different electronic tools. In particular, the lines of the table represent the degree or training translators received (bachelor’s, master’s, special courses or seminars in translation, and courses on cat tools). The columns of the tables represent selected electronic tools: translation memory software (tm), mt software or service (mt),

48

Zaretskaya, Pastor and Seghiri View iew the whole text while translating

Combine ine words from the tm or termbank Integated ntegated terminology management Integrated mt system Possibility to save tms as a ffile ile on pc orts unlimited number of languages Supports Fill tm from parallel texts online Adjustable segmentations settings Automatic atic access to terminology resources Store data in the cloud Access to publicly available tms Real time quality check Web-based version of the tool High working speed Simple interface, easy to learn Adjustable keyboard shortcuts Work on different os than Windows Supports ts big number of document formats Supports pports formats from other cat tools Share tm 0 Inconvenient Figure 2.3

Not important

100

200

Not so useful

300 Useful

400

500

600

Essential

Usefulness of features of a cat tool.

electronic textual corpora, electronic tools for compiling and managing corpora, terminology management software, and tools for terminology extraction. Thus, each cell of the tables shows the percentage of all translators with the corresponding training that reported using or having used the corresponding type of tools. We can see clearly that the highest percentages are observed almost always in the ‘Courses cat’ line, except for Corpora and Corpora tools. It has to be mentioned that the number of respondents who used corpora and corpora

49

User Perspective on Translation Tools Table 2.4 Education and training in translation and use of electronic tools.

ba ma Courses Courses cat None

tm

mt

Corpora Corpora tools Terminology te management

78.1% 86.7% 78.4% 92.1%

46.5% 46.4% 51.9% 54.4%

17.5% 25.1% 13.0% 19.0%

13.3% 28.0% 30.4% 30.4%

58.8% 36.3% 65.5% 77.5%

26.8% 25.7% 28.8% 34.6%

66.9% 43.1% 10.7%

66.7%

44.7%

23.2%

Table 2.5 Computer competence and use of electronic tools.

Advanced Experienced Average Poor

tm

mt

Corpora Corpora tools Terminology te management

83.7% 72.8% 59.5% 25.0%

54.5% 19.0% 41.5% 11.8% 37.8% 11.1% 14.3% 0%

35.1% 26.7% 25.0% 0%

67.0% 51.8% 43.1% 25.0%

32.1% 18.7% 15.5% 14.3%

tools was very small compared to other tools, therefore these numbers are less representative. Based on these data, we can assume that specialised courses on cat tools provide the most efficient training for using electronic tools. Nevertheless, courses on translation and university education also seem to make a contribution: for most tools, the lowest percentage is observed in the group with no education in translation. To summarise, there are two main observations we can make based on ­Table 2.4. Firstly, courses on cat tools seem to be the most helpful for adopting different types of tools. Secondly, translators who finished specialised courses on translation and university degree in the field were more likely to use e­ lectronic tools than those who did not have any training at all. However, as specialised courses show higher percentages, it makes us wonder whether the currently provided university education is not offering all the necessary ­training on electronic tools. In other words, even though the education in translation helps to adopt electronic tools to some extent, many translators have to resort to some additional courses to add to the training provided by the

50

Zaretskaya, Pastor and Seghiri

university. Finally, it can be surprising to see that the percentage of corpora tools users is significantly higher for those who had no education in translation. Note that this percentage is calculate on the total number of corpora users, which was rather small to draw conclusions for this specific case. Similarly, we considered how the usage rates of different tools depend on the respondents’ computer skills. Thus, participants of the survey were asked to auto-evaluate their computer competence as ‘Advanced user’, ‘Experienced user’, ‘Average’ or ‘Poor’. As in the case of education, we look at how many translators in each of these groups use different types of tools (Table 2.5). We can see that there is a clear relation between the level of it competence and the percentage of users for all tools: more advanced computer users are more likely to adopt electronic tools than translators with poor computer competence. This means that translators’ understanding of how to benefit from electronic tools very much depends on their general computer skills. To summarise the findings on translators’ knowledge of electronic tools, the survey showed that majority of translators who responded were not familiar with all the variety of tools that were at their disposal. Often they did not know how they could benefit from certain type of tools, and sometimes even were not aware that those tools existed. Courses on cat tools and specialised courses on translation seem to be an efficient way for a translator to get acquainted with the technological assistance those tools can provide. University education in translation is a less strong factor when it comes to translators’ adoption of tools. And finally, computer competence seems to be directly related with usage of translation tools. 3.4 Features of cat Tools State-of-the-art cat tools include an astonishing amount of functionalities that perform all sorts of tasks that make part of translation process. In fact, not all of them are used by all translators. In the survey, we tried to find out whether there are features that are absolutely essential for all translators, and whether there are ones that none of the translators find useful; or, on the contrary, there is no agreement on that and it all depends on translators’ individual tastes and habits. Thus, the respondents were given a list of functionalities and characteristics that today’s tools have, and were asked to label them as ‘essential’, ‘useful’, ‘not so useful’, ‘not important’, or ‘inconvenient’. The statistics of answers for this question are illustrated in Figure  2.3. There is a number of features that are considered essential by many users, such as integrated terminology management system, possibility to save tm on your own pc, high working speed, simple interface and smooth learning curve, support for a big number

User Perspective on Translation Tools

51

of ­document formats, and support for formats originated from other tm software. In general, all of the mentioned features were regarded as useful, while some of them, such as storing data in the cloud, web-based version of the tool, or different os versions seem less important for respondents. Additionally, many users mentioned the auto-propagation feature, which allows to include translated segments of the text in the tm automatically while working on one text so that they can be used later throughout the whole text. Translators were also asked to describe their favourite and most hated feature in their own words in a comment box. Out of 403 translators who provided their comments, a remarkable number of 103 mentioned concordance search as one of their favourite features. This is an interesting finding considering that most translators reported that they did not use corpora. However, in fact, concordance search is the main purpose of corpora for translators, which allows them to see the searched word or phrase in context. This points to the fact that, even though most translators reported that they did not use corpora, they do use their translation memories as parallel corpora. Other favourite features were autopropagation (12 participants), autosuggest (16), easy to use interface (19), and terminology management (22). Interestingly, terminology management also appears among the most hated features (7 participants), along with not user-friendly interface (12) and bad handling of tags (12). The fact that ­terminology management appears both among the most favourite and most annoying features can be a sign that it is a useful feature as all translators find it necessary to manage their terminology, but they do not like the feature’s current implementations. 3.5 Translation Tools and User Requirements The survey covered a broad range of tools of different kinds, from translation memory systems to terminology extraction tools, electronic dictionaries, and many others, but there are some general tendencies that can be noticed in translators’ requirements. User-friendliness is one of them. In fact, it seems to be an important issue that currently available tools cannot fully handle. ­Software usability is defined by the international iso standard for software quality as ‘a set of attributes that bear on the effort needed for use, and on the ­individual assessment of such use, by a stated or implied set of users’ (iso/iec 9126, 1991). Even though many commercial cat tools include all the features rated as useful or essential by the respondents, often they are implemented at the cost of usability. In translators’ own words, ‘some of the functionalities are too complicated’, ‘it is still complicated to learn how to use’, ‘too many features to learn’, etc. Thus, 12 out of 133 translators mentioned that their software is not user-friendly, and that was the most annoying characteristic of the software

52

Zaretskaya, Pastor and Seghiri

they could think of. In addition, 8 out of 104 mentioned user-friendliness when commenting about general improvements they would like to see in translation tools (‘the longer the training needed the worse a cat tool is to us’). Another tendency, which is in fact related to usability, is the growing multifunctionality of cat tools. Even though having many features in one tool makes it complicated to learn, translators prefer that to having a separate standalone tool for each functionality: they require additional installation p ­ rocess, an ­effort to learn and get used to, and often additional costs. Apart from that it would produce compatibility issues between different file formats. Thus, translators mostly prefer systems of different functions to be integrated in one translation tool. For instance, terminology management was performed within a cat tool by more than a half of the survey respondents, and only a very small number (6%) used a standalone terminology management tool. Similarly, 21% had a te system integrated within a cat tool, and only 4% had a standalone te system; 35% had the quality assurance (qa) feature integrated in their translation software, 11% had a standalone qa tool, and 14% used both types. mt systems also followed this line. According to the comments, many respondents prefer to use the mt system integrated in their cat tool, and in case they do not have one, they prefer to resort to online mt services. mt feature in a cat tool allows users to see the translation suggestions coming from an mt engine and use them directly, modify them, or discard them. About a third of all mt users reported having an mt feature in their cat tools, and an equal amount did not have it. Surprisingly, another third part of the respondents did not know whether their cat tool had an integrated mt system. Finally, only about 13% of mt users reported having standalone mt software installed on their computer. The most common obstacle for using such mt systems was the lack of awareness of such systems and also, but less commonly, their high price, bad quality of translation, difficulty of implementation, or translators simply did not find them useful. Thus, we can summarise the observations made above by saying that translators prefer tools with a full set of functionalities, which are implemented in an intuitive and easy-to-use way. Speed is a crucial aspect of translator’s work, and they need a tool that is fast, easy to use, and permits to automatise all the non-creative parts of the work and organise the data. The question is, therefore, how to achieve the trade-off between simplicity and multi-­functionality. The survey method allowed us to list the features that translators like and ­dislike. However, as we have seen, the goal of the developers should not consist only in including as many functionalities as possible, but also to take into account the way they are implemented. Therefore, research on translators’ ­requirements must, apart from user surveys, focus on experimental case

User Perspective on Translation Tools

53

s­tudies in a real-life translation s­etting, which would measure translators’ speed and performance, as well as the quality of the final translations produced. Based on these data we can investigate how the tools’ functionalities intervene with each other, how accessible they are to the translator, how the translator interacts with them, and whether they fit into translators’ workflow and make them more productive. 4 Conclusions Understanding user needs is one of the goals of research on translation technologies. In this work we report on how we applied the questionnaire method in order to discover the needs and attitudes of professional translators regarding translation tools, first of all the ones that are designed specifically for translators, such as translation memory software, and the ones that are widely used by them, such as machine translation. Considering the general picture of translators’ knowledge and use of different types of tools, it appears that most of them only use tm software on a regular basis. We discovered that translators’ adoption of different kinds of tools strongly depends on their education. Firstly, respondents with some translation training were more likely to use or at least try electronic tools than respondents with no training in translation. Secondly, specialised courses on translation and cat tools were more helpful when it comes to learning how to use these tools than university education, as the usage rates were higher for those who participated in such courses. This shows that university education, even though it provides some training on electronic tools to prepare students to meet the job market requirements, does not fully keep up with the pace of the technology development in the industry. Another related finding was that translators’ self-assessed knowledge of computers is also directly connected with their adoption of tools: knowledge and skills in it coincide with higher usage rates. Even though only tm software was reported to be used on a regular basis by the majority of translators, in fact, this software nowadays allows to perform many different tasks apart from the retrieval of tm matches, such as concordance search, terminology management, quality assurance, alignment, and others. According to the survey results, translators see software multi-­ functionality as an advantage, as they prefer to work with one tool suitable for all tasks rather than having to resort to a separate tool each time, which is complicated in many ways. On the other hand, too many features and settings makes the software less user-friendly and more time-consuming to learn.

54

Zaretskaya, Pastor and Seghiri

The most important characteristics that a tm system should have, according to the respondents, are high working speed, simple interface, support for a big number of document formats including formats from other tm software, integrated terminology management system, automatic consistency and completeness check (and quality assurance in general), analysis for invoicing, and auto-propagation. A simple analysis of state-of-the-art tools is enough to compare them in terms of offered features and functionalities. However, evaluation of a tool’s working speed and usability is more difficult to carry out using quantitative methods. We argue that research on user requirements within the translation technology field should in addition to user questionnaires investigate translators’ performance, productivity and satisfaction in real-life case studies. Acknowledgements Anna Zaretskaya is supported by the People Programme (Marie Curie Actions) of the European Union’s Framework Programme (FP7/2007–2013) under rea grant agreement No 317471. The research reported in this chapter has been ­partially carried out in the framework of the research group Lexytrad and the projects NOVATIC and INTELITERM. References Blancafort, H., Heid, U., Gornostay, T., Méchoulam, C. & Daille, B. (2011). User-centred Views on Terminology Extraction Tools: Usage Scenarios and Integration into MT and CAT Tools. Proceedings of the TRALOGY Conference ‘Translation Careers and Technologies: Convergence Points for the Future’. Bowker, L. & Corpas-Pastor, G. (2015). Translation Technology. In Mitkov, R. (ed.), Handbook of Computational Linguistics. Oxford University Press, 2nd edition. Bowker, L. & Marshman, E. (2009). Better Integration for Better Preparation. Bringing Terminology and Technology more fully into Translator Training using the CERTT Approach. Terminology, 15 (1), pp. 60–87. Bowker, L., McBride, C. & Marshman, E. (2008). Getting more than you paid for? Considerations in Integrating Free and Low-cost Technologies into Translator Training Programs. Redit: Revista electrónica de didáctica de la traducción y la interpretación (1), pp. 26–47. DePalma, D.A. and Kelly, N. (2009). The Business Case for Machine Translation. Technical report, SDL, AMTA, EAMT.

User Perspective on Translation Tools

55

Dillon, S. & Fraser, J. (2007). Translators and TM: An Investigation of Translators’ Perceptions of Translation Memory Adoption. Machine Translation, 20 (2), pp. 67–79. Doherty, S., Gaspari, F., Groves, D., van Genabith, J., Specia, L., Burchardt, A., Lommel, A. & Uszkoreit, H. (2013). QTLaunchPad – Mapping the Industry I: Findings on Translation Technologies and Quality Assessment. European Comission Report. Retrieved from: http://www.qt21.eu/launchpad/sites/default/files/QTLP_Survey2i.pdf (Consulted on 3/03/2017). Federico, M., Cattelan, A. & Trombetti, M. (2012). Measuring User Productivity in Machine Translation enhanced Computer assisted Translation. Proceedings of the Tenth Conference of the Association for Machine Translation in the Americas (AMTA). Flanagan, K. (2015). Subsegment Recall in Translation Memory – Perceptions, Expectations and Reality. The Journal of Specialised Translation ( JoSTrans), 23, pp. 64–88. Fulford, H. & Granell Zafra, J. (2005). Translation and Technology: A Study of UK Freelance Translators. The Journal of Specialised Translation ( JoSTrans), 4, pp. 2–7. Gornostay, T. (2010). Terminology Management in Real Use. Proceedings of the 5th International Conference Applied Linguistics in Science and Education. Gough, J. (2011). An Empirical Study of Professional Translators’ Attitudes, Use and Awareness of Web 2.0 Technologies, and Implications for the Adoption of Emerging Technologies and Trends. Linguistica Antverpiensia, New Series, Themes in Translation Studies 10. Retrieved from https://lans-tts.uantwerpen.be/index.php/LANSTTS/article/view/284/182 (Consulted on 3/03/2017). ISO/IEC 9126. (1991). Software engineering – Product quality. Koby, G.S. (2001) Editor’s Introduction, in Krings H.P. Repairing Texts: Empirical Investigations of Machine Translation Post-Editing Processes, pp. 1–23. Koehn, P. & Haddow, B. (2009). Interactive Assistance to Human Translators Using ­Statistical Machine Translation Methods. Machine Translation Summit XII, pp. 73–80. Lagoudaki, E. (2006). Translation Memories Survey 2006: Users’ Perceptions around TM Use. Proceedings of the International Conference Translating & the Computer 28, pp. 15–16. ASLIB. Läubli, S., Fishel, M., Massey, G., Ehrensberger-Dow, M. & Volk, M. (2013). Assessing Post-Editing Efficiency in a Realistic Translation Environment. Proceedings of MT Summit XIV Workshop on Post-Editing Technology and Practice, pp. 83–91. Ortiz-Martínez, D. & Casacuberta, F. (2014). The new THOT Toolkit for Fully-automatic and Interactive Statistical Machine Translation. Proceedings of the Demonstrations at the 14th Conference of the European Chapter of the Association for Computational Linguistics, pp. 45–48. Gothenburg, Sweden. Parra Escartín, C. (2015). Creation of new TM Segments: Fulfilling translators’ wishes. Proceedings of the RANLP 2015 Natural Language Processing for Translation M ­ emories (NLP4TM) Workshop. Hissar, Bulgaria.

56

Zaretskaya, Pastor and Seghiri

Parra Escartín, C. & Arcedillo, M. (2015). Machine translation evaluation made fuzzier: A study on post-editing productivity and evaluation metrics in commercial settings. Proceedings of the MT Summit XV, Research Track (pp. 131–144), Miami (Florida). Plitt, M. & Masselot, F. (2010). A productivity test of statistical machine translation post-editing in a typical localisation context. The Prague Bulletin of Mathematical Linguistics, 93, 7–16. Starlander, M. & Morado Vázquez, L. (2013). Training translation students to evaluate CAT tools using EAGLES: a case study. Proceedings of the 35th Translating and the Computer Conference, London. ASLIB. TAUS/LISA (2011). Translation Interoperability Survey Retrieved from: http://videolec tures.net/w3cworkshop2011_vandermeer_perspectives/ and http://videolectures .net/site/normal_dl/tag=563177/w3cworkshop2011_vandermeer_perspectives_01 .pdf (consulted on 3/03/2017). Torrez Domínguez, R. (2012). Use of Translation Technologies Survey. Technical report. TradOnline (2011). Translation Industry Survey 2010/2011 – WHAT’S NEW SINCE 2008? Retrieved from http://www.tradonline.fr/medias/docs_tol/translation-survey-2010/ page1.html (consulted on 3/03/2017). Willis, G. (2005). Cognitive interviewing: a tool for improving questionnaire design. ­Thousand Oaks, CA. Zhechev, V. (2014). Analysing the post-editing of machine translation at Autodesk. In O’Brien, S., Balling, L.W., Carl, M., Simard, M. & Specia, L. (Eds.) Post-editing of ­Machine Translation: Processes and Applications (pp. 2–24). Cambridge Scholars Publishing, Newcastle upon Tyne.

chapter 3

Assessing Terminology Management Systems for Interpreters Hernani Costa, Gloria Corpas Pastor and Isabel Durán-Muñoz Abstract This chapter describes and compares current Terminology Management Systems (tms) with a view to establishing a set of features in order to assess the extent to which terminology tools meet the specific needs of interpreters. As in translation, domainspecific terminology becomes a cornerstone in interpreting when consistency and accuracy are at stake. Therefore, an efficient use and management of terminology will enhance interpreting results. As a matter of fact, interpreters have limited time to prepare for new topics and they have to carry out searches and preparation prior to an interpretation and have it accessible during the interpreting service. Fortunately, there are an ever-growing number of applications capable of assisting interpreters before and during an interpretation service, even though they are still few compared to those devoted to translators. Although these tools appear to be quite similar, they provide different kinds of features, which result in different degrees of usefulness, as it can be observed in the last section of this paper.

Keywords interpreter’s needs – interpretation service – interpreting – language technology – terminology management systems – preparation phase

1 Introduction Interpreting can be distinguished from other types of translation processes by its immediacy. Following Pöchhacker (2007/2011: 10), ‘Interpreting is performed here and now for the benefit of people who want to engage in communication across barriers of language and culture’. Currently, there is no universally ­accepted classification of interpreting modes, since authors and interpreting

© koninklijke brill nv, leiden, ���8 | doi 10.1163/9789004351790_005

58

Costa, Corpas and Durán

institutions, such as iti or dg Interpretation at the European Commission, propose their own classifications. However, the most frequent interpreting modes encountered in the literature and offered by company services are based on the three following criteria: timing/delay of relaying the translated message, direction of interpreting and setting/purpose of the interaction. Depending on the timing/delay of relaying the translated message, the main categories of interpreting are simultaneous interpreting and consecutive interpreting. Simultaneous interpreting is defined as a translated message that is given at roughly the same time that the source message is produced. In consecutive interpreting the interpreter waits until the speaker has finished before beginning the interpretation and takes notes in the meantime. Depending on the direction of interpreting, we can distinguish unidirectional interpreting and bi-lateral or bi-directional interpreting. Unidirectional interpreting occurs in situations in which the message is conveyed to a passive audience, and bi-lateral or bi-directional interpreting happens when the interpreter mediates/facilitates communication/dialogue between two parties (also called liaison interpreting). Depending on the setting/purpose of the interaction, we can distinguish: • Conference: Simultaneous interpreting at international conferences and ­formal meetings, with interpreters working in pairs; • Business: Interpreting at smaller or less formal company meetings, factory visits, exhibitions, product launches, government meetings and accompanying delegations etc.; • Police and court: Interpreting for the police and courts, the probation service, solicitors, arbitrations and tribunals etc.; • Community: Interpreting for individuals and organisations such as the nhs, social services in matters of health and welfare, the local government, ­not-for-profit or charitable organisations and at community events. Teleinterpreting (also remote interpreting) is an important modality of interpreting provided by a remote or offsite interpreter via telephone (over the phone interpreting) or via video (video remote interpreting). This is usually done in consecutive mode, but simultaneous interpreting is possible ­depending on the capabilities of the telecommunication technology used. Other modalities are whispered interpreting, sign language interpreting, sight interpreting and others. These modes of interpretation are extremely demanding and, consequently, it is nearly impossible for interpreters to collect the relevant specialised information during the interpretation service itself. They frequently face different

Assessing Terminology Management Systems for Interpreters

59

settings and specialised fields in their interpretation services and yet they always need to provide excellent results. They might be called to work for specialists that share a background knowledge that is totally or partially unknown to laypersons and/or outsiders (Will, 2007). When interpreters lack the necessary background knowledge or experience, they usually need to perform extensive searches for specialised knowledge and terminology in a very efficient way in order to supply this deficit and acquire the required information. In this sense, interpreters are required to find the relevant information for their service prior to interpretation and have it accessible during the process. As it is well known, interpreting is an extremely strenuous task, since it involves a great deal of effort in terms of decoding, memorising and encoding a message (Tripepi Winteringham, 2010: 88). Therefore, interpreters should, as other professionals do, benefit from the development of technology, which will bring about a considerable improvement of their working conditions (Costa et al., 2014b). Where language technologies are concerned, advances have been observed due to the confluence of telecommunications and digital data processing systems in the last decades (Pöchhacker, 2007/2011: 168). However, language technology developments need more systematic research. To date, a limited number of studies have focused on the needs of interpreting technology (­ Moser-Mercer, 1992; Berber, 2008; Braun, 2006; Kalina, 2010), to ­Computer-Assisted Interpreter Training (cait) (Gran et al., 2002; de Manuel Jerez, 2003; Blasco Mayor, 2005; Sandrelli & de Manuel Jerez, 2007) or on Computer-­Assisted Interpreting (cai) tools (Kelly, 2009; Tripepi Winteringham, 2010; Costa et al., 2014a/2014b/2015; Fantinuoli, 2018/forthcoming). ­Although some interpreters have shown some degree of reluctance to use language technologies in their profession (see Berber, 2008), it is clear that cai tools represent an important improvement in the field of interpretation and thus in the multilingual communication context. Nevertheless, the solutions tailored to interpreters’ needs are few and still far behind (Costa et al., 2014a/2014b). In this paper, we aim to shed some light on a specific type of technology targeting interpreters – Terminology Management Systems (tms) – and to carry out a comparative analysis of several of those tools in order to assess their relevance. 2

Interpreter’s Terminology Needs

The potential use of computers to improve interpreters’ working conditions was pointed out by Gile (1987) a long time ago. However, very little progress

60

Costa, Corpas and Durán

has been made so far. As Will (2015: 179) states: ‘the software industry seems not to be interested in this part of the market’. We find some attempts to fill in this existing gap, but they are mainly ‘private projects’ carried out by active interpreters (Will, ibid.) or research studies such as Costa et al. (2014a), who offer a tentative catalogue of current language technologies for interpreters, or Will (ibid.), whose aim is to provide a series of transparent and objective criteria for the use and development of terminology management software for conference interpreters. As a rule, most interpreters seem to be unaware of the opportunities offered by language technologies. As far as terminology is concerned, interpreters continue to store information and terminology on scraps of paper or excel spreadsheets, while the use of technologies and terminology management tools is still very low. A study conducted by Moser-Mercer (1992: 507) rejected the assumption that ‘interpreters’ needs are identical to those of translators and terminologists’ and intended to ‘survey how conference interpreters handle terminology documentation and document control and to offer some guidelines as to the interpretation-specific software tools for terminology and documentation management’. The results of this study include some key findings, such as the conclusion that most of the respondents were interested in exchanging terminological information and that they were open to using computers in their profession. According to these findings, Moser-Mercer (1992) highlighted that ‘software developers targeting the conference interpreting market must provide a tool that meets the specific needs of the interpreters and not just market translation tools’ (ibid: 511). More recent studies have also studied interpreters’ current needs and practices regarding terminology management (Rodríguez & Schnell, 2009; Bilgen, 2009), and they also share the same findings: interpreters require specific tools to meet their needs, which are different from translators and terminologists. According to a survey conducted by Bilgen (2009), 85% of respondents are open to using computers, yet conventional methods still prevail over the use of computerised methods of terminology management. The author observed that respondents had little or no experience with terminology management software, and those with some experience were most dissatisfied with the money and time they had to invest in them, and their overall experience was mediocre (ibid: 66). Respondents indicated that their priorities were different from those identified in terminology literature in terms of terminological information stored, and the way in which term records are structured. This is an important aspect that differentiates the needs of interpreters and translators as regards definitions and contexts (Bilgen, 2009). Due to their working conditions, translators usually prefer to consult multiple definitions and contexts to find the best solution for the translation problem. On the ­contrary, interpreters will rarely have the time to go over multiple definitions, contexts,

Assessing Terminology Management Systems for Interpreters

61

etc. to find the right one, and thus, they will need to store the most concise information to be able to consult it in the quickest and easiest way. Their responses in this survey also showed that the way they retrieve terminological information was context-specific, and that there was also a significant variation among individual interpreters. Flexibility is, therefore, of great importance to interpreters due to the variation of their context-specific terminology management practices, and on their individual preferences regarding the storage, organisation and retrieval of terminological information (ibid: 92). Rodríguez and Schnell (2009), after a thorough analysis of interpreters’ needs and in order to meet their requirements as regards terminology management tools, propose the possibility of developing small databases that vary according to the area of speciality or according to the conference and client. These mini-databases would be multilingual and include an option allowing the interpreter to switch the source and target languages. This assumption is in line with the Function Theory (Bergenholtz & Tarp, 2003; Tarp, 2008) and electronic multifunctional dictionaries (Spohr, 2009), which both defend the need to elaborate terminological entries according to potential users. Rodríguez and Schnell (2009) recognise five features that would distinguish the interpreters’ mini-databases from the terminology databases intended for translators: • • • • •

speed of consultation; intuitive navigation; possibility of updating the terminology record in the interpretation booth; considerable freedom to define the basic structure; multiple ways of filtering data.

Accordingly, they also suggest the abandonment of the usual terminology methodology if the intention is to provide interpreters with specific glossaries tailored to their needs. The authors propose the use of a semasiological and associative methodology instead of the onomasiological approach as the latter would slow down the interpretation process due to the extra cognitive effort required by onomasiological structures. Bearing those features in mind, the next sections will describe and compare several tms developed for or by interpreters to assess the extent to which these terminology tools meet the specific needs of the interpreters. 3

A Brief Survey of tms

It is a well-known fact that terminology work is present in the whole process of preparation prior to an interpretation service. For example, interpreters

62

Costa, Corpas and Durán

b­ ecome familiar with the subject field by searching for specialised documents, by extracting terms and looking for synonyms and hyperonyms, by finding and developing acronyms and abbreviations and by compiling a glossary. According to Rodríguez and Schnell (2009), interpreters tend to compile in-house glossaries tailored to their individual needs as the main way to prepare the terminology of a given interpretation. As previous studies and surveys have shown, this terminology management carried out by interpreters is frequently done manually or with very little help from technology. However, in the last decade a wealth of Terminology Management Systems (tms) that interpreters could use to quickly compile, store, manage and search within glossaries have been developed. They can be typically used to prepare an interpretation, in consecutive interpreting or in a booth. Even though most of these tms have not been specifically developed for interpreters but for translators, there are some that cater for the needs of both translators and interpreters (Durán-Muñoz, 2012; Costa et al. 2016). Due to space constraints, only the tms developed for interpreters that are currently available, together with some other tms that can be useful in their interpreting tasks, are described in detail below, classifying them in three different categories: standalone, webbased and mobile tms. 3.1 Standalone tms Intragloss1 is a commercial Mac os X software created specifically to help interpreters when preparing for an event by allowing them to manage glossaries. This application can be simply defined as a glossary and document ­management tool created to help the interpreter prepare, use and merge different glossaries with preparation documents, in more than 180 different ­languages. It allows them to import and export glossaries to and from plain text, Microsoft Word and Excel formats. Every glossary imported to, or created in, is assigned to a domain glossary (considered the highest level of knowledge), which contains all the glossaries from the sub-areas of knowledge, named ‘assignments’. The creation of an assignment glossary can be done in two different ways: either by extracting automatically all the terms from the domain glossary that appear in the imported documents, or by highlighting a term in the document, searching for it on search sites (such as online glossaries, terminology databases, dictionaries and general Web pages) and manually adding the new translated term to the assignment glossary. It is important to mention that the online search can be made within Intragloss. Another interesting feature is that Intraglosss allows users to copy assignment glossaries 1 http://intragloss.com/.

Assessing Terminology Management Systems for Interpreters

63

and assignment entries from one assignment to another. The domain glossary may be multilingual as it can i­nclude several bilingual assignment glossaries. By way of an example, if there are two assignment glossaries English/French and Dutch/English, in the same domain, the domain glossary will be French/ English/Dutch, i.e. multilingual. Finally, Intragloss also permits to manually add meta-information to each glossary entry. In short, Intragloss is an intuitive and easy-to-use tool that facilitates the interpreters’ terminology management process by producing glossaries (imported or created ad hoc), by searching on several websites simultaneously, by highlighting all the terms in the documents that appear in the domain glossary and by comparing different language versions of a document. However, it is currently platform dependent and only works on Mac os X platforms. InterpretBank2 is a simple terminology and knowledge management software tool designed both for interpreters and translators using Windows and Android. It helps to manage, learn and look up glossaries and term-related ­information. Due to its modular architecture, it can be used to guide the interpreter during the entire workflow process, starting from the creation and management of multilingual glossaries (TermMode), passing through the study of these glossaries (MemoryMode), and finally allowing the interpreter to look up terms while in a booth (ConferenceMode). InterpretBank also has an Android version called InterpretBank Lite. This application is specifically designed to access bi- or trilingual glossaries previously created with the desktop version. It is useful when working as a consecutive, community or liaison interpreter, when a quick look up at the terminology list is necessary. InterpretBank has a user-friendly, intuitive and easy-to-use interface. It allows us to import and export glossaries in different formats (Microsoft Word, Microsoft Excel, simple text files, Android and tmex) and suggests translation candidates by taking advantage of online translation portal services, such as Wikipedia, MyMemory and Bing. However, it is platform-dependent (it only works on Windows and Android), does not handle documents (only glossaries) and requires a commercial license. Interplex ue3 is a user-friendly multilingual glossary management program that can be used easily and quickly in a booth while the interpreter is working. Instead of keeping isolated word lists, it allows the user to group all terms relating to a particular subject or field into multilingual glossaries that can be searched in an instant. This program permits to have several glossaries open at the same time, which is a very useful feature if the working domain is ­covered 2 http://www.interpretbank.com/. 3 http://www.fourwillows.com/.

64

Costa, Corpas and Durán

by more than one glossary. Similar to the previous analysed programs, Interplex ue also enables users to import and export glossaries to and from Microsoft Word, Excel, and simple text files. Interplex ue runs on Windows; nevertheless, it has a simpler version for iOS devices, one named Interplex Lite, for iPhone and iPod Touch, and another named Interplex hd, for iPad. Both glossaries and multi-glossary searchers offer the functionality of viewing expressions in each of the defined languages. In general, Interplex ue has a user-friendly interface and it is regularly updated. It allows import and export of glossaries from and to Microsoft Word and Excel formats. However, it is also platform dependent (only works on ­Windows and iOS), does not handle documents, only glossaries, and requires a commercial license. sdl MultiTerm Desktop4 is a commercial tms developed for Windows that provides one solution to store and manage multilingual terminology. ­MultiTerm was first launched in 1990 by Trados GmbH but in 2005 the company was acquired by sdl,5 which renamed MultiTerm to sdl MultiTerm. Today, sdl MultiTerm is a terminology management tool commercialised by sdl as a standalone application, which has been improved according to translators’ needs. Alternatively, MultiTerm can be used within the sdl Trados Studio6 as an integrated tool. As translators/interpreters can easily edit and add terminology within sdl Trados Studio, MultiTerm helps to improve the efficiency of the translation process and promotes high-quality translated content with real-time verification of multilingual terminology. This application is very comprehensive because it allows storage of an unlimited number of terms in a vast number of languages; imports and exports glossaries from and to different technology environments, such as Microsoft Excel, xml, tbx and several other proprietary formats; it permits users to manually add a variety of meta-data information, such as synonyms, context, definitions, associated project, part-of-speech tags, urls, etc. Apart from the previous mentioned descriptive fields, MultiTerm also allows the user to insert illustrations for the terms in the terminology database (which can be stored either locally or, for collaborative purposes, in a remote server). This visual reference feature is very useful especially to interpreters and translators dealing with unfamiliar terms. Moreover, MultiTerm has an advanced search feature that permits the user to search not only the indexed terms but also in their descriptive fields, or create filters to make custom searches within specific fields, like language, definition, 4 http://www.sdl.com/cxc/language/terminology-management/multiterm/. 5 http://www.sdl.com. 6 http://www.sdl.com/products/sdl-trados-studio.

Assessing Terminology Management Systems for Interpreters

65

part-of-speech, etc. Nevertheless, the most interesting feature about MultiTerm is its concept-oriented feature, i.e. each entry in MultiTerm corresponds to a single concept, which can be described by different terms in both source and target language. This detail is very important because it allows the user to centralise and customise the terms with more information, such as different possible translations and their corresponding contexts. In general, MultiTerm can be seen as an advanced multilingual tms with an intuitive and easy-to-use interface. Although MultiTerm was originally designed for translators, it can also be used by interpreters. Its main advantage to interpreters, when compared with other terminology tools, is twofold: it allows users to add several translation terms in one entry and enables them to customise a wide variety of descriptive fields, such as illustrations, associated projects, definitions, etc. However, it can only be used on Windows, it does not handle documents and there is no demo version available. AnyLexic7 is an easy-to-use tms developed for Windows with a simple and intuitive interface. It was not designed for any particular terminological requirement, instead it aims to help the interpreter prepare, use and manage different glossaries or dictionaries. AnyLexic can be described as a robust terminology management tool, as it enables users to easily create and manage multiple mono-, bi- or multilingual glossaries in any language and to import and export glossaries to and from Microsoft Excel, plain text and A ­ nyLexic ­Exchange Format (aef). In addition, each entry in the glossary can have multiple translation equivalents in the target language along with notes. The search for records in the database allows users to combine different options, such as search for all source terms or translation candidates and associated notes. In addition, the search can be performed within one or multiple glossaries. ­Another interesting feature in AnyLexic is the way that records can be displayed using different templates with configurable text colour, background colour, font size and text format. Besides, it is possible to customise the template for displaying the records. With the purpose of simplifying the teamwork process, this tool has an additional option to exchange any glossary with other AnyLexic users by either using the aef proprietary format or by accessing a remote glossary, a very useful feature for collaborative interpreting and/or translation projects. In general, AnyLexic is an easy and convenient terminology database managing software for working with terminology, creating, editing and exchanging glossaries. However, it only works on Windows platforms and even though an evaluation version is available for 30 days, it requires a commercial license. 7 https://anylexic.com/.

66

Costa, Corpas and Durán

Lingo8 is a commercial Windows terminology management tool designed to create and manage terminology databases, whether mono- or multilingual. It can import from and export to tmx and plain text. Its main features are: the creation and management of any number of specialised glossaries/dictionaries in any language; it can handle large files (i.e. over 50K entries); it allows users to have several glossaries open at the same time; and it has a rapid and easily configurable search functionality that can be customised to search for all terms, translation candidates and associated descriptive fields, either in all glossaries or in a specific one. Another interesting feature is the drag and drop functionality, which enables users to easily insert words into Microsoft documents, for instance. Lingo is a simple and user-friendly software that offers an effective way to create and manage multilingual glossaries in any language. Additionally, it permits users to manually add an infinite number of customised fields into each entry, such as definitions, urls, synonyms, antonyms, contextual information, notes or any other desirable field. However, it is platform dependent and does not import from or export to common formats like Microsoft Word or Excel. UniLex9 is a free terminology management tool created by Acolada GmbH for Windows. It aims to help interpreters and translators prepare, use and manage bilingual glossaries or dictionaries in approximately 30 different languages. UniLex offers a variety of search functions and the possibility to combine user glossaries or dictionaries with a full range of dictionaries available in the UniLex series (e.g. Blaha: Pocket Dictionary of Automobile Technology German/ English), which can be acquired as single user versions or as network versions for collaborative purposes. UniLex can also be used in a network environment, which allows users to exchange glossaries or dictionaries. Nevertheless, this additional feature requires a commercial license. In general, UniLex is not only capable of managing user bilingual glossaries or dictionaries, but also dictionary titles from renowned publishers, which are sold by the company to be consulted within UniLex. However, it only works on Windows and does not handle multilingual glossaries. TermX10 is a simple and easy-to-use commercial tms created by Translex Publishing for Windows. Apart from the usual functionalities that tms offer (such as add, view, search, edit and remove terminology), TermX permits users to add contextual information (relating to the use of the term in a specific context), source information (how and where the term was collected) and up 8 9 10

https://www.lexicool.com/soft_lingo2.asp. http://www.acolada.de/unilex.htm. http://www.translex.co.uk/software.html.

Assessing Terminology Management Systems for Interpreters

67

to 6 translation equivalents for each individual source term entry. Similar to Intragloss, this tool also allows the user to associate a term to a domain, which then can be used as a filter to search for terminology in a specific sub-area of knowledge. TermX provides a native format for the management and exchange of terminology, as well as import and export capabilities in the most widely used storage formats, like csv (Comma Separated Values), plain text, ms Excel, xml, rtf, html, MultiTerm and pdf. In short, TermX aims to help interpreters and translators prepare, use and maintain multilingual glossaries in any language outside of the Computer-­ assisted Translation (cat) environment whilst making all data readily a­ vailable for import and use in the cat environment when needed. Terminus11 is a commercial tms designed by interpreters for interpreters working on Windows. It enables them to organise multilingual terminology (up to 5 languages per glossary) into different subjects. Terminus associates terminology with one or more subjects (i.e. domains). Each term has one main subject and as many additional subjects as the user needs. These descriptors are important as they allow users to search and export specific terminology from these pre-defined subjects. Moreover, when searching for terminology, they can be used to limit the search (e.g. display all the terms stored in a particular subject). Especially on extremely large databases, this may reduce the number of terms that match the search criteria. Terminus is an easy-to-use flexible tool as it enables the users to classify terms into different subjects, to import terminology lists from plain text and ms Excel files, to export results alphabetically or grouped together by the main subject and sorted within each subject to plain text, rtf and pdf. Another interesting feature is the way that records can be displayed by using different colours for different languages. Tables  3.1–3.2 provide a comparative summary of the main features that characterise the tms described above. Overall punctuations have been assigned for relevance and wealth of functionalities. 3.2 Web-based tms Interpreters’ Help12 is a powerful and free tms designed not only to manage multilingual glossaries but also to manage job assignments and clients. Assignments can be created for both personal and community usage, the last one permits to share assignments privately with other Interpreters’ Help

11 12

http://www.wintringham.ch/cgi/ayawp.pl?T=terminus. https://interpretershelp.com/.

68 Table 3.1

Costa, Corpas and Durán Comparing standalone tms: Intragloss, InterpretBank, Intraplex, sdl MultiTerm and AnyLexic (Part 1/2).

Feature

Intragloss 1 (2014)

InterpretBank Intraplex sdl Multi AnyLexic 4 3.102 (2014) 2.1.1.47 (2012) Term 2014 (2013) (2011)

Manages multiple glossaries (no = 0; yes = 10) No. of possible working languages (≤100 = 4; >100 = 7; unlimited = 10) No. of languages per glossary allowed (≤3 = 5; ≥4 = 10) No. of descriptive fields (non = 0; 1 = 3; [2–5] = 7; >5 = 10) Handles documents (no = 0; yes = 10)

yes (10)

yes (10)

yes (10)

180 (7)

35 (4)

unlimited (10) unlimited (10)

unlimited (10)

2 (5)

2 (5)

unlimited (10) unlimited (10)

unlimited (10)

4 (7)

4 (7)

non (0)

>5 (10)

1 (3)

no (0)

no (0)

no (0)

no (0)

yes (5)

yes (5)

yes (5)

yes (5)

ms Word, Excel and Plain Text (3) ms Word, Excel and Plain Text (3)

ms Word, Excel and other cat formats (5) ms Word, Excel and other cat formats (5)

Excel, Plain Text and aef (3) Excel, Plain Text and aef (3)

yes (5)

ms Word, Excel, tmex and Plain Text (4) ms Word, Excel, tmex, Android and Plain Text (4) yes (5)

no (0)

no (0)

no (0)

English (1)

English (1)

English (1)

English + 5 other English + 10 languages (5) other languages (5)

yes (pdf, ms Word, Pages and Keynote files) (10) Unicode compatibility yes (5) (no = 0; yes = 5) Imports from (1 = 1; ms Word, Excel 2 = 2; 3 = 3; [4–5] = 4; and Plain >5 = 5) Text (3) Exports to (non = 0; ms Word and 1 = 1; 2 = 2; 3 = 3; Excel (2) [4–5] = 4; >5 = 5) Embedded online search for translation candidates (no = 0; yes = 5) Interface’s supported languages (1 = 1; [2–5] = 3; >5 = 5)

yes (10)

yes (10)

69

Assessing Terminology Management Systems for Interpreters Feature

Intragloss 1 (2014)

InterpretBank Intraplex sdl Multi AnyLexic 4 3.102 (2014) 2.1.1.47 (2012) Term 2014 (2013) (2011)

Remote glossary exchange (no = 0; yes = 5) Well-documented (no = 0; yes = 5) Availability (proprietary without demo = 1; proprietary with demo = 3; free = 5) Operating System(s) (1 = 1; 2 = 3; ≥3 = 5) Other relevant features (subjective analysis = max. 5)

no (0)

no (0)

no (0)

yes (5)

yes (5)

yes (5)

yes (5)

yes (5)

yes (5)

yes (5)

Final Mark

proprietary proprietary without demo with demo (3) (1)

proprietary proprietary with- proprietary with demo (3) out demo (1) with demo (3)

Mac os X (1)

Windows and Android (3) allows to high- the Memorylight terms in Mode helps to the documents memorise bilingual and merge a glossary with glossaries (4) a document making it annotated to be printed (5)

Windows and iOS (3) permits to have several glossaries open at the same time (2)

67

55

60

Windows (1)

Windows (1)

it is a concept oriented-tool and permits to add illustrations into each entry (5)

allows to share within a group of AnyLexic users (1)

77

64

Table 3.2 Comparing standalone tms: Lingo, UniLex TermX and Terminus (Part 2/2). Feature

Lingo 4 (2011)

UniLex 0.9 (2007)

TermX (2013)

Terminus 3.1 (2009)

Manages multiple glossaries (no = 0; yes = 10) No. of possible working languages (≤100 = 4; >100 = 7; unlimited = 10) No. of languages per glossary allowed (≤3 = 5; ≥4 = 10) No. of descriptive fields (non = 0; 1 = 3; [2–5] = 7; >5 = 10) Handles documents (no = 0; yes = 10)

yes (10)

no (0)

yes (10)

yes (10)

unlimited (10) 30 (4)

unlimited (10) unlimited (10)

unlimited (10) 2 (5)

6 (10)

5 (10)

>5 (10)

2 (7)

>5 (10)

2 (7)

no (0)

no (0)

no (0)

no (0)

70

Costa, Corpas and Durán

Table 3.2 Comparing standalone tms: Lingo, UniLex TermX and Terminus (Part 2/2). (cont.) Feature

Lingo 4 (2011)

UniLex 0.9 (2007)

Unicode compatibility (no = 0; yes = 5) yes (5) no (0) Imports from (1 = 1; 2 = 2; 3 = 3; tmx and Plain Plain Text (1) [4–5] = 4; >5 = 5) Text (2)

TermX (2013)

Terminus 3.1 (2009) yes (5) Excel and Plain Text (2)

Embedded online search for translation candidates (no = 0; yes = 5) Interface’s supported languages (1 = 1; [2–5] = 3; >5 = 5) Remote glossary exchange (no = 0; yes = 5) Well-documented (no = 0; yes = 5) Availability (proprietary without demo = 1; proprietary with demo = 3; free = 5) Operating System(s) (1 = 1; 2 = 3; ≥3 = 5) Other relevant features (subjective analysis = max. 5)

no (0)

no (0)

yes (5) ms Word, Excel and other cat formats (5) ms Word, Excel and other cat formats (5) no (0)

English (1)

English + 3 (3)

English (1)

English (1)

no (0)

no (0)

no (0)

no (0)

yes (5) proprietary without demo (1) Windows (1)

yes (5) proprietary with demo (3)

permits to add – an unlimited number of descriptive fields (5)

availability to import and export from and to cat tools (5)

demo version only limits the number of entries (1)

Final Mark

64

68

56

Exports to (non = 0; 1 = 1; 2 = 2; 3 = 3; tmx and Plain Plain Text (1) [4–5] = 4; >5 = 5) Text (2)

yes (5) no (0) proprietary free (5) with demo (3) Windows (1)

Windows (1)

27

rtf, pdf and Plain Text (3)

no (0)

Windows (1)

­ embers. When sharing an assignment with team members, they can comm ment on it, view assignment details, view and edit glossaries related to the assignment, download assignments’ files and view the assignment’s client page. ­Interpreters’ Help also has the option to make a glossary publicly available to the Interpreters’ Help community. Apart from that, this tool keeps a history of all the assignments, it allows users to easily find assignments by client and

Assessing Terminology Management Systems for Interpreters

71

material that was used for a previous assignment, upload assignment files and attach glossaries. Moreover, it permits them to create, edit, search and view glossaries; to add, view, remove and edit entries; to add an unlimited number of translation terms; to add a variety of contextual information (such as comment, category, definition, acronym, amongst others); to easily move or remove columns; to add a glossary to the Favourites group; to add to and remove tags from a glossary; to export a glossary to Excel or pdf; to import from ms Word, Excel, Libreoffice/Openoffice and csv; to view a printable version of a glossary, and to copy glossaries, either duplicate our glossaries or copy a public one to our account. Interpreters’ Help also has a Mac os version called Boothmate, which permits the user to access glossaries offline. This standalone version synchronises with the website and can be used in the booth even without an Internet connection. It is important to mention that BoothMate only allows users to search for terminology, not to edit term entries or glossaries. Interpreters’ Help can be considered one of the most complete tms freely available on the market. Both versions were designed to be a companion tool not only for users who need to search for glossaries in the booth, but also to those who are looking for a user-friendly and straightforward terminology management tool. Examples of the most innovative and consequently more expensive webbased terminology management solutions on the market today are WebTerm,13 Acrolinx,14 Termflow15 and flashterm.16 Apart from the basic options offered by the aforementioned web-based tms (e.g. create, edit, view, remove and group terms into domains; add contextual information to each entry; import from and export to e.g. plain text or csv; manage multilingual glossaries; and, share glossaries with a group of users), these tools offer more sophisticated features, such as: • Extract multilingual terminology from translation memories, pdf, xml, etc. (e.g. Acrolinx and Termflow); • Import and export terminology in industry-standard exchange formats (e.g. olif, xml, mtf, tbx, tmx, martif, csv, sdl’s MultiTerm format) (e.g. Acrolinx, WebTerm and Termflow); • Advise whether a translation term is preferred or prohibited in a specific domain (e.g. Acrolinx, Termflow and flasterm); 13 14 15 16

https://www.star-group.net/en/products/web-based-terminology.html. https://www.acrolinx.com/platform-services/terminology-management/. http://www.termflow.de/. https://www.flashterm.eu/home.

72

Costa, Corpas and Durán

• Integrate a reference database to store client instructions, internal procedures, employee contact information and other useful information to the interpretation or translation service (e.g. LogiTerm and WebTerm) • Administrative control in which the project manager can select the fields that are displayed, which functions can be used and which settings can be changed (e.g. WebTerm, Acrolinx, Termflow, and flasterm). Bearing this in mind, these tools can be considered more sophisticated than the standalone tms previously mentioned since they include more advanced features and offer professional support, as they were specially designed for commercial purposes. Although they were not built to help interpreters d­ uring the interpretation process, they can be extremely useful before the interpreting service as they allow them to store and share terminology more easily, especially for companies who have a considerable number of employers or for freelance interpreters in a collaborative environment. Due to space constraints, only some of the available web-based tms on the market can be mentioned. Nevertheless, there are some tms worth to ­mention, such as: • AcrossTerm:17 a centralised tms for the entire company terminology; • i-Term:18 a state-of-the-art terminology and knowledge management tool, which allows users to store, structure and search online for knowledge about concepts; • Multitrans Prism:19 an innovative client-server software solution that integrates project and business management, translation memory, and terminology management; • TermWiki:20 a seamless collaborative tms that aims to collect every term in every subject in the world and make it available in every language. It permits users to search for translation candidates. Table  3.3 provides a comparative summary of the main features that characterise the web-based tms mentioned above. As in previous cases, overall punctuations have also been offered to serve as a quick guide or checklist for interpreters.

17 18 19 20

https://www.across.net/en/. http://www.iterm.dk/. http://linguistech.ca/MultiTrans_EN. http://es.termwiki.com/.

Assessing Terminology Management Systems for Interpreters

73

3.3 Mobile tms Mobile terminology applications (or tms apps) are undoubtedly the next step in this ever-evolving domain of term management. tms apps are systems that have been developed or optimised for small handheld devices, such as mobile phones, smartphones, iPads or pdas, among others. Some of the most popular ones are Glossary Assistant and The Interpreter’s Wizard. Glossary Assistant21 is a user-friendly multilingual glossary management application created by a professional team of interpreters for Android devices. Specially designed for simultaneous/consecutive interpreting, Glossary Assistant provides a comfortable viewing of glossaries on Android-tablets (limited on smartphones). It enables users to create, remove and manage multilingual glossaries (glossaries can be maintained for up to 10 languages); to add, edit and remove entries to/from a glossary; to search for terms either in a specific language or in all languages, and to re-arrange and sort columns by language and alphabetically, respectively. The glossaries can be imported and exported to/from Unicode plain text files. In order to import glossaries from third-party applications, Glossary Assistant only requires those glossaries to be stored as a tab delimited Unicode text file. Glossary Assistant has a free (4 glossaries maximum, with a maximum of 250 rows per glossary) and a commercial version (which does not have restrictions). A pc version is also available for free without restrictions. In both versions, pc and Android, all the glossaries are internally stored in a database and they can be exchanged between them. The Interpreter’s Wizard22 is a free iPad application capable of managing bilingual glossaries in a booth. It is a simple, fast and easy-to-use application that helps the interpreter to search and visualise terminology in seconds. The system includes a rapid and easily configurable search functionality that can be customised to search for all terms, translation candidates either in all glossaries or in a specific one. Nevertheless, all the imported glossaries need to be previously created and converted online to the proper format, and it does not allow users to export glossaries. Table 3.4 provides a comparative summary of the main features that characterise these two mobile tms. As in previous cases, overall punctuations have also been offered to serve as a quick guide or checklist for interpreters. 3.4 Comparative Analysis Although the aforementioned Terminology Management Systems (tms) can be used to prepare a given interpretation of any kind according to the 21 22

http://swiss32.com. http://download.cnet.com/The-Interpreter-s-Wizard/3000-2124_4-75560236.html.

csv and tbx (2) no (0)

yes (5) >5 (5) >5 (5) no (0)

unlimited (10) >5 (10) no (0) yes (10)

unlimited (10) unlimited (10) no (0) yes (10) yes (5) yes (5) ms Words, Excel, Open/ >5 (5) Libreoffice and csv (4) Excel and pdf (2) >5 (5) no (0)

no (0)

yes (5) >5 (5)

no (0) yes (10)

unlimited (10)

3 (7)

unlimited (10)

unlimited (10)

yes (10)

flashTerm (2015)

yes (10) yes (10)

>5 (10)

>4 (10)

no (0)

>5 (10)

yes (5) –

no (0) yes (10)

>5 (10)

>4 (10)

unlimited (10) >100 (7)

yes (10)

unlimited (10)

yes (10)

yes (10)

Termflow (2013)

yes (10)

Acrolinx (2014)

Manages multiple glossaries (no = 0; yes = 10) N° of possible working languages (≤100 = 4; >100 = 7; unlimited = 10) N° of languages per glossary allowed (≤3 = 5; ≥4 = 10) N° of descriptive fields (non = 0; 1 = 3; [2–5] = 7; >5 = 10) Handles documents (no = 0; yes = 10) Remote glossary exchange (no = 0; yes = 10) Unicode compatibility (no = 0; yes = 5) Imports from (1 = 1; 2 = 2; 3 = 3; [4–5] = 4; >5 = 5) Exports to (non = 0; 1 = 1; 2 = 2; 3 = 3; [4–5] = 4; >5 = 5) Embedded online search for translation candidates (no = 0; yes = 5)

WebTerm 6 (2014)

Interpreters’ Help beta (2014)

Feature

Table 3.3 Comparing web-based tms: Interpreters’ Help, WebTerm, Acrolinx, Termflow and flashterm.

74 Costa, Corpas and Durán

Final Mark

Interface’s supported languages (1 = 1; [2–5] = 3; >5 = 5) Well-documented (no = 0; yes = 5) Availability (proprietary without demo = 1; proprietary with demo = 3; free = 5) Other relevant features (subjective analysis = max. 5) – clean and straightforward tms that allows users to add an unlimited number of translation terms (5) 77 76 74

simple interface and allows users to add illustrations (3)

yes (5) yes (5) proprietary with- proprietary with out demo (1) demo (3)

yes (5) free (5)

English (1)

>5 (5)

English (1)

75

English and Deutsch (2) no (0) proprietary without demo (1) –

78

clean interface and allows users to add illustrations (3)

English and Deutsch (2) yes (10) proprietary with demo (3)

Assessing Terminology Management Systems for Interpreters

75

76

Costa, Corpas and Durán

i­nterpreters’ requirements identified in Section  2, these systems differ from one another in their functionalities, practical issues, degrees of user-friendliness and target audience (i.e. individual or enterprise usage). Therefore, it is necessary to establish a set of specific and measurable features that permit us to assess and distinguish the different tools concerning individuals’ and companies’ needs in such a way that the results would be useful for both potential customers as well as to the designers of such systems. Departing from the conclusions drawn from the literature review (see Sections 1 and 2) and the description of the terminology tools analysed in Sections 3.1, 3.2 and 3.3 (standalone, web-based and mobile tms, respectively), this section provides an extensive analysis of these tms based on our own practical set of measurable features. For instance, the ‘freedom to define the basic structure’ identified by Rodríguez and Schnell (2009) was reformulated into several practical measurable features, such as ‘No. of descriptive fields’, ‘No. of working languages’ and ‘No. of languages per glossary’. Moreover, the possibility of ‘developing multilingual mini-­databases’, also identified in their study, was reconsidered as measurable features by means of the following criteria: ‘Manages multiple glossaries’ and ‘No. of l­ anguages per glossary’. Another example is the ‘Remote Glossary ­Exchange’ measurable feature, which was inferred from the study conducted by Bilgen (2009), who identified the need to exchange terminological information. After a careful analysis of the priorities for the design and features to be included in a terminology management tool reported in Moser-Mercer (1992); Bergenholtz and Tarp, 2003; Tarp, 2008; Spohr, 2009; Rodríguez and Schnell (2009); Bilgen (2009) – see Section 2 for more details, we identified 15 main ­features. Although some of them are pointed out as fundamental due to their extreme importance when assisting interpreters before and during an interpretation service, others are mostly related with the tools’ design and surrounding. In an attempt to standardize these 15 features into a discriminative 0–100 scoring system, we used the EAGLES framework for the evaluation of nlp (Natural Language Processing) systems as a reference to divide these features into two categories: fundamental and secondary. The EAGLES (1996) report includes formalisms of evaluation procedures for various types of systems according to their general quality characteristics and their definitions: functionality, reliability, usability, efficiency, maintainability and portability. It is important to mention that the EAGLES methodology used as a starting point the iso 9126 standard for software quality (iso/iec, 1991). Although all the reported characteristics are important to any software, in this work our main focus is on the functionality of the software. Thus, we considered fundamental all the features related to the software’s functionality (‘A set of attributes that

Assessing Terminology Management Systems for Interpreters

77

Table 3.4 Comparing mobile tms: Glossary Assistant and The Interpreter’s Wizard.

Feature

Glossary Assistant 1.2 (2015)

The Interpreter’s Wizard 2.0 (2011)

Manages multiple glossaries (no = 0; yes = 10) No. of possible working languages (≤100 = 4; >100 = 7; unlimited = 10) No. of languages per glossary allowed (≤3 = 5; ≥4 = 10) No. of descriptive fields (non = 0; 1 = 3; [2–5] = 7; >5 = 10) Handles documents (no = 0; yes = 10) Unicode compatibility (no = 0; yes = 5) Imports from (1 = 1; 2 = 2; 3 = 3; [4–5] = 4; >5 = 5) Exports to (non = 0; 1 = 1; 2 = 2; 3 = 3; [4–5] = 4; >5 = 5) Embedded online search for translation candidates (no = 0; yes = 5) Interface’s supported languages (1 = 1; [2–5] = 3; >5 = 5) Remote glossary exchange (no = 0; yes = 5) Well-documented (no = 0; yes = 5) Availability (proprietary without demo = 1; proprietary with demo = 3; free = 5) Operating System(s) (1 = 1; 2 = 3; ≥3 = 5) Other relevant features (subjective analysis = max. 5)

yes (10)

yes (10)

unlimited (10)

unlimited (10)

10 (10)

2 (5)

non (0)

non (0)

no (0) yes (5)

no (0) yes (5)

Plain Text (1)

Proprietary Format (1)

Plain Text (1)

non (0)

no (0)

no (0)

English (1)

English (1)

no (0)

no (0)

yes (5) proprietary with demo (3)

no (0) free (5)

Final Mark

53

Android and WiniOS (iPad) (1) dows (3) user-friendly and quick performance (1) intuitive interface (4) 39

78

Costa, Corpas and Durán

bear on the existence of a set of functions and their specified properties. The functions are those that satisfy stated or implied needs’), such as the management of multiple glossaries, the number of possible working languages permitted by the tool, how many of these languages can be used at the same time per glossary, the number of descriptive fields allowed per glossary entry and the possibility of managing terminology with preparation documents. The remaining 10 features, which are related to reliability, usability, efficiency, maintainability and portability were categorised as secondary. In detail, the features classified as fundamental to a terminology tool were given 10 points and 5 points to the secondary ones – except for web-based tms, in which we removed one feature and considered 6 as fundamental and 8 as secondary. Then, these features were used to evaluate the seventeen tools (9 standalone, 6 web-based and 2 mobile) presented in Sections 3.1, 3.2 and 3.3 and to assess which one is the most complete, both considering each sub-group separately and all the tools together. The first feature clarifies whether the tools were designed to handle multiple glossaries in their interfaces at the same time (Manages multiple g­ lossaries). The next two features are somehow related. The No. of possible working languages describes how many different working languages are permitted by the application. Then, considering these working languages, how many of them can be used at the same time per glossary (No. of languages per glossary allowed). The next feature is related to all types of descriptive fields that these tools allow users to add to each glossary entry (N° of descriptive fields). The possibility of managing terminology with preparation documents (Handles documents) is another relevant feature for interpreters seeking tools capable of highlighting terms in documents, for example. Equally, import is the ­Unicode support (Unicode compatibility) as it provides a unique number for every character, no matter what the platform, the program, or the language is. In other words, an application that supports full Unicode means that it has support for any ascii or non-ascii language, such as Hebrew or Russian, two non-ascii languages. Imports from and Exports to, as its name suggests, represents the supported input and output formats. The Embedded online search for translation candidates is a relevant add-in for terminology tools, as it ­permits users to focus the search for terminological candidates within the tool. Despite the fact that all the tools have English as a default language, the support of multiple languages (Interface’s supported languages) is another important feature as it would definitely increase the number of potential users that a terminology tool can reach. The Remote glossary exchange feature is important when cooperating with other working partners remotely is required, as in ­collaborative interpreting and crowd-sourcing. The next three features are related to the

Assessing Terminology Management Systems for Interpreters

79

available documentation, their availability and platform dependency (Well-­ documented, Availability and Operating System(s), respectively). Finally, the last row presents some unique characteristics along with some relevant comments (Other relevant features). Based on this comparative analysis, none of the investigated terminology tools exhibit all the desirable features. Nevertheless, sdl MultiTerm, TermX and Intragloss are the best-classified standalone tms with 77, 68 and 67 points out of 100, respectively (see Table 3.1–3.2). This is not surprising because sdl MultiTerm is the most expensive standalone tool nowadays available on the market and, apart from that, it has been in development for more than 20 years. Also developed for commercial purposes, TermX was created by a team of professionals focused on linguistic services, such as translation and terminology management. The score of Intragloss, released in 2014 as a stable version, is also not surprising due to its novelty and design purposes, i.e. it was specifically developed by interpreters for interpreters and, thus, it is entirely tailored to their needs. All three offer a user-friendly interface to easily store, manage and search for multilingual terminology and definitions. On the other hand, UniLex, Intraplex and Terminus got the worst scores due to the lack of features offered (27, 55 and 56, respectively). Regarding the remaining tools (AnyLexic, Lingo, InterpretBank), they have similar features, which resulted in similar scores (64, 64 and 60, respectively). It is worth mentioning that the three best-classified tools were released between 2013 and 2014 and those that got lower scores were released between 2007 and 2012, which means that recent standalone tms are better designed to assist interpreters. Sharing terminology is extremely important because it allows users to improve glossaries, making them more uniform, complete and correct across ­subjects and domains, which can only be accomplished collaboratively. Moreover, sharing terminology is the only way to collect most terms in most subjects in the world and make this knowledge available in every language. Bearing this in mind, web-based tms take advantage of cutting-edge technologies to fulfil the need for sharing terminology. As we can see in Table 3.3, all the 6 web-based tms analysed got similar scores, ranging from 74 (Acrolinx) to 78 ­(flahterm). This means that they have similar features and should be investigated in more detail by those who are looking for commercial web-based tms, especially the prices and the technical support provided. It is also important to note that all these tools have the release date between 2013 and 2015. Apart from the common options offered by traditional tms, these web-based systems also integrate a Content Management System (cms). In other words, web-based tms, not only offer a set of features to manage terminology in a collaborative

80

Costa, Corpas and Durán

environment, but also provide procedures to manage the entire workflow, i.e. a cms that allows users to manage reference databases to store client instructions, internal procedures, employee contact information, amongst other additional information related to the interpretation or translation service. Despite the fact that mobile tms do not reach acceptable scores when compared with standalone and web-based tms (Glossary Assistant: 53; The Interpreter’s Wizard: 39 – see Table 3.4) nor they offer the necessary comfort to manage terminology, they still play an important role when a quick search for terminology is required, e.g. while in a booth. To sum up, web-based programs obtained higher average scores compared with standalone and mobile tms (76, 60 and 46, respectively). These results can be explained by the companies’ effort and the cutting-edge technology used during their development. Another fact that contributes to the increasing interest in web-based tms is that nowadays companies are more orientated towards developing centralised systems in order to provide uniform services to both staff and clients. Nevertheless, this effort requires higher investment in equipment and manpower to maintain these systems and consequently make them more expensive compared with standalone or mobile tms. The only exception is the Interpreters’ Help tool (still in beta). 4 Conclusion This paper presents a comprehensive and up-to-date review of the currently available tms on the market as well as an overview of the most relevant features that these tools should have in order to help interpreters before and during the interpretation process. Seventeen terminology tools have been described and compared with the aim of assessing them on the basis of a set of 15 features previously identified and a scoring system. This comparative analysis aims at highlighting some of the features that interpreters can expect from the terminology management tools currently available on the market. In addition, the results obtained could guide interpreters when choosing specific tools for a given interpretation project, i.e. the tms(s) that would best cater for their specific needs, in order to help them work more efficiently, store and share terminology more easily, as well as save time when looking for a specific feature most suited to a specific interpreting service. Sharing terminology is extremely important because it allows users to improve terminology by enhancing term coverage and consistency within and throughout domains in a collaborative fashion. Although most of the analysed

Assessing Terminology Management Systems for Interpreters

81

tms could be considered very flexible when searching for terminology within glossaries and can help interpreters carry out their terminology management, it appears that none of them can fulfil all interpreters’ needs. It is worth mentioning that some tools require a steep learning curve (e.g. Lingo) while others require a significant financial investment (e.g. sdl MultiTerm, WebTerm and flashterm). Moreover, some tools are fairly basic and more orientated towards creating and managing bilingual or multilingual glossaries rather than more comprehensive terminology records with supporting information (e.g. UniLex and The Interpreter’s Wizard). Our main findings suggest that most tms are not envisaged to be used by interpreters. Therefore, tms do not fulfil completely the needs of this group of end-users as regards speed of consultation, intuitive navigation, possibility of updating the terminology record in the interpretation booth, freedom to define the basic structure, multiple ways of filtering data and sharing information, etc. Conversely, those tools devoted to interpreters (and mainly developed by interpreters) are fairly basic and only include a limited number of features. Another important observation is that the most comprehensive, userfriendly and successfully evaluated systems are standalone tms, which are also greater in number (if considering purely tms). This fact reinforces the idea that most tms are not addressed to interpreters as their final users, but rather to translators. Interpreters need the information and terminology gathered during the preparation phase in the interpretation service and it is not always possible to use standalone versions. On the other hand, web-based tms are more recent and have been created with cutting-edge technology, which may result in standalone tms losing the race to web-based tms in the short run. Interestingly enough, mobile tms are not performing as well as the others. This seems to be in contradiction to the extensive usage of apps and the requirement of accessing websites from a number of different devices these days. Mobile tms should be seen as a portable interface/middleware to a web-based or standalone tms, especially suitable for quick terminology searches, although it should also be acknowledged that mobile tms are still far away from offering the same degree of comfort to manage terminology and/or web-based content. Given that quality terminology management is a top priority for interpreters, there seems to be a pressing need to design terminology management tools tailored specifically to assist interpreters both prior to and during their interpreting services. In this vein, it would be necessary to ascertain interpreters’ terminology needs (as opposed to translators’), and then, devote more efforts to the development of web-based and, particularly, mobile tms in order to

82

Costa, Corpas and Durán

provide on-site consultation of glossaries, terminologies, lists of proper names and conversion figures, etc. No doubt, technology-assisted interpreting will offer a challenging and fruitful research niche for many years to come. We are just at the beginning of this long and winding road… Acknowledgements Hernani Costa is supported by the People Programme (Marie Curie Actions) of the European Union’s Framework Programme (FP7/2007–2013) under rea grant agreement n. 317471. The research reported in this work has also been partially carried out in the framework of the R&D Project for Excellence TERMITUR (ref. n. HUM2754, 2014–2017), the vip: Voice-text integrated system for InterPreters (ref. n. FFI2016-75831, 2017–2020), and the INTERPRETA 2.0 (ref. n. PIE17-015, 2017–2019). References Berber, D. (2008). ICT (Information and Communication Technologies) in Conference Interpreting: a survey of their usage in professional and educational settings. ­C ETRA. Retrieved from http://isg.urv.es/cetra/article_CETRA2007_berber.pdf (Consulted on 3/03/2017). Bergenholtz, H. and Tarp, S. (2003). Two opposing theories: On H.E. Wiegand’s recent discovery of lexicographic functions. Hermes. Journal of Linguistics, 31, pp. 171–196. Bilgen, B. (2009). Investigating Terminology Management for Conference Interpreters. MA dissertation, University of Ottawa, Ottawa, Canada. Blasco Mayor, M.J. (2005). El reto de formar intérpretes en el siglo XXI. Translation Journal, 9 (1). Retrieved from http://accurapid.com/journal/31interprete2.htm (Consulted on 3/03/2017). Braun, S. (2006). Multimedia communication technologies and their impact on interpreting, in Carroll M., Gerzymisch-Arbogast H. and Nauert S. (eds.). Audiovisual Translation Scenarios. Proceedings of the Marie Curie Euroconferences MuTra: Audiovisual Translation Scenarios. Copenhagen, Denmark. Retrieved from http://www .euroconferences.info/proceedings/2006_Proceedings/2006_proceedings.html (Consulted on 3/03/2017). Costa, H., Zaretskaya, A., Corpas Pastor, G. and Seghiri, Miriam. (2016). Terminology Extraction Tools: Are they Useful for Translators? MultiLingual. April/May.

Assessing Terminology Management Systems for Interpreters

83

Costa, H., Corpas Pastor, G. and Durán Muñoz, I. (2015). An Interpreters’ Guide to ­Selecting Terminology Management Tools. NATO Conference on Terminology Management. Brussels, Belgium. November. Costa, H., Corpas Pastor, G. and Durán-Muñoz, I. (2014a). Technology-assisted Interpreting. MultiLingual 143, 25(3), pp. 27–32. April/May. Costa, H., Corpas Pastor, G. and Durán Muñoz, I. (2014b). A Comparative User Evaluation of Terminology Management Tools for Interpreters. Proceedings of the 25th Int. Conf. on Computational Linguistics (Coling’14), 4th Int. Workshop on Computational Terminology (CompuTerm’14), pp. 68–76. Dublin, Ireland. August, 2014. de Manuel Jerez, J. (2003). Nuevas tecnologías y formación de intérpretes. Editorial Atrio, Granada. Durán-Muñoz, I. (2012). Meeting Translators’ Needs: Translation-oriented Terminological Management and Applications. The Journal of Specialised Translation. 18, pp. 77–92. Retrieved from http://www.jostrans.org/issue18/art_duran.pdf (Consulted on 3/03/2017). EAGLES. (1996). Evaluation of natural language processing systems. Final report. Eagles document EAG-EWG-PR.2. EAGLES Evaluation Working Group. Technical report. October. Fantinuoli, C. (2018). Computer-assisted Interpreting: Challenges and Future Perspectives. This volume. Gile, D. (1987). La terminotique en interprétation de conférence: un potentiel à exploiter. Meta: Translators’ Journal, 32 (2), pp. 164–169. Gran, L., Carabelli, A. and Merlini, R. (2002). Computer-assisted interpreter training, in Garzone G. and Viezzi M. (eds.) Interpreting in the 21st Century. Challenges and Opportunities. Selected Papers from the 1st Forlì Conference on Interpreting Studies, 9–11 November 2000, Amsterdam/Philadelphia, John Benjamins, pp. 277–294. Kalina, S. (2010). New Technologies in Conference Interpreting, in Lee-Jahnke H. and Prunc E. (eds.). Am Schnittpunkt von Philologie und Translationswissenschaft. ­Festschrift zu Ehren von Martin Forstner. Bern, Peter Lang, pp. 79–96. Kelly, N. (2009). Moving toward machine interpretation. TCWorld, January/February 2009 Issue, pp. 15–17. Moser-Mercer, B. (1992). Banking on Terminology: Conference Interpreters in the ­Electronic Age. Meta: Translators’ Journal, 37(3), pp. 507–522. Pöchhacker, F. (2007/2011). Introducing Interpreting Studies. London and New York: Routledge, 2nd/3rd edition. Rodríguez, N. and Schnell, B. (2009). A Look at Terminology Adapted to the Requirements of Interpretation. Language Update, 6(1), pp. 21–27. Retrieved from http://www .btb.termiumplus.gc.ca/tpv2guides/guides/favart/index-eng.html?lang=eng&lettr =indx_autr8gijKBACeGnI&page=9oHAHvmFzkgE.html (Consulted on 3/03/2017).

84

Costa, Corpas and Durán

Sandrelli, A. and de Manuel Jerez, J. (2007). The impact of information and communication technology on interpreter training: state-of-the art and future prospects. The Interpreter and Translator Trainer (ITT), 1 (2), pp. 269–303. Spohr, D. (2009). Towards a Multifunctional Electronic Dictionary Using a Metamodel of User Needs. eLexicography in the 21st century: New challenges, new applications. Louvain-La-Neuve, Belgium. Presses Universitaires de Louvain. Tarp, S. (2008). Lexicography in the Borderland Between Knowledge and Non-­ knowledge: General Lexicographical Theory with Particular Focus on Learner’s Lexicography. Lexicographica: Series maior. Walter de Gruyter, 1st edition. Tripepi Winteringham, S. (2010). The usefulness of ICTs in interpreting practice. The Interpreters’ Newsletter, 15, pp. 87–99. Retrieved from http://www.openstarts.units .it/dspace/bitstream/10077/4751/1/TripepiWinteringhamIN15.pdf (Consulted on 3/03/2017). Will, M. (2007). Terminology Work for Simultaneous Interpreters in LSP Conferences: Model and Method. In Gerzymisch-Arbogast H. and Budin G. (eds.). Proceedings of the Marie Curie Euroconferences MuTra: LSP Translation Scenario. EU-High-Level Scientific Conference Series, Vienna, Austria. Will, M. (2015). Zur Eignung simultanfähiger Terminologiesysteme für das Konferenzdolmetschen. trans-kom, 8 (1), pp. 179–201. Retrieved from http://www.trans-kom .eu/bd08nr01/trans-kom_08_01_09_Will_Konferenzdolmetschen.20150717.pdf (Consulted on 3/3/2017).

chapter 4

Human Translation Technologies and Natural Language Processing Applications in Meaning-based Translation Learning Activities Éric Poirier Abstract This chapter describes how human translation (ht) technology and natural language processing (nlp) applications can be of use in the design of meaning-based translation learning activities for a professional translation training course. Meaning-based translation learning activities are part of a new instrumental approach aiming at the operationalisation of meaning-based operations (source language understanding, meaning­transfer, target language drafting) through iterative and replicable learning tasks. The instrumental approach makes use of ht technology as one of the three groups of translation tools identified by Bowker (2002) which also includes computer-­ aided translation (cat), the commonly-used term for machine-assisted translation (mat), and machine translation (mt), a diminutive of human-assisted machine translation (hamt). The instrumental approach involves task-based and objectively assessable and replicable learning activities on processing meaning in translation operations. The activities suggested in this chapter are all replicable in different language pairs and involve the processing of meaning by means of ht and nlp applications. They are also measurable in the context of grade-based assessment and traditional (instructional) teaching practices. To the best of our knowledge, those activities with their intensive use of ht and nlp applications have not been suggested elsewhere. The instrumental approach is centered on what technology and tools can do in the resolution of meaning-based translation difficulties and in the validation of correct performing of crucial translation operations.

Keywords human translation (ht) – natural language processing (nlp) – instrumental approach – task-based activity – human learning – meaning

© koninklijke brill nv, leiden, ���8 | doi 10.1163/9789004351790_006

86

Poirier

1 Introduction According to Bowker (2002: 4 and 7), human translation (ht) technology such as word processors, spelling and grammar checkers, electronic resources (terminology data banks and bilingual dictionaries) and the Internet is one of the three types of technology used in translation, the others being computer-aided translation (cat), the most commonly used term for referring to machine-­ assisted human translation (maht), and machine translation (mt), a diminutive for human-assisted machine translation (hamt). The recognition of ht processes in translation technology and the active role humans are playing in the implementation of cat and mt tools are a reminder that translation is in essence a meaning-based human activity. Whatever the translation results one can obtain with translation technology, a well-informed human, preferably a professional trained in translation, will always be required to check for their accuracy and aptness. This might have to do with the fact that most translation technologies for ht, cat and mt just process forms and texts which in their turn support meaning and messages and do not process meaning at all. For translation technologies and applications concerned with the processing of meaning and messages in translation, one must turn to natural language processing (nlp) which includes computational linguistics and, by definition, semantics and meaning processing as well.1 This paper presents an instrumental approach to using ht and nlp applications for the correct translation of messages and interlanguage transfer of meaning. 2

Translation Teaching Using Technology

Translation is a complex cognitive activity that is hard to replicate exactly with different people, and even with the same individual at different times, as the experiment conducted by Gile has shown (2005: 75–100). The individual and cultural variations of communication understanding, information content, language acceptability, norms and idiolects are all making translation teachers’ job very complicated if their goal is to make learners produce professional quality translations. Even if teachers are proficient and experts in translation, their expertise is not to be replicated strictly but transferred in principle with, 1 Most current machine translation systems such as the one provided in Google use a probabilistic model which does not process meaning but textual segments or chains of characters, as opposed to rule-based systems which process linguistic and symbolic data such as information provided in dictionaries and by syntax and semantic (meaning) analysis.

Human Translation Technologies

87

inevitably, acceptable individual (and socio-cultural) stylistic shifts or deviations. Learners therefore, cannot do without numerous empirical and controlled trials and errors in the context of practical learning activities. From the teachers’ perspective, getting as many translation exercises as possible might look like the way to go. Besides the fact that the simple traditional translation exercises have been described as depressing (Kelly, 2005: 97) for learners, these typical exercises are also diverting teaching and learning efforts on translation modalities related to contingent parameters of a particular document or text genre, as opposed to meaning-based translation techniques that apply regardless of documents or text genres. The complexity of human translation processes is due to both the cognitive operations related to knowledge and information processing in understanding source text, transferring meaning, and drafting it in the target language, as well as to the vast number of conceptual and linguistic objects to which these cognitive processes can be applied to. There is no known method agreed on as regards the counting of translation objects. Since Chomsky and the finite set of recursive rules defining an infinite number of sentences, translation objects are probably finite even if there is an infinite number of source language and target language sentences. It is therefore conceptually difficult to reduce this complexity of translation to its core and replicable processes that should be addressed in translation training sessions of limited duration. This difficulty has an impact on the empowerment of the teacher and the learners and is a source of confusion in the methods, processes and techniques to teach and to learn, especially in a methodology-oriented or beginner’s course. Instead of coping with much learning difficulty by simulating the whole set of complex operations with traditional translation activities, we suggest a new instrumental approach in order to operationalise course content, instructions and learning activities with technological tools and electronic resources such as online bilingual dictionary, terminological records and natural language processing applications. ­Operationalisation of course content involves the identification, separation and simulation of single iterative and replicable tasks that are involved and combined sequentially in the translation process, from reading and understanding the source text, to the transfer of meaning and drafting of final target language text.2 2 From a learning perspective, cat tools, bilingual concordancers and mt engines uses are counter-productive in the operationalisation of translation processes by compelling learners to take cognitive shortcuts from the series of meaning-based tasks required in human translation. Those shortcuts might be valuable for practicing experienced translators but for

88

Poirier

The chapter presents first the crucial role of meaning in the context of ht learning and in the designing of learning activities in the instrumental approach. The organisation of the chapter then follows the sequentially-­ structured tasks involved in the translation process, and suggests new pedagogical activities making use of resources and tools commonly used in human translation. Learning activities presented in detail below are the contextual identification and detection of subject fields or domains associated with terms and expressions (which involve a conceptual structuration of fields of expertise), the understanding and analysis of texts with a simplified phrasal-based representation of meaning in sentences, the effective seizing of lexical senses in source text and the effective finding of bilingual correspondences in bilingual dictionaries entries. All these tasks are representative of professional translator’s competencies and skills. With non-subjective and task-based learning activities, the instrumental approach contributes to enhance translation learning in online as well as onsite environments by operationalizing and structuring the learning process of translation operations. This new approach to teaching translation opens up new opportunities for learners on the uses of ht technologies and nlp ­applications not only for translation operations but also in providing better scientific and technical consulting services. Ultimately, this new approach may point to substantial enhancements of ht tools and nlp applications used for translation purposes. 3

Meaning in Translation Technology and Pedagogy

A core principle in the teaching of translation first needs to be recognized; that is, the centrality of meaning. Garnier (1985: 40), as cited in Guidère (2010: 79) states that meaning is very largely recognised as having primacy in the translation operation. Several approaches to translation and theories in linguistics acknowledge the centrality of meaning: the interpretative approach in teaching translation and interpretation, as described in Seleskovitch and Lederer (1989); the meaning-text theory (mtt) in linguistics, as described in Mel’čuk (1981); the meaning-based translation learning manual of Larson (1998), the ­translation learning manual of Delisle and Fiola (2013). In English translation learners and beginners it seems disempowering in the way that these tools take them away from meaning-based operations. Through repetition of attested solutions, novel translation solutions become progressively inaccessible to the beginners as their core activities are centered on slavishly replicating already existing material.

Human Translation Technologies

89

studies, specialists and academics seem to be less explicit about the importance of meaning and more suspicious of mental processes and cognitive ­reality of meaning, although there are exceptions (like Larson). Most authors recognise the existence of extra-linguistic meaning codes and linguistic meaning (lexical and grammar, at least). By acknowledging that a translation process is an act of communication, one implicitly admits that there is a meaning that is communicated and mediated through translation. Scarpa (2010: 85) for instance defines specialised translation as an interlinguistic communication of information through documents written in special languages. As we have seen, translation operations are complex, especially as regards meaning. With the help of ht technologies and nlp applications, the instrumental approach aims to break up complex operations involved in translating into sequences of simpler tasks. For instance, translation involves two primary operations: the understanding (or analysis) of the source text and its drafting (synthesis) in the target language. As a central principle in translation, meaning needs to be processed in both operations. For learners, this processing is not to be confused with text processing that is very similar except that it does not take the contextual meaning into account. With the help of ht tools and nlp applications, it is possible to break up the analysis and synthesis operations in a sequence of smaller mandatory tasks like the correct understanding of the subject field or domain of the text to be translated, the seizing of the grammatical meaning of each sentence to be translated, and the proper search of a relevant translation in a bilingual dictionary (not all good translations can be found in dictionaries, but the modeling of translation need to recognise at first this important step in the efficient processing of professional translation solutions). Those three simple tasks within the general process of translating are just examples of what functional uses of ht technology and nlp applications can provide for a better understanding and modeling of cognitive operations in translation. The interest of the meaning-based approach for teaching translation is that the meaning must be studied in connection with communication forms and codes. It cannot be studied alone since it is an intangible phenomenon attested indirectly through forms. Meaning is always mediatised and cognitively processed through linguistic features such as morphemes, tokens (word forms), texts and lexical units, or parts of. Even a social or cultural code may contribute to the interpretation of meaning. As regard the instrumental approach, technology in translation is used to process all forms, codes and means used to communicate meaning. Meaning is mediated through technology. Although the instrumental approach shares many features of translation learning and teaching with the competency approach of the pacte group

90

Poirier

(­ operationalisation of competence, task-based approach), its global approach towards technology and meaning differ from the conventional translation competencies approach where technology use is confined in an instrumental sub-competence that is ‘predominantly procedural knowledge related to the use of documentation resources and information and communication technologies applied to translation’ (Hurtado Albir, 2015: 259). The main interest of technology-mediated meaning in the instrumental approach is that it recognises iterative and replicable processes that can be modelised and operationalised in learning activities with ht technologies and nlp applications. The instrumental approach takes its origin from the beginning of a book on pedagogy by City E.A. et al. (2009). Because of its role in the instrumental approach to translation technology in translation teaching, online and onsite, it is worth citing it: There are only three ways to improve student learning at scale. The first is to increase the level of knowledge and skill that the teacher brings to the instructional process. The second is to increase the level and complexity of the content that students are asked to learn. And the third is to change the role of the student in the instructional process. That’s it. If you are not doing one of these three things, you are not improving instruction and learning. Everything else is instrumental. That is, everything that’s not in the instructional core can only affect student learning and performance by somehow influencing what goes inside the core. city e.a. et al., 2009: 24

This statement contributes to translation teaching by reminding us of what are the three fundamental elements (what the authors call the instructional core) which may have a lasting impact on the quality of learning; that is content, teacher (their skills and knowledge), and learners (their active efforts). The adjective instrumental means here that all the non-core instructional features of a course exist or may be used uniquely to have an impact on the three instructional cores. In other words, everything (even technologies) that you had in a training course must contribute directly to one of the three instructional cores to improve learning. What is interesting is that the statement implies that there is no learning improvement (or maybe more extremely no learning at all) if a course feature (such as one mediated with technology) does not enhance or improve instructional cores of a course. As regards translation technology, this view has interesting consequences. First, technology is generally not part of the instructional cores of any course, and of translation training courses, except for courses dedicated to the use of

Human Translation Technologies

91

technology tools as translation aids. Second, if one wants to enhance the learning of translation through technology, then one has to integrate technological features that have an impact on one of the instructional cores, and most significantly on the content of the course and on the efforts that learners put in their learning. The agreed principle that translation is a ‘savoir-faire’ which can only be learned through practice is at the origin of this last requirement. The demonstration of the first requirement has been evidenced negatively with the ‘relative’ failure of translation memory courses that were designed as a translation learning course per se. Translation memory tools are just tools, they are not and should not be the primary focus of translation learning.3 Translation is a genuine language activity that is situated in a social and cultural context or environment. Because of that, translating a specific text in a translation class or course does not much contribute to the learning of learners more than what is needed to know to translate this specific text. It appears to be a good reason why learners feel bored (as reported above) when translating texts in translation teaching courses. They might have expected knowledge more pervasive that relates to translation, in general, not only to the translation of a specific text. On the content side of the instructional core, the instrumental approach is centered on generating abstract knowledge that is iterative and replicable and that can be used for several texts and in fact throughout the professional life of learners. This knowledge is also very different from source language proficiency and involves instead textual correlations between source language segments and target language segments. Since meaning is intangible and linked to forms and codes, technology may, therefore, play an instrumental role in the discovery and advance of abstract and generic knowledge on translation. 4

Instrumentation of Subject Fields and Domains

Translation is a communication act dealing with all human activities and fields of knowledge, and with all text-types and communication methods. There is no universal classification of translation specialisation either in fields of knowledge or text types, although specific communication methods, like oral or sign languages, serve to distinguish interpretation from translation. Specialisation of translators in one activity or field of knowledge is technically possible, but most translators tend to work in a large field or similar fields of 3 This is not at all to say that there is no need for learning how to use translation memory software in a translation learning program.

92

Poirier

knowledge so that this content-based delimitation of competencies will not negatively impact translator’s business opportunities. For example, one translator could decide to specialise in the translation (and terminology) of sports but less likely in a particular sport like tennis or golf. The main reason for this grouping is the ability for translators to have enough clients to make a living while making synergies in the knowledge and expertise required to translate efficiently and accurately (in comparison with non-professional translators). Too much specialisation theoretically involves a limitation of business opportunities, and too much broadening in one’s practice involves the risk of not being able to demonstrate adequate expertise and knowledge that is expected from field-based content specialists. This enlarging approach to translation fields of practice has led to conventionally agreed translation specialisations sharing similar knowledge, ­training institution and programs as well as interests in communication practices. Translation specialisations (excluding interpretation services) generally recognised are: scientific and technical translation, economic, business and financial translation, administrative and legal translation, literary and advertising translation, localisation, audio-visual and multimedia translation, as well as general translation, dealing with (in theory) translation principles overlapping all translation specialisation and (more practically) with non-specialised texts. It is effectively possible to ungroup these seven designations and make them individual specialisations like scientific translation, technical translation, economic translation, business translation, and so on. In any case translation specialisations may vary from 7 to 13 different fields of practice. In some translation specialisations, it is not very clear which similarities are involved, and the category is defined by opposition or by contrast with other fields of specialisation (Gouadec, 2009). For example, scientific translation seems to stand in contrast with literature translation more than for any other inherent or organic criteria. Other criteria for classification are the document types that are most commonly translated. Operating manuals and owner’s manuals are translated with a scientific and technical translation approach, as opposed to a literary approach. As well, accounting information is most often translated into a business, financial or economic context where for instance number uses are domain specific (different from general use). The classification of translation subdomains is also dependent on ­knowledge-based domain hierarchies that are much more numerous than translation specialisations. Some initial activities in an introductory course may use technology tools such as online terminological databases (and eventually corpus-based tools, but that topic is complex and beyond the scope of this chapter) to make learners aware of translation specialisations and their close link with knowledge-based domains of human activities as organised in

Human Translation Technologies

93

e­ ducation institutions: humanities (literature, law, philosophy, history, etc.), health sciences, engineering, industrial trades (carpentry, mechanics, insurance, etc.), management (business, finance), computer sciences, performing arts, natural sciences (biology, chemistry, physics, mathematics, astronomy, etc.), to name a few and without being comprehensive. This simple but factual knowledge does probably explain why translators do generally acquire in their training and in their practice a strong general culture. The next section describes a group of iterative and replicable learning activities that may be used to make learners aware of specialised terms belonging to knowledge-based domains. 4.1 Find the One Odd Out Activity This learning activity consists simply of identifying in a group of terms the one that doesn’t belong in the subject field or domain of the others, as indicated by the record of the terms in a terminological data bank (it could be TermiumPlus or iate–The eu’s multilingual term base). This activity is much like the game called ‘find the odd one out’. It involves analysis of the records found with a character search in a terminological database. When the search gives multiple records, learners must select a relevant record and/or eliminate all the others based on the comparison of keywords used as their subject field or domain determination. Only the subject fields or domains of the record to be chosen as being representative or sharing similarities with other terms are to serve as group definition criteria. If this choice is difficult or undecidable for learners, it is possible to indicate which record is to be selected. An example is provided in Table 4.1. below with details of the reasoning to apply to find the correct solution. The most immediate benefit of this activity is to invite learners to read all the relevant records of a specific term and go beyond the first potential translation found with a precise chain of characters. It especially encourages them not to take unnecessary risks by relying systematically on the most common meaning of an expression. The ‘critical’ reading and analysis of terminological records has become an important skill to master in a knowledge-based trade such as professional translation. The activity models this competency and invites learners to seize the relevant subject fields or domains of a text to do specialised content searches in order to get a better understanding of the notions and concept organisations in a given text. Learners will also be familiarised with the criteria to use to exclude terms and their meanings according to the co-occurrence of other terms and meanings in context. With this kind of activity, the instrumental approach will help learners to activate a general organisation of subject fields and domains and their respective multidimensional relationships. This activity might also be considered as an exploration of the hyperonym semantic relationship in as much as the belonging of a term to a domain can

94 Table 4.1

Poirier Find the odd one out! Answers provided from iate terminology database.

Terms

Field(s) as indicated in iate terminology database

ges

Environment, Industry [record in information technology and dataprocessing may be retained at first but later rejected] Environment, Energy, Climate

réchauffement climatique ozonosphère

Environment [it should be clearer now that the keyword is environment, i.e. the odd term should not belong to environment.] indice de smog Environment conditionnement Transport, Humanities, Building and Public Works, No subde l’air ject, Electronic and electrical engineering, Earth sciences, Land Transport, Energy, Industry [the term has not been classified as belonging directly to the environment field of knowledge] éco-blanchiment One record with the following domains: Environmental policy, Marketing, Public opinion Which one is the ‘Conditionnement de l’air’ [by deduction from the analysis odd? of the domains of all terms] be associated with this peculiar semantic relationship. Since there have been no studies on the criteria of domain attributions for terms, it is difficult to determine to what extent the hyperonym relationship is involved in the attribution of subject fields or domains to terms. For that reason, the hyperonym hypothesis cannot be rejected. One important pitfall to be avoided is that the group of terms chosen for that activity needs to be cautiously selected because it’s possible to find groups of terms for which several groupings and solutions may be possible. Since the attribution of subject fields and domains to terms involves some form of knowledge constructions and terminological conventions, the experience gained with one class may serve to another class as regards the selection of groups of terms. Some groups may provide for richer analysis than other ones. An extension of this activity is possible for term identification criteria. This other activity might be suggested for advanced learners or terminology classes. On a conceptual level, it would be interesting to create an activity to make learners aware of the differences between an agrammatical sequence of characters or words, a linguistic expression, a textual unit, a multiword unit, and a

Human Translation Technologies

95

complex terminological unit, as well as when and how those differences are applicable.4 The last step of this activity is to ask learners to classify in which translation specialisation domain the group of terms would belong to. That specialisation, described in the previous section, should be provided beforehand to learners. Neutral definitions also need to be offered to learners since the criteria of specialisation vary slightly from one translation region to the other (i.e. the markets and work environments in Europe are different from North American, which are different from Africa and Asia). In relation to this activity, another aspect of knowledge classification, in general, has been very difficult to apply non-subjectively in the design of valuable learning activities. Even when learners were invited to use the ten main classes of the Dewey Decimal Classification or its equivalent in translation or in library science, the criteria to apply in the categorisation of texts were difficult to replicate among learners. Without clear and neutral criteria of ­classification of knowledge, which would require enormous work on an epistemological level, we have not been able to design iterative learning activities on the categorisation of texts in a particular subject field or domain. Still, the interest of this task is strong in translation since the final decision on the interpretation of words and expressions often depends on this text classification ability. 5

Instrumentation of Grammatical Meaning

As a consequence of the centrality of meaning in human communication and in translation, understanding the source text is a prerequisite for translation into any target language. No translation is even possible without decoding or interpretation of source message. For teachers and learners, it is very helpful to know or to detect when a wrong understanding of the source text is the cause of inadequate translation. A complete representation of what one understand is not at reach for now in science, but it’s possible to operationalise and represent grammatical meaning and rationally explain related understanding errors with common syntactic analysis tools. 5.1 An Instrumental Approach to Syntax As a formal representation of grammatical meaning establishing dependency and coordination relationships among lexical elements of a sentence, and for which we have good descriptions of their lexical meanings (in monolingual and bilingual dictionaries), syntax is the perfect tool to conceptualise critical 4 N-grams technology might be of some help in this regard.

96

Poirier

elements of textual meaning in translation. The instrumental approach takes advantage of grammatical analysis technologies now available on the Web. These tools are used to illustrate visually correct and incorrect representations of source language grammatical meaning. If learners design and build an ­incorrect representation of source language grammatical analysis for a sentence, there is a strong possibility of incorrect understanding, unless there is a representation error. In the instrumental approach, grammatical meaning has to be systematically modelised at the phrasal level, but the modeling doesn’t need to be systematically done at the lexical level. The detailed representation of the lexical level is needed only when an interpretation difficulty arises. Unlike grammatical analysis in linguistics where all the terminal elements of a sentence (and its phrases) need to be identified and tagged, the instrumental approach proposes a simplified phrasal analysis that can be used to represent meaning for translation purposes. The objective of the phrasal analysis is not to create a deep syntactical analysis. Its goal is to show learners how to identify and tag phrases in sentences and to represent their relationships in the sentence. Units of analysis are not content words but propositions and four types of phrases such as subject, predicate, phrasal modifiers and sentence modifiers. Phrasal modifiers are attached to the phrase they modify, but they often need to be identified because of their strong similarities with sentence modifiers. In order to operationalise this representation, translation learners need to become proficient in the use of brackets to separate and identify syntactic constituents in a sentence. Brackets in the syntactical analysis are very similar to html tags; they both contain an identifying keyword (the type of the phrase such as np, vp, ap, etc.) and are used in pairs to define the beginning and the end of phrases in the sentence. What is interesting for learning is that the correct identification of phrases implies a correct analysis of terminal elements in the phrase (lexical units), even if those constituting elements are not represented formally. Although this exercise might be fairly easy in one’s native language, the difficulty level is much higher in the case of a second language user, as it is the case for most professional translators which generally translate from their second language to their mother tongue. The difficulty level is also increased for complex sentences where different subjects and predicates are coordinated or subordinated. A secondary benefit of this activity comes from the fact that metalinguistic names referring to phrases in a sentence and basic analysis of sentences may differ from one language to another. That forces learners to develop and acquire sentence understanding through the prism of another metalinguistic referential system, and not simply calque their native language metalinguistic apparatus to the second language. The phrasal analysis of the instrumental approach gives the opportunity to translation learners to apply

97

Human Translation Technologies

second language analysis skills which will be instrumental in the understanding of second language sentences that they will translate. Unlike lexical meaning which applies to lexical units in any language, the grammatical meaning is not transferable directly from one language to another; some features are unique to one language and culture. Translators need in that case to find and use several acceptable grammatical alternatives in the target language. Some examples in English grammar are the implicit meaning relationships of compound elements in complex terms, as opposed to French and probably other Roman languages where the meaning relationships of compound elements are made explicit (with prepositions). Another lexicogrammatical distinctiveness of English are the gerund syntactical structures and interpretations as being both nominal and verbal (Aarts, 2008). The next section describes a group of iterative and replicable learning activities designed to make learners experiment simplified phrasal analysis with a nlp generator of graphical syntax trees. 5.2 Drawing a Syntax Tree Activity Once learners have acquired the uses of brackets to represent phrases in sentences, they are given some examples of equivocal sentences for which there are two meanings or interpretations. The simplified phrasal analysis with bracket representation of the meaning will make it possible to visualise the two different meanings of the sentence. A classic example is the sentence ‘I saw the man with the binoculars’. Simply put, one reading makes the phrase ‘with the binoculars’ independent from the nominal phrase ‘the man’ (so that it could be attached semantically to the subject and the whole sentence) while the other will attach the same phrase to the nominal phrase ‘the man’ (so that the meaning will be ‘a man holding or using binoculars’). The same phrase could then be analyzed as a phrasal modifier (dependent on the nominal phrase) or a sentence modifier (dependent on the whole sentence). The bracket representations of this equivocal sentence are provided to the translation learners, as in Table 4.2 below, where S etc. stands for sentence, sp stands for subject phrase, vp stands for verb phrase and sm stands for sentence modifier: Table 4.2 Bracket representations of grammatical meanings.

Meaning description

Bracket representation

Meaning 1, ‘with the binocular’ as sentence modifier

[S [sp i] [vp saw the man] [sm with the binoculars]]

Meaning 2, ‘with the binocular’ as phrasal modifier

[S [sp i] [vp saw the man with the binoculars]]

98

Poirier

Even though the two representations are sensibly different, this meaning difference is intangible as regards the surface form of the sentence (‘I saw the man with the binocular’). This is the reason why in order to seize the d­ ifference, it is useful to have a graphical representation tool or application that makes this difference tangible. A free nlp tool such as phpSyntaxTree by IronCreek ­Software (2003) is just the application to use for making this difference tangible, and for creating a simplified grammatical representation of meaning in learning human translation. When learners have their bracket representation ready, they copy it in the phpSyntaxTree text box to automatically generate a syntax tree with the bracket representation provided. As explained above, the simplified phrasal analysis allows learners to identify and tag the sentence nodes, the subject phrase nodes and the verb phrase nodes. A lot of grammatical ambiguities can be represented at the phrasal level, and there is no need to detail the representation at the terminal level (although this type of analysis needs to be done in order to identify and tag the phrases in the sentence). If we input the previous bracket representations in one of these tools, we can generate the two syntax trees in Figures 4.1 and 4.2: S

sp

vp

sm

I

saw the man

with the binoculars

Figure 4.1

Sentence modifier meaning representation. S

sp

vp

I

saw the man with the binoculars

Figure 4.2

Phrasal modifier meaning representation.

99

Human Translation Technologies

As shown in the figures, triangles represent phrases in the sentence, and the structural relationships of triangles represent a visual representation of the simplified grammatical meaning of phrases. As explained earlier, there is a different representation of the phrasal modifier meaning and the sentence modifier meaning for the same tangible segment. The phrasal modifiers are integrated to the phrase they modify while sentence modifiers are represented as independent phrase groups in the sentence. In that sentence, the resolution of the ambiguity involves fairly simple sentence and phrasal analysis, but translators are often required to understand and analyze much more complicated sentences, such as this one which is the first sentence of an article published in The Economist (2013) on the definition of a civil war ‘Not every scrap involving armed groups in the same polity is a civil war: on that much the experts agree’. This orthographic sentence contains two propositions or grammatical sentences identified with the symbols S1 and S2, as shown in Figure 4.3 below: S1 sp1

vp1 s2

much the experts agree on that sp2

vp2

Not every scrap involving armed groups in the same polity is a civil war Figure 4.3

Complex sentence meaning representation.

A correct bracket representation of its meaning is shown in table 4.3 below: Table 4.3 Bracket representation of a complex sentence.

[S1 [sp1 much the experts] [vp1 agree on that [S2 [sp2 Not every scrap involving armed groups in the same polity] [vp2 is a civil war]]]] This logical and meaning-based rearrangement of phrases and propositions makes it much easier to understand the information content of the sentence and greatly facilitates the transfer of information and its translation into another language.

100

Poirier

The instrumental approach in the representation of grammatical meaning gives learners means to make a tangible representation of grammatical meaning so that they are able to validate their correct understanding of sentences before translating. The learning activity is a technological and pedagogical simulation of what needs to be done mentally. This activity on the identification and visual representation of grammatical meaning also shows that k­ nowledge of word meanings (and bilingual dictionaries) are simply not the only meaning features that translators need to handle. Translators also need to become proficient in the understanding of the phrase organisation within source and second language sentences. From the teacher’s point of view, the instrumentation of grammatical ­meaning helps to operationalise the highly cognitive process of grammatical meaning seizing and makes it possible for learners to visualise an incorrect interpretation of the sentences that often leads to an incorrect translation in the target language since one can only translate what one understood. Another benefit of this activity is that it shows the importance of understanding not only words from a second language but their specific structure and construction within phrases and sentences. This knowledge contributes to the required contrastive approach that is needed when translators and learners redraft the meaning of the message in the source language in the target language, which has its own requirements regarding word collocations and phrasal constructions. 6

Instrumentation of Sense Seizing and Correspondent Selection

It has been shown by Gile (2005: 75–100) that the wording of a sequence of events (nouns) simply listed has a high degree of variation in the organisation of primary and secondary information. The replicable experiment was realised in 1982 with a group of about 18 students (some were second language ­speakers, but their results were excluded for the sake of the analysis of native speaker results). The sequence of events was, in fact, the proper name of French presidents followed by a question mark, indicating a question as to who could succeed to the current president. The same experiment can be done in American English with the following sequence of words in Table 4.4 below: Table 4.4 Sequence of events to paraphrase.

Clinton -> Bush -> Obama -> ?

Human Translation Technologies

101

For eight native speakers of French, Gile recorded eight different statements that range from a correct understanding with various degrees of explicitations (schematic wording to fully natural question such as ‘Who will succeed to Barack Obama as the president of the United States?’) to incorrect understandings of what was asked to redraft (such as ‘Is this the correct chronological order of United States presidencies?’). 6.1 The Neutralisation of the Individual Variations There is no doubt that the kinds of variations illustrated above are at play in translation assignments where students are asked to translate from the same source text. This is precisely the conclusion at which comes Gile too, reflecting on the accuracy of translations. However, what happens when learners use the same bilingual documentation source? The next activity that can be used in the instrumental approach to teaching translation is focussing on the neutralisation of the individual variations in redrafting information and meaning with the help of a bilingual dictionary for beginner, or a more detailed documentation source for advanced learners (described below). In the learning activity, all learners must use the same ­bilingual resource to select the proper sense expressed in an occurrence and its corresponding translation in the target language. From a teacher’s point of view, the use of the same documentation source allows for the neutralisation of variations in translation which are due to the use of different documentation sources. This condition makes it possible to verify first if translation learners know how to use the dictionary. More importantly, the translation teacher can verify if their use of the bilingual dictionary is tailored to the meaning-based requirements of a translation task (the correct understanding of a particular segment that needs to be translated). The understanding of translation learners can be operationalised non-subjectively and unbiasedly, provided the tools used by learners be known so that their reasoning or thinking process can be tracked for assessment purposes. This activity is also useful for learning translation because it makes learners aware of different word uses and senses which are a very significant source of meaning variations. For instance, it has been said that the verbal lexical unit set in English may have as much as 400 different senses and uses. Since individual variation in redrafting meaning is somehow inescapable, it makes sense to make learners aware of the importance to start with the correct information and interpretation of each word in the source text. It even seems that the interpretation of words in complex terms and phrases may follow specific rules as we have shown in the interpretation of business in complex terms in a recent paper (Poirier, 2015). The next section describes a group of iterative

102

Poirier

and ­replicable learning activities designed to assess the understanding skills of learners in sense seizing and correspondent selection. 6.2 Sense Seizing and Correspondent Matching Learning Activity The activity consists of translating the French noun ‘valorisation’ using only the information contained in the corresponding entry of the word in a selected bilingual dictionary, such as the one shown in Figure 4.4 below. Valorisation formes /valↄrizasjↄ˜/ nom féminin a. (Économie, Finance) (= mise en valeur) [de région, terrain, patrimoine] development (= augmentation de la valeur) [de produit, monnaie, titres] increase in value b. (= augmentation du mérite, du prestige) [de diplôme, compétences] increased prestige [de profession, activité] improved status [de personne] improved self-esteem (Psychologie, Psychiatrie) self-actualization spéc c. (Écologie) [de déchets] recovering

Figure 4.4

Robert and Collins (2004) entry provided to learners.

The selection of the word ‘valorisation’ is inspired by a bulletin published by ­Anglocom (n.d.)5 rightly titled Valorisation. This paper shows very interestingly that the French word ‘valorisation’ has potentially 14 n ­ ominal correspondents and no less than 16 different verbal correspondents. Clearly, each translation correspondent does not necessarily determine a precise meaning or sense of the lexical unit ‘valorisation’ in the source language. Still, the French-English dictionary should recognise a larger amount of senses to ‘valorisation’ than the ones that are recorded in monolingual French dictionaries. This document shows that contrastive analysis of senses and the taking into account of different translations seem to provide more sense distinctions than the traditional monolingual analysis where no more than three senses are generally

5 Translation agency based in Québec City. The full document is available at http://www .anglocom.com/documents/toolbox/Valorisation.pdf.

Human Translation Technologies

103

a­ ssociated with the noun ‘valorisation’. On a theoretical level, this situation calls for the definition of mapping criterion of translation and correspondents on source language senses and the distinction between senses and translation variants. In the example above, the bilingual dictionary entry is used for the purpose of sense seizing and correspondent selection. In the French-English bilingual entry used (Robert & Collins, 2004, online ­version), the word ‘valorisation’ has only three different subentries: a, b, and c. Each of these numbers refers to a particular sense identifier and subentry of the noun ‘valorisation’. In the case of a, some periphrases are listed as a form of meaning description with a direct correspondent associated to each ­periphrase. For b, just one periphrase is listed, and different collocations are listed with each their own proper correspondent showing that the head noun is translated slightly differently depending on the collocate. In the case of c, the sense is characterised in a different field label (ecology) which is identified by a very specific use of the word that cannot be associated or described with the two others. Table 4.5 shows a question that has been drafted for the instrumentation of the sense seizing and correspondent selection learning activity. Table 4.5 Example of a correspondent selection question and response options.

Using the entry VALORISATION in the Robert & Collins English-French dictionary available online, find the subentry where is given the correct ­correspondent of the particular use of the word ‘valorisation’ in the following statements to be translated. Q1. Le projet aide le groupe cible à découvrir de nouvelles sources de ­valorisation et à éviter la dépendance aux drogues, la délinquance et le décrochage. Your answer: [response options: a, b or c] Q2. Next occurrence of ‘valorisation’ in a sentence. Your answer: [response options: a, b or c] Q3. Etc. In the learning activity designed with the instrumental approach, the bilingual dictionary entry serves as a referential representation of potential correspondents for the word ‘valorisation’, organised by meaning or senses. Learners must, therefore, read the occurrences of the word and seize which sense described in the bilingual dictionary fits the meaning used in the sentence provided. The bilingual dictionary used for reference distinguishes three different senses (a, b, and c) and could be used in beginner classes. A more detailed bilingual documentation source such as Anglocom’s bulletin discussed above

104

Poirier

could be used for advanced learners of translation. In this case, since numerous translation solutions are provided, the teacher would need to establish clear criteria to follow in the selection of acceptable and unacceptable translation solutions. The drafting of translation questions with more numerous response adds up to the difficulty of designing good generic response options fitting many or several occurrences of ‘valorisation’. Also, with more potential translation solutions, the criteria to use need to be made explicit and tangible, which is often difficult and not always possible. Overall, this activity represents a replicable and iterative solution to address the lack of contextual understanding of words in the text among learners, as most translation teachers have experienced and as have advocated Kussmaul (1995: 106) very early in translation studies. The activity is not designed to make learners aware of bilingual dictionaries defects and issues, but there is no doubt that it could certainly show some concrete examples of these problems for translators. 7

Summary and Conclusion

In this chapter, we have shown different uses of human translation (ht) technology and natural language processing (nlp) applications in the design of meaning-based translation learning activities for a professional translation training course. This use of ht technology and nlp applications is part of the instrumental approach in translation teaching which aims at modeling simple tasks involved in complex translation processes. The chapter describes three examples of instructional content that can be taught with technology in translation classes. Each of these practical learning scenarios are illustrated with a concrete example of activity that can be offered to translation learners: instrumentation of subject fields and domains (Find the one odd out activity), instrumentation of grammatical meaning (Drawing a syntax tree activity) and instrumentation of sense seizing and correspondent selection (correspondent selection learning activity). The learning activities are based on repetitive and similar (iterative) meaning processing tasks that are part of the instructional core of learning to translate. The replicability and iteration of meaning-based tasks are very significant features for the study of translation processes and methods. With the use of technology, the instrumental approach supports a rational and non-subjective approach to the assessment of meaning processing for translation purposes. Because of its non-subjective description and assessment of tasks involved in translation, the instrumental approach is very efficient in teaching translation, whether in an online or onsite environment.

Human Translation Technologies

105

Another benefit of the instrumental approach for translation teaching is that the use of technology is for once designed for human translators and human translation learning and not for machine translation (mt) or humanassisted machine translation (hamt) applications. The contribution of technology is essential to the development of meaning-based translation learning. It may also be of primary importance in the development of scientific and shareable data for the advancement of knowledge on human translation and in translation studies. References Aarts, B. (2008). Approaches to the English gerund. In Trousdale F. and Gisborne, N. (Eds.), Constructional Approaches to English Grammar, Berlin, Boston: De Gruyter Mouton, pp. 11–31. Anglocom (n.d.). Valorisation. Retrieved from http://www.anglocom.com/documents/ toolbox/Valorisation.pdf (Consulted on 03/03/2017). Bowker, L. (2002). Computer-aided Translation Technology: A Practical Introduction, ­Ottawa: University of Ottawa Press. City, E.A., Elmore, R.F., Fiarman, S.E. & Teitel, L. (2009). Instructional Rounds in ­Education: A Network Approach to Improving Teaching and Learning, Harvard Educational Press, pp. 24–27. Delisle, J. & Fiola, M.A. (2013). La traduction raisonnée: Manuel d’initiation à la traduction professionnelle de l’anglais vers le français (3rd ed.), Ottawa: University of ­Ottawa Press. Garnier, G. (1985). Linguistique et traduction. Éléments de systématique verbale comparée du français et de l’anglais, Caen: Paradigme. Gile, D. (2005). La traduction. La comprendre, l’apprendre, Paris: Presses universitaires de France. Gouadec, D. (2009). Profession: traducteur, Paris: La Maison du dictionnaire. Guidère, M. (2010). Introduction à la traductologie (2nd ed.), Bruxelles: De Boeck. Hurtado, A. (2015). The Acquisition of Translation Competence. Competences, Tasks and Assessment in Translator Training, Meta 60 (2), pp. 256–280. IATE – The EU’s Multilingual Term Base. Luxembourg: Translation Centre for the Bodies of the European Union. Retrieved from http://iate.europa.eu (Consulted 03/03/2017). IronCreek Software (2003). phpSyntaxTree – drawing syntax trees made easy. Retrieved from http://ironcreek.net/phpsyntaxtree/ (Consulted on 03/03/2017). Kelly D. (2005). A Handbook for Translator Trainers: Translation Practices Explained, Manchester, UK & Northampton MA: St. Jerome Publishing. Kussmaul, P. (1995). Training the Translator. Amsterdam/Philadelphia: John Benjamins.

106

Poirier

Mel’čuk, I.A. (1981). Meaning-Text Models: A recent trend in Soviet linguistics. ­Annual Review of Anthropology 10, pp. 27–62. http://www.annualreviews.org/doi/ abs/10.1146/annurev.an.10.100181.000331 (Consulted on 03/03/2017). Larson, M.L. (1998). Meaning-Based Translation: A Guide to Cross-Language Equivalence (2nd ed.), Lanham, Mar.: University Press of America. Poirier, E. (2015). The interpretation of business in specialized expressions and compound terms for translation purposes, Intralinea 17. Retrieved from http://www .intralinea.org/specials/article/the_interpretation_of_business_in_specialized _expressions (Consulted on 03/03/2017). Robert & Collins (2004). Grand Robert et Collins (bilingue) [e-dictionary], Paris: Le Robert. Scarpa, F. (2010). La traduction spécialisée. Une approche professionnelle à l’enseignement de la traduction. Ottawa: Presses de l’Université d’Ottawa. Translated and adapted from Italian into French by M.A. Fiola. Seleskovitch, D. & Lederer M. (1989). Pédagogie raisonnée de l’interprétation. Paris: ­Didier Érudition. TermiumPlus (2017). The Government of Canada’s terminology and linguistic data bank. Ottawa: Translation Bureau. Retrieved from http://www.btb.termiumplus .gc.ca/ (Consulted on 03/03/2017). The Economist (2013). Defining Conflicts – What makes it a war? (2013). Retrieved from http://www.economist.com/news/briefing/21589432-some-say-killing-25-people -yearenough-others-suggest-1000-what-makes-it-war (Consulted on 03/03/2017).

part 2 cat and cai Tools

∵

chapter 5

Monitoring the Use of newly Integrated Resources into CAT Tools: A Prototype Aurélie Picton, Emmanuel Planas and Amélie Josselin-Leray Abstract In this chapter, we discuss the impact of the integration of digital linguistic data into the software environment of translators. We have studied more particularly the contribution of knowledge-rich contexts (KRCs, Meyer, 2001) to specialised translation. We carried out our research within the framework of the anr cristal which is funded by the French government (ANR-12-CORD-0020), and whose main objective is to develop innovative techniques for extracting krcs useful for translators from comparable corpora. Our strategy consisted in testing the use of different types of krc by translators working in a cat environment specifically designed for our experiment. Our careful observation of translators’ behaviour and the significant number of participants (42) have led us to draw some initial conclusions about the characteristics and patterns of the use of krcs, and their complementarity with traditional resources used by professional translators. This study provides us with a basis for further related research both on the ergonomics of computer-assisted translation tools and the integration of new resources useful for translators.

Keywords cat tools – computer aided translation – knowledge-rich context – translator’s workstation – translator monitoring – translation process – translation resources

1 Introduction The long-running battle waged by corpus linguists also involved in translation studies has started to bear fruit: professional translators are becoming increasingly aware of how useful corpus-related tools can be in their daily work. This heightened awareness can be accounted for by several factors, such as:

© koninklijke brill nv, leiden, ���8 | doi 10.1163/9789004351790_007

110

Picton, Planas and Josselin-Leray

• The unrelenting zeal of researchers in translation studies who have been emphasizing the benefits of using corpora for translation practice for over 10 years (e.g. Bowker, 2012; Duran-Muñoz, 2012; Kübler & Aston, 2010; Varantola, 2003); • The introduction of training sessions in translation programs that are no longer theoretical sessions but more practical, hands-on sessions that show the trainee translator how corpora can be integrated into the translation process, (e.g. Aston, 1999; Beeby et al., 2009; Bernardini, 2006; Bowker, 1998; Fantinuoli, 2013; Kübler, 2011; Zanettin et al., 2014). The efforts made by several computational linguists towards the greater availability – especially to translators – of some corpus-processing tools, such as: • Monolingual or bilingual concordancers (e.g. AntConc or ParaConc) • Corpus-compiling tools based on keywords (e.g. BootCAT, ConQuest) • Corpus-based term extraction tools (e.g. TermoStat, TermSuite) However, it cannot be denied that, in 2015, the most efficient tools are still in the hands of researchers and are not easily accessible to the average translator (e.g. Bowker, 2004; 2011; Fantinuoli, 2015; Kübler, 2014). A possible lead to change this situation, already suggested in the 1990s by Atkins (1996) or by Bowker (2012), is to work on the integration of ‘corpus processing’ tools into cat tools. However, to date, the integration of these tools into existing cat tools (e.g. Star Transit,1 Multicorpora Multitrans,2 Kilgray MemoQ3) has remained slow. Even in the most recent tools such as the ‘LiveDocs Corpus’ ­option of MemoQ, the use of so-called ‘corpora’ is currently limited to the querying of text databases (which are called ‘corpora’ but are in fact nothing more than the documents compiled into a translation memory or a data bank) by means of a keyword or a character string search, and whose result is the display of some text fragments such as translation memory (bi)segments or extracts from various documents. With this situation in mind, we aim at fuelling the debate over the development of new cat tools that integrate other functions related to fine-grained analyses of corpora. The translator can resort to a corpus for a variety of reasons when translating. We will focus more particularly on one specific aspect: the integration of a special type of contexts automatically retrieved from comparable corpora – Knowledge-Rich Contexts.

1 2 3

1 https://www.star-group.net/en/products/translation-and-localization.html, last consulted on 3/03/2017. 2 http://prism.multicorpora.com/acton/fs/blocks/showLandingPage/a/5698/p/p-0022/t/ page/fm/0, last consulted on 3/03/2017. 3 https://www.memoq.com/en/, last consulted on 3/03/2017.

Monitoring the Use of Integrated Resources into cat Tools

111

In Section 2 of this chapter, we first present an overview of the notion of ‘context’ in translation, its link with corpora and its importance for translation. We then focus on the notion of ‘Knowledge-Rich Contexts’ and their relevance for translation. Section 3 explores cat tools interfaces, how contextual information is used in these tools, and how to keep track of translation actions while monitoring experiments involving translators. Section 4 offers an overview of the final protocol we set up to monitor the translators’ actions and, more particularly their use of krcs while translating. In Section 5, we present the first main results obtained from this protocol. These results offer precious hints to discuss and to understand better what ‘integrating corpus-based data’ could mean in today’s cat tools. 2

Context in Translation

This section first explores the notion of context in translation – a central, but fuzzy notion whose definition needs to be refined, especially given the specific needs of translators. It then shows how corpora have changed the way traditional resources used by translators – such as dictionaries – present contextrelated information, and introduces the concept of ‘Knowledge-Rich Contexts’ for translation. 2.1 A Key Notion with a Wide Scope ‘Context-bound’ (Varantola, 1994), ‘context-dependent’ (Varantola, 1998: 181), ‘context-sensitive’ (Rogers & Ahmad, 1998: 95; Atkins, 2007) – there is a wide variety of terms used by linguists to stress how crucial contextual information is for translation and translators. According to the findings from Atkins and Varantola’s survey on dictionary use (Atkins & Varantola, 1997: 31), the type of expressions causing the most problems for translators are ‘not hard words’ but ‘very general words whose translation is highly context-dependent’. This is also underlined by Rogers and Ahmad (1998: 195), who indicate that ‘one of the translator’s prime needs is for context-sensitive information’. But what exactly is meant by ‘context’ in that case? Context is a fuzzy notion whose definition as far as translation is concerned can be particularly broad. For instance, Melby and Foster (2010) define it as ‘a set of resources that need to be available to translators’ and Costa (2006) uses the term to refer to ‘les réalités, linguistiques ou non, qui entourent le texte et sa production’. It can encompass the translator’s physical environment, the context of situation, as well as what Halliday (1999) calls ‘co-text’, which pertains to language in use. The distinction that seems most fundamental thus lies

112

Picton, Planas and Josselin-Leray

between the extra-linguistic or non-verbal context and the intra-linguistic or verbal context. Even though both are extremely important for translators, we will only focus on the latter. More precisely, our definition of context, along with the Encyclopaedia Universalis (Fuchs, n.d.), will refer to the linguistic surroundings of a particular utterance, i.e. the linguistic units that precede and follow a specific linguistic unit element within an utterance, at sentence level or on a larger scale. 2.2 The Contextual Needs of Translators Even when narrowed down to co-text, context as seen from the translators’ needs perspective is still a many-sided notion. It is in turn an umbrella term for a great variety of linguistic information, which in fact goes beyond sheer lexical information, as underlined by Varantola (1998: 181): ‘The information sought [by translators] is broader in scope; they often want to know how the expression behaves grammatically, and what kind of lexical, sentence, paragraph or text environment it normally occurs in. At a higher level, they wish to know whether the expression is appropriate for the context, subject field, text type or register in question’. Varantola mentions the need for ‘a continuum of contextual, pragmatic and encyclopaedic information’ which is summed up in Figure 5.1 below. Bowker (2011, 2012) has also tried to sum up what type of contextual information is needed by translators. She underlines that ‘[t]ranslators may sometimes not even know what it is that they need: they are seeking for inspiration, associations, similar examples, parallel situations that can be adapted. (…) [i]t is often a case of I don’t know what I’m looking for, but I’ll recognize it when I see it’ (Bowker, 2012: 391). The type of contextual information needed ranges from information about usage (which includes information about collocations), information about the frequency of a term, information about lexical and conceptual relations, information about style, register and genre to information about usages to avoid. All this information, which can be either source-text related or target-text related, is much needed by the translator at various stages of the translation process.

Equivalent > gramm. collocation> lexical collocation > examples > idiomatic usage > longer passage > parastructure > text structure > stylistic information > encyclopedic information Figure 5.1

Contextual needs as a continuum of contextual, pragmatic and encyclopaedic information (Varantola, 1998: 182).

Monitoring the Use of Integrated Resources into cat Tools

113

2.3 Towards the End of the ‘Context-Free vs. Context-Sensitive’ Debate? The resources traditionally used by translators, such as dictionaries, have been criticised for not providing enough contextual information, such as usage examples (Varantola, 1994: 607). Bowker (2011: 214–215) underlines that, in ­existing term banks too (e.g. Termium, le Grand Dictionnaire Terminologique), most records present terms out of context, or only in a single context – even though, back in the early 60s, the creators of the very first term banks, especially dicautom, intended only to provide terms in context instead of providing lists of terms in isolation (Bachrach, 1987: 98–100). In later term banks though, some kind of contexts were to be provided, as we will see below (see 3.2). As far as dictionaries are concerned, it is widely acknowledged that the information contained is mostly context-free, insofar as what is presented is ‘prototypical uses’ (Varantola, 1994: 607) or what is considered ‘most suitable’ (Atkins, 2007: 144) while what translators need is highly context-dependent (Atkins, 2007: 143; Varantola, 1994: 607). However, already in 1994, Varantola suggested an access to electronic corpora, such as the ones used by lexicographers during the dictionary-­ compiling process, might be able to bridge the gap. This was also advocated by B ­ owker (2012: 390) nearly two decades later, when she suggested ‘capturing the ­intermediary steps of lexicographic research’, which nowadays is always corpus-based. All the evidence collected regarding the contexts in which the headwords are found: e.g. their ‘grammatical and collocational behaviours’, the various relationships ‘that hold between words and their underlying concepts’ (Bowker, 2012: 391), and a direct access to a large number of kwic concordances can prove more useful to the translator than a finalised dictionary entry. Some publishing houses have decided to take that step and to include access to some corpus information, such as concordances (see for instance the electronic version of the Oxford Advanced Learner’s Dictionary, 8th edition, 20104), making the dictionary an integrated or ‘hybrid’ tool, as explained by Granger (2012: 5). However, this mostly applies to learners’ dictionaries. Moreover, although translators ‘both value and appreciate having access to distilled lexical knowledge’, as Bowker (2012: 390) puts it, they are under time constraints, which, among other reasons, makes it necessary for them to have access to pre-sorted information. One step in this direction could be the integration of Knowledge-Rich Contexts. 4

4 Oxford Advanced Learner’s Dictionary, 8th edition, 2010, A.S. Hornby & J. Turnbull (ed.), ­Oxford: Oxford University Press.

114

Picton, Planas and Josselin-Leray

2.4 Knowledge-rich Contexts for Translation Knowledge-Rich Contexts (from now on krcs) are defined by Meyer (2001: 281) as ‘context[s] indicating at least one item of domain knowledge that could be useful for conceptual analysis’. Although the existing studies about krcs originally focused mainly on text-based terminology or ontology-building (Auger & Barrière, 2008; Aussenac-Gilles & Séguéla, 2000), more recently, several papers (cf. Bowker, 2012), have shed light on the importance of such contexts for translators: having access to usage information for a given term or to semantic and conceptual relationships between terms – be it in the source language or in the target language – is essential for translators. The (semi-)automatic extraction of krcs thus seems of particular interest as an attempt to provide translators with pre-sorted and relevant information from corpora. In this research, we propose to work on the integration of (semi-) automatically extracted krcs from corpora, following the propositions of Hmida et al. (2014) and Lefeuvre (2013). We distinguish two types of useful krcs for translation: linguistic krcs (providing information regarding collocations) and conceptual krcs (providing information regarding the definition of a concept and conceptual relations, such as hypernymy, synonymy, etc.). In the next section, we investigate how this contextual information can be used into the usual cat tools and what the specific needs of translators are. We propose an interface that integrates a series of pre-sorted krcs and that allows us to monitor the use of these contexts by translators. 3

Designing an Interface for Examining the Use of krcs

This section is devoted to an overview of the various interfaces of the most popular cat tools, together with a discussion of the way context-related information is integrated into those tools. In keeping with our experiment objective, it ends with a description of the various methods used to record the translator’s many activities during the translation process. Analysis of the Interface Elements of a tws (Translator’s WorkStation) Generally speaking, and as far as we know, publishers of computer-assisted professional translation software5 give little detail about their theoreti­ cal groundings or the ergonomic choices they made when designing their 3.1

5

5 Different names were put forward for this type of software. In this document, we use the term ‘Translator’s WorkStation’ (tws) (Hutchins, 1998).

115

Monitoring the Use of Integrated Resources into cat Tools Table 5.1

Interface elements of different software. Star Trados Déjà Vu Wordfast OmegaT MemoQ Wordbee Wordfast Trados Transit Workbench Classic Pro Studio

Available from One window Double window Tabbed windows

1991 − + +

1992 + − −

1993 − + +

1999 + − −

2000 + − +

2006 − + +

2008 − + +

2008 − + +

­products.6 However, some ‘meta’ users and researchers have contributed significantly to such tools by laying the basis for their development. We have selected four elements for the development of our software prototype model, as they seem to us essential for the designing of such an interface. 1

Integration of a Double Window Displaying Both the Source Text and Target Text Aligned Side by Side As from 1980, when the very concept of an assisted-translation tool was still at the initial stages, Kay (1980) suggested displaying the source text and target text side by side on the same page. In the early years of twss, some pc software tools such as Trados Workbench (1992), Wordfast Classic (1999) or OmegaT (2000) did not factor in this suggestion for technical reasons.7 Since then, most tools have used a double window (Table 5.1). Today the double window is presented in tabular format: the left-hand column shows the source segments, the right-hand column contains the target segments and each line includes a segment to be translated (Figure 5.2).

2 Tabbed Windows Displaying Several Resources The large number of translation resources likely to be used by translators, and the space required for them, make it impossible to display them clearly and simultaneously in a single window. The solution offered by twss currently available to provide access to this wealth of information lies in interfaces including tabbed windows. The various tabs generally provide access to: 6 7

6 In 37 years of existence, the Translating and the Computer Conference has published only 5 such publications, among which Planas (2005) or Lengyel (2011). 7 For example, the challenge of integrating this tool into Microsoft Word.

2009 − + +

116

Figure 5.2

Picton, Planas and Josselin-Leray

Example of present-day tws (2015): a MemoQ screenshot.

• • • •

Translation units stored in translation memories Terms from terminological databanks Excerpts from context-related document search A viewing window displaying the document layout in its final format (e.g. the lower left side of the previous screenshot of MemoQ) • Results of automatic queries on specialised web portals (e.g. Studio’s search engine Web LookUp or MemoQ Web Search). 3 Keyboard Shortcuts Translators spend a lot of time typing on a keyboard. Moving one hand to grasp the mouse, pointing and clicking with it, then putting their hands back to the keyboard are time-consuming operations. Translators repeat these operations over and over again thus wasting valuable time, hence the increasing relevance of keyboard shortcuts. Obviously, this point held twss developers’ attention from very early on (Figure 5.3). 4 Customisation Options Habits and ergonomic preferences vary from translator to translator. Consequently, the layout of tws sub-windows – among other interface elements – can

Monitoring the Use of Integrated Resources into cat Tools

Figure 5.3

117

Example of Wordfast Pro 4 screen for keyboard shortcuts management.

also vary according to translators’ preferences or according to the task being performed by a translator. Following in the wake of such precursors as Star Transit, most of today’s twss allow customising the layout, positioning and size of interface elements within the window. 3.2 Provision of Contexts in twss The second central point of our study deals with the inclusion of contexts in twss. In the seventies, long before twss ever existed, terminological databases of large organisations hosted in central servers already included sentencebased context examples for illustrating the use of specific terms. As early as 1973, Eurodicautom terminological database (cee) included terms with relevant contexts, and was followed by Termium (University of Montreal) in 1974 (see 2.3.). Goffin (1973: 251) gave the example of a phraseological glossary in five languages also published by the European Commission in 1968 in Luxemburg (Figure 5.4). At the same time, Krollmann (1981), Arthern (1978), Kay (1980), or Melby (1981) advocated access to source and translated texts via a ­document referencing system. Kay (1980) and Melby (1981) suggested the use of ‘­bilingual concordances’ for generating sets of keyword searchable phrases aligned side by side. Schwab (1981) put forward the idea of kwic lists. This function was first installed in the reftex system at Odense University in Denmark ­(Kjærsgaard, 1987).

118

Picton, Planas and Josselin-Leray

Figure 5.4

Extract from L’acier dans les industries chimiques’, European Commission,8 1968.

As a result, several developments occurred in various tools such as: – The KonText concordancer (KonText is short for Knowledge on Text) of the Translator’s Workbench Project (Ahmad et al., 1989) that allows translators to view the frequency of words, statistics, concordances and collocations. It also enables the user to browse words using the conceptual relations between them thanks to the marvin tool. It should be noted that – in line with what our cristal project proposes typically and automatically – those relations (‘belong to’, ‘is part of’, ‘causes’) are identified within the text using a semi-automatic tracking of markers such as ‘is made of’, ‘carries out’, ‘is placed on’; – The Sadler and Vendelmans (1990) syntax concordancer that allows translators to refer to aligned bilingual syntactic trees; – TransSearch (Macklovitch et al., 2000), an internet-based concordancer and precursor of Linguee, which makes it possible to do monolingual or bilingual searches in a bilingual corpus aligned side by side. Although the eighties and nineties saw a multitude of new ideas, current twss have only kept the following functionalities of concordancers: – Search of plain text string, without inflected analysis of equivalent strings; – Contexts presented in the form of plain monolingual or bilingual lists, classified by measuring their closeness in terms of edit distance; 8

8 ‘L’acier dans les industries chimiques’, Phraseological Glossary Deutsch-Français-ltalianoNederlands-English, ivth Steel Congress, 1968, European Commission. Retrieved on 2nd 15/05/2016 from http://bookshop.europa.eu/is-bin/INTERSHOP.enfinity/WFS/EU-Bookshop -Site/fr_FR/-/EUR/ViewPublication-Start?PublicationKey=XK1455003.

Monitoring the Use of Integrated Resources into cat Tools

119

– Contexts extracted from either translation memory units, or monolingual or parallel reference texts (see Gow (2003)’s classification). 3.3 Methods Used to Observe the Translator’s Tasks The third central element of our study focuses on the different methods for observing the translator’s tasks. Among various approaches, Göpferich and Jääskeläinen (2009) list the ­following methods: • Think Aloud Protocols (taps), where translators comment on their choices as their work progresses. These comments are recorded on a tape recorder (Jääskeläinen, 1989; Künzli, 2001). • Dialogue protocols, where the decisions about translation are made through a dialogue between peer translators, and are also recorded on tape ­(Varantola, 1998). • Retrospection, in which the translation is explained after it has been done. • Integrated problem and decision reporting (ipdr) in which translators note down and explain points they deem critical. • Questionnaires, interviews and diaries are commonly used off-line to get more information from translators. Although these various procedures obviously give quite satisfying results concerning the translation process, they come under considerable ­criticism – e­ specially the tap (Ehrensberger-Dow & Massey, 2013; Dragsted & Carl, 2013). The main criticism originates from the fact that these procedures are generally intrusive and do not satisfy the ‘ecological validity’ principle ­(Ehrensberger-Dow & Massey, 2008): they do not comply with the traditional working ­environment of a translator. To make up for this limitation, the 2000s saw the advent of key logging software: e.g. InputLog (Leitjen & Van Waes, 2006), Translog 2000, Translog 2006 (Schou et al., 2009) and Translog ii (Carl, 2012) that allow the recording of the translators’ textual production activity without intrusion (Göpferich et al., 2008) (Alves & Vale, 2011).9 Additionally, current devices are rather inconspicuous (e.g. cameras are embedded into the sound bar of the monitor) so as not to disturb the translator’s ecological environment. Generally speaking, the study of the above-mentioned procedures enables us to identify the following trends: 9

9 Using eye-trackers, Göpferich et al. (2008) have studied the translation process phases, Alves & Vale (2011) the drafting and revision phases of the translation process.

120

Picton, Planas and Josselin-Leray

• The combination of different techniques leads to more explicit results. This is what Alves (2003, vii) calls ‘triangulation’: ‘the need to apply several instruments of data gathering and analysis in their attempts to throw light on the nature of the process of translation’. • The drafting of a preliminary questionnaire or interview to understand the translators’ background and translation habits better; • The drafting of retrospective interviews conducted just after translation (for short memory matters) to have a better understanding of translation choices. 3.4 Elements Chosen for Argos10 With those various elements in mind, we wish to introduce an interface prototype model called (Figure 5.5). In terms of design, we have kept all four key interface elements previously mentioned: separate windows for the source text and the target text; tabbed-windows so as to show more data; keyboard shortcuts for basic operations; and finally possible customization using the mouse. A sub-window is dedicated to the display of krcs: one tab for source krcs, one tab for target krcs. krcs related to a given term appear when the term is selected in the source text (see ‘cinder cone’ in the screenshot) or entered in the dedicated search field (to the right of the cristal logo). The user can then select/unselect krcs by just clicking on them. These searches and selections were logged: each time a krc list was displayed, the user had to click on at least one krc to show which ones were of best help. The choices were kept and displayed in case the same list of krcs was selected again. The other resources were displayed in tabbed-windows situated at the bottom of the interface. As a complement to Argos logging, we also used BBFlashback to record the screen, and all keyboard activity, in whichever windows, was recorded so as to provide a means of identifying clearly in which window the recorded edition was executed. Post-editing processes made it possible to compile one or several users’ logs, in order to extract different kinds of information: which terms were looked for 10

10

It should be noted that a recent paper (Daems et al., 2006) presents a software package quite similar to ours, using a combination of Casmacat and InputLog interfaces to examine the influence of the use of resources on translation: Casmacat records the translation while InputLog records the use of external resources (such as dictionaries and encyclopaedias). This requires a subtle combination of both loggings. We thought it better to create an interface in which all resources were available in a single log so as to appraise their use more precisely.

Monitoring the Use of Integrated Resources into cat Tools

Figure 5.5

121

Argos interface.

and in which resources; which krcs were most frequently used; what was the sequence of resource use, etc. 4 Protocol With the aim of assessing the relevance of integrating krcs into a cat tool, we set up an experiment with trainee translators, who were asked to translate a short text in English using Argos. After a 15-minute training session on the environment, the participants were given two hours to translate the text and indicate which krcs were the most useful. Their activity was recorded and saved. ­Immediately after the translation task, they were asked to fill in the ­questionnaire. We then conducted recorded interviews. More precisely, our protocol relied on the following elements: 4.1 Participants 42 second-year Master’s students enrolled in a professional translation programme – 24 from the cetim, Université Toulouse 2 Jean Jaurès, Toulouse and

122

Picton, Planas and Josselin-Leray

18 from the Faculté des Humanités, Université Catholique de l’Ouest, Angers – participated in the experiment. They were all French native speakers, were all familiar with cat tools,11 and had followed English-into-French translation courses. 4.2 Text On the basis of previous experiments with translators (e.g. Bowker, 1998;­ Künzli, 2001; Varantola, 1998), the text to be translated was (i) written in ­English to be translated into French; (ii) 150-words long and to be translated in less than two hours; (iii) dealing with a subject that was (a) technical enough to necessitate terminology search, but (b) not too technical for the students, and (c) familiar enough to us so that we could assess the quality of the translations at the end of the experiment, (iv) containing a number of collocational and syntactic difficulties, (v) with a very clear structure. We selected an extract from a popular-science book on volcanology12 describing the two phases in which cinder cones are built. 4.3 Resources The participants were provided with two types of resources: (i) lexicographic resources: the Robert and Collins dictionary (English  French),13 ­Termium,14 the Grand Dictionnaire Terminologique,15 (all three online), and some entries of a specialised bilingual dictionary of volcanology;16 (ii) krcs: for each term in the text, we selected different types of krcs that had been automatically extracted (Section  2.4.). We tried to collect different types of krcs for each term, such as definitions, hypernyms, synonyms and collocations. These were taken partly from a comparable, French–English, popular-science corpus compiled by Josselin-Leray (2005), and partly from reliable documents found on the Internet. Firstly, a dozen one-sentence long contexts were selected for each 11 12 13 14 15 16

11

12 13 14 15 16

In a later experiment, in 2015, 8 professional translators were invited to participate in a similar experiment in order to compare their use of krcs with that of students. The results have not been published yet. What’s so hot about volcanoes? Wendell A. Duffield (2011), Mountain Press. https://www.collinsdictionary.com/dictionary/english-french. Last consulted on 03/03/2017. http://www.btb.termiumplus.gc.ca/tpv2alpha/alpha-fra.html?lang=fra. Last consulted on 03/03/2017. http://www.granddictionnaire.com/. Last consulted on 03/01/2017. Dictionnaire bilingue des Sciences de la Terre (anglais/français) (2013), Michel J.-P. et al. Paris, Dunod, 5th edition. Relevant entries were converted into electronic form.

Monitoring the Use of Integrated Resources into cat Tools (i)

123

Tephra is any material ejected explosively from a volcano (ash, lapilli, cinder, and spatter (conceptual KRC, hypernymy and definition)

(ii) Au-dessous de 64 mm de diamètre, les projections sont dites lapillis (conceptual KRC, hypernymy and definition) (iii) Les fontaines de lave , hautes de presque 60 mètres, se composent d'une myriade de lambeaux de magma. (conceptual KRC, meronymy) (iv) Entre deux plaques divergentes, se trouve une zone d'accrétion où s'épanchent des laves fluides et denses, de type basaltique. (linguistic KRC, collocation) Figure 5.6

Examples of krcs in French and English.

term in the source language (English). Secondly, we tried to anticipate possible equivalents in the target language (French) for each term and provided contexts for each. As described in Section 2.4, we made a distinction between conceptual krcs and linguistic krcs. Half of the krcs provided were linguistic while the other half were conceptual (Figure 5.6). We also added some ‘Knowledge Poor Contexts’, supposed to be of no use to the translator. 4.4 Questionnaire and Interviews The translation task was complemented by an online questionnaire about the main translation difficulties, the use of resources and krcs, the relevance of krcs, the stages of the translation process when krcs were needed most, the interface, and general information (age, experience, degrees, etc.). Then, ‘focus group’ interviews lasting approximately 20 minutes were conducted with all the participants, divided into smaller groups (about 5 students). Six of the participants were interviewed individually and were asked to comment on their own actions and choices while translating. For this purpose, the interviewer and interviewee viewed some sequences of the BBFlashback videos to be discussed. Based on all the data recorded by Argos during the experiment, the results were obtained by ‘triangulation’ (Section 3.3). 5

Results and Discussion

The results obtained through this protocol provide some ideas to fuel the debate over the development of new cat tools with corpus-based data. More precisely, we would like to discuss three important elements that contribute

124

Picton, Planas and Josselin-Leray

to the definition of what ‘integration’ means when talking about corpora and cat tools for translators. 5.1 The Quality of Resources to be Integrated: The Example of krcs The quality of resources to be provided to translators is a central issue, and a balance between the quality and quantity of available data has to be found. However, as mentioned in Section 2, context quality in translation is a complex issue. Our findings have identified five aspects that contribute to ensure optimum quality (hence relevance) of knowledge-rich contexts to be provided to translators. The first three aspects relate to the type of information that should ideally be included in contexts to be integrated in a tool, while the other two concern data format. 1

A Useful krc is a Context That Has Similarities with the Segment to be Translated in the Source Text During the interviews, several participants pointed out the reassuring feeling they experienced when krcs were closely related to the source text or to the translation they were considering in the target language (Figure 5.7). This perception is backed up by observation of the most frequently chosen contexts according to the segments to be translated (Figure 5.8).

2 A Useful krc is a Context Originating from an Identifiable Source Interviews and questionnaires make it possible to highlight a very strong response from the participants who want to know the origin of the context resources displayed, so they can be assured of their reliability and relevance. During our experiment, we had not actually specified the source. Figure 5.9– 5.10 present a few extracts from interviews and questionnaires to illustrate this point. Indicating the source also has a significant impact on the choice of krcs by translators. Indeed, one of the contexts most frequently chosen because of its relevance (13 participants out of 42) was among the few that provided its source (Figure 5.11). .

In diesem Abschnitt werden die Optionen beschrieben, die Sie mit dem Befehl cmdjob verwenden können.

Figure 6.1 A shortened example of a segment editing session.

144

Moran, Lewis and Saam

tool. This is still a prototype based on OmegaT that only replays keystroke actions within a segment. It is intended as a means of ensuring that event data is interpreted correctly. With regard to reliability, as the software is used to conduct tests offline there is no risk that time data is affected by server load or network latency. This is a major advantage of a desktop-based approach over a web-server based one. For the first productivity test we carried out using iOmegaT software, documentation from a Welocalize client called Autodesk,20 a software publisher specialising in Computer Aided Design (cad) software, was translated and post-edited by 24 translators into 12 languages. Two translators per language pair were paid for two days each. This resulted in approximately 48 days of uad. This test was considered to be an academic study. According to rules governing research ethics at Trinity College Dublin all translators who participated signed a form to show they understood the kind of data that was being gathered. In subsequent tests carried out by Welocalize translators remained anonymous to us (e.g. the Dell21 productivity test discussed below). In order to evaluate the translators’ perception of the usability of the cat tool in this test we requested that they fill out a non-mandatory questionnaire with a set of standard questions (Brooke, 1996) after the productivity test had been completed. Mainly translators used various versions of sdl Trados Studio and we wanted to see that they were reasonably comfortable using OmegaT. The answers were on a scale of 1 to 5, where 1 was ‘Strongly disagree’ and 5 was ‘Strongly agree’, and 15 out of the 24 translators who participated in the test completed it. The results are shown in Table 6.1. The positive feedback shown in Table 6.1 regarding ease of use for OmegaT was mirrored by the fact that translators were able to work with the software with very little training. We provided translators with a three-minute video on how to get started with OmegaT and a second two-minute video on how to configure it (e.g. to change the font size and switch on the spellchecker). No interactive training was provided and written instructions were presented on a single page. A day prior to the productivity test translators were given a 30-minute throw-away test to allow them to get used to the tool but this proved unnecessary so it was not repeated for subsequent tests. We provided synchronous support for translators via Skype. Only two translators availed of this. Both were cases that resulted in small fixes to the software. Synchronous support was not provided for subsequent tests.

20 21

http://www.autodesk.com. http://www.dell.com.

145

User Activity Data in cat Tools Table 6.1

Translator feedback on iOmegaT/OmegaT.

Question

Average

I found OmegaT unnecessarily complex I think that I would like to use OmegaT frequently I thought OmegaT was easy to use I found the various functions in OmegaT were well integrated I thought there was too much inconsistency in OmegaT I think that I would need the support of a technical person to be able to use OmegaT in my own work I would imagine that most people would learn to use OmegaT very quickly I found OmegaT very cumbersome to use I needed to learn a lot of things before I could get going with OmegaT

1.6 3 4 3.4 2.4 1.8 4.2 1.8 1.7

3.1 uad and Information Security The uad is stored locally on the translator’s pc until it is manually sent by email or (S)ftp to the client along with the translation project package so at all points data security can be maintained. Translators in the tests were only identifiable via an id. At any point the translator could read the instrumentation files in cat-uad format in a directory called/instrumentation within the translation project directory. When designing the data format, we aimed to make it maximally readable for human translators as well as machine-readable. Thus, despite the fact that the xml files consume more disk space, element names are fully spelled out, e.g. is used instead of . In its current form one hundred days of cat-uad take up about 1 gb of space when uncompressed but 129mb when compressed. 3.2 OmegaT and gms Systems While cat tools can be used on their own for smaller projects and accounts, enterprise-scale translation account work is often administered using a Globalisation Management System (gms). GlobalSight22 is an open-source gms developed by Welocalize. While it provides a web-based cat tool that can be useful for review or maintenance of translation memories, this feature proved 22

http://www.globalsight.com.

146

Moran, Lewis and Saam

unpopular with translators during translation. Many prefer to work offline. In 2013 Welocalize funded an interoperabity project between OmegaT and GlobalSight. As of OmegaT, 3.08 Update 1 round-tripping of GlobalSight localization kits is fully supported. This means OmegaT can parse the GlobalSight dialect of xliff while preserving metadata and leveraging large translation memories so any company that uses GlobalSight is now able to use OmegaT or iOmegaT on live production projects to capture translator productivity data. iOmegaT also works with other management systems used by enterprise (large volume) translation customers. Thus far productivity tests have been carried out on source files that originated in both sdl WorldServer and sdl  tms.23 We have developed a suite of interoperability utilities to process files for translation in iOmegaT. In general, if files can be translated with GlobalSight, sdl Trados Studio or OmegaT they can be translated with iOmegaT and with a little engineering effort most dialects of xliff can be processed in OmegaT. 4 Analysis In this section we will discuss some examples of analyses that can be generated on the basis of cat-uad. In the productivity tests for Dell and Autodesk (­ clients of Welocalize) from which the data was drawn all translators were aware that they were participating in productivity tests. All translators except Japanese in the Autodesk test translated the same source material. Both systems use Moses-­based24 Statistical mt with some enhancements. Spot-­checking was carried out to control for quality. Any ht or mt segment that required more than seven minutes of time (including self-review) was considered an outlier and discarded for analysis. ht segments accounted for 20% of the total segment count for Autodesk. The average total word count in the ht segment class was approximately 500 words for ht and 3,500 for mt per handoff to the translator. For Dell the ht ratio was increased resulting in approximately 1,400 words for ht and 2,300 words for mt. 4.1 Segment Level uad – mt/ht Ratio Figure 6.2 and Figure 6.3 show ht versus mt ratios for various translators and various language pairs where English is the source language for two ­productivity 23 24

http://www.sdl.com. http://www.stat-mt.org.

147

1800 1600 1400 1200 1000 800 600 400 200 0

ht mt Average mt Average ht

ch s1 ch s2 ch t1 ch t2 cs 1 cs 2 de 1 de 2 es 1 es 1 fr1 fr2 it1 it2 ja1 ja2 ko 1 ko 2 pl 1 pl 2 pt 1 pt 2 ru 1 ru 2

Words Per Hour

User Activity Data in cat Tools

Translator id

1800 1600 1400 1200 1000 800 600 400 200 0

ht mt Average mt Average ht

ch _s ch i_tr _s 1 i ch _tr2 _ti ch _tr _t 1 de i_tr _d 2 de e_t _d r1 es e_tr _e 2 es s_tr _e 1 s es _tr2 _lx es _tr1 _lx fr_ _tr2 ca fr_ _tr ca 1 fr_ _tr2 fr fr_ _tr1 fr pt _tr2 _b pt r_tr _b 1 pt r_tr _p 2 pt t_tr _p 1 t ru _tr2 _r u ru _tr _r 1 u_ tr2

Words Per Hour

Figure 6.2 ht versus mt translation throughput ratios for Autodesk data from 2012.

Translator id

Figure 6.3 ht versus mt translation throughput ratios for Dell data from 2012.

tests (Autodesk and Dell respectively). Both translation and self-review 6 time was accounted for. For a more in-depth discussion of the importance of accounting for self-review time when measuring post-editing productivity and a more detailed description of the Autodesk productivity test, see Moran et al. (2014). As can be seen by the averages, translators translated between 550 and 600 Words Per Hour (wph) for ht and mt on both accounts. However, in the case of Autodesk translators have an average throughput of nearly 800 (wph) when post-editing. In the case of Dell they are only slightly faster when post-editing. In other words, mt was more useful on the Autodesk account than on Dell (though this has improved since). From these graphs we can see how certain translators seem to be far more productive than others when post-editing. For example, though es1 in Autodesk can translate faster than average at 800 wph, s/he can post-edit at more

148

Moran, Lewis and Saam

than 1,400 wph. fr_ca_tr1 in Figure 6.4 for Dell is slower than average for ht but second fastest for mt. The same source files were translated by fr_ca_tr2 but ht and mt speeds were almost the same. This is quite a typical result. It seems that mt is not a technology that suits all translators under all circumstances. In the Autodesk study in Figure  6.2 almost all translators were faster because of mt. In the Dell study in Fig 6.3, 6 translators were slower with mt and most were largely unaffected. The small net gain resulted from a small number of translators like fr_ca_tr1, pt_br_tr2 and ru_ru_tr1. This graph illustrates the point that unless mt is high in terms of utility, it is hard to achieve a return on investment for mt without being able to identify translators who benefit from mt like these ones. The graphs also show that even when the source material and mt is the same, translators vary greatly in terms of their translation speed with and without mt. As in-house translators are fewer in number than freelance translators (­Garcia, 2007) it is particularly important to be able to have a means of identifying freelance translators whose translation speed is enhanced by mt. We have found that measuring ht versus mt ratios over short periods leads to unreliable statistics due to problems associated with small sample sizes. For this reason, it is less costly and thus preferable to be able to measure pe speed over days and weeks in production translation projects using shunted productivity tests like the ones presented here or, ideally, to measure it intermittently or on an ongoing basis during production translation without the project management overhead of a productivity test. A good example of a company taking this approach is described by ibm in Roukos et al. (2012). 4.2 Keystroke uad – Typing Speed Figure 6.4 and Figure 6.5 show typing speed for Autodesk and Dell translators. We have restricted the languages to non-Asian languages as Input Method Editors (imes), commonly used for these languages. We measured typing speed by identifying unbroken bursts of at least 5 keystrokes where the time between keystroke is under 300 ms. In line with standard practice when measuring typing speed, we consider a word to be 5 characters (though we consider a keystroke to be a character). Using these parameters, we were able to identify approximately 1,096 samples on average per translator with a minimum of 193 samples on the Dell data. For Autodesk the numbers were similar, with an average sample count of 1,149 and a minimum sample count of 247. For Dell the standard deviation for Words Per Minute was 806 versus 755 for Autodesk. ht and mt made little difference in terms of typing speed, where ht was faster by two wph in both cases. We did not correlate these typing speed numbers with other more traditional measures of typing speed used to measure the performance of

User Activity Data in cat Tools

149

120 Words Per Minute

100 80

TYPING SPD (WPM) TS HT (WPM) TS MT (WPM) Average

60 40 20 0

cs1 cs2 de1 de2 qa1 es1 fr1 fr2 it1 it2 pl1 pl2 pt1 pt2 ru1 ru2 nonime Translator id

Figure 6.4 Typing speed in words per second on Autodesk 120 100 Words Per Minute

80

60

TYPING SPD (WPM) TS HT (WPM) TS MT (WPM) Average

40 20

de

de

_d

e_

t _d r1 e_ es tr2 _e s es _tr1 _e s_ es tr2 _lx es _tr1 _lx _ fr_ tr2 ca fr_ _tr1 ca _t fr_ r2 fr_ fr_ tr1 fr_ pt tr2 _b pt r_tr _b 1 r_ pt tr2 _p pt t_tr1 _p t ru _tr2 _r u ru _tr1 _r u_ tr2

0

Translator ID Figure 6.5 Typing speed in words per second on Dell

t­ypists, as we were more interested in relative speed between translators. As  we  only  ­measure  fast bursts we would expect that the estimates shown above would be higher than these measures. Interestingly, we found no correlation between typing speed and overall productivity. Even the slowest typist above (pt_pt_tr1 in Figure  6.5) would translate 3,600 wph instead of 400 wph if all that were involved in the act of translation were typing. It seems the secret to fast translation is to be able to think fast. However, this result should not be interpreted as a suggestion that learning to touch-typing is time wasted. In fact, we take the opposite view. Predictive typing technology like autosuggest in Trados, Muses in MemoQ and a similar (though less automated) function in OmegaT where proposals are fleeting presented on the screen are already commonly used by many translators and interactive mt25 also provides proposals that change frequently. These technologies may be harder to use for translators who have to frequently switch their gaze from screen to keyboard as proposals near the cursor in the target segment only remain in view for a very short period of time. 25 e.g. http://www.statmt.org/iamt.

150 5

Moran, Lewis and Saam

Conclusion and Further Work

In this paper, we discussed some of the previous work that has been carried out to gather and analyse uad in cat tools. We discussed some of the advantages of desktop-based cat tools and why we chose OmegaT as the cat tool to adapt with the uad logging feature. We also discussed why the software is based on an offline or desktop-based cat tool and presented some analyses gleaned from the data. These analyses showed that there is a good deal of variation in the impact that mt can have on the productivity of individual translators and that typing speed and overall productivity are not correlated across a large set of translators. In conclusion, we can state that uad in cat tools is a source of useful information that can be used to improve translation workflows that involve mt in cat tools. This research is part of an ongoing effort to develop software and ­approaches to help quantify the impact on translator productivity of computational linguistic technologies and training in their use. In the next subsections, we will outline some of the next steps planned. 5.1 A Common Standard for cat-uad cat-uad provides a very useful means of tying individual segment sessions with information about mt for A/B testing purposes like ht/mt but it is useful in general for A/B testing. The same technique is often used by website or mobile phone app developers to test new features. It can also provide useful information for CAT tool developers so they can replicate problems. Though a number of research projects mentioned previously have made use of User Activity Data, as yet no common standard for this kind of data exists. This is not a trivial problem. cat tools vary in terms of features and functionality but they overlap in more ways than they differ so a common format seems like a sensible approach. As various technologies like full-sentence mt, interactive mt and Automatic Speech Recognition already make a big difference to words-per hour productivity, translators, translation agencies and end buyers of translation who use various cat tools and various mt engines would benefit from many of the reports that cat-uad facilitates. However, to achieve this a translator’s privacy should be respected. Figure 6.6 shows an example of how this could be implemented in a cat tool on the basis of a wireframe diagram. The option of not sharing linguistic data would solve the problem of sharing data with a third-party (e.g. the cat tool publisher, a company specialized in the analysis of User Activity Data or a researcher), particularly for individual translators. uad containing linguistic data can, of course, be shared with a contracting agency. If a translator wishes to remain anonymous or an agency wishes to preserve translator anonymity

User Activity Data in cat Tools

151

URL to reporting service Username Password Send User Activity Data Remain anonymous Don’t send linguistic data

Figure 6.6 A wireframe mockup to illustrate a privacy model in a proprietary cat tool.

from a client (e.g. a larger translation agency or end buyer) it would be possible to do so using an anonymous id. Privacy models in desktop-based cat tools from a translator perspective are discussed further in Moran and Lewis (2015). 5.2 Instrumentation in Other cat Tools We are currently developing the software further as a commercial offering including a web application to manage the data discussed here and display reports similar to those shown here under the working title WordFace Analytics.26 In addition to OmegaT, we have developed a plugin for Trados Studio 2014+ to gather uad and other proprietary cat tools have shown interest in the concept. Our aim is to develop a commercial product based on a computing architecture called an Extract-Transform-Load (etl) pipeline that can be used to visualize uad across a range of cat tools to make it easier for buyers and translators to assess the utility of mt and other technologies and techniques that can enhance translation speed. Acknowledgements This research is supported by the Science Foundation Ireland (Grant 12/CE/ I2267) as part of cngl, the Centre for Global Intelligent Content (www.cngl. ie) at Trinity College Dublin and as part of Technology and an Innovation Development Award Feasibility Grant (12/TIDA/I2424) titled ‘iOmegaT – An Instrumented Replayable Computer-aided Translation Tool’. The iOmegaT Workbench developed under this grant has been licensed to a small number of companies including Welocalize, and Hewlett Packard. Welocalize were also closely involved in the development of the software. 26

http://www.wordface-analytics.com.

152

Moran, Lewis and Saam

References Aziz, W., Castilho, S. & Specia L. (2012). PET: A Tool for Post-editing and Assessing Machine Translation. Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12). Istanbul, Turkey. Bota, L., Schneider, C. & Way A. (2013). COACH: Designing a new CAT Tool with Translator Interaction. Machine Translation Summit XIV, Main Conference Proceedings, pp. 13–320, Nice, France. Brooke, J. (1996). SUS-A quick and dirty usability scale. In Jordan P. et al. (Eds.) Usability Evaluation in Industry, pp. 189–194 CRC Press. Carl, M. (2012). Translog-II: A Program for Recording User Activity Data for Empirical Reading and Writing Research. Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12). Istanbul, Turkey. Garcia, I. (2007) Power shifts in web-based translation memory. Machine Translation, 21 (1), 55–68. Moran, J., Lewis, D. & Saam, C. (2013). Analysis of Post-Editing Data: A Productivity Field Test using an Instrumented CAT Tool In O’Brien, S, Winther Balling, L., Carl, M, Simard M. & Specia, L (eds.), Post-Editing of Machine Translation: Processes and Applications. Newcastle upon Tyne, United Kingdom: Cambridge Scholars Publishing. Moran, J. & Lewis, D. (2015). Towards a CAT tool agnostic standard for User Activity. Localisation Focus, 14 (2), pp. 56–61. Moran, J., Lewis, D. & Saam, C. (2014). Towards desktop-based CAT tool instrumentation. AMTA 2014 Workshop on Post-Editing Technology and Practice (WPTP 2014), pp. 99–110. Vancouver, Canada. Plitt, M. & Masselot, F. (2010). A Productivity Test of Statistical Machine Translation Post-Editing in a Typical Localisation Context. The Prague Bulletin of Mathematical Linguistics, 93, pp. 7–16, Prague. Roukos, S., Ittycheriah, A. & Xu, J.-M. (2012). Document-Specific Statistical Machine Translation for Improving Human Translation Productivity. Computational Linguistics and Intelligent Text Processing, Springer, pp. 25–39. Tatsumi M. (2009). Correlation Between Automatic Evaluation Metric Scores, Postediting speed, and Some other Factors. Proceedings of MT Summit XII, Ontario, Canada, pp. 332–339. Zhechev, V. (2012). Machine Translation Infrastructure and Post-editing Performance at Autodesk. AMTA 2012 Workshop on Post-Editing Technology and Practice (WPTP 2012), pp. 87–96. San Diego, USA.

chapter 7

Computer-assisted Interpreting: Challenges and Future Perspectives Claudio Fantinuoli Abstract During the last decades, information technology has played a central role in the language services industry. Translators and technical writers take advantage of dedicated software to reuse already translated texts, to adhere to a customer-specific corporate language, to grant terminology consistency, and so forth. The final goal is to increase quality and productivity. Even if information technology did not have the same impact on conference interpreting, also the profession is undergoing some changes. Computer-­assisted interpreting (cai) tools have entered the profession only in recent years, but other, more general resources had already influenced the way interpreters work. This is not only challenging the way interpreting is performed, but it may have an impact on the cognitive processes underlying the interpreting task, even on some basic assumptions and theories of interpreting, for example the cognitive load distribution between different tasks during simultaneous interpreting. Yet, the academic debate is starting to take notice of these changes and their implications only now. As a consequence, it almost failed to shed light on and address the challenges that lay ahead: there have been relatively few empirical investigations on the impact of cai tools; interpreting models have not been adapted accordingly; the didactics of interpreting has received almost no new technologies in their curricula and no proposal has been advanced to increase the quality of cai tools and to meet interpreters’ real needs.

Keywords computer-assisted interpreting – productivity – cai tools – information technology – interpreters – interpretation

1 Introduction During the last decades, Information and Communication Technology (ict) has played a central role in many language-related professions: translators and © koninklijke brill nv, leiden, ���8 | doi 10.1163/9789004351790_009

154

Fantinuoli

technical writers take advantage of dedicated software to reuse textual parts or already translated sentences, to adhere to a customer-specific corporate language, to grant terminology consistency, and so forth. With their final goal to increase quality and productivity, the use of such tools has become so ubiquitous that their presence is mostly taken for granted. On interpreting, however, ict did not have the same major impact as on other professions, as confirmed by the fact that the manner in which interpreting is performed today has basically remained the same over the years. Yet, the profession also underwent some important changes with regard to new technological advances. The World Wide Web with its unprecedented richness of subject and terminological information, for example, has changed the way interpreters prepare their assignments (cf. Kalina, 2009; Fantinuoli, 2011), allowing them to deal more effectively with the complexity and variability of the topics they are called upon to interpret (cf. Tripepi Winteringham, 2010). Laptops and tablets in the booth allow interpreters to look up reference material and specialised terminology while interpreting (cf. Fantinuoli, 2016; Costa et al., 2014, 2018/forthcoming; Will, 2015), with implications both on the cognitive processes underlying the interpreting task as well as on the preparatory activity needed to perform well. Finally, the use of remote interpreting has been adopted in some interpreting settings and its diffusion is increasing (cf. Mouzourakis, 1996; Riccardi, 2000; Andres and Falk, 2009). Notwithstanding the above-mentioned uses of technology in the modern interpreting workflow, the attitude of many practitioners towards interpreter-­ specific technologies is rather negative. The fact that many of them have shown some degree of reluctance to the use of ict (cf. Tripepi Winteringham, 2010) is illustrated by the results of several surveys in professional settings (cf. BerberIrabien, 2010; Valentini, 2002) and individual papers (cf. Roderick, 2014).1 Pym (2011: 4) describes the general attitude of professional interpreters towards technological transformation with the following words: ibm headphones and wires enabled conference interpreters to form a profession […] So what happens when the technology moves to the next level, in this case allowing for remote video-interpreting […]. The established conference interpreters will swear until they are blue in the face that quality work only comes from their being in attendance at the conference, to witness the speaker’s every gesture, to imbibe the atmosphere 1 Roderick (2014, 18), for example, repeatedly speaks of ‘alienation due to the use of new technology’ and that ‘[it in the booth] can lead the interpreter to lose sight of the first aim of interpreting as we learn it, namely conveying meaning and facilitating communication’.

Computer-assisted Interpreting

155

of the event, to hobnob with the eminences they are called upon to render. No matter the empirical evidence for or against, the professional group that gained its mystique with an old technology will resist the advance of the new technology, at least until it can turn the new to suit its own strategic purposes. Resistance to technological change is usually a defense of old accrued power, dressed in the guise of quality. The lack of interest for or the aversion to new technologies is not only limited to the practitioners. Judging by the small number of studies on technologies published to date, a similar attitude seems also to be typical for the academic debate, as I will point out in Section 2. When discussing ict in the field of interpreting it is important to differentiate technologies depending on the level at which they interact with the interpreter and the interpreting task. I would like to propose a clear distinction here between two groups of technologies which I will call, for lack of better terms, the process-oriented on the one hand, and the setting-oriented technologies on the other. The first group comprises terminology management systems, knowledge extraction software, corpus analysis tools and the like. They are process-oriented because they are designed to support the interpreter during the different sub-processes of interpreting and, consequently, in the various phases of an assignment, i.e. prior to, during and possibly after the interpreting activity proper, independent of the modality. They are an integral part of the interpreting process and are directly linked to and might have an influence on the cognitive processes underlying the task of interpreting. Process-oriented technologies are the distinctive element of computer-assisted or computeraided interpreting (cai), which can be defined as a form of oral translation, wherein a human interpreter makes use of computer software developed to support and facilitate some aspects of the interpreting task with the overall goal to increase quality and productivity. In this context, cai tools are all sorts of computer programs specifically designed and developed to assist interpreters in at least one of the different sub-processes of interpreting, for instance knowledge acquisition and management, lexicographic memorisation and activation, etc. The second group, the setting-oriented technologies, comprises ict tools and software ‘surrounding’ the interpreting process proper, such as booth consoles, remote interpreting devices, training platforms, etc. They are settingoriented as they primarily influence the external conditions in which interpreting is performed or learned, but can be considered somewhat marginal with respect to the main cognitive processes underlying interpreting. Settingoriented technologies were central in the development of some interpreting

156

Fantinuoli

modes (one thinks of simultaneous interpreting, for example) and future developments, for example in the area of remote interpreting, may have a major impact on the interpreter profession, its status and the working conditions, but they will not radically change the core upon which the activity of interpreting is based.2 This classification is obviously an over-generalisation. Every technology could be placed in a continuous scale between these two extremes, depending on the perspective of the researcher and the way interpreters use them. So, for example, it is not easy to decide which of the two categories should be addressed by the Consecutive Pen3 (Orlando, 2014). The decision will clearly vary according to the fact that the Pen is used in the didactics of consecutive interpreting to capture simultaneously the video of the notes and the audio, in order for teachers to provide better advice to their students, or as a hybrid mode of interpreting (the so called Consec-simul with notes). With further advances in both process and setting-oriented technologies and due to the fact that we are getting accustomed to using digital devices in almost all walks of life, it is plausible to expect that the influence of both groups of technologies on all aspects of interpreting – profession, didactics and research – will increase in the years to come. The focus of this chapter, however, is solely on process-oriented technologies, i.e. cai tools, as this appears an underrepresented subject within interpreting studies in general, and technologyrelated studies, in particular. The fact that interpreters increasingly rely on software, both interpreter-specific and not, to support their daily professional life (just think about the presence of laptops and tablets in the booth) makes new technology in interpreting an interesting research subject which requires to be analysed in detail. The rest of the chapter is structured as follows: Section  2 gives a brief overview of the major studies on information and communication technology. Section 3 introduces ­process-oriented ­technologies in 2 To draw a parallel with the translation profession, process-oriented technologies (cai tools) can be considered the interpreter’s counterpart of computer-assisted translation (cat) tools, both having an influence on the translation process and product, on the workflow, etc. Setting-­oriented technologies are similar to translation process external technologies, like computers, e-mails and so forth, which have obviously revolutionised the way translators work, their status etc., but have only marginally changed the translation process and subprocesses (see for example Austermühl, 2001). 3 This technology refers to a digital pen used to take notes and to capture data on a special paper. It integrates a built-in microphone, a speaker and an infra-red camera. A program synchronises what is being recorded as handwriting with the audio recorded at the same moment. The user can tap on a word on the notebook to hear the part of the speech related to it.

Computer-assisted Interpreting

157

more detail, distinguishing between first and second-­generation cai tools and presenting the features solutions available at the moment on the market offer. Section 4 presents the major challenges that interpreting studies need to address to bridge the emerging gap between the developing profession and the research activities in this discipline, arguing for more empirical research to understand the influence of cai tools on the interpreting workflow and to guide the future development of new tools. Finally, the conclusions summarise the topics introduced in this chapter and present some future perspectives. 2

New Technologies and Interpreting Studies

In the past, the academic interest for the topic of ict in the domain of interpreting has been very marginal and the number of studies published very small (Berber-Irabien (2010) points out that only 1.12% of titles included in the cirin Bulletin from 2003 to 2008 were technology-related). The situation is now slowly changing and the interest in new technologies has increased over the years. In the last cirin Bulletin (Gile, 2015b), for example, 7 items out of 64 were explicitly dedicated to some technological aspect of interpreting. The three main areas of interest are remote interpreting,4 especially telephone and video interpreting, computer-assisted interpreter training5 and computer-­ assisted interpreting software. A bibliometric analysis shows that the majority of studies conducted to date concentrate on the first two areas, while studies focussing on our object of interest, software designed to assist interpreters, play a secondary role. The first publications dealing in some way with cai tools can be dated back to the period around the turn of the millennium, but the subject has started to gain a significant momentum only during the last few years after the first interpreter-specific programs had entered the market. Some papers have pointed out how general, not dedicated, tools such as search engines, online glossaries and so forth have changed the way interpreters access and elaborate knowledge (Kalina, 2009, 395); others have analysed the terminological competence interpreters need and how it can be managed with the help of computer programs (cf. Rütten, 2007; Will, 2009); others have proposed and designed programs to help interpreters manage and access c­ onference-related

4 For an overview see Tripepi Winteringham (2010) and Andres and Falk (2009). 5 For an overview see Carabelli (1997), Gran et al. (2002), Sandrelli and Jerez (2007) and Lim (2014).

158

Fantinuoli

terminology and information (cf. Fantinuoli, 2009; Stoll, 2009; Fantinuoli, 2012); some others, finally, have reviewed the tools available on the market (cf. Costa et al., 2014). cai tools are strictly related to terminology and knowledge acquisition and their integration in the interpreting process. Consequently, particular attention has been devoted to these topics. Rütten (2007) analysed the role of and relationship between information and knowledge from the point of view of conference interpreting. Extending the classical concepts of terminology and terminology management to the broader field of knowledge and information management, she describes knowledge as a combination of language, content and situational knowledge, pleading for a knowledge representation in the classical model of Wüster (Rütten, 2007: 83): concept, object and designation and their reciprocal relation. Rütten articulates the workflow of knowledge acquisition on the basis of Kalina’s phases with dedicated ‘learning’ operations for each phase. Eventually, she identifies a progression during the course of preparation in the data-information-knowledge continuum (ibid: 113): from simple and sparse data to the establishment of a complex knowledge system. In a case study, the author analyses the entire process of interpreting (from the assignment to the post-elaboration of information) under the perspective of information and knowledge processing. Based on her observations, she speculates on the manner in which a computer program could be integrated in the interpreting’s workflow. She proposes a software model that should support the interpreter during the entire interpreting process. It consists of three components: • a language-oriented terminology module • a content-oriented documentary module • a situation-oriented overview module As Rütten (2007) points out, the software’s single components are not to be seen as completely independent from each other, as terminology data always contains extra-linguistic information, documents are also a source for terminology, etc. The idea of developing a tool for all interpreting phases, which integrates both linguistic as well as non-linguistic information, laid the foundations for extending the scope of first-generation cai tools, which, as pointed out in Section 1, were only focussed on the management of multilingual lists of word equivalences. Another aspect very much debated academically is the lexical and conceptual gap between interpreters and event participants, especially when working on specialised subjects (Morelli and Errico, 2007). This depends on the fact

Computer-assisted Interpreting

159

that interpreters generally do not share the same level of specialised expertise as the conference participants (cf. Will, 2009; Fantinuoli, 2011; Fantinuoli, Fantinuoli, 2016, 2017) to fill this gap, interpreters do preparatory work prior to the beginning of the interpreting task. When preparing for an assignment, they typically use the reference material at their disposal to gain as much information on the subject as possible. A central point of this preparatory work is the collection and management of terminological information. It is evident, as pointed out by many scholars (cf. Morelli and Errico, 2007), that terminology plays a central role in any language mediated activity, as ontologies and term collections are required to create the knowledge system needed to achieve a precise and shared comprehension. According to generally accepted terminological standards, the collected information should be organised in a complex terminological repository, as introduced in Arntz et al. (2009). Yet, interpreters’ glossaries are generally a context-free list of terms and their translations (Will, 2009). They are concise, complied according to personal needs and contain also very infrequent terms. This praxis poses several problems: on the one hand, simple word equivalences in two or more languages do not allow a clear term disambiguation. On the other hand, compiling glossaries – even if they are reduced to mere terms and their possible equivalence – is a time-consuming task. In fact, it is not possible to know exactly beforehand, i.e. before the end of the event, what will really be needed during the interpreting task (for example infrequent terms). The obvious tendency is to invest a lot of time processing terminological information that will never be used in the course of interpretation. As a consequence, interpreters need to anticipate topics and settings of the assignment, resolving beforehand the possible problems that may arise during the interpreting task. This calls for a very effective and specific way to constitute the relevant terminological and encyclopaedic knowledge. In that regard, Will (2009) describes the complexity of the knowledge systems that must be mastered by interpreters in order to perform an interpretation of excellent quality and proposes an interpreting-oriented terminology approach, the so called dot.6 He applies the context-related term model of Gerzymisch-Arbogast (1996), which considers possible deviations from the unique correlation between concept and designation, as defined by Wüster (1991). According to this principle, since terminology is embedded in texts, it can be ‘contaminated’ by the knowledge system itself. According to Will, these potential deviations of the meaning are not taken into consideration in 6 dot is the abbreviation of the German ‘Dolmetschorientierte Terminologiearbeit’ (Interpreteroriented terminology work).

160

Fantinuoli

context-independent word lists (simple glossaries), and this may lead to incorrect translations, for example in case of polysemy or terminologisations (Will, 2009: 6). In order to solve this problem, Will pleads for what he defines as ‘detective work’, an approach to terminological work which allows interpreters to represent terms in context: from the term and the term definition to the specific knowledge system. This relationship can be constituted by comparing an individual term structure to its systematic reference meaning. The result consists of Terminological Knowledge Entity (tke), the ‘smallest complete knowledge unit for understanding and producing technical texts’ (Will, 2007: 69). Grouping together the individual entities established in tkes, it is possible to constitute complex structures which are the basis of text comprehension and production. If such mental structures of knowledge are dynamic, they may allow interpreters to give meaning to what they hear, for example, through principles such as deduction, inference and anticipation (Morelli & Errico, 2007).7 In most general terms, preparation must allow interpreters to gain a systematic overview of the knowledge systems and the terminologies involved in the event as well as their ranking in terms of importance and priority. The knowledge systems that emerge can ultimately be recorded in a glossary and used during interpretation. As far as cai tools are concerned, many authors point out how they could offer a practical support to better rationalise and organise the process of knowledge constitution and its use before and during the task of interpreting (cf. Will 2009; Rütten 2007; Stoll 2009; Tripepi Winteringham 2010). Even if the interest among practitioners, especially among the new generation, and some scholars seems to have increased during the last years, the overall impression is that the applied use of cai tools has remained marginal in the growing body of interpreting studies, as confirmed by the small number and scope of publications dedicated to the topic. This is particularly true for experimental studies. In the context of specific tools dedicated to preparation, Xu (2015) experimentally investigated how a corpus-based terminology preparation, which integrates the building of small comparable corpora as well as the use of automatic term extractors and concordance tools, can improve trainee interpreters’ performances. The results show that the test groups consistently had better 7 For example, Chernov (2004) viewed prediction or the so-called expectation-based processing as being fundamental to the interpreting process. He distinguishes between message elements that are new and those that are already known (thema-rhema progression) and argues that the attention of interpreters is on the new components of the message which are processed on the basis of probability prediction based on available knowledge. This knowledgedriven processing is common to many interpreting models.

Computer-assisted Interpreting

161

terminology performance during simultaneous interpreting: they interpreted more terms correctly, had higher terminology accuracy scores and made less term omissions. Furthermore, they also had higher holistic simultaneous interpreting performance scores than the control groups. These results suggest that the Corpus Driven Interpreters Preparation (cdip) (Fantinuoli, 2006; Gorjanc, 2009) can help interpreters improve their performance when working on specialised topics. In order to implement cdip, Fantinuoli (forthcoming) proposes a corpus-based cai tool specifically developed to support interpreters during the preparatory phase. In the same context, two studies have also focused on automatic corpus construction and terminology extraction: Fantinuoli (2006) proposed an automatic terminology extraction in order to provide interpreters with a preliminary list of highly specialised monolingual terms for the conference preparation while Xu and Sharoff (2014) evaluated and assessed the amenability for interpreters of several term extraction methods. Even if the accuracy of the extraction methods is not perfect, both studies stated that the use of small specialised corpora and automatic terminology extraction may facilitate interpreters in their preparation. The only papers focussing on the implementation of real cai tools are dedicated to the projects InterpretBank (Fantinuoli, 2009, 2011, 2012, 2016), Lookup (Stoll, 2009) and CorpusMode (Fantinuoli, 2017). In these studies, the authors describe the development and features of the three cai tools, discussing the theoretical framework for the implemented solutions: the actual requirements in terms of linguistic and extra-linguistic knowledge needed by professional interpreters and how they can be constituted are analysed; a general structure of the interpreters’ workstation is presented on the basis of advances in terminology management and information retrieval approaches; finally, the components of the workstation are implemented. More recently, first attempts at empirically analysing cai tools, both in the context of interpreting quality as well as in the didactics, has been made. Gacek (2015), for example, tried to answer the question whether the use of terminology tools in the booth improves the interpreter’s performance in terms of terminological quality. Based on the experimental data and comments obtained, the study shows that the use of interpreter-specific terminology software8 during the interpreting task is more efficient in improving the terminology rendition (correctness and completeness) than other solutions (paper glossaries). Even if the study is interesting as it empirically suggests that the negative attitude of some practitioners, claiming for example that such tools, at least in the booth, are somewhat unnatural (Tripepi Winteringham, 2010), is unfounded, it lacks 8 The tool used in the experiment was InterpretBank.

162

Fantinuoli

of a robust experimental setup (the testers were unexperienced in the use of the tool, the text was manipulated in order to have a small set of terminological stimuli which did not allow for other translation strategies to be applied, etc.) and statistical analysis. To overcome these shortcomings, more controlled tests with different settings (tester, stimuli, etc.) and advanced statistical measures are needed. With this in mind, Biagini (2016) compared the performance of interpreters dealing with a specialised text characterised by a high terminological density, both with a cai tool9 as with a paper glossary and developed a rigorous experimental setting in order to control the independent variables at stake (the testers were selected according to stringent criteria, i.e. they had gone through the same amount of practice with the tool, were provided with the same glossary etc.) and used statistical tests to grand for the reliability of the data. The analysis of the results shows that under certain conditions cai tools improve the overall interpretation quality in terms of terminology accuracy and completeness of the interpreted text. Another almost unexplored area has to do with the didactics of cai tools. As some universities recognised the need to adapt their curricula to the emerging use of new technologies in interpreting, a pilot study was conducted at the University of Bologna to understand how to integrate cai tools in the curriculum (Prandi, 2016). The aim of the experimental study was to collect information on the students’ approach to such tools10 in the booth. The analysis of audio/video as well as keylogging data shows that experience plays a key role in helping user integrate the tool in their workflow and that most testers were able to conduct effective terminology searches (with an average 90% rate of terms correctly identified). As a drawback, the author stressed the tendency of some testers to rely too much on the software, with obvious negative consequences on the overall performance. The result of this first study seems to indicate that cai tools can be successfully integrated in the curricula of future interpreters, provided they already have robust experience in interpreting (for example at the level preceding the final exams, as the texts need to be of a rather specialised nature) and enough time to understand how to adapt their interpreting strategies to the use of the tool. Similar conclusions were drawn by Biagini (2016) who correlated the empirical findings of his experiment with the responses provided by the participants in a questionnaire.

9 10

The tool used in the experiment was InterpretBank. The tool used in the experiment was InterpretBank.

Computer-assisted Interpreting

163

3 Process-oriented cai Tools: An Overview As pointed out above, all language-related professions have been influenced by computer applications over the years. Differently from translation and technical writing, where a plethora of software has been developed to assist humans during their work, the software targeting interpreters remains very limited in number and scope. They have become more popular in recent years, but their impact on the profession has so far been marginal. There are several interdependent reasons for this: • many professional interpreters have been generally reluctant to accept the idea of software supporting them during the interpreting process, maybe because this could possibly raise doubts about the pure intellectual activity of interpreting; • many practitioners consider the use of cai tools in the booth as unnatural, the reason being that it is a time-consuming and distracting activity (cf. Tripepi Winteringham, 2010, 4). • the cognitive processes of interpreting, especially simultaneous interpreting, have not been completely ascertained (cf. Will, 2009, 19), making it quite difficult to design software able to smoothly integrate with the interpreting process; • too little effort has been invested in systematically investigating the role of terminology and knowledge acquisition in the interpreting process and, most importantly, the role of software tools in the interpreting process; • from an economic point of view, interpreting plays a marginal role in the language industry. Consequently software houses committed to the development of tools for language professionals have never invested time and money in the design and implementation of software for interpreters; • differently to the translation industry, where cat tools are recognised as a cost cutting factor, the economic gain in using dedicated software is not clearly measurable; • universities offering courses in conference interpreting do not usually, or only marginally, introduce novice interpreters to the topic of computer-­ aided interpreting. Notwithstanding the above-mentioned reasons, a small number of pieces of software have been developed during the last 15 years or so. Differently to cat tools, which are nowadays very similar in terms of design and functionalities, dedicated tools for interpreters are quite heterogeneous. This can be mainly

164

Fantinuoli

related to two reasons. Firstly, cai tools are relatively new and less widespread; this has deprived them of many test and improvement phases, which are crucial to achieve software maturity and find the golden standard in terms of functionalities desired by the professional group. Secondly, no major investigation has been attempted by scholars to understand the type of software and functionalities required by interpreters in order to optimise their performances. In fact, as interpreting is still considered a rather individual task, tool design nowadays reflects more the ideas and habits of the respective developer, generally an interpreter himself, than the needs of the interpreter community. cai tools can be distinguished according to several criteria, for example the workflow phases covered or the presence of a simultaneous modality, which takes account of the time constraints of simultaneous interpreting, as this is a crucial element of the profession. If it is going to be used in the booth, the terminology-lookup mechanisms needs to behave quite differently from that implemented in translation-oriented terminology tools. In order to reduce the cognitive load needed to look up a term, cai tools may in fact use algorithms designed to reduce the number of strokes needed to input the search word, to correct typing errors, to discriminate results according to the conference topics, their relevance, etc. Depending on their architecture and functionality spectrum, cai tools can be broadly divided into two groups: first-generation cai tools, proposed for the first time about 15 years ago and, more recently, second-generation cai tools. First-generation tools are programs designed to manage terminology in an ­interpreter-friendly manner. Being very simple in terms of architectural design and functionalities, they support interpreters in managing multilingual glossaries similar to ms Word or Excel lists, but do not envisage any other specific supporting activity of the interpreting process (such as information retrieval). The list of first-generation software is comprised of Interplex,11 Terminus,12 Interpreters’ Help,13 LookUp and DolTerm. Only Interplex, Terminus and Interpreters’ Help are still maintained and are commercially available. Designed to manage multilingual glossaries, they are basically graphic interfaces to store and retrieve terminological data from a database. They are different from terminology management systems for terminologists and translators as they use simple entry structures and offer basic functionalities to look up glossaries in the booth. All tools can store additional information to the terms in explicitly or implicitly dedicated fields and allow the categorization of entries through 11 12 13

http://www.fourwillows.com/interplex.html. http://www.wintringham.ch/cgi/ayawp.pl?T=terminus. http://www.interpretershelp.com.

Computer-assisted Interpreting

165

a one-tier categorisation system (Interplex, Interpreters’ Help) or a multi-tier system (Terminus, LookUp and DolTerm). In order to search the database, the user enters a string of text (the term or part of it) in the search mask and presses the enter key. None of the first-generation tools implement any sort of advanced search algorithm to take account of the time constraints of the interpreting task, such as misspelling correction, progressive search in one or more glossaries, etc. As a simple and user-friendly solution to store and access the terminology in the booth during interpretation, first-generation cai tools can be treated as a simplified version of traditional terminology management systems (such as Multiterm) with an easy-to-use search functionality. If such tools can undoubtedly be considered a first step towards the optimisation of some aspects of the interpreting task (for example, making the use of paper glossaries in the booth superfluous and making it easier to reuse previously compiled glossaries), they are far from becoming a complete interpreter’s workstation which is able to take into account the other aspects of the interpreting process, as indicated by the literature summarised in Section 2. With the goal of extending the limited scope of first-generation cai software, second-generation tools build on first academic research and investigations on terminology and knowledge management. They present a holistic approach to terminology and knowledge for interpreting tasks and offer advanced functionalities that go beyond basic terminology management, such as features to organise textual material, retrieve information from corpora or other resources (both online and offline), learn conceptualised domains, etc. The second-generation tools developed to date are InterpretBank14 and Intragloss.15 They exploit more advanced computational approaches to offer professional interpreters a supporting toolset suitable for different phases of the interpreting process, from preparation to interpretation in the booth. InterpretBank is a prototype developed between 2008 and 2012 as part of a doctoral research project at the University of Mainz/Germersheim (Fantinuoli, 2009, 2012, 2016). The modular structure of the tool aims at covering the different phases of the interpreting task, as defined by Kalina (2007). For the preparatory phase, for example, it comprises automatic translation and terminological retrieval from online resources, which helps to speed up the glossary creation procedure, the integration of the preparatory material handed out by the conference organiser, a concordancer to look up terms in real context (in sentences extracted from the conference material), a memorisation utility to learn the glossary prior to the conference and so forth. Intragloss focuses 14 15

http://www.interpretbank.com. http://www.intragloss.com.

166

Fantinuoli

on the preparatory phase of an assignment and presents a novel approach to glossary building, as it is based on the interaction between preparatory texts and the terminological database. On one hand, it allows filling a glossary by highlighting a term in the preparatory document and searching for its translation in online resources such as glossaries, databases, dictionaries, etc. On the other, it automatically extracts all the terms from the domain glossary that appear in the preparatory documents, thus directly linking the texts with the available terminology repository. If classified according to the presence of the simultaneous modality, the only tool which implements a solution for looking up terms in the booth, taking into consideration the time-constraints and peculiarities of the simultaneous modality, is InterpretBank. The tool uses a dedicated utility to increase its usability in the booth by reducing and focussing the mass of information at the interpreter’s disposal. The conference modality seeks to diminish the cognitive load needed to query the database by means of fuzzy search, which acts as an error correction mechanism for misspelling in the word typed by the interpreter or present in the glossary, stopwords exclusion for reducing the number of matches displayed as a query result, dynamic search in the glossary to avoid the use of the enter button, progressive search in a hierarchical structure of glossaries according to their relevance for the actual conference and so forth. 4 Investigating cai Tools: The Challenges That lie Ahead Computer-assisted interpreting is slowly changing the interpreting landscape and the statements of some scholars are very clear with regards to the potentiality of cai tools: It may be thus assumed that, in the practice of the profession, interpreting rendition may benefit from the use of technological aids. cai may indeed be a major breakthrough in the interpreting field as it may provide a powerful solution enabling interpreters to improve both the quality and productivity of their interpretation services. Tripepi Winteringham, 2010: 3

However, quality and productivity shifts related to the introduction of cai tools in the interpreting workflow have not been the object of scientific investigation so far, as pointed out in Section 2. Research, especially with empirical and quantitative methods, is therefore urgently needed. Its goal should be to analyse the positive and negative influence of cai tools in the interpreter’s

Computer-assisted Interpreting

167

­performance and their role in the cognitive processes underlying the interpreting task. The questions that should be addressed, among others are: as typing an unknown word on a keyboard requires an additional time-consuming effort, how would this affect the efforts balance of interpreting? Could the activity of searching for a term result in distraction and loss of concentration for the interpreter? For what kind of texts should interpreters access terminology in real-time? First empirical tests have been conducted at some universities (see ­Section 2), but a lot of preliminary work is still required, primarily with respect to the ‘methodology’ to be adopted in the experimental setting. This is somewhat crucial if one considers the high number of variables at stake in the interpreting process and, consequently, in the design of experiments (interpreter’s experience, personal attitude towards cai tools, text typology, to name but a few). The fact that we still need to understand what kind of experimental designs are bound to give the best fruits is not surprising if one considers that this is uncharted territory in which new theories still need to be developed. The real challenge, however, concerns the finding of proper ways to operationalise the research questions. Once the scientific hypothesis has been formulated, how should the interpreted texts (possibly a corpus of interpreted texts) be investigated in order to identify concrete manifestations of the use of cai tools? What sort of things do we need to search for and what sort of techniques do we apply to locate them? How can we triangulate results in order to account for all variables involved in the experimental setup? In analogy to descriptive translation studies (cf. Toury, 1995), both productoriented and process-oriented research could help us to formulate tentative answers to the above mentioned questions. In this context, under productoriented research we understand the quantitative and qualitative analysis of interpreted texts produced by interpreters with the support of some sort of cai tool. Product-oriented studies are generally of a comparative nature, as they tend to investigate both the source and target texts by means, for examples, of contrastive linguistics, contrastive pragma-linguistics, contrastive pragmatics and contrastive discourse analysis (cf. Vandepitte, 2008) or texts produced under different conditions (with, without or with different cai tool). From a methodological point of view, such studies could profit from a corpusbased approach, for example when comparing interpreted texts with a corpus of comparable, non-interpreted texts, as is now common practice in studies of translation universals and the like (Baker, 1995). Product-oriented research could be employed to measure if and how the use of cai tools influences the interpreter’s performance and to discover if their use left some sort of ‘fingerprint’ (Gellerstam, 1986) in the interpreted texts. Even if quality criteria in

168

Fantinuoli

i­nterpreting are not easy to define and may diverge greatly depending on the perspective adopted and the conference context,16 experimental tests, comparing for example the interpreters performance with and without tools, could be the basis for inferencing the influence of tools on the overall performance of interpreters from a set of predetermined findings and observations. As most cai tools are strictly related to the lexical level of interpreting and, not-surprisingly, terminology is considered crucial in meetings of technical nature (cf. aiic, 1995), product-oriented studies could help to analyse the terminological rendition of interpreters working with or without cai tools in the booth. We may look at terminology accurateness as an indication of successful use of cai tools for terminology access tasks. Given the fact that functional communication could also be achieved without the use of a specific terminological unit, but applying other translation strategies or tactics (for example paraphrasing, using a hyponym, etc.), the terminological rendition should be evaluated in correlation with other parameters of the interpreted text, such as completeness, fluidity, semantic or functional correspondence and all related strategies applied (removing redundancies, anticipation through discourse inference, compression, etc.). Similar empirical tests could be used to investigate different aspects of the use of cai tools in different phases of the interpreting assignment. Not only the more obvious terminological lookup in the booth, as mentioned above, should be in focus, but also the use of special tools during the preparation, for example in the constitution of the knowledge background, the lexicographical memorisation and so forth. Are differences measurable in terms of invested time and quality output? On the other hand, process-oriented research in interpreting studies, encompassing foremost cognitive aspects and methods, could shed light on the brain of interpreters, as seen in the described product-oriented research. For example, what happens when conference interpreters are simultaneously exposed to sensory information on different input channels? As suggested by Seeber (2012), a multimodal setting is not limited to the realm of remote interpreting, but it applies to most ordinary conference interpreting scenarios nowadays. Computer-assisted interpreting is by definition a multimodal scenario, as it adds to the traditional stimuli the parameter of the use of a cai tool (which could also be broken up in several parts, such as keyboard use, search of the right information on screen, read of relevant information, etc.). Is it possible to reduce the cognitive load during interpreting to allow for terminology lookup activities? If yes, which strategies could be applied in order to smoothly 16

Not to mention that the quality of an interpretation is never inherent in the interpretation itself but attributed to it by some instance (cf. Zwischenberger, 2010).

Computer-assisted Interpreting

169

incorporate the results of a terminological query in speech? Process-oriented research could help to understand the effects of cai tools on the interpreting task (the process), an area, where very little progress has been made during the last years. However, new methods (eye-tracking, Electroencephalography (eeg), etc.) should be explored for process-based research and, if successful, attempts should be made to use them systematically in experimental tests (cf. Seeber, 2013). Besides helping to describe the characteristics of computer-­ assisted interpreting, both product and process-oriented research could allow us to better define what a tool specifically designed for interpreters should offer and how it should be integrated in the interpreting process. As introduced in Section 3, first and second generation cai tools are more or less based on the personal ideas of their developers, mostly interpreters themselves, and lack any experimental support in their design decisions. Particular attention should therefore be devoted to the way knowledge in general, and in particular lexicological knowledge, should be structured and presented to the user in order to cope with the limits proposed in literature. The question of information visualisation is very much debated in cognate disciplines, as it should be in interpreting. The analysis of the limits of available software coupled with a better insight into the cognitive processes of interpreting could allow the scientific community to propose cai tools that move from the representation of simple linguistic equivalences (the typical structure of interpreter glossaries) to a new, interpreter-friendly way to represent a specific domain and its terminology. Finally, another important topic that should be addressed concerns the didactics of interpreting. If cai tools (as all other interpreted-related technologies) are slowly redefining the professional landscape, there is no reason why advantages and shortcomings of their use should not be properly addressed in the training of future interpreters. At the moment, the number of universities actively engaged in teaching new technologies in interpreting courses is very limited,17 while much of the work is done outside the regular programs in the form of seminars and workshops offered, for example, by professional associations. If we want future generations of interpreters to be prepared to address technological changes, the topic should be recognised as an important part of the didactic objectives of any educational institution. Consequently, a debate on when and how the topic should be taught needs to be initiated.

17

In 2014, the University of Innsbruck introduced a curricular course dedicated to Technologies for interpreters in their Master of Conference Interpretation and the University of Surrey is much devoted to the teaching of remote and mobile interpreting.

170 5

Fantinuoli

Conclusions and Future Research

New technologies are slowly reshaping the landscape of professional interpreting and there is reason to believe that the pace of change will increase during the next years. The challenge for interpreting scholars is to research the use of evolving cai tools, assess their feasibility, analyse the strategies interpreters may need to adopt and, eventually, transfer this knowledge to the training of a new generation of interpreters. The emerging role of both process and setting-oriented technologies has started to be recognised by researchers and first studies on the subject have been published recently. Yet, the majority of studies is of a general or theoretical nature, while the number of empirical studies is still almost insignificant. However, in order to shed light onto the advantages and disadvantages of cai tools, the way they are affecting the interpreting process and the tasks interpreters can perform better with their help and those which cannot, research on new technologies needs to be performed not only on the basis of naturalistic methods (such as corpus analysis), but empirical experiments should be conducted also in stringently controlled experimental conditions. Both process and product-oriented research in this area are required. There are obvious difficulties that still need to be addressed: experimental design must be optimised and robust experimental methods must be imported into empirical interpreting research,18 as it is the case with written translation process research. In order to understand how to operationalise the research hypothesis, much of exploratory research is still required, and modern experimental techniques such as eye-tracking, eeg, etc., should be tested and applied if proven successful. ict is advancing quickly and is opening new perspectives in the area of cai tools. Speech recognition, for example, could represent the next step in the 18

This point of view, however, has been very much criticised by many researchers and practitioners, their main concern being the ecological validity of the tasks and environmental conditions under which they have been called to interpret (cf. Gile, 2015a: 54). Even if interpreting is generally viewed as a strategic activity (a fact that should make it difficult to conduct research in the same way as in cognate disciplines, for example psychology), the careful design of experimental settings and the proper control of all variables at stake is a prerequisite for obtaining reliable data. With all due respect to the concerns expressed by many scholars, this will mean, for example, that source texts should not be selected only according to the fact that they are ‘real’ (i.e. typical for the profession), but they should be selected, and manipulated, in order to provide controlled material for the analysis of the dependent variables at syntactical, lexical or semantic level.

Computer-assisted Interpreting

171

evolution of cai tools. It could be used to automatically extract terminology in real-time from the interpreter’s database or to show name entities, numbers and the like on the interpreter’s monitor. Would this influence the interpreting process? Would it facilitate the interpreting task or determine a cognitive overload? Again, to find an answer to these and other questions, empirical interpreting studies are required. This seems the only way for interpreting studies to keep pace with an evolving profession. References AIIC (1995). Survey on Expectations of Users of Conference Interpretation. Andres, D. and Falk, S. (2009). Information and Communication Technologies (ICT) in Interpreting – Remote and Telephone Interpreting. In Andres, D. and Pöllabauer, S., editors, Spürst Du wie der Bauch rauf runter?/Is everything all topsy turvy in your tummy? – Fachdolmetschen im Gesundheitsbereich/Health Care Interpreting, pp. ­9–27. Martin Meidenbauer, München. Arntz, R., Picht, H., and Mayer, F. (2009). Einführung in die Terminologiearbeit. Olms, Hildesheims. Austermühl, F. (2001) Übersetzen im Informationszeitalter – Überlegungen zur Zukunft fachkommunikativen und interkulturellen Handelns im Global Village. Trier: WVT Wissenschaftlicher Verlag. Baker, M. (1995). Corpora in translation studies: An overview and some suggestions for future research. Target, 7(2), pp. 223–243. Berber-Irabien, D.-C. (2010). Information and Communication Technologies in Conference Interpreting. Lambert Academic Publishing. Biagini, G. (2016). Glossario cartaceo e glossario elettronico durante l’interpretazione simultanea: uno studio comparativo. Tesi di laurea, SSLiMIT Trieste. Carabelli, A. (1997). IRIS Interpreters’ Resource Information System. Una banca dati interattiva per la formazione di interpreti e traduttori. Unpublished dissertation, SSLMIT-Università degli Studi di Trieste. Chernov, G.V. (2004). Inference and anticipation in simultaneous interpreting: a probability-­prediction model. Amsterdam & Philadelphia: John Benjamins. Costa, H., Corpas Pastor, G., and Durán-Muñoz, I. (2014). A Comparative User Evaluation of Terminology Management Tools for Interpreters. In Proceedings of the Workshop on Computational Terminology (CompuTerm’14), 25th International Conference on Computational Linguistics (COLING’14). Costa, H., Corpas Pastor, G., and Durán-Muñoz, I. (2018). Assessing Terminology Management Systems for Interpreters. This volume.

172

Fantinuoli

Fantinuoli, C. (2006). Specialized Corpora from the Web for Simultaneous Interpreters. In Baroni, M. and Bernardini, S., editors, Wacky! Working Papers on the Web as Corpus., pp. 173–190. Bologna: GEDIT. Fantinuoli, C. (2009). InterpretBank: Ein Tool zum Wissens- und Terminologiemanagement für Simultandolmetscher. In Tagungsband der internationalen Fachkonferenz des Bundesverbandes der Dolmetscher und Übersetzer e.V. (BDÜ), pp. 411–417, Berlin. Fantinuoli, C. (2011). Computerlinguistik in der Dolmetschpraxis unter besonderer Berücksichtigung der Korpusanalyse. Translation: Corpora, Computation, Cognition. Special Issue on Parallel Corpora: Annotation, Exploitation, Evaluation, 1(1), pp. 45–74. Fantinuoli, C. (2012). InterpretBank – Design and Implementation of a Terminology and Knowledge Management Software for Conference Interpreters. Doctoral Thesis, University of Mainz. Fantinuoli, C. (2016) InterpretBank. Redefining Computer-Assisted Interpreting Tools. Proceedings of the Translating and the Computer 38 Conference, pp. 42–52. London: Editions Tradulex. Fantinuoli, C. (2017). Computer-assisted Preparation in Conference Interpreting. In Translation & Interpreting, Vol 9, No 2, pp. 24–37. Gacek, M. (2015). Softwarelösungen für DolmetscherInnen. master, Uniwien, Vienna. Gellerstam, M. (1986). Translationese in Swedish Novels translated from English. In Wollin, L. and Lindquist, H. (eds.), Translation Studies in Scandinavia, pp. 88–95. CWK Gleerup, Lund. Gerzymisch-Arbogast, H. (1996). Termini im Kontext Verfahren zur Erschliessung und Ubersetzung der textspezifischen Bedeutung von fachlichen Ausdrucken. Tübingen: Francke (UTB). Gile, D. (2015a). The Contributions of Cognitive Psychology and Psycholinguistics to Conference Interpreting: A Critical Analysis. In Ferreira, A. and Schwieter, J.W., editors, Benjamins Translation Library, 115, pp. 41–64. Amsterdam/Philadelphia: John Benjamins. Gile, D. (2015b). THE CIRIN BULLETIN. Conference Interpreting Research Information Network. Technical Report 50. Gorjanc, V. (2009). Terminology Resources and Terminological Data Management for Medical Interpreters. In Andres, D. and Pöllabauer, S., editors, Spürst Du, wie der Bauch rauf-runter? Fachdolmetschen im Gesundheitsbereich. Is Everything all Topsy Turvy in your Tummy? Healthcare Interpreting, pp. 85–95. Meidenbauer, München. Gran, L., Carabelli, A., and Merlini, R. (2002). Computer-assisted interpreter training. In Garzone, G. and Viezzi, M., editors, Benjamins Translation Library, 43, pp. 277–294. Amsterdam/Philadelphia: John Benjamins. Kalina, S. (2007). ‘Microphone Off’ – Application of the Process Model of Interpreting to the Classroom. Kalbotyra, 57(3), pp. 111–121.

Computer-assisted Interpreting

173

Kalina, S. (2009). Dolmetschen im Wandel – neue Technologien als Chance oder Risiko. In Tagungsband der internationalen Fachkonferenz des Bundesverbandes der Dolmetscher und Übersetzer e.V. (BDÜ), pp. 393–401, Berlin. Lim, L. (2014). Examining Students’ Perceptions of Computer-Assisted Interpreter Training. The Interpreter and Translator Trainer, 7(1), pp. 71–89. Morelli, M. and Errico, E. (2007). Le microlingue nell’interpretazione: esperienze professionali e didattiche. Tradurre le microlingue scientifico-professionali, pp. 347–372. Turin: Utet. Mouzourakis, P. (1996). Videoconferencing: Techniques and challenges. Interpreting, 1(1), pp. 21–38. Orlando, M. (2014). A Study on the Amenability of Digital Pen Technology in a Hybrid Mode of Interpreting: Consec-Simul with Notes. Translation & Interpreting, 6(2), pp. 39–54. Prandi, B. (2016). The Use of CAI Tools in Interpreters’ Training: A Pilot Study. In Proceedings of the 37 Conference Translating and the Computer, London. Pym, A. (2011). What Technology does to Translating. Translation & Interpreting, 3(1), pp. 1–9. Riccardi, A. (2000). Die Rolle des Dolmetschens in der globalisierten Gesellschaft. In Kalina, S., Buhl, S. & Gerzymisch-Arbogast, H. (eds.), Dolmetschen: Theorie – Praxis – Didaktik – mit ausgewählten Beiträgen der Saarbrücker Symposien, pp. 75–87. Röhrig. Roderick, J. (2014). Interpreting: a Communication Profession in a World of NonCommunication. Revue Internationale d’études en langues modernes appliquées, 7, pp. 9–18. Rütten, A. (2007). Informations- und Wissensmanagement im Konferenzdolmetschen. Frankfurt am Main: Peter Lang. Sandrelli, A. and Jerez, J.d.M. (2007). The Impact of Information and Communication Technology on Interpreter Training. The Interpreter and Translator Trainer, 1(2), pp. 269–303. Seeber, K.G. (2012). Multimodal input in Simultaneous Interpreting: An eye-tracking experiment. In Zybatov, L.N., Petrova, A., and Ustaszewski, M. (eds.), Proceedings of the 1st International Conference TRANSLATA, Translation & Interpreting Research: Yesterday – Today – Tomorrow, pp. 341–347. Peter Lang, Frankfurt am Main. Seeber, K.G. (2013). Cognitive Load in Simultaneous Interpreting: Measures and Methods. In Ehrensberger-Dow, M., Göpferich, S. and O’Brien, S. (eds.), Interdisciplinarity in translation and interpreting process research. Target 25(1), pp. 18–33. Stoll, C. (2009). Jenseits simultanfähiger Terminologiesysteme. Trier: Wvt Wissenschaftlicher Verlag. Toury, G. (1995). Descriptive Translation Studies and Beyond. Amsterdam/Philadelphia: John Benjamins.

174

Fantinuoli

Tripepi Winteringham, S. (2010). The usefulness of ICTs in interpreting practice. The Interpreters’ Newsletter, 15, pp. 87–99. Valentini, C. (2002). Uso del Computer in Cabina di Interpretazione. Tesi di laurea, ­S SLiMIT Bologna. Vandepitte, S. (2008). Remapping Translation Studies: Towards a Translation Studies Ontology. Meta: Journal des traducteurs, 53(3), pp. 569–588. Will, M. (2007). Terminology Work for Simultaneous Interpreters in LSP Conferences: Model and Method. In Proceedings of the EU-High-Level Scientific Conference Series MuTra, pp. 65–99. Will, M. (2009). Dolmetschorientierte Terminologiearbeit. Modell und Methode. Tübingen: Gunter Narr Verlag. Will, M. (2015). Zur Eignung simultanfähiger Terminologiesysteme für das Konferenzdolmetschen. trans-kom 8 (1), pp. 179–201. Retrieved from http://www.trans-kom .eu/bd08nr01/trans-kom_08_01_09_Will_Konferenzdolmetschen.20150717.pdf (Consulted on 03/03/2017). Wüster, E. (1959/1991). Einführung in die allgemeine Terminologielehre und terminologische Lexikographie. Würzburg: Ergon. Xu, R. (2015). Terminology Preparation for Simultaneous Interpreters. Doctoral Thesis, University of Leeds. Xu, R. and Sharoff, S. (2014). Evaluating Term Extraction Methods for Interpreters. In Proceedings of the 4th International Workshop on Computational Terminology (Computerm), pp. 86–93, Dublin, Ireland. Zwischenberger, C. (2010). Quality Criteria in Simultaneous Interpreting: An International vs. A National View. The Interpreters’ Newsletter, 15, pp. 127–142.

part 3 Machine Translation

∵

chapter 8

The ACCEPT Academic Portal: A Pre-editing and Post-editing Teaching Platform Pierrette Bouillon, Johanna Gerlach, Asheesh Gulati, Victoria Porro and Violeta Seretan Abstract The advance of machine translation in the last years is placing new demands on professional translators. This entails new requirements on translation educational curricula at the university level and exacerbates the need for dedicated software for teaching students how to leverage the technologies involved in a machine translation workflow. In this chapter, we introduce the ACCEPT Academic Portal, a user centred online platform which implements the complete machine translation (mt) workflow and is specifically designed for teaching purposes. Its ultimate objective is to increase the understanding of pre-editing, post-editing and evaluation of machine translation. The platform, publicly available at http://accept-portal.unige.ch/academic, is built around four main modules, namely, the Pre-editing, Machine Translation, Post-editing, and Evaluation module. The Pre-editing module provides checking resources to verify the compliance of the input text with automatic and interactive pre-editing rules, based on a shallow analysis of the text. The Translation module translates the raw and preedited versions of the input text using a statistical mt system, and highlights the differences between the two translations for easy identification of the impact of pre-editing on translation. The Post editing module allows users to improve translations by postediting the output text freely, manually or with the help of interactive and automatic post-editing rules. Finally, the Evaluation module provides support for eliciting user feedback. At the end of the workflow, a summary and statistics on the whole process are made available to users, for reference purposes. The ACCEPT Academic Portal was developed in the framework of the ACCEPT European project and, to the best of our knowledge, it is the only online environment integrating advanced pre-editing and post-editing technology into a complete mt workflow. Through its simple and userfriendly interface, as well as its pedagogically motivated functionalities that enable experimentation, visual comparison and documentation, the ACCEPT Academic Portal is a unique tool allowing to study the interactions between mt-related processes and to assess the contribution of new technologies to translation.

© koninklijke brill nv, leiden, ���8 | doi 10.1163/9789004351790_010

178

Bouillon et al.

Keywords translation technology – translation teaching – translation workflow – pre-editing – post-editing – machine translation – software tool – online platform

1 Introduction The increasing adoption of machine translation (mt) in the translation industry is placing new demands on professional translators and translation students. Understanding the advantages and limitations of the technologies involved in the mt workflow is an essential part of the translators’ skill set. This entails new requirements on translation educational curricula and, consequently, exacerbates the need for dedicated software for teaching students how these technologies may interact to offer a better final output. In this chapter, we describe the ACCEPT Academic Portal (henceforth, aap), a user-centred online platform which implements a complete mt ­workflow and is specifically designed for teaching purposes. The platform was developed in the framework of the ACCEPT European project (2012–2014) devoted to improving the automatic translation of user-generated content (www.acceptproject.eu). To make the ACCEPT technology accessible to a wider public and, in particular, to teachers and students, we undertook the task of transforming the ACCEPT demonstration portal (Seretan et al., 2014) into an easy-to-use, fully-integrated platform combining pre-editing, mt and post-editing into a single workflow. While tools exist for each of these individual processes (as can be seen in Section 2), to the best of our knowledge they have never been combined into a single platform. By enabling experimentation with the multiple processes involved in the mt workflow, aap provides a unique environment to study interactions between these processes. The platform, publicly available at http://accept-portal.unige.ch/academic, allows users to automatically or interactively pre-edit a text with different types of pre-editing rules and to evaluate the impact on translation quality and on human post-editing, using metrics such as time and keystrokes. The platform relies on the Acrolinx iq checker (Bredenkamp et al., 2000), a rule-based engine that uses a declarative formalism for language checking in which error patterns are detected based on shallow parsing. The ultimate objective of aap is to increase the understanding of the importance of pre-editing, post-editing and evaluation of mt. This chapter is structured as follows. In Section 2, we provide a teaching-­ oriented survey of translation tools that integrate pre-editing, mt and

The ACCEPT Academic Portal

179

­post-editing. In Section 3, we present the design choices and main functionalities of the aap platform, detailing the underlying technologies developed in the framework of the ACCEPT European project. In Section  4, we describe a use case for teaching, as carried out at the Faculty of Translation and ­Interpreting of the University of Geneva in the spring 2015 semester. Section 5 concludes the chapter. 2

Related Work

In this section, we review the existing technology, developed either in academic or industrial contexts, that aims at integrating pre-editing, machine ­translation and post-editing to the benefit of the human translator. We adopt the perspective of the teacher who plans to use such technology in the c­ lassroom, and investigate to what degree it is readily available for use in a teaching context, what type of language phenomena are taken into account – e.g., ­casing, punctuation, lexicon, syntax, style – and whether the pre-editing and post-editing processes are manual or automated. An important aspect considered in our survey is the usability of tools and, in particular, the installation effort, on the grounds that the need for advanced technical expertise for setting up a tool may hinder its adoption in a teaching context. 2.1 Integration of the Pre-editing and Machine Translation Processes Pre-editing is an area that is intensively researched, especially in the contexts of Controlled Languages (cl) – see O’Brien (2003) for a typology of cl rules, and Kuhn, 2014 for a detailed review of cls for English –, lexical normalisation (e.g., Han et al., 2013), as well as text simplification and source reordering for smt – see Shardlow (2014) and Bisazza and Federico (2015) for recent reviews. There are numerous reports on the positive impact of pre-editing on the performance of mt systems, for instance, in rule-based settings (O’Brien and Roturier, 2007), statistical settings (e.g. Aikawa et al., 2007) and example-based settings (e.g., Way and Gough, 2005).1 However, to the best of our knowledge, with the exception of the ACCEPT Academic Portal, there is no integrated offthe-shelf tool available that users can readily experiment with in order to test the effects of pre-editing on mt results. In some machine translation tools – e.g., Microsoft Translator Hub2 – it is possible to apply spell checking on the source text before it gets translated 1 The acronyms rbmt, smt and ebmt are used to refer to these mt paradigms, respectively. 2 http://www.bing.com/translator.

180

Bouillon et al.

and post-edited; but in these cases, we cannot properly speak of pre-editing. Outside a translation context, we can find open-source online demonstrators for various language checkers, e.g. LanguageTool3 (Naber, 2003) and After the Deadline4 (Mudge, 2010), which are capable of highlighting errors not only at the level of spelling, but also at the level of grammar and style. In both cases, the application of editing rules is performed in an interactive way and there is no integration with mt. Another tool, the Acrolinx iq checker (a proprietary tool; Bredenkamp et al., 2000), allows users to create customised editing rules in a formalism similar to regular expressions and to apply them in different environments, such as Microsoft Office, sdl Passolo, Alchemy Catalyst, and Adobe InDesign. Nevertheless, aside from the present implementation of the Acrolinx iq checker (as described in Section 1), rules cannot be freely tested online and the checker is not coupled with mt. The above-mentioned tools do not really take advantage of the whole gamut of pre-editing operations which have been proposed in cl and mt research as means to obtain increased translatability. Their use for teaching pre-editing and mt is therefore limited. 2.2 Integration of the Machine Translation and Post-editing Processes The integration of mt with post-editing is much more common than with preediting. Vieira and Specia (2011) provide a comparative analysis of translation tools that enable the integration of machine translation into human translation pipelines. Their analysis is focused on a series of features, among which the existence of a ‘spell/grammar/style checker’, which is the basic type of support expected from a post-editing perspective. Most of the tools studied (5/9) do provide support for language checking; the others (4/9) rely on default language checking plug-ins installed on I­nternet browsers. However, most of the tools mentioned do not provide support for automatically fixing recurrent basic errors that are typical of mt output, for instance, lack of agreement or word order errors. In addition, most of the existing tools do not provide support for automatic post-editing (ape). ape would avoid translators having to deal repeatedly with basic errors, which represent one of the ‘most annoying aspects of post-editing’ (O’Brien & Moorkens, 2014). The research on ape is relatively recent. Guzmán (2008) proposed, for instance, a regular-expression-based approach aimed at correcting a limited range of grammatical errors in English- > Spanish rbmt output. Simard et al. (2007) proposed an smt-based approach to ape, in which human post-edited 3 https://languagetool.org. 4 http://www.polishmywriting.com.

The ACCEPT Academic Portal

181

data is used to train a monolingual translation system where the mt output is seen as the source and the corrected version as the target. Béchara et al. (2011) extended this approach by including source language information available through word alignment. For the time being, the research on ape did not lead to the release of actual translation tools, except for Depfix (Rosa et al., 2012). This tool relies on parsing to fix specific syntax-related errors for English- > Czech translation, pertaining to agreement, negation, genitive structures, and valence of verbs and nouns. 2.3 Survey of Translation Tools from a Teaching Perspective We performed our own review of translation tools from the perspective of ­usefulness for our scenario, namely, that of a teacher looking for tools to demonstrate and use in the classroom. We considered the following tools: – post-editing oriented tools: pet (Aziz et al., 2012), Casmacat (Alabau et al., 2013),5 and MateCat (Federico et al., 2014);6 – general (web-based) translation tools: Google Translator Toolkit,7 Reverso Localize,8 and Microsoft Translator Hub.9 Among the analysed tools, pet and Casmacat require a complex installation procedure and are therefore out of reach for the non-expert user. MateCat is very usable, fully integrated and, within a few clicks, takes the user through the processes of project creation, data analysis and post-editing (or professional translation outsourcing). Google Translator Toolkit also enables project creation and provides an intuitive post-editing interface, in which the machine translation, if picked up, can be edited in a segment-by-segment fashion. Spell checking can be performed on the target side. The look-and-feel is altered by pop-up editing windows and the relatively complex user interface. Reverso Localize is a fullyfledged tool, easy to use, which provides support for collaborative work and offers an interface for in-context10 post-editing of web sites whose distinctive feature is the highlighting of differences between the raw translation and the 5 http://www.casmacat.eu/index.php?n=Installation.HomePage (accessed March, 2017). 6 https://www.matecat.com. 7 https://translate.google.com/toolkit. 8 http://localize.reverso.net. 9 http://www.bing.com/translator. 10 See Roturier (2015) for a discussion on the importance of in-context post-editing for ­localization quality assurance.

182

Bouillon et al.

revised ­version. Microsoft Translator Hub features a very intuitive and usable interface. It performs spell checking on the source side, and its distinguishing feature is the two-directional alignment between source and target words. However, this tool does not provide a proper post-editing environment. Summing up, there are currently only few tools that implement the mt workflow, even partially, and that are readily available for use in a teaching context. They include online tools like language checkers (e.g., LanguageTool, After the Deadline) and post-editing platforms (e.g., MateCat, Google Translator Toolkit, Reverso Localize). While these tools do not show the degree of sophistication attained by state of the art technologies (e.g., Acrolinx for preediting; Casmacat and pet for post-editing), they are a viable solution until the latter become more user-friendly. To our knowledge, the ACCEPT Academic Portal (aap) is the only ready-touse platform which integrates advanced pre-editing and post-editing technology into a complete mt workflow. The design and the functionalities of aap are presented in the next sections. 3

Platform Design and Functionalities

The ACCEPT Academic Portal (aap) is built using the JavaScript framework AngularJS. The platform offers a minimalistic and user-friendly interface that integrates and regroups the original ACCEPT software (see Section 1) into a complete mt workflow, allowing users to subject a text to a sequence of processes until the desired output is reached. The interface is currently available in English. aap is built around four main modules, the Pre-editing, Translation, Postediting and Evaluation modules: – The Pre-editing module of aap relies on a jQuery plug-in that implements a lightweight version of the original ACCEPT ‘real-time’ pre-editing plug-in (designed to function without an external dialogue). The checking process relies on the Acrolinx iq engine (Bredenkamp et al., 2000), a rule-based engine that uses a combination of nlp11 components and enables the development of declarative rules. The rules are written in a formalism similar to regular expressions, based on the syntactic tagging of the text. The preediting rules applied on the platform were developed during the A ­ CCEPT European project. They were designed to improve the translatability of 11

nlp stands for Natural Language Processing.

The ACCEPT Academic Portal

183

technical forum data, which is particularly prone to errors (Gerlach et al., 2013; Gerlach, 2015; ACCEPT deliverable D2.2). – The Translation module uses a statistical system built specifically to deal with technical forum data in the framework of the ACCEPT European project. This is a phrase-based Moses system, fully described in ACCEPT ­deliverables D4.1 and D4.2. – The Evaluation module is based on the evaluation framework developed in the last phase of the ACCEPT European project. It allows for eliciting user feedback on the quality of the output in terms of metrics such as fluency. Detailed information about the evaluation framework can be found in ­ACCEPT deliverable D5.3. – Finally, the Post-editing module implements the ACCEPT post-editing client, described in detail in Roturier et al. (2013). This module integrates postediting rules developed with the same technology as the pre-editing rules. Each of the modules can be individually activated. Two additional components, the Start and Statistics pages, are used to configure the workflow and visualise the results, respectively. A help button is available in each of the modules and pages, giving access to online help which provides information about usage of the platform as well as underlying resources, such as the available pre-editing and post-editing rules. In the remaining of this section, we detail the functionalities of each of the platform modules. 3.1 Start Page The Start page (see Figure 8.1) allows users to select data and define the translation workflow. The workflow can be applied to different types of data: – sample data (text from a technical forum post); – custom data consisting of text inserted by users in the in-line editor; – custom data consisting of text files uploaded by users. The selection of the modules has to be done at this stage. The available modules can be combined in different ways, thus creating different possible processing scenarios. Below we provide several typical examples: – – – –

Pre-editing and Machine Translation; Pre-editing, Machine Translation and Post-editing; Pre-editing, Machine Translation, Post-editing and Evaluation; Machine Translation and Evaluation.

184

Bouillon et al.

The aap platform currently provides support for two language pairs, namely French- > English and English- > French. More language combinations will be added following the integration of external mt systems, such as Bing. 3.2 Pre-editing Module The Pre-editing module (see Figure 8.2) provides checking resources to verify the compliance of the input data with pre-editing rules. It allows users to test interactive and automatic rules for English and French. The rules are grouped into sets and can be activated and deactivated individually by clicking on the

Figure 8.1

Screen capture of the portal start page.

Figure 8.2

Screen capture of the Pre-editing module.

The ACCEPT Academic Portal

185

downwards arrow of the corresponding button. Besides applying rules, users can also edit the text manually. The pre-editing resources used in the platform are a product of the ­ACCEPT project, described in detail in ACCEPT D2.2, Gerlach et al. (2013), and Gerlach (2015). These resources rely on the Acrolinx technology (Bredenkamp et al., 2000), which allows the description of error patterns in a formalism similar to regular expressions. Since the focus of the project was machine t­ ranslation of user-generated content, these rules were designed specifically to improve translatability of this content and, more precisely, of technical forum data. As a consequence, the rules focus on different aspects of this type of data that present specific readability issues and difficulties for mt. They are essentially monolingual, but in some cases they may depend on the language combination. The rules can be split into two main groups: rules for humans and rules for the machine (as in O’Brien, 2003). Rules of the first group focus on linguistic improvement, thereby making content more accessible to human readers, but also to automatic processes. The rules of the second group focus on specific phenomena that are problematic for mt systems, and may perform transformations that degrade the content. The rules for humans include the following main categories: Spelling and grammar: these rules correct non-word and real word errors, such as word confusions, tense/mode confusions or agreement errors. – Punctuation and spacing: these rules correct punctuation issues, including elision and hyphenation. – Style rules: these rules focus on style issues typical for the informal language used in community content, such as cleft sentences, direct questions, and use of present participle or incomplete negation. – Simplification rules: in an approach similar to Controlled Languages, these rules are designed to improve readability and reduce ambiguity, for example by splitting long sentences. – Normalisation rules: these rules replace unconventional abbreviations and anglicisms by standard equivalents. The rules for the machine include the following main categories: – Reformulation: these rules were designed specifically for the smt system included in the portal, performing transformations to increase similarity with the system’s training data. – Reordering: these language-pair specific rules reorder the source text to bring it closer to word order used in the target language.

186

Bouillon et al.

– Clitics (French only): a number of rules performs transformations of French clitics, which were found to be problematic for translation. – Informal second person (French only): this rule addresses the register mismatch between French and English by converting the informal second person in French (tu) into its formal correspondent (vous), which is much more frequent in the training data (Rayner et al., 2012). Table  8.1 and Table  8.2 illustrate the application of several pre-editing rules to sentences in French and English, respectively. For instance, the last row of ­Table 8.1 shows an example in which two pre-editing rules are applied: a first that changes the order of verb and subject, as required for interrogative sentences (tu as → as-tu), and a second that replaces an abbreviation by the corresponding full form (tuto → tutoriel). The mt output is also shown for each example, once for the original sentence and once for the modified sentence, to illustrate the impact of the transformations. Table 8.1

Examples of pre-editing in French and impact on machine translation into English.

Raw French Source oups j’ai oublié, j’ai sa aussi. English smt Oops I forgot, I have its also. output French Source avez vous des explications ou astuces pour que cela fonctionne? English smt Have you explanations or tips for output it to work? French Source J’ai apporté une modification dans le titre de ton sujet. English smt I have made a change in the title of output tone subject. French Source Il est recommandé de la tester sur une machine dédiée. English smt It is recommended to the test on a output dedicated machine. French Source Tu as lu le tuto sur le forum? English smt You have read the Tuto on the output forum?

Pre-edited oups j’ai oublié, j’ai ça aussi. I have forgotten, I have this too. Avez-vous des explications ou astuces pour que cela fonctionne? Do you have any explanations or tips for it to work? J’ai apporté une modification dans le titre de votre sujet. I have made a change in the title of your issue. Il est recommandé de tester ça sur une machine dédiée. It is recommended to test it on a dedicated machine. As-tu lu le tutoriel sur le forum? Have you read the tutorial on the forum?

187

The ACCEPT Academic Portal

Table 8.2 Examples of pre-editing in English and impact on machine translation into French.

English Source French smt output

English Source French smt output English Source French smt output English Source French smt output

Raw

Pre-edited

It is possible to setup Custom Power Schemes to use. Il est possible de le programme d’installation personnalisée les différents modes de gestion de l’alimentation à utiliser. So it’s not always the obvious cause. Il n’est pas toujours évident la cause You have to remember that the software is not perfect. Vous devez vous rappeler que le logiciel n’est pas parfait. The interface module are ready. L’interface module sont prêts.

It is possible to set up Custom Power Schemes to use. Il est possible de configurer les différents modes de gestion de l’alimentation personnalisé à utiliser. Therefore it’s not always the obvious cause. Par conséquent, il n’est pas toujours évident la cause. You must remember that the software is not perfect. Vous devez vous rappeler que le logiciel n’est pas parfait. The interface module is ready. L’interface module est prête.

The pre-editing rules for French are grouped into three sets (ACCEPT D 2.2, Gerlach et al., 2013). Each set is accessible through the corresponding button in the bottom left-hand corner of the pre-editing view, ‘Automatic’, ‘Interactive’, and ‘Silent’ (see Figure 8.2). The first two sets contain the rules for humans. The ‘Automatic’ set regroups rules that treat unambiguous cases and have unique suggestions. They can therefore be applied automatically with no human intervention. The ‘Interactive’ set contains rules that have either multiple suggestions or no suggestion, thus requiring human intervention. Finally, the third set of rules grouped under the ‘Silent’ button contains the rules for the machine that should not be visible to end-users and thus applied automatically before being sent to mt. As for the pre-editing rules for English, they are grouped into two sets ­(ACCEPT D 2.2). In the portal, they are accessible through the two buttons in the lower part of the pre-editing interface, ‘Automatic’ and ‘Interactive’. As in the case of French, the ‘Automatic’ button regroups rules that treat unambiguous cases and have unique suggestions, therefore allowing for automatic application of the set. The ‘Interactive’ button regroups rules that require human intervention.

188

Bouillon et al.

Figure 8.3

List of automatic pre-editing rules for French.

Given the pedagogical nature of the portal, all rules, both for French and for English, can be applied interactively by the user by clicking on the corresponding button, even if they were conceived to be applied automatically. For both languages, the complete list of rules in each set can be accessed by clicking on the downwards arrow of the corresponding button (i.e., ‘Automatic’, ‘Interactive’ and ‘Silent’). Each rule can be activated and deactivated individually (see Figure 8.3). The rule description, including rule application examples, is available in the platform documentation and is accessible with a simple click on the information button next to the rule.12 In the current version of the portal, the set of available rules is fixed. Future extensions might include the possibility for users to customize or create new rules, thereby allowing students to learn to define rules for specific phenomena, test them and study their impact on translation quality. This extension would however considerably increase the complexity of the task, as rule development requires familiarisation with the rule formalism, as well as some knowledge of the nlp components used by the pre-editing technology. Nonetheless, we consider this feature worth developing, since it would allow users 12

For the sake of readability, we do not discuss in this chapter the technical details related to the rule description using the Acrolinx formalism. The interested reader is referred to Gerlach et al. (2013) for a more in-depth presentation, including examples of actual rule descriptions.

The ACCEPT Academic Portal

Figure 8.4

189

Screen capture of the translation module.

to get more actively involved in the pre-editing process, and would increase the attractiveness of the portal for a different category of users. 3.3 Translation Module The Translation module translates the raw and pre-edited version of the input text using the ACCEPT smt system developed during the ACCEPT European project and described in ACCEPT deliverables D4.1 and D4.2. The interface of this module is shown in Figure 8.4. The module highlights the differences between the two translations for easy identification of the impact of pre-editing on translation. The user can select the version (raw or pre-edited) to be sent to the post-editing module. 3.4 Post-editing Module The Post-editing module (see Figure 8.5) allows users to improve translations by freely post-editing the text. It also allows interactive checking with Acrolinx post-editing rules for French and English, which are specifically designed for correcting mt output and readability issues (Porro et al., 2014; ACCEPT 2.4). The interface shows the source (for bilingual post-editing scenarios), the mt output and the sentence currently post-edited. The post-editing activity is recorded in an xliff file, for maximal interoperability (Roturier et al., 2013). This file contains detailed segment-level ­information on the actions performed during the post-editing process: for each revision, it reports keystrokes, editing time, number of accepted rule

190

Bouillon et al.

Figure 8.5

Screen capture of the Post-editing module.

­suggestions (if any), and user comments. The post-editing report can be exported at this stage or at the end, together with all session data, on the Statistics page. The help button at the bottom of the page provides access to the online taus post-editing guidelines.13 The rules available in the Post-editing module are meant to help post-editors speed up their work by reducing the number of edits they have to perform, and to improve their working experience by enhancing text readability. Like the pre-editing rules, they are essentially monolingual, but in some cases they may depend on the language combination. Automatic post-editing (ape) rules exist for both language pairs, English- > French and French- > English. The post-editing rules can be split into two groups depending on the type of phenomena they treat: standard errors and mt-specific errors. The rules in the first group target standard grammar and spelling errors, while the rules in the second group focus on errors that are produced specifically by smt systems. Standard post-editing rules include the following main categories: – Spelling and grammar: these rules correct non-word and real word errors, such as word confusions, tense/mode confusions or agreement errors. – Punctuation and spacing: these rules correct punctuation issues, including elision and hyphenation. 13

https://www.taus.net/academy/best-practices/postedit-best-practices/machine -translation-post-editing-guidelines.

191

The ACCEPT Academic Portal Table 8.3 Examples of post-editing of English mt output.

Rules for English Error detected

Post-edited correction (suggested or expected)

This window is ouvre. Donc I have not access the history. You can copy the details in the clipboard and paste the in a reply. The scan has allowed to detect threats minor. You can you express openly. I just buy new servers.

This window is ouvre. Donc I have not access the history. You can copy the details in the clipboard and paste in a reply. The scan has allowed to detect minor threats. You can express openly. I just bought new servers.

mt-specific rules include the following main categories: – Reordering and reformulation: these rules reorder the constituents (pos elements) of a sentence; this error is generally due to a literal rendering of the source language structure (e.g., ‘The scan has allowed to detect threats minor’). – Duplicated word or pos: these rules detect duplicated words or cases with two successive words of the same pos (e.g., ‘J’ai mis à jour mon les pilotes’). Sample post-editing rules for English and French mt output are provided in Table 8.3 and Table 8.4, respectively. The post-editing rules could be further split into rules with one or multiple suggestions and low precision, and rules with unique suggestion and high precision. As in the case of pre-editing rules, the post-editing rules for French that have a unique suggestion and a high precision can be applied automatically before the manual post-editing process begins. On the contrary, the rules that have multiple suggestions, and are thus less precise, are set aside to be applied interactively during the main manual process (ACCEPT D2.4, Porro et al., 2014). Nevertheless, given the pedagogical nature of the portal, all rules are applied interactively during the post-editing process so that the users can observe how the rules work and apply the changes themselves. The French

192

Bouillon et al.

Table 8.4 Examples of post-editing of French mt output.

Rules for French Error detected

Post-edited correction (suggested or expected)

Je n’ai accès à distance. Le Norton technicien m’a conseillé de […] Je suis en espérant As tu lu ça ? Blocage des appels Pas de message […] Il n’a pas faire ça Il a demandé à Si je savais […]. J’espère que cette aide. J’ai mis à jour mon les pilotes.

J’ai accès à distance. Le technicien Norton m’a conseillé de […] J’espère As-tu lu ça ? Blocage des appels. Pas de […] Il n’a pas fait ça Il a demandé à si je savais […]. J’espère que ça aide. J’ai mis à jour mes pilotes.

post-editing rules are grouped into a single set, which is accessible through the button ‘Interactive’ of the post-editing interface (see Figure 8.5).14 As for post-editing rules for English (ACCEPT D2.4), they are not precise enough to be applied automatically, and therefore require human intervention. They are also accessible through the ‘Interactive’ button of the post-­ editing interface (see Figure 8.5). Once the main post-editing task is completed, the text undergoes a final revision phase. During this final check, the user moves on to a monolingual interface (see Figure 8.6), where he or she may choose to interactively apply a set of rules that will help verify that no errors were left or introduced during the post-editing phase. The interactive rules used in this phase are actually the pre-editing rules targeting basic spelling, grammar and style mistakes (see Section 3.2). 3.5 Evaluation Module The Evaluation module can be used to manually evaluate the quality of the target sentences. By default, the evaluation is organised on a sentence by 14

Unlike pre-editing, the detailed list of post-editing rules is not displayed to the user. This is because of the technical choices related to the application of rules in bunches in a predefined order, to avoid undesirable interactions.

The ACCEPT Academic Portal

Figure 8.6

Screen capture of the revision interface in the Post-editing module.

Figure 8.7

Screen capture of the Evaluation module.

193

s­ entence basis and focuses on fluency by means of a simple question: ‘How fluent is the text?’. The answer scale to this question ranges from ‘flawless’ to ‘incomprehensible’ (see Figure 8.7). The module has to be activated in the Start Page at the beginning of the workflow. If the Post-editing module is activated in the Start Page, the evaluation will focus on the post-edited sentences. However, if only the Translation module is activated, the sentences shown for evaluation will be those of the raw mt output. The results of the manual evaluation are shown in the Statistics page by means of a bar chart, as described in the next section.

194

Bouillon et al.

3.6 Statistics Page At the end of the workflow, the Statistics page shows a summary of the entire process (see Figure 8.8). This summary presents, for each of the modules activated in the Start page, statistics or graphics on the activities performed within the module. The summary concerning the Start page includes the language combination chosen and the number of words in the original text submitted to the workflow. In the summary concerning the Pre-editing module, the user will find information on the specific rule sets applied to the original text. The Translation summary comprises information on the mt system used, the time required to translate the text and the number of words in the output text. If the Evaluation module was activated, the summary will present the results of the evaluation by means of a bar chart. Finally, the Post-editing summary comprises statistics about the post-­ editing actions performed during the task. Statistics are shown on a sentence per sentence basis and include keystroke information as well as editing time and thinking time spent in the task. The ‘editing time’ begins when the user starts typing or editing the text of the current segment, while the ‘thinking time’ comprises the time between the end of an editing phase and the beginning of the next one (this is the time the user might spend thinking about what to edit in the next segment). A complete post-editing report can be exported into an xliff file, thus allowing the users to analyse the information separately and process it using other tools. Details on the interpretation of this xliff file can be found in Roturier et al. (2013) and ACCEPT D5.6. All versions of the text produced in each step (pre-edited version, chosen translation, final output), as well as the xliff report, can be downloaded together at this final stage. 4

Use Case for Teaching

In this section, we illustrate a use case for the portal as carried out in June 2015 in a Master course at the Faculty of Translation and Interpreting of the University of Geneva. The objective of the task is to give students insights into the impact of pre-editing on mt output and post-editing effort, both on the segment level, by studying the impact of individual rules, and on the document level, by computing automatic scores and performing a basic fluency evaluation. The task is designed to be performed in groups of two students. Table 8.5 describes the instructions for the task and the phenomena that are illustrated by each step of the task.

The ACCEPT Academic Portal

Figure 8.8

195

Screen capture of the statistics page.

Table 8.5 Use case for teaching – Instructions handed out to students.

Instructions Setup This task should be performed in groups of two • Go to the ACCEPT Academic Portal: http://accept-portal.unige.ch/academic/ • Use the default demo login to connect. • On the Start page, in the Language pair controls, select English as source language and French as target. • Keep the default text and click Next. Pre-editing Objective: explore different types of pre-editing rules In this module, you will apply rules to correct and simplify the text. • A first set of rules is applied automatically. Changed words or phrases are highlighted in blue. • A second set of rules can be applied interactively by clicking the Interactive button.

196

Bouillon et al.

Table 8.5 Use case for teaching – Instructions handed out to students (cont.)

• You can also perform corrections or simplifications manually (e.g. by applying known controlled language rules). ! For this step, both members of the group should perform exactly the same transformations, in order to produce two identical pre-edited texts. • Click Next. Translation Objective: see the impact of individual rules on mt at the segment level This module allows you to compare the translations of the raw and pre-edited ­versions of the text. Differences are highlighted in blue. • For the next steps of this task, one of the members of the group will work with the translation of the raw text, while the other will work with the translation of the ­pre-edited text. Select the desired translation and click Next. Post-editing Objective: apprehend the difficulties of bilingual post-editing In this module, you will post-edit the mt output sentence by sentence. • Perform a minimal post-edition. The objective is to perform as few changes as ­possible to produce a usable translation. Refer to the taus guidelines and to the glossary provided at the end of this document. • Once post-editing is complete, click Finish and check. Check the underlined errors, if any, and click Download. ter score Objective: quantify the impact of pre-editing on the technical post-editing effort on the document level In this step you will compute the ter score for the raw and pre-edited versions • In the.zip archive produced by the portal, you will find a file called translatedData. txt (mt output) and a file called postEditedData.txt (post-edited version) • Use the Asiya-Online (http://asiya.lsi.upc.edu/demo/asiya_online.php) tool to compute the ter score for the mt output, using the post-edited version as reference translation. Do this for the raw and pre-edited versions. • Compare the obtained ter scores.

197

The ACCEPT Academic Portal Report Write a report for this task, including:

Judgements

• The ter score for each of the two versions (raw and pre-edited version) • A brief discussion (max 1 page) of: • the difficulties encountered during the post-editing task • the impact of pre-editing on this task. 10 9 8 7 6 5 4 3 2 1 0

Figure 8.9

1

2 3 Usefulness from 1 (not useful) to 4 (very useful)

4

Usefulness of the exercise with the ACCEPT academic portal.

A version of this exercise was given to Master students in communication and information technologies at the Université Libre de Bruxelles (ulb). This Master includes a course in computational linguistics which contains a 9-hour mt module about architecture (rbmt, smt), evaluation (human and automatic), pre-editing and post-editing. The exercise was given at the end of the mt module without further explanation and was optional. Students had to complete the exercise at home without supervision and then fill in a short questionnaire. 22 out of 40 students completed the task and questionnaire. Students were first asked to rate the usefulness of the exercise as a practical complement to the course on a four-point scale from 1 (not useful) to 4 (very useful). The average score was 3.2 (n = 22). A second question asked students to rate the user-friendliness of aap, again on a four-point scale from 1 (difficult to use) to 4 (very easy to use). The average score was 2.9 (n = 22). The score distributions for both questions are reported in Figures 8.9 and 8.10, respectively. In addition to the scores, students were asked to give their opinion on the strengths and weaknesses of aap. The answers show that the proposed exercise was a good complement to the course, very well received by students. 15 of the

198

Bouillon et al.

Judgements

8 6 4 2 0

1

2

3

4

User-friendliness from 1 (difficult) to 4 (very easy) Figure 8.10

User-friendliness of the ACCEPT academic portal.

22 students explicitly indicated as a strength that aap is easy to use (‘L’interface est très bien faite. Nous voyions directement où nous en sommes dans le processus de traduction. De plus, nous pouvons modifier le texte et voir la différence entre le texte source et le texte cible’; ‘Simplicité d’utilisation’15). Among the weaknesses, a recurring concern is the usability of the post-editing module, criticized in 4 of the 15 negative comments. Indeed, the look-and-feel of this module is different from that of other modules, as it inherits the original form of the ACCEPT post-editing plug-in, and still has to be redesigned. It is also worth mentioning that in a relatively high proportion of cases (4/15), the students’ negative comments were actually related to the mt performance, rather than the p ­ ortal’s ­interface and functionalities (‘peu clair, mauvaise traduction en réponse’; ‘Le portail ne prend pas en compte certaines tournures de phrase d’une langue à l’autre et traduit littéralement’16). 5 Conclusion The rapid pace of adoption of machine translation in the past years is placing new demands on professional translators. This situation entails new requirements on translation educational curricula at the university level and 15

16

‘The interface is well designed. We can directly see at which stage of the translation process we are. Additionally, we can modify the text and see the differences between source and target’; ‘Ease of use’. ‘Unclear, bad translation in the output’; ‘The portal does not consider turns of phrases that are different from one language to the other, it translates literally’.

The ACCEPT Academic Portal

199

­exacerbates the need for dedicated software for teaching students how to leverage the technologies involved in a machine translation workflow. In this chapter, we described the ACCEPT Academic Portal (aap), an online application which integrates into a single platform the multiple processes involved in the mt workflow – from pre-editing to post-editing and evaluation – and we proposed a use-case for teaching. Thanks to its simple and user-friendly interface, as well as its pedagogically-motivated functionalities enabling experimentation, visual comparison and documentation, aap represents a unique tool for the study of interactions between mt-related processes, and for the assessment of new technologies for translation. We carried out a small evaluation experiment with a group of students, in order to assess the user-friendliness and usefulness of the ACCEPT Academic Portal. The results showed that students generally appreciate the portal functionalities and positively value the proposed exercise as a complement to their mt course. Work is in progress to integrate external translation engines into the p ­ ortal – e.g., Bing – and thus to increase the platform’s coverage in terms of language pairs. However, adding a new source or target language to the system also requires porting a subset of editing rules that is more generally applicable to the new language, as well as defining a new set of rules that is language-­ specific. This is a time-intensive process, and one that relies on the availability of language resources (part-of-speech taggers). Yet, this extension is feasible, as such resources exist for an increasing number of languages. The existing rules are portable across domains (Gerlach, 2015), but, to a limited extent, they are genre-specific (about 15% of the rules deal with phenomena that are characteristic of user-generated content, the original application of the ACCEPT project). Further developments are also foreseen for the Evaluation module of the platform, in which we plan to plug automatic computation of evaluation metrics such as ter, as a complement to human evaluation. Finally, we plan to conduct a larger-scale evaluation of the portal in terms of usability, by collecting feedback from more users and assessing in new settings the suitability of the platform for translation technology teaching. Acknowledgements The research leading to these results has received funding from the European Community’s Seventh Framework Programme (FP7/2007–2013) under grant agreement n° 288769.

200

Bouillon et al.

References ACCEPT D2.2 (2013). Definition of pre-editing rules for English and French ( final version). Retrieved from http://www.accept.unige.ch/Products/D2_2_Definition_of_ Pre-Editing_Rules_for_English_and_French_with_appendixes.pdf (Consulted on 03/03/2017). ACCEPT D2.4 (2014). Definition of post-editing rules for English, French, German and Japanese. Retrieved from http://www.accept.unige.ch/Products/D-2-4-Definition -of-Post-editing-Rules.pdf (Consulted on 03/03/2017). ACCEPT D4.1 (2012). Baseline machine translation system. Retrieved from http:// www.accept.unige.ch/Products/D_4_1_Baseline_MT_systems.pdf (Consulted on 03/03/2017). ACCEPT D4.2 (2013). Report on robust machine translation: domain adaptation and linguistic back-off. Retrieved from http://www.accept.unige.ch/Products/ D_4_2_ Report_on_robust_machine_translation_domain_adaptation_and_linguistic_back -off.pdf (Consulted on 03/03/2017). ACCEPT D5.3 (2012). Adapted evaluation portal prototype to allow for the collection of user ratings. Retrieved from http://www.accept.unige.ch/Products/D5.3_Adapted_ Evaluation_Portal_Prototype.pdf (Consulted on 03/03/2017). ACCEPT D5.6 (2013). Browser-based client demonstrator and adapted post-editing environment and evaluation portal prototypes. Retrieved from http://www.accept .unige.ch/Products/D_5_6_Browser-based_client_demonstrator_and_adapted_ post-editing_environment_and_evaluation_portal_prototypes.pdf (Consulted on 03/03/2017). ACCEPT D9.2.4 (2014). Survey of evaluation results – Version 2. Retrieved from http:// www.accept.unige.ch/Products/D9-2-4-Survey-Evaluation-Results.pdf (Consulted on 03/03/2017). Aikawa, T., Schwartz, L., King, R., Corston-Oliver, M. & Lozano, C. (2007). Impact of controlled language on translation quality and post-editing in a statistical machine translation environment. In Proceedings of MT Summit XI, pp. 10–14. Copenhagen, Denmark. Alabau, V., Bonk, R., Buck, C., Carl, M., Casacuberta, F., García-Martínez, M., González, J., Koehn, P., Leiva, L., Mesa-Lao, B., Ortiz, D., Saint-Amand, H., Sanchis, G. & Tsoukala, C. (2013). CASMACAT: An open source workbench for advanced computer aided translation. The Prague Bulletin of Mathematical Linguistics, 100, pp. 101–112. Aziz, W., Castilho, S. & Specia, L. (2012). PET: A tool for post-editing and assessing machine translation. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12). Istanbul, Turkey. Béchara, H., Ma, Y. & van Genabith, J. (2011). Statistical postediting for a statistical MT system. In MT Summit XIII: the Thirteenth Machine Translation Summit, pp. 308– 315. Xiamen, China.

The ACCEPT Academic Portal

201

Bisazza, A. & Federico, M. (2015). A survey of word reordering in statistical machine translation: Computational models and language phenomena. Manuscript submitted for publication. Bredenkamp, A., Crysmann, B. and Petrea, M. (2000). Looking for errors: A declarative formalism for resource-adaptive language checking. In Proceedings of the Second International Conference on Language Resources and Evaluation, pp. 667–673. Athens, Greece. Federico, M., Bertoldi, N., Cettolo, M., Negri, M., Turchi, M., Trombetti, M., Cattelan, A., Farina, A., Lupinetti, D., Martines, A., Massidda, A., Schwenk, H., Barrault, L., Blain, F., Koehn, P., Christian, B. & Germann, U. (2014). The MateCat tool. In Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: System Demonstrations, pp. 129–132. Dublin, Ireland. Gerlach, J. (2015). Improving Statistical Machine Translation of Informal Language: A Rule-based Pre-editing Approach for French Forums. Geneva, Switzerland: University of Geneva. Gerlach, J., Porro, V., Bouillon, P. & Lehmann, S. (2013). La préédition avec des règles peu coûteuses, utile pour la TA statistique des forums ? In Actes de la 20e conférence sur le Traitement Automatique des Langues Naturelles (TALN), pp. 39–546. Sables d’Olonne, France. Guzmán, R. (2008). Advanced automatic MT post-editing. Multilingual Computing, 19(3), pp. 52–57. Han, B., Cook, P. & Baldwin, T. (2013). Lexical normalization for social media text. ACM Transactions on Intelligent Systems and Technology, 4 (5:1), pp. 5:27. Kuhn, T. (2014). A survey and classification of controlled natural languages. Computational Linguistics, 40(1), pp. 121–170. Mudge, R. (2010). The design of a proofreading software service. In Proceedings of the NAACL HLT 2010 Workshop on Computational Linguistics and Writing, pp. 24–32. Los Angeles, California. Naber, D. (2003). A rule-based style and grammar checker. (Unpublished Master’s Thesis). Universität Bielefeld. O’Brien, S. (2003). Controlling controlled English: An analysis of several controlled language rule sets. In Proceedings of EAMT-CLAW, pp. 105–114. Dublin, Ireland. O’Brien, S. and Moorkens, J. (2014). Towards intelligent post-editing interfaces. ­Paper presented at FIT XXth World Congress 2014, Berlin, Germany, 4–6 August 2014. O’Brien, S. and Roturier, J. (2007). How portable are controlled languages rules? A comparison of two empirical MT studies. In Proceedings of MT Summit XI, pp. 345–352. Copenhagen, Denmark. Porro, V., Gerlach, J., Bouillon, P. & Seretan, V. (2014). Rule-based automatic post-­ processing of SMT output to reduce human post-editing effort. In Proceedings of Translating and the Computer 36, pp. 66–76. London, England.

202

Bouillon et al.

Rayner, M., Bouillon, P. & Haddow, B. (2012). Using source-language transformations to address register mismatches in SMT. In Proceedings of the Conference of the Association for Machine Translation in the Americas (AMTA). San Diego, USA. Rosa, R., Mareček, D. & Dušek, O. (2012). DEPFIX: A system for automatic correction of Czech MT outputs. In Proceedings of the Seventh Workshop on Statistical Machine Translation, pp. 362–368. Montréal, Canada. Roturier, J. (2015). Localizing Apps: A practical guide for translators and translation students. Routledge: London, New York. Roturier, J., Mitchell, L. & Silva, D. (2013). The ACCEPT post-editing environment: A flexible and customisable online tool to perform and analyse machine translation post-editing. In Proceedings of MT Summit XIV Workshop on Post-editing Technology and Practice, pp. 119–128. Nice, France. Seretan, V., Roturier, J., Silva, D. & Bouillon, P. (2014). The ACCEPT Portal: An online framework for the pre-editing and post-editing of user-generated content. In Proceedings of the Workshop on Humans and Computer-Assisted Translation, pp. 66–71. Gothenburg, Sweden. Shardlow, M. (2014). A survey of automated text simplification. International Journal of Advanced Computer Science and Applications, 4, pp. 58–70. Simard, M., Goutte, C. & Isabelle, P. (2007). Statistical phrase-based post-editing. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics, pp. 508–515. Rochester, New York. Vieira, L.N. & Specia, L. (2011). A review of translation tools from a post-editing perspective. In Proceedings of the Third Joint EM+/CNGL Workshop Bringing MT to the User: Research Meets Translators (JEC’11), pp. 33–42. Luxembourg. Way, A. & Gough, N. (2005). Controlled translation in an example-based environment: What do automatic evaluation metrics tell us? Machine Translation, 19(1), pp. 1–36.

chapter 9

The Challenge of Machine Translation Post-editing: An Academic Perspective Celia Rico, Pilar Sánchez-Gijón and Olga Torres-Hostench Abstract The emergence of machine translation (mt) in professional translation practice has evolved from a topic of conversation among practitioners to promote a tangible change in the translation industry. The aim of this chapter is, then, to shed light on mt in professional and academic contexts by promoting a fresh approach to teaching using translation technology, and dealing with the needs and expectations of translators. Our work stems from considering the following key question: if the translation industry already considers post-editing as a viable service for almost any translation area, how should the academic world respond to this challenge? This question is addressed from three perspectives: (a) the evolution of translation technology and how post-editing has had an impact on the industry; (b) academic research paths in post-editing; and (c) training post-editors in a higher education context.

Keywords machine translation – translation – post-editing – training – academia – translation quality

1 Introduction The emergence of machine translation (mt) in professional translation practice has evolved from a topic of conversation among practitioners to promote a tangible change in the translation industry. With the rise of translation memories, key agents in the industry, whether major providers of translation s­ ervices or users of linguistic services, are putting pressure on other professionals involved in this industry (medium and small translation service providers and,

© koninklijke brill nv, leiden, ���8 | doi 10.1163/9789004351790_011

204

Rico, Sánchez-Gijón and Torres-Hostench

ultimately, individual translators) to incorporate mt into their daily tasks. The integration of mt into the workflow is a technological change, but above all it entails a change in translation practice. Many translators are reluctant to accept post-editing jobs as rates are lower, commissions may conflict with their concept of quality in translation and whether it is acceptable to offer a ‘good enough translation’. While the industry is still finding its bearings in this new scenario, unresolved questions continue to stoke the debate. What terms should be stipulated in contracts between suppliers and customers of translation services? How can translation quality be defined? How should post-­editors be trained? Working from this context, what is the role of universities? How can they reflect the current developments in, and demands of, both the translation industry and academia without disrupting the learning process? Research in this direction has already started in those academic institutions closely linked to the translation industry, resulting in significant contributions to post-editing practices. This has led to developing post-editing (and pre-editing) guidelines, designing and testing post-editing tools and implementing metrics for estimating (and predicting) translation quality. However, the industry requires a bold response from researchers in translation, one that accepts the risks and challenges in defining the skills needed to engage in post-editing, how to develop them or how to identify individuals best equipped for this task. Switching perspective, if we consider the interests of students opting for further training in postgraduate courses, we see again how universities are expected to provide a comprehensive education that facilitates access to the labour market in a rapidly growing area such as post-editing. Following the experience gained in this field through the research network ‘Post-It: Training post-editors’, together with related research initiatives, the authors of this chapter attempt to set out the position of universities on mt post-editing, both in terms of research and training. The aim of this work is, then, to cast light on mt in professional and academic contexts by promoting a fresh approach to teaching using translation technology, and dealing with the needs and expectations of translators. Our work stems from considering the following key question: if the translation industry already considers post-editing as a viable service for almost any translation area, how should the academic world respond to the challenge? This question is addressed in the sections that follow: Section 2 focuses on the evolution of translation technology and the impact of post-editing on the industry; Section 3 presents research questions and methodologies that should be applied in post-editing research; and Section 4 introduces different aspects of post-editors training in a higher ­education context.

The Challenge of Machine Translation Post-editing

2

205

The Evolution of Translation Technology or How Bar-Hillel was Right

The evolution of translation from craftsmanship to an industrial process goes hand in hand with technology and is directly affected by it. In the last 40 years, the job of the translator has shifted its object of attention from documents on paper, translated manually without the assistance of a single tool, to digital content embedded in all kinds of gadgets, apps and devices, incorporating a myriad of formats and also serving a multitude of purposes. There is now a whole new set of features which can be added to the most conventional definition of the discipline: it is digital, online, free, progressively dependent on data (either as a reference in translation memories or for training mt systems), social, collaborative, crowdsourced, customised, etc. These are all attributes that necessarily signal a transformation in the nature of translation. At the same time, the market, valued at us$37.19 billion in 2014 and growing at an annual rate of 6.23% (DePalma et al., 2014), is still chasing a never-ending dream: fully automated tasks and maintaining a balance between unstable variables such as cost, quality and turnaround. Even if recent developments in mt give the ‘magical’ impression that part of this dream has come true, the human portion of the workflow remains the key value in translation. In the early days mt research focused on achieving ‘Fully Automatic High Quality Translation’ but back then, Bar-Hillel (1960: 91–163) was right when he declared that ‘the idea of inventing a method for fully automatic high quality translation (fahqt) is just a dream which will not come true in the foreseeable future’. So true in fact that even today, when we see mt reaching a popularity peak, we are beginning to realise that our efforts should not be aimed at fahqt but rather on faut or ‘Fully Automatic Useful Translation’ (Van der Meer, 2006), that is to say, on producing mt output that offers ‘good enough’ quality based on user type and levels of acceptance. This change in expectations is the reason behind the growing interest in post-editing as a task. Post-editing Has (Almost) Always Been There but Only Now is Having an Impact on the Industry The latest survey on the mt market (Van der Meer & Ruopp, 2014) reveals that post-editing of mt output accounted for 2.47% of revenues (us $828.02 million), with 38.63% of the 1,119 respondents reporting that they offered postediting services. In the last couple of years or so, an apparently new role has emerged in the Language Service Provider (lsp) world: that of the post-editor. The debate in the industry is still open regarding for whom, how and when postediting is appropriate, if at all. While views differ widely among t­ranslators, 2.1

206

Rico, Sánchez-Gijón and Torres-Hostench

the fact is that the industry is progressively incorporating mt as a service and clients are requesting it. Even though recent developments in mt have placed post-editing at the centre of the debate, the truth is that post-­editing as a profession itself can be traced back to 1985, with reports of work carried out at the Pan American Health Organization. mt had been implemented some years earlier for translating from Spanish into English, with a productivity of 4,000 to 10,000 words a day per post-editor (Vasconcellos & León, 1985: 134). Postediting was then defined as ‘adjusting the machine output so that it reflects as accurately as possible the meaning of the original text, [with an emphasis on] adjusting relatively predictable difficulties’ (Vasconcellos, 1987: 411). From then on, the term post-editor was progressively used in the works of Arnold et al. (1994: 33–35), Clarck (1994: 302) and Somers (1997), among others. There are two milestones as regards research into this field: Krings and Koby (2001), Allen (2003). While the former concentrate on the analysis of the cost and effort associated with post-editing as compared to conventional human translation, the latter approaches the topic from a practical perspective, giving specific guidelines and actual examples of the task. We can see how the role of the post-editor has been gaining momentum over the last decade in all aspects of research: didactic approaches (O’Brien, 2002 and 2011a; O’ Brien, Roturier & Almeida, 2009; Torrejón & Rico, 2002; Rico & Torrejón, 2012); the integration of mt with a commercially available post-editing solution (Beaton and ­Contreras, 2010); measuring post-editing effort related to mt output quality (Guerberof, 2009a, Thicke, 2011, Plitt & Masselot, 2010, Roturier, 2004, Specia & Farzindar, 2010, Specia, 2011); automated post-editing (Lawson-Tancred, 2008); assessing­and developing post-editing tools (Vieira & Specia, 2011, Aziz et al., 2012); post-editing­as a viable alternative to conventional translation (­Garcia, 2011, Sánchez-Gijón & Torres-Hostench, 2014); and estimating productivity (Guerberof, 2009b, O’Brien, 2011b). Similarly, relevant journals such as JosTrans, Tradumàtica or Machine Translation have devoted monograph issues to this topic (Brunette­& Gerber, 2013; de la Fuente, 2012; O’Brien & Simard, 2014). All of these taken together with numerous debates held in online professional forums, training courses and frequent job offers, definitively underline the key role of post-editing in the translation industry. Now, for a better understanding of what post-editing involves we would like to work from the definition put forward by the Centre for Next Generation for Localisation (cngl) and the Translation Automation User Society (taus) in January 2011: Post-editing is the correction of machine-generated translation output to ensure it meets a level of quality negotiated in advance between client and post-editor.

The Challenge of Machine Translation Post-editing

207

From this definition we can consider the following series of task-related processes (Rico & Torrejón, 2012): • Source text: reading the source text and using it as a reference to recognise patterns for reformulating in the target text (morphological, syntactic or semantic) or to make decisions on textual coherence. • mt: reading the mt output either entirely or in segments, directing attention to elements which need further confirmation in the source text and evaluating whether reformulation is necessary. • Target text production: producing a new text either from old elements already present in the text or adding new ones. This involves language correction according to post-editing guidelines and distinguishing between full and light revision (Allen, 2003; taus/cngl, 2011). • mt output evaluation: making positive or negative evaluations of the mt output and comparing it with the source text, which, in turn, is related to defining quality in terms of client’s expectations. • Documentation: choosing dictionaries to be used (if any), collocations to be found, parallel texts to be consulted and/or asking for informants when necessary. • Writing: writing linearly, deleting or inserting elements, leaving a gap, marking specific elements, overwriting and rewriting. • Management: reporting feedback to allow for mt improvement and/or source content optimization, collecting samples of different post-editing issues in order to facilitate training of other fellow post-editors in the team, and keeping up-to-date with the latest advances in the field of mt and pre/ post-editing tools. There seems to be a general agreement that post-editing should be conducted by translators as, ‘only a translator can judge the accuracy of a translation, [as] the one best able to pick up errors in the machine translation, he has a fund of knowledge about the cross-language transfer of concepts, and he has technical resources at his disposal which he knows how to use in the event of doubts’ (Krings & Koby, 2001: 12). If this is the case, then what is the role of universities in training post-editors? In this respect, the numerous challenges that lay open for academia beg the question as to whether these changes are having a minimal impact on the working conditions of professional translation or rather a change of paradigm is needed in Translation Studies if these are to account for the many advances in translation technology. Following Rozmyslowicz’s proposal (2014: 148) we should be leaving aside demonizing patterns that consider mt (and post-editing as an associated task) as the ultimate externalization of the translator, and free ourselves from the idea that it involves

208

Rico, Sánchez-Gijón and Torres-Hostench

a ‘mechanized and automated objectification of everything that has become suspicious about the linguistic past of translations studies’. So, we should look beyond the instrumentalist agenda that concentrates on the technical properties of the technologies involved (Kenny & Doherty, 2014: 276), and approach the research on and teaching of post-editing from a balanced integration of procedural and declarative knowledge, competences development and harmonization of demands from the industry. This approach is developed further in the following section. 3

Research Paths in Post-editing

As mentioned earlier, with the emergence of digital products with embedded text and the appearance of specific translation tools to deal with them, research on post-editing developed in the translation sector has been one step ahead of academic research. Since Translation Studies has begun to pay attention to post-editing as a new translation modality worth exploring, a broad range of possibilities of cooperation between the industry and academia have emerged. One of the first research questions that Translation Studies have had to tackle is how to define post-editing. To delve further into these descriptive research questions implies collaboration between academia and the industry. On the one hand, the industry provides researchers with research evidence (data), instruments and tools needed for an in-depth description of post-editing as a translation modality. On the other hand, researchers contribute by designing research projects, applying the most appropriate methodologies for each research question, and ensuring that results obtained in each observational research, empirical test or experiment really address the question posed. Apart from the descriptive approach, the translation industry and academic researchers are also interested in studies that deal with how to improve postediting results. Here, it is extremely important to first define the research issue clearly. Enhancing post-editing involves many different issues and not all of them can be addressed at the same time or by analysing the evidence obtained through the same observational research, empirical test or experiment. The following are attempts to classify these research issues: • Product-oriented research. This includes questions such as how to determine the quality of raw mt output before or after post-editing, how to analyse whether there are significant differences between translated from raw and post-edited translations, or if improvements to the use of language in

The Challenge of Machine Translation Post-editing

209

the original text could result in a better quality raw mt output that could be easier to post-edit. The objects of study for these questions are mainly postedited texts. • Process-oriented research. This addresses research questions which usually deal with looking for differences between translation and post-editing processes, apart from the fact that post-editing involves the use of an mt system. Most of the time these studies begin with an observational research that reveals the weaknesses of the process. Afterwards these studies become the starting point for technical or procedural improvements. • Productivity-oriented research. This attempts to find out under what conditions raw mt output post-editing is more productive than translating. To do this it is important to first decide what kind of productivity is to be observed, whether productivity is just being measured in terms of time invested to perform a (translation or post-editing) task, or if factors such as human and technical resources should also to be taken into account. Most of the time only productivity in terms of time invested is observed and analysed, but both research questions and results seem to be aimed at productivity in general, without paying enough attention to human and technical investment, which also represent a cost and, consequently, have an effect on productivity. • Cognitive research. This category is gaining ground in Translation Studies, although it is also attracting interest from the translation industry. Research questions from this category try to find out how the translator and posteditor black box works. They deal basically with two different approaches. On the one hand, post-editing might be used as a methodology to gain insights into the decision-making process of translators when facing translation problems. On the other hand, now that post-editing has begun to be a full-time professional task for translators, it has come to the attention of some translation companies that post-editing seems to have a psychologically stressing component that is not usually found in translation tasks. To find out whether this component really exists and how to avoid it are the objectives of this kind of research study. Although it may seem a minor point, methodological issues in research on post-editing play an important role in achieving reliable and comparable results. That is the reason why valuable results can only emerge for both the translation industry and Translation Studies by posing a viable hypothesis, searching for or collecting the evidence needed to address this and applying the appropriate methods of analysis and research on the post-editing modality. Up to now, many of the studies carried out in this field give very valuable

210

Rico, Sánchez-Gijón and Torres-Hostench

insights into what post-editing involves. But in most cases, studies show some weaknesses in one (or more than one) of the following areas: • The nature of research evidence. Data driven research usually needs as much data as possible to demonstrate conclusive results. Results obtained from poor data in terms of quantity or homogeneity may be inconclusive or just not transposable to any other post-editing framework. • Incoherence between the research question and research evidence. Even though research evidence may be quantitatively and qualitatively useful, they might not be appropriate to answer a particular research question. For example, if the research question deals with the skills of professional posteditors, data obtained from non-professionals (such as students) will not shed any light. The same applies if certain post-editing circumstances are the focus of observation (such as language combination, use of tools, application of particular guidelines, etc.), but the data used were obtained under different circumstances. Unfortunately this is not always easy to take into account, particularly when textual evidence from the translation industry (translated and post-edited texts) are being analysed. • Incoherence between the research question and methodology. Choosing the right research methodology is essential to reach concluding results. It is difficult to isolate just one factor within complex processes, analyse it and draw conclusions without paying attention to the other factors involved in the process. In translation, and particularly in post-editing, it is very difficult to isolate just one factor in the whole process, for instance, the performance of professionals post-editing a particular text. One cannot assume that other factors involved (e.g. previous experience in the same topic, in the same kind of task, in the use of the same tools, etc. of the participants in the test) will not play any role nor affect the final results. That is why it is crucial to carefully design the research tests following the appropriate methodology and using suitable research instruments. Failure to take the above weaknesses into account before embarking on a ­research project may result in misinterpretations of the results obtained rendering them less useful or applicable to other post-editing frameworks than originally anticipated. If research projects are methodologically sound, both the industry and academia will benefit from the results obtained. The industry will cast light on how to improve tools and productivity, while academia will further develop the post-editing skills that should be part of any translators’ training program. What follows are some insights on how to train post-editors at a graduate level.

The Challenge of Machine Translation Post-editing

4

211

Training Post-editors in Higher Education Contexts

This section presents a case study from the thematic network for training posteditors: Post-it: Training post-editors.1 This case study has a two-fold objective: to present a successful post-editing training experience and to reflect on the results for further improvement. The experience took place in a post-editing seminar integrated into the course Audiovisual Translation and Localization, for fourth-year students at the Universitat Autònoma de Barcelona. The seminar was attended by thirty students who had no previous knowledge of post-editing. It consisted of 20 hours work for the students, distributed across three different learning contents: theory classes (3 hours), practical classes (2 hours) and working in small groups outside class (15 hours). The contents of the theory classes were related to the evolution of mt, types of mt systems, mt pre-editing and basic preediting criteria, mt post-editing and basic post-editing criteria. The practical class consisted of post-editing a 20-sentence text using taus Dynamic Quality Framework (dqf)2 (taus, 2014) and analysing the results on productivity and quality together in the class. Finally, students worked in groups and were given three sets of tasks. The first concerned post-editing: each student translated the same text selected by the teacher with a different online mt system (Google Translator, Bing or Apertium). Students then identified and classified the errors made by the selected mt system in a text processor file. Afterwards they post-edited the raw mt according to the criteria given in class. The second set of tasks was related to pre-editing. Taking into account the mistakes corrected in the first step, the students pre-edited the original text so that these errors could be avoided to a certain extent and translated the pre-edited text with the same mt system used before. The students checked whether mt mistakes reoccurred in the raw translation and analysed which pre-editing changes were more likely to prevent mt errors. Then the students analysed both post-editing (with and without pre-editing) and made suggestions for pre-editing and postediting. The third set of tasks comprised making a comparison of mt systems. The three students of the group prepared a final group analysis comparing the three systems and presenting their conclusions. The students followed the instructions without any difficulty, completed all the given tasks successfully and grasped the concepts of pre-editing and postediting. The following observations were made after analysing the students’ work: 1 POST-IT: https://sites.google.com/a/tradumatica.net/postit_en/. 2 taus Dynamic Quality Framework https://www.taus.net/quality-dashboard-lp.

212 1

2 3

4

Rico, Sánchez-Gijón and Torres-Hostench

They were able to correctly identify mt system mistakes and classify the errors. In particular, they detected: untranslated text; omissions (for instance, articles); unnecessary repetitions; literal translations; grammar mistakes such as in concordances, in genre and number, prepositions, verb tense mistakes, wrong use of subordinates and impersonal structures; punctuation mistakes; style mistakes and orthography mistakes. They were able to identify those text features that make the process of machine translation more difficult, such as compound sentences, compound words, synonyms, polysemy, ambiguity and figurative language. They were able to pre-edit the original text to simplify it and make postediting easier. For instance, they made changes to the punctuation of the original text, changed ambiguous or polysemic words, changed subjunctive verb tenses into indicative and the conditional verb tense into present, used more passives, modified the word order, simplified syntax, changed idioms and figurative language to unambiguous words and sentences and made terminology coherent. They realised that the simplification of the text was essential to enhance the machine translation and make post-editing easier. They correctly applied the post-editing criteria learned in the theory classes. In particular, they verified that no information from the original was missing; they corrected all spelling mistakes (Spanish accents, apostrophes, hyphens, inverted commas, etc.); they corrected syntax (i.e. ­literal translations, non-sense sentences, etc.); they ­corrected the grammar mistakes, such as the incorrect use of prepositions; they corrected the wrong terminology; they corrected the punctuation; and last but not least, they checked for bias in relation to the assignment.

With this training input we observed that the students not only learned the basic techniques for post-editing, but, more importantly, they acquired a new perspective on machine translation that changed their preconceptions on machine translation. Before the task, their opinion on machine translated was based on assumptions such as: 1

Machine Translation Systems Do Not Translate Well: At first sight the machine translated pretty well. There was no sentence that we could not understand or was completely mistranslated. This says a lot for a machine. Compared to the other two mt systems, this system produces quite a decent translation taking into account our low expectations of mt.

The Challenge of Machine Translation Post-editing

213

Machine translation, as it could not be otherwise, has limitations because it is a machine. We have certain preconceived notions of what a machine translation system is and how it works. We thought that we would find a translation of a very poor quality and we would have to correct most of it, if not all. 2

Machine Translation is the Translator’s Enemy: In conclusion, leaving aside that mt is useful for translators and it has to be accepted as a work tool (and not seen as our enemy), it is clear that all translations need to be revised and more editing, and this is exactly what post-editing is about. In this task we used a translation method [mt] that scares us a little bit, partly because we do not trust it – we have been told in all our courses that it is not a tool that we should use to work, and we have never used it to work – but also because customers prefer to buy translation services taking into account price criteria instead of quality criteria, and human translators lose customers. Human translators have to look for other ways to make profits, so pre-editing and post-editing may be a way to make profits as well.

3

Human Translators are Superior: The machine translation system will not pick up all the details but a human translator will. The mt system has neither the capacity to think nor take translation decisions regarding the end reader, context or function of the text, etc. It does not have the capacity to consider all the fundamental issues related to translation that makes a good quality translation and, all said and done, all companies try to give a reliable image. We also think that human translator criteria should always prevail over the machine’s criteria. mt systems are seen as a form of cheating because the quality achieved by mt systems is not the same as the one achieved by a person.

Nevertheless, in the group work the students were surprised by the results of machine translation: However, after having analysed the results and having observed the weak features of the system as well as the strong ones, there is no doubt that this mt system shows promise.

214

Rico, Sánchez-Gijón and Torres-Hostench

After this work of pre-editing and post-editing, we have realised that the quality of mt is better than expected. (…) with this task we have been positively surprised by the mt system. (…) we have to admit that it does not translate badly and we expected a worse result. When we started this group work we did not give much credibility to any mt system. However, after finishing these tasks we have to admit that some systems are better than others and that the quality offered varies a lot across these systems. Obviously it cannot be compared to a human translation, but with some effort from the translator, the result offered by the machine translation can be acceptable and fulfil the same function as a human translation. mt Systems surprised us for the better and the worse, but mostly for the better, as the changes we had to make to the raw mt were insignificant because most problems were stylistic and they did not prevent understanding the text. Inevitably, many questions arise for translation lecturers and universities from the students’ answers. For how long are universities going to ignore machine translation? (Piqué Huerta & Colominas, 2013) Do we want to see how the evolution of machine translation leaves our students unprepared? Fortunately, at the end of the training sessions the students themselves drew their own conclusions. It seems they will not turn their backs on machine translation anymore and they will not rule out mt post-editing in the future: Despite this not being the ideal job for a graduate in translation and interpreting, working in post-editing may be one of our first working experiences or a complement to the translation task. We have learned that mt is unthinkable without the human value that translators represent. Therefore, we do not have to be afraid of machine translation but rather to add it to our future professional opportunities. As the saying goes: “if you can’t beat them, join them”. 5 Conclusions The industry needs universities to play an active role in the challenges posed by machine translation post-editing. Ideally, this should be: (a) analysing the evolution of translation technology and the impact of post-editing on the

The Challenge of Machine Translation Post-editing

215

translation industry; (b) research into post-editing; (c) training post-editors in higher education. In short, academia should do more than teaching postediting skills. As seen in this chapter, advances have been made in the development of post-editing (and pre-editing) guidelines, the design and testing of post-­ editing tools and the implementation of metrics for estimating (and predicting) translation quality. However, assuming that the final aim is to improve post-editing results, there is still much more that can be done by academia. This chapter has put forward suggestions and examples on research design for studies on post-editing. As pointed out in Section 3, research questions on post-editing could not only adopt a cognitive approach but also be product-­ oriented, process-oriented and/or productivity-oriented. Some examples and suggestions have been given concerning the weaknesses of research evidence, inconsistencies between the research question and research evidence and between the research question and methodology. Nonetheless, whatever kind of research on post-editing may be carried out, there is no doubt that this professional modality of translation is starting to change the translation paradigm in ­Translation Studies. Apart from the fact that the starting point is a source text which is in the same language as the target text (although it might not be considered a ‘text’ strictly speaking due to incoherencies, inconsistencies and language errors in general it may contain), post-editing also introduces a shift in terms of skills needed to produce a functional translation. In fact, post-­editing is even changing the concept of functional translation itself, depending on the degree of visibility of the target text, that is, the use that the client will do of the translation. All these issues open a new range of challenging research questions that Translation Studies researchers are already tackling. Finally, from a training perspective, the authors present a successful experience in which the students not only learned basic post-editing techniques, but, more importantly, they acquired a new perspective on machine translation that changed to some degree their preconceptions on machine translation. The fact is that we should design teaching scenarios that engage students and teachers in a community in which learning is the result of interactions, reflections, and experiences. The authors also hope that this chapter will change to some degree preconceptions about machine translation post-editing by translation studies research and teaching. The authors also feel confident that post-editing research and training will be an important step forward in the given a new paradigm in Translation Studies.

216

Rico, Sánchez-Gijón and Torres-Hostench

Acknowledgements This study was supported by the Spanish Ministry of Economy and Competitiveness. A Spanish State programme for Research, Development and Innovation geared towards meeting challenges posed by society (Grant Number FFI2013-46041-R). References Allen, J. (2003). Post-editing. In Somers H. (Ed.) Computers and Translation: A translator’s guide. Amsterdam/Philadelphia: John Benjamins. Retrieved from http://www .mt-archive.info/Bar-Hillel-1960-App3.pdf (Consulted on 03/03/2017). Arnold, D.L., Balkan, S., Meijer, Humphreys R. & Sadler, L. (1994). Machine Translation: An Introductory Guide. Manchester: NCC Blackwell. Retrieved from http://www.es sex.ac.uk/linguistics/external/clmt/MTbook/ (Consulted on 03/03/2017). Aziz, W., Sousa, S.C.M. & Specia, L. (2012). PET: a tool for post-editing and assessing machine translation. Paper presented at the The Eighth International Conference on Language Resources and Evaluation, LREC’12, Istanbul, Turkey. Bar-Hillel, Y. (1960). The present status of automatic translation of languages. Advances in computers 1(1), pp. 91–163. Beaton, A. & Contreras, G. (2010). Sharing the Continental Airlines and SDL postediting experience. Paper presented at the AMTA 2010, The Ninth Conference of the Association for Machine Translation in the Americas. Denver, Colorado. Retrieved from http://www.mt-archive.info/AMTA-2010-Beaton.pdf (Consulted on 03/03/2017). Brunette, l. & Gerber, L. (2013). JosTrans, Special issue on Machine translation and the Working Methods of Translators, 19. Retrieved from http://www.jostrans.org/issue19/ issue19_toc.php (Consulted on 03/03/2017). Clarck, R. (1994). Computer-assisted translation: The state of the art. In Dollerup C. & Lindegaards A. (Eds.), Teaching translation and interpreting, II: insights, aims, visions, pp. 301–308. Amsterdam: John Benjamins. de la Fuente, R. (2012). Posedición, cambio de paradigma? Tradumàtica: traducció i tecnologies de la informació i la comunicació 10, pp. 147–149. DePalma, D., Vijayalaxmi, H., Pielmeier, H. & Stewart, R.G. (2014). The Language Services Market 2014. Cambridge, Massachusetts: Common Sense Advisory. Garcia, I. (2011). Translating by post-editing: is it the way forward? Machine Translation, 25(3), pp. 217–237. Guerberof, A. (2009a). Productivity and quality in MT post-editing. Paper presented at the MT Summit 2009 Workshop 3. Retrieved from http://www.mt-archive.info/MTS -2009-TOC.htm (Consulted on 03/03/2017).

The Challenge of Machine Translation Post-editing

217

Guerberof, A. (2009b). Productivity and quality in the post-editing of outputs from translation memories and machine translation. Localization focus, 7(1), 11–21. Kenny, D. & Doherty, S. (2014). Statistical machine translation in the translation curriculum: overcoming obstacles and empowering translators, The Interpreter and Translator Trainer, 8(2), pp. 276–294. Krings, P. & Koby, G.S. (Eds.). (2001): Repairing Texts. Empirical Investigations of Machine Translation Post-Editing Processes. Kent: The Kent State University Press. Lawson-Tancred, H. (2008). Monolingual translation: automated post-editing. Multilingual, vol. April/May, pp. 40–46. O’Brien, S. (2002). Teaching post-editing: a proposal for course content. Paper presented at the 6th EAMT Workshop, Teaching Machine Translation, November 14–15, 2002. European Association for Machine Translation, pp. 99–106. Retrieved from http:// www.mt-archive.info/EAMT-2002-OBrien.pdf (Consulted on 03/03/2017). O’Brien, S. (2011a). Introduction to Post-Editing: Who, What, How and Where to Next? Retrieved from http://www.cngl.ie/node/2542 (Consulted on 03/03/2017). O’Brien, S. (2011b). Towards predicting post-editing productivity. Machine Translation, 25(3), pp. 197–215. O’Brien, S. & Simard, M. (2014). Machine Translation. Special Issue: Post-Editing 28 (3–4), December 2014 Retrieved from http://link.springer.com/journal/10590/28/3/ page/1 (Consulted on 03/03/2017). O’Brien, S. Roturier, J. & Almeida, G.D. (2009). Post-Editing MT Output. Views for the researcher, trainer, publisher and practitioner. Paper presented at the MT Summit 2009 tutorial. Retrieved from http://www.mt-archive.info/MTS-2009-OBrien-ppt .pdf (Consulted on 03/03/2017). Piqué Huerta, R. & Colominas, C. (2013). Les tecnologies de la traducció en la formació de grau de traductors i intèrprets. Revista Tradumàtica, 11. http://revistes.uab.cat/ tradumatica/article/view/43/pdf (Consulted on 03/03/2017). Plitt, M. & Masselot, M. (2010). A Productivity Test of Statistical Machine Translation The Prague Bulletin of Mathematical Linguistics, 93, pp. 7–16. Rico, C. & Torrejón, E. (2012). Skills and profile of the new role of the translator as MT post-editor. Tradumàtica, 10, pp. 166–178. Roturier, J. (2004). Assessing a set of Controlled Language rules: Can they improve the performance of commercial Machine Translation systems? Paper presented at Aslib, Translating and the Computer 26. London. Rozmyslowicz, T. (2014). Machine Translation: A Problem for Translation Theory. New Voices in Translation Studies 11, pp. 145–163. Sánchez-Gijón, P. & Torres-Hostench, O. (2014). MT Post-editing into the mother tongue or into a foreign language? Spanish-to-English MT translation output postedited by translation. Proceedings of the Third Workshop on Post-Editing Technology and Practice, pp. 5–19, AMTA 2014.

218

Rico, Sánchez-Gijón and Torres-Hostench

Somers, H. (1997). A practical approach to using machine translation software: Postediting the source text. Translator 3 (2), pp. 193–212. Specia, L. (2011). Exploiting Objective Annotations for Measuring Translation Post-­ Editing Effort. Proceedings of the EAMT. Leuven, Belgium. Retrieved from http:// www.ccl.kuleuven.be/EAMT2011/ (Consulted on 03/03/2017). Specia, L. & Farzindar, A. (2010). Estimating Machine Translation Post-Editing Effort with HTER. AMTA-2010 Workshop Bringing MT to the User: MT Research and the Translation Industry. Denver, Colorado. Retrieved from http://amta2010.amtaweb .org/ (Consulted on 03/03/2017). TAUS (2014). TAUS Dynamic Quality Framework. Retrieved from https://evaluate .taus.net/evaluate/dqf/dynamic-quality-framework#evaluate-quality (Consulted on 03/03/2017). TAUS/CNGL (2011). TAUS/CNGL Machine Translation Post-Editing Guidelines. Retrieved from https://www.taus.net/academy/best-practices/postedit-best-practic es/machine-translation-post-editing-guidelines (Consulted on 03/03/2017). Thicke, L. (2011). Improving MT results: a Study. Multilingual, February, pp. 37–40. Torrejón Díaz, E. & Rico Pérez C. (2002, November). Controlled Translation: A new teaching scenario tailor-made for the translation industry. Paper presented at the 6th EAMT Workshop, Teaching Machine Translation, European Association for Machine Translation, pp. 107–116. Retrieved from http://www.mt-archive.info/EAMT -2002-Torrejon.pdf (Consulted on 03/03/2017). Van der Meer, J. (2006). The Emergence of FAUT: Fully Automatic Useful Translation. Keynote at the 11th Conference of the European Association for Machine Translation. Retrieved from http://www.mt-archive.info/EAMT-2006-VanderMeer.pdf (Consulted on 03/03/2017). Van der Meer, J. & Ruopp, A. (2014). Machine Translation Market Report. The Netherlands: TAUS. Vasconcellos, M. (1987). A comparison of MT postediting and traditional revision. In Kummer K. (Ed.). Proceedings of the 28th Annual Conference of the American Translators Association, pp. 409–415. Medford, NJ: Learned Information. Vasconcellos, M. & León M. (1985). SPANAM and ENGSPAN: Machines Translation at the Pan American Health Organization. Computational Linguistics 11, pp. 122–136. Retrieved from http://acl.ldc.upenn.edu/J/J85/J85-2003.pdf (Consulted on 03/03/2017). Vieira, L. & Specia, L. (2011). A Review of Machine Translation Tools from a Post-Editing Perspective. Paper presented at the 3rd Joint EM+/CNGL Workshop Bringing MT to the User: Research Meets Translators (JEC 2011), Luxembourg.

chapter 10

scate Taxonomy and Corpus of Machine Translation Errors Arda Tezcan, Véronique Hoste and Lieve Macken Abstract Quality estimation (qe) and error analysis of machine translation (mt) output remain active areas in Natural Language Processing (nlp) research. Many recent efforts have focused on machine learning (ml) systems to estimate the mt quality, translation errors, post-editing speed or post-editing effort. As the accuracy of such ml tasks relies on the availability of corpora, there is an increasing need for large corpora of machine translations annotated with translation errors and the error annotation guidelines to produce consistent annotations. Drawing on previous work on translation error taxonomies, we present the scate (Smart Computer-aided Translation Environment) mt error taxonomy, which is hierarchical in nature and is based upon the familiar notions of accuracy and fluency. In the scate annotation framework, we annotate fluency errors in the target text and accuracy errors in both the source and target text, while linking the source and target annotations. We also propose a novel method for alignment-based inter-annotator agreement (iaa) analysis and show that this method can be used effectively on large annotation sets. Using the scate taxonomy and guidelines, we create the first corpus of mt errors for the English-Dutch language pair, consisting of statistical machine translation (smt) and rule-based machine translation (rbmt) errors, which is a valuable resource not only for nlp tasks in this field but also to study the relationship between mt errors and post-editing efforts in the future. Finally, we analyse the error profiles of the smt and the rbmt systems used in this study and compare the quality of these two different mt architectures based on the error types.

Keywords machine translation – error classification – bilingual corpus

© koninklijke brill nv, leiden, ���8 | doi 10.1163/9789004351790_012

220

Tezcan, Hoste and Macken

1 Introduction According to Rinsche and Portera-Zanotti (2009: 5), ‘it seems very likely that the use of machine translation will grow to cater for exponentially rising translation needs in increasingly globalized contexts, combined with a considerable lack of properly skilled human resources’. However, while Machine Translation (mt) systems slowly become usable in some restricted scenarios, mt quality in unrestricted domains is highly variable. In fact, in order to obtain translations of high, publishable quality, humans still need to intervene in the translation process and do this most frequently by post-editing mt output (i.e. a ‘translateand-revise’ approach). Despite the continuous improvements, mt systems are far from perfect and translations generated by mt systems often contain errors. Even though there are many automatic metrics for quality assessment, these metrics do not provide additional information about the nature of errors that are made by mt systems. Instead, they provide scores at sentence or document level. Automatic error analysis is, therefore, essential to improve current mt systems and understand the relationship between specific mt errors and post-editing efforts. mt Quality Estimation (qe) aims to provide an estimate of the quality of unseen automatic translated sentences without relying on reference translations. Most qe systems treat this problem as a supervised Machine Learning (ml) task (Bach et al., 2011; Specia et al., 2013; Biçici, 2013; de Souza et al., 2014). Such systems need labelled training data and use a variety of algorithms to train models on the given training set based on features that describe the source text, the mt output and the word and phrase alignments between the source text and the mt suggestion. The trained models are then used for automatic quality estimation of new mt output. As the accuracy of such ml tasks relies on available corpora, there is an increasing need for large corpora of machine translations annotated with translation errors. Several translation error taxonomies have been proposed, each with a specific goal in mind. Mateo (2014: 75) states that ‘the search for a unique method for Translation Quality Assessment (tqa) that could achieve full objectivity in every situation, context and for every type of text seems illusionary’. Quite apart from the fact that there are different ways of classifying errors, manual error annotation itself is a subjective task, which leads to disagreements between annotators (Popović & Burchardt, 2011). Moreover, inter-annotator agreement (iaa) on mt error annotations is difficult to measure, given the different aspects involved in annotation disagreements, such as error category, and the position and the span of errors (Stymne & Ahrenberg, 2012). Referring to MT-specific errors, Flanagan (1994) claims that mt quality can be difficult

scate Taxonomy and Corpus of Machine Translation Errors

221

to quantify for a number of reasons, including the possibility of obtaining multiple correct translations, the difficulty to define error boundaries, the cause of errors not always being apparent and the accumulation of errors. Lommel et al. (2014) also report that a simple list of issue types and definitions along with general guidelines proved insufficient to guide annotators when faced with unfamiliar issues in mt output. These observations underline the need for detailed annotation guidelines in order to ensure the quality of the error annotation task and to minimize disagreements among multiple annotators. In this chapter, we describe the scate (Smart Computer-aided Translation Environment) mt error taxonomy, which is organised hierarchically and contains different subcategories based on the well-known distinction between accuracy and fluency. We show how this taxonomy was used to build an annotated corpus of mt errors, which will be used as training corpus to develop an automatic qe system for mt. We also discuss our methodology to capture the iaa on a large-scale error annotation task. To the best of our knowledge, the scate corpus of mt errors is the first effort of its kind to build an EnglishDutch corpus of both smt and rbmt output, labelled with fine-grained error annotations. It differs from the existing error-annotated corpora in that it links the accuracy errors annotated in the mt output to the corresponding source text fragments they originate from, which allows us to study the impact certain source text characteristics have on mt quality. 2

Related Work

There has been a considerable amount of work dedicated to classifying mt errors (Flanagan, 1994; Vilar et al., 2006; Costa et al., 2015), human errors (sae J2450, 1999; lisa, 2007; Wisniewski et al., 2014), and both types of translation errors combined (Lommel et al., 2014; Daems et al. 2014). To classify mt errors, Flanagan (1994) proposes a list of fine-grained categories, which were identified by observing the most frequent error types (such as spelling, rearrangement, category, conjunction and clause boundary) and ­underlines that while some error categories may well apply to all languages, additional language-pair specific error categories should complement this categorization. The taxonomy suggested by Vilar et al. (2006), which inspired other work on translation error classification (Avramidis & Koehn, 2008; Popović & Burchardt, 2011), uses a hierarchical scheme and divides the mt errors into five broad categories: omissions, word order errors, incorrect words, unknown words and punctuation errors. Some additional error categories were added to this list to account for English-Chinese mt errors. Farrús et al. (2011) and

222

Tezcan, Hoste and Macken

Costa et al. (2015) propose linguistically-motivated, language-­independent ­taxonomies, which classify the mt errors into different linguistic levels. In this respect, the orthographic, lexical and semantic levels are common in both taxonomies. In an attempt to handle the highly variable subjectivity of quality evaluation of mt output at sentence level, White (1995) was in favour of drawing a distinction between adequacy1 and fluency. Whereas adequacy is concerned with how much of the source content and meaning is also retained in the target text, fluency is concerned with the extent to which the translation flows well, regardless of sentence meaning. This distinction between accuracy and fluency was suggested to break down human translation quality judgments into separate and smaller units (White, 1995). Toury (2000) drew a similar distinction by referring to the adherence to the norms of the source text as adequacy and adherence to the norms of the target text as acceptability.2 Lommel et al. (2014) took the distinction of accuracy and fluency as a basis for translation error classification in the Multidimensional Quality Metrics (mqm). On the assumption that no single, fixed metric can be used to assess translation quality for a wide range of different translation tasks, mqm is intended to be used in a customized manner, by selecting a relevant subset of issues from the full list of 108 errors types. While most of the taxonomies described in this section provide guidelines on the way error types should be annotated, the level of detail varies. Most of these guidelines do not inform annotators about the annotation spans, nor do they include examples, especially for the potentially confusing error categories. The work of Lommel et al. (2014) and Daems et al. (2014) contains detailed guidelines as to how and where errors should be annotated, which improves the applicability of these taxonomies in new studies. A number of error taxonomies that focus on final translation output (human translations or post-edited machine translations) have been introduced in the localisation industry, such as the lisa qe Model (2007) and sae J2450 (1999). Both metrics provide a list of error types and rank errors according to severity to calculate quality scores, which raises another concern, namely subjectivity. Due to either the scope of the approach (lisa qa Model) or the limited applicability (sae J2450), these metrics often need to be customized to meet specific requirements (Lommel et al., 2014). Nevertheless, the idea of using error weights to reflect the required post-editing effort seems valuable. 1 Adequacy and accuracy are often considered synonyms. 2 Toury’s definition of acceptability is broader than the definition of fluency and also encompasses problems related to the target text as a whole and the target text in context.

scate Taxonomy and Corpus of Machine Translation Errors

223

One of the key requirements of demonstrating the validity and reliability of an assessment method is that the annotations can be applied consistently, yielding a high iaa (Lommel et al., 2014). According to Stymne and Ahrenberg (2012), even though ‘subjectivity’ is a key word in earlier studies on translation error analysis, iaa received relatively little attention. In the meantime, while more and more iaa results are being reported on translation error analysis (Lommel et al., 2014; Daems et al., 2013; Costa et al., 2015), there is still no consensus on how iaa should be measured for this task. A limited number of error-annotated corpora of machine translations have been described in the literature. The Terra corpus (Fishel et al., 2012) is based on the error taxonomy suggested by Vilar et al. (2006) and contains four language pairs (English-Czech, French-German, German-English and EnglishSerbian) with translations coming from different mt systems, including two smt systems, one rbmt and one deep-syntactic mt system. The largest subpart of the Terra corpus contains 252 sentences. The TaraXU Corpus (Avramidis et al., 2014) contains data from two different annotation schemes: a shallow and a more fine-grained variant. This corpus comprises four language pairs (English-German, French-German, Spanish-German and English-Czech) and contains a different number of sentence pairs, depending on the language pair and the mt system used, ranging from 83 to 523 sentences for the fine-grained error annotations and 1492 to 1798 sentences for the shallow error annotations. In this corpus, mt output was obtained from six different mt systems, including two rbmt and four smt systems. To the best of our knowledge, the scate corpus of mt errors is the first effort of its kind, currently consisting of 2963 sentence pairs of smt output and 698 sentence pairs of rbmt output. As such, this corpus offers a large body of fine-grained error annotations. It differs from the existing error-annotated corpora in that it links the accuracy errors annotated in the mt output to the corresponding source text fragments they originate from, which allows us to gain insights into the nature of mt errors and determine the source text fragments that are problematic for an mt system. 3

The Scate Error Taxonomy

The MT-specific properties and the error classification approach, which is based on how errors are detected, make the scate error taxonomy unique. We define the taxonomy in detail considering these two main aspects while comparing it to existing taxonomies and by providing examples of error annotations.

224

Tezcan, Hoste and Macken

3.1 Properties of the Error Taxonomy for mt By using manual error annotations in order to develop a better mt qe system within the scate project, we first defined a list of key properties the error taxonomy should have. These properties were defined by studying previous efforts on translation error analysis and can be summarised as follows: Focus on MT-specific errors: Given the fact that current mt systems do not necessarily use language as humans do, we can expect such systems to produce different types of translation errors. The taxonomy should be able to reflect the typical errors that mt systems generate. For example, copying unknown words from source to target or making grammatical errors is more likely to occur in mt output than in human translations. Distinction between accuracy and fluency errors: It is important that the error taxonomy allows the accuracy and fluency errors to be labelled separately, so that both aspects of translation quality can be analysed. Since this distinction can reduce the complexity of the error annotation task, it can also maximize iaa and the quality of the annotated corpus. An automatic error detection system might also benefit from this distinction by looking for fluency errors in the target text only and accuracy errors in source and target text together. Annotating errors on both source and target text: Even though translation errors can be annotated in the target text, annotating the corresponding part of the source text extends the error representation to the bilingual level. Linking the error annotations on both source and target provides extra alignment information among errors. Moreover, by also annotating errors in the source text, source-specific errors, such as ‘omissions’, can be annotated. Multiple errors on the same text span: Different types of translation errors can occur in the same text span. As Vilar et al. (2006) state, error types are not mutually exclusive and it is not infrequent that one kind of error causes also another one to occur. An ideal taxonomy allows all the error-related information to be collected. In addition to these key characteristics, the scate taxonomy can also be characterized as a hierarchical error taxonomy with error categories based on linguistic notions. The hierarchical structure allows errors to be analysed on different levels of detail, ranging from coarse- to fine-grained classifications. Linguistically motivated error categories allow annotators to apply common linguistic knowledge to categorize errors and allow them to examine the relationship between linguistic errors and the mt quality or the post-editing effort.

225

scate Taxonomy and Corpus of Machine Translation Errors

While the error taxonomy is based on common linguistic notions, the complexity of the mt output can still be confusing, especially when there are multiple, equally plausible explanations for an error. By formalizing the decisionmaking process, confusion can be reduced to a minimum (Lommel et al., 2014). To this end, we provided detailed guidelines and annotation examples together with definitions on annotations spans with a view to minimizing different interpretations of error definitions and thus fostering agreement between the annotators. Several existing error taxonomies show similarities with the scate mt error taxonomy. Based on the properties discussed above, an overview of the existing mt error taxonomies that were described in Section 2 is given in Table 10.1. The distinguishing feature of the scate taxonomy compared to its nearest counterparts mainly lies in the fact that it links the accuracy errors annotated in the mt output to the corresponding source text they originate from. 3.2 Error Classification In the scate mt error taxonomy, errors are classified according to the type of information that is needed to be able to detect them. We refer to any error that can be detected in the target text alone as a fluency error. Errors that can only be detected by looking at both source and target sentences are considered as accuracy errors. These two main categories are then split further into subcategories as shown in Figure 10.1.

Table 10.1 Comparison of the characteristics of different error taxonomies. We name the ­existing taxonomies based on the names of the authors.

distinction of accuracy and fluency source-target mappings multiple annotations in the same span error hierarchy linguistically motivated categories detailed guidelines

Flanagan

Vilar

Lommel

Costa

Daems

–

–



–



– 

– 

– –

– 

– 

– 

 –

 

 

 

–

–



–



226

Figure 10.1

Tezcan, Hoste and Macken

scate mt error taxonomy.

Fluency errors are concerned with the well-formedness of the target language, regardless of the content and meaning transfer from the source language and can be detected by analysing the target text alone, as exemplified in Figure 10.2. There are five main error categories under Fluency: grammar, lexicon, orthography, multiple errors and other fluency errors. ‘Grammar’ errors include all errors against the grammatical rules of the target language (in our case Dutch) with respect to word order, word form, missing or extra words and multi-word syntax. ‘Lexicon’ errors are concerned with words that either do not exist in the target lexicon or words that belong to the target lexicon but do not fit in a given context. ‘Orthography’ errors include spelling, capitalization and punctuation errors. While the first three main categories discussed above are based on common linguistic notions, and have also been used in previous error taxonomies with similar definitions, the error category ‘multiple errors’ is not common in existing taxonomies. Only in the taxonomy proposed by Lommel et al. (2014) do we notice a similar category referred to as ‘unintelligible’. ‘Multiple errors’ correspond to a combination of fluency errors that are difficult to identify separately, e.g. a word order error combined with wrong word forms and wrong lexical choices. When multiple errors accumulate on a particular text span, it becomes difficult for annotators to distinguish different types of errors and this can result in different interpretations of the translation errors and lower iaa. ‘Multiple errors’ groups such text fragments under a single error category.

scate Taxonomy and Corpus of Machine Translation Errors

Figure 10.2

227

An example source sentence and its machine translation, which contains a number of ‘fluency’ errors.

In Figure 10.2, the words ‘de … bereik (the … range)’ are marked as a ‘grammar – word form (agreement)’ error due to a lack of concord between the noun and the determiner. ‘nva bereik (nva range)’ and ‘go staal (go steel)’ are compounds in Dutch and need to be written together, leading to ‘orthography – spelling (compound)’ errors. Finally, a ‘grammar – word order’ error indicates the wrong word order in the string ‘go staal is’. This single error annotation also points to the correct order ‘is go staal’ by marking the group of words that need to switch places separately. Accuracy errors, on the other hand, are concerned with the extent to which the source content and the meaning is represented in the target text and can only be detected when both source and target sentences are analysed together, as exemplified in Figure 10.3. Accuracy errors are split into the following main categories: addition, omission, untranslated, do-not-translate (dnt), mistranslation, mechanical, bilingual terminology, source errors and other accuracy errors. While ‘addition’ errors refer to target words not represented in the source, ‘omission’ errors refer to source words not represented in target text. ‘Untranslated’ errors refer to words that are not translated in the target but are copied instead, when they should have been translated and ‘dnt’ errors correspond to source words that have been unnecessarily translated into the target language, when they should not have been translated. ‘Mistranslation’ errors refer to source content that has been translated incorrectly. In contrast to some of the taxonomies (Costa et al., 2015; Lommel et al., 2014), we also define partof-speech (pos) errors as ‘mistranslation’ errors since a change in pos category can only be detected when source and target texts are analysed together. The category of ‘Source’ errors allows annotating errors in the source sentence, which can be useful to identify mt errors that do not only originate from the mt system. Even though punctuation and capitalization errors are common to all mt error taxonomies described in Section 2, we defined an additional category ‘mechanical’ under accuracy. This category refers to non-meaning related errors such as punctuation and capitalization errors that are only visible when source and target are analysed together.

228

Figure 10.3

Tezcan, Hoste and Macken

An example source sentence and the corresponding mt output with ‘accuracy’ errors.

In Figure  10.3, the translations ‘resultaten (outcomes)’ and ‘mededeling (announcements)’ are marked as ‘mistranslation – word sense’ errors due to the wrong selection of word senses for the corresponding English words. ‘Onderzoek (research)’ is not a translation of ‘search’ and is marked with ‘mistranslation – other’ error category. For each accuracy error, the corresponding source and target words are annotated and linked. Certain similarities can be observed between some of the accuracy and fluency error categories in the error taxonomy, such as ‘extra words’ vs. ‘addition’, ‘missing words’ vs. ‘omissions’ or ‘orthography – capitalization’ vs. ‘­mechanical – capitalization’. As the main distinction between accuracy and fluency errors in the taxonomy is based on the type of information that is needed to be able to detect them, accuracy errors do not necessarily imply fluency errors, or vice versa for that matter, as is clearly shown in the examples in Figures 10.4 and 10.5. It should be also clarified that content and function words can both be annotated under accuracy and fluency errors since they are not explicitly associated with any of these two main error categories. In the example in Figure 10.4, ‘to lose’ is literally translated as ‘te verliezen’ in Dutch, which should have been translated as ‘verliezen’. The annotator detects that there is an extra word in the MT output (‘te’), which causes the sentence to be grammatically incorrect. This extra word, however, does not amount to an ‘addition’. In Figure  10.5, we can see that the annotator was able to detect a comma that should not be there, and annotated it as an ‘­orthography – ­punctuation’ error by reading the target text only. The transfer of the apostrophe to the wrong location in the target and the wrong transfer of the capitalization of the source phrase ‘BIOLOGICAL ORIGIN’ could only be detected by analysing the source and target texts together, which led to ‘mechanical’ error annotations. In the scate mt taxonomy, all error categories additionally contain an extra subcategory ‘other’, which corresponds to errors that fall within an overarching category but do not belong to any of its sub-categories. This category ensures that while an observed error may not fit the description of existing

scate Taxonomy and Corpus of Machine Translation Errors

229

Figure 10.4

‘Extra word’ errors, which can be detected in the target text alone, do not necessarily amount to ‘addition’ errors.

Figure 10.5

‘Capitalization’ and ‘punctuation’ errors are not always visible in the target text. In this example, two such errors are annotated as accuracy errors.

subcategories in the taxonomy, it can still be annotated. Furthermore, it provides useful information about the coverage of the taxonomy and the clarity of error categories. Finally, it must be noted that the scate error taxonomy does not claim to be exhaustive. Even though it is initially designed to cover most common smt and rbmt errors in the English-Dutch language pair, the top categories of this taxonomy are based on language independent notions (such as ‘accuracy and fluency’) and the language specificity increases with each lower level in the error hierarchy (such as ‘orthography – spelling (compound)’). This hierarchical structure allows customization of error categories for error classification tasks in other language pairs and for other mt architectures. For a more detailed description of the error categories and examples for each category, together with the annotation guidelines, please visit http://users.ugent.be/~atezcan/. 4

Corpus of Machine Translation Errors

As mentioned in Section 2, we built a corpus of mt errors consisting of two data sets: an smt data set of 2963 sentences and an rbmt data set of 698 sentences. We used Google Translate3 as the smt and Systran4 as the rbmt engine to obtain the mt output for all source sentences. The source sentences in the corpus of smt errors were extracted from the Dutch Parallel Corpus5 (Macken et al., 2011) and consist of an equal number of sentences from three different 3 http://translate.google.com (June, 2014). 4 systran Enterprise Edition, version 7.5. 5 http://www.kuleuven-kulak.be/DPC.

230

Tezcan, Hoste and Macken

Table 10.2 Number of sentences, number of words and average sentence length (in number of words), listed for the different data sets used for error annotation.

smt

Sentences Words Average sentence length

smt-sub

source

target

source

2,963 54,015 18.2

2,963 698 54,112 12,867 18.2 18.4

rbmt target

source

target

698 12,866 18.4

698 12,867 18.4

698 12,917 18.5

text types: external communication, non-fiction literature and journalistic texts. A text type-balanced subset of the smt data set was used to build the corpus of rbmt errors. Two pairs of annotators (all with a Master’s degree in Translation) were assigned to the annotation task on the smt and rbmt outputs separately and the annotations were made using the brat rapid annotation tool.6 In order to compare the error profiles of the two mt architectures, we extracted an smt sub-corpus from the larger smt corpus, which contains the sentences that were also annotated in the rbmt corpus. The statistics of each data set are presented in Table 10.2. To maximize the consistency and reliability of the annotations, the taxonomy, the accompanying error annotation guidelines and the corpus of smt errors were all developed in parallel, interactive cycles of iaa analysis and revision. After the necessary revisions, the annotations in the corpus of smt errors were adapted to the final version of the taxonomy. The final version of the taxonomy and the annotation guidelines were used to build the corpus of rbmt errors. Therefore, we provide iaa results only for the corpus of rbmt errors in Section 5. 5

Inter-Annotator Agreement

Determining iaa for translation error analysis is far from trivial as it might involve – depending on the taxonomy being used – error detection, error categorization and error span annotation. Different methods have been used (Stymne and Ahrenberg, 2012; Lommel, Uszkoreit & Burchardt, 2014; Daems, 6 http://brat.nlplab.org.

scate Taxonomy and Corpus of Machine Translation Errors

231

Macken & Vandepitte, 2013; Costa et al., 2015) and in most cases, the comparison of the number of annotated errors per sentence and Cohen’s kappa coefficient7 is used to calculate iaa. Both, however, have shortcomings for the translation error analysis task. 5.1 Alignment-based Inter-annotator Agreement Comparing the number of error annotations per sentence without taking the error span into account has the potential of interpreting agreement incorrectly, as annotators might indicate errors of the same category at different l­ ocations. In the worst-case scenario, this method could point to a perfect agreement when there is no agreement at all. An example is provided in Figure 10.6, in which we can see a source segment in English and its mt output in Dutch, ­annotated by two annotators. In this example, both annotators detected a ‘mistranslation – word sense (content word)’ error in the same output (apart from other errors) in two different locations. For this sentence, comparing the number of errors for each annotator would incorrectly lead to a perfect agreement score on this error category. Cohen’s kappa (Cohen, 1960) coefficient, which attempts to take the probability of random agreement into account, has been widely used in the nlp community to assess iaa. However, for annotation schemes in which annotators are allowed to indicate multiple errors on the same, or on overlapping, text spans (as is shown in Figure 10.7), without the alignment information between the annotations, it is not entirely clear how kappa should be calculated. In order to assess iaa for the purpose of translation error analysis, we evaluated the two underlying tasks of the error annotation process, viz. error detection and error categorization, separately. To assess iaa on error detection, we considered the problem of error detection as a binary task, deciding for each word whether it was part of an annotation span or not and calculated Cohen’s Kappa at word level.

Figure 10.6

A source segment in English and its mt output in Dutch containing annotations of ‘mistranslation’ errors provided by two different annotators.

p​(A)​ − p(E ) 1 − p(E )

7 ​ κ = ​_________       ​, where κ is the kappa value, p (A) is the probability of the actual agreement and p (E) is the probability of agreement by chance.

232

Tezcan, Hoste and Macken

Figure 10.7

Annotations coming from annotator 1 (dark blue and dark red) and annotator 2 (light blue and light red). All Annotations

All Annotations

1 Isolated

Overlapping

Overlapping

Single Overlap Extract: Text Span Similarity Threshold (t)

2 3 4

Multiple Overlaps

Exact Span Similar Span (t) Single Overlap

Aligned

5 Isolated

Multiple Overlaps 6 Not-Aligned

(a)

Figure 10.8

(b)

Illustrations of (a) text span similarity threshold determination and (b) the ­annotation alignment procedure, which uses the text span similarity threshold to align annotations with similar spans.

To assess iaa on error categorization, we proposed a novel methodology for alignment-based iaa. In this methodology, we first automatically aligned the overlapping annotations coming from the two annotators and then calculated iaa for error categorization on this aligned annotation set using Cohen’s kappa. The alignment procedure is schematically presented in Figure 10.8 and basically consists of two steps. In a first step, the text span similarity threshold is being determined, whereas in a second step this threshold is used to align similar annotations. The annotations coming from two annotators are first grouped as ‘overlapping’ if the text spans match or intersect. If not, they are categorized as ­‘isolated’. Overlapping annotations can further be subdivided into ‘single ­overlap’ (when a single annotation from each annotator overlaps) and ‘multiple overlaps’ (when multiple annotations are involved).

scate Taxonomy and Corpus of Machine Translation Errors

233

Aligning the overlapping annotations from the two annotators when the annotations span the same text fragments is a straightforward task. When the text spans do not match exactly, it becomes difficult to align annotations, especially in the case of ‘multiple overlaps’. Therefore, we first analyse the text span similarities of the annotation pairs in the ‘single overlap’ set, which span different text fragments. To be able to make this analysis, we first define text span similarity (S) of a set of annotations as: ​S = ​|a​1​ f​ − a​2​ f​ |​ + ​|a​1​ l​ − a​2​ l​|​ where a1 and a2 denote the annotations coming from the first and the second annotator, respectively, and f and l denote the index value of the first and the last characters of an annotation within the corresponding sentence, respectively. We focus on this subset of the annotations (spanning different text fragments) in the ‘single overlap’ set to define a text span similarity threshold (t), which is calculated as the average S value of the annotation pairs when S > 0. In our data set, t was set at 14 characters. This threshold is used in the second step to align annotations based on text span similarity. To align the annotations, we started working on the ‘overlapping’ annotation set (set 1). Each overlapping annotation pair with an exactly matching text span (S = 0), is simply aligned (set 2), moved to the ‘aligned’ set and removed from the ‘overlap pairs’ set. This latter ‘filtering step’ allows us to collect additional alignments for the remaining annotations in the ‘overlap pairs’ set or leave isolated annotations behind. If multiple annotation pairs with ‘exact spans’ are found, the alignment is made only between the annotations with the same category when available.8 We use t to align annotations based on text span similarity (when annotation text spans differ less than the t value, set 3) and move them to the ‘aligned’ set. If multiple annotation pairs with ‘similar spans’ are found, the alignment is made only if the annotations belong to the same error category. Once the annotations with the ‘exact’ and ‘similar’ spans are aligned, the annotation pairs in the ‘single overlaps’ set (set 4) are aligned and moved to the ‘aligned’ set. The alignment process can be illustrated with the example given in ­Figure 10.9. In this example, apart from other potential errors, the ‘grammar – word form (agreement)’ annotations from the two annotators have exactly matching text spans and can easily be aligned. After filtering the aligned annotations, the remaining ‘orthography – spelling’ and ‘orthography – ­punctuation’ 8 If there are multiple annotation pairs with exact span and matching categories, no alignment is made.

234

Tezcan, Hoste and Macken

Figure 10.9

1800 1600 1400 1200 1000 800 600 400 200 0

1550 1398 713

overlap aligned

Example error annotations coming from two annotators indicated as red (and pink) and grey, respectively.

ann 1 1690

929 725

overlap - not aligned

1550 1398

420498

548

isolated

713

1692

ann 2

807 771 838 586 569

overlap - overlap - not isolated aligned aligned

original data alignment on ‘exact’ spans alignment on ‘exact’ & ‘similar’ spans

Figure 10.10 Number of annotations from each annotator, in three annotation groups (isolated, overlaps – aligned and overlaps – not aligned), when different alignment steps are applied.

errors from the two annotators can additionally be aligned based on text span similarity, given that text span similarity is under the text span similarity threshold. After all alignments are made, we can collect the isolated annotations (set 5). Any overlapping annotations that are left in the ‘overlap pairs’ are marked as ‘Not-Aligned’ annotations (set 6). In Figure 10.10, we show the impact of using the alignment procedure by reporting the number of annotations that become available for iaa analysis, after applying the annotation alignment steps for the annotations with ‘exact’ and ‘similar’ spans on the corpus of rbmt errors. The original annotation set (indicated in the first bars of each a­ nnotation group, for each annotator) consists of 713 overlapping annotations that are aligned, 1690 and 1692 overlapping annotations that are not aligned and 420 and 569 isolated annotations (from annotators 1 and 2, respectively). By aligning annotations on the basis of ‘exact’ spans (indicated in the second bars of each annotation group) and afterwards on the basis of ‘similar’ spans (­indicated in the third bars of each annotation group), we reduced the number of annotations in the ‘overlap no-alignment’ group by more than 58% for each annotator by moving annotations to the ‘isolated’ and the ‘overlap – aligned’ groups. As the ‘isolated’ annotation sets reflect errors that are detected by one annotator but not the other, the number of annotations in these sets affect the iaa

scate Taxonomy and Corpus of Machine Translation Errors

235

on ­error detection. For iaa on error classification, we focus on the annotation pairs in the ‘overlap – aligned’ groups to obtain a large-scale analysis, which consists of 1550 annotations (68% and 72% of all overlapping annotations in total, for annotator 1 and 2, respectively). 5.2 Inter-annotator Agreement Results We report the iaa results on the rbmt corpus using Cohen’s kappa (κ) and observed agreement scores. Firstly, we measured iaa on error detection, considering the binary decision for each word being an error or not. The obtained kappa score for error detection is 0.66. Secondly, we used the annotation alignment procedure, described in Section 5.1, to measure iaa for error categorization from taxonomy levels 1 to 3 (level 1 representing the main distinction of ‘accuracy’ and ‘fluency’, while levels 2 and 3 represent deeper levels in the error hierarchy) and provide the iaa results in Table 10.3. In this table, we report Cohen’s kappa coefficient in two different stages. First, only for the annotations that the annotators agree on at a higher level (κ1) and then, for all annotations (κ2). Although the exact interpretation of the kappa coefficient is difficult, according to Landis and Koch (1977), 0.6–0.8 points to substantial and 0.8–1.0 points to almost perfect agreement. While kappa scores inform us of the degree of agreement, they fail to give more insight into the type of error annotations on which the annotators disagree. In order to gain a deeper understanding of the nature of these Table 10.3 iaa results based on Cohen’s kappa coefficient.

Taxonomy Level 1 Taxonomy Level 2 Taxonomy Level 3

κ1

κ2

0.78 0.91 0.89

0.78 0.78 0.70

Table 10.4 Percentage of annotations for the aligned annotation pairs from the two a­ nnotators on taxonomy level 1. Error categories in the first row represent the annotations from annotator 1 and the categories in the first column from annotator 2. Observed ­agreement in this annotation set is 89%.

Accuracy Fluency

Accuracy

Fluency

43.4% 5.7%

5.3% 45.6%

Addition dnt Grammar Lexicon Mech. Mistrans. Multi Omi. Ortho. Untrans.

0.2% 0% 0% 0% 0% 0.1% 0% 0% 0% 0%

Add.

0% 4.2% 0% 0% 0% 0.4% 0% 0% 0% 0%

dnt

0% 0% 27.8% 0.8% 0% 0% 0.1% 0% 0.7% 0%

Gra. 0% 0% 1.4% 7.% 0% 0% 0% 0% 0.5% 0%

Lex. 0% 0% 0% 0% 0.4% 0% 0% 0% 0% 0%

Mech. 0% 1.2% 0% 0% 0.1% 37.8% 0% 0% 0% 0.3%

Mistrans. 0% 0% 0.6% 0% 0% 0% 0.2% 0% 0.1% 0%

Multi 0% 0% 0% 0% 0% 0% 0% 1.2% 0% 0%

Omi. 0% 0% 0.3% 0% 0% 0% 0% 0% 11.5% 0%

Ortho.

0% 0% 0% 0% 0% 0% 0% 0% 0% 2.9%

Untrans.

Table 10.5 Percentage of annotations for the aligned error annotations from the two annotators on taxonomy level 2, given the agreement on taxonomy level 1. Error categories in the first row represent the annotations from annotator 1 and the categories in the first column those from annotator 2. Observed agreement in this annotation set is 93.3%.

236 Tezcan, Hoste and Macken

scate Taxonomy and Corpus of Machine Translation Errors

237

d­ isagreements, we calculated percentages of the aligned error annotations from the two annotators, assuming that they agree on the higher level of the taxonomy. The annotation matrix in Tables 10.4 and 10.5 give more information on the nature of disagreements and provide us with a more complete picture about the iaa. In the rbmt data set, the most pronounced disagreements can be observed in the following error categories: ‘lexicon’ vs. ‘grammar’ (total of 2.2%), ‘mistranslation’ vs. ‘dnt’ (total of 1.6%), and ‘multiple’ vs. ‘grammar’ (total of 0.7%). Such information is useful to detect confusion between certain categories, which can be used to revise the error taxonomy, error definitions and/or annotation guidelines further. 6

Error Statistics

In this section, we analyse the errors in different data sets in the scate corpus of mt errors in detail and discuss the differences between the error profiles of the smt and rbmt systems. Table 10.6 gives an overview of the number of errors for the three layers of the scate error taxonomy and for each data set in the corpus of mt errors. To resolve the disagreements between the annotators, a consolidation step has been included. Only the annotations both annotators agreed on were retained in the final corpus. We can make a number of observations about the translation quality of both mt systems by analysing the distribution of errors they made. Given that the mt output in the ‘SMT-sub’ and ‘rbmt’ data sets draw on the same source segments, they can be compared. Looking at the totals, we see that the rbmt system yielded more errors (3070) than the smt system (2256). 68% of the smt errors were fluency errors, compared to 59% in the rbmt system, and 32% of the smt errors were accuracy errors compared to 41% in the rbmt system. In order to gain more insight into the distribution of errors per error category, we analyse the annotations in more detail. 6.1 Accuracy Errors As is shown in Figure 11, the large majority of accuracy errors made by both systems are ‘mistranslation’ errors (65% and 74% for the smt and rbmt systems, respectively). The proportions of the remaining accuracy errors vary between the two systems. While proportionally, the smt system made more ‘omission’ and ‘untranslated’ errors, the rbmt system made more errors on the ‘dnt’ category. Considering that an rbmt system typically uses a syntactic parser to analyse the source sentence and words, it is not surprising that such a system

238

Tezcan, Hoste and Macken

Table 10.6 Number of errors per category, per data set.

Level 1 Total Accuracy

Level 2

Mistranslation

dnt Untranslated Addition Omission Terminology Source Mechanical

Fluency

Other Grammar

Orthography

Lexicon

Multiple Errors Other

Level 3

mwe pos Word Sense Partial Other

Capitalization Punctuation Other

Word Form Word Order Extra Word Missing Word Multi-Word Syntax Other Spelling Capitalization Punctuation Other Non-existent Lexical Choice

smt 10051 3241 2088 564 198 948 164 214 63 283 185 531 0 21 70 45 18 7 0 6810 4171 1059 1334 481 1128 158 11 1117 736 44 335 2 1022 163 859 497 3

SMT-sub 2256 732 474 135 46 202 39 52 14 67 41 115 0 1 20 14 3 3 0 1524 936 243 309 104 251 26 3 243 134 14 94 1 232 51 181 112 1

rbmt 3070 1265 940 270 53 569 3 45 115 61 61 34 0 2 52 3 47 2 0 1806 855 142 376 159 153 24 1 284 95 55 134 0 527 64 463 140 0

scate Taxonomy and Corpus of Machine Translation Errors SMT -sub 03%

RBMT

00%

% 02

06%

00% 04% 05%

09%

09%

00% 00%

05% 03%

16% 65%

74% Figure 10.11

239

Mistranslation Omission Untranslated Addition Mechanical DNT Source Other

Proportions of ‘accuracy’ errors per mt system.

makes fewer ­‘omission’ errors than a statistical system, which utilizes automatically generated word and phrase alignment models in combination with a language model, which tries to maximize the fluency of the output without taking the source content into account. The ‘dnt’ errors refer to source words that are translated in the target language, while they should have been simply copied. One of the explanations for the number of these errors can be attributed to the way Named Entities (nes), which should invariably be copied from source to target, are handled ­differently in both systems. nes need to be explicitly encoded in rbmt dictionaries. For our experiments, we relied on the default dictionaries of the rbmt system, and therefore, it is not surprising that the rbmt system made more ‘dnt’ errors (9%) than the smt system (2%), which can learn how to translate these nes from its training data. The ‘untranslated’ errors refer to source words that are copied to target, while they should have been translated. The smt system relies on the bilingual training data for vocabulary coverage and does not use explicit dictionaries to translate words. The ‘unknown’ words get copied to the target sentence in smt systems and this can be the main source of ‘untranslated’ errors, which account for 9% of accuracy errors of the smt system, compared to 5% in the case of rbmt. Looking into the type of ‘mistranslation’ errors that both systems make, (see Figure 10.12) we notice that the rbmt system made more ‘word sense’ errors compared to the smt system (61% and 43%, respectively). This category makes up the majority of ‘mistranslation’ errors in both systems. However, while it is true that the ‘partial’ errors constitute 8% of all ‘mistranslation’ errors in the smt system, the rbmt system makes almost no errors in this category. ‘Partial’ errors are defined as the incorrect and partial translation of Dutch separable verbs.

240

Tezcan, Hoste and Macken

Such errors in the smt case can be attributed to word order differences in the English and Dutch languages (subject-verb-object and subject-object-verb, respectively) and the use of phrase alignment models to translate between such languages, which often fail to complete long-distance alignments accurately. 6.2 Fluency Errors The rbmt system made more fluency errors (1806) than the smt system (1524). By looking at the proportions of fluency errors per MT system in Figure 10.13, we see that one of the problem areas for the rbmt system seems to be related to making the right lexical choices. The number of ‘lexical choice’ errors which the rbmt system makes (527) is more than double the number of smt errors in the same category (232). The reason that the smt system makes better lexical choices can be linked to the language model component it uses, which prefers more fluent and context-dependent translations. This is in contrast to the rbmt system, which relies heavily on bilingual dictionaries for making correct SMT-sub

RBMT

08%

05%

10%

06% 00% Word Sense

43%

11%

MWE Other

29% 61%

POS Partial

28%

Figure 10.12 Proportions of ‘mistranslation’ errors per mt system. SMT-sub 07%

RBMT 08%

15% 29% 16%

47%

61% 16% Figure 10.13 Proportions of ‘fluency’ errors per mt system.

Grammar Orthography Lexicon Multiple

scate Taxonomy and Corpus of Machine Translation Errors SMT-sub 03% 00% 27%

RBMT 03% 00% 26%

18%

17% Word Form Word Order Extra Word Missing Word Multi-Word Syntax Other

19% 11% 33%

241

44%

Figure 10.14 Proportions of ‘grammar’ errors per mt system.

lexical choices, without considering the context in which the target words are used. Even though the rbmt system makes more fluency errors, it makes fewer grammatical errors than the smt system, because it contains syntactic analysis components. Given that the grammatical errors make up the majority of errors in both systems (61% and 47% for the smt and rbmt systems, respectively), it seems that generating grammatically correct translations is challenging for both type of systems. Looking into grammar errors in detail, in Figure 10.14, we see that the smt system proportionally made much more errors on the ‘word form’ and ‘missing word’ categories (26% and 27% respectively), compared to the rbmt system (17% and 18% respectively). Syntactic analysis and transfer steps, which help build target syntactic constructions, can be attributed to the relative success of the rbmt system in these error categories. The smt system, however, proportionally made fewer errors when it came to ‘word order’ and ‘extra word’ categories (33% and 11%, respectively), compared to the rbmt system, for which these errors make up 44% and 19% of all grammar errors. Further analysis of long and short-distance word order errors is needed to gain more insight into what causes errors of this category. 7

Conclusions and Future Work

In this chapter, we present the scate mt error taxonomy and the scate corpus of mt errors for the English-Dutch language pair. Designed in a hierarchical structure, the scate taxonomy draws a distinction between accuracy and fluency errors at the top of the error hierarchy. All errors that can be detected in the target text alone are defined as fluency errors, whereas errors that can

242

Tezcan, Hoste and Macken

only be detected at bilingual level (source and target) are defined as accuracy errors. In addition to drawing this main distinction, we also link the accuracy errors to source and target text, thus providing extra alignment information with respect to the mt errors. In order to assess iaa for translation error analysis, we evaluated the two underlying tasks of the error annotation process, viz. error detection and error categorization, separately. To measure iaa on error categorization, we proposed a new method for alignment-based iaa, which involves the alignment of annotations coming from different annotators based on text span similarities prior to iaa analysis. We used iaa to first revise the taxonomy, error definitions and annotation guidelines while a corpus of smt errors was being developed. After establishing our taxonomy, we applied it to the translations of an rbmt system, reported high iaa using Cohen’s kappa and observed agreement in terms of error annotations in this data set. We also compared the error profiles of the two types of mt architectures. Even though the rbmt system made more accuracy and fluency errors, it still made fewer errors on certain categories, such as the ‘untranslated’, ‘omission’, ‘capitalization’ and ‘grammar’ errors. This study helps us understand the strengths and weaknesses of these different mt architectures, in respect of the types of errors they make. These findings can lead to improvements in hybrid systems, which combine these two different approaches in mt. Additionally, these findings can help us understand the impact of different error types on post-editing effort. In the future, we would like to measure this impact, considering the post-editing effort in relation to the number of edits and post-editing time. Other plans for future work include the automatic detection of certain error types and applying the findings of this study to improving existing mt quality estimation systems. Acknowledgements This research has been carried out in the framework of the scate project9 funded by the Flemish government agency iwt (iwt-sbo 130041). 9 The project coordinator of the scate project is Dr. Vincent Vandeghinste. The scate consortium consists of ccl, Centre for Computational Linguistics – University of Leuven; esat/psi, Centre for the Processing of Speech and Images – University of Leuven; liir, Language Intelligence & Information Retrieval – University of Leuven; qlvl, Quantitative Lexicology and Variational Linguistics – University of Leuven; lt3, Language and Translation Technology Team – University of Ghent; edm, Expertise centre for Digital Media – Hasselt University.

scate Taxonomy and Corpus of Machine Translation Errors

243

References Avramidis, E. & Koehn, P. (2008). Enriching Morphologically Poor Languages for ­Statistical Machine Translation. ACL, pp. 763–770. Avramidis, E., Burchardt, A., Hunsicker, S., Popovic, M., Tscherwinka, C., Vilar, D. & U ­ szkoreit, H. (2014). The TaraXU Corpus of Human-Annotated Machine ­Translations. Proceedings of LREC 2014, pp. 2679–2682. Bach, N., Huang, F. & Al-Onaizan, Y. (2011). Goodness: A method for measuring machine translation confidence. Proceedings of the 49th Annual Meeting of the ­Association for Computational Linguistics: Human Language Technologies-Volume 1, pp. 211–219. ­Association for Computational Linguistics. Biçici, E. (2013). Referential translation machines for quality estimation. In Proceedings. of the Eighth Workshop on Statistical Machine Translation, pp. 343–351. A ­ ssociation for Computational Linguistics. Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and p­ sychological measurement, 20 (1), pp. 37–46. Costa, Â., Ling, W., Luís, T., Correia, R. & Coheur, L. (2015). A linguistically motivated taxonomy for Machine Translation error analysis. Machine Translation, 29 (2), 127–161. Daems, J., Macken, L. & Vandepitte, S. (2013). Quality as the sum of its parts: A twostep approach for the identification of translation problems and translation quality assessment for HT and MT+ PE. In MT Summit XIV Workshop on Post-Editing Technology and Practice, pp. 63–71. European Association for Machine Translation. Daems, J., Macken, L. & Vandepitte, S. (2014). On the origin of errors: A fine-grained analysis of MT and PE errors and their relationship. Language Resources and Evaluation Conference, pp. 62–66. European Language Resources Association (ELRA). de Souza, J.G.U., Buck, C., Turchi, M. & Negri, M. (2014). FBK-UPV-UEdin participation in the WMT14 Quality Estimation shared-task. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pp. 322–328. Farrús, M., Costa-Jussa, M.R., Marino, J.B., Poch, M., Hernández, A., Henríquez, C. & Fonollosa, J.A. (2011). Overcoming statistical machine translation limitations: error analysis and proposed solutions for the Catalan–Spanish language pair. Language resources and evaluation, 45(2), pp. 181-208. Fishel, M., Bojar, O. & Popović, M. (2012). Terra: a Collection of Translation Error-­ Annotated Corpora. In LREC, pp. 7–14. Flanagan, M. (1994). Error classification for MT evaluation. In Technology Partnerships for Crossing the Language Barrier: Proceedings of the First Conference of the Association for Machine Translation in the Americas, pp. 65–72. Landis, J.R. & Koch, G.G. (1977). The measurement of observer agreement for categorical data. biometrics, pp. 159–174.

244

Tezcan, Hoste and Macken

LISA QA Model. (2007). The Globalization Industry Primer: An Introduction to Preparing your Business and Products for Success in International Markets. Geneva: The ­Localization industry Standards Association (LiSA). Lommel, A., Uszkoreit, H. & Burchardt, A. (2014). Multidimensional Quality Metrics (MQM): A Framework for Declaring and Describing Translation Quality Metrics. Revista tradumàtica: traducció i tecnologies de la informació i la comunicació, 12, pp. 455–463. Lommel, A., Popović, M. & Burchardt, A. (2014). Assessing Inter-Annotator Agreement for Translation Error Annotation. In Automatic and Manual Metrics for Operational Translation Evaluation Workshop Programme, pp. 5–15. Macken, L., De Clercq, O. & Paulussen, H. (2011). Dutch parallel corpus: a balanced copyright-cleared parallel corpus. Meta: Journal des traducteursMeta:/Translators’ Journal, 56(2), pp. 374–390. Mateo, R.M. (2014). A deeper look into metrics for translation quality assessment (TQA): a case study. Miscelánea: A Journal of English and American Studies, 49, pp. 73–94. Popović, M. & Burchardt, A. (2011). From human to automatic error classification for machine translation output. In 15th International Conference of the European Association for Machine Translation (EAMT 11), pp. 265–272. Rinsche, A. & Portera-Zanotti, N. (2009). Study on the size of the language industry in the EU. European Commission-Directorate General for Translation, Brussels. SAE J2450 Society of Automotive Engineers Task Force on Translation Quality Metric (1999). Specia, L., Shah, K., De Souza, J.G. & Cohn, T. (2013). QuEst-A translation quality estimation framework. In ACL (Conference System Demonstrations), pp. 79–84. Stymne, S. & Ahrenberg, L. (2012). On the practice of error analysis for machine translation evaluation. In LREC 2012, pp. 1785–1790. Toury, G. (2000). The nature and role of norms in translation. The translation studies reader, 2. Vilar, D., Xu, J., d’Haro, L.F. & Ney, H. (2006). Error analysis of statistical machine translation output. In Proceedings of LREC, pp. 697–702. White, J.S. (1995). Approaches to black box MT evaluation. In Proceedings of Machine Translation Summit V, pp. 10. Wisniewski, G., Kübler, N. & Yvon, F. (2014). A Corpus of Machine Translation Errors Extracted from Translation Students Exercises. In LREC 2014, pp. 26–31.

APPENDIX 1

scate mt Error Taxonomy: Examples of Most Frequent mt Errors In the following examples, only the words belonging to the specific error category are highlighted. Other errors (belonging to other error categories) are not considered. The scate mt error taxonomy allows different error types annotated on the same text span.

i

Accuracy Errors

A Mistranslation

A source text fragment has been translated incorrectly.

i

Multi-word Expression

ii

Word Sense Disambiguation

The translation is incorrect (and often too literal) because the source sentence contains a multi-word expression such as an idiom, a proverb, a collocation, a compound or a phrasal verb. (en): The answer shines out from almost every article. (nl): Het antwoord schijnt uit bijna elk artikel. Remark: The translation is incorrect and too literal. A correct translation would be ‘komt naar voren in’, which means ‘comes forward, emerges’.

The target text fragment refers to a different (and a wrong) sense of the corresponding source text fragment. • Function Word (en): The urban projects are the ideal testing places for working with integrated teams… (nl): De stedelijke projecten zijn de ideale proeftuinen voor met geïntegreerde teams te werken… Remark: ‘voor’ means ‘before’ in this context. A correct translation would be ‘om’.

© koninklijke brill nv, leiden, ���8 | doi 10.1163/9789004351790_013

246

APPENDIX 1

• Content Word (en): The results can be disastrous for mother and child. (nl): De resultaten kunnen funest zijn voor moeder en kind. Remark:  The translation ‘resultaten’ means ‘findings’. A correct translation in this context would be ‘gevolgen’ which means ‘consequences’.

ii

Fluency Errors

A Grammar

Errors regarding the grammatical rules of the Dutch language.

i

Word Form

A wrong word form has been used. • Agreement Two or more words do not agree with respect to gender, number, or person. (en): Urban dynamics are strongly related to… (nl): Stedelijke dynamiek zijn sterk gerelateerd aan… Remark: ‘dynamiek’ is a singular noun and does not agree with the plural form of the verb ‘be (zijn)’

ii

Word Order

iii

Extra Words

iv

Missing Words

Wrong word order. (en):  In the future, the central city will undoubtedly continue to play an important role. (nl):  In the toekomst zal de centrale stad ongetwijfeld blijven een belangrijke rol spelen. Remark:  The correct word order is ‘een belangrijke rol blijven spelen’

One or more extra words make the target sentence grammatically incorrect. (en):  This is also our view, although the urban authorities often still see things differently. (nl):  Dit is onze mening, hoewel de stedelijke autoriteiten vaak nog dingen anders te zien.

One or more missing words make the target sentence grammatically incorrect.

scate mt Error Taxonomy

247

• Function Word (en):  This is equal to 1.500 euro net on withholding tax of 25%. (nl):  Dit is gelijk aan 1,500 euro netto op roerende voorheffing van 25%. Remark:  After ‘op’ the target sentence requires an article for grammatical correctness. In this example there are no additional ‘omission’ errors. • Content Word (en):  The way in which clients talk about their collaboration with Company1 bears out his words. (nl):  De manier waarop klanten over hun samenwerking met Company1 bevestigt zijn woorden. Remark:  After ‘Company1’ the target sentence requires a verb to be grammatically correct. In this case the missing verb is the translation of ‘talk about’ in Dutch, which is an additional ‘omission’ error for this sentence pair.

B Lexicon

Errors regarding the use of the lexicon in the Dutch language.

i

Lexical Choice

The word(s) is a part of the Dutch lexicon but another word(s) should be used for generating a correct Dutch sentence. • Function Word (en):  The cities are bursting with life. (nl):  De steden barsten met het leven. Remark:  A correct translation of ‘bursting with life’ would be ‘barsten van het leven’. The incorrect lexical choice of ‘met’ can be detected in the Dutch sentence alone. An additional error of type ‘mistranslation – word sense disambiguation’ accompanies this fluency error. • Content Word (en):  …contrary to what the Reuters report indicates. (nl):  …in tegenstelling tot wat het rapport Reuters geeft. Remark:  ‘Geeft’ literally means ‘(to) give’ and the incorrect lexical choice can be detected in the Dutch sentence alone. An additional error of type ‘mistranslation – partial’ accompanies this fluency error. A correct translation and a lexical choice would be ‘aangeeft (indicates)’.

C Orthography

Errors related to the method for writing Dutch language.

248

APPENDIX 1

i Spelling Spelling error

• Compound: Errors Related to the Spelling of Compounds (en):  The wind energy sector in Europe consumes 700.000 tonnes of steel per year. (nl):  De windenergie sector in Europa verbruikt 700.000 ton staal per jaar. Remark:  ‘windenergiesector’ should be written in one word

Index Academic portal 4, 177, 178, 179, 182, 195, 197, 198, 199 Automatic post-editing 39, 177, 180, 190 Bilingual corpus 118, 219 Bilingual dictionary 87, 89, 101, 102, 103, 122 Booth 61, 62, 63, 71, 76, 80, 81, 154, 154n, 155, 156, 161, 162, 163, 164, 165, 166, 168 cai tool 59, 153, 155, 156, 156n2, 157, 158, 160–171, 173 cat tool 1–3, 37, 38, 41, 42, 44, 46–54, 56, 70, 87n, 109–111, 114, 121–124, 129, 131, 137–145, 150–152, 163 Collocation 100, 103, 112–114, 118, 122, 123, 125, 129, 207, 245 Comparative analysis 59, 76, 79, 81, 180 Computer-assisted interpreting 3, 59, 83, 153, 166, 168, 172 Computer-assisted translation 1, 3, 37, 109, 156n2, 202, 216 Controlled language 179, 185, 196, 200, 201, 217 Corpus 4, 38, 92, 109, 110, 113, 118, 122, 132–135, 155, 161, 167, 170, 172, 219, 221, 223, 224, 229, 230, 234, 235, 237, 241–244 Dictionary 11, 13, 66, 84, 89, 101–103, 106, 111, 113, 113n, 122, 127, 128, 132, 136 Didactics 153, 156, 161, 162, 169 Editing time 189, 194 Electronic tool 2, 4, 5, 37, 38, 41, 44, 45, 47–50, 53 Ergonomics 16, 109, 129–131 Error classification 219, 221–223, 225, 229, 235, 243, 244 Error taxonomy 4, 219, 221, 223–226, 228, 229, 237, 241, 245 Evaluation 4, 14, 24, 54, 56, 65, 78, 83, 132, 133, 152, 171, 172, 177, 178, 182, 183, 192–194, 197, 199, 200–202, 207, 216, 222, 243, 244 External resource 9–11, 14, 16, 17, 19, 30, 32, 120n

Focus group 14, 123, 124 Glossary 62–65, 67–71, 73, 74, 76–79, 117, 160, 162, 165, 166, 196 ict 82, 153–155, 157, 170, 171 Information behaviour 9, 15, 31, 35 Information seeking 31, 33 Information technology 3, 31, 94, 153 Instrumental approach 85–91, 93, 96, 97, 100, 101, 103–105 Inter-annotator agreement 4, 219, 220, 230, 231, 235, 244 Interpretation vii, 1–3, 20, 57–59, 61, 62, 72, 76, 78, 80, 81, 84, 88, 89, 91, 95–97, 100, 101, 106, 153, 159, 160, 162, 165, 166, 168n, 169n, 194, 225, 226, 235 Interpretation service 57–59, 61, 78, 81, 92, 166 Interpreter vii, viii, 1–4, 57–67, 72, 73, 76, 78–84, 153–174, 217 Interpreting 1, 3, 5, 33–35, 57–60, 62, 65, 72, 73, 79, 81–84, 135, 153–174, 179, 194, 214, 216, 231 Interpreting technology 1, 3, 59 iOmegaT 3, 137, 142, 144–146, 151 Knowledge system 158–160 Knowledge-rich context 109–111, 113, 114, 124, 135 krc 3, 109, 111, 114, 120–129, 131, 132 Language technology 57, 59 Learning activity 2, 85, 87, 88, 90, 93, 95, 97, 100–104 Log 120n, 139, 143 Machine translation error 4, 219, 229, 243, 244 Machine Translation vii, 1–4, 13, 37, 39, 41, 53–56, 85, 86, 86n, 105, 132, 134, 137, 138, 152, 177–181, 183, 185–187, 198–203, 207, 212–220, 222, 223, 227, 229, 243, 244 Machine translation post-editing 4, 55, 56, 138, 152, 202, 203, 214, 215, 217, 218

250 Machine Translation system vii, 86n, 212, 213, 217 Meaning-based translation 2, 85, 87, 88, 104–106 Monitor 111, 114, 119, 138, 171 mt 1, 3, 4, 10, 13, 14, 17n7, 32, 35, 39–41, 46–49, 52, 54–56, 85–87, 105, 137–140, 142, 143, 146–152, 177–182, 184–187, 189–194, 196–209, 211–214, 216–231, 237, 239–245 mt error 211, 219–223, 225–227, 229, 237, 241, 242, 245 mt quality 138, 219–221, 224, 242 nlp application 39, 85, 86, 88–90, 104 Online platform 177, 178 Online questionnaire 41, 123, 125, 127, 128 Online resource 1, 2, 9, 10, 13, 14, 16, 31, 33, 36, 165, 166 Post-editing 4, 14, 16, 17n7, 32, 39, 40, 55, 56, 120, 133, 138, 139, 143, 147, 152, 177–183, 189–194, 196–220, 222, 224, 242, 243 Post-editing rule 177, 183, 189–192, 192n, 200 Pre-editing 4, 177–180, 182–192, 194–197, 199–202, 204, 211, 213–215 Pre-editing rule 177, 178, 183, 184, 186–188, 190–192, 195, 200 Preediting 180, 182, 211 Preparation phase 57, 81 Process-oriented research 9, 13, 167–169, 209 Product-oriented research 167, 168, 170, 208 Productivity 2, 3, 10, 15, 16, 30, 32, 37, 38, 40, 54, 55, 56, 137–139, 142, 144, 146–150, 152–155, 166, 206, 209–211, 216, 217 Productivity-oriented research 209 Productivity test 56, 137, 138, 144, 146–148, 152, 217 Professional translation vii, 2, 33, 40, 85, 89, 93, 104, 114, 121, 181, 203, 207 Quality estimation 219, 220, 242–244 Readability 143, 185, 188n, 189, 190 Resource vii, viii, 1–5, 9–14, 14n, 15–17, 17n, 18, 19, 19n, 20, 21, 23, 24, 26, 27, 29–33, 48, 86–88, 90, 101, 109, 111, 113, 115, 120,

Index 120n, 121–131–133–140, 152, 153, 165, 166, 171, 172, 177, 183–185, 199–201, 207, 209, 216, 219, 220, 243 scate 4, 219, 221, 223–226, 228, 229, 237, 241, 242, 242n, 245 Segment Level A/B Testing 139, 140 Software 20, 38, 39, 45–48, 51–55, 60, 62, 63, 65, 66, 73, 7891n, 109, 114, 114n, 115, 119, 120n, 134, 135, 137, 141, 142, 144, 150, 151, 154–158, 161–165, 169, 172, 177, 178, 182, 187, 199, 201, 218 Teaching viii, 1, 4, 38, 85–91, 101, 104, 105, 135, 169, 169n, 177–182, 194, 195, 199, 203, 204, 208, 215–218 Terminology 1, 2, 13, 38, 39, 42, 45–55, 57, 59–67, 70–73, 76, 78–84, 86, 92, 94, 95, 106, 114, 122, 125, 132, 133, 135, 139, 140, 153–155, 158, 159, 159n, 160–169, 171, 172, 174, 212, 227 Terminology extraction 39, 46–48, 51, 54, 83, 161 Terminology management 1, 2, 42, 45, 46, 48, 50–55, 57, 59, 60–66, 71, 73, 76, 78, 79, 81–83, 155, 158, 161, 164, 165, 171 Terminology management system 1, 2, 45, 50, 54, 57, 59, 62, 76, 155, 164, 165, 171 Terminology management tool 46, 52, 60, 61, 64–66, 71, 78, 81–83, 171 Textual corpora 45, 46, 48 tms 2, 57, 59, 61, 62, 64–82, 146 Training post-editor 203, 204, 207, 211, 215 Translatability 180, 182, 185 Translation vii, 1–5, 9–17, 17n, 18–20, 29–47, 49–57, 60, 63–67, 70–73, 76, 79, 80, 85, 86, 86n, 87, 87n, 88, 88n, 89–91, 91n, 92, 93, 95–102, 102n, 103–106, 109–112, 114–116, 119, 120, 120n, 121–129, 131–143, 145–152, 155, 156n2, 159, 160, 162–168, 170–174, 177–181, 183, 185, 186, 188, 189, 194, 196, 198, 198n, 199–224, 226, 228–231, 237, 239–242, 242n, 243–247 Translation learning activity 85, 88, 104 Translation memory vii, 38, 39, 42, 45, 47, 51, 53, 55, 73, 91, 91n, 110, 119, 134, 135, 140, 141, 152

Index Translation process vii, 2, 3, 9–13, 15–17, 17n, 18, 29, 31–36, 38, 44, 50, 57, 64, 87, 87n, 88, 89, 104, 109, 110, 112, 114, 119, 119n, 123, 125, 127–129, 131–134, 156n2, 170, 179, 198n15, 220 Translation project 65, 137, 138, 145, 148 Translation software 46, 52, 114 Translation task 2, 17n7, 20, 101, 121, 123, 129, 131, 132, 134, 209, 214, 222 Translation teaching 86, 90, 91, 104, 105, 135, 178 Translation technology 2, 4, 5, 9, 13, 32, 54, 86, 88, 90, 105, 137, 178, 199, 203–205, 207, 215, 242n Translation tool vii, 2, 37, 38, 41, 46, 50–53, 60, 85, 109, 178, 181, 202, 208, 218 Translation training 2, 37, 38, 53, 85, 87, 90, 104

251 Translation workflow vii, 150, 177, 178, 183, 199 Translator productivity 137, 138, 146, 150 Translator research style 9, 17, 21, 23, 24, 27, 28, 32 Triangulation 20, 120, 123 Usability 2, 4, 37, 51, 52, 54, 78, 144, 152, 166, 179, 198, 199 User Activity Data 3, 133, 137, 139, 150, 152 User requirement 37, 42, 51, 54 User survey 2, 37–39, 41, 42, 52 User-generated content 4, 178, 185, 199, 202 Web-based 1, 46, 48, 51, 67, 72, 73, 76, 78, 80–82, 139–141, 145, 181 Workflow vii, 4, 32, 38, 43, 45, 53, 63, 65, 141, 150, 154, 156n2, 157, 158, 162, 164, 166, 177, 178, 182, 183, 193, 194, 199, 204, 205