Learning Outcomes, Academic Credit and Student Mobility 9781553395560

Table of contents :
Cover
Title
Copyright
Contents
Acknowledgements
Biographies
1 Introduction
2 Foundations for Learning Outcomes and Credit Transfer
3 The Gulf Between Hope and Practice: South African Experiences with Learning Outcomes and Credit Transfer
4 Foundations and Reform Measures: Credit Transfer and Learning Outcomes Policy and Practice in the United States
5 Learning Outcomes for Credit Transfer: Reflections on the Australian Experience
6 International Student Mobility Based on Learning Outcomes and Workload: The European Credit Transfer and Accumulation System
7 Learning Outcomes, Progression, and Qualifications: Considerations for Vocational and Higher Education in the United Kingdom
8 Shifting Paradigms in Postsecondary Education: Historical, Conceptual, and Theoretical Frameworks Governing Outcomes-Based Approaches to Credit Transfer
9 The Overarching System: Structures, Policies, and Mechanisms Impacting Learning Outcomes and Credit Transfer
10 Adoption, Adaptation, and Implementation: The Complexities of Using Learning Outcomes to Advance Credit Transfer in Institutional Contexts
11 Summary of Findings and Recommendations

Citation preview

Learning Outcomes, Academic Credit, and Student Mobility Edited by Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

Queen’s Policy Studies Series School of Policy Studies, Queen’s University McGill-Queen’s University Press Montréal & Kingston · London · Ithaca

Copyright © 2020 School of Policy Studies, Queen’s University at Kingston, Canada

School of Policy Studies Publications Program Robert Sutherland Hall 138 Union Street Kingston, ON K7L 3N6 www.queensu.ca/sps/ All rights reserved. The use of any part of this publication for reproduction, transmission in any form, or by any means (electronic, mechanical, photocopying, recording, or otherwise), or storage in a retrieval system without the prior written consent of the publisher— or, in case of photocopying or other reprographic copying, a licence from the Canadian Copyright Licensing Agency—is an infringement of the copyright law. Enquiries concerning reproduction should be sent to the School of Policy Studies at the address above.

Library and Archives Canada Cataloguing in Publication Title: Learning outcomes, academic credit, and student mobility / edited by Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon. Names: Arnold, Christine, 1985- editor. | Wilson, Mary, 1967- editor. | Bridge, Jean, 1949- editor. | Lennon, Mary Catharine, 1978- editor. Series: Queen’s policy studies. Description: Series statement: Queen’s policy studies | Includes bibliographical references. Identifiers: Canadiana (print) 20200328158 | Canadiana (ebook) 20200328204 | ISBN 9781553395546 (softcover) | ISBN 9781553395553 (HTML) | ISBN 9781553395560 (PDF) Subjects: LCSH: Students, Transfer of. | LCSH: School credits. | LCSH: Competency-based education. Classification: LCC LB3064 .L43 2020 | DDC 371.2/914—dc23

Contents Acknowledgements................................................................................................................................................ v Biographies ................................................................................................................................................................... vii 1

Introduction ............................................................................................................................................................1 Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

2

Foundations for Learning Outcomes and Credit Transfer ....................... 19 Elisa Tortola, Christine Arnold, and Zanele Myles

3

The Gulf Between Hope and Practice: South African Experiences with Learning Outcomes and Credit Transfer .................... 51 Stephanie Allais

4

Foundations and Reform Measures: Credit Transfer and Learning Outcomes Policy and Practice in the United States .............. 79 Debra Bragg

5 Learning Outcomes for Credit Transfer: Reflections on the Australian Experience ......................................................................................................................... 115 Peter Noonan 6

International Student Mobility Based on Learning Outcomes and Workload: The European Credit Transfer and Accumulation System.......................................................................................................................... 141 Robert Wagenaar

iv

Contents

7 Learning Outcomes, Progression, and Qualifications: Considerations for Vocational and Higher Education in the United Kingdom ........................................................................................................................................ 167 Chris Winch 8

Shifting Paradigms in Postsecondary Education: Historical, Conceptual, and Theoretical Frameworks Governing Outcomes-Based Approaches to Credit Transfer .............................................. 199 Christine Arnold, Mary Wilson, Michael Potter, and Leesa Wheelahan

9

The Overarching System: Structures, Policies, and Mechanisms Impacting Learning Outcomes and Credit Transfer ..................................... 225 Mary Catharine Lennon, Paola Borin, Brian Frank, and Krista Pearson

10 Adoption, Adaptation, and Implementation: The Complexities of Using Learning Outcomes to Advance Credit Transfer in Institutional Contexts........................................................................................................................... 249 Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers 11 Summary of Findings and Recommendations ..................................................... 283 Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

Acknowledgements We are thankful for the generous support and encouragement of the Ontario Council on Articulation and Transfer (ONCAT). ONCAT has supported the work of the Ontario research consortium and international scholars and has provided both the impetus and the funding for the purposes of publication. The Ontario research consortium has benefitted from the contributions of those investigating and disseminating ideas about outcomes-based approaches to forging credit transfer arrangements and student pathways. We have also benefitted from the guidance and recommendations of stakeholders from colleges, universities, and quality assurance and research bodies, as well as other non-governmental and governmental agencies who share our interest in advancing student access and success. We acknowledge with appreciation the assistance of these individuals in shaping and sustaining this work. Specifically, we wish to acknowledge Nicole Fallon for the expertise and guidance she delivered throughout our time working together, and Nicola Simmons for the facilitation of an informative writing retreat that guided the direction this publication has taken. In addition, we wish to recognize the incredible contributions Elisa Tortola provided in researching, organizing, and authoring the “Foundations for Learning Outcomes and Credit Transfer” chapter (see chapter 2) for inclusion in this publication and extend our gratitude to our research assistant Zanele Myles, whose hard work and dedication made this publication possible. Finally, we would like to thank our reviewers Fiona McQuarrie, professor of Business at the University of the Fraser Valley and special projects coordinator at the British Columbia Council on Admissions and Transfer (BCCAT), and Ken Norrie, professor emeritus of Economics at McMaster University and first vice-president, research, at the Higher Education Quality Council of Ontario (HEQCO) and the editorial team at McGill-Queen’s University Press for their comments and contributions.

Biographies Stephanie Allais Stephanie Allais is the research chair of Skills Development and professor of Education at the Centre for Researching Education and Labour at University of the Witwatersrand. Her research is located in the sociology and political economy of education, focused on relationships between education and work. She was a fellow at the Centre for Educational Sociology at the University of Edinburgh, and before this managed and conducted research into qualifications frameworks in sixteen countries for the International Labour Organization (ILO). Dr. Allais has worked in government, distance education, trade union education, teaching high school, teaching adult basic education and training, and leading a student organization. She served on many committees by appointment of ministers of education in South Africa, and has been involved in numerous policy processes. Christine Arnold Christine Helen Arnold is an assistant professor in the Adult Education/Postsecondary Studies program in the Faculty of Education at Memorial University of Newfoundland. She holds a BA (Honours), BEd (Intermediate/Senior), and MEd in Teaching, Learning and Development from Brock University in St. Catharines, Canada and a PhD in Higher Education from the Ontario Institute for Studies in Education at the University of Toronto in Toronto, Canada. Her research interests include the student experience in postsecondary education, with a focus on student affairs/services and student mobility/transitions. More specifically, she studies student transitions in postsecondary education and the extent to which organization and information frameworks support movement.

viii

Biographies

Paola Borin Paola Borin is a curriculum development consultant at Ryerson University in the Office of the Vice Provost, Academic, with a background in postsecondary education policy, curriculum design, and teacher education from Stanford University, York University, and Brock University. Dr. Borin develops strategies and processes to enhance program curricula working at the department and the institution level on policy and practice to promote and enhance both curriculum development and assessment. Dr. Borin’s research assesses the impact of policies on curriculum, and instruction in postsecondary education. Debra Bragg Debra Bragg is the director of Community College Research Initiatives at the University of Washington and an emeritus endowed professor of higher education and founding director of the Office of Community College Research and Leadership at the University of Illinois at Urbana-Champaign. Her current research focuses on transitions within postsecondary education and between education and employment. Dr. Bragg currently leads the study of transfer partnerships that demonstrate high baccalaureate attainment rates for students of colour and low-income students, and she researches the growing phenomenon in the United States of applied baccalaureate degrees conferred by community colleges. In addition, she is a fellow of the American Educational Research Association (AERA). Jean Bridge Jean Bridge is an adjunct research professor with Brock University, Centre for Digital Humanities. Her research and creation in art and immersive/interactive media has, over three decades, been published and exhibited world-wide. Bridge has been instrumental in the development of interdisciplinary curriculum and cross-sector partnerships in applied creative fields and in building college-university collaborations across Ontario. She has innovated in student mobility through the conceptualization and production of an outcomes-based tool for comparison of specialized, multi-disciplinary programs and has investigated alternative academic credentials that validate and showcase student achievement.

Biographies

ix

Brian Frank Brian Frank is the associate dean of Teaching and Learning, the DuPont Canada chair in Engineering Education Research and Development in the Faculty of Engineering and Applied Science, and a professor in the Department of Electrical and Computer Engineering at Queen’s University. His research interests are in assessment in postsecondary education, complex problem solving, and supporting credit transfer and student mobility. Jovan Groen Jovan Groen is a senior research and evaluation analyst at the University of Ottawa. He has recently transitioned into this role following several years teaching course design and as acting director of the university’s Centre for University Teaching. With a background in curriculum development, Jovan has worked closely with academic programs in different processes of curriculum assessment, development, and review. Stemming from his research interests in postsecondary and adult education, Jovan’s work focuses on the scholarship of teaching and learning and transformative learning in postsecondary education settings. Mary Catharine Lennon Mary Catharine Lennon is a senior policy advisor at the Postsecondary Education Quality Assessment Board (PEQAB), a research associate with the Centre for the Study of Canadian and International Higher Education at the Ontario Institute for Studies in Education at the University of Toronto, and currently sits on the Board of Directors of the International Network of Quality Assurance Agencies in Higher Education (INQAAHE). She has an academic and professional background in international and comparative higher education policy research, development and advice. Her areas of expertise include system design, competency frameworks, quality assurance, evaluation, and learning outcomes. She has worked with institutional, provincial, inter-provincial, and international educational agencies including the Higher Education Quality Council of Ontario (HEQCO), Council of Ministers of Education, Canada (CMEC), and the Association of Commonwealth Universities (ACU).

x Biographies

Zanele Myles Zanele Myles is a master of education graduate in Curriculum Studies and Learning with a focus on special education at Memorial University of Newfoundland in St. John’s, Canada. She currently operates a special education tutoring business, Study Corner, which supports student success. She holds a BA (Honours) from the University of Guelph in Guelph, Canada and BEd (Primary/Junior) from Queen’s University in Kingston, Canada. Her research interests are in special education, which includes an emphasis on programming and curricula in self-determination; student inclusion within the classroom; and students with learning disabilities and how their experiences within the school system impact their learning. Peter Noonan Peter Noonan is a professor of Tertiary Education Policy in the Centre for Research on International Education Systems at Victoria University. He is also a fellow of the Graduate School of Education at the University of Melbourne. Professor Noonan is a member of the Editorial Board for the Policy Reviews in Higher Education Journal. His major research interests are in the design, governance, and funding of tertiary education systems. He was a member of the Expert Panel for the Review of Australian Higher Education (Bradley Review) and chaired the Expert Panel for the recent Review of the Australian Qualifications Framework (AQF). Krista Pearson Krista Pearson is the registrar at Algonquin College and holds a BA from Dalhousie University in Halifax, Canada, BEd from Acadia University in Wolfville, Canada, MEd from Nipissing University in North Bay, Canada, and a PhD from Lakehead University in Thunder Bay, Canada. Her areas of research focus on social justice and equity in a postsecondary education, research and practice in access to education, and international education. Dr. Pearson’s additional interests include organizational, curriculum, and policy development; and strategic enrolment with a focus on student and community development. Mary Pierce Mary Pierce is the dean of Business, Information Technology and Part-

Biographies

xi

Time Studies at Fanshawe College. Mary has been innovative and tireless in her efforts to build a dynamic and thriving academic environment. Mary has been a leader in the development of online education, international exchange programs for students, and pathway opportunities and community partnerships. She has been chair of the provincial ONCAT transfer project for business programs in Ontario and past chair of the Ontario Heads of Business. Mary holds a BA (Honours) in Economics from the University of Toronto in Toronto, Canada and an MA in Education from Central Michigan University in Michigan, United States. In 2015, Mary was recognized with a National Leadership Excellence Award by Colleges and Institutes Canada. Prior to her career in education, Mary had a senior-level career in business with Procter and Gamble, S.C. Johnson Wax, The Blackburn Group, and Citigroup International. Michael Potter Michael Potter is a teaching and learning specialist at the University of Windsor’s Centre for Teaching and Learning. Although his scholarship now focuses on storytelling and the use of nihilist, existentialist, and pragmatist philosophy in university teaching, he has published a variety of academic and popular works. He is the author of Bertrand Russell’s Ethics (2006), co-author of Leading Effective Discussions (2008), and Learning Outcomes Assessment: A Practitioner’s Handbook (2015), and co-editor of a special issue of the Canadian Journal for the Scholarship of Teaching and Learning focused on the role of the Arts and Humanities in the scholarship of teaching and learning (2015). Elisa Tortola Elisa Tortola is a master of education graduate in Educational Leadership at Memorial University in St. John’s, Canada and a Fellow of the School of Graduate Studies. She holds a BA (Honours) from York University in Toronto, Canada. Her research interests include influencers of student success and equitable pathways to postsecondary education. As an accomplished university administrator in Toronto with more than a decade of experience in postsecondary international student admissions and recruitment, Elisa has served in both institutional and national-level advisory roles to inform policy and enact positive change. She has worked closely with various Canadian consular missions abroad to profile Canadian postsecondary education globally and

xii

Biographies

has presented at numerous international conferences about national admission practices and the postsecondary education system. Robert Wagenaar Robert Wagenaar is a professor of History and Politics of Higher Education and director of the International Tuning Academy at the University of Groningen. The Academy is an education and research centre with focus on the reform of higher education programs. It runs a bi-annual SCOPUS, ERIC, and Web of Science indexed Tuning Journal for Higher Education. From 2003 until mid 2014 he was director of Undergraduate and Postgraduate Studies at the Faculty of Arts of the same university. He has been directly involved in European Union higher education policy initiatives, such as the development of the European Credit Transfer and Accumulation System (ECTS) since 1989. His research interests are in higher education innovation and policy making. Leesa Wheelahan Leesa Wheelahan is a professor at the Ontario Institute for Studies in Education at the University of Toronto where she holds the William G. Davis Chair in Community College Leadership. She started as a college teacher in Australia in 1994 and has researched and worked in pathways development ever since. Her research interests are underpinned by a concern with social justice in access to and the outcomes from postsecondary education. Dr. Wheelahan leads the Pathways to and through Education and Work research group at the Ontario Institute for Studies in Education. Her research and publications focus on the role of knowledge in curriculum, pathways, colleges, and postsecondary education policy. Richard Wiggers Richard Wiggers is the dean of the Cornwall campus of St. Lawrece College and former associate dean of the Department of Liberal Studies at Mohawk College. He holds a BA (Honours) from Carleton University in Ottawa, Canada, MA from the University of Ottawa in Ottawa, Canada, and a PhD from Georgetown University in Washington, United States. He has authored or co-authored nearly two dozen publications and edited collections, and had previous careers with the federal (Justice, Secretary of State, and Indian Affairs) and New Brunswick

Biographies

xiii

(Postsecondary Education, Training, and Labour) governments. From 2009–2016 he served as executive director, Research and Programs with the Higher Education Quality Council of Ontario (HEQCO) leading research projects on student services, teaching and learning, graduate enrolment expansion, and work integrated learning. Mary Wilson Mary Wilson is the director of the Centre for Academic Excellence at Niagara College of Applied Art and Technology. She holds an EdD in Theory and Policy Studies from the Ontario Institute for Studies in Education at the University of Toronto in Toronto, Canada. Dr. Wilson’s research and practice interests include educational development, postsecondary education policy, curriculum theory and practice, the history of postsecondary education, and postsecondary student learning experiences. Christopher Winch Christopher Winch is a professor of Educational Philosophy and Policy in the School of Education, Communication, and Society at King’s College London. He was head of the Department of Education and Professional Studies at King’s College from 2008 until 2012. A longstanding member of the Philosophy of Education Society of Great Britain, he was the chair from 2008 until 2011. He has worked in primary, further, and higher education. He researches mainly in the fields of philosophy of education, professional, and vocational education with an emphasis on professional knowledge and judgment and on comparative European vocational education and training (VET). In addition, Dr. Winch is also the author of the well-known publication Teachers’ Know-How: A Philosophical Investigation (2017).

1

Introduction Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

Aims of This Study This publication aims to critically assess the conceptual foundations, assumptions, and implications of using learning outcomes for the purposes of postsecondary credit transfer and student mobility. The study comprises a critical review of current approaches to the use of learning outcomes in furtherance of credit transfer across national and international jurisdictions. It employs a multivalent examination of their potential impacts in the unique context of Ontario, Canada. The written selections in this publication are the culmination of a multi-year study entitled Learning Outcomes for Transfer (LOFT), funded by the Ontario Council on Articulation and Transfer (ONCAT). Established in 2011, ONCAT was created to enhance student pathways and reduce barriers for students looking to transfer among Ontario’s forty-five public postsecondary institutions. The organization is member-driven and works with all public colleges and universities to progress the system of credit transfer while respecting institutional autonomy. The study was undertaken through collaborations and contributions from three scholarly groups: an Ontario research consortium, invited international scholars, and an assembly of provincial stakeholders. The first group, the Ontario research consortium (composed of Ontario college and university experts in credit transfer and learning outLearning Outcomes, Academic Credit, and Student Mobility, edited by Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon. Montréal and Kingston: McGill-Queen’s University Press, Queen’s Policy Studies Series. © 2020 The School of Policy Studies, Queen’s University at Kingston. All rights reserved.

2

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

comes) were the leaders of this research and cooperated on the careful selection of the second group, which was composed of five prominent international scholars who are acknowledged experts on matters of learning outcomes and credit transfer. The international scholars represent South Africa, the United States, Australia, Europe, and the United Kingdom and were invited to contribute contextually situated papers that consider (1) the theoretical and conceptual foundations that have led to the current application/redaction of learning outcomes for credit transfer; (2) the inherent assumptions associated with the valuing or devaluing of learning outcomes for credit transfer; and (3) the perceived implications of development and implementation at the system(s), institutional, administrative, and student levels. Distinguished international scholars were selected from those countries with significant outcomes-based experiences and education reforms, providing a platform for learning within the Ontario context. Many of these countries were early adopters of 1980s and 1990s outcomes-based and standards-based reforms and have been navigating this territory/terrain for some time with several successes and failures (Allais, Raffe, Strathdee, Wheelahan, and Young 2009). The Ontario research consortium members reviewed these papers and produced commentary that discusses the practical challenges and opportunities of using outcomes-based approaches for credit transfer within Ontario according to three themes (historical, conceptual, and theoretical frameworks; systems and structures; and institutions and curriculum). Recommendations were then drawn from this commentary for consideration within the province. Lastly, the project has benefited from a cycle of consultation with the final group of provincial stakeholders at various levels. This highly inclusive approach has been designed to gather as great a range of perspectives and insights as possible to inform the observations and recommendations that have emerged from this study. A Unique Moment for Critical Exploration The Ontario postsecondary system is experiencing a unique moment with the critical exploration of learning outcomes as a tool for credit transfer. In their distinct ways, colleges and universities in the province of Ontario have established broad-based implementation of learning outcomes to inform the structure of curriculum development and quality assurance activities across credentials. There is a provincial qualifications framework that incorporates the requirements and expec-

Introduction

3

tations of postsecondary credentials from undergraduate certificates, diplomas, and degrees through postgraduate programs. And finally, there is a concerted, province-wide effort to expand and improve student access and pathways with the support of government, its arm’slength agencies, and the postsecondary institutions themselves, with active experimentation through projects and policy discussions. With such factors in place, a comprehensive examination of the lessons to be learned from parallel endeavours in other jurisdictions and a determination of recommended practices and policies for further consideration is timely for the province and will be of great interest to scholars and practitioners across the country and beyond our borders. In Ontario, as well as in many other jurisdictions around the world, learning outcomes have been perceived to hold promise as a potential tool to inform credit transfer: Learning outcomes facilitate a clear comparison of credentials at the course, year, and program levels and, as such, can play a vital role in credit transfer, by both improving existing pathways to maximize student success and by increasing the overall number of transfer opportunities. When discipline experts from various institutions, sectors, and jurisdictions come together to discuss their subject area through a learning outcomes lens, they gain greater understanding of what is expected of students in each program. The clarity and trust that results from such work enables partners to more confidently build partnerships and pathways among programs and institutions, which ultimately expedites the often lengthy and complex articulation process. (Ontario Council on Articulation and Transfer 2015, 2)

Exploration and experimentation with regard to the use of learning outcomes to inform advances in student mobility and pathways have been supported by ONCAT, and these early forays have provided sufficient grounds for further consideration of outcomes-based approaches. This foundation has provided an ideal opportunity to conduct a largescale, comprehensive evaluation of learning outcomes for the purposes of student mobility across jurisdictions. We are hopeful that this work will help guide decision makers in Ontario—and in other jurisdictions—in determining whether they should invest in such an approach or consider alternatives, and if so, how and to what extent. The work outlined is necessary for informed decision-making with respect to the associated frameworks, policies, and procedures.

4

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

Over the past decade, significant, related scholarship has emerged in Ontario from independent researchers as well as scholars working in association with the Higher Education Quality Council of Ontario and the Centre for the Study of Canadian and International Higher Education at the Ontario Institute for Studies in Education at the University of Toronto. Although there are robust bodies of literature specific to student access, mobility, and credit transfer, as well as the use of learning outcomes as an organizing principle for curriculum design and assessment of student learning, there is a paucity of scholarship that studies the intersection of these concepts and their viability for informing recognition of previously completed postsecondary education. This publication will make a unique contribution to deliberations regarding this intersection through an assessment of the conceptual foundations, assumptions, and implications of using learning outcomes as a tool for credit transfer. The publication is primarily situated within the Ontario postsecondary education policy landscape and is therefore directly relevant to this jurisdiction. However, there are valuable insights for provinces across Canada and countries around the world, and the publication is intended to contribute thoughtfully to the work that Raffe (2011) refers to as policy learning. Credit Transfer and Student Mobility in Ontario Ontario’s college and university sectors have developed along relatively parallel trajectories in which credit transfer was not well structured into the larger system’s design (Skolnik 2016a; Wheelahan, Moodie, Lennon, Brijmohan, and Lavigne 2016). The establishment of ONCAT has signalled a turn toward a more linked system with greater emphasis on lifelong learning and the consequent need to maximize transfer from one qualification to another with optimal credit granted for prior study (Tuck 2007; Wheelahan et al. 2016). Improved credit transfer makes it possible to advance economic and social inclusion and to reduce costs of postsecondary education for all stakeholders. While Ontario’s universities were created over time through founding legislative acts that laid out distinct provider mandates for undergraduate and graduate degrees and which collectively constitute this branch of postsecondary learning, the Ontario college sector is relatively new, having been created in the 1960s as consisting of vocational certificate- and diploma-granting institutions separate from universities

Introduction

5

(Government of Ontario 1965). Unlike the junior college model in the United States, which offers associates degrees, Ontario colleges were not originally intended to be feeder institutions for progression to university baccalaureate degrees. Requisite Reforms Ontario has taken a mixed approach to credit transfer in the decades following the establishment of the college sector. In acknowledgement that better linkage between the two sectors would advance student mobility, reports such as Vision 2000 (Council of Regents for Colleges of Applied Arts and Technology 1990) recommended colleges adopt consistent, system-wide program standards; broader curriculum through increased emphasis on general education and transferable skills; and creation of advanced standing agreements in related areas of study. A subsequent Task Force on Advanced Training called for a planned system of advanced training with colleges granting degree-level credentials (Task Force on Advanced Training 1993). In addition, the 1995 Pan-Canadian Protocol on the Transferability of University Credits established principles for the development of degree completion agreements between colleges and universities (College-University Consortium Council 1999). Implementation of some of these reforms began in 2000 with the Postsecondary Education Choice and Excellence Act, which enabled colleges to offer a limited number of applied baccalaureate degrees and also allowed for the creation of college–university collaborative programs (Government of Ontario 2000). The introduction of college baccalaureate degrees—if only on a modest scale—supported progression while maintaining two parallel sectors (Skolnik 2016a). The encouragement of collaboration between the sectors opened the door to more creative linkages between colleges and universities. Subsequent legislation (Government of Ontario 2002) and key policy directives (Ministry of Training, Colleges and Universities [MTCU] 2003) however, more fully established colleges as career-focused institutions with the establishment of provincially defined, outcomes-driven vocational program standards; essential employability skills; and general education requirements meant to conform to the province’s emerging qualifications framework. By contrast, universities established university undergraduate degree-level expectations (UDLEs) for baccalaureate degrees expressed as learning outcomes (Ontario

6

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

Council of Academic Vice-Presidents 2005). The Ontario Qualifications Framework (OQF), which addresses the entire range of postsecondary credentials (Ministry of Training, Colleges and Universities 2009), is based upon these two distinct constructions of outcomes. It is worth noting that the communication of outcomes in the UDLEs mirrors the language of the OQF, but program standards are written in terms of workplace competency rather than in the specific language of the OQF, a reality that impedes determination of transfer equivalency between programs whose outcomes are differently aligned. New Centralized Credit System In 2011, the Government of Ontario announced spending of nearly $74-million CAD over five years to operate a new centralized system that facilitates transfers among institutions. The new centralized credit system aims included: (1) reducing the need for students to repeat coursework, (2) developing more transfer opportunities, (3) providing on-campus advisors/counsellors and orientation programs, and (4) refining a centralized website that assists students in identifying credits for transfer (Ministry of Training, Colleges and Universities 2011). As a result of this initial funding and subsequent renewals, ONCAT has, since its inception, pursued a range of activities that address problems of credit transfer through examination of practices, support of research, and information sharing. The organization has sought to directly engage stakeholders in the search for solutions that work for students and institutions within the constraints of the current system. The ONTransfer website (https://www.ontransfer.ca/) is a resource for students seeking and sorting existing transfer options. Its database of course equivalencies is a resource to help faculty and staff make decisions about credit transfer. The research ONCAT funds investigates student demand and patterns of transfer and develops analyses, principles, and decision-making tools. The most important of its activities is the seeding of on-the-ground collaborative initiatives that have established college/university curriculum committees and that are testing in practice effective methods and models for bilateral and multilateral agreements that transfer blocks of credit between colleges and universities. Categories of Credit Transfer Current Ontario transfer practices fall into two fundamental categories: (1) students seeking individual assessment of credits that can be trans-

Introduction

7

ferred from one institution to another; and (2) students transferring within prearranged articulation agreements. Individual evaluations of the credits for transfer requires the determination of course or level equivalency and advanced standing. This assessment of credit uses topline comparison of courses typically based on transcripts and course outlines. Individual assessment of credits may apply to students who have undertaken a partial or full credential. Alternatively, prospective arrangements for transfer of a block of credits are made between programs where there is some affinity between fields of study and it is presumed students may wish to pursue future education. Block transfer agreements are developed through a variety of processes at either the program or course level to determine the credit to be awarded for prior study. The typical process for degree completion involves examination of the provincial program standards applicable to the sending program; examination of the sending program’s curriculum and courses; assessment of these against the requirements of the receiving program; determination of the number of credits, entry point, and gaps between the transferrable credits and the entry point; and specification of bridge course(s) that may be required. Outcomes are increasingly used as a reliable and systematic index of student learning on which to base these agreements. Other measures or currencies commonly used in the evaluation of credit for transfer include content, curriculum, sequence, credit weight, learning resources, evaluations, assessments, and grading structures. Most transfer agreements in Ontario are bilateral although a growing number of multilateral agreements are being developed. Nevertheless, it is useful to make distinctions between pathway agreements that guarantee entry into programs and that offer advanced postsecondary education to students who have successfully completed specific credentials; generic pathways that relate institutions and families of programs to both facilitate entry and grant credit transfer for individual students; specific pathway agreements such as those coordinated between existing programs whose curricula are mapped; and enhanced pathway agreements that grant more access and credit than is standard. Enhanced pathways are based on collaborative curriculum, teaching, and learning assessment and require high levels of trust. Despite improvements to the credit transfer ecosystem in Ontario, there remain a number of deficiencies and limiting assumptions that impede the province’s capacity to achieve a fully linked system with less friction in student mobility. In particular, increased consistency in

8

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

standards; approved rules and processes; established trust between institutions; confirmed coherence in curriculum and pedagogy; varied entry mechanisms; and developed admission guidelines based on most recent qualification(s) are necessary principles that should guide transfer of students and transfer of credit (Wheelahan et al. 2016). Moreover, a patchwork of agreements exists between partnering institutions that are not applicable across a greater diversity of programs and institutions; pathways are often developed with limited consultation with potential employers, and adequately sustaining credit transfer pathways that are already in existence can be challenging (Wheelahan et al. 2016). Ontario and Canada in a Broader Policy Context Since 2006 Ontario has been striving to develop a postsecondary education system focused on accountability, accessibility, and quality, and the continued commitment of government is aimed at supporting student achievement, ensuring high-quality provision, and establishing fair accountability measures of success (Shanahan and Jones 2007). This work has involved a number of policy actors including government, arm’slength agencies, college and university non-governmental associations, and organizations responsible for quality assurance and credit transfer. Unlike some federated systems (such as the United States or Australia), the Canadian federal government does not have any jurisdiction, or federal ministry, over educational issues. Each province or territory is responsible for organizing, funding, and directing K–12 and postsecondary education independently.1 Lacking strong federal involvement necessitates a number of actors in postsecondary education policy, at both national and provincial/territorial levels, which creates discrete realms of power in the policy and quality assurance landscape (Skolnik 2016b). The numerous actors and discrete realms of authority in the Canadian policy and quality assurance landscape are a significant hindrance to sweeping changes. What has occurred, instead, have been small changes within the parameters of the existing systems that are articulating expectations within traditional courses and programs. Because of this, Canada has not had significant external pressure to revise the system (Haskel 2013). Unlike in Europe or South America, there have 1. The exception to this is Aboriginal education, which was formerly under the purview of the Ministry of Aboriginal Affairs and Northern Development and now falls under the purview of Indigenous Services Canada (ISC). See https://www.sac-isc.gc.ca

Introduction

9

been no international or regional forces to support large or strategic moves toward a system overhaul or competency-based education model. Instead, seeing the potential of the model, a variety of activities have taken place and are slowly moving the system forward. Incremental Approaches to Change Instead, an incremental approach to change is occurring in the various spheres of influence and control. Ad hoc and unaligned, but with the dedication of many major players, the interests and activities incorporating learning outcomes at the system level, in quality assurance activities, institutions, and the classroom, has increased. Similarly, student demand for flexible learning pathways and lifelong learning has pushed the system to be more connected and inclusive. While there hasn’t been a system overhaul, instead there have been tentative steps toward incorporating learning outcomes, and a gradual move toward a transfer system based on student competencies rather than a system based on administrative design (Lennon 2014). Each of the ten provinces and three territories have their own system and structure for higher credential provision. Universities typically provide undergraduate and graduate degrees, while colleges generally provide more technical diplomas similar to the description of a short cycle qualification (as being typically practically based and occupationally specific) in UNESCO’s 2011 International Standard of Classification of Education (UNESCO Institute for Statistics 2011). Some provinces, such as British Columbia, are structured to allow two years of college credits to be transferred into a bachelor’s degree. Quebec, alternatively, has a system where two years of college is mandatory for entrance into bachelor’s degree programs. Others, such as Ontario, maintain fairly discrete college and university sectors. The variety of system structures is important to understanding the breadth of work necessary to implement common expectations both within and across the provinces. Governments have differing involvement in the day-to-day activities of the institutions, varying by province as well as by sector. Traditionally, as colleges are more aligned with labour market needs, the governments maintain a strong role in determining program offerings, establishing program standards, and operating quality assurance frameworks. Alternatively, Canadian universities are given autonomy in all these aspects. The very small private sectors of postsecondary education provision in Canada are heavily regulated by the provincial governments.

10

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

Issues of Recognition The challenges of these disparate systems create issues of recognition both across the country and within provincial systems. There are significant challenges in student mobility, credit transfer, and articulation despite the volume of movement across provincial boundaries. In a survey of forty universities across the nation, it was found that 18 percent of students lost prior credits when transferring to another province (Heath 2012a). Similar research within Ontario highlights that students transferring between institutions within the province are also likely to lose prior credits, particularly if they are transferring from a college to a university (Heath 2012b). For decades, most provinces have operated transfer activities on a “case-by-case” model of course credit acceptance (Junor and Usher 2008), and many are still working on developing course-to-course credit transfer using course hour equivalences. One explanation for the continued work in this area is that there is no common definition of a credit: in some cases, it is twenty-four hours of classroom time and in others thirty-nine (Higher Education Strategy Associates 2012). Authors and Advice on Reading Authors One of the unique strengths of this publication rests in the diversity of experienced and knowledgeable perspectives among the contributing authors. The Ontario research consortium members were composed of experienced practitioners and scholars from across Ontario’s colleges and universities in various roles (faculty members, registrars, academic administrators, and educational developers), who selected the international scholars and have reflected upon and analyzed their invited perceptions, highlighting lessons learned and recommending future directions for investigation. The five invited international scholars represent South Africa, the United States, Australia, Europe, and the United Kingdom, and are listed below with their appropriate affiliations: • Dr. Stephanie Allais (Research Chair of Skills Development and Professor of Education at the Centre for Researching Education and Labour, University of the Witwatersrand, Johannesburg, South Africa) • Dr. Debra Bragg (Director of Community College Research Initiatives, University of Washington, Washington, United States, and

Introduction 11

Emeritus Endowed Professor of Higher Education and Founding Director of the Office of Community College Research and Leadership, University of Illinois at Urbana-Champaign, Illinois, United States) • Dr. Peter Noonan (Professor of Tertiary Education Policy in the Centre for Research on International Education Systems, Victoria University, Melbourne, Australia, and Fellow of the Graduate School of Education, University of Melbourne, Melbourne, Australia) • Dr. Robert Wagenaar (Professor of History and Politics of Higher Education and Director of the International Tuning Academy, University of Groningen, Groningen, The Netherlands) • Dr. Chris Winch (Professor of Educational Philosophy and Policy in the School of Education, Communication, and Society, King’s College, London, England) The biographies provided in this publication for the Ontario research consortium and international scholars demonstrate the broad-based nature of analysis derived from those working in both theory and practice at the departmental, institutional, and systems level; provincially, nationally, and internationally; as well as within the cognate and applied disciplines across colleges and universities. Our intentions are that this publication will inform further questions on policy and practice directions for all levels of the system in Ontario and contribute to comparative insights and experiences for international colleagues. We hope that this publication will have broad appeal for scholars and decision makers in postsecondary education, particularly faculty members, registrars, academic administrators, educational developers, and credit transfer advisors and evaluators. This publication is an edited collection, and therefore each written selection complements the others and the works can be read as a whole or in their separate parts. Reading Guide This introduction establishes the procedural approach that informs the work and introduces the themes of both policy and practice that will be examined in depth, setting the stage for the findings and recommendations set forth in the conclusion (chapter 11). In addition, it succinctly summarizes the current state of credit transfer and articulation work in Ontario and comments upon early explorations of the use of learning

12

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

outcomes as a tool for informing credit transfer. Encouraging lifelong learning and ensuring ease of postsecondary student access and mobility are common concerns and priorities for educational institutions and governments; however, credit transfer systems and the development of pathways remain challenging and the resource-intensive work can result in uneven achievement of intended purposes. We are attentive to the identification of areas of perceived peril and promise in the pursuit of outcomes-based approaches to building student pathways. The foundational chapter (chapter 2) provides a comprehensive literature review and environmental scan of significant initiatives, programs, and projects that involve outcomes-based approaches to credit transfer and student mobility. It was essential that we possessed an informed understanding regarding national and international discussions, debates, and current activities within this area prior to proceeding. The chapters by international scholars (chapters 3–7) provide an intentionally curated selection of international perspectives from South Africa, the United States, Australia, Europe, and the United Kingdom and comprise five foundational papers from those working in jurisdictions with active, related policy and project work. Our international contributors are well-established leaders in this field with a history of publications on credit transfer and learning outcomes. They represent a range of stances on the use of learning outcomes as a tool for transfer— from highly critical to mixed opinions and positive takes on the future potential. Collectively, they possess experience in vocational, liberal, and professional education as well as the cognate disciplines and their areas of expertise include educational policy, philosophy, political economy, and leadership and governance. Their papers provide the wisdom of experience and insights into the critical questions, issues and conditions, and activities necessary for consideration in the Ontario context and are arranged in alphabetical order within the publication: • Chapter 3 by Stephanie Allais opens the publication by sharing experiences from South Africa in the use of learning outcomes with specific reference to the process of teaching and learning, and credit accumulation and transfer. Allais provides an overview of the South African education and training system, an appraisal of the National Qualifications Framework and what it aimed to achieve, and an exploration of some lessons for other countries wanting to use learning outcomes to improve credit transfer and accumulation.

Introduction

13

• Chapter 4 by Debra Bragg centres on developments in the United States and the use of learning outcomes with specific reference to equity, credential completion, and credit accumulation and transfer. Bragg provides a historical perspective on credit transfer explaining the national focus on postsecondary education following World War II that emphasized community college education, a review of transfer reforms in the United States for increased efficiency and emerging transfer initiatives that focus on inequitable learning outcomes, and a synopsis of promising changes that may inform the Canadian context. • Chapter 5 by Peter Noonan centres on developments in Australia and the use of learning outcomes with specific reference to credit accumulation and transfer. Noonan provides an examination of credit transfer between the higher education and Vocational Education and Training (VET) sectors and outlines that this has been an important, but intermittent, policy priority for the past three decades; an overview of national policies and guidelines that have been put in place and refined to guide institutional practices; and an explanation of the purposes of the Australian Qualifications Framework (AQF), which guides all qualifications offered by Australian education institutions with policies on credit transfer, recognition of prior learning, and advanced standing. • Chapter 6 by Robert Wagenaar centres on developments in Europe and the use of learning outcomes with specific reference to student workload and credit accumulation and transfer. Wagenaar provides an introduction to the European Community Action Scheme for the Mobility of University Students, which was the start of the transfer program for students and teachers, an overview of the European Credit Transfer and Accumulation System (ECTS) initiated as a six-year pilot as a result of the Bologna Process, and an outline of the overarching European qualifications frameworks, Tuning subject area frameworks, and required feasibility study on Measuring and Comparing Achievements of Learning Outcomes of Higher Education in Europe (CALOHEE). • Chapter 7 by Chris Winch centres on developments in the United Kingdom and the use of learning outcomes with specific reference to progression and credit accumulation and transfer. Winch provides a history of outcomes-based qualifications, an application of epistemic ascent that indicates the importance of levels of increasing cognitive difficulty in curriculum, and an argument that

14

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

if learning outcomes are conceived of more like standards related to curricula and notional study hours, then design flaws can be eliminated. The Ontario research consortium chapters (chapters 8–10), which follow the contributions of the international scholars, consist of three papers that respond to the contributions of the international scholars. The response chapters are organized around three integrative themes (historical, conceptual, and theoretical frameworks; systems and structures; and institutions and curriculum) that were identified in the international scholars’ chapters. The themes selected explore the nature and utility of credentials and outcomes taxonomies, frameworks, and tools; the role of trust and relationship-building in establishing and maintaining credit transfer and pathway arrangements; and an exploration of roles, responsibility, agency, and voice in the ongoing work of examining the use of learning outcomes as a tool for credit transfer. • Chapter 8 by Arnold, Wilson, Potter, and Wheelahan reflects on historical, conceptual, and theoretical frameworks by investigating considerations of policy learning versus borrowing, the relative influences of intrinsic and institutional logics, and the nature of pedagogic discourse, which can be used to understand these historical origins and their effects on policy, practice, and pedagogy. • Chapter 9 by Lennon, Borin, Frank, and Pearson reflects on systems and structures by investigating the goals and explicit or implicit purposes of policy activities, the role of qualifications frameworks and quality assurance as a means to implement policies in learning outcomes and credit transfer, and the role of organization and leadership that can impact or influence the policy landscape. • Chapter 10 by Bridge, Groen, Pierce, and Wiggers reflects on institutions and curriculum by investigating educational providers’ adaptation of learning outcomes designed to serve both system goals and institutional requirements, the diverse range of institutional players that leverage and interpret system-wide outcomes frameworks and standards, and the patterns, potentials, and success factors that reside in Ontario outcomes-based transfer models, initiatives, and tools. Each of the three response chapters reflects upon key issues raised in the international scholars’ chapters and their implications for Ontario.

Introduction

15

Finally, the concluding chapter (chapter 11) provides the accumulated insights from all the chapters to summarize the findings and formulate recommendations for Ontario as it continues to consider the future exigencies of an outcomes-based approach to credit transfer. References at the end of each chapter throughout the publication identify sources and serve as bibliographic tools to support further scholarship. A variety of acronyms and nomenclature are used within this publication, as the various backgrounds of the international scholars lends itself to varied meanings of learning outcomes and credit transfer terminology in relation to the contexts in which these scholars are working. While we initially determined that it was important to provide a glossary of terminology, we realized that the variations among such a large group of scholars coming together for this study made it nearly impossible. As such, a review of the commonly held definitions of competencies and learning outcomes are provided in the foundation chapter (chapter 2), and the international scholars describe the terms they use within the bodies of their papers for clarity and consistency. It is important to note when reading though that the concept of “higher education” is understood differently across the jurisdictions involved in this publication. Colleges and universities (all postsecondary education qualifications regardless of vocational/occupational nature) are designated as part of the higher education system in North America. In contrast, in the United Kingdom, Europe, Australia, and South Africa further education is regarded as a distinct sector from higher education (Wheelahan 2004). References Allais, S., D. Raffe, R. Strathdee, L. Wheelahan, and M. Young. 2009. Learning from the First Qualifications Frameworks. Geneva, Switzerland: International Labour Office, & Skills and Employability Department. http://www.ilo.org/employment/Whatwedo/Publications/working-papers/WCM_041902/lang--en/index.htm College-University Consortium Council (CUCC). 1999. The Ontario College-University Degree Completion Accord. Summary Proceedings of the CUCC Forum, Toronto, ON. Council of Regents for Colleges of Applied Arts and Technology of Ontario. 1990. Vision 2000: Quality and Opportunity. Toronto, ON: Ministry of Training, Colleges and Universities. Government of Ontario. 1965. Bill 153: An Act to Mend the Department

16

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

of Education Act. 27th Legislature, 3rd session. http://cclp.mior.ca/ Reference%20Shelf/PDF_OISE/Basic%20Documents.pdf ———. 2000. Post-secondary Education Choice and Excellence Act, SO 2000, c 36, Sched. (2000). https://www.ontario.ca/laws/statute/00p36 ———. 2002. Colleges of Applied Arts and Technology Act, SO 2002, c 8, Sched. F. https://www.ontario.ca/laws/statute/02o08f?search=ontario+colleges+of+applied+arts Haskel, B. 2013. “Where There’s a Will …: Reforming Postsecondary Education in Canada’s and the European Union’s Decentralized Systems.” Canadian Public Administration, 56 (2): 304–321. Heath, N. 2012a. Student Mobility in Canada Across Canadian Jurisdictions. Windsor, ON: Pan-Canadian Consortium on Admissions and Transfer. ———. 2012b. Student Mobility Within the Province of Ontario: Supplement to the Report on the Pan-Canadian Survey of Student Mobility. Windsor, ON: Pan-Canadian Consortium on Admissions and Transfer. Higher Education Strategy Associates. 2012. Changing Times, Changing Places: The Global Evolution of the Bachelor’s Degree and the Implications for Ontario. Toronto, ON: Higher Education Quality Council of Ontario. Junor, S., and A. Usher. 2008. Student Mobility and Credit Transfer: A National and Global Survey. Toronto, ON: Educational Policy Institute. Lennon, M. C. 2014. “Incremental Steps Towards a Competency-Based Education System in Ontario.” Tuning Journal for Higher Education, 2 (1): 59–89. Ministry of Training, Colleges and Universities. 2003. Frameworks for Programs of Instruction. http://www.tcu.gov.on.ca/pepg/documents/FrameworkforPrograms.pdf ———. 2009. Ontario Qualifications Framework (OQF). http://www. tcu.gov.on.ca/pepg/programs/oqf/ ———. 2011. Policy Statement for Ontario’s Credit Transfer System. https://www.ontransfer.ca/files_docs/content/pdf/en/news_ and_events/news_and_events_2.pdf Ontario Council of Academic Vice-Presidents. 2005. Ontario Council of Academic Vice-Presidents (OCAV) Guidelines for University Undergraduate Degree Level Expectations. http://cou.on.ca/reports/ guidelines-for-university-undergraduate-degree-level-expectations/ Ontario Council on Articulation and Transfer. 2015. ONCAT Position Paper: Learning Outcomes in Credit Transfer. http://67.21.231.62/sites/

Introduction

17

default/files/oncat_positon_paper_on_learning_outcomes.pdf Raffe, D. 2011. “National Qualifications Frameworks: What Can Be Learnt from the International Experience?” Journal of Contemporary Educational Studies/Sodobna Pedagogika, 4: 66–80. Shanahan, T., and G. Jones. 2007. “Shifting Roles and Approaches: Government Coordination of Post-secondary Education in Canada, 1995–2006.” Higher Education Research and Development, 26 (1), 31–43. Skolnik, M. 2016a. “(How) Do Quality Assurance Systems Accommodate the Differences Between Academic and Applied Higher Education?” Higher Education, 71 (3): 361–378. ———. 2016b. “Situating Ontario’s Colleges between the American and European Models for Providing Opportunity for the Attainment of Baccalaureate Degrees in Applied Fields of Study.” Canadian Journal of Higher Education, 46 (1): 38–56. Task Force on Advanced Training. 1993. No Dead Ends: Report of the Task Force on Advanced Training to the Minister of Education and Training. Toronto, ON: Ministry of Education and Training. Tuck, R. 2007. An Introductory Guide to NQFs: Conceptual and Practical Issues for Policy-makers. Geneva: International Labour Office. https:// www.ilo.org/wcmsp5/groups/public/@ed_emp/@ifp_skills/documents/instructionalmaterial/wcms_103623.pdf UNESCO Institute for Statistics. 2011. International Standard Classification of Education (ISCED). http://uis.unesco.org/en/topic/international-standard-classification-education-isced Wheelahan, L. 2004. Community Colleges for Australia. Melbourne, Victoria: Department of Education and Training. http://www.eduweb. vic.gov.au/edulibrary/public/highered/postsece/Community_colleges_for_Au stralia.pdf Wheelahan, L., G. Moodie, M. C. Lennon, A. Brijmohan, and E. Lavigne. 2016. Student Mobility in Ontario: A Framework and Decision Making Tool for Building Better Pathways. Toronto, ON: Centre for the Study of Canadian and International Higher Education, OISE-University of Toronto.

2

Foundations for Learning Outcomes and Credit Transfer Elisa Tortola, Christine Arnold, and Zanele Myles

Introduction Learning outcomes have become somewhat ubiquitous within postsecondary education; and yet, discussions continue to arise regarding agreed-upon meaning, adoption, implementation, and application. The purpose of this chapter is to lay the foundation for the chapters that follow by delivering a comprehensive literature review and scan of significant international and supranational examples of outcomes-based mobility initiatives specifically at the intersection of learning outcomes and credit transfer. An informed understanding of national and international discussions, debates, and current activities within these areas was essential prior to proceeding with our collective work. This chapter provides an overview of the importance of credit transfer and current transfer rates, reviews the complexities and challenges of learning outcomes, highlights the intended function of taxonomies and qualification frameworks in an attempt to provide better clarity surrounding credential or qualification attainment, and outlines international and supranational examples of outcomes-based mobility initiatives. The international and supranational examples of outcomes-based mobility initiatives that are described herein are not representative of all outcomes-based initiatives, but rather have been selected purposeLearning Outcomes, Academic Credit, and Student Mobility, edited by Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon. Montréal and Kingston: McGill-Queen’s University Press, Queen’s Policy Studies Series. © 2020 The School of Policy Studies, Queen’s University at Kingston. All rights reserved.

20

Elisa Tortola, Christine Arnold, and Zanele Myles

fully for the intersection that they feature between learning outcomes and credit transfer. There are limited outcomes-based initiatives intentionally designed with student mobility at the forefront, and as such we have attempted to provide several worthwhile examples for consideration. While the literature and initiatives outlined may seem fragmented and resource-intensive at times, there are significant movements occurring nationally and internationally, and it is important to take a moment to pause and critically reflect on the entire breadth of work and the lessons learned. Foundational Literature and Environmental Scan Credit Transfer and Why It Matters Given the rise in student mobility and the demands of the knowledge-based economy, postsecondary institutions face continued pressure to produce more graduates more rapidly and to facilitate a smooth transition for students transferring between institutions. Student mobility and credit transfer are essential components of ensuring credential completion for students across disciplines and programs. In Canada, a recent report by the Government of Canada (2017) suggests that 76.1 percent of new jobs projected between 2017 and 2026 will require a postsecondary education credential. Coupled with this is the fact that students may attend multiple institutions during the pursuit of their postsecondary credentials. According to Hinchley and Greenlee (2018), Ontario postsecondary students often make a program or institutional change between their first and second years of study. The authors report that 18.5 percent of Ontario students changed their program of study by their second year, 2.3 percent transferred to a different institution within Ontario, and 0.4 percent transferred to another institution within Canada during this same period to complete their credential. Although these statistics may indicate relatively low numbers of students transferring within Canada, Heath (2012) posits students may not complete their credential(s) if institutions do not allow for a smooth transition and the recognition of previously completed studies. The author also suggests that students may transfer institutions due to program choice, or unavoidable life circumstances such as family relocation or employment opportunities. In the United States, Shapiro et al. (2018) examined the mobility patterns of approximately 2.8 million new postsecondary education stu-

Foundations for Learning Outcomes and Credit Transfer

21

dents and determined that within the first six years of their studies, 38 percent of those students transferred to another institution before completing their credential, representing nearly two out of every five students. Likewise, Mullin’s (2012) report on behalf of the American Association of Community Colleges, emphasizes the important role community colleges have played in fostering the transfer of students to universities for degree-level studies in the United States. The report states that 82 percent of community college students, granted full credit transfer, earned their bachelor’s degree at the receiving institution within the time observed, whereas only 42 percent completed degree studies after receiving partial credit for their previous studies. However, despite the importance of student mobility and credit transfer within postsecondary education systems, there are several impediments that arise for students and institutions. Chase (2010) describes the typical credit transfer review process as being influenced by three factors: “(a) the type of accreditation of the sending institution, (b) whether articulation or transfer agreements existed, and (c) the similarity of coursework at the sending institution to coursework offered at the receiving institution” (103). Generally, institutions without formal agreements in place with the sending institution review requests for credit transfer on a case-by-case and course-by-course basis, a process that can negatively impact both the student and the institution. This traditional process exposes students to subjective evaluations with a potential outcome that credit may not be granted, ultimately resulting in lost time and increased costs (CMEC 2012). As a result, at the national level, the Council of Ministers of Education, Canada (CMEC) established a working group to address the issue of student mobility and develop a national approach to improve credit transfer across Canada. Given the vast provincial differences in assessing credit transfer, CMEC has acknowledged that this process has been developing slowly over several years (CMEC 2012). To improve the credit transfer process, institutions, governments, and supranational organizations have proposed and implemented various educational reforms. These include but are not limited to the use of learning outcomes, taxonomies, the introduction of qualifications frameworks and meta-frameworks, and arguably one of the largest-scale educational reforms—the Bologna Process—all delineated in the sections that follow.

22

Elisa Tortola, Christine Arnold, and Zanele Myles

Learning Outcomes—Complexities and Challenges Articulating clearly defined learning outcomes for individual courses or programs may assist with their evaluation during the credit transfer process. As learning outcomes for credit transfer move from localized to more globalized applications, it is critical to establish agreed upon and widely understood meanings for effective reform to take place. However, determining a universally suitable definition has proved challenging not only because the terms must apply to varied disciplines and programs (i.e., vocational, college, and university), but also because the selected terminology can have very different meanings from one language to another. Birtwistle, Brown, and Wagenaar (2016) suggest that one of the fundamental obstacles to this challenge is that language is often “culturally and historically bound” (209); to illustrate this point, the authors explain how the term “competences” carries a meaning that is different in the United Kingdom from what it is in Europe and North America (210). The lack of a universally accepted definition of learning outcomes has hindered their widespread adoption (Adam 2006; Allan 1996; Blaich and Wise 2011; Deller, Brumwell, and MacFarlane 2015; Meda and Swart 2018). Experts in the area of learning outcomes such as Adam (2006) have espoused that, although this may be the case, the similarities in the varied definitions are striking. While there are varying understandings of learning outcomes in Germanic culture (Winch, see chapter 7), in accordance with Anglophone traditions, a learning outcome is a statement that often utilizes similar descriptors that highlight what a learner or student should know, understand, or be able to do at the end of a specified learning period (Birtwistle, Brown, and Wagenaar 2016; Wagenaar 2018). A sample definition of the term can be found in the work of Zakani, Frank, Turner, and Kaupp (2016) who posit that “learning outcomes are statements that clearly describe what a learner is expected to know and be able to do at the end of a period of study, which might be a unit, a course, or a credential” (2). The learning period referenced may take the form of a course, a module, a series of educational experiences, or an entire degree or diploma program. A review of the current literature quickly brings to light many similarities in the language used to describe learning outcomes. Terms such as what a learner should know, understand, or be able to do, as well as references to the skills or abilities attained by the learner, are common descriptors found in learning outcome statements within an educa-

Foundations for Learning Outcomes and Credit Transfer

23

tional context (Adam 2008; Biggs and Tang 2011; Gallavara et al. 2008; Lennon et al. 2014). The process of achieving learning outcomes and the expected timeline for completion are also quite intentional and are often oriented around a learning activity such as a course, a credential, a program, or a series of collegiate experiences (Carter, Coyle, and Leslie 2011; Klein-Collins 2012; Potter and Kustra 2012; Zakani et al. 2016). All of these rely on the stipulation that the outlined goals must be measured, verified, demonstrated, or objectively assessed (Ascough 2011; Hussey and Smith 2002; Kennedy, Hyland, and Ryan 2009; Shipley 1995). The literature surrounding learning outcomes also includes some differences in definitional language, such as including in the goals an expected change in student behaviour or a focus on the depth and breadth of a student’s performance (Allan 1996; Lennon 2014). It may also allude to the fact that learning outcomes may be broad or narrow in scope (Adam 2008), that they may highlight a standard of competence (Lokhoff et al. 2010), and that they are transferable skills that are not exclusively academic in nature (Caspersen et al. 2014). These nuances in descriptive or definitional language may create confusion between the terms learning outcomes and competencies, as similar descriptors appear in both, often resulting in the perception that the terms are interchangeable. Some have described learning outcomes as representative of a pedagogical approach that places the student at the centre of learning, realigning the focus from what the instructor hopes to accomplish to what the student will achieve at the end of the learning period (Barr and Tagg 1995; Sin 2014). However, it is important also to consider that learning outcomes may be specified differently depending on where they are situated. Within a qualifications framework, for example, they may be specified more loosely without the constraint of time. As Wheelahan (2011) explains, “Qualifications that are based on outputs sever the link between the institution and learning outcomes because they are based on the premise that learning outcomes can be defined independently of when, how or where learning takes place” (327). When learning outcomes occur within a program, course, module, or curriculum, on the other hand, they generally refer to achievement at the end of the specified learning period, allowing for assessment and progression to the next level of learning. In 2016, Lennon offered a simple way to consider the variety of actors involved, goals, target population, and activities in learning outcomes (see Table 2.1; Lennon 2016, 21). The taxonomy below demonstrates the complexity of how learning outcomes are used, par-

24

Elisa Tortola, Christine Arnold, and Zanele Myles

Table 2.1 Taxonomy of Learning Outcomes Initiatives Common Goals

Actors

Target Audience

Strategy Type

Transparency

Programs

Students

Articulation

Teaching and learning

Institutions

Public/employers

Implementation

Institutional improvement/ quality

Discipline associations

Faculty (course design)

Measurement

System design

Quality assurance Program agencies/accredi- (curriculum detation bodies velopment)

Labour market alignment and economic development

National governments

Institution (accountability)

International coordination (and comparison)

International/ regional government or nongovernmental organizations

System level (coordination and accountability)

Note: Reprinted from M. C. Lennon (2016), “In Search of Quality: Evaluating the Impact of Learning Outcomes Policies in Higher Education Regulation.” Ph.D. Dissertation, University of Toronto.

ticularly considering that the elements are not mutually exclusive and often span many columns and rows. Because of this maze of activities, attempting to define a learning outcome becomes increasingly complicated. Similarly, the term competencies, like learning outcomes, also lacks a widely agreed-upon definition (Adam 2006). Terms such as skills, knowledge, and understanding, as well as ability, are all used to describe what a competent learner will gain following a successful learning experience (Lokhoff et al. 2010; Mitchell, Feltham, and Trotter 2013). Competencies are often broken down into two categories, competency-based education (CBE) and competency-based training (CBT).

Foundations for Learning Outcomes and Credit Transfer

25

CBE finds its roots in teacher education (Burke 1989; Burke et al. 1975; Elam 1972) and is more closely linked with vocational programs at the postsecondary level (Allan 1996).1 This type of education differs from traditional programs in that it allows for a more flexible completion period and provides a customized pace of learning through its modular design (Capella University 2017; Ford 2014; Malan 2000; Nodine 2016; Tuxworth 1994). Competency-based training, on the other hand, tends to be more heavily focused on training learners for the workforce (Parker and Walters 2008), although undergraduate programming also uses CBT, it has more often been associated with preparing learners for industry-defined roles that involve demonstrating mastery skills or performance ability, or attaining a predefined level of uniformity in order to comply with industry-defined standards (Guthrie 2009; Hodge and Harris 2012; Hodge 2007; Mulcahy 2000; Okoye and Isaac 2015). The European Commission (as cited in Adam 2006) posits that learning outcomes name certain evaluative competencies that a student must attain to progress; it is the associated nomenclature of these terms that leads to confusion. Taxonomies and Qualifications Frameworks Further to lacking an agreed-upon definition, the implementation of learning outcomes also requires the simultaneous introduction of agreed-upon methods of evaluation. Because of their referential and comparative functions, taxonomies and qualifications frameworks have been positioned to serve as calibration instruments that use learning outcomes statements as a signalling device for evaluating equivalencies and distinctions. For learning outcomes to assist in the progression towards credential attainment, the expected outcomes must be measurable to demonstrate and confirm achievement within or across tranches of differentiation among qualifications. One commonly explored option of evaluation is the use of taxonomies. Taxonomies have featured predominately in the assessment of learning outcomes as a classification tool (Duan 2006). They provide a means to measure outcomes at various levels of learning through the use of descriptive terms that clearly specify expectations. Some of the 1. Although some may associate vocational education with the traditional trade programs—automotive, carpentry, plumbing, and electrical—within postsecondary education vocational programs may also refer to various professional credentials such as those in medicine and engineering.

26

Elisa Tortola, Christine Arnold, and Zanele Myles

most frequently referred to taxonomies include Bloom’s Taxonomy of Educational Objectives and the Structure of Observed Learning Outcomes (SOLO) Taxonomy. In the revised version of Bloom’s Taxonomy (featured in the graphic below) each level represents a more complex cognitive stage of learning, with lower-order skills listed in the bottom half, that students must advance through to progress (Adams 2015). Similarly, the SOLO Taxonomy (Biggs and Collis 1982) also offers a hierarchical scale of cognitive analysis to measure learning. SOLO, however, features five hierarchical levels beginning with pre structural and progressing in complexity to end with extended abstract (Biggs and Tang 2011; Lucas and Mladenovic 2009; Lueg, Lueg, and Lauridsen 2016). Taxonomies serve an important purpose in supporting the assessment of clearly articulated learning expectations while also providing an opportunity for curriculum design and programming that align with learning outcomes, whereas qualifications frameworks support student mobility (Carter et al. 2011; Galloway 2008; Raffe 2003). There are numerous examples of taxonomies, such as Bloom’s Taxonomy, that have been successfully implemented to classify learning outcomes (see Figure 2.1; Duan 2006). Incorporating action verbs found in taxonomies allows learning outcomes to be written not only in a way that is observable and measurable, but also with a clarity that is universally understood and palatable to a wide audience, including students, parents, and academics (Stanny 2016). Critique, identify, define, compose, and debate are all examples of action verbs found in Bloom’s Taxonomy (Adam 2008; Carter et al. 2011). Employing taxonomies assists in providing clarity for curriculum and program creation by explicitly describing the expectations of a student’s knowledge acquisition, skill, or ability following a prescribed learning period (Adam 2008). In addition to exploring the use of taxonomies as evaluation tools, it is also important to consider the role that qualification frameworks have played in attempting to provide better clarity surrounding credential or qualification attainment. Qualifications frameworks differ from taxonomies in that they are decoding devices for interpreting and understanding credentials or qualifications such as degrees, certificates, or experiential learning. They can assist potential employers in evaluating applicants’ credentials or be used to build bridges either within or between the employment, education, and training sectors by demonstrating how qualifications can relate to one another and how they may be combined (Allais, Raffe, and Young 2009). As an example, the Scottish Credit and Qualifications Framework (SCQF) is a 12-level

Foundations for Learning Outcomes and Credit Transfer

27

Figure 2.1 Bloom’s Taxonomy

Note. Reprinted from Vanderbilt University Center for Teaching. https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/

national qualifications framework (Raffe 2011). This framework, which has existed for almost two decades, was designed to create a national language that would describe the various levels of education in Scotland as well as define credit points obtained after each level. The SCQF was intended to assist in creating a seamless opportunity for student mobility and lifelong learning through a nationally implemented qualifications framework (Raffe 2003). It is important to understand and acknowledge the differences in the design and purpose of some key qualifications frameworks. The European Qualifications Framework (EQF), for example, is referred to as a meta-framework (Allais et al. 2009; Bohlinger 2008; Raffe, Gallacher, and Toman 2008). The EQF is not a national qualifications framework in that it does not belong to any specific European Union member country and does not “include binding mechanisms of recognition addressed to individuals” (Bohlinger 2008, 98). Instead, it acts as an umbrella framework positioned above national frameworks to guide their de-

28

Elisa Tortola, Christine Arnold, and Zanele Myles

velopment and encourage cross-recognition with other frameworks (Raffe, Gallacher, and Toman 2008). The EQF is loosely designed, so as to encourage pathways and linkages between “equivalent branches or sectors of learning in different countries” (Raffe, Gallacher, and Toman 2008, 64); as such, it relies on the voluntary cooperation of various nations for its implementation and use (Bohlinger 2012; Fernie, Pilcher, and Smith 2014; Young 2008). Unlike the SCQF, which determines credits achieved at each level, the European Credit Transfer and Accumulation System (ECTS) and the European Credit System for Vocational Training (ECVET) were designed specifically to support the credit transfer process and encourage mobility (Howieson and Raffe 2013). Examples of the Assessment and Implementation of Learning Outcomes Despite the many challenges faced by institutions and organizations, they continue to explore, implement, and assess learning outcomes. Institutions of higher learning are required to demonstrate to their constituents and to government that they are providing a service that merits the qualifications awarded upon successful completion. They are required to account for the public funding provided. Many are beholden to governments and external accrediting bodies for funding and the maintenance of their accreditation and are obligated to ensure that they are providing quality education to the students they serve (Coates 2016; Kuh and Ikenberry 2009; Kuh, Jankowski, Ikenberry, and Kinzie 2014). Measurable learning outcomes have become the method of choice for many institutions and for many countries. This was demonstrated on a large scale with the Bologna Process (Caspersen et al. 2014) and in the reforms, policy changes, and national frameworks that are being created or fine-tuned around the globe. The difficulty for postsecondary institutions lies in the implementation and assessment of these measures. Implementation requires sustained commitment over a long period and collaboration across sectors such as government branches and various postsecondary institutions (Raffe 2007). For example, when referring to the large-scale SCQF, Raffe (2007) credits the success of this twenty-one-year process to its voluntaristic structure, its flexible design and rules, the setting of realistic goals, and a development and implementation model based on cooperation and partnerships. The various participants involved in implementing the SCQF had a clear stake and benefit in its success. Colleges and universities benefited from the prospect of seamless mobility for students across postsecondary in-

Foundations for Learning Outcomes and Credit Transfer

29

stitutions; professional organizations benefited from their programs gaining recognition in the new system; and postsecondary education organizations benefited from leading the implementation (Raffe 2007). In the Canadian context, one of the most notable examples of learning outcomes implementation has taken place in the province of Ontario. By the mid-2000s, the Ontario college sector had seen the validation of vocational program standards, essential employabilities skills, and general education requirements as core to non-degree qualifications. Following the establishment of college degree granting by Ministerial Consent in 2000, the inclusion of degree-level expectations, which greatly correspond to the university undergraduate degree-level expectations (UDLEs), was one set of comparative descriptors among others for all postsecondary credentials in the Ontario Qualifications Framework (Ministry of Colleges and Universities, n.d.; Ministry of Training, Colleges and Universities 2009). Learning outcomes adoption followed a similar timeline for Ontario universities. This implementation began in the early 2000s and was precipitated in part by an awareness of the rising interest in learning outcomes in Europe, where the ECTS was employing learning outcomes and student workload measures within qualifications frameworks (European Higher Education Area, n.d.), and the shift from faculty-centred to learner-centred education in the United States, where Robert Barr and George Tagg (1995) had published their watershed article, “From Teaching to Learning—A New Paradigm For Undergraduate Education,” in Change magazine (1995). In December 2005, the Ontario Council of Academic Vice-Presidents formally adopted the UDLEs (Ontario Universities Council on Quality Assurance 2019) which resulted in universities embarking on the implementation of learning outcomes via the “Guidelines for University Undergraduate Degree Level Expectations.” These guidelines were created to serve as a framework for describing expectations of Ontario university graduates’ attributes and performance. Also critical in this implementation of learning outcomes in Ontario postsecondary education was the inclusion of learning outcomes as an explicitly named evaluative criteria in the quality assurance frameworks for both the colleges and the universities. The Randall Report (2010) for the Ontario College Quality Assurance Service and the 2010– 2011 report on the Council of Ontario Universities Quality Council (2011) both reference the use of learning outcomes as a quality benchmark, thereby triggering the generation of policy and practice levers that further drove the common adoption of learning outcomes through

30

Elisa Tortola, Christine Arnold, and Zanele Myles

quality assurance processes. The challenges that quickly emerged for the implementation of learning outcomes across program qualifications included a lack of localized expertise in developing, implementing, and evaluating a learning-centred curriculum, along with the existing stress of academic workloads and a history of not prioritizing curriculum leadership, coupled with the requisite contributions in tenure and promotion (Hubball et al. 2007). For successful implementation, collaboration was credited as being necessary, which is a sentiment echoed by many scholars (Hubball et al. 2007; Kuh et al. 2014; Raffe 2007), as well as the requirement for flexible and creative course structures (Hubball et al. 2007). International Initiatives and Projects—Models and Strategies for Student Movement As we look toward international models and strategies that promote the movement of students across institutions and across jurisdictions, there are some key initiatives to consider. In the next few sections, we highlight a sampling of influential projects at the intersection of learning outcomes and credit transfer that have taken place in Europe, the United States, and Australia. These initiatives vary in scope and impact, from transnational examples to more localized ones, each providing a unique lens on international efforts aimed at improving student mobility. In order to ensure a broad spectrum of examples, some of the initiatives chosen include the transnational Bologna Process, the regional and state-level efforts of the United States, Australia’s national efforts in the AQF (Australian Qualifications Framework), and finally the supranational project known as AHELO (Assessment of Higher Education Learning Outcomes). Europe The Bologna Process. Arguably one of the largest and most influential transnational initiatives is the Bologna Process. The 1999 Bologna Process (named for the Italian city where it was created) was the voluntary signing of a declaration by twenty-nine European education ministers indicating their willingness to participate in harmonizing and strengthening the European higher education system (Ministerial Conference Bologna 1999, 2016). A document viewed as a response to global pressures and growing competition on the world stage (Dobbins, Knill, and Vögtle 2011). Its founding principles included

Foundations for Learning Outcomes and Credit Transfer

31

1. The creation of a system that would allow for seamless student mobility by awarding easily understood and comparable degrees. 2. The potential to receive credit for the attainment of non-curricular skills obtained through lifelong learning as well as the development of a credit system to facilitate mobility between institutions throughout Europe, such as the ECTS. 3. A commitment toward quality assurance with “a view to developing comparable criteria and methodologies” (Ministerial Conference Bologna 1999, 2016). 4. The identification of two cycles of learning at the higher level—one the undergraduate (bachelor) degree, which it was determined must consist of a minimum of three years of study, and the second graduate studies (master’s degree). It was also declared that undergraduate study should be designed to qualify students directly for work in the European labour market (Ministerial Conference Bologna 1999; 2016). The Bologna Process also recognized the role of teachers, researchers, and administrative staff, to allow for ease of mobility of personnel among various institutions within Europe (Ministerial Conference Bologna 1999, 2016). Bologna represented a significant transnational effort to create what was termed the European Higher Education Area (EHEA) by the year 2010 (Dobbins, Knill, and Vögtle 2011). Included in the declaration was recognition that certain key areas would require attention and cooperation by the European university signatories, namely curriculum development and the integration of programs of study, a focus on training and research, and the exploration of memorandums of understanding to maximize student mobility between institutions of various European nations (Ministerial Conference Bologna 1999, 2016). Since the original signing in 1999, the ministerial conference takes place roughly every two years, and there are currently forty-eight participating countries. The Bologna Follow-Up Group (BFUG) organizes each conference and is made up of members from the signatory countries as well as representatives from the European Commission and additional advising organizations. The BFUG was also tasked with monitoring the progress and implementing agreed-upon measures by participating nations (Dobbins, Knill, and Vögtle 2011; Vukasovic, Jungblut, and Elken 2017). Each conference provides an opportunity for additional changes or new measures to be introduced. In 2003, for example, a three-cycle system that includes the addition of the doctor-

32

Elisa Tortola, Christine Arnold, and Zanele Myles

al level of study was introduced into the communiqué (The Bologna Process 2018; Dobbins, Knill, and Vögtle 2011). Following was the introduction of an all-encompassing qualifications framework utilizing learning outcomes that was adopted by the EHEA in 2005 and a requirement for all participating countries to create individual national frameworks by 2010 (Pálvölgyi 2017). The focus of the Bologna Process continues to be quality assurance and the recognition of qualifications for ease of student and employee mobility across Europe (The Bologna Process 2018). The Bologna Process and Tuning Disciplines. Tuning is a process that brings together faculty and subject matter experts across institutions to determine and articulate the essence of a specified discipline (Wagenaar, see chapter 6). This collaboration produces a consensus of reference points that institutions may utilize to build a student-centred curriculum and develop learning outcomes. As learning outcomes became an important element of the Bologna Process, they brought with them the challenge of introducing a new, unfamiliar concept. The response to this challenge was the Tuning Project. Funded by the European Commission and introduced in 2000, the Tuning Project provided a tool to assist participating nations in meeting Bologna objectives (Pálvölgyi 2017). As Lennon et al. (2014) write, “Faculty members from different countries were brought together to formulate common student learning expectations in order to support program reform. The process was extremely successful and provided a way for academia to have a voice in the system-level overhaul” (4). For its adopters, Tuning provided a methodology and system that would allow for calibration, development, and evaluation of programs and courses to align with the aims of the Bologna Process (González and Yarosh 2013). Furthermore, Tuning contributes to curriculum design in subject areas that includes markers for progression that are distinguished using learning outcomes while also facilitating the use of a common framework respected across international higher education. (Pálvölgyi 2017). Tuning is described as a bottom-up approach in that individual institutions are tasked with making the shift from a teacher-focused methodology to a learner-centred view of teaching (Birtwistle, Brown, and Wagenaar 2016). This new approach required the use of learning outcomes to define output-based programming and represented a significant paradigm shift. In Canada, the United States, Latin America,

Foundations for Learning Outcomes and Credit Transfer

33

Africa, Australia, Russia, and many others (Tuning Academy, n.d.), Tuning has been taken up as a multinational, multicultural, collaborative project. It requires innovation among institutions to implement the use of learning outcomes so as to enable effective decision-making autonomy. This bottom-up approach is a key element of the process as it places the power to influence national policies in the hands of educators (Pálvölgyi 2017). United States The Degree Qualifications Profile. Currently, there are no national or state-level qualifications frameworks in the United States. However, there are a number of regional and state-level projects such as the Degree Qualifications Profile (DQP), which are based on voluntary participation and often funded by philanthropic foundations. The DQP was developed collaboratively across regions and states and is meant to be a shared set of reference points for degrees at the associate, bachelor, and master’s levels that enhances educational productivity. The DQP was prompted in 2009 by the setting of a national goal to achieve the world’s highest proportion of college graduates by 2020. To this end, the Lumina Foundation led a series of discussions and meetings that engaged “United States and European educators, association leaders, government officials, and postsecondary education executives” and produced a series of white papers leading to the creation of the Degree Qualifications Profile in 2011 (Lumina Foundation, n.d., para. 3). The DQP is a non-compulsory meta-framework, developed to encompass American values and recognize the diversity within the United States education system while providing a benchmark for curriculum improvement and evaluation by utilizing learning outcomes (Bragg, see chapter 4). The DQP documentation is clear that its goal is not to standardize American degrees but to create clarity about the expectations of what graduates should know and be able to do following a course of study (Lumina Foundation, n.d.). The DQP framework consists of five learning categories that capture expected learning outcomes: (1) specialized knowledge, (2) broad and integrative knowledge, (3) intellectual skills, (4) applied and collaborative learning, and (5) civic and global learning (Adelman et al. 2014, 5). The framework is designed to support institutions in the evaluation of students pursuing an associate’s, bachelor’s, or master’s degree program across all disciplines of study, by providing a common language which may be used

34

Elisa Tortola, Christine Arnold, and Zanele Myles

to compare credentials and programs. Although the DQP was created as a guide to be applied broadly across all subject areas with a focus on achieving learning outcomes that would lead to a credential, creating discipline-specific learning outcomes through Tuning is encouraged. Adelman et al. (2014) describe how faculty with field-specific knowledge are asked to create learning outcomes that pertain to their area of expertise. The DQP differentiates itself from previous approaches in the United States that have tried to ensure the accountability of institutions of higher learning—by creating a meta-framework that has been developed over time through a voluntary consultative process with faculty and staff at more than four hundred institutions. Adelman et al. (2014) explain how learning outcomes within the DQP are described using active verbs such as identifies, categorizes, and prioritizes to ensure measurable assessment and comparisons. Qualitative learning outcomes are used to measure students’ success in earning their degrees in place of traditional quantitative measures, such as the number of credits completed (Adelman et al. 2014). Western Association of Schools and Colleges (WASC). The WASC Senior College and University Commission (WSCUS), a division of the Western Association of Schools and Colleges (WASC), piloted the DQP for voluntary participation of its member institutions from 2012–2014 (Western Association of Schools and Colleges, n.d.). As an accrediting body, WASC’s Degree Qualifications Profile requires that member institutions can clearly define the meaning, quality, and integrity of the degrees they confer. In an effort to assist institutions with this task, the DQP was offered as an optional framework that members could choose to adopt and incorporate at their institutions: “offering a point of reference and a common framework for talking about the meaning of degrees, but without prescriptions or standardization” (Western Association of Schools and Colleges, n.d., para. 5). Western Interstate Commission for Higher Education (WICHE) and Liberal Education and America’s Promise (LEAP). Regional organizations such as the Western Interstate Commission for Higher Education (WICHE) assist in facilitating the sharing of resources and collaboration among their constituents (Western Interstate Commission for Higher Education 2018). One of WICHE’s (2017) key programs is the Interstate Passport that facilitates “block transfer of completed lower-division general education attainment based on student learning outcomes rather than on specific courses and credits” between member

Foundations for Learning Outcomes and Credit Transfer

35

institutions (Shea 2019, para. 1). Shea (2019) explains that as of May 2019, more than thirty-eight thousand students have earned block credit from one of the thirty-two member institutions. These institutions currently represent fourteen states and many other institutions are in the process of joining this initiative. The nine learning outcomes outlined in the Interstate Passport are drawn from the Liberal Education and America’s Promise (LEAP) learning outcomes initiative, which was developed by the Association of American Colleges and Universities (AAC&U; Association of American Colleges and Universities, n.d.). As a national organization, AAC&U’s LEAP initiative promotes essential learning outcomes, principles of excellence, high-impact educational practices, authentic assessments (using student work and faculty rubrics), and students’ signature work (which encourages institutions to require students to submit their choice of a cross-disciplinary project that will be valued toward their degree). These services and offerings are shared with institutions of higher learning across the country (Association of American Colleges and Universities, n.d., para. 4). These essential learning outcomes and rubrics form a core resource for the institutions participating in the WICHE Interstate Passport. Australia Unlike the United States’ regional and state-level initiatives, Australia has introduced a large-scale, national qualifications framework in an attempt to facilitate student mobility and credit transfer. The Australian Qualifications Framework (AQF). Australia’s AQF was first introduced in 1995, revised in 2011, and its latest review was conducted and released in October 2019. In December 2019 the Australian Government accepted all the recommendations outlined in the newly released report, as they relate to higher education, along with accepting “the aims of the recommendations of the review in relation to vocational education, contingent on further discussions with state and territory governments” (Australian Government 2019, para. 4 ) (Noonan, see chapter 5).2 In the fourth edition implementation handbook, the Australian Qualifications Framework Advisory Board (2007), acknowledged the varying linkages with industry and institutions that are held by “…the schools sector, vocational education and training 2. For a copy of the updated report recommendations see: https://www.education.gov. au/australian-qualifications-framework-review-0

36

Elisa Tortola, Christine Arnold, and Zanele Myles

sector and higher education sector...” the AQF was expected to “…connect these in a coherent single Framework incorporating qualification titles and guidelines” (1). The goal was to create a national qualifications framework that would be comprehensive and consistent, while at the same time allow for flexibility (Australian Qualifications Framework Advisory Board 2007). The AQF has, overall, been “well regarded internationally and this has contributed to the high standing of Australian qualifications internationally” (Wheelahan 2011, 334) and lauded for its creation of a national VET system and general acceptance by all the sectors—likely due to sectoral autonomy and its voluntary participation structure (particularly for the higher education sector; Wheelahan 2011). However, there were several challenges faced by Australia that are not unlike those faced by other nations. The initiative was meant to facilitate a smooth transition for student mobility and credit transfer between education sectors based on fundamentally different curricular ideologies—namely those between the schools sector, the VET sector, and the higher education sector (Keating 2008; Wheelahan 2011). This resulted in a somewhat fragmented use of the AQF by the various education sectors. Although the AQF was accepted by all stakeholders (Keating 2008) the VET sector to which the AQF was most closely aligned adopted it more readily, utilizing it for credit transfer specifically within the VET sector itself, making it less effective for transfer between VET and higher education (Wheelahan 2011). Keating (2008) explains that the VET system is “…based upon industry competency standards and thus the currency of the VET sector is radically different to that of the university sector” (64). In its initial introduction, the AQF included a specific objective to create a national VET system stating that the AQF: “…encourages the provision of more and higher quality vocational education and training through qualifications that meet individual, workplace and vocational needs, thus contributing to national economic performance…” (Australian Qualifications Framework Advisory Board 2007). No similar notation for the higher education or schools’ sector was made, which has also been suggested to explain the strong affinity between the AQF and VET sector (Wheelahan 2011). Notwithstanding, this connection between the AQF and VET sector, the higher education and schools’ sectors have also maintained participation and interest. This has been attributed to the fact that each sector has been granted ownership of setting their own qualifications within the AQF (Wheelahan 2011), al-

Foundations for Learning Outcomes and Credit Transfer

37

though as Keating (2008) states, “it also comes at the cost of some discontinuity and inconsistency” (10). Three suggestions from the recent AQF review were highlighted in the Australian Government media release in December 2019, which recommend: • Senior secondary students can study subjects at school that count towards a vocational training qualification or university degree. • Recognition of microcredentials to allow providers to offer short, highly-targeted courses. • VET and higher education to have clear and flexible entry and exit points, as well as pathways within and between, to allow students to mix and match the subjects they study to meet their education requirements. (Australian Government 2019, para. 2) Further, although not considered a key driver for the AQF, it is worth noting that in an effort to align more closely with Tuning, Australia also introduced the Learning & Teaching Academic Standards project, which sought to develop learning outcomes across several disciplines (Gallagher 2013). Historically one of the top receiving countries of international students, (Curaj et al. 2012), Australia recognized that to maintain its competitiveness in the international higher education market, particularly considering the competition created by the Bologna Process and the Tuning Project, several changes would need to take place (Chase 2010; FitzGibbon 2014). Supranational Frameworks/Tools The OECD’s Assessment of Higher Education Learning Outcomes (AHELO) Through its introduction of AHELO, the Organisation for Economic Co-operation and Development (OECD, n.d.) hoped to facilitate the use of learning outcomes on a global scale by providing “data to governments, institutions and students…on…what students at the end of their first (bachelor level) degrees know and are able to do” (para. 1). The OECD describes these key stakeholder benefits in the following manner: for governments AHELO provides a tool to assess the quality of their tertiary education against international standards; for institutions it allows for benchmarking of learning outcomes against international standards and provides opportunity to ameliorate teaching and learning within their institutions; and for students it allows them to compare their personal achievements against the learning outcomes of their institution, as well as national and international standards (para. 1).

38

Elisa Tortola, Christine Arnold, and Zanele Myles

The AHELO study developed common expectations and assessed student performance at the end of a bachelor’s degree in seventeen countries around the world in three different knowledge areas to determine if a global assessment is possible and valuable (Tremblay, Lalancette, and Roseveare 2012). The pilot collected data from 249 institutions of higher education in seventeen countries and regions—including Canada (Ontario)—in an effort to mitigate the gap of information on the effectiveness of learning outcomes (OECD 2013a, 2). In the executive summary of the data analysis, the OECD concluded: “The AHELO feasibility study demonstrated that it is feasible to develop instruments with reliable and valid results across different countries, languages, cultures and institutional settings” (OECD 2013b, 1). Despite this seemingly positive outcome, the study also brought to light numerous tensions. The Ontario analysis noted an important issue regarding how various jurisdictions wanted to use the results of the international learning outcomes assessment as either system-level benchmarks or institutional and program-level improvement mechanisms. This inherently leads to questions of “identifying the primary goal of a large-scale international assessment that supports stakeholder needs, and creating a framework that is fit-for-purpose” (Lennon and Jonker 2014, 19). At a more fundamental level of concern about assessment, the AHELO initiative is not without controversy and criticism. In a letter addressed to OECD Secretary-General Angel Gurría, the then-president of the American Council on Education, Molly Corbett Broad, and the president and CEO of Universities Canada, Paul Davidson, outlined their collective concerns (M. C. Broad and P. Davidson, personal communication, May 7, 2015), such that AHELO failed to recognize the diversity of institutional academic courses and missions by attempting to create a standard solution for all institutions worldwide. Nevertheless, by pointing to teaching and learning as the central quality assurance issue and catalyst for AHELO, the OECD provides justification for a tool that will evaluate this broadly and fill an existing gap as a trustworthy resource that can be used by all (Shahjahan and Torres 2013). Measuring and Comparing Achievements of Learning Outcomes in Higher Education in Europe (CALOHEE) CALOHEE is the most recent project (started in January 2016) funded by the European Union Tuning Academy (Tuning CALOHEE, n.d.a.), which is an extension of the Bologna Process discussed previously

Foundations for Learning Outcomes and Credit Transfer

39

(Wagenaar, see chapter 6).3 The purpose of the project is to “develop multi-dimensional tests for five fields of study using similar methodology but tailoring them to the characteristics of the fields, thereby making it possible to compare students’ performance in a Europe-wide context” (Zlatkin-Troitschanskaia et al. 2017, 8). As Wagenaar (2018) explains, CALOHEE’s aim is threefold: to create benchmarks for the five designated areas of academic study, to provide comprehensive assessment frameworks for these areas, and to respect the individual missions and profiles of both the postsecondary institutions and the various degree programs. According to the Tuning CALOHEE (n.d.b.) website, the five subject areas of focus include engineering (civil engineering), healthcare (nursing), humanities (history), natural sciences (physics), and social sciences (education).4 The objective is to allow the academic community to play a central role in this process by initiating a bottom-up approach (Zlatkin-Troitschanskaia et al. 2017). One of the main motivations for the development of CALOHEE was the discouraging implementation levels of two of Tuning’s most critical reforms: “the introduction of the concepts of active learning and the student-centred approach” (Wagenaar 2018, 171). A deficiency in academic staff training and comprehension of these concepts, as well as students’ struggle with understanding them, are deemed the culprits of this disappointing result (Birtwistle, Brown, and Wagenaar 2016; González and Wagenaar 2003). To rectify this, CALOHEE aims to facilitate degree program updates so that the learning outcomes stipulated meet the goals not only of the program but also of students and society at large. The idea is to provide a clear reference along with the required materials and approach needed to implement the changes. Achieving this will create confidence for students in securing future employment and inspire civic engagement, as well as contribute to students’ personal growth. Providing students with a clear understanding of the competencies that their degree will afford them and how this will serve them in the future will also inspire them to become more engaged and invested in their learning (Wagenaar 2018). Despite comparative assessment being the main goal of CALOHEE, Wagenaar (2018) explains that it also hopes to affect quality through the updating of “existing subject area and sectoral frameworks” and “the development of meaningful ‘assessment frameworks’ which are 3. See https://www.calohee.eu/why-calohee/ 4. See https://www.calohee.eu/why-calohee-2/

40

Elisa Tortola, Christine Arnold, and Zanele Myles

drawn from these” (172). Tuning uses a more precise approach to the creation of learning outcomes statements than previous models, which focused on using the most fitting “verb.” Instead, Tuning suggests five elements must be incorporated when creating learning outcomes statements: “verb, type, subject, standard and scope/context” (Wagenaar 2018, 172). This “assessment framework” will require a new method of evaluation, given the limitation of traditional rubrics to evaluate international study programs or programs in different subject areas. The new European Subject Area Assessment Framework, which will be developed in the context of the CALOHEE feasibility study and will provide details beyond the traditional subject area assessment of what a student should “know, understand, and be able to do” upon completion of his or her studies. As Wagenaar (2018) writes, “The CALOHEE feasibility study should thus provide a solid basis for constructing reliable and sustainable sets of assessment items for each of the five subject areas covered by the feasibility study” (173). Wagenaar (2018) describes the instrumental role that the AHELO feasibility study played in the creation of CALOHEE. More specifically, the lessons learned from AHELO’s failings helped to inform the CALOHEE project to avoid these pitfalls. One of the primary differences is CALOHEE’s bottom-up approach, placing the academic community at the centre of its implementation and the reform of Europe’s higher education system, versus the top-down approach taken by AHELO. CALOHEE involves seventy academics and six student representatives from a wide range of countries building on the framework of the EHEA and worldwide Tuning projects. Despite Tuning’s global reach, it has managed to remain culturally contextual to regional differences, an element the AHELO Feasibility Study did not factor in, which caused a misinterpretation of results and a struggle with cohesion. A fundamental flaw of AHELO, according to Wagenaar (2018), is the “separation of a ‘generic skills and competencies strand’ and two ‘subject specific knowledge and skills strands’ for respectively, civil engineering and economics” (173). In Tuning these are considered inseparable, “based on the argument that generic competencies are not only developed in the framework of a domain of knowledge but are also perceived differently between educational sectors” (Wagenaar 2018, 173). CALOHEE’s first goal is in facilitating an assessment framework that will have the ability to reliably measure and compare the achievement of learning outcomes nationally and internationally. As a future goal, however, it hopes to implement state-of-the-art computer-based assess-

Foundations for Learning Outcomes and Credit Transfer

41

ment with the ability to assess deep knowledge and understanding as well as higher-level skills. Future computer-based assessment is considered an achievable goal considering the pace of current technological development, a reality that did not exist a decade ago when AHELO was introduced (Wagenaar 2018). Conclusion Within the context of dynamic and fluid postsecondary education systems across the globe, reform efforts have striven to produce and use outcomes-based approaches as a means to support the importance of student learning and the advantages of student mobility among institutions. This comprehensive review of the foundations that support an understanding of learning outcomes and credit transfer and the literature that surrounds them demonstrates how education reform continues to identify and address the future needs of students and fluctuating global economies, within diverse societal and cultural contexts. The ways in which nations and supranational organizations respond to these needs vary. There is now general agreement that learning outcomes at the program and unit level are beneficial aids in determining student achievement with respect to intended levels of learning (Wagenaar 2018). Our examination of the implementation of learning outcomes has drawn a line between the use of taxonomies in creating learning outcomes statements (which tend to remain static), and the creation and deployment of various qualifications and supranational frameworks (which are dynamic and often revised based on lessons learned and the intended purpose of such frameworks, in communications, reform, or system transformation (Allais, Raffe, and Young 2009, 25). Whatever the method of choice, outcomes have the potential in designing and determining equivalency of learning experiences for the purpose of student mobility. References Adam, S. 2006. “An Introduction to Learning Outcomes: A Consideration of the Nature, Function and Position of Learning Outcomes in the Creation of the European Higher Education Area. Article B. 2.31.” In the EUA Bologna Handbook. Brussels, Belgium: European University Association. ———. 2008, February. “Learning Outcomes Current Developments in Europe: Update on the Issues and Applications of Learning Out-

42

Elisa Tortola, Christine Arnold, and Zanele Myles

comes Associated with the Bologna Process.” In Edinburgh: Scottish Government. Presented to the Bologna Seminar: Learning Outcomes Based Higher Education: The Scottish Experience. http://aic.lv/ace/ace_ disk/2007_09/sem07_09/LO_edinb/Edinburgh_Feb08_Adams.pdf Adams, N. E. 2015. “Bloom’s Taxonomy of Cognitive Learning Objectives.” Journal of the Medical Library Association, 103 (3): 152–153. Adelman, C., P. T. Ewell, P. L. Gaston, and C. G. Schneider. 2014. The Degree Qualifications Profile: A Learning-Centered Framework for What College Graduates Should Know and Be Able to Do to Earn the Associates, Bachelor’s or Master’s Degree. http://degreeprofile.org/wp-content/ uploads/2017/03/DQP-web-download-reference-points-FINAL.pdf Allan, J. 1996. “Learning Outcomes in Higher Education.” Studies in Higher Education, 21 (1): 93–108. Allais, S., D. Raffe, and M. Young. 2009. Researching NQFs: Some Conceptual Issues. Geneva: International Labour Organization. http://www. ilo.org/public/libdoc/ilo/2009/109B09_294_engl.pdf Ascough, R. S. 2011. “Learning (About) Outcomes: How the Focus on Assessment Can Help Overall Course Design.” Canadian Journal of Higher Education, 41 (2): 44–61. Association of American Colleges and Universities. n.d. Liberal Education and America’s Promise. https://www.aacu.org/leap Australian Government. 2019, December 9. A New Future for VET and Higher Education. https://ministers.education.gov.au/tehan/ new-future-vet-and-higher-education ———. 2019, October 24. Australian Qualifications Framework Review. https://docs-edu.govcms.gov.au/documents/review-australian-qualifications-framework-final-report-2019 Australian Qualifications Framework Advisory Board. 2007. Implementation Handbook. https://www.aqf.edu.au/sites/default/files/ aqf-implementation-handbook-fourth-edition-2007.pdf Barr, R. B., and J. Tagg. 1995. “From Teaching to Learning: A New Paradigm for Undergraduate Education.” Change: The Magazine of Higher Learning, 27 (6): 12–26. Biggs, J., and C. Tang. 2011. Teaching for Quality Learning at University: What the Student Does. 4th edition. Maidenhead, UK: Open University Press. Biggs, J., and K. Collis. 1982. Evaluating the Quality of Learning: The SOLO Taxonomy (Structure of the Observed Learning Outcome). New York, NY: Academic Press. Birtwistle, T., C. Brown, and R. Wagenaar. 2016. “A Long Way to Go …

Foundations for Learning Outcomes and Credit Transfer

43

A Study on the Implementation of the Learning-Outcomes Based Approach in the EU.” Tuning Journal for Higher Education, 3 (2): 429–463. Blaich, C., and K. Wise. 2011. From Gathering to Using Assessment Results: Lessons from the Wabash National Study. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). Bohlinger, S. 2008. “Competences as the Core Element of the European Qualifications Framework.” European Journal of Vocational Training, 42 (1): 96–112. ———. 2012. “Qualifications Frameworks and Learning Outcomes: Challenges for Europe’s Lifelong Learning Area.” Journal of Education and Work, 25 (3): 279–297. Burke, J. W., ed. 1989. Competency Based Education and Training. London, UK: Falmer Press. Burke, J. B., J. H. Hansen, W. R. Houston, and C. Johnson. 1975. Criteria for Describing and Assessing Competency Programs. Syracuse, NY: National Consortium of Competency-Based Education and Training. Capella University. 2017. Competency-Based Education: A Policy Primer. http://s2.q4cdn.com/585385277/files/doc_downloads/white_papers/Competency-Based-Education-Embargoed.pdf Carter, I., J. Coyle, and D. Leslie. 2011. “Easing the Transfer of Students from College to University Programs: How Can Learning Outcomes Help?” Canadian Journal of Higher Education, 41 (2): 10–27. Caspersen, J., N. Frølich, H. Karlsen, and P. O. Aamodt. 2014. “Learning Outcomes Across Disciplines and Professions: Measurement and Interpretation.” Quality in Higher Education, 20 (2): 195–215. Chase, M. M. 2010. “Student Transfer Policies and Practices in the United States and Europe: Mobility without Loss of Credit.” Journal of Student Affairs Research and Practice, 47 (1): 99–119. Coates, H. 2016. “Assessing Student Learning Outcomes Internationally: Insights and Frontiers.” Assessment & Evaluation in Higher Education, 41 (5): 662–676. http://dx.doi.org/10.1080/02602938.2016.1 160273 Council of Ministers of Education, Canada (CMEC), Credit Transfer Working Group. 2012. Report of the CMEC Credit Transfer Working Group, 2012. Council of Ministers of Education, Canada. http://cmec. ca/Publications/Lists/Publications/Attachments/311/CTWG%20 Report%202012_EN.PDF Council of Ontario Universities Quality Council. 2011. Annual Report: June 2010 to June 2011. https://oucqa.ca/wp-content/up-

44

Elisa Tortola, Christine Arnold, and Zanele Myles

loads/2012/12/Annual-Report-of-the-Quality-Council-2010-11.pdf Curaj, A., P. Scott, L. Vlasceanu, and L. Wilson. 2012. European Higher Education at the Crossroads: Between the Bologna Process and National Reforms. Dordrecht, The Netherlands: Springer Netherlands. Deller, F., S. Brumwell, and A. MacFarlane. 2015. The Language of Learning Outcomes: Definitions and Assessments. Toronto, ON: Higher Education Quality Council of Ontario. Dobbins, M., C. Knill, and E. M. Vögtle. 2011. “An Analytical Framework for the Cross-Country Comparison of Higher Education Governance.” Higher Education, 62 (5): 665–683. Duan, Y. 2006. “Selecting and Applying Taxonomies for Learning Outcomes: A Nursing Example.” International Journal of Nursing Education Scholarship, 3 (1): 1154. Elam, S. 1972. “Performance-Based Teacher Education: What Is the State of the Art?” Quest, 18 (1): 14–19. European Higher Education Area. n.d. Three-Cycle System. http:// www.ehea.info/page-three-cycle-system Fernie, S., N. Pilcher, and K. Smith. 2014. “The Scottish Credit and Qualifications Framework: What’s Academic Practice Got to Do with It?” European Journal of Education, 49 (2): 233–248. FitzGibbon, J. 2014. Learning Outcomes and Credit Transfer: Examples, Issues, and Possibilities, a BCCAT Special Report. Vancouver, BC: British Columbia Council on Admissions and Transfer. Ford, K. 2014. “Competency-Based Education: History, Opportunities, and Challenges.” UMUC Center for Innovation in Learning and Student Success, 10: 24. Gallagher, M. 2013. “Standards-Based Accountability in Higher Education in Australia.” In Measuring the Value of a Postsecondary Education, edited by K. Norrie and M. C. Lennon, 53–85. Montreal, QC: McGill-Queen’s University Press. Gallavara, G., E. Hreinsson, M. Kajaste, E. Lindesjoo, C. Solvhjelm, A. Sorskar, and M. S. Zadeh. 2008. Learning Outcomes: Common Framework—Different Approaches to Evaluation Learning Outcomes in the Nordic Countries. Joint Nordic Project 2007–2008. Lysaker, Norway: Nordic Quality Assurance Network for Higher Education (NOQA). Galloway, A. 2008. Use of Taxonomies in Assessing Higher-Order Skills. https://www.sqa.org.uk/files_ccc/ResearchReport10_Taxonomies. pdf González, J., and R. Wagenaar, editors. 2003. Tuning Educational Structures in Europe: Final Report Phase One, Bilbao and Groningen. http://

Foundations for Learning Outcomes and Credit Transfer

45

tuningacademy.org/wp-content/uploads/2014/02/TuningEUI_Final-Report_EN.pdf González, J., and M. Yarosh. 2013. “Building Degree Profiles. The Tuning Approach.” Tuning Journal for Higher Education, 1: 37–69. Government of Canada. 2017, October 3. Imbalances Between Labour Demand and Supply (2017–2026). Canadian Occupational Projection System (COPS). http://occupations.esdc.gc.ca/sppc-cops/[email protected]?lid=16%20&fid=1&lang=en Guthrie, H. 2009. Competence and Competency-Based Training: What the Literature Says. National Centre for Vocational Education Research. https://files.eric.ed.gov/fulltext/ED507116.pdf Hinchley, C. and E. Greenlee. 2018, April. The Education Longitudinal Linkage Platform (ELLP): Linking Data for Insights into Student Pathways and Graduate Outcomes. Presented at the ONCAT 7th Annual Student Pathways in Higher Education Conference, Toronto, ON. https:// oncatconference.com/2018/documents/papers/F3EN.pdf Heath, N. 2012. Student Mobility in Canada Across Canadian Jurisdictions. https://pccatweb.org/media/1244/pccat_mainreport_final-en-fulldocument-with-logos.pdf Hodge, S. 2007. “The Origins of Competency-Based Training.” Australian Journal of Adult Learning, 47: 179–209. https://files.eric.ed.gov/ fulltext/EJ797578.pdf Hodge, S., and R. Harris. 2012. “Discipline, Governmentality and 25 Years of Competency-Based Training.” Studies in the Education of Adults, 44 (2): 155–170. Howieson, C., and D. Raffe. 2013. “The Paradox of Scotland: Limited Credit Transfer in a Credit-Based Lifelong Learning System.” Oxford Review of Education, 39 (3): 366–384. Hubball, H., N. Gold, J. Mighty, and J. Britnell. 2007. “Supporting the Implementation of Externally Generated Learning Outcomes and Learning-Centered Curriculum Development: An Integrated Framework.” New Directions for Teaching and Learning, 112: 93–105. Hussey, T., and P. Smith. 2002. “The Trouble with Learning Outcomes.” Active Learning in Higher Education, 3 (3): 220–233. Kennedy, D., Á. Hyland, and N. Ryan. 2009. “Writing and Using Learning Outcomes: A Practical Guide. Article C3.4-1.” EUA Bologna Handbook. Brussels, Belgium: European University Association. Keating, J. 2008, October 31. Qualifications Systems and National Qualifications Frameworks. Paper presented at the Monash University-ACER Centre for the Economics of Education and Training Annual Confer-

46

Elisa Tortola, Christine Arnold, and Zanele Myles

ence, Melbourne, Australia. http://www.monash.edu/education/ non-cms/centres/ceet/docs/conferencepapers/2008jackkeating. pdf Klein-Collins, R. 2012. Competency-Based Degree Programs in the U.S.: Postsecondary Credentials for Measurable Student Learning and Performance. Chicago, IL: Council for Adult and Experiential Learning. Kuh, G., and S. Ikenberry. 2009. More Than You Think, Less Than We Need: Learning Outcomes Assessment in American Higher Education. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). Kuh, G., N. Jankowski, S. Ikenberry, and J. Kinzie. 2014. Knowing What Students Know and Can Do: The Current State of Student Learning Outcomes Assessment in U.S. Colleges and Universities. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). Lennon, M. C. 2014. “Incremental Steps towards a Competency-Based Post-Secondary Education System in Ontario.” Tuning Journal for Higher Education, 2 (1): 59–89. ———. (2016) In Search of Quality: Evaluating the Impact of Learning Outcomes Policies in Higher Education Regulation. PhD Dissertation, University of Toronto. Lennon, M. C., B. Frank, R. Lenton, K. Madsen, A. Omri, and R. Turner. 2014. Tuning: Identifying and Measuring Sector-Based Learning Outcomes in Postsecondary Education. Toronto, ON: Higher Education Quality Council of Ontario. Lennon, M. C., and L. Jonker. 2014. AHELO: The Ontario Experience. Toronto, ON: Higher Education Quality Council of Ontario. Lokhoff, J., B. Wegewijs, K. Durkin, R. Wagenaar, J. Gonzalez, A. K. Isaacs … M. Gobbi. 2010. “A Tuning Guide to Formulating Degree Programme Profiles Including Programme Competences and Programme Learning Outcomes.” Duesto, Spain: Competences in Education and Recognition Project (CoRe)/European Commission. Lucas, U., and R. Mladenovic. 2009. “The Identification of Variation in Students’ Understandings of Disciplinary Concepts: The Application of the SOLO Taxonomy within Introductory Accounting.” Higher Education: The International Journal of Higher Education and Educational Planning, 58 (2): 257–283. Lueg, R., K. Lueg, and O. Lauridsen. 2016. “Aligning Seminars with Bologna Requirements: Reciprocal Peer Tutoring, the SOLO Taxonomy and Deep Learning.” Studies in Higher Education, 41 (9): 1674–1691.

Foundations for Learning Outcomes and Credit Transfer

47

Lumina Foundation. n.d. The Birth and Growth of the Degree Qualifications Profile. http://degreeprofile.org/wp-content/uploads/2016/06/ Birth-Growth-DQP.pdf Malan, S. P. T. 2000. “The ‘New Paradigm’ of Outcomes-Based Education in Perspective.” Journal of Consumer Sciences, 28 (1): 22–28. Meda, L., and A. J. Swart. 2018. “Analysing Learning Outcomes in an Electrical Engineering Curriculum Using Illustrative Verbs Derived from Bloom’s Taxonomy.” European Journal of Engineering Education, 43 (3): 399–412. Ministerial Conference Bologna 1999. 2016, October 26. http://www. ehea.info/cid100210/ministerial-conference-bologna-1999.html Ministry of Colleges and Universities. n.d. “A History of the Program Standards.” http://www.tcu.gov.on.ca/pepg/audiences/colleges/ progstan/history.html Ministry of Training, Colleges and Universities. 2009. Ontario Qualifications Framework (OQF). http://www.tcu.gov.on.ca/pepg/programs/oqf/ Mitchell, A. D., M. Feltham, and L. Trotter. 2013. Mapping the Ontario Advanced Diploma: European and American Outcomes for Business. Toronto, ON: Colleges Ontario. Mulcahy, D. 2000. “Turning the Contradictions of Competence: Competency-Based Training and Beyond.” Journal of Vocational Education and Training, 52 (2): 259–280. Mullin, C. M. 2012, October. Transfer: An Indispensable Part of the Community College Mission (Policy Brief 2012-03PBL). Washington, DC: American Association of Community Colleges. Nodine, T. R. 2016. “How Did We Get Here? A Brief History of Competency-Based Higher Education in the United States.” The Journal of Competency-Based Education, 1 (1): 5–11. Ontario Universities Council on Quality Assurance. 2019, February 15. Quality Assurance Framework. https://oucqa.ca/wp-content/uploads/2019/10/Quality-Assurance-Framework-and-Guide.pdf Organisation for Economic Co-operation and Development (OECD). 2013a. Assessment of Higher Education Learning Outcomes Feasibility Study Report, Volume 1: Design and Implementation. Executive Summary. http://www.oecd.org/education/skills-beyond-school/ AHELO%20FS%20Report%20Volume%201%20Executive%20Summary.pdf ———. 2013b. Assessment of Higher Education Learning Outcomes Feasibility Study Report, Volume 2: Data Analysis and National Experiences. Ex-

48

Elisa Tortola, Christine Arnold, and Zanele Myles

ecutive Summary. http://www.oecd.org/education/skills-beyond-school/ AHELO%20FS%20Report%20-%20Volume%202%20Executive%20Summary.pdf ———. n.d. AHELO Main Study. http://www.oecd.org/education/skills-beyond-school/ahelo-main-study.htm Okoye, K. R. E., and M. O. Isaac. 2015. “Enhancing Technical and Vocational Education and Training (TVET) in Nigeria for Sustainable Development: Competency-Based Training (CBT) Approach.” Journal of Education and Practice, 6: 66–69. https://files.eric.ed.gov/fulltext/ EJ1081283.pdf Pálvölgyi, K. 2017. “Implementation through Innovation: A Literature-Based Analysis of the Tuning Project.” Higher Learning Research Communications, 7 (2). Parker, B., and S. Walters. 2008. “Competency Based Training and National Qualifications Frameworks: Insights from South Africa.” Asia Pacific Education Review, 9: 70–79. https://files.eric.ed.gov/fulltext/ EJ811119.pdf Potter, M. K., and E. Kustra. 2012. A Primer on Learning Outcomes and the SOLO Taxonomy. Centre for Teaching and Learning, University of Windsor. http://www.uwindsor.ca/ctl/sites/uwindsor.ca.ctl/files/ primer-on-learning-outcomes.pdf Raffe, D. 2003. “’Simplicity Itself’: The Creation of the Scottish Credit and Qualifications Framework.” Journal of Education and Work, 16 (3): 239–257. ———. 2007. “Making Haste Slowly: The Evolution of a Unified Qualifications Framework in Scotland.” European Journal of Education, 42 (4): 485–502. ———. 2011. “Are ‘Communications Frameworks’ More Successful? Policy Learning from the Scottish Credit and Qualifications Framework.” Journal of Education and Work, 24 (3–4): 283–302. Raffe, D., J. Gallacher, and N. Toman. 2008. “The Scottish Credit and Qualifications Framework: Lessons for the EQF.” European Journal of Vocational Training, 42 (1): 59–69. Randall, J. 2010. Review of Ontario College Quality Assurance Service: Report of the Independent External Panel. https://www.ocqas.org/ wp-content/uploads/2015/01/jrandall_final_report_mgmt_brd.pdf Tuning Academy. n.d. Tuning Projects. http://tuningacademy.org/projects Shahjahan, R. A., and L. E. Torres. 2013. “A ‘Global Eye’ for Teaching and Learning in Higher Education: A Critical Policy Analysis of the

Foundations for Learning Outcomes and Credit Transfer

49

OECD’s AHELO Study.” Policy Futures in Education, 11 (5): 606–620. Shapiro, D., A. Dundar, F. Huie, P. K. Wakhungu, A. Bhimdiwali, A. Nathan, and H. Youngsik. 2018, July. Transfer and Mobility: A National View of Student Movement in Postsecondary Institutions, Fall 2011 Cohort. Signature Report no. 15. Herndon, VA: National Student Clearinghouse Research Center. https://nscresearchcenter.org/ signaturereport15/ Shea, P. 2019, Sept. 6. “Interstate Passport: Streamlining Transfer for Increased Student Success.” The Beacon, 2 (1). https://myemail.constantcontact.com/NWCCU--The-Beacon-Volume-2-Issue-1-Updated.html?soid=1131614120805&aid=3PZvx9lCuD4 Shipley, D. 1995, Oct. 27. “Transforming Community Colleges Using a Learning Outcomes Approach.” Workshop sponsored by Advanced Education Council of British Columbia & the Centre for Curriculum & Professional Development in Richmond, BC. (ERIC Document Reproduction Service No. 388346). Sin, C. 2014. “Lost in Translation: The Meaning of Learning Outcomes Across National and Institutional Policy Contexts.” Studies in Higher Education, 39 (10): 1823–1837. Stanny, C. J. 2016. “Reevaluating Bloom’s Taxonomy: What Measurable Verbs Can and Cannot Say about Student Learning.” Education Sciences, 6 (4): 37. The Bologna Process and the European Higher Education Area. 2018, September 17. https://ec.europa.eu/education/policy/higher-education/bologna-process_en Tremblay, K., D. Lalancette, and D. Roseveare. 2012. AHELO: Feasibility Study Report: Volume 1: Design and Implementation. Paris, France: Organisation for Economic Co-operation and Development. Tuning CALOHEE. n.d.a. “What is CALOHEE?” https://www.calohee. eu/why-calohee-2/ ———. n.d.b. “Why CALOHEE?” https://www.calohee.eu/why-calohee/ Tuxworth, E. 1994. “Competence-Based Education and Training: Background and Origins. In A Collection of Readings Related to Competency-Based Training, edited by Deakin University, 109–123. Victoria, Australia: Victorian Education Foundation. Vukasovic, M., J. Jungblut, and M. Elken. 2017. “Still the Main Show in Town? Assessing Political Saliency of the Bologna Process across Time and Space.” Studies in Higher Education, 42 (8): 1421–1436. Wagenaar, R. 2018. “What Do We Know—What Should We Know?

50

Elisa Tortola, Christine Arnold, and Zanele Myles

Measuring and Comparing Achievements of Learning in European Higher Education: Initiating the New CALOHEE Approach.” In Assessment of Learning Outcomes in Higher Education, edited by O. Zlatkin-Troitschanskaia, M. Toepper, H. Anand Pant, C. Lautenbach, and C. Kuhn, 169–189. Cham, Switzerland: Springer. Western Association of Schools and Colleges. n.d. Degree Programs: Meaning, Quality, and Integrity of Degrees. https://www.wscuc.org/ resources/handbook-accreditation-2013/part-iii-wasc-quality-assurance/institutional-report/components-institutional-report/3-degree-programs-meaning-quality-and-integrity-degrees Western Interstate Commission for Higher Education (WICHE). 2018. About Us. https://www.wiche.edu/about WICHE. 2017. Interstate Passport: A Network of Colleges and Universities Providing Students Nationwide General Education Transfer. https://interstatepassport.wiche.edu/ Wheelahan, L. 2011. “From Old to New: The Australian Qualifications Framework.” Journal of Education and Work, 24 (3–4): 323–342. https:// www.tandfonline.com/doi/abs/10.1080/13639080.2011.584689?journalCode=cjew20 Young, M. 2008. “Towards a European Qualifications’ Framework: Some Cautionary Observations.” Journal of European Industrial Training, 32 (2/3): 128–137. Zakani, S., B. Frank, R. Turner, and J. Kaupp. 2016. Framework for Transferability between Engineering and Technology Programs. Toronto, ON: Ontario Council on Articulation and Transfer. Zlatkin-Troitschanskaia, O., H. A. Pant, C. Lautenbach, D. Molerov, M. Toepper, and S. Brückner. 2017. Modeling and Measuring Competencies in Higher Education. Wiesbaden, Germany: Springer Fachmedien.

3

The Gulf Between Hope and Practice: South African Experiences with Learning Outcomes and Credit Transfer Stephanie Allais

Introduction This chapter shares some experiences from South Africa in the use of learning outcomes with specific reference to credit accumulation and transfer. South Africa was one of the early adopters of an outcomes-based qualifications framework, and credit transfer and accumulation were one of many aims for this framework. There is very limited systemic research into the effects of the qualifications framework on credit transfer and accumulation. A piece of research commissioned by the South African Qualifications Authority (SAQA), which is the body in charge of the National Qualifications Framework (NQF), found that the outcomes-based system seemed to have led to less, and not more, transfer across the system (Blom, Parker, and Keevy 2007). There is certainly no positive evidence that learning outcomes improved credit transfer and accumulation. But the main evidence is indirect: The outcomes-based qualifications framework itself was not viable, and it collapsed (Allais 2007b, 2011). This chapter, therefore, provides a history and analysis of the broader problems with the design and implementation of the Learning Outcomes, Academic Credit, and Student Mobility, edited by Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon. Montréal and Kingston: McGill-Queen’s University Press, Queen’s Policy Studies Series. © 2020 The School of Policy Studies, Queen’s University at Kingston. All rights reserved.

52

Stephanie Allais

outcomes-based framework in South Africa. The chapter also argues that where the new outcomes-based qualifications were adopted, this is likely to have hampered credit accumulation and transfer from technical and vocational education into higher education. Finally, the chapter suggests that the over-reliance on levels as a mechanism for credit transfer and accumulation has meant a lack of attention to institutional and substantial arrangements. It has also meant that there is no focus in South Africa on where the problems are—and there is very little insight into the extent to which there is even an issue. There is also no place that learners can turn to for arbitration when they are denied credit. The structure of the chapter is as follows: The first section provides an overview of the South African education and training system, in order for readers to locate the policy analysis which follows and make sense of the specific use of some terms. This is followed by a very brief history of how the National Qualifications Framework was initially developed and the substantial changes which have been made to the original model. The second and longest section provides an overview of what the qualifications framework aimed to achieve and why the implementation of it ran into so many problems. This part of the chapter explains the first two arguments in relation to credit accumulation and transfer—that the qualifications framework did not work in its own right, and that because some parts of the system tried to use it and others did not, transfer is likely to have become more difficult. It also argues that understanding why the framework did not work in its own right sheds light on the inherent limitations of an over-reliance on learning outcomes. The final section explores some suggested lessons for other countries that are considering the use of learning outcomes to improve credit transfer and accumulation. History and Context The chapter starts with a brief overview of the South African education and training system, and the contextual challenges facing South Africa (currently, but also in the 1990s when learning outcomes were introduced). The Provision of Education and Training in South Africa South Africa has universal access to primary education, Grades 1 to 7, and is close to achieving universal access to Grades 8 and 9; in other words, for children up to fifteen years of age (Taylor and Shindler 2016).

The Gulf Between Hope and Practice: South African Experiences 53

However, it is well-established that systematic underperformance continues to be a major problem, with even most countries that are much poorer outperforming South African school children in repeated evaluations. After age fifteen, South African children have the option of choosing between continuing with high school or shifting to a Technical and Vocational Education (TVET) college. There are fifty public TVET colleges, all of which are multi-campus institutions, created through mergers of 152 former technical colleges as well as skills centres and some colleges of education. In practice, the vast majority of those who remain in education (by 2013 just over 86 percent of the age cohort) remain in the school system (Taylor and Shindler 2016). The TVET colleges occupy an uneasy position in the education and training system: They offer a set of qualifications that were designed as alternatives to the senior secondary qualification obtained in the school system, which is referred to in South Africa as the matric. However, they function as postsecondary institutions, as the majority of students enrolling at them have finished senior secondary education, in the main with weak educational achievements. Approximately 20 percent of the age cohort manages to access university education. South Africa has twenty-six public universities, which are also multi-campus institutions following a set of mergers in the early 2000s, as well as amalgamations of what in South Africa used to be referred to as technikons, and incorporations of about one hundred teacher-training colleges. The technikons were something like the polytechs of the United Kingdom or the colleges of Canada. They are now either integrated into universities—which are then named comprehensive universities—or they are called universities of technology. Universities offer certificates, diplomas, and degrees at levels 5–10 on the National Qualifications Framework (NQF). South Africa’s university system expanded dramatically in the past thirty years, with an increase of 225 percent since 1986 and 87 percent since 1994 (Taylor and Shindler 2016). This is widely seen to have been at the expense of quality: Throughput and graduation rates are very low, with about one in four students (excluding those in distance education) graduating in the specified time of three years for most undergraduate degrees. Only 35 percent of the total intake including distance education, and 48 percent excluding distance education, graduate within five years, and the estimates are that some 55 percent of the intake never graduate (Council on Higher Education 2015).

54

Stephanie Allais

South Africa currently has two national state departments, under two ministries that are responsible for education and training. The Department of Basic Education is responsible for schools, and the Department of Higher Education and Training is responsible for universities and private higher education institutions, TVET colleges (public and private), community education, and training colleges—incorporating adult education centres—as well as the bodies that are supposed to function as labour market intermediaries and gather information on training needs, i.e., the Sectoral Education and Training Authorities (SETAs). Other ministries and departments are involved in education. Early childhood development is the joint responsibility of the Departments of Health, Social Development, and Basic Education. Research at the university level is also shaped by the Department of Science and Technology. Qualifications are issued or overseen by three statutory bodies: the Council for Higher Education (which oversees the universities), the Quality Council for Trade and Occupations (which issues certificates for trade and occupational qualifications excluding those issued by universities), and Umalusi (schools and adult education). These regulatory bodies are governed by acts of parliament. They are each responsible for what is referred to as a sub-framework of the NQF. The South African Qualifications Authority is responsible for the entire framework, including articulation among the three sub-frameworks. The framework currently has ten levels. As will be discussed below, various aspects of these arrangements are recent, and some of them are likely to change in the coming years. I, therefore, turn to a brief history of the qualifications framework. A Brief History of the South African Qualifications Framework Introduction, Review, and Changes In 1994, South Africa transitioned from the iniquitous system of apartheid to a constitutional democracy. The first education and training legislation passed by the first democratically elected parliament was the South African Qualifications Authority Act, passed in 1995 (Republic of South Africa 1995). Through this act, a National Qualifications Framework was created, and the South African Qualifications Authority was created to develop and implement the framework. The education system at this point in South Africa was the respon-

The Gulf Between Hope and Practice: South African Experiences 55

sibility of the Ministry of Education. Both this ministry and the Ministry of Labour were given oversight of the qualifications framework and the qualifications authority, because the framework was intended to integrate education and training, improve responsiveness to industry needs, improve credit transfer and accumulation across the system, and support the recognition of learning for those who had been denied access to education through apartheid but had gained knowledge and skills at their place of employment. The framework was designed as one of the few completely comprehensive qualifications frameworks in the world. A grid of eight levels and twelve fields was created to encompass the entire education system at all levels and in all sectors. A set of new structures were designed to populate this framework: There were twelve National Standards Bodies, which were stakeholder bodies intended to oversee the creation of qualifications in each of the (then) twelve fields of the qualifications framework. Under these bodies, a large number of ad hoc Standards Generating Bodies (SGBs) were created. The SGBs, composed of experts and interest groups, developed the new outcomes-based qualifications and part qualifications, which were called unit standards (South African Qualifications Authority 2000a, 2000b, 2000d). Unit standards were supposed to capture specific learning outcomes, which learners could then accumulate one at a time and ultimately be awarded a qualification (Allais 2011). The intention was that these new outcomes-based qualifications would populate the (then) eight levels and twelve fields of the framework, and all previous qualifications would gradually be phased out. South Africa would then have a set of national qualifications that specified the learning outcomes required by communities, workplaces, and educational providers, none of which had a direct relationship to a specific educational provider. Any educational provider would be able to apply to a quality assurance body to be accredited and, if they could prove appropriate capacity, would be able to offer any qualification on the national framework (South African Qualifications Authority 2000c). All education institutions would be obliged to redesign their programs on the basis of these specified outcomes, or to develop new programs to meet the requirements of specified outcomes. In other words, the qualifications framework was an entirely new approach to designing qualifications, which was intended to change the roles of educational institutions and educators (Allais 2012b). By June 1998 the first unit standards were registered on the frame-

56

Stephanie Allais

work; more followed in 1999 (South African Qualifications Authority 1999). By 2001 there were sixty-five SGBs registered and developing new outcomes-based unit standards and qualifications. Despite initial wide support for both the qualifications framework and the idea of learning outcomes, difficulties soon emerged (Breier 1998; Cooper 1998; Muller 1998). One of the problems worth noting, as it seems to be common to some other outcomes—or competency-based systems, is that many people felt alienated by the new terminology and structures (Departments of Education and Labour 2002; Lugg 2007). An outcomes-based curriculum had also been implemented at the Grade 1 level in the school system, and it had also met with considerable difficulties (Chisholm 2003; Taylor 2000; Taylor and Vinjevold 1999). The then-minister of education announced in 2000 that the framework would be reviewed. A group referred to as the International Study Group was constituted for this purpose. The report of this group recommended substantial changes to the framework (Departments of Education and Labour 2002). The two departments made no clear decisions based on this review, and a lengthy period of confusion and inaction on the part of government followed (Allais 2011). During this period of ongoing review (2000–2008), the development of the framework continued, based mainly on the original design. By March 2005, 696 new unit standards-based qualifications and 8,208 unit standards had been registered. However, only a tiny number of learners had been awarded these new qualifications, only a tiny percentage of potential education and training providers had been accredited to offer them, and there were many qualifications and unit standards that no provider was accredited to offer (Allais 2007a; South African Qualifications Authority 2005). Although, as already discussed, the intention was for the qualifications framework to replace all existing qualifications, in 1997 the qualifications authority started to register existing qualifications on an “interim basis” (South African Qualifications Authority 1997). They set a deadline by which these qualifications were supposed to expire, but then the deadline was repeatedly extended. By 2005 the qualifications authority started to refer to them as “provider qualifications” (South African Qualifications Authority 2005). This shows that they had accepted that these qualifications would not be phased out and would remain a permanent feature of the qualifications framework. During the review period, a framework for higher education qualifications was released, which had more levels for qualifications after senior secondary education and following this, the framework was changed from eight

The Gulf Between Hope and Practice: South African Experiences 57

to ten levels. By 2005 the National Standards Bodies were disbanded (Allais 2007a). Late in 2008, a set of bills were drafted to create substantial changes to the framework. It was split into the three linked frameworks, and these continue today, each with a quality assurance council (Republic of South Africa 2008a, 2008b, 2008c). In 2009 political change in the ruling African National Congress led to more change for education and training: A new president, Jacob Zuma, created a cabinet with two education ministries, splitting the then-Ministry of Education into the current Ministries of Basic Education and Higher Education and Training. The SETAs and the Quality Council for Trades and Occupations were moved from the Ministry of Labour to the Ministry of Higher Education and Training, requiring still more legislative changes (Allais 2011). Change Continues The new minister of higher education and training released a White Paper in 2013 intended to provide direction for the creation of a single post-school education and training system (Department of Higher Education and Training 2013). A draft national fifteen-year plan to implement this White Paper was released in December 2017. The system has been, and continues to be, in a state of flux and at the time of going to print in early 2020, had not yet been formally approved (Taylor and Shindler 2016). Educational institutions have been merged, renamed, and located under different regulatory bodies. The landscape of regulatory bodies has changed and may change again. The White Paper suggests that further change to the qualifications framework is undesirable, but at the same time advocates for simplification of this framework. The area of most policy contestation is the mid-level qualifications aimed at preparation for work, at the level of senior secondary as well as lower levels of university education. When Hopes and Dreams Hit Implementation Challenges Pinning Hopes on the South African National Qualifications Framework South African educational reformers and policymakers had extremely high expectations about what learning outcomes could achieve. Education policy had been central to apartheid policy; education was therefore at the centre of much of the struggle against apartheid.

58

Stephanie Allais

Apartheid institutionalized social and political fragmentation. The apartheid state had created pseudo-independent states (Bantustans) to keep black South Africans contained in rural areas as well as maintaining the lie of “separate peaceful coexistence” that was the official apartheid narrative. Some of these Bantustans had separate education administrations. Within the official boundaries of South Africa, there were separate education administrations for each of the racial groups created by the apartheid classification system (which are specific to South Africa). Thus, at the time of transition to democracy, South Africa’s education system was “complex and collapsed,” with “high levels of adult and matriculation illiteracy, dysfunctional schools and universities, discredited curricula and illegitimate structures of governance” (Chisholm 2003, 269). Learning outcomes were perceived as a mechanism that could transcend the specificities of different contexts and integrate this fragmentation to create a coherent, nationally integrated education system (Allais 2007a). The idea of integration was also seen as a way of overcoming the dichotomy of academic and everyday knowledge; the divide between general academic and vocational education; the distinction between mental and manual training, and between theory and practice; divisions between natural and social sciences in the curriculum; and the distinction between pure and applied research (Motala 2001). There was a strong call from activists involved in adult basic education for nationally recognized certificates, in a sector that had been highly fragmented and sporadic (Aitchison 2002). The apartheid curriculum was authoritarian and highly prescriptive, with a narrow range of centrally prescribed textbooks that openly pushed the ideological agenda of the state. It was also widely seen as dominated by rote-learning. Chanting was a prevalent pedagogy particularly in black schools. Reformers saw learning outcomes as a tool to place the learner at the centre of the education system, and a mechanism to ensure active learning and critical thinking (Allais 2007a). Learning outcomes would, it was hoped, provide a benchmark against which evaluation could be conducted regardless of whether learners had been in formal or informal learning contexts, and thereby support workers to have their existing skills and knowledge recognized. This was a crucial issue given that black workers were denied promotion because they lacked formal qualifications, but were sometimes doing the same work as their better-paid white counterparts (Webster 1985). The trade union movement in South Africa, and in particular the National

The Gulf Between Hope and Practice: South African Experiences 59

Union of Metalworkers (with considerable engagement from its Australian counterparts) was intensively involved in the ideas that led to the qualifications framework. Their vision included breaking down barriers to education and training and linking the world of education and the world of work (Lugg 2007). An outcomes-based National Qualifications Framework was seen as a mechanism to create a clear relationship between skills, grading, and wages, thus enabling workers to move up a career-path (Industrial Strategy Project 1994, 67). At the same time, learning outcomes were seen as a tool for industry to ensure that they could obtain people with the right skills and knowledge, by involving people from workplaces in the specification of learning outcomes. Learning outcomes were seen as a tool to improve quality, as all education providers would have to meet the same standard. The vision was for learning outcomes to provide clear standards against which course designers could design courses, educators could teach, assessors could assess, and regulators could evaluate the quality of provision and assessment. Learning outcomes would then be a significant tool for learner mobility because they would provide a clear specification of what learners had achieved to institutions of education and to workplaces (Allais 2007a). None of this happened in practice. The following section reflects on some of the problems that led to the collapse of these policies. The Disconnection Between Qualifications and Provision Although the qualifications framework was seen as the vehicle to “completely transform [emphasis added] the disparate education and training system” (South African Qualifications Authority 2005d, 24), in fact, the bulk of the education system never offered the qualifications and unit standards that were developed through its systems and structures. Many qualifications, comprised of many unit standards, in turn, comprised of specific learning outcomes and assessment criteria and range statements—the features familiar to competency-based training systems—were developed, but few were offered, enrolled for, and awarded (Allais 2007a). By 2006, for example, there were awards against eighty-one qualifications, out of the total of 818 new qualifications that had been designed. Ninety-seven qualifications had already expired by November 2006, many before ever having been offered. To some extent, this would be inevitable, given that students take time to move through an education

60

Stephanie Allais

system. There were awards against 986 unit standards out of the 10,214 unit standards which had been developed; 9,228 of them had no course designed against them, or awards made against them (Allais 2007a). What all of these numbers mean is that new outcomes-based qualifications were created and registered on the framework, but there were no educational programs designed against them or assessment conducted against them. Many hundreds of qualifications and unit standards were developed which have never been taught, assessed against, or awarded. The university system continued to offer programs based on the “interim” or “legacy” qualifications: qualifications which, although they were cosmetically changed into outcomes-based formats and registered on the qualifications framework, were institution-specific, and issued in relation to specific programs designed by institutions. These qualifications were not designed through the processes and structures of the qualifications framework, and did not correlate with the central requirements of its model—the specification of outcomes separate from any institutions, learning programs, or educational inputs (Allais 2007a). Learners in secondary schools and most learners in vocational colleges were also issued with certificates against qualifications designed by the Department of Education, as opposed to the qualifications framework structures (Umalusi 2006). Where there was qualification change in the formal education system, specifically for the senior-secondary school and vocational college systems, the new qualifications were not composed of outcome statements and were not designed through the structures and systems of the qualifications framework (Allais 2007a). So, there was the curious dichotomy between a large system of educational provision with associated systems for awarding qualifications that existed without the new qualifications registered on the qualifications framework, and the new systems and structures for designing qualifications as well as new qualifications that existed with very little corresponding educational provision. In other words, the formal education and training system did not comply with the original qualifications or quality assurance model of the qualifications framework and did not use the unit standards-based qualifications designed by the standards generating bodies; instead they continued to design their own qualifications. And while many new qualifications and unit standards were developed by the standards generating bodies, most of these were never used by any providers (Allais 2007a). As discussed above, although the existing university qualifications

The Gulf Between Hope and Practice: South African Experiences 61

were supposed to be scrapped, this never happened, and eventually the qualifications authority stopped even pretending that universities would offer the new national qualifications. Universities went through a cosmetic process of inserting learning outcomes into all their qualifications and learning programs. This was done internally, and not in relation to the outcomes-based qualifications developed through the structures of the qualifications framework. An outcomes-based curriculum was introduced to the school system in primary school, but in a very different way to the qualifications framework systems. The new curriculum did away with disciplinary areas or subjects, through the specification of learning outcomes. Teachers were provided with sixty-six specific outcomes, which were supposed to contain “the specific knowledge, understanding, skills, values and attitudes which should be demonstrated by learners in the context of each learning area” (Spreen 2001, 112), and each outcome, like the unit standards and whole qualifications of the NQF, was associated with assessment criteria. Teachers were then supposed to design curricula by selecting appropriate content. The assumption was that the outcomes themselves would provide sufficient information for teachers. This was an entirely new role for teachers: Curriculum development had never been part of their role. They were not trained for it, nor was time allocated in their working day for it. And this situation was exacerbated because the majority of teachers were very poorly trained, and themselves the product of a weak education system. The problems that predictably (with the clarity of hindsight) unfolded were so substantial that a review was called for very soon after implementation started, and substantial revisions to the curriculum were quickly made—notably the inclusion of lists of specified content. However, this “add-on” approach led to an extremely complex set of curriculum documentation: Instead of a single syllabus document, there were three separate, lengthy documents—a curriculum statement, a learning program guideline, and assessment guidelines, which still assumed a substantial role for teachers in curriculum design, and which sometimes contradicted each other. This was complex, cumbersome, and confusing for teachers (Allais 2010). The association of outcomes-based education with the democratization project in South Africa made people reluctant to confront the problems associated with it, and it remained official policy until late 2009 when the minister of education declared it dead (Motshekga 2009). What underpinned the problems?

62

Stephanie Allais

Narrow Fragmented Learning Outcomes One of the problems which emerged in South Africa is that the unit standards and new outcomes-based qualifications contained such narrow and fragmented learning outcomes that they were essentially unusable by educational providers. Some examples of the qualifications developed include (sourced from the SAQA website, http://www.saqa.org.za): Examples of Level 2 Qualifications National Certificate: Retail Shop Floor Practices Certificate: Reception Operations and Services National Certificate in Steel Tube and Pipe Manufacturing (Seamless Hot-Finished or Welded or Cold-Formed) National Certificate: Air-Conditioning, Refrigeration and Ventilation (also at level three) National Certificate: Bread and Flour Confectionary Baking National Certificate: Contact Centre Support National Certificate: Macadamia Production and De-Husking National Certificate: Victim Empowerment and Support Examples of Level 3 Qualifications National Certificate in Quality Checking of Tyres and Tyre Components National Certificate: Cigarette Filter Rod Production National Certificate: Construction Painting National Certificate: Fast Food Services National Certificate: Food and Beverage Processing: Oil and FatBased Product Processing National Certificate: Jewellery Manufacture in a Mass Production Environment National Certificate: Seed Processing and Packaging Examples of Level 4 Qualifications National Certificate: Community-Based Language Practice Further Education and Training Certificate: Manufacturing and Assembly Operations Supervision Further Education and Training Certificate: Craft Enterprise National Certificate: Food and Beverage Manufacturing Technology: Spray Dried Food Product Technologist

The Gulf Between Hope and Practice: South African Experiences 63

Further Education and Training Certificate: Real Estate Further Education and Training Certificate: Pipeline Operations Further Education and Training Certificate: Victim Empowerment Coordination Further Education and Training Certificate: Community Facilitation in Society and Environment Interactions Examples of Level 5 Qualifications1 National Certificate: Resolving of Crime National Diploma: Animal Production National Certificate: Emergency Services Operations National Certificate: Maintenance of High-Speed Production Processes (Fast-Moving Consumer Goods) National Diploma: Footwear Technology National Certificate: Information Technology: Systems Support These qualifications comprised unit standards containing specific learning outcomes. Some examples of unit standard titles are provided below. Examples of Unit Standard Titles Apply Self-Management Through the Concepts of Positive Self-Esteem and Resiliency Use a Personal Budget to Manage Own Money Demonstrate Knowledge and Understanding of the Characteristics of Burial Societies in South Africa Facilitating [sic] a Numeracy Learning Programme in the Reception Year Interact with People in Textiles Processes Access, Process, Adapt and Use Data from a Wide Range of Texts Develop and Implement the Creative Process Apply Maritime Geography Demonstrate a Basic Understanding of the Physiological Processes in Plant Growth and Development Apply Knowledge of Anatomy, Physiology, and Medical Terminology Relevant to Phlebotomy Demonstrate Knowledge of Environmental Systems and Ecology 1. This is the first level of higher education.

64

Stephanie Allais

Control Traffic Manage One’s Own Development Identify and Describe Learning Processes Attend to and Handle a Domestic Violence Incident Communicate Orally with Relevant Stakeholders in the Recovery of Debt Each of these titles represents many pages of documentation, as there are many specifications that unit standards have to include. Consider below the assessment criteria for one specific outcome, bearing in mind that a specific outcome was a part of a unit standard, and each unit standard had a number of specific outcomes, associated with a number of assessment criteria and other specifications. Assessment Criteria for Specific Outcome “Wash hands effectively” Assessment Criterion 1: Applies soap or hand-washing detergent. Assessment Criterion 2: Explains why soap and water needs to be used for washing hands. Assessment Criterion 3: Lathers hands for a minimum of ten seconds. Assessment Criterion 4: Explains why the lather needs to be on the hands for at least ten seconds. Assessment Criterion 5: Washes palms, backs of hands, between fingers, and under jewelry. Assessment Criterion 6: Explains the areas where most of the dirt and germs can collect on hands. Assessment Criterion 7: Rinses and dry hands. Assessment Criterion 8: Closes taps after use. Assessment Criterion 9: Dries his/her hands thoroughly after washing. Assessment Criterion 10: Explains why hands need to be dried. Assessment Criterion 11: Explains why hands should not be dried on clothing. Assessment Criterion 12: Explains why hand washing is important. Assessment Criterion 13: Gives three examples of when one needs to wash hands. Assessment Criterion 14: Explains the proper hand-washing techniques. Assessment Criterion 15: Gives an example of health problems that can be prevented by hand washing.

The Gulf Between Hope and Practice: South African Experiences 65

This is a somewhat extreme example, but many others can be provided—with numerous assessment criteria for packing groceries at a checkout counter, or for sweeping floors. Many of the unit standards became very narrowly specified, with specific outcomes, assessment criteria, range statements, and so on; there was also an upward spiral of specifications into assessment, moderation, and verification specifications—which could only be conducted by individuals assessed and registered against the assessment, moderation, and verification unit standards. One explanation for the low uptake of the new outcomes-based qualifications is the sheer practical difficulty of working with such a system. The underpinning problem here was an exaggerated idea of the extent to which learning outcomes on their own capture the essence of a required standard, and an exaggerated idea that there is some “sameness” across a wide range of knowledge areas and learning experiences which can be captured and understood through statements of learning outcomes (Allais 2012a). The South African qualifications framework was premised on these assumptions. But when qualification designers started designing outcome statements, it must have become obvious to them that there was room for interpretation. And they clarified through specification, anticipating ways in which people might interpret the outcomes differently. This phenomenon is observed in other systems, in particular in competence-based training systems as well as the original qualifications framework in New Zealand, which start from similar assumptions, and lead to “trivialized” outcomes (Young 1996, 28), which “grow like mould and become unwieldy” (Knight 2001, 373). As Wolf (1995) argues, “The attempt to map out free-standing content and standards leads, again and again, to a never-ending spiral of specification” (1995, 55). It is the combination of the need for and lack of transparency that leads to increasing elaboration of the “standards,” as well as the development of increasingly narrow standards, and explains how in South Africa we ended up with fifteen assessment criteria as well as detailed “range statements” for the specific outcome of washing hands. The alternative is broadly specified learning outcomes, such as: “The learner will be familiar with the main theories of the discipline”

or “The learner will conduct independent research”

66

Stephanie Allais

or “The learner will understand the general tenets of traditional and modern sociological thinking.”2

But educators and policymakers using such broadly specified outcomes have to accept that these will be interpreted differently, as “familiar,” “the main theories,” and “independence,” among others, would all be contested in different educational institutions. Such contestation is not bad unless the system relies on the outcomes as the bearers of sufficient meaning that regulatory bodies can judge courses against them, and unless it is assumed that all courses designed against them are substantively the same. There are two points to draw out of this for credit accumulation and transfer. The first is that an over-reliance on learning outcomes collapses in on itself eventually, which means it cannot assist. The over-specification of outcomes in South Africa, in the quest to “create” standards of common national practice, led to unworkably complex policy documents that, as I have demonstrated elsewhere, undermined the integrity of curricula and the work of educational institutions (Allais 2007b), and it did not, in fact, lead to uniformity of provision, even with the detailed specifications. With colleagues, I conducted research (Allais, King, Bowie, and Marock 2007) that demonstrated that in spite of these highly specified qualifications, different educational providers interpreted the same learning outcomes in widely divergent manners. The second point is that credit accumulation and transfer systems have to factor in other mechanisms and systems, and not rely only on learning outcomes and assume that the specification of learning outcomes will lead to the inevitable sameness of learning offerings. Another problem that occurred in South Africa is that because some parts of the system did attempt to use the new qualifications, movement from those parts of the system into universities may have become even more difficult. It was mainly in vocational and occupational education, community development, skills training, and adult education that attempts were made to use the new qualifications. These are the weakest parts of the South African education and training system: They are low-status in comparison to universities, which are a powerful voice in society, and 2. The discussion below draws from a paper presented to the Higher Education Quality Council in South Africa by Allais and Shalem (2005).

The Gulf Between Hope and Practice: South African Experiences 67

in relation to schools, which are constantly in the public spotlight. In general, they do not have strong well-established curricula and qualifications, or a long history of well-respected qualifications. This led to an even greater gulf between those parts of the education system and university education (Allais 2012b). Wheelahan (2011) makes a similar case in relation to the Australian qualifications framework: She argues that because the vocational education system is competency-based, and the university system is curriculum-based, credit transfer across the two has been difficult. In short, learning outcomes have been seen to lead to a hardening of boundaries instead of making them more porous. Level Descriptors, Learning Outcomes, and Boundary Crossing Another key mechanism in South Africa that was supposed to facilitate credit transfer, and which remains a feature of the qualifications framework, is level descriptors. Level descriptors are broadly specified outcomes or competencies which are supposed to capture what it means to be competent at a particular level of a qualifications framework. Having such a specification is intended to facilitate comparison of qualifications across different institutions, fields, and countries, because they are supposed to demonstrate that different qualifications lead to the same broad outcomes or levels of competence. In South Africa, there are ten categories that level descriptors describe: scope of knowledge, knowledge literacy, method and procedure, problem solving, ethics and professional practice accessing, processing and managing information, producing and communicating information, context and systems, management of learning, accountability. Consider the descriptors for level four of the South African qualifications framework. This level is supposed to be roughly equivalent to the twelfth year of formal schooling (the end of senior secondary education in South Africa). Descriptors for Level Four of the South African Qualifications Framework • Scope of knowledge, in respect of which a learner is able to demonstrate a fundamental knowledge base of the most important areas of one or more fields or disciplines, in addition to the fundamental areas of study and a fundamental understanding of the key terms, rules, concepts, established principles and theories in one or more fields or disciplines;

68

Stephanie Allais

• Knowledge literacy, in respect of which a learner is able to demonstrate an understanding that knowledge in one field can be applied to related fields; • Method and procedure, in respect of which a learner is able to demonstrate an ability to apply essential methods, procedures, and techniques of the field or discipline to a given familiar context, and an ability to motivate a change using relevant evidence; • Problem solving, in respect of which a learner is able to demonstrate an ability to use own knowledge to solve common problems within a familiar context, and an ability to adjust an application of a common solution within relevant parameters to meet the needs of small changes in the problem or operating context with an understanding of the consequences of related actions; • Ethics and professional practice, in respect of which a learner is able to demonstrate an ability to adhere to organizational ethics and a code of conduct, and an ability to understand societal values and ethics; • Accessing, processing and managing information, in respect of which a learner is able to demonstrate a basic ability in gathering relevant information, analysis and evaluation skills, and an ability to apply and carry out actions by interpreting information from text and operational symbols or representations; • Producing and communicating information, in respect of which a learner is able to demonstrate an ability to communicate and present information reliably and accurately in written and in oral or signed form; • Context and systems, in respect of which a learner is able to demonstrate an understanding of the organization or operating environment as a system within a wider context; • Management of learning, in respect of which a learner is able to demonstrate a capacity to take responsibility for own learning within a supervised environment, and a capacity to evaluate own performance against given criteria; and • Accountability, in respect of which a learner is able to demonstrate a capacity to take decisions about and responsibility for actions, and a capacity to take the initiative to address any shortcomings found (sourced from the SAQA website, http://www.saqa.org. za). The process of arriving at these ten categories in South Africa was

The Gulf Between Hope and Practice: South African Experiences 69

long, drawn out, and highly contested because they are by no means self-evident; when international level descriptors are considered, what is clear is how many different categories of descriptors there are, and how these differ across countries (Allais 2014). Once the categories have been agreed on, it is similarly not self-evident what each category means: What, for example, is “knowledge literacy” or “accountability” to someone not involved in the creation of these descriptors? It could also be argued that not all qualifications should enable learners to achieve higher competency levels in all ten categories. For example, “working as part of a group” should not be necessary for all qualifications at any particular level. There could also be considerable debate about what it would mean to have a higher level of competency. For example, South African learners at level one are required to be able to Demonstrate an understanding that knowledge in a particular field develops over a period of time through the efforts of a number of people and often through the synthesis of information from a variety of related sources and fields.

While learners at level four must be able to Demonstrate an understanding that knowledge in one field can be applied to related fields.

Is the former at a “lower level” than the latter? Finally, what each specification means is open to a huge range of interpretation. For example, to “solve common problems within a familiar context” could mean many different things. It is far from clear that, using this as a criterion, people would easily be able to adjudicate between different qualifications and make judgments about whether they are at the same level, and there is no evidence that the level descriptors have been used in this way. Lessons and Implications One of the key arguments to be made from the analysis above is that when too much weight is placed on learning outcomes as a policy mechanism, in the absence of specialized and well-trained communities of trust who share a good understanding of their knowledge fields, the tendency to over-specification is inevitable. If standards are not already widely understood within the community of professionals—the teachers and specialists who are teaching and assessing learners—and

70

Stephanie Allais

trusted by members of the public, outcome statements are open to very different interpretations. In an attempt to contain these differences, outcome developers make outcome statements more and more specific, but in the process, the statements become narrower and longer and, consequently, more difficult for curriculum designers, teachers, and assessors to work with. Is there still a role for learning outcomes? The following section will consider this issue, followed by a brief discussion about some other lessons from credit transfer and accumulation policy in South Africa. A Role for Learning Outcomes? As discussed above, the most positive outcome of the development of learning outcomes in South Africa was that the process of developing learning outcomes brought people together who had never engaged with each other before and played a role in building these vital communities of trust. Although learning outcomes per se are not integral to such a process, they could be argued to provide a useful focus to such discussions. If, therefore, the process is emphasized, and not the product, it could be argued that there may be some role for this approach. Further, it could be argued that a different version of an outcomes-based approach could start with a group of people who already have a shared understanding of what the “outcomes” or “standards” should be, and that the learning outcomes can play a role in structuring decisions and judgments. If they are a useful way of generating discussion and debate, and hence a better and more explicit understanding of standards, they assist only those who participate in the process of creating the standards. The written standards on their own do not assist those who were not part of this process. Some argue that even in such a case they could distort judgments because experts become trapped in technicist processes, which are likely to distort the lens through which they look at courses. This is because when experts have to make judgments against learning outcomes, they are using requirements which are not internal to the tradition of knowledge areas or practices in which they are expert (Shalem, Allais, and Steinberg 2004). This is particularly the case where there is a strong or over-reliance on learning outcomes as a primary benchmark against which learners are assessed or courses are evaluated, accredited, and compared, because the more the reliance on learning outcomes, the more the learning outcomes tend to be over-specified. The complex

The Gulf Between Hope and Practice: South African Experiences 71

and detailed specifications tend to trap experts in a bureaucratic mode of judging against those specifications, instead of drawing on their field of expertise—disciplinary knowledge, or fields of practice—they bring these to bear when making judgments. The implication is that over-specified learning outcomes may not just be unbeneficial, but they may get in the way of good substantive judgments. A different approach could be one that uses outcomes as useful statements of aim that enable course designers to describe their understanding of their field of knowledge. Course designers, and not an external body, design these statements, deriving them specifically from what they are trying to teach. This means that instead of starting with outcomes and designing the content down from them, this approach derives the aims from within the logic and emphasis specific to the content of the knowledge field or practice. This would make the relationship between content and aims a descriptive one and would not require speculations on which content best serves which aim. This approach may be similar to that described by Raffe (2009), in what he refers to as “outcomes-referenced” National Qualifications Frameworks, which use outcomes together with various “input” factors. In other words, outcomes can be useful when it is clear what they are the outcome of—when they are seen as something that is embedded in a body of knowledge or field of practice, and not something that stands aside from it. Using outcomes in this manner could form part of constructive professional discussions between colleagues or across institutions, in which trust is built over time into what each other’s learning programs actually entail. What needs to be noted, however, is that if outcomes are used in this way, then most of the claims made for them—that they cross borders, enable validation/recognition of prior learning, enable comparisons of different learning programs—no longer hold, because they are embedded in particular contexts. The claims made about what learning outcomes on their own, separate from curricula or learning programs, can achieve, rest on the idea that they have a meaning disembedded from their context. This raises the question: What is a context—how wide, how narrow, how specific? When I argue that learning outcomes are embedded in a context, I am referring primarily to a body of knowledge, skills and expertise, or a field of learning. When there is a high degree of cohesion in a disciplinary or professional community in terms of the knowledge and expertise required, it is easier for members of that community to

72

Stephanie Allais

speak to each other using a shorthand—like learning outcomes. There is an implicit understanding of what the learning outcome relates to, and the knowledge and expertise in which it is embedded. This could be a large group or a small group, depending on the discipline as well as on the tightness of the community of practice. In fields where disciplines have fairly hierarchical or widely agreed bodies of knowledge, or where there are very strong and organized professional communities, it is more likely that they can use learning outcomes as a common language. When doctors say a learning outcome of the medical course must be detailed insight into anatomy, they are likely to have a reasonably shared sense of what such an understanding entails, and how much anatomy needs to be learnt by the students. There is nothing in the outcome itself that discloses the information—a lay person would have no idea, just from reading “detailed insight into anatomy,” how detailed exactly this should be, and how much anatomical knowledge must be mastered. Where a strongly held communal understanding of the relevant body of knowledge or field of expertise is lacking, the learning outcomes do not in themselves build that common language and can detract from it. This is why strong institutional relationships are beneficial to credit transfer systems—because institutions build up, over time, insight into each other’s curricula, and develop a sense of what they can expect learners to have mastered when they accept a learner from a neighbouring—or far away—institution. And, as argued above, when learning outcomes are discussed in relation to curricula—in other words, as something like aims or objectives have traditionally been discussed in relation to curricula—they can be a useful part of a discussion, and a process of building trust between two institutions in each other’s curricula. It is ironic to note that movement possibilities across national and state borders are often poor for medical practitioners—but this is due to political and economic reasons, related to national priorities of governments, or occupations themselves setting up barriers to protect their income levels. Other Lessons from South Africa South Africa has just passed a new policy on articulation (Republic of South Africa 2017). The policy has nine principles, spells out roles and responsibilities of the key regulatory bodies as well as education pro-

The Gulf Between Hope and Practice: South African Experiences 73

viders, and requires that “new qualifications and part qualifications submitted to SAQA for registration contain clear articulation routes, and statements elaborating articulation in a clear and practical way. Where articulation is not possible reasons must be provided” (Republic of South Africa 2017, 9). One striking feature of this policy is that, like its predecessors, it makes no reference to any attempt to quantify the problem: In other words, what has never been established in South Africa is the extent to which credit accumulation and transfer are problems, and where the major problems are. The area seems to be dominated by anecdote, and a feeling that there is a problem. For example, it is commonly cited that students from the National Certificate Vocational, a qualification offered in TVET colleges that is officially equivalent to the senior secondary certificate, are denied access to university. But what is not clear is the percentage of these students who are denied access, relative to the very many students with a senior secondary certificate who are also denied access, and what the grounds are for them not getting into university. In other words, is the underlying problem access from the college certificate, or is it that there are too few spaces in university education? There is research currently underway to investigate models of good practice, but the extent of the problem remains unquantified. Another problem that has dogged this area is that there is nowhere for students to turn if they feel they have been unfairly denied credit accumulation or transfer. Consider two examples of recent cases. A student enrolled for a BA at one of South Africa’s most prestigious universities. He had completed about 85 percent of a B. Juris, a discontinued legal degree, from a far less prestigious university. The prestigious university would not recognize a single credit, and he started his BA from scratch. Was this a valid decision, based on an analysis that the credits the student had accumulated were either irrelevant to a BA or of an inappropriate standard for a degree from the prestigious university? Or was it prejudice on behalf of a university that can set its own terms? The student does not know, and the perception that credit accumulation and transfer is impossible remains. Similarly, a student at another prestigious university wanted to transfer within the same university from an M.Phil in Research to an MA in Monitoring and Evaluation. She wanted to transfer at the point at which she had completed her course work and was about to start her dissertation; the person who was to supervise her had started the new MA in Monitoring and Evaluation. But the second program would not

74

Stephanie Allais

recognize her course work as credit toward the MA, and she was told to start again. It is possible that an underlying problem here is the ways in which university departments have been turned into cost centres, which are under pressure to gain income and therefore tend to prefer students to do additional coursework. In both instances, students had no place to appeal to outside of the receiving university/department. A central institution that could receive complaints and reach substantive judgments on them, even if it did not have legal power to overturn the decision of a university, could have two useful effects: It could increase policymakers’ knowledge of where specific problems are and how much of a problem they are, and it could make public where decisions seem to be substantively unfair, thus improving insight into the issue for policymakers, institutions, and students. In short, problems with credit transfer tend, internationally, to be very poorly quantified, and policy should start from attempting to quantify the degree of the problem and key points where there seem to be problems. Finally, there are two additional points from South Africa that require brief mention. First, our outcomes-based qualifications framework introduced an enormous amount of policy complexity, and a raft of new institutions, in a context in which most institutions were weak. This added to the difficulty of implementation. Second, our qualifications policy was very heavily borrowed—modelled on the Australian competence-based training system, which was in turn modelled on the British National Vocational Qualifications, although we expanded the model to the entire education and training system. The key message to reiterate, therefore, is: Proceed carefully, and with many other pieces of the puzzle in place. When thinking about credit transfer, learning outcomes cannot be more than a rope to assist over a small fissure. When they are expected to be a bridge over a gorge, they are likely to turn the gorge into a canyon. References Aitchison, J. 2002. “Postcards to the New Prisoners in the Global Classroom.” (Inaugural lecture presented at the University of Natal, Pietermaritzburg, South Africa.) Allais, S. 2007a. “The Rise and Fall of the NQF: A Critical Analysis of the South African National Qualifications Framework” (doctoral dissertation. University of the Witwatersrand, Johannesburg, South Africa.)

The Gulf Between Hope and Practice: South African Experiences 75

———. 2007b. “Why the South African NQF Failed: Lessons for Countries Wanting to Introduce National Qualifications Frameworks. European Journal of Education, 42 (4): 523–547. ———. 2010. “Outcomes-Based Education: Understanding What Went Wrong.” In Retrieving Teaching: Critical Issues in Curriculum, Pedagogy and Learning, edited by Y. Shalem, and S. Pendlebury, 27–40. Cape Town, South Africa: Juta. ———. 2011. “The Changing Faces of the South African National Qualifications Framework.” Journal of Education and Work, 24 (3–4): 343–358. ———. 2012a. “Claims versus Practicalities: Lessons about Using Learning Outcomes.” Journal of Education and Work, 25 (3): 331–334. ———. 2012b. “Why Solving On-Going Problems with the NQF Matters.” In Shaping the Future of South Africa’s Youth: Rethinking PostSchool Education and Skills Training, edited by H. Perold, N. Cloete, and J. Papier, 9–28. Cape Town, South Africa: African Minds. ———. 2014. Selling Out Education: National Qualifications Frameworks and the Neglect of Knowledge. Rotterdam, The Netherlands: Sense. Allais, S., M. King, L. Bowie, and C. Marock. 2007. The “F” Word: The Quality of the “Fundamental” Component of Qualifications in General and Further Education and Training. Pretoria, South Africa: Umalusi. Allais, S., and Y. Shalem. 2005, January. “Why the Case for Outcomes-Based Standards in Quality Assurance Is a Circular Argument.” (Presented at the Higher Education Quality Council standards setting seminar in Pretoria, South Africa.) Blom, R., B. Parker, and J. Keevy. 2007. The Recognition of Non-Formal and Informal Learning in South Africa: Country Background Report Prepared for the OECD Thematic Review on Recognition of Non-Formal and Informal Learning. Hatfield, South Africa: SAQA. Breier, M. 1998. “Three Questions for the Designers of the NQF.” In Reconstruction, Development, and the National Qualifications Framework, edited by K. Pampallis. Johannesburg, South Africa: Centre for Education Policy Development and Education Policy Unit. Council on Higher Education. 2015. VitalStats Public Higher Education 2013. Pretoria, South Africa: CHE. Chisholm, L. 2003. “The State of Curriculum Reform in South Africa: The Issue of Curriculum 2005.” In State of the Nation: South Africa 2003–2004, edited by J. Daniel, A. Habib, and R. Southall, 268–289. Pretoria, South Africa: HSRC Press. Cooper, L. 1998. “The Implications of the National Qualifications Framework for Emancipatory Education in South Africa.” In Reconstruction,

76

Stephanie Allais

Development, and the National Qualifications Framework, edited by K. Pampallis. Johannesburg, South Africa: Centre for Education Policy Development and Education Policy Unit. Departments of Education and Labour. 2002. Report of the Study Team on the Implementation of the National Qualifications Framework. Pretoria, South Africa: Departments of Education and Labour. Department of Higher Education and Training. 2013. White Paper for Post-School Education and Training: Building an Expanded, Effective and Integrated Post-School System. Pretoria, South Africa: Department of Higher Education and Training. Industrial Strategy Project. 1994. “Industrial Strategy for South Africa: Recommendations of the ISP.” South African Labour Bulletin, 18 (1): 48–75. Knight, P. 2001. “Complexity and Curriculum: A Process Approach to Curriculum-Making.” Teaching in Higher Education, 6 (3): 369–381. Lugg, R. 2007. Making Different Equal? Social Practices of Policy-Making and the National Qualifications Framework in South Africa between 1985 and 2003. London, UK: University of London. Motala, E. 2001. “Policy Analysis Capacity in South Africa.” In Implementing Education Policies: The South African Experience, edited by Y. Sayed and J. Jansen, 240–249. Cape Town, South Africa: University of Cape Town Press. Motshekga, A. 2009. Statement by Minister of Basic Education, Angie Motshekga, on curriculum review process, National Assembly. Accessed November 20, 2009. http://www.search.gov.za/info/ Muller, J. 1998. “NQF and Outcomes-Based Education: Pedagogic Models and Hard Choices.” In Reconstruction, Development, and the National Qualifications Framework, edited by K. Pampallis. Johannesburg, South Africa: Centre for Education Policy Development and Education Policy Unit. Raffe, D. 2009. “Towards a Dynamic Model of NQFs.” In Researching NQFs: Some Conceptual Issues, edited by S. Allais, D. Raffe, and M. Young, 22–43 (Employment Sector Working Paper No. 44). Geneva, Switzerland: ILO. Republic of South Africa. 1995. South African Qualifications Authority Act. Pretoria, South Africa: Government Gazette. ———. 2008a. General and Further Education and Training Quality Assurance Amendment Act (Vol. 523[31785]). Pretoria, South Africa: Government Gazette. ———. 2008b. National Qualifications Framework Act (Vol. 524). Pretoria,

The Gulf Between Hope and Practice: South African Experiences 77

South Africa: Government Gazette. ———. 2008c. Skills Development Amendment Act (Vol. 521[31666]). Pretoria, South Africa: Government Gazette. ———. 2017. Articulation Policy for the Post-School Education and Training System (Government Gazette number 40545 No. 12). Pretoria, South Africa: Government Printer. South African Qualifications Authority. 1997. SAQA Bulletin, 1 (1). ———. 1999. Annual Report: 1998/1999. Pretoria, South Africa: SAQA. ———. 2000a. SGB Manual Part One. Pretoria, South Africa: SAQA. ———. 2000b. SGB Manual Part Two. Pretoria, South Africa: SAQA. ———. 2000c. The National Qualifications Framework and Quality Assurance. Pretoria: The South African Qualifications Authority. ———. 2000d. The National Qualifications Framework and Standards Setting. Pretoria: SAQA. ———. 2005. Annual Report: 2004/2005. Pretoria: SAQA. Shalem, Y., S. Allais, and C. Steinberg. 2004. “Outcomes-Based Quality Assurance: What Do We Have to Lose? Journal of Education, (34): 51–77. Spreen, C. 2001. “Globalization and Education Policy Borrowing: Mapping Out Outcomes-Based Education in South Africa.” (PhD thesis, Teachers College, Columbia University.) Taylor, N. 2000, November. “‘Anything but Knowledge’: The Case of the Undisciplined Curriculum.” (Paper presented to the Designing Education for the Learning Society conference, Enschede, The Netherlands.) Taylor, N., and J. Shindler. 2016. Education Sector Landscape Mapping South Africa. Johannesburg, South Africa: Joint Education Services. Taylor, N., and P. Vinjevold. 1999. Getting Learning Right: Report of the President’s Education Initiative. Johannesburg, South Africa: Joint Education Trust. Umalusi. 2006. Umalusi’s Role in Monitoring Standards of Qualifications and Curricula. Report of an Umalusi Workshop. Pretoria, South Africa: Umalusi. Webster, E. 1985. Contesting Skill-Deskilling and the Transition to Machinofacture. Cast in a Racial Mould: Labour Process and Trade Unionism in the Foundries. Johanesburg, South Africa: Raven Press. Wheelahan, L. 2011. “From Old to New: The Australian Qualifications Framework. Journal of Education and Work, 24 (3–4): 323–342. Wolf, A. 1995. Competence-Based Assessment. Edited by H. Torrance. Buckingham, UK: Open University Press.

78

Stephanie Allais

Young, M. 1996. “The Outcomes Approach to Education and Training: Theoretical Grounding and an International Perspective. In IMWG Conference on the National Qualifications Framework. Proceedings, edited by T. Coombe, 22–40. Johannesburg, South Africa: Technikon SA Conference Centre: HSRC on behalf of the Inter-Ministerial Working Group on Education and Training.

4

Foundations and Reform Measures: Credit Transfer and Learning Outcomes Policy and Practice in the United States Debra Bragg

Introduction Within this work, the transfer of students between community colleges and universities in the United States is detailed. This chapter begins with a brief historical perspective on transfer, where I explain the national focus on higher education following World War II that emphasized community college education, including reinforcing the importance of baccalaureate completion. This discussion lays a foundation for the main focus of my chapter, transfer reforms in the United States. Predominantly focused on increased efficiency, transfer reforms emphasizing learning outcomes are limited in the United States’ context; however, my chapter recognizes the importance of emerging transfer initiatives that focus on inequitable learning outcomes. I close the chapter by highlighting promising changes occurring in the United States that may inform the Canadian context. Origin and Evolution of Credit Transfer in the United States The beginning of transfer between colleges and universities in the United States is attributed to the formation of “junior” colleges, which is Learning Outcomes, Academic Credit, and Student Mobility, edited by Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon. Montréal and Kingston: McGill-Queen’s University Press, Queen’s Policy Studies Series. © 2020 The School of Policy Studies, Queen’s University at Kingston. All rights reserved.

80

Debra Bragg

an idea that emerged in the late 1800s when the question of where to begin and end high school was debated. Prior to this time, few people attended school beyond the elementary level, with most ending their formal education in eighth grade at the age of thirteen. Thus, for most Americans, college was not a consideration or even an interest. Rather, it was seen as a luxury for higher-income individuals who were also “determined to have the talents to be able to attain a baccalaureate degree or advanced degree” (Gilbert and Heller 2013, 420). College admissions were overtly discriminatory in enrolling primarily White males, with limited opportunities for other groups. However, following the expansion of colleges and universities under the Land Grant Act of 1862—called the first Morrill Act—the second Morrill Act was the genesis of today’s historically Black colleges and universities. The expansion of college education opportunities allowed for an increased focus on enrolling student populations previously denied access (see, for example, Thelin 2004). By the early 1900s, American university presidents were attempting to find ways to strengthen their new, fledgling institutions. Some wrote with admiration of Germany’s universities, which offered undergraduate instruction as well as graduate education and research, but this model would require substantial numbers of students. How would they grow their enrollments to achieve this lofty vision? Notably, William Rainey Harper, president of the University of Chicago, reached out to local high school administrators with proposals to create college admission options for their students. One of these superintendents resided in Joliet, Illinois, where an arrangement was forged between the University of Chicago and the Joliet school district for the high school to offer coursework that would transfer to the university for college credit. Thus, Joliet Junior College (JJC), the first formally known junior college in the nation, was born in 1901. Following suit, relationships were created between JJC and other Midwest universities, including the University of Illinois and Northwestern University, though the number of students matriculating to these universities was small, as was true at the University of Chicago. Despite the modest enrollments, higher education leaders in other parts of the country soon followed suit, with leaders of the University of California at Berkeley and Stanford University advocating for junior colleges to be formed in that state (Cohen, Brawer, and Kisker 2014). Looking at these early years, some scholars have argued that transfer between colleges and universities would not have emerged without the

Foundations and Reform Measures 81

advocacy and support of elite university leaders, but they also acknowledge that this support came with a price (see, for example, Brint and Karabel 1989). University personnel decided what transfer coursework was worthy of credit and who would be admitted, ultimately determining how transfer would operate in the United States higher education context. Whereas junior colleges would recruit students and prepare them for upper-division instruction, it was the prerogative of the universities to decide who would be admitted and how much credit would be granted from lower-division studies. Thus, the relationship between community colleges and universities began with the balance of power tipped in favour of “senior” institutions that would determine access to the most prized undergraduate degree, the baccalaureate. By the mid-twentieth century, enrollments in junior colleges had grown substantially from their beginnings, due in part to the influx of students into technical coursework to prepare workers for World War II and further growth after the war. The 1947 Commission on Higher Education, appointed by President Harry Truman—one of the few modern United States presidents without a college degree—was stacked with education and political leaders who favoured educating the United States citizenry to levels higher than high school. As part of a progressive educational agenda, the Commission recommended that “junior” colleges be renamed “community” colleges to reflect a more comprehensive mission thought necessary to benefit a larger proportion of the nation’s population. The Commission prescribed that by 1960 college enrollment should reach nearly half of college-age individuals, stating, “at least 49 percent of the population had the ability to complete a minimum of two years of higher education, while 33 percent had the ability to continue to an advanced liberal degree or specialized professional education” (cited in Gilbert and Heller 2013, 420). The Commission also recommended that higher education rectify inequities in college enrollment by overcoming discriminatory practices in admissions and enrollment by race, ethnicity, gender, religion, and country of origin. Community colleges, with their emphasis on transfer as well as vocational, community, and adult education, were encouraged to adopt an “open access mission” that could play a democratizing role for higher education. The Truman Commission’s recommendations were mostly ignored as the United States entered a conservative political era focused on building the post–World War II economy; however, by the mid-1960s, the nation began to shift to a more favourable political landscape for

82

Debra Bragg

social, cultural, and educational reforms, including in the Civil Rights Movement. The Higher Education Act of 1965 was passed at this point, formally recognizing that education beyond high school is a national priority for all Americans. The new federal law did not dictate that states adopt community colleges; however, it authorized federal financial aid for students attending community colleges, which incentivized states to expand this form of higher education institution. The response of states was swift and widespread in that most states adopted legislation to create or expand community colleges having an open-access mission and comprehensive curriculum, as envisioned by the Truman Commission. Community colleges grew at a rate of nearly one per week from the mid- to late-1960s (Cohen et al. 2014), with transfer or vocational education that was treated as a separate track to prepare students to enter employment rather than matriculate to the university level (Bragg 2012). An important but unexpected outgrowth of the comprehensive community college was a decline in transfer from the community college to the university during the 1970s (Anderson, Sun, and Alfonso 2006). While many factors were operating at the time, expansion of vocational education was a contributor, which was accompanied by public debate about the value of the baccalaureate degree. Concern about the quality of all public education in the United States continued into the 1980s when President Reagan formed the National Commission on Excellence in Education (1983), which authored A Nation at Risk and called for increasing the academic preparation of compulsory-age students for higher education as a means of improving the economy. Also, at this time, Parnell (1985), president of the American Association of Community and Junior Colleges, sought federal policy to prepare more students to transition from high school to community colleges, specifically promoting the idea that postsecondary education should be an option for the majority of high school students. Parnell (1985) believed that an approach to education called technical preparation (tech prep), which integrates general education with vocational education beginning in high school and extending to community colleges, would enhance students’ educational and economic opportunities. First referenced in federal legislation in 1984, but first authorized with federal appropriations in 1990, tech prep reaffirmed the vocational track as distinct from transfer education in community colleges. The growth of community college enrollments in the 1980s was substantial, resulting in transfer becoming more fully endorsed by state

Foundations and Reform Measures 83

systems of higher education such that states began to “negotiate the requirements for students’ movement from institution to institution” (Anderson et al. 2006, 263). The state role in promoting institution-to-institution agreements expanded, with some states advocating for statewide articulation agreements that would increase transfer students’ attainment of baccalaureate degrees. The adoption of statewide agreements grew through the 1990s and into the 2000s, with some states legislating rules for operating transfer in state statutes (see Table 4.1, next three pages). Seeking to understand the impact of statewide agreements on baccalaureate attainment, Anderson et al. (2006) conducted one of only a few quantitative studies on statewide articulation agreements and baccalaureate attainment by comparing the attainment rates for students enrolling in states with articulation agreements to those enrolling in states without them. Results of Anderson et al. (2006) were undoubtedly disappointing for transfer advocates who promoted statewide articulation agreements to improve baccalaureate attainment, in that the research showed statewide agreements had no statistical effect on attainment after controlling for individual characteristics known to explain variation in baccalaureate completion. In discussing this result, Anderson et al. (2006) referenced an earlier 50-state study of transfer and articulation by Ignash and Townsend (2000), whose research supported statewide articulation agreements but recognized other transfer-related policies and practices needed to improve baccalaureate attainment. In this regard, Anderson et al. (2006) endorsed the previous recommendation of Ignash and Townsend (2000), who suggested that, in order for state-level transfer systems to be effective, states need to implement a comprehensive set of policies and practices that do not rely solely on administrative rules. For example, they recommended faculty engagement in transfer decision-making, core curriculum guaranteed to count toward the baccalaureate degree, and counselling and advisement services focusing on student success. In a more recent study of the impact of statewide articulation agreements and related transfer policies on baccalaureate attainment, LaSota and Zumeta (2016) examined state-level transfer and articulation policy using a rubric created by Smith (2010) that categorizes transfer and articulation policy strategies according to credit transfer rules, common course numbering, formal transfer guides, and so forth. This multivariate analysis found a positive impact on baccalaureate attainment in states having transfer guides and common course numbering, but

Statewide Transfer Policy*

Yes

No

Yes

Yes

Yes

Yes

Yes

No

Yes

No

No

No

Yes

No

State

Alabama

Alaska

Arizona

Arkansas

California

Colorado

Connecticut

Delaware

Florida

Georgia

Hawaii

Idaho

Illinois

Mississippi

No

Yes

No

Yes

Yes

Yes

No

No

Yes

Yes

Yes

Yes

Yes

Yes

Statewide Articulation Guide*

State-by-State Transfer and Articulation Reform

Table 4.1

Yes

Yes

Yes

Yes

Yes

Yes

No

Yes

Yes

Yes

Yes

Yes

No

Yes

Transfer Data Reporting*

Yes

No

Yes

Yes

Yes

Yes

No

No

Yes

Yes

Yes

Yes

No

Yes

Transferable Core of Lower-Division Courses**

No

No

No

No

No

Yes

No

No

Yes

No

Yes

Yes

No

No

Statewide Common Course Numbering**

Yes

Yes

Yes

Yes

Yes

Yes

No

No

Yes

No

Yes

No

No

Yes

Statewide Guaranteed Transfer of Associate’s Degree**

Yes continued

No

No

Yes

No

Yes

No

No

Yes

No

No

No

No

Yes

Statewide Reverse Credit**

84 Debra Bragg

Statewide Transfer Policy*

No

No

Yes

Yes

Yes

No

Yes

No

Yes

Yes

Yes

Yes

Yes

State

Missouri

Montana

Nebraska

Nevada

New Hampshire

New Jersey

New Mexico

New York

North Carolina

North Dakota

Ohio

Oklahoma

Oregon

No

Yes

No

Yes

Yes

No

Yes

Yes

Yes

Yes

Yes

No

No

Statewide Articulation Guide*

Yes

Yes

Yes

No

Yes

Yes

No

Yes

No

Yes

No

Yes

Yes

Transfer Data Reporting*

State-by-State Transfer and Articulation Reform, continued

Table 4.1

Yes

Yes

Yes

Yes

Yes

No

Yes

Yes

Yes

Yes

No

Yes

Yes

Transferable Core of Lower-Division Courses**

No

No

No

Yes

No

No

Yes

No

No

Yes

No

No

No

Statewide Common Course Numbering**

Yes

Yes

Yes

No

Yes

No

Yes

Yes

Yes

Yes

No

No

Yes

Statewide Guaranteed Transfer of Associate’s Degree**

continued

Yes

No

No

No

No

No

No

No

No

Yes

No

No

Yes

Statewide Reverse Credit**

Foundations and Reform Measures 85

Yes

Yes

Yes

Yes

Yes

Yes

No

Yes

Yes

Yes

No

Yes

Pennsylvania

Rhode Island

South Carolina

South Dakota

Tennessee

Texas

Utah

Vermont

Virginia

Washington

West Virginia

Wisconsin

Wyoming

Yes

Yes

No

No

Yes

No

No

No

No

No

Yes

Yes

Yes

Statewide Articulation Guide*

Yes

Yes

No

No

Yes

No

No

Yes

Yes

No

Yes

Yes

No

Transfer Data Reporting*

** Anderson and Millard (2014).

Yes

State

* Smith (2010).

Statewide Transfer Policy*

State-by-State Transfer and Articulation Reform, continued

Table 4.1

No

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

Yes

No

Yes

Transferable Core of Lower-Division Courses**

Yes

Yes

No

No

No

No

Yes

Yes

Yes

Yes

No

No

No

Statewide Common Course Numbering**

Yes

Yes

Yes

Yes

Yes

No

Yes

No

Yes

Yes

No

No

Yes

Statewide Guaranteed Transfer of Associate’s Degree**

No

No

No

Yes

No

No

No

Yes

No

No

No

Yes

No

Statewide Reverse Credit**

86 Debra Bragg

Foundations and Reform Measures 87

ther components of state transfer systems had little or no effect. Some o critical aspects of this study’s findings seemed to support the recommendation of Ignash and Townsend (2000) that statewide articulation agreements are unlikely to produce positive results without the implementation of transfer policies and practices at the institutional level, where faculty and students reside and experience learning. Table 4.1 compares transfer and articulation policies and practices implemented by states as of 2010 and again in 2014 (Anderson and Millard 2014; Smith 2010). By comparing these results, we can see that state transfer policies and practices continue to evolve, with more states adopting common course numbering, core lower-division courses, and reverse credit transfer. These results are also consistent with a recent study of state transfer and articulation policy research in ten states. In this analysis, Hodara et al. (2016) gathered data on transfer policy and practice and concluded that a comprehensive approach to transfer and articulation yields more positive outcomes for transfer students, including transfer credit and baccalaureate attainment outcomes. These scholars recommended that states increase their investment in researching the impact of transfer policies, practices, and student outcomes to improve state transfer systems. Taken together, these studies point to the uniquely important positioning of transfer at the state level. Contrary to other countries, where the federal government plays a major role in governing transfer, the lion’s share of responsibility for transfer occurs within the jurisdiction of the fifty states. Efforts to improve transfer mostly focus on reforms coordinated by state education agencies, including agencies that have fiscal responsibility for flowing state funds to colleges and universities. However, even with this fiduciary responsibility, state agencies tend to have weak regulatory authority over transfer; thus, institutions respond with varying levels of compliance. Filling this gap, often community colleges and universities create memoranda of agreements to guide student transfer relative to particular baccalaureate programs. These program-to-program agreements are beneficial when institutions support them but have varied support and impact when they are not endorsed fully by campus leaders. Current Status and Context for Credit Transfer During the eight years of President Obama’s administration, states were encouraged to increase the percentage of Americans with high-quality

88

Debra Bragg

credentials (mostly degrees but also certificates) to 60 percent of the national population by 2020, raising the mid-twentieth century benchmark set by the 1947 Truman Commission by about 10 percent. Advocating for this goal to come to fruition, the Bill and Melinda Gates Foundation, the Lumina Foundation, the Kresge Foundation, and other philanthropic organizations encouraged the federal and state governments to adopt this particular goal, or a similar goal, by 2020 or 2025 (Lumina Foundation, n.d.). Over the twenty-year period from 1995 to 2015, the nation’s college completion rate has increased from 33 percent to 46 percent when associate’s degrees and baccalaureate degrees are included in the count (Ryan and Bauman 2016). Some of this change is attributable to the establishment of state completion goals to increase the proportion of citizens with a college education. Importantly, transfer reforms are included in a long list of changes that have contributed to improving the nation’s college completion rate despite the lack of an explicit federal policy dedicated to transfer. Over the last two decades, philanthropic organizations have been influential in supporting a wide range of reforms to higher education, especially community college education. Comprising nearly 40 percent of higher education undergraduate students in the United States (National Center for Education Statistics 2016), community colleges have been the focus of numerous efforts to improve credential attainment and college completion at the community college level. Seeking to fill gaps in federal and state support, philanthropies have focused their support on these institutions, which enroll the most diverse student populations in United States higher education. With transfer viewed as an extension of the community college education, it is logical that philanthropies have viewed transfer as an extension of two-year postsecondary education. Though not as extensive an investment as other community college-focused reforms of developmental education and online education, additional resources dedicated to transfer, including providing investments to clearer and more streamlined transfer pathways, has been a priority (see, for example, Bailey, Jaggars, and Jenkins 2015). It is also important to mention that, while college enrollment is relatively high (and college completion is also increasing), approximately thirty-five million citizens aged twenty-five and older have some college, but no degree, which equates to 17 percent of American adults (Ryan and Bauman 2016); the rate of college completion is lower than other developed nations. The reasons so many students accumulate college credits but no degree are complex, including the relative ease

Foundations and Reform Measures 89

of accessing higher education through community colleges, the high mobility of college students within states and across the United States, and the challenges that students face paying for college due to escalating tuition costs (see, for example, Sanchez 2014). Looking at student mobility patterns alone, recent research shows a high rate of enrollment at multiple colleges and universities, creating the necessity to transfer credits from one school to another. Shapiro, Dundar, Wakhungu, Yuan, and Harrell (2015) reported that, among students who started postsecondary education in 2008, 45 percent of students who transferred did so more than once. These findings confirm what de los Santos and Wright (1989) found decades earlier, in that students swirl among multiple higher education institutions, with the students who move back and forth recognized euphemistically as swirlers. Student mobility is an important concern as students attempt to move college credits from one institution to another, accumulating enough credit to finish a college degree. Increased pressure on institutions to accept credits via transfer further complicates the already complex historical relationship between community colleges and universities. As noted, the incidence of transfer between higher education institutions, mostly community college and universities, is extensive. Of the 20.2 million students who attend public or private degree-granting higher education institutions, 6.7 million enroll in institutions designated as two-year institutions (National Center for Education Statistics 2015). Of this total group of college enrollees, 33 percent of United States college students transfer from one institution to another or are co-enrolled at more than one institution. For students who begin higher education at a public community college, institutions that have the most demographically diverse populations of all public higher education institutions in the United States, the percentage of students who transfer is especially high, at 42 percent. The National Student Clearinghouse Research Center (NSC; 2017) also reports on the incidence of transfer among United States college students and shows a similar result, in that approximately one in three first-time college students transfers within five years (Hossler et al. 2012). Among these transfer students, 16 percent transfer after one year, 37 percent after the second year, and 25 percent after a third year. As college students become more mobile, moving between institutions and co-enrolling, transfer patterns are also diversifying. The historic definition of “transfer” offered by higher education scholars in the United States has expanded from reflecting student movement from a

90

Debra Bragg

community college that acts as a “sending” institution to a university that acts as a “receiving” institution (known as “vertical transfer”) to “lateral transfer” that reflects transition between institutional types (e.g., two-year to two-year and four-year to four-year). Also increasingly common is the practice of “reverse transfer,” which refers to students beginning at a baccalaureate institution and physically leaving the university to transfer themselves back to a community college (Townsend and Dever 1999, 5), or students maintaining their physical presence at the university but transferring back credits earned at the university to attain their associate’s degree (Taylor and Bragg 2015). Added to these transfer patterns is the growing trend for states to authorize community colleges to confer baccalaureate degrees, mostly in the form of applied baccalaureates, which enables students to attain both the associate’s and bachelor’s degrees from a historically designated associate-degree-granting institution (Bragg and Soler 2017). With so many transfer pathways, including baccalaureate degrees offered by community colleges, the notion of swirling through college (de los Santos and Wright 1989) has become the new normal for students. Looking at state-level statistics also provides insight into the large variation in transfer patterns throughout the United States. The NSC state-level data files provide some of the best data for measuring transfer, and although the NSC does not operate in all fifty states, it is implemented in the vast majority of them, with national coverage reaching over 90 percent. With the NSC’s expanding research capacity in recent years, we have an increasingly more complete and transparent picture of transfer by state and throughout the country. For example, we now know transfer results by state, including the percentage of bachelor’s degree earners who enrolled in public two-year colleges, as well as four-year completion rates by students who started at two-year public colleges. These statistics are important because they reveal differences in college completion between states and are consistent with results of a separate study by Jenkins and Fink (2016) that also relied on NSC data (see the last column of Table 4.2 on the next four pages and more discussion below).1

1. Table 4.2 is also notable because it updates earlier results reported by Trick (2013) on state-by-state transfer rates in the United States for the Higher Education Quality Council of Ontario.

Number of 2015–16 Bachelor’s Degree Earners with Prior Enrollment at Two-Year Public Institutions*

15,556

N/A

24,255

6,750

108,416

12,469

7,427

3,136

50,607

14,254

2,812

4,208

36,351

State

Alabama

Alaska

Arizona

Arkansas

California

Colorado

Connecticut

Delaware

Florida

Georgia

Hawaii

Idaho

Illinois

61%

38%

45%

32%

64%

45%

34%

46%

61%

49%

67%

N/A

59%

Percentage of 2015–16 Bachelor’s Degree Earners with Prior Enrollments at Two-Year Public Institutions*

20%

13%

10%

8%

19%

N/A

12%

14%

14%

11%

N/A

N/A

NA

Four-Year Completion Rate for Students who Started at 2-Year Public Institution**

Four-Year Bachelor’s Completion Rates for Students with Prior Enrollment at Two-Year Colleges

Table 4.2

17%

11%

10%

8%

16%

N/A

10%

10%

15%

8%

10%

N/A

13%

continued

Six-Year Bachelor’s Completion Rate for CCRC Community College Cohort***

Foundations and Reform Measures 91

Number of 2015–16 Bachelor’s Degree Earners with Prior Enrollment at Two-Year Public Institutions*

13,763

11,932

12,273

9,877

9,852

1,640

15,142

13,349

28,437

15,126

9,337

20,289

1,780

State

Indiana

Iowa

Kansas

Kentucky

Louisiana

Maine

Maryland

Massachusetts

Michigan

Minnesota

Mississippi

Missouri

Montana

31%

53%

69%

45%

52%

27%

48%

27%

46%

44%

70%

56%

32%

Percentage of 2015–16 Bachelor’s Degree Earners with Prior Enrollments at Two-Year Public Institutions*

11%

17%

16%

19%

14%

13%

19%

8%

N/A

11%

N/A

22%

N/A

Four-Year Completion Rate for Students who Started at 2-Year Public Institution**

18%

12%

15%

12%

13%

11%

18%

7%

10%

8%

16%

16%

N/A

continued

Six-Year Bachelor’s Completion Rate for CCRC Community College Cohort***

Four-Year Bachelor’s Completion Rates for Students with Prior Enrollment at Two-Year Colleges, continued

Table 4.2

92 Debra Bragg

Number of 2015–16 Bachelor’s Degree Earners with Prior Enrollment at Two-Year Public Institutions*

8,087

4,607

4,228

18,031

4,124

45,048

20,794

3,197

24,224

12,698

12,471

27,302

2,279

State

Nebraska

Nevada

New Hampshire

New Jersey

New Mexico

New York

North Carolina

North Dakota

Ohio

Oklahoma

Oregon

Pennsylvania

Rhode Island

24%

33%

59%

67%

38%

54%

42%

36%

52%

45%

34%

62%

59%

Percentage of 2015–16 Bachelor’s Degree Earners with Prior Enrollments at Two-Year Public Institutions*

N/A

15%

11%

N/A

13%

23%

12%

19%

13%

14%

12%

N/A

15%

Four-Year Completion Rate for Students who Started at 2-Year Public Institution**

N/A

14%

11%

17%

9%

12%

10%

15%

7%

17%

12%

10%

13%

continued

Six-Year Bachelor’s Completion Rate for CCRC Community College Cohort***

Four-Year Bachelor’s Completion Rates for Students with Prior Enrollment at Two-Year Colleges, continued

Table 4.2

Foundations and Reform Measures 93

10,844

1,343

15,411

89,191

13,735

1,413

25,809

16,239

5,911

10,867

1,559

828,450

State

South Carolina

South Dakota

Tennessee

Texas

Utah

Vermont

Virginia

Washington

West Virginia

Wisconsin

Wyoming

Total

74%

31%

40%

52%

48%

29%

44%

75%

50%

25%

48%

17%

11%

N/A

11%

10%

N/A

N/A

19%

16%

2%

17%

Four-Year Completion Rate for Students who Started at 2-Year Public Institution**

**Shapiro et al. (2016, February).

Percentage of 2015–16 Bachelor’s Degree Earners with Prior Enrollments at Two-Year Public Institutions*

*National Student Clearinghouse Research Center. (2017). ***Jenkins and Fink (2006, January).

Number of 2015–16 Bachelor’s Degree Earners with Prior Enrollment at Two-Year Public Institutions*

18%

8%

5%

13%

15%

N/A

N/A

15%

16%

2%

10%

Six-Year Bachelor’s Completion Rate for CCRC Community College Cohort***

Four-Year Bachelor’s Completion Rates for Students with Prior Enrollment at Two-Year Colleges, continued

Table 4.2

94 Debra Bragg

Foundations and Reform Measures 95

Using the NSC datasets to conduct a state-by-state analysis of transfer and baccalaureate attainment, Jenkins and Fink (2016) reported wide variation in student transfer outcomes with respect to baccalaureate attainment. They further analyzed these and supplementary data on state and institutional demographics, funding, and other variables to understand the variation in transfer among the states. Their study produced limited explanatory results about state policy, but it did provide informative results pertaining to institutions. By studying the incidence of transfer and baccalaureate attainment at the institutional level within states, they preliminarily identified factors influencing (or failing to influence) transfer and baccalaureate attainment rates. For example, counter to earlier research that emphasized student demographics as a predominant factor in transfer success, Jenkins and Fink (2016) noted institutional control and funding (i.e., public governance and funding versus private governance and funding) as important factors in baccalaureate attainment. Jenkins and Fink (2016) also found transfer students who attended more highly selective institutions had a higher incidence of baccalaureate attainment. Similarly, the more affluent the student body, the higher the baccalaureate attainment of transfer students. These results parallel findings obtained by Dowd, Cheslock, and Melguizo (2008), reinforcing the importance of understanding how institutional factors impact transfer baccalaureate attainment. Rather than operating from a student deficit perspective that characterizes students as a problem to be fixed (see further discussion of problematizing the student rather than the higher education system in Chase et al. 2014), these studies highlight ways that historical and cultural contexts influence transfer student outcomes. To this end, the stratification of higher education institutions, and therefore college enrollment by student demographics, has to be accounted for to understand fully how transfer works and how it impacts student outcomes. So, for example, the extent to which transfer reforms enhance the educational experiences and outcomes of underserved students, especially racial and ethnic students, has to be taken into account to gain a full picture of the United States transfer landscape. The following sections present transfer reforms in the United States, beginning with reforms that address problems with credit transfer attainment and college credentialing (especially focusing on the baccalaureate degree, but also the associate’s), and then turning to reforms that attempt to link transfer to learning outcomes.

96

Debra Bragg

Credit-Focused Transfer Reforms As noted, efforts to close the gap in baccalaureate attainment through transfer focus extensively on state policies to improve the mobility of college credit between institutions. Recognizing that state governments originate and control transfer policies, reforms to improve credit transfer attainment emanate from state bureaucracies that administer the structures and rules that govern students’ transfer experiences and outcomes. Transfer and articulation are embedded into core administrative functions of state systems in ways that influence student behaviour, both positively and negatively. Examples of reforms that are focusing on this credit mobility problem include offering prior learning assessment and awarding credit for prior learning. Also, offering students dual admission to two- and four-year institutions at the point of entry into college, and implementing university centres on community colleges wherein students do not have to leave their two-year campus to obtain their baccalaureate degree are further examples. Two additional reforms that are spreading throughout the country are reverse credit transfer and baccalaureate degrees conferred by community colleges. These two reforms that are largely focused on ensuring college credit attainment apply to college degrees and are addressed below, but first I briefly summarize the results of two studies that speak to the characteristics of effective credit transfer policies and practices. Seeking to understand the transfer landscape nationally, Hodara and colleagues (2016) led a recent study of transfer policy involving ten states, wherein the team confirmed the importance of reforms that address credit loss. These researchers pointed to the need for guaranteed credit for general education and prerequisite major courses to enable transfer students to transfer credits and begin their studies at the upper-division level without wasting time and money on re-taking lower-division courses not accepted by universities. Hodara et al. (2016) recommended that transfer guarantees should be extended to more programs of study, addressing not only the most popular majors but all majors, and clearly suggesting and fully specifying lower-division courses that are guaranteed to transfer. Her team also recommended improvements in transfer advising, so that students are supported in developing the “transfer college knowledge” (iv) that is needed to succeed in navigating different institutional environments. Hodara et al. (2016) called for improved data systems and more research on the mobility of college credits in the transfer process. In concluding, this

Foundations and Reform Measures 97

team recommended adoption of the “guided pathways model” (40) conceived by Bailey, Jaggars, and Jenkins (2015) to ensure that transfer students have sufficient structure and advising support to map out and follow through on their intended college trajectories. Many of the recommendations made by Hodara et al. (2016) correspond closely with an earlier report by Kisker, Wagoner, and Cohen (2011). Basing their suggestions on many years of research conducted and/or compiled by the University of California at Los Angeles (UCLA) Center for the Study of Community Colleges, Kisker, Wagoner, and Cohen (2011) conducted case studies of four states and described a number of reforms that her team expected to impact student transfer outcomes positively. Table 4.3 (on the next two pages) presents these reforms, along with their identification with the four states included in this study. Similar to Hodara et al. (2016) and Jenkins and Fink (2016), as well as many other transfer researchers who came before them, the transfer policies and practices that Kisker, Wagoner, and Cohen (2011) highlight focus on improving college credit transfer to advance baccalaureate degree attainment. As I’ve noted previously, these reforms are driven mostly by states and therefore require compliance on the part of community colleges and universities to achieve improved outcomes. Given that the focus of many, if not most, transfer reforms is on improving the movement of credit from institution to institution to count toward a baccalaureate degree, I have chosen to delve more deeply into reforms that are being implemented on a fairly wide scale in the United States. The first reform is Credit When It’s Due (CWID), which focuses on reverse credit transfer, and the second is applied baccalaureate degrees that are implemented by universities as well as community colleges, with an increasing number of states granting two-year colleges the statutory authority to confer their own baccalaureate degrees. First, the national CWID network began in 2012 and evolved to include sixteen states that received funding from a group of six philanthropic foundations to facilitate the completion of the associate’s degree when students transfer to the university without first attaining the associate’s. This initiative was formed to address a long-standing problem wherein large numbers (some studies estimate as high as 70 percent) of community college students transfer to universities without receiving an associate’s degree (see, for example, McCormick and Carroll 1997). Despite results showing the positive impact of attaining an associate’s degree prior to transfer, along with concomitant strategies that states have encouraged to improve associate’s degree completion

Lower-division, pre-major, and early-major pathways are guaranteed to transfer toward the new major at the receiving institution. Shifts the focus from any credit attainment to the application of credit toward a major. Students who have attained an associate’s degree prior to transferring are granted junior status with all the rights and privileges appropriate for this upper-division rank, including consideration for scholarships and acceptance into selective majors.

Students are guaranteed admission if they have completed a transfer Arizona, Ohio, associate’s degree (and typically a 2.0-grade point average) and Washington may also be given priority for major choice over out-of-state and non-degree transfer students. Associate’s and baccalaureate degree credit limits, typically a few Arizona, New Jersey, credits or courses beyond the standard credit-hours (e.g., 60 credits Washington for semester-based or 90 credits for quarter-based associate’s degree programs). continued

Common lower-division, pre-major, and early major pathways

A focus on credit applicability

Junior status upon transfer

Guaranteed and/or universal priority admission

Associate’s and/or bachelor’s degree credit limits

Unknown

Unknown

Arizona, Ohio, Washington

Arizona, New Jersey, Ohio, Washington

General education packages or modules that are common across the state’s community colleges that transfer all coursework as a block to public universities.

Common general education package

Evident in the Four States

Definition

Policy/Practice

Center for the Study of Community Colleges Recommended Transfer Policies and Practices

Table 4.3

98 Debra Bragg

Upper-division coursework taken at the lower-division level (typically from a community college although it could be other sending institutions) is counted as meeting the upper-division course requirement.

Acceptance policy for upper-division courses

Note: Adapted from Kisker, Wagoner, and Cohen (2011).

Definition

Policy/Practice Ohio

Evident in the Four States

Center for the Study of Community Colleges Recommended Transfer Policies and Practices, continued

Table 4.3

Foundations and Reform Measures 99

100

Debra Bragg

(see, for example, Crook, Chellman, and Holod 2012; Crosta and Kopko 2014; Shapiro et al. 2013), the pattern of vertical transfer without the associate’s degree continues, precipitating the creation and implementation of reverse credit transfer policy to address this concern (Taylor and Jain 2017). Though the research is just beginning on reverse credit transfer, an estimated nearly 20,000 additional associate’s degrees have been conferred through reverse credit transfer in the sixteen states participating in CWID (Wheatle et al. 2017). This number reflects a unique count of conferred associate’s degree completion through reverse credit transfer, with four states having reported degree conferral for the first time for 2015–2016. All states participating in CWID continue to scale their reverse credit transfer initiatives, which will continue the growth of associate’s degree numbers into the future. Though the CWID study is too new to report baccalaureate attainment rates for students who participated in reverse credit transfer, Taylor and Giani (2016) used data from two CWID states (Minnesota and Hawaiʻi) to investigate the characteristics of students who were eligible for and received an associate’s degree through reverse credit transfer in order to understand who this policy may benefit, paying particular attention to concerns about equity in the transfer process. Inferential findings revealed a statistically significant association between age and income status and completion of receiving reverse transfer associate’s degrees in both states. Reverse credit transfer eligible students who were between eighteen and twenty-four years of age and who were Pell Grant recipients were more likely to receive an associate’s degree via reverse credit transfer than older students (age twenty-five and older) and non-Pell recipient students. In terms of race/ethnicity, Asian students in Hawai‘i and White students in Minnesota were more likely to receive reverse credit transfer associate’s degrees than students having other racial/ethnic backgrounds. This study reveals mixed results for underserved student groups, with some evidence of the benefit of reverse credit transfer for Pell recipients but less so for historically underrepresented groups. The second reform is the applied baccalaureate (AB) degree that is conferred by community colleges. Lagging behind Canada in the implementation of baccalaureate degrees by community and technical colleges, AB degrees in the United States are defined as baccalaureate credentials that focus largely on vocational curriculum that has historically been considered as terminal education offered at the two-year

Foundations and Reform Measures 101

associate of applied science (AAS) degree level. In a national study of AB degrees conferred by both community colleges and universities, Townsend, Bragg, and Ruud (2009) defined the AB degree as “a bachelor’s degree [that is] designed to incorporate applied associate courses and degrees once considered “terminal” or non-baccalaureate level while providing students with higher-order thinking skills and advanced technical knowledge and skills” (iv). States, and often institutions within states, use different titles for the AB degree, which seems to be due in part to the high degree of autonomy that state governments (and in some states, the institutions within higher education systems) have to determine college degree titles. Degree titles, such as the Bachelor of Applied Science (BAS), Bachelor of Technology (BT) and Bachelor of Applied Technology (BIT) are used commonly in association with AB degrees in the United States. Whereas virtually every state in the country authorizes at least one four-year baccalaureate degree-granting institution to confer some form of AB degree, the conferral of AB degrees by community colleges is less common but growing among the states (Townsend, Bragg, and Ruud 2009). Currently, the Community College Baccalaureate Association (n.d.) reports that twenty-four states authorize an estimated 650 community colleges to confer AB degrees in the United States. A few states, such as Florida and Washington, have authorized the majority of their community colleges to confer these degrees, but this level of authorization is highly unusual; most states authorize a relatively small number of community colleges to confer AB degrees (Bragg and Soler 2017). In some states that authorize community colleges to confer AB degrees, there is a clear and direct link between community college AB degree policy and transfer and articulation policy. For example, in the state of Washington, the leadership of AB degree authorization and conferral resides with the chief administrator of transfer for the state community and technical college system, making transfer and degree conferral a logical extension of the transfer process. The AB degrees conferred by community and technical colleges in Washington help to fill a gap in higher education capacity at the four-year level in this state. Efforts led by the state focus on coordinating transfer between the community and technical colleges and universities, or the community and technical colleges offering their own baccalaureate degrees. However, this alignment is not typical in states with more capacity for higher education and where the power dynamics favour universities that pre-

102

Debra Bragg

vent AB degree conferral by community colleges from gaining support from state policymakers. In all of these reforms, only a modest amount of research has been conducted, and often those studies do not sufficiently examine issues of transfer and college credit and degree attainment for historically underserved student populations. This is unfortunate because it means states may miss identifying reforms that reproduce inequitable outcomes for students of colour, low-income, and first-generation college students who begin their collegiate studies at the community college level in disproportionate numbers. A critical policy analysis conducted by Chase et al. (2014) suggests the lack of an equity lens is pervasive in state transfer policy in the United States. By analyzing equity (and inequity) in transfer policies in numerous states, delving most deeply into transfer policy in California, the state with the largest community college enrollment in the United States, Chase et al. (2014) reveal a dearth of attention paid to racial and ethnic students in either accessing and attaining transfer education. This, despite long-standing evidence of inequities between these groups and White students (see, for example, Brint and Karabel 1989; Dougherty 1994). The following section offers some early insights into how learning outcome reforms that more explicitly focus on inequities for students of colour may be addressed. Learning Outcomes Reforms The state priority to improve college credit transfer is well established. Decades of state policy work to standardize, coordinate, and confer credit according to rules and regulations have created transfer bureaucracies that impact the learning experiences of students. Rules and regulations that govern credit attempt to normalize the transfer function in ways that replicate the traditional transfer experience; however, the fact that these rules and regulations exist suggests transfer is still outside what is considered a normal college experience. For underserved students, the complexity and burden of transferring to and from multiple institutions, each with its policies and procedures—despite state bureaucracies—makes clear why it is important to understand how transfer relates to students’ college learning experiences. To address some of these complexities, in 2011 the Lumina Foundation convened a diverse group of stakeholders from across the United States and Europe to begin the process of creating the Degree Qualifications Profile (DQP) that was informed by the Bologna Process. Know-

Foundations and Reform Measures 103

ing higher education systems in the United States would be unlikely to replicate the Bologna Process without considerable adaptation, the Lumina Foundation (n.d.) described the DQP as representing “distinctly American values and emphases” (1). Lumina believed that higher education in the United States would benefit from a framework reflecting the diversity of American higher education while also establishing curricular and evaluation benchmarks that would contribute to curriculum improvement within and across institutions. Through the support of Lumina, the American Association of Colleges & Universities (AAC&U) and other national organizations participated in creating, implementing, and testing the DQP in higher education systems and institutions across the country. One of these organizations is the National Institute for Learning Outcomes Assessment (NILOA), which offers technical assistance to higher education institutions interested in using the DQP, including those seeking to improve transfer and articulation. While transfer is not the primary focus of NILOA’s agenda, materials produced by NILOA researchers and partners reveal ways faculty and staff involved in transfer can capitalize on using the DQP to align curriculum, learning outcomes, and learning outcomes assessments (see, for example, Jankowski, Kinzie, and Marshall 2016; Marshall, Jankowski, and Vaughn 2017). Another initiative that links learning outcomes to transfer is the AAC&U’s Quality Collaboratives project, which deliberately connects learning outcomes to transfer reforms occurring within and between institutions. Humphries, McCambly, and Ramaley (2015), focusing on addressing the “transfer ‘problem’” (13) with a particular focus on transfer by underserved student populations, link learning outcomes for racially diverse and low-income transfer students to reforms attempting to address issues with time to degree completion, excessive credits, and high cost. They further argue that transfer reforms that focus on credit-oriented reform and do not focus on linking teaching and learning to transfer in deliberate ways will fall short of success. They suggest that transfer reforms focusing exclusively on credit attainment will not go deep enough to “transform foundational practices and policies in ways that recognise the reality of today’s students and today’s institutional contexts” (14). A primary focus of the Quality Collaboratives initiative was to partner community colleges and universities to focus on educational quality simultaneously with college completion, shifting the focus to student learning and also to reforms demonstrating more equitable outcomes.

104

Debra Bragg

Partner institutions that participated in this initiative brought together college and university decision makers, faculty, support staff, and others to identify levers for change that create opportunities to link improvements to the transfer process to enhanced student learning, and ultimately, to increased college completion. The focus on student learning that involves faculty more centrally was the most innovative aspect of this work, and unique among transfer reforms, as I have noted in this chapter. The project deliberately minimized the focus on policy strategies resulting in state and institutional targets, such as raising transfer completion rates by 5 percent in a given number of years, and instead focused on identifying learning and equity goals consistent with AAC&U’s Liberal Education and America’s Promise (LEAP) initiative, which is aligned to Lumina’s DQP. The Quality Collaboratives initiative facilitated meetings involving the faculty of partner institutions to align learning outcomes to transfer and completion goals using data disaggregated by student sub-group to assess and improve transfer pathways and student learning. These strategies were designed to enable institutions and institutional partners to hold themselves accountable to the “demonstrated achievement of quality learning outcomes” (Humphries, McCambly, and Ramaley 2015, 25). One other important goal was to take lessons from this bottom-up initiative to inform state bureaucracies on ways to improve transfer policies and practices. A unique aspect of the Quality Collaboratives project was its explicit recognition that inequitable outcomes are a legacy of existing state transfer bureaucracies. For this reason, the AAC&U adopted the equity-inclusive model developed by Alicia Dowd (2003) as an integral part of its learning outcomes approach. Counter to credit-focused transfer reforms that link purported improvements to achieving increased efficiency, Dowd’s model centres transfer reform on equity by engaging faculty in inquiry processes that support their understanding of students’ experiences and outcomes, and sometimes also engaging students as active stakeholders in improving learning. Dowd suggests higher education institutions should be held accountable for improving equity in student learning outcomes that are linked to degree completion outcomes, and she further argues that a comprehensive approach to transfer requires many stakeholders to play a role in the students’ learning experience or little will change. Drawing lessons from Dowd’s proverbial playbook, research on high-performing transfer partnerships (Meza, Bragg, and Blume 2018; Yeh 2018) builds on studies of transfer and equity (see, for example,

Foundations and Reform Measures 105

Dowd et al. 2009) to determine how historical patterns of inequity in college access and outcomes for racial and ethnic groups are replicated or disrupted by partnerships between colleges and universities. While this study is still underway, preliminary results confirm gaps in college retention and completion between racial and ethnic student groups and White student groups in three states, with some gaps being a few percentage points and others showing a much larger and more troubling gulf. Some factors that appear to contribute to institutional pairings (of which some are considered true transfer partnerships) that yield favorable results are top leadership commitment to transfer, faculty and staff engagement and support for various aspects of the transfer function, and close geographic proximity of institutions. Future research is focused on understanding how strong transfer partnerships improve outcomes for transfer students, particularly racial and ethnic student groups, and—similar to the Quality Collaboratives initiative—using these lessons to inform state transfer bureaucracies. Unique among the few existing transfer initiatives focusing on learning outcomes is the Interstate Passport project administered by the Western Interstate Commission for Higher Education (WICHE), which facilitates the transfer of student learning between institutions located in the western United States. This initiative focuses on increasing college completion rates among transfer students attending member institutions. This project incorporates research using the NSC data on transfer student performance for quality assurance. The project focuses on learning associated with lower-division general education as a whole rather than individual courses, and it uses “block transfer” to allow for the matching of learning outcomes with competency-based outcomes. The project provides for processes that link learning outcomes to proficiency criteria in nine knowledge and skill areas (e.g., skills: communication, literacy knowledge: natural science, human society), again tied to the AAC&U’s LEAP essential learning outcomes (Western Interstate Commission for Higher Education [WICHE] 2016). Complimentary partnerships between institutions involving the faculty within and across institutions are central to the Interstate Passport initiative. Though this project is relatively young, and no outcomes are known to exist, it is expanding from the current sixteen participating states to include additional states and institutions. The focus on learning outcomes and competency-based transfer has also become evident in a few states participating in CWID reverse credit transfer. For example, Hawaiʻi has used its extensive information

106

Debra Bragg

technology infrastructure to create a competency-based framework to articulate general education competencies to upper-division courses that students can count toward associate’s degree requirements for reverse credit transfer. Data on associate’s degree completion show Hawaiʻi is expanding the number of students who are being awarded associate’s degrees via reverse credit transfer at a rate higher than other states. Because this approach is not based on course-to-course articulation but rather a competency attainment model that reflects learning in broad disciplinary categories, it represents another example of how learning outcomes are being linked to transfer. However, an important lesson from this state’s efforts to award credit based on competencies is that a relatively sophisticated data and information-sharing system is needed to support this effort, and despite a large federal investment in state longitudinal data systems many states do not have such systems in place. The fact that Hawaiʻi operates higher education with a single higher education system contributes greatly to why and how its data capacity is sufficient to support this competency-based, learning-outcomes-oriented approach to transfer. Summarizing this section on learning outcomes, I think it is important to acknowledge again that the strong focus on credit attainment guided by states emphasizes efficiencies in credit conferral far more than learning outcomes, though more attention is being paid to transfer and learning outcomes than in the past. For example, Kisker, Wagoner, and Cohen (2011) and Mullin (2012) have used their informed positions in nationally recognized organizations such as the Center for the Study of Community Colleges at UCLA and the American Association of Community Colleges to urge higher education systems and institutions to centre learning outcomes in transfer reforms. More recently, the Aspen Institute emphasized the importance of learning expectations, experiences, and outcomes in their study of essential practices employed by community colleges and universities, noting the importance of transfer pathways aligned to high-quality instruction and structured student advising (Wyner et al. 2016). While an important step to linking transfer to learning outcomes, this approach does not explicitly emphasize the equity lens that Dowd (2003) and other abovementioned initiatives have emphasized in the transfer reform conversation. Without this clear-eyed focus on ways in which transfer serves historically underserved populations, especially racial and ethnic students, it is difficult to imagine how transfer will improve substantially in the US context. To this end, research that is delving into inequitable transfer out-

Foundations and Reform Measures 107

comes for underserved students, appropriately named by Crisp and Nuñez (2014) as the “racial transfer gap,” deserves further mention here. In their research, Crisp and Nuñez suggest reform is needed that focuses on understanding and improving the experiences of transfer students of colour, including understanding how institutional curriculum and other support services strengthen (or weaken) student outcomes. Other research that explicitly examines transfer by racial and ethnic students suggests a student’s sense of belonging and identity as a legitimate college student impacts their transfer experience and outcomes (Bensimon and Dowd 2009). To counter the racial transfer inequities, these researchers point to the importance of “transfer champions” who guide and support underserved students in pursuing their transfer programs of study. Jain et al. (2011) have also contributed insightful findings pertaining to underserved populations, noting the importance of understanding how institutional culture and campus climate influences racial minority transfer students, including focusing on support services that are sensitive to these students’ lived experiences as college learners. Final Thoughts As this chapter has noted, transfer has a long history that is rooted in an uneven power dynamic between community (initially named “junior”) colleges that send students to universities. Despite tremendous growth in college enrollments, including large numbers of students who transfer vertically but also laterally, in reverse, and to multiple institutions, institutions continue to struggle despite transfer bureaucracies perpetuating policies, rules, and regulations to improve baccalaureate attainment rates. Though some patterns and preferences for how states administer transfer systems are evident, such as the adoption of statewide articulation agreements and other transfer-related policies (e.g., common course numbering and transfer guides), there is wide variation among states in the adoption and implementation of these policies, and there is also limited evidence suggesting these state policies positively impact transfer student outcomes. In fact, the limited research that has been conducted on state transfer and articulation policies and practices tends to show limited or no effects on student outcomes. To improve transfer, some higher education systems and institutions are pushing harder on reforms focusing on improving efficiency, primarily by improving the transfer of college credits. Efforts at “guided

108

Debra Bragg

pathways” steer institutions in this direction, with a heavy focus on improving internal functions such as advising, technology, and data utilization (Bailey, Jaggars, and Jenkins 2015). Important from the standpoint of shoring up processes that appear critical to enabling students to matriculate between institutions, while evident, the focus on learning outcomes appears less pervasive than in the AAC&U Quality Collaboratives (Humphries, McCambly, and Ramaley 2015), WICHE’s Interstate Passport initiative (WICHE 2016), and high-performing transfer partnerships (Meza, Bragg, and Blume 2018). Drawing on these initiatives for lessons to link the transfer reform to learning outcomes, including partnering institutions in deliberate ways to address the needs of underserved students, may result in improved outcomes for all students. Though originating from different places, these latter initiatives point to centring reform at the institutional level and allow lessons to bubble up to improve state transfer bureaucracies. Enabling a richer and deeper understanding of institutions, each with their own historical, cultural, curricular, and policy contexts, may enable improvements in transfer to emerge that better serve the racial and ethnic learners who make up an increasingly larger proportion of college students in the United States. The lessons that I glean for the Learning Outcomes for Transfer project are simple to state but potentially difficult to execute. Improving transfer policies and practices relies on the implementation of changes at the institutional level where faculty, staff, and students can be engaged. These changes need to occur within institutions but also in conjunction with other institutions, recognizing that transfer takes different forms and engages a diversity of learners, especially historically underserved student populations who have limited experience with college. This is precisely why transfer reforms need to happen on the ground, where teaching and learning is the predominant function. State transfer bureaucracies that focus on credit mobility and conferral can set boundaries for institutions to operate, but they remain marginal to the learning that transfer students experience as they navigate within and between institutions. If transfer is to improve, it will require the active engagement of institutional and partnership reforms that are dedicated to ensuring college learning is more impactful and effective for underserved students. If this occurs, all students will benefit.

Foundations and Reform Measures 109

References Anderson, G., J. C. Sun, and M. Alfonso. 2006. “Effectiveness of Statewide Articulation Agreements on the Probability of Transfer: A Preliminary Policy Analysis.” The Review of Higher Education, 29 (3): 261–291. Anderson, L., and M. Millard. 2014. 50-State Comparison: Transfer and Articulation Policies. http://www.ecs.org/transfer-and-articulation-policies-db/ Bailey, T. R., S. S. Jaggars, and D. Jenkins. 2015. Redesigning America’s Community Colleges: A Clearer Path to Student Success. Cambridge, MA: Harvard University Press. Bensimon, E. M., and A. Dowd. 2009. “Dimensions of the Transfer Choice Gap: Experiences of Latina and Latino Students Who Navigated Transfer Pathways.” Harvard Educational Review, 79 (4): 632– 659. Bragg, D. 2012. “Career and Technical Education.” In Understanding Community Colleges, edited by J. Levin and S. Kater, 187–202. London, UK: Routledge/Taylor Francis. Bragg, D., and M. Soler. 2017. “Policy Narratives on Applied Baccalaureate Degrees: Implications for Student Access to and Progression through College in the United States.” Journal of Vocational Education and Training, 39 (1): 123–146. Brint, J., and S. Karabel. 1989. The Diverted Dream: Community Colleges and the Promise of Educational Opportunity in America,1900–1985. New York, NY: Oxford University Press. Chase, M., A. Dowd, L. B. Pazich, and E. Bensimon. 2014. “Transfer Equity for ‘Minoritized’ Students: A Critical Policy Analysis in Seven States.” Educational Policy, 28 (5): 669–717. Cohen, A., F. Brawer, and K. Kisker. 2014. The American Community College (6th Ed). San Francisco, CA: John Wiley & Sons. Community College Baccalaureate Association. n.d. https://www.accbd.org Crisp, G., and A. M. Nuñez. 2014. “Understanding the Racial Transfer Gap: Modeling Underrepresented Minority and Nonminority Students’ Pathways from Two-to Four-Year Institutions.” Review of Higher Education, 37 (3): 291–320. Crook, D., C. Chellman, and A. Holod. 2012. Does Earning an Associate Degree Lead to Better Baccalaureate Outcomes for Transfer Students? New York, NY: City University of New York Office of Policy Research.

110 Debra Bragg

https://www.cuny.edu/about/administration/offices/ira/opr/papers/associate_degree.pdf Crosta, P., and E. Kopko. 2014. Should Community College Students Earn an Associate Degree before Transferring to a Four-Year Institution? New York, NY: Community College Research Center, Teachers College, Columbia University. https://www.luminafoundation.org/files/resources/associate-degree-before-transfer.pdf de los Santos, A. G., and I. Wright. 1989. “Community College and University Transfers.” Educational Record, 79 (3/5): 82–84. Dougherty, K. 1994. The Contradictory College. Albany, NY: SUNY Press. Dowd, A. C. 2003. “From Access to Outcome Equity: Revitalizing the Democratic Mission of the Community College.” The Annals of the American Academy of Political and Social Science, 586 (1): 92–119. Dowd, A., M. Chase, L. B. Pazich, and E. Bensimon. 2009. “Transfer Equity for ‘Minoritized’ Students: A Critical Policy Analysis of Seven States.” (Paper presented at the Association for the Study of Higher Education (ASHE), Vancouver, BC.) http://www.deanza.edu/ir/ deanza-research-projects/transferequityarticle.pdf Dowd, A., J. Cheslock, and T. Melguizo. 2008. “Transfer Access from Community Colleges and the Distribution of Elite Higher Education.” Journal of Higher Education, 79: 442–472. Gilbert, C. K., and D. Heller. 2013. “Equity, Access, and Community Colleges: The Truman Commission and Federal Higher Education Policy from 1947 to 2011.” Journal of Higher Education, 84 (3): 417–443. Hodara, M., M. Martinez-Wenzl, D. Stevens, and C. Mazzeo. 2016, May. “Improving Credit Mobility for Community College Transfer Students: Findings and Recommendations from a 10-State Study.” Portland, OR: Education Northwest. http://educationnorthwest.org/ resources/improving-credit-mobility-community-college-transfer-students. Hossler, D., D. Shapiro, A. Dundar, M. Ziskin, J. Chen, D. Zerquera, and V. Torres. 2012, February. Transfer & Mobility: A National View of Pre-Degree Student Movement in Postsecondary Institutions (Signature Report 2). Reston, VA: National Student Clearinghouse Research Center and the Project on Academic Success at Indiana University. http://www.studentclearinghouse.info/signature/2/NSC_Signature_Report_2.pdf Humphries, D., H. McCambly, and J. Ramaley. 2015. The Quality of a College Degree: Toward New Frameworks, Evidence, and Interventions. Washington, DC: AAC&U.

Foundations and Reform Measures 111

Ignash, J., and B. Townsend. 2000. “Evaluating State-Level Articulation Agreements According to Good Practice.” Community College Review, 28 (3): 1–21. Jain, D., A. Herrera, S. Bernal, and D. Solorzano. 2011. “Critical Race Theory and the Transfer Function: Introducing a Transfer Receptive Culture.” Community College Journal of Research and Practice, 35 (3): 252–266. Jankowski, N., J. Kinzie, and D. Marshall. 2016. “Degree Qualifications Profile & Tuning: Student Learning Impact and IR.” http://www. learningoutcomeassessment.org/Presentations/Jankowski_Kinzie_ Marshall_AIR_2016.pdf Jenkins, D., and J. Fink. 2016. Tracking Transfer: New Measures of State and Institutional Effectiveness in Helping Community College Students Attain Bachelor’s Degrees. New York, NY: Community College Research Center, Teachers College, Columbia University. http://ccrc. tc.columbia.edu/media/k2/attachments/tracking-transfer-institutional-state-effectiveness.pdf Kisker, C. B., R. L. Wagoner, and A. M. Cohen. 2011. Implementing Statewide Transfer & Articulation Reform: An Analysis of Transfer Associate Degrees in Four States. http://centerforcommunitycolleges.org/index.php/ download_file/view/26/ LaSota, R., and W. Zumeta. 2016. “What Matters in Increasing Community College Students’ Upward Transfer to the Baccalaureate Degree: Findings from the Beginning Postsecondary Student Study 2003–2009.” Research in Higher Education, 57: 152–189. Lumina Foundation. n.d. “Lumina Foundation Strategic Plan for 2017 to 2020.” Indianapolis, IN: Author. https://www.luminafoundation. org/files/resources/lumina-strategic-plan-2017-to-2020.pdf Marshall, D. W., N. A. Jankowski, and T. Vaughan, III. 2017, December. Tuning Impact Study: Developing Faculty Consensus to Strengthen Student Learning. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment (NILOA). http://www.learningoutcomesassessment.org/documents/TuningImpactStudy.pdf Meza, E. A., D. D. Bragg, and G. Blume. 2018, February. Including Racial Equity as an Outcome Measure in Transfer Research (Transfer Partnerships Series, Data Note 2). Seattle, WA: Community College Research Initiatives, University of Washington. McCormick, A., and C. D. Carroll. 1997. Transfer Behavior among Beginning Postsecondary Students: 1989 to 1994. Washington, DC: National

112 Debra Bragg

Centre of Education Statistics, U.S. Department of Education. http:// nces.ed.gov/ and link to the report: https://files.eric.ed.gov/fulltext/ED408929.pdf Mullin, C. M. 2012, October. Transfer: An Indispensable Part of the Community College Mission (Policy Brief 2012-03PBL). Washington DC: American Association of Community Colleges. http://www.aacc.nche. edu/Publications/Briefs/Documents/AACC_Transfer_to_LUMINA.pdf National Center for Education Statistics. 2015. Digest of Educational Statistics, 2015. Washington, DC: Institute of Education Sciences, United States Department of Education. ———. 2016. Table 303.70: Total Undergraduate Fall Enrollment in Degree-Granting Postsecondary Institutions, by Attendance Status, Sex of Student, and Control and Level of Institution. In Digest of Education Statistics 2016. https://nces.ed.gov/programs/digest/d17/tables/ dt17_303.70.asp National Commission on Excellence in Education. 1983. “A Nation at Risk.” Washington, DC: United States Department of Education. https://www2.ed.gov/pubs/NatAtRisk/title.html National Student Clearinghouse Research Center. 2017. “The Role of Two-Year Public Institutions in Bachelor’s Attainment.” Herndon, VA: Author. https://nscresearchcenter.org/wp-content/uploads/ SnapshotReport26.pdf Parnell, D. 1985. The Neglected Majority. Washington, DC: American Association of Community and Junior Colleges. Ryan, C., and K. Bauman. 2016, March. Educational Attainment in the United States: 2015, Population Characteristics (Report No. P20-578). Washington DC: United States Census Bureau, United States Department of Commerce. https://www.census.gov/content/dam/Census/library/publications/2016/demo/p20-578.pdf. Sanchez, C. 2014, March 14. How the Cost of College Went from Affordable to Sky-High, Morning Edition, National Public Radio. https://www. npr.org/2014/03/18/290868013/how-the-cost-of-college-wentfrom-affordable-to-sky-high Shapiro, D., A. Dundar, P. Wakhungu, X. Yuan, and A. Harrell. 2015. Completing College: A State-Level View of Student Attainment Rates (Signature Report No. 8a). Herndon, VA: National Student Clearinghouse Research Center. https://nscresearchcenter.org/signaturereport8-statesupplement/ Shapiro, D., A. Dundar, P. Wakhungu, X. Yuan, A. Nathan, and Y. A.

Foundations and Reform Measures 113

Hwang. 2016, February. Completing College: A State-Level View of Student Attainment Rates (Signature Report No. 10a). Herndon, VA: National Student Clearinghouse Research Center. https://nscresearchcenter.org/wp-content/uploads/NSC_Signature_Report_12_ StateSupp.pdf Shapiro, D., A. Dundar, M. Ziskin, Y. Chiang, J. Chen, A. Harrell, and V. Torres. 2013. Baccalaureate Attainment: A National View of the Postsecondary Outcomes of Students Who Transfer from Two-Year to Four-Year Institutions (Signature Report No. 5). Herndon, VA: National Student Clearinghouse Research Center. https://nscresearchcenter.org/signaturereport5/ Smith, M. 2010. State Notes: Transfer and Articulation. Boulder, CO: Education Commission of the States. http://www.ecs.org/clearinghouse/90/70/9070.pdf Taylor, J., and D. Bragg. 2015. “Optimizing Reverse Transfer Policies and Processes: Lessons from Twelve CWID states.” Champaign, IL: Office of Community College Research and Leadership (OCCRL), University of Illinois at Urbana-Champaign. https://www.luminafoundation.org/files/resources/optimizing-reverse-transfer.pdf Taylor, J., and M. Giani. 2016. “Reverse Transfer: Taking Stock and Moving Forward.” Update on Research and Leadership, 27 (1). Champaign, IL: Office of Community College Research and Leadership, University of Illinois at Urbana-Champaign. Taylor, J. L., and D. Jain. 2017. “The Multiple Dimensions of Transfer: Examining the Transfer Function in American Higher Education.” Community College Review, 45 (4): 273–293. https://doi. org/10.1177/0091552117725177 Thelin, J. 2004. A History of American Higher Education (2nd ed.). Baltimore, MD: John Hopkins University Press. Townsend, B., and J. Dever. 1999. “What Do We Know about Reverse Transfer Students?” New Directions for Community Colleges, 106: 5–15. Townsend, B., D. D. Bragg, and C. M. Ruud. 2009. “Development of the Applied Baccalaureate. Community College Journal of Research and Practice, 33 (9): 686–705. Trick, D. 2013. College-to-University Transfer Arrangements and Undergraduate Education: Ontario in a National and International Context. Toronto, ON: Higher Education Quality Council of Ontario. Western Interstate Commission on Higher Education (WICHE). 2016. The Interstate Passport: A New Framework for Transfer. Boulder, CO: Author. http://www.wiche.edu/passport/about/overview

114 Debra Bragg

Wheatle, K., J. Taylor, D. Bragg, and J. Ajinkya. 2017. The Potential of Degree Reclamation: A Path to Reclaiming the Nation’s Unrecognised Students and Degrees. Washington, DC: Institute for Higher Education Policy. Wyner, J., K. C. Deane, D. Jenkins, and J. Fink. 2016. The Transfer Playbook: Essential Practices for Two- and Four-Year Colleges. New York, NY: Columbia University, Teachers College, Community College Research Center. Yeh, T. L. 2018, January. Introduction to the High-Performing Transfer Partnerships Study (Transfer Partnerships Series, Data Note 1). Seattle, WA: Community College Research Initiatives, University of Washington.

5

Learning Outcomes for Credit Transfer: Reflections on the Australian Experience1 Peter Noonan

Introduction Current Arrangements Credit transfer between the higher education and Vocational Education and Training (VET) sectors has been an important, but intermittent, policy priority in Australia for the past three decades. Over that period, national policies and guidelines have been put in place and refined to guide institutional practices. The Australian Qualifications Framework (AQF) includes a Pathways Policy, the purpose of which is “to maximise the credit that students can gain for learning already undertaken” (Australian Qualifications Framework Council 2013, 77). The AQF 1. This paper was prepared prior to the 2019 Federal election in Australia which saw the return of the Liberal\National Party government. It also precedes the Review of the Australian Qualifications Framework (AQF) which was chaired by the author. The Review included recommendations to improve credit transfer policies in Australia. A copy of the Review report and an associated research paper on credit and pathways is available through the Australian Government Department of Education online document library website docs-edu.govcms.gov.au Learning Outcomes, Academic Credit, and Student Mobility, edited by Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon. Montréal and Kingston: McGill-Queen’s University Press, Queen’s Policy Studies Series. © 2020 The School of Policy Studies, Queen’s University at Kingston. All rights reserved.

116 Peter Noonan

Pathways Policy only provides guidance to institutions—it is not and cannot be used for registration or audit of providers in either sector. Nonetheless, all public universities and many private higher education providers have policies on credit transfer, recognition of prior learning, and advanced standing. Many universities also have established agreements with individual VET providers that guarantee VET students graduating with VET diplomas and advanced diplomas credit toward higher education qualifications (some Certificate IV courses carry credit as well). Some universities guarantee entry with credit for graduates from partner VET providers (for example, Charles Sturt University 2018). Australia does not have a set of tertiary education institutions whose core role includes the provision of credit-based pathways into higher education degrees, such as in many community colleges in the United States and some colleges in Canada. Nonetheless, most public Technical and Further Education (TAFE) institutions in all Australian states and territories and some non-governmental VET providers have in place agreements that specify minimum levels of credit with partner universities. However, the proportion of students transitioning from VET to higher education varies significantly between universities, largely reflecting institutional mission and prestige (Chesters, Watson, and Hagel 2013). The numbers of students involved are also small. In 2015 approximately 35,000 students commenced higher education based on their prior studies in VET, out of 700,000 commencing higher education students (international and domestic) of which 300,000 were government funded students (it is not possible to determine how many of these students commenced with credit from VET). Nonetheless, credit-based pathways from VET to higher education are important for many students and tertiary education institutions. Drivers, Enablers, and Barriers to Credit Transfer The most comprehensive study into credit transfer in Australia, Giving Credit Where Credit Is Due (Phillips KPA 2006), identified the primary drivers of credit transfer arrangements as government policy and directions, efficiency and cost savings, competition for students, institutional mission, employer and student needs, and convergence of the VTE (vocational and technical education) and higher education sectors (iii). The study also identified the major enablers of credit transfer as institutional leadership, mutual respect, and commitment (among partner

Learning Outcomes for Credit Transfer: Reflections on the Australian Experience 117

institutions). Information provisions for students, so that they can take advantage of credit transfer opportunities, is also critical. Giving Credit Where Credit Is Due also identified major barriers to credit transfer pathways arising from differences between the higher education and VET sectors. The primary barriers identified were funding and accountability, attitudes and culture, administrative issues, curriculum and qualification design, assessment, and lack of resources. Issues related to curriculum, qualification design, and assessment fundamentally relate to the different ways the higher education and VET sectors define outcomes for learners. In this regard Giving Credit Where Credit Is Due made the following observations: Associated with basic differences in purpose between the VTE and higher education sectors, there are also differences in the way in which qualifications are structured and described and in the approaches each sector takes to curriculum and assessment. Mapping of equivalence of student outcomes between the sectors becomes much more difficult when the way in which curriculum is designed, described and assessed in each sector is very different. The use of non-graded assessment in the VTE sector is a significant barrier to admission of VTE students to higher education. (Phillips KPA 2006, v)

The differences between the sectors in relation to qualification design and assessment are long-standing, and despite several major reviews are essentially unchanged even today. The AQF requirements for individual qualifications are cross-referenced in the standards governing the registration of VET providers (VET Quality Framework) and higher education providers (Birmingham 2015). In higher education, under the 2015 Threshold Standards, learning outcome requirements for higher education qualifications are defined as follows: The specified learning outcomes for each course of study encompass discipline-related and generic outcomes, including: •

Specific knowledge and skills and their application that characterize the field(s) of education or disciplines involved;

•

Generic skills and their application in the context of the field(s) of education or disciplines involved;

118 Peter Noonan

•

Knowledge and skills required for employment and further study related to the course of study, including those required to be eligible to seek registration to practise where applicable; and

•

Skills in independent and critical thinking suitable for lifelong learning. (Birmingham 2015, Section 1.4)

Outcomes from VET qualifications in Australia are defined more narrowly through the requirements of individual units of competence, which “define the skills and knowledge to operate effectively and how they need to be applied to perform effectively in a workplace context” (Australian Skills Quality Agency, n.d., para. 1). Walls and Pardy (2010) argue that the effect of these differences in approach is that The issue of reconciling the skills-based competencies of VET with the codified knowledge of higher education in order to more clearly navigate the boundaries—or the crazy paving—remains complicated. In practice it is learning equivalence that remains the point of impasse for achieving equitable credit transfer arrangements. A means for establishing equivalence is imperative to ensuring that credit is recognised and awarded without prejudice. (8)

To understand how these differences developed, and their effects, it is necessary to look at how the current arrangements for credit transfer between VET and higher education evolved. The Evolution of Credit Transfer Policy The issue of credit transfer in Australian tertiary education was first placed squarely on the national policy agenda in Higher Education: A Policy Discussion Paper, which was released by the Australian Government in December 1987 (Dawkins 1987). The discussion paper set out the government’s broad objectives for higher education and canvassed far-reaching reform proposals. It preceded Higher Education: A Policy Statement, a formal statement of government higher education policy intent, which was released in July 1988 (Dawkins 1988). The discussion paper reflected the then-minister for employment education and training John Dawkins’s determination to challenge many of the prevailing orthodoxies in universities, including their perceived lack of responsiveness to learner needs. Consistent with this perspective, the discussion paper asserted:

Learning Outcomes for Credit Transfer: Reflections on the Australian Experience 119

The difficulty in Australia of obtaining credit for previous study is well known. Unfortunately, institutions have been slow to respond. This inaction has affected school students proceeding to TAFE and higher education courses, former apprentices proceeding to TAFE and advanced education paraprofessional courses, TAFE graduates of the latter courses who proceed to diploma and degree courses and, finally, undergraduates and postgraduates who move across or within higher education institutions. (Dawkins 1987, 39)

The discussion paper canvassed a range of system- and institutionallevel initiatives to improve credit transfer arrangements within the higher education sector and between the higher education and the TAFE sectors.2 It articulated the radical principle that the onus should be on the institution to demonstrate why credit should not be provided, rather than on the student to demonstrate why credit should be provided. One of the system-level measures proposed in the discussion paper included the potential to link funding to the availability of credit transfer (based on the principle that government funding should not be used to fund higher education institutions where students had already acquired the required level of skills and knowledge). Possible institutional measures in the discussion paper included the development of two-year courses in both higher education and TAFE, for which automatic credit would be provided by participating institutions. However, in the government’s subsequent formal policy statement (Dawkins 1988), the far-reaching systemic and institutional options to improve credit transfer outlined in the discussion paper were not pursued. Rather, the policy statement outlined a series of credit transfer principles based on guidelines developed jointly between the then Australian Vice-Chancellors Committee (AVCC) representing universities and the peak body representing the heads of the former Colleges of Advanced Education. (TAFE was not involved in the development of these principles.) The policy statement foreshadowed an expectation that institutions would adopt the credit transfer principles as a condition of access to government funding as an institution in the new Unified National System of Higher Education (UNS)—the centrepiece of the policy statement. The principles outlined in the policy statement included 2. TAFE as a set of public institutions preceded the broader concept of the national VET system, which comprises TAFE as well as nationally recognized governmental and non-governmental VET providers.

120

Peter Noonan

the potential development of credit-based, two-year TAFE diplomas and associate diplomas to provide credit-based pathways from TAFE to higher education. The approach set out in the policy statement— setting a general policy framework and identifying principles to guide institutional actions rather than mandating system-wide requirements for credit transfer—has subsequently typified the approach to credit transfer by successive governments in Australia. Significantly, an option to incorporate TAFE diplomas and associate diplomas into the higher education funding system and even for TAFE institutions to become higher education providers was discarded in the policy statement. This was in part because the states—who controlled TAFE—had not supported the Commonwealth taking over funding of TAFE diplomas and associate diplomas, and also because the government was pursuing a parallel reform process to improve the skills base of the Australian workforce. As a consequence, the policy statement added an important caveat to its general support for credit transfer: Credit transfer and articulation are not objectives to be attained at any cost. Some TAFE courses have a structure and integrity which do not lend themselves to full articulation and credit transfer. Such courses must not be made more theoretical simply to meet the needs of a minority (albeit an important minority) who may wish to transfer at a later date. In such cases, appropriate credit plus bridging courses should be provided where numbers warrant. (Dawkins 1988, 38)

The retreat from the specific options outlined in the discussion paper to the principles outlined in the policy statement reflected a general softening in the overall approach to higher education policy by the government, particularly in response to criticisms about potential governmental intrusion into the autonomy of universities. The government’s primary focus for higher education reform was on restructuring higher education institutions to establish the UNS, new governance arrangements to support the system, new funding and accountability arrangements, and the reintroduction of student tuition charges through the establishment of the income contingent student loans scheme—the Higher Education Contribution Scheme. However, there were other factors at play as well. There was a growing view in the government and in trade unions and employer bodies that a strong and distinctive national VET system was required to deepen and broaden the skills base of the Australian workforce to help make Australian industry more competitive in world markets.

Learning Outcomes for Credit Transfer: Reflections on the Australian Experience 121

Central to this objective, Australia’s narrow, occupationally based industrial awards were to be transformed to broader skills-based awards, with these awards including job classifications and rates of pay directly aligned to VET qualifications. Critical to this process was that the process for identifying new skills requirements would be driven by industry, that is, by employers and trade unions through the industrial relations process, rather than by education and training providers through courses. To achieve this objective, all VET qualifications were to transition to a Competency Based Training (CBT) model, which was defined and assessed in terms of functional competence directly linked to job roles and workplace performance. Learner outcomes were to be no longer graded; rather, learners were deemed to be “competent” or “not yet competent” and assessed at the unit of competence level. The CBT model contrasted with the existing TAFE course model which, although vocational and occupationally focused in character, was based on course curriculum and learning outcomes for individuals rather than on specific job competencies. The key elements of these reforms were outlined in other major skills and training policy statements which were also released in 1987 (Dawkins and Holding 1987) and in 1988 in parallel with the higher education reform process. Because of these concurrent policy processes, the higher education and VET sectors subsequently followed distinctly different trajectories. The higher education reforms resulted in a fundamental reshaping of the higher education system, primarily within the paradigm of the existing Australian university model, with the associated loss of the more vocationally focused Centres for Adult Education (CAE) sector (with which TAFE had a natural alignment). In VET, a series of major reforms led to the emergence of a new national VET system, the centrepiece of which was a national framework for the recognition of VET providers and qualifications. Under this framework, VET providers registered in one state could operate and issue qualifications in all states, and qualifications or part qualifications were also to be recognized in all states. This entire system was underpinned by the national competency standards which in effect provided the “exchange currency” to establish equivalence in learning outcomes. The national recognition approach in the VET sector differs from credit transfer significantly as national VET qualifications, and learning outcomes are automatically recognized regardless of provider and therefore recognized by all registered providers (rather than by students applying for credit recognition).

122

Peter Noonan

Recognition of Prior Learning (RPL) and Recognition of Current Competence (RCC) were also introduced as part of this system of national recognition. They were regarded as integral to the workforce skills agenda based on equity considerations—the right of existing workers to have current skills and competence recognized—and to efficiency by avoiding the costs of training where the relevant skills already existed. Learners applying for RPL or RCC are assessed against the relevant national competency standards rather than by course curriculum and length. From the outset, the decision to adopt CBT based on industry-defined competency standards as the central characteristic of VET qualifications was strongly contested, particularly by TAFE leaders and practitioners and by the education sector generally. It was contested for two reasons; firstly, it was seen to move the locus for the design and content of qualifications from educational institutions to industry, and secondly, the CBT model, with its task and workplace performance orientation, was seen by its critics as too narrow in terms of the broad learning needs of individual students and in terms of meeting social and community needs outside of the workplace. Guthrie (2009) argues that the CBT model adopted in Australia was strongly based on the functional competency approach in use in the United Kingdom, and based around a system of national vocational qualifications. It was workplace-focused and performance-oriented, like its United Kingdom counterpart. Australia, therefore, drew heavily on the United Kingdom experience and literature, and many of the issues raised about CBT and its implementation had parallels. Like its counterpart, the Australian conception has tended to downplay the importance of underpinning knowledge and a holistic view of the ‘craft concept’ compared with, say, the German and Austrian models of competence. (8)

In its first iteration, competency standards were used to underpin the existing TAFE curriculum course model. TAFE curriculum developers used national competency units to develop TAFE curriculum. However, this was an uneasy arrangement as TAFE course outcomes and assessment requirements continued to be expressed in terms of learning outcomes for individuals, rather than in terms of workplace competency standards. The development and accreditation processes were also unwieldy as competency standards had to be developed and approved and courses then developed and accredited. It also proved to

Learning Outcomes for Credit Transfer: Reflections on the Australian Experience 123

be too time-consuming and expensive to develop curriculum in all of the emerging areas of competency standards. Consequently, in 1996 the decision was taken to directly align competency standards to VET qualifications through National Training Packages. National curriculum (or learning resources) continued to be developed in major VET qualification areas as part of National Training Packages, but its use was not mandated. Rather, it was anticipated that over time VET providers would develop their own learning and assessment resources—an expectation which has largely been unfulfilled. From the outset, universities eschewed direct and narrow linkages between courses and specific workforce roles. In 1992, the AVCC rejected competency-based training, arguing that While the development of knowledge, skills and understanding is central to the role of universities it is not the responsibility of universities to shape and re-shape programs of study in response only to changes in current professional or workforce needs… Competencies are necessary but not sufficient outcomes of university education. (AVCC, quoted in Guthrie 2009, 9)

There were also concerns in the higher education sector about the implications of CBT for cross-sectoral assessments, for example, difficulty in identifying course structures and underpinning skills and knowledge and the loss of graded assessment in VET. (Graded assessment enabled universities to select the highest achieving students where there was competition for higher education places.) Nonetheless, in parallel with the introduction of CBT in VET, from 1993 to 1996 the AVCC undertook a credit transfer project funded by the Commonwealth government. The project involved the development of a set of Credit Transfer Principles that individual universities could consider in setting their own policies. It also included pilots across twelve fields that helped to establish minimum levels of credit for (then) TAFE Associate diplomas into university degrees. However, these pilots were based on existing TAFE curricula-based courses, and as these courses were progressively phased out, the sharp difference between the VET competency model and the higher education discipline-based knowledge model was exposed. In summary, the decision to introduce CBT—as a form of outcomes-based learning in VET—had as one of its central purposes automatic credit recognition within the VET system, but was implemented notwithstanding the reservations in the higher education sector of the

124

Peter Noonan

implications for credit transfer to higher education. Indeed, most of the principal advocates of CBT in VET actively opposed options for the design of VET qualifications to incorporate credit transfer outcomes, arguing that these outcomes did not constitute job roles or relate to workplace performance. In 1997, a report prepared by Coopers & Lybrand (1997) for the then Australian National Training Authority (ANTA), which was established in 1992 to oversee the national VET system, concluded: For VET—higher education, future credit transfer and articulation arrangements may experience additional difficulties, primarily because the change will bring competency standards into clear focus. The tertiary sector remains unconvinced about CBA and retains a lack of confidence in the capacity of CBT to ensure the acquisition of underlying skills and knowledge, which are considered essential for tertiary study. (7)

However, the major priority for the VET system at the time was the rapid development and implementation of National Training Packages across key industry and occupational areas. It was not until 2000 that the AVCC and ANTA jointly commissioned a major study, Pathways to Partnerships (Carnegie 2000), to assess the extent of the impact of the introduction of CBT on credit transfer and articulation. Gaining the support of the AVCC for the study was important, as it signalled a willingness by at least some universities to continue to progress credit transfer as policy priority notwithstanding the reservations in the higher education sector about CBT and National Training Packages. Pathways to Partnership researched the impact of National Training Packages and the VET national recognition framework on cross-sector arrangements. Its objective was to develop a combined ANTA/AVCC national policy on cross-sectoral collaboration. The project included case studies in the Agriculture, Tourism and Hospitality, Community Services, and Telecommunications sectors. The case studies showed a mixed picture in relation to the impact of the introduction of CBT and National Training Packages on cross-sectoral arrangements. However, the results of a survey of institutions, Industry Training Advisory Boards (which at that stage developed National Training Packages), industry groups, and government officials was revealing. Survey outcomes showed overwhelming support for a statement put to respondents that “each sector has a central focus and client base which should be the central driver for qualifications development” (Carnegie 2000, 172).

Learning Outcomes for Credit Transfer: Reflections on the Australian Experience 125

Support for this statement clearly implied that cross-sectoral collaboration was a secondary consideration in qualification design if accommodating the different requirements of each sector compromised the core purpose of each sector. The survey results also showed how a significant majority of the respondents felt: • The introduction of National Training Packages would significantly impact current arrangements; and • That using competencies directly rather than courses is more difficult in developing cross sectoral linkages because there is less understanding and acceptance of this approach in Higher Education. (Carnegie 2000, 186–187) However, in analyzing these results, the study noted that universities had slightly lower levels of agreement to these statements than VET sector providers. This suggested that VET providers anticipated greater problems in terms of views and responses of higher education providers than higher education providers themselves—although perceived problems were high in both sectors. Pathways to Partnerships made specific recommendations to improve National Training Packages in areas such as qualification content equivalence, greater emphasis on higher order technical skill and knowledge, and the development of generic competencies across National Training Packages. However, although some or parts of these recommendations were picked up in the process of the continual improvement of National Training Packages, ongoing resistance to the incorporation of credit transfer outcomes within National Training Packages meant that the recommendations of the Pathways to Partnerships report were never fully or properly considered in the VET sector. Rather, credit transfer considerations continued to be excluded from the guidelines and requirements for National Training Package development. In its response to Pathways to Partnerships, the AVCC supported the thrust of the recommendations, but continued to argue that …There are real difficulties associated with the move away from a curriculum based approach when making cross-sectoral assessments. Universities have highlighted concerns regarding the identification of course structures, underpinning skills and knowledge and graded assessments which could be pursued as part of the ongoing quality assessment processes by the VET sector. (AVCC 2001, 2)

126

Peter Noonan

Sitting behind the AVCC’s concerns was the influence of perceived hierarchies between “higher” education and technical and vocational education offered through TAFE. These hierarchies were evident in the higher education sector itself, between universities with their research missions and roles in preservation and advancement of scholarship in cognate disciplines and the CAEs with their primary focus on applied learning in technical, paraprofessional, and professional disciplines. The absorption of the CAEs into the university system at the same time as the unilateral decision to introduce CBT into the VET system, with its primary focus on industry-defined workplace outcomes, reinforced—rather than reduced—the effect of these traditional hierarchies. Although professional and vocational provision was (and still is) a central role of universities, occupational hierarchies and the strong desire by many professional bodies to maintain and control professional standards often worked against—rather than helped to develop—occupational pathways across the sectors. In any event, the proposed joint policy between ANTA and the AVCC never eventuated. The Evolution of the AQF The Australian Qualifications Framework (AQF) was developed and introduced in 1995. The AQF had as one of its major objectives the establishment of learning pathways, including through credit transfer and articulation. The AQF’s initial form was a twelve-level framework that largely reflected the existing qualifications issued by the VET, higher education, and school sectors. As such it was a relatively weak framework (Wheelahan 2008) that largely replicated (and perhaps even entrenched) the differing approaches between the VET and higher education sectors to qualification development and design. As part of the AQF, a set of principles was developed to assist institutions and systems in the development and implementation of credit transfer and articulation arrangements. However, these principles were set at high levels and were not designed to redress the difference in course outcome definitions. This left in place a situation where national policies aimed at maximizing credit transfer between the sectors through the AQF sat uneasily within a broader policy context that drove differentiation between the sectors, particularly in terms of how learner outcomes were identified and expressed.

Learning Outcomes for Credit Transfer: Reflections on the Australian Experience 127

Addressing the Gap in Qualification Design The Effect of Differences in the Definition of Learning Outcomes The preceding analysis indicates that learning outcomes are defined and expressed differently between the VET and higher education sectors in Australia and that, while the AQF has been effective in promoting credit transfer and in providing guidance to institutions, it does not bridge the gap created by these differences—even in its revised and more coherent form (see below). The key question then arising concerns the extent to which this difference serves as a major barrier to the development and implementation of credit transfer pathways between VET and higher education. The High-Level Review of National Training Packages argued: We do not accept that the reluctance of some universities to offer fair and reasonable credit transfer for VET qualifications is sufficient reason to cause a re-think of the Training Package model. We have the sense that the tide may have turned and that as RTOs and Universities come to understand the Training Package model better, graded assessment is introduced, and the Associate Degree introduced, pathways into higher education will open up. (Schofield and McDonald 2004, 31)

Giving Credit Where Credit Is Due reached a similar conclusion: While some institutions have managed to work very effectively within the Training Package approach, others have found that it brings constraints and limitations. We have concluded that Training Packages themselves are not necessarily the problem, but that some improvements could be made to enhance opportunities for credit transfer. These are issues associated more with conveying information about the Training Packages and with building maximum opportunities for credit transfer into their development than issues associated with the fundamental approach of Training Packages. (Phillips KPA 2006, 30)

Walls and Pardy (2010) highlight the difference between research findings and actual practice when they conclude that “the research highlighted criticism of competency-based training and a refusal to recognize comparable VET learning. As an increasing number of institutional and individual achievements of credit transfer illustrate, such instances are becoming the exception” (25).

128

Peter Noonan

The fact that credit transfer arrangements have been put in place between many individual TAFE institutions and some private providers with universities, as well as some private higher education providers, suggests that the differences in how learning outcomes are defined, qualifications designed, and assessment undertaken does not constitute an insurmountable barrier to credit transfer between the sectors. However, establishing these arrangements required high levels of commitment and trust between partner institutions, particularly to address the inherent differences in approach between the sectors. Moreover, establishing equivalence in learning outcomes using a common framework is no guarantee that credit transfer and credit recognition will easily follow. Content equivalence between courses, even within a common framework, still has to be established, and institutions issuing credit have to assure themselves that learners have achieved the required outcomes. Credit transfer practices within the higher education sector vary significantly, largely reflecting institutional prestige and competition for students, even within a common approach to learning outcomes. Measures to Redress Difference in Learning Outcomes Given that learning outcomes themselves have continued to be defined and expressed differently between the higher education and VET sectors, a range of measures has been identified over time to assist in the process of establishing equivalence for credit or in ensuring that individual learners have equivalent skills and knowledge (where gaps exist). These measures fall into three categories: enhancements to National Training Packages, processes between institutions, and refinements to the AQF. Enhancements to National Training Packages Specific options and recommendations to enhance National Training Packages to facilitate the process of establishing equivalence or of better aligning qualifications have emerged from the major credit transfer studies highlighted above and from other projects. In summary form, major options and themes emerging from these studies include the following: • Use of supplementary learning resource materials that better specify the learning and assessment process used by the VET provider.

Learning Outcomes for Credit Transfer: Reflections on the Australian Experience 129

• Requirements for National Training Package developers to develop cross-sector qualification linkages (at the diploma and advanced diploma levels) as part of the National Training Package development process and for the engagement of higher education staff in this process. • Professional development for staff with expertise and standing in both sectors who can help facilitate the process of qualification mapping. • The development of higher education qualification linkage benchmarks. • Broadening the purpose of VET qualifications and the definition of competence to explicitly require capacity for further learning. • The development of pathway qualifications to higher education not specifically related to occupational outcomes. • Better specification in Training Packages of underpinning skills and knowledge, greater emphasis on higher order technical skills and knowledge, and the development of additional units in generic areas such as maths, writing skills, research, and study skills. • The introduction of graded assessment at least as an option for students considering further studies in higher education. However, there has been ongoing resistance from some key industry bodies and within the former group of Industry Skills Councils to broadening the focus and purpose of Training Packages, to broadening the definition of competence, and to the inclusion of specific measures to improve credit transfer arrangements. The Training Package development template required developers to list credit transfer outcomes, but this is largely an information requirement rather than part of the development process. Processes Between Institutions As there has been little change to the National Training Package model, the responsibility for establishing equivalence for credit transfer has fallen to individual universities and VET providers, principally TAFE institutions. From the case studies undertaken for Giving Credit Where Credit Is Due, a range of measures that institutions and systems have developed to address the gaps between the design of qualifications in the higher education and VET sectors emerged. These measures include the following:

130

Peter Noonan

• The establishment of joint committees and working parties, particularly at the course level, to map the content of Training Packages against higher education courses and to identify areas where credit can be provided as well as where gaps exist. • The development of additional VET learning resources, particularly where gaps in underpinning and disciplinary knowledge are evident or in generic areas such as maths, research, report writing, and critical enquiry. • The development of optional VET units for students wishing to pursue credit transfer pathways. • The use of graded assessment in some VET providers to assist higher education institutions to select the best-prepared VET students. • Transitional or preparatory programs in higher education to address specific gaps in learners’ skills and knowledge or to assist them to transition to the higher education environment. (Phillips KPA 2006) As there have been no systematic reviews or case studies undertaken in recent years, it is difficult to assess the extent to which these and other practices are still employed or widely employed. It is likely that after the initial mapping and negotiation process has taken place, higher education institutions have become more familiar with Training Packages and in assisting students to transition through these pathways; the intensive processes required initially to establish equivalence may no longer be required. A further consideration is that the substantial growth in higher education since 2011 has resulted in much greater diversity in student cohorts entering higher education, with universities needing to adjust their pedagogical practices to accommodate the different backgrounds and varying levels of skills and knowledge of this broader mix of students as a consequence. Assisting students who have been given some level of credit either from VET or from prior learning or experience forms part of this process, perhaps resulting in some universities being less prescriptive about credit requirements than in the past. Revisions to the AQF Since its inception in 1995, there have been important revisions to the AQF to improve credit transfer pathways. In 2004, the associate degree was introduced as a new qualification in the AQF. Similar in concept to

Learning Outcomes for Credit Transfer: Reflections on the Australian Experience 131

the associate degree in the United States and Canada, the associate degree in Australia was designed to sit alongside VET diplomas and advanced diplomas but to map directly and with credit into degree programs. The TAFE sector advocated strongly for the associate degree to be a VET qualification outside of the National Training Package and CBT model. However, resistance to this proposal from advocates of the existing National Training Package model in the VET sector on the one hand and from universities on the other led to the inclusion of the associate degree within the AQF as a higher education qualification. This has significantly limited its potential use as an instrument for credit transfer from VET to higher education, as although VET providers offer associate degrees, they must be accredited as higher education providers to do so. As such, the associate degree has been used for credit transfer within the higher education sector, but associate degree enrollments have remained low since its introduction and are not a major feature of the Australian tertiary education landscape. Major changes were made to the AQF itself in 2011 following findings and recommendations from reviews that highlighted the inconsistencies in how qualifications were conceived and described in the AQF. The 2008 Review of Australian Higher Education, known as the Bradley Review (Australian Government and Bradley 2008), was particularly important in this regard, as the government’s response to the Bradley Review provided the authorizing environment required to effect fundamental changes to the AQF. The 2011 changes to the AQF introduced consistent learning taxonomies and levels across the qualifications in the AQF in the form of learning outcomes across the domains of knowledge, skills, and the application of knowledge and skills. The separate qualification descriptors for VET and Higher Education diplomas and advanced diplomas have also been replaced by a common set of descriptors. The National Qualifications Pathway Policy was also developed as part of the revised AQF. While primarily focusing on the obligations of institutions in terms of policies, provision, and information on credit transfer, the policy also recommended the following levels of credit be adopted by institutions: • 50 percent credit for an Advanced Diploma or Associate Degree linked to a three-year bachelor’s degree; • 37.5 percent credit for an Advanced Diploma or Associate Degree linked to a four-year bachelor’s degree;

132

Peter Noonan

• 33 percent credit for a Diploma linked to a three-year bachelor’s degree; and • 25 percent credit for a Diploma linked to a four-year bachelor’s degree (AQFC 2013, 79). While the revised AQF provides a much stronger and coherent basis for the development and alignment of qualifications, in practice the traditional drivers of qualification development and design in the VET and higher education sectors remain. This continues to limit the effectiveness of the AQF as a tool for the design of more coherent qualification pathways between the VET and higher education sectors. Future Possibilities and Challenges Notwithstanding the differences in the purpose and design of qualifications highlighted in this report, the notion that competency-based learning models clearly differentiate VET and higher in Australia is a not as straightforward as it seems. Graduates from many higher education courses must meet standards set by regulatory and professional bodies to legally work in many professions (law and many health professions). Even where standards are not legally enforceable, membership of the relevant professional body serves as a de facto requirement for employment in the profession (engineering and accounting). Standards have been developed to create greater transparency in the assessment of professional and occupational capability and to move away from input measures such as course length, course completion, or professional experience. Some of these professional standards are based on the same essential structure as VET competency standards: unit descriptors, elements, and performance criteria (Australasian Professional Legal Education Council 2000). Others use a simpler structure (unit title and performance criteria) and include essential knowledge as well as performance teaching requirements (Australian Institute for Teaching and School Leadership, n.d.). The application and enforcement of professional standards are increasing; as accountability for professional performance rises, risk management and contractual requirements will begin to legally enforce decisions to deregister individuals from professional practice. For example, all states in Australia have now adopted mandatory national standards for teaching. Professional standards are also used for the initial assessment of applicants for skilled migration and work visas and for the right to work

Learning Outcomes for Credit Transfer: Reflections on the Australian Experience 133

in Australia in the relevant profession or occupation. However, the difference between higher education and VET in the use of competency or professional standards is that while relevant higher education courses are designed to ensure that learners can meet professional standard requirements, they have broader purposes and are not directly and rigidly aligned to the relevant competency standards. There has also been a progressive and intensifying shift to outcomes-based learning in Australian higher education, consistent with international trends that emphasise a move away from input to outcome-based measures (Trembley, Lalancette, and Roseveare 2012, 36). This trend has been driven by several factors: a focus on graduate attributes and capabilities, better alignment with professional and occupational standards, greater accountability for outcomes, the rise of rankings and international benchmarking systems, and the development of standards for the accreditation of higher education providers. In principle, it should be possible to revisit the many proposals for the enhancement of National Training Packages and to consider the development of vocational qualification pathways that span the VET and higher education sectors particularly: If we begin to view vocational education and training as a continuum of offerings through distinctive forms of qualifications, requiring distinctive pedagogy and assessment, then a view about the future of VET, grounded in a contemporary definition and understanding of occupational and professional competence can emerge. This will create the potential for different forms of institutions and types of qualifications across providers in the broader tertiary education sector. (Noonan 2010, 19–20)

These developments are most likely to work where there are tight linkages between occupations and career structures which span the VET and higher education sectors, for example, in areas such as nursing, child care and early learning, allied health, agriculture, and potentially financial services (Yu, Bretherton, and Buchanan 2013). However, at present, there is little policy priority in either the higher education or VET sectors to pursue these possibilities, with each sector largely preoccupied with its own policy challenges. Future arrangements for credit transfer between VET and higher education may also be challenged by current and emerging changes in the policy, funding, and the regulatory landscape for higher education in Australia. Specifically, there has been a subtle but important shift

134

Peter Noonan

in emphasis between the 2015 Higher Education Threshold Standards and the initial 2011 Threshold Standards as they relate to credit transfer. The 2011 Threshold Standard required that: The higher education provider ensures that it maintains processes to provide for the recognition of prior learning, credit transfer and articulation of awards. These processes are designed to maximise the credit students may gain for learning already undertaken [emphasis added], subject to preserving the integrity of learning outcomes and/or discipline requirements of the award to which it applies. (Department of Industry, Innovation, Science, Research and Tertiary Education 2011, section 3)

The 2011 Threshold Standard reflected the intent of the AQF Pathways Policy to maximize credit students may gain. However, the 2015 Threshold Standards removed the requirement for an accredited higher education institution to maintain processes designed to maximize the credit students may gain and instead stipulated: Credit through recognition of prior learning is granted only if: a. students granted such credit are not disadvantaged in achieving the expected learning outcomes for the course of study or qualification, and b. the integrity of the course of study and the qualification are maintained. (Birmingham 2015, section 1)

This important change reflects two drivers: a focus in the revised standards on the student “lifecycle” such that standards are organized and grouped across the student experience (from admissions to certification) and the need for the standards to be clear and enforceable, including the basis upon which students are admitted to higher education courses. The Threshold Standard relating to student admission requires that “admitted students have the academic preparation and proficiency in English needed to participate in their intended study, and no known limitations that would be expected to impede their progression and completion” (Birmingham 2015, 3). The effect of this shift in emphasis on the level of credit transfer in Australia is difficult to assess. The advisory board established to oversee the AQF, which had played a major role in its reform and in developing national credit transfer policies and principles, has been dis-

Learning Outcomes for Credit Transfer: Reflections on the Australian Experience 135

banded, and there is no national agency with specific responsibility for monitoring and advising on cross-sectoral pathways or credit transfer. National policy objectives for credit transfer have sat uneasily within an increasingly fractured tertiary education system that has seen significant growth in funding and enrollments in higher education and declining funding and enrollments in VET (Noonan 2015; Fowler 2017). Reputational damage to the VET sector from inappropriate practices of some private VET providers has exacerbated these structural anomalies between the VET and higher education sectors and has led to caution in the higher education sector about providing universal credit for VET students in relevant courses. Rather, credit will in the main continue to be limited to graduates from institutions where universities have formal agreements in place, meaning that credit availability will be limited to students enrolled with some providers and not others. The introduction of demand-driven funding for higher education since 2012 in Australia has enabled universities to directly enroll students who in the past may have enrolled with credit through VET pathways. It has also allowed universities to enroll greater numbers of students using VET as the basis of admission than when higher education places were capped. The number of students admitted to university on the basis of previous VET studies rose from 26,167 in 2011 to a 35,297 in 2014 before a slight decline to 34,987 in 2015 as higher education enrollments plateaued. However, overall higher education commencements rose significantly over this period from 489,917 in 2011 to 569,065 in 2015 (including international students), and VET partnerships remain an important strategic priority for them. However, the Commonwealth government has decided to effectively end the demand-driven funding system for higher education by placing financial caps on universities for government-funded undergraduate places for 2018 and 2019 and then only providing additional funding across all universities commensurate with population growth. Universities can enroll additional students within that cap, and some may look to partnerships with VET providers, particularly TAFE. This would allow for the expansion of enrollments through credit-based learning pathways as a lower-cost delivery option, for example, by having the first two years of a degree delivered and funded through the VET sector, with full credit toward a university degree (building on existing examples). For VET providers with a major focus on diplomas and advanced diplomas, guaranteed credit-based pathways into higher education through partnerships with universities will also be an attrac-

136

Peter Noonan

tive proposition. However, higher education provision by VET providers is not likely to grow significantly (as VET providers are not likely to attract government higher education funding due to the requirements of the Higher Education Threshold Standards). The government has also announced a review of the AQF. This review creates the potential for a renewed focus on credit-based learning pathways. Peak bodies representing TAFE institutes and private VET providers are also likely to argue for leading providers to be able to develop and self-accredit courses outside of the CBT system of VET qualifications, creating the potential for co-designed pathways between VET and higher education institutions. The government has also foreshadowed a review of the current higher education providers’ categories that define the different types of higher education providers. Some commentators, including TAFE Directors Australia (the peak body representing TAFE institutions), have argued for the establishment of “teaching only” tertiary education institutes spanning the VET and higher education sectors. This model would shift the focus from credit transfer from VET to higher education to credit transfer within the higher education sector, a process that could be far more easily undertaken within a common framework of qualification design and learning outcomes. However, the financial caps now placed on the higher education sector would seem to limit the scope for the development of new non-university higher education providers, and renewed focus credit-based pathways between VET and higher education is the most likely outcome from these reviews. The Australian Labor Party (the major opposition party) has announced that it will undertake a widespread review of postsecondary education spanning the VET and higher education sectors. While the terms of reference for the review have not been released, it is clear that the review could significantly transform the landscape of tertiary education in Australia. The focus may well be on the future skill needs of the Australian workforce, including the capacity for people to flexibly move across the tertiary system and through a renewed role for TAFE in that system. These developments all suggest that there will be a renewed focus on credit-based learning pathways from VET to higher education, and even from higher education to VET. However, the benefits of this focus will only be realized if a more consistent and coherent approach to defining, developing, and assessing learning outcomes between the higher education and VET sectors is developed, understood, and widely used.

Learning Outcomes for Credit Transfer: Reflections on the Australian Experience 137

References Australasian Professional Legal Education Council. 2000. Competency Standards for Entry Level Lawyers. https://www.leocussen.edu.au/ wp-content/uploads/2016/11/Competency_Standards_for_Entry_ Level_Lawyers-1.pdf Australian Government, and D. Bradley. 2008, December. Review of Australian Higher Education. Final Report. https://www.mq.edu. au/__data/assets/pdf_file/0013/135310/bradley_review_of_australian_higher_education.pdf Australian Institute for Teaching and School Leadership. n.d. Australian Professional Standards for Teachers. https://www.aitsl.edu.au/teach/ standards Australian Skills Quality Agency. n.d. Training Packages. https://www. asqa.gov.au/about/australias-vet-sector/training-packages Australian Qualifications Framework Council (AQFC). 2013, January. AQF Qualifications Pathways Policy. Second Edition. https://www.aqf. edu.au/sites/aqf/files/aqf-2nd-edition-january-2013.pdf Australian Vice-Chancellors Committee (AVCC). 2001. Cross-Sector Qualification Linkages between Higher Education and VET: AVCC Response to the VETASSESS Report “From Pathways to Partnerships.” https://www.voced.edu.au/content/ngv%3A33618 Birmingham, S. 2015, October 7. Higher Education Standards Framework (Threshold Standards) 2015. https://www.legislation.gov.au/Details/ F2015L01639/9f923e9b-a57e-42d4-b0b6-12f928e387a0 Carnegie, J. 2000. Pathways to Partnerships. ANTA/AVCC Report and Draft Policy Guidelines. https://www.voced.edu.au/content/ ngv%3A19310 Charles Sturt University. 2018, January 30. “Information for Future Students.” http://futurestudents.csu.edu.au/study-options/pathways/tafe/guaranteed-entry Chesters, J., L. Watson, and P. Hagel. 2013, November 6. A Half-Open Door: Pathways for VET Award Holders into Australian Universities. https://www.ncver.edu.au/research-and-statistics/publications/ all-publications/a-half-open-door-pathways-for-vet-award-holdersinto-australian-universities Coopers & Lybrand. 1997. Impact of Competency Based Assessment on Credit Transfer and Articulation Arrangements: Final Report to the Australian National Training Authority. https://www.voced.edu.au/content/ngv%3A42892

138

Peter Noonan

Dawkins, J. S. 1987. Higher Education: A Policy Discussion Paper. Canberra, Australia: Australian Government Publishing Service. ———. 1988. Higher Education: A Policy Statement. Canberra, Australia: Australian Government Publishing Service. Dawkins, J. S., and A. C. Holding. 1987. Skills for Australia. Canberra, Australia: Australian Government Publishing Service. Department of Industry, Innovation, Science, Research and Tertiary Education. 2011, December. “Higher Education Standards Framework (Threshold Standards) 2011. https://www.legislation.gov.au/Details/F2013C00169 Fowler, C. 2017. The Boundaries and Connections between the VET and Higher Education Sectors: “Confused, Contested and Collaborative.” Adelaide, Australia: NCVER. Guthrie, H. 2009. “Competence and Competency-Based Training: What the Literature Says.” Adelaide, Australia: NCVER. Noonan, P. 2010. “The Future of VET: The Case for a New VET Settlement.” In The Future of VET: A Medley of Views, edited by F. Beddie and P. Curtin, 16–20. Adelaide, Australia: NCVER. http://www.academia.edu/16553638/The_Future_of_VET_a_medley_of_views ———. 2015, October 28. A Model for Tertiary Education Funding in Australia. (Presentation transcript). http://www.mitchellinstitute.org. au/presentations/a-model-for-tertiary-education-funding-in-australia/ Phillips KPA. 2006, June. Giving Credit Where Credit Is Due: A National Study to Improve Outcomes in Credit Transfer and Articulation from Vocational and Technical Education to Higher Education. Final Report. http:// www.educationcouncil.edu.au/site/DefaultSite/filesystem/documents/Reports%20and%20publications/Archive%20Publications/ Higher%20Ed/Giving%20Credit%20Where%20Credit%20is%20 Due.pdf Schofield, K., and R. McDonald. 2004, April. Moving on… Report of the High Level Review of Training Packages. https://digitised-collections. unimelb.edu.au/bitstream/handle/11343/115460/scpp-00185nat-2004.pdf?sequence=1 Trembley, K., D. Lalancette, and D. Roseveare. 2012. Assessment of Higher Education Learning Outcomes. Paris, France: Organisation for Economic Co-operation and Development. http://www.oecd.org/education/skills-beyond-school/AHELOFSReportVolume1.pdf Walls, S., and J. Pardy. (2010, May 5. “Crediting Vocational Education and Training for Learner Mobility.” https://www.ncver.edu.au/re-

Learning Outcomes for Credit Transfer: Reflections on the Australian Experience 139

search-and-statistics/publications/all-publications/crediting-vocational-education-and-training-for-learner-mobility Wheelahan, L. 2008. “Neither Fish nor Fowl: The Contradiction at the Heart of Australian Tertiary Education.” Journal of Access Policy & Practice, 5 (2): 133. https://research-repository.griffith.edu.au/bitstream/handle/10072/20801/50501_1.pdf%3bsequence=1 Yu, S., T. Bretherton, and J. Buchanan. 2013, November 26. “Defining Vocational Streams: Insights from the Engineering, Finance, Agriculture and Care Sectors.” https://www.ncver.edu.au/research-and-statistics/publications/all-publications/defining-vocational-streams-insights-from-the-engineering,-finance,-agriculture-and-care-sectors

6

International Student Mobility Based on Learning Outcomes and Workload: The European Credit Transfer and Accumulation System Robert Wagenaar

Introduction The European Community Action Scheme for the Mobility of University Students, in short, the ERASMUS Programme, established in 1987 was the start of a transfer program for students and teachers that has not been equalled anywhere else in the world. Its main intention was (and is) to promote the European integration process and to develop a European mindset. From the beginning it was well understood that such a transfer scheme presumed a theoretical and conceptual foundation. This became the European Credit Transfer and Accumulation System (ECTS), which was initiated as a six-year pilot, but was widely implemented from 1995 on. As a result of the Bologna Process that started in 1999, ECTS gradually transformed from a transfer system (based on the notion of student workload) into a transfer and accumulation system. This change was initiated by the European Commission-supported grassroots initiative Tuning Educational Structures in Europe (founded in 2001). It implied a change of paradigm from exLearning Outcomes, Academic Credit, and Student Mobility, edited by Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon. Montréal and Kingston: McGill-Queen’s University Press, Queen’s Policy Studies Series. © 2020 The School of Policy Studies, Queen’s University at Kingston. All rights reserved.

142

Robert Wagenaar

pert-driven to student-centred education, requiring new models for the design and delivery of educational programs. The need was also felt to develop conceptual frameworks based on the concept of competences and learning outcomes. This resulted in general overarching European qualifications frameworks and Tuning subject area frameworks. These were thought necessary to guarantee not only the comparability and compatibility of degree programs, but also to respect minimum standards. After a construction phase of around seven years, an implementation phase of the Bologna Process followed; that is, the actual application of the learning outcomes approach as an integral part of the paradigm change. From 2010 awareness arose that the desired reforms were not taking place in the majority of European countries as widely or as in depth as was intended. This was confirmed by Bologna Process progress reports. The disconnect between the desired reforms and the actual situation triggered Tuning to take a bold new initiative in 2016— in close consultation with the European Commission—to implement a feasibility study on Measuring and Comparing Achievements of Learning Outcomes of Higher Education in Europe (CALOHEE), of which so-called assessment frameworks are the basic instrument. Evolution of the European Credit Transfer System Some thirty years ago, there was reason to celebrate at the premises of the European Commission, in particular its Task Force Human Resources, Education, Training and Youth. For a decade it had worked very hard to develop and launch the ERASMUS Programme. Its establishment was not at all self-evident, with education being perceived as the prime responsibility of the national states. A number of factors played a decisive role to convince the member states to agree with the action scheme. First, there was the success of the small-scale European Communities (EC) 1976 Action Programme, which concentrated on setting up transnational joint study programs, short study visits, and university networks, and involved some five hundred higher education institutions by 1984. Second, in the mid-1980s discomfort was felt by policymakers, resulting from growing political cynicism about the EC. It was thought necessary to underpin economic policies—that is, the development of one single market—by highlighting the cultural dimension of the EC as a binding factor for integration and, in this setting, to give a more prominent place to education (Corbett 2005; European Commission 2006).

International Student Mobility Based on Learning Outcomes and Workload 143

In 1984 the European Council (Commission of the European Communities 1985) published a call to “strengthen and promote the European identity and image both for its citizens and the rest of the world” (Adonnino 1985, 5). As a follow-up to this call a high-level ad hoc committee, which represented the heads of state, was established on “People’s Europe,” chaired by the Italian Pietro Adonnino. This committee was given the assignment to come up with concrete initiatives within half a year “involving the citizens of Europe more determinedly in the construction of the Community” (Adoninno 1985, 31). In its final report published in June 1985, it proposed not only a comprehensive program of European inter-university exchanges and studies open to a significant part of EC students, but also the development of a European academic credit transfer scheme to facilitate mobility (Commission of the European Communities 1985). It took a further two years to get the mobility program in place. The basic assumption of setting up the resulting ERASMUS Mobility Scheme was to stimulate a European mindset among new generations of students. In terms of numbers, the program developed over time into one of the most successful flagship programs of the European Communities, since 1993 named the European Union. From its start in the academic year 1987–1988, nearly four million students obtained a mobility grant, starting with some 3,244 students in the first year, and growing in the academic year 2013–2014 to 212,208 (European Commission 2015a, 2017). This initiative became the main driver for the internationalization of higher education, but also a means for reform. Indispensable for this last aspect proved to be the establishment of the ECTS. To develop this system a five-year pilot project was set up in 1988, to be launched one year later. In retrospect this pilot can be seen as the start of a process that would result in a real and widely accepted European Credit Transfer and Accumulation System (abbreviated as ECTS) based on two integrated parameters: student workload and learning outcomes. This reflects the paradigm shift (to be) made from staff-driven to student-centred education and active learning (González and Wagenaar 2006; Wagenaar 2006). With the main motive identified for setting up a large-scale mobility program—that is, contributing to the European integration process by developing a European spirit and identity among students—the focus of this chapter is on how this was accomplished. Which steps were taken to develop a theoretical and conceptual framework and methodology thought conditional for a successful system for transfer and recogni-

144

Robert Wagenaar

tion of periods of studies in the European Union, later the much larger group of European countries involved in the Bologna Process? This is a process which aimed to develop a globally competitive and attractive European Higher Education Area (EHEA). What were the underlying assumptions and the implications of the policies that were defined to arrive at the aimed results? ERASMUS, ECTS, and Workload Immediately after the launch of the ERASMUS Mobility Scheme it became clear that recognition of studies after transfer would become an issue as a result of the variety and diversity of higher education programs in Europe. This is also what the Adonnino ad hoc committee foresaw. Being aware of the sensitivity of the issue at stake, it phrased its proposal rather carefully so as to “examine the possibility of introducing a European system of academic credits transferable throughout the Community (European Academic Credit Transfer System). This system would be implemented by means of bilateral agreements or on a voluntary basis by universities and higher education establishments which, by arrangement with one another, would determine the procedures for academic recognition of such credits” (Commission of the European Communities 1985, 25). In a note it offered a sort of blueprint: Under this system, which has proved its worth in the United States, each course followed in a higher-education establishment as part of the normal curriculum entitles the student to a credit which can be taken into account in other establishments at a similar level or re-assessed with other credits to entitle him to a diploma or certificate corresponding to the whole of the studies thus undertaken. (Commission of the European Communities 1985, 25)

In developing the design for the ECTS pilot project, the European Commission and its administrative body for education, the ERASMUS Bureau, decided after initial research not to opt for the United States Carnegie System, based on the so-called credit hour. Instead, it chose a more recent approach (applied successfully in the northern part of Europe since the second half of the 1970s or the first half of the 1980s, depending on the country) to base a credit system for higher education on “typical student workload” and not on “contact hours” as the measurement instrument for transfer and recognition. This was thought to be a more flexible and reliable approach to organizing student mobility.

International Student Mobility Based on Learning Outcomes and Workload 145

It has to be stipulated that, besides northern Europe, no other European countries had a credit system in place (Task Force Human Resources, Education, Training and Youth 1993). A model was decided on in which sixty credits represented the workload of all educational activities (contact hours, laboratory work, placements, self-study, etc.) to be executed in a formal program during a full academic year. This was a unique number, not used anywhere else yet, that would allow for easy calculation of credits for trimesters and (half) semesters. Higher education institutions were invited to participate on a voluntary basis in the ECTS pilot, which covered five different disciplines. After a selection process, initially, it involved eighty-one institutions and three consortia, which grew to 146 in 1992 after a second call and selection round (Task Force Human Resources, Education, Training and Youth 1993). Success of the pilot was not guaranteed, with doubts being expressed both within the EC and the higher education world that the development of a comprehensive European-wide system was feasible. A key principle of mutual trust between institutions was identified, with a focus on respect for the academic decisions of the other and prior agreement on the content of the exchange programs of students. This principle was supported by transparency tools such as the development of course catalogues according to a fixed format, and a system of learning agreements and transcripts of records for outgoing and incoming students to document the achievements of students. These instruments still stand today and have not substantially changed. From the very start it was intended that mobility students should participate in regular courses of the “host” institution, not in special programs. Besides developing a sustainable credit system for transfer, the main contribution of the pilot scheme was indeed the building of trust and confidence between partners from different countries. Although the variation in reputations of higher education institutions was acknowledged, it became evident that there were no substantial differences in the quality of outcomes of the learning process, although the approaches toward teaching, learning, and assessment differed, in particular between the northern and southern parts of Europe. Already in 1992 the Commission engaged Coopers & Lybrand to carry out a mid-term evaluation to see whether a further extension of the use of ECTS should be recommended. Although the Commission came up with a list of recommendations, including one to arrange for initial funding for the introduction of the system, further extension was thought realistic (Task Force Human Resources, Education, Training

146

Robert Wagenaar

and Youth 1993). Indeed, in the years after the termination of the pilot phase, ECTS was rolled out to many more institutions with the support of a growing team of ECTS (academic) counsellors. Ministerial Declarations and Tuning To make a long story short, ECTS got a serious boost as a result of the Sorbonne Declaration, a joint declaration on harmonization of the architecture of the European higher education sector, signed in 1998 in Paris by the ministers of education of the four largest European Union countries. The basic idea underpinning this document was to make European higher education more efficient and more competitive by introducing a two-cycle structure comparable to the Anglo-Saxon bachelor-master model (Ravinet 2005; Sorbonne Joint Declaration 1998). This initiative triggered the formulation of the Bologna Declaration, which was signed by twenty-nine European countries in 1999 and is seen as the starting point of a process which is still continuing today. Identified as one of the six objectives was the “establishment of a system of credits—such as in the ECTS system—as a proper means of promoting the most widespread student mobility” (Bologna Declaration 1999, 3). In practice, the “such as” disappeared soon in the ongoing discourse. Furthermore, the document stipulated the necessity to promote European dimensions in higher education, in particular with regard to curricular development, inter-institutional cooperation, mobility schemes and integrated programs of study, and training and research (Bologna Declaration 1999; Gaston 2010). In the meantime, a growing concern had developed among the group of ECTS counsellors about the lack of recognition of studies taken at another higher education institution. In 2000, at one of the regular meetings of the team, it was noted that many institutions upheld the practice that the content of units were compared one by one to decide whether there was sufficient overlap for recognition. This was contrary to the ECTS philosophy, which promoted that periods of studies should be taken as the basis for recognition, allowing for flexibility and diversification of (the content of) learning. After further discussion this resulted in the establishment of the Tuning Educational Structures in Europe project, co-financed by the European Commission and the higher education institutions involved. This project was designed as a grassroots initiative to give academics a say in the process initiated by the Bologna Declaration. In other words, its agenda stressed the modernization of

International Student Mobility Based on Learning Outcomes and Workload 147

higher education programs in Europe. It defined four aims: (1) develop tools to identify professional profiles and desired outcomes in terms of knowledge, skills and competences in five subject areas to bring about a high level of Europe-wide convergence in higher education; (2) facilitate transparency by introducing a common credit accumulation system (ECTS) by restructuring the present transfer system; (3) develop a model curriculum structure for each area, enhancing the recognition and European integration of diplomas; and (4) ultimately, promote mobility of students and teachers (University of Groningen 2000). The project obtained the full support of the European Rectors’ Conferences as well as the national authorities of the European Union member states. The Tuning agenda was aligned with existing European Union policy documents in which concern was expressed about the present content of higher education programs and the actual needs of society, in particular the labour market (European Commission 1991, 1997). Tuning took a wider perspective by expressing the intention to also cover personal development and active citizenship. For this reason, it introduced the notion of “generic competences” to be made explicit together with the “subject-specific competences” for each of the disciplines involved. Competence is to be understood here as a dynamic combination of cognitive and meta-cognitive skills; knowledge and understanding; interpersonal, intellectual, and practical skills; and ethical values. In Tuning terms, level of competence—making someone more competent as the result of a learning process—should be expressed as (intended and achieved) learning outcomes. As a consequence, this approach started to promote a student-centred approach replacing the staff-driven one, implying that a paradigm shift regarding teaching, learning, and assessment would be required (Barr and Tagg 1995; González and Wagenaar 2003; Tagg 2003). Tuning introduced an eight-step model, which at a later stage was redefined into a ten-step model, to (re)design and enhance degree programs (González and Wagenaar 2006; Lokhoff et al. 2010). The result of all this was the development of a language to be understood and shared by all stakeholders, in particular the educational sector. This was thought necessary as an instrument for the transfer and recognition of periods of studies, including full degrees. To contribute further to the enhancement and relevance of higher education as well as to making study programs compatible and comparable—a necessity for developing one European Higher Education Area—it introduced the notion of common reference points for disciplinary fields, which

148

Robert Wagenaar

in Tuning are named subject areas. This required the development of comprehensive sets of descriptors, which were gradually published from 2007 after a process of validation. The publications, entitled Reference Points for the Design and Delivery of Degree Programmes in…, each covered a set of items: (1) introduction of the subject area; (2) map of typical degrees in the field; (3) map of typical occupations of graduates; (4) level cycle descriptors framed in terms of competences to develop and learning outcomes to be achieved; (5) student workload and ECTS; (6) most appropriate approaches regarding teaching, learning and assessment; and (7) quality enhancement (Tuning Educational Structures in Europe, n.d.). The underlying idea was that by having a common—well-accepted—methodology for program design, delivery, and enhancement, as well as a shared set of documents in place as reference points (comparable to the United Kingdom Quality Assurance Agency benchmark papers), recognition of studies would be significantly improved. The identification of competences to be developed in each program, as well as the consequential formulating of these in learning outcomes to be achieved, should allow for the recognition of qualifications and periods of study without applying course-to-course comparison (which is very inefficient anyway). It would also allow for much more flexibility in what should be learned and how it should be taught, learned, and assessed in meeting internationally agreed learning outcomes descriptors. Initially, Tuning chose to develop reference points for traditional disciplinary academic fields, to allow for easy identification in and between countries. Over time its approach was extended to regulated professions, such as nursing and medicine, and to multidisciplinary and interdisciplinary fields of study, such as European studies and gender studies. Also, frameworks were made for disciplines belonging to the domain/sector of “performing and creative disciplines,” such as music, theatre, and fine arts (Tuning Educational Structures in Europe, n.d.). Learning Outcomes and Conceptual Frameworks In the setting of Tuning, ECTS was turned from a transfer system into a transfer and accumulation system. This revision was made possible by no longer focusing on student workload only but by also focusing on the assessed outcomes of the learning process. In other words, to be awarded credit, students were required to meet the learning outcomes

International Student Mobility Based on Learning Outcomes and Workload 149

set for each course unit. This key feature would allow for obtaining credit for any type of learning unit, including placements, laboratory work, etc., after summative but also formative assessment. It also introduced the notion that ECTS should be applied as a tool for planning, designing, and implementing degree programs to facilitate the feasibility of studies and quality enhancement, including the tracking of student success. When developing ECTS as a transfer system—in the first half of the 1990s—the notion of “relative values” of credits was applied, “reflecting the quantity of work each course demands in relation to the total quantity of work required to complete a full year of academic study at a given institution” (González and Wagenaar 2006, 228). This allowed for a patchwork of modules having different weights. By extending the system to an accumulation system, credits obtained—by definition having “absolute value”—constituted fixed building blocks of a degree program (González and Wagenaar 2006). As a result, the extension also initiated discussions about modularization and the accumulation of credits in so-called learning strings to avoid patchwork. By now, it seems around 90 percent of degree programs in EU countries are more or less modularized (Center for Higher Education Policy Studies 2010; European Union Association 2010). The implication of an accumulation system is that credits based on the learning outcomes approach relate not only to the amount but also to the level of work to be established. A reference can be made here to the many taxonomies that have been developed over time, such as the one developed by Benjamin Bloom, who distinguishes six levels (Anderson and Krathwohl 2000; Bloom, Engelhart, Furst, Hill, and Krathwohl 1956). Tuning developed an approach that would allocate credits to courses and use ECTS as an instrument for the planning and designing of curricula, as well as determine the workload of a typical learner for each unit (González and Wagenaar 2006). In August 2004 the first ECTS Users’ Guide was published, based on the notions of student workload and learning outcomes (European Commission 2004). In later editions of the Users’ Guide the key features defined in the guide were not substantially changed. This might not come as a surprise, given the high level of consistency of members of the group of overlapping ECTS and Tuning experts responsible for the successive editions of the guide. The most current version, published in 2015, has been endorsed by all countries involved in the Bologna Process (European Commission 2015b). They have by now included ECTS in their higher education leg-

150

Robert Wagenaar

islation. That does not mean that the higher education institutions in those countries have all implemented ECTS according to the features of a learning outcomes/student workload-based system. A number of studies show that higher education institutions and their academics still struggle with the concept of learning outcomes. However, the notion that programs should be based on student workload is generally accepted, although here also the higher education institutions in a number of countries are still challenged with the concept and its correct implementation (European Commission, Education, Culture and Audiovisual Executive Agency 2015; European Student Union 2015; European University Association 2015). Having said this, ECTS is by now the unchallenged instrument to organize transfer and recognition of studies, although the state of implementation in different higher education institutions and countries differs. This is confirmed by the 2014 ERASMUS impact study (European Commission 2014). Its large-scale survey shows that 96 percent of the higher education institutions list “recognition of ECTS as the most important aspect of organizational framework of mobility” (19), and 91 percent of the institutions claim that they have implemented recognition of ECTS credits. Tuning developed its conceptual frameworks for subject areas, dubbed reference points (not standards) for the design and delivery of degree programs, which were launched in May 2001 at the same time as European-level initiatives were undertaken. A group of policymakers and quality assurance experts close to or overlapping the Bologna Follow-Up Group—that is, civil servants charged with overseeing the implementation of reforms agreed by European ministers of higher education—realized early in the process that the Bologna agenda of implementing a higher education system essentially built on two main cycles, and for easily readable and comparable degrees, would never succeed without clear descriptors. These should define shared minimum quality standards of what should be the outcomes of learning— in general terms—of a first- and second-cycle program (BA and MA). This led to the informal Joint Quality Initiative. In a short time span, it developed general descriptors for first and second cycle programs and at a later stage also for two-year associated degree (as part of the first cycle) and third-cycle or doctoral studies. These together became known as the “Dublin Descriptors,” named after the place where they were initially drawn up (Joint Quality Initiative 2004; Leegwater 2015). In March 2002, at the Amsterdam Bologna seminar, meant to develop input for the next meeting of ministers of education in the framework

International Student Mobility Based on Learning Outcomes and Workload 151

of the Bologna Process, it was concluded that there is “a widely-shared consensus that the ‘Dublin Descriptors,’ defining key outcomes for Bachelors and Masters programs in general…are useful. These generic descriptors are complementary to the more specific outcomes of the Tuning project…which have been developed at the level areas of knowledge (‘disciplines’)” (Westerheyden and Leegwater 2003, 97). This became official from 2003, when the ministers asked to develop national and overarching qualifications frameworks. “First and second cycle degrees should have different orientations and various profiles in order to accommodate a diversity of individual, academic and labor market needs,” and qualifications should be described, according to the ministers, in terms of “workload, level, learning outcomes, competences and profile,” borrowed from the Tuning terminology (Conference of European Ministers Responsible for Higher Education 2003, 4). The outcome of all this was the Qualifications Framework for the European Higher Education Area (EHEA), in which the Dublin Descriptors made up the core (Bologna Working Group on Qualifications Frameworks 2005). It was endorsed by the ministers at their Bergen meeting in 2005, at which also the European Standards and Guidelines for Quality Assurance were confirmed. These, defined under the direction of the European Association for Quality Assurance in Higher Education, have since become the cornerstone for quality assurance and accreditation in most of the European countries. In 2015 they were updated and endorsed again by the ministers of education (European Association for Quality Assurance in Higher Education 2005, 2015). To complicate matters, the European Commission initiated during the same years the development of an overarching European Qualifications Framework for Lifelong Learning (EQF for LLL), in an attempt to combine vocational education and training and higher education in one framework. This framework was officially endorsed by the European Parliament and the Council of Ministers in 2008 as a formal “recommendation” to the member states (European Commission 2008). As a result, higher education institutions and also Tuning were confronted with two competing frameworks, of which only the qualifications framework for the EHEA was (and is) based on ECTS credits. Although the Bologna Follow-Up Group concluded that the frameworks were compatible, this was actually more wishful thinking than reality. The founding philosophy differs (Bologna Follow-Up Group 2007). While the EQF’s main concern is the application of knowledge and skills in society, the focus of the qualifications framework for the EHEA is more

152

Robert Wagenaar

related to the learning process itself—it applies descriptors which cover different areas or “dimensions” of learning: knowledge and understanding; application of knowledge and understanding in relation to problem solving, making judgments, communicating information, conclusions, etc.; and learning capability. Notwithstanding the differences, by having theoretical or conceptual frameworks in place a sustainable base was created to assess and compare the quality of learning in (trans)national perspective, a condition for fair transfer and recognition of studies. This was and should be perceived as a major step forward. Noting that the gap between overarching qualifications frameworks and disciplinary frameworks was rather wide, Tuning took the initiative (again supported by the European Commission) to develop so-called sectoral qualifications frameworks to allow for bridging the two levels. It distinguished six sectors: natural sciences, engineering, healthcare, social sciences, humanities, and performing and creative disciplines. In the timespan 2008–2011, it defined sectoral frameworks for all mentioned sectors, except natural sciences and healthcare due to an absence of concrete projects and therefore funding (Tuning Education Structures in Europe, n.d.). As part of this endeavour, it decided to merge the philosophies of the two overarching European frameworks to have a firm basis for its sectoral qualifications frameworks. These in turn allowed for preparing Tuning reference points documents that would be better aligned with the overarching frameworks (CALOHEE 2016; Wagenaar 2013; 2018). In practice, sectoral frameworks have proven to be a reliable basis for designing and delivering multi-, inter-, disciplinary and transdisciplinary degree programs, because they are more abstract than disciplinary frameworks. Good examples in this respect are area and life sciences programs (Euroculture, n.d.; University of Utrecht, n.d.). Criticisms Over time, opposition also developed regarding the wish to harmonize European higher education, although it was not very well organized and was limited to a few countries, mainly Germany, Austria, and Spain, and some individual academics. Criticism can be grouped in two arguments. Firstly, there was fear that higher education was being handed over to economic interests as a result of the World Trade Organization’s General Agreement on Trade in Services discussions. The Bologna Process was labelled a neoliberal reform, an attempt to

International Student Mobility Based on Learning Outcomes and Workload 153

economize higher education. In the drive to modernize there was far too much focus on notions such as the “knowledge economy,” the “knowledge society,” and the return rate approach (Lorenz and Valeyre 2006). In reality there proved to be consensus among policymakers and higher education leaders that higher education should remain a public good. The second argument applied was that the Process lowered the quality of education and undermined the related academic freedom of staff to form new generations of academics, focusing in particular on the division of long studies in two-cycle programs. Symbolic became the slogan: “In former times I was a poet and a philosopher; now I am a bachelor” (Peschar 2012, 613). It was argued that the workload of students had been increased due to packing five-year studies into three years, that mobility had decreased, and that first-cycle degrees were not accepted by the labour market. All these arguments were unfounded by evidence (Peschar 2012). Nevertheless, the European Parliament and the Council of Ministers implicitly answered these criticisms in 2006 by publishing a Recommendation on Key Competences for Lifelong Learning, identifying eight basic competences. The Recommendation stated that “key competences for lifelong learning are a combination of knowledge, skills and attitudes appropriate to the context. They are particularly necessary for personal fulfilment and development, social inclusion, active citizenship and employment” (European Parliament 2006, 13). However, it was also stressed that these were “essential in a knowledge society and guarantee more flexibility in the labour force, allowing it to adapt more quickly to constant changes in an increasingly interconnected world” (European Parliament 2006, 13). These competences were seen as “a major factor in innovation, productivity and competitiveness,” and a contribution “to the motivation and satisfaction of workers and the quality of work” (Navarro 2017, 2). As a follow-up, two years later, the Commission launched its New Skills for New Jobs initiative, meant in particular for higher education, followed in 2010 by a report of a European Commission expert group entitled New Skills for New Jobs: Action Now (European Commission 2010). As such the European Union and its member states did indeed focus on economic prosperity, but with an eye toward civic awareness and personal satisfaction. One might have expected that respect for and use of national languages—a sensitive issue in the European context—would have played a key role in the development of the instruments outlined above. Remarkably, it did not really do so, although its importance was well

154

Robert Wagenaar

understood, operating as it was in an educational environment. From the start of the ERASMUS Mobility Scheme and the development of ECTS, it was acknowledged that a common working language—English—would be a requirement for success. Serious criticism of this common-sense strategy never occurred. A practical approach was applied by translating many of the key European documents into national languages, acknowledging the English version as the original source. At the same time, key national documents were translated into English to make them widely accessible. While the first editions of the ECTS Users’ Guide were translated into the languages of the European Union member countries, the most recent ones have only been published in English. However, from the introduction of ECTS it was obligatory that each course catalogue be published in the native language and another widely spoken language, which in practice is English. All Tuning documents were prepared in English by international groups of experts. General Tuning documents were translated into several languages; the frameworks and reference points documents were not. Without any doubt this limits (and has limited) their application in national contexts. Experience has taught that it is quite critical to have glossaries in all languages involved that offer well-defined definitions of the main terms used. Definitions are perceived as conditional for a correct interpretation or understanding of each term. Making the translations of terms and their definitions is seen as a national responsibility. Ministries, accreditation authorities and quality assurance agencies, and rectors’ conferences have a shared role to play in this respect. What was heavily debated, however, was the language proficiency of students in a program whose native language is different from the language of study. This has been and continues to be experienced as a major issue both for degree students and mobility students. In general, higher education institutions have decided to apply minimum standards that are based on the widely accepted descriptors of the Common European Framework of Reference for Languages: Learning, Teaching, Assessment, initiated by the Council of Europe (Council of Europe, n.d.). This reference framework distinguishes three general levels, A (lowest), B, and C, and six sublevels, A1, A2, B1, B2, C1, and C2 (near native). Typically, higher education institutions apply B2 as the minimum for admission to a program or its individual course units. This standard is used for course units offered in the national language and in English. Meanwhile, a growing number of higher education institutions in Europe offer degree programs or parts of them in English, in particular at the master’s level.

International Student Mobility Based on Learning Outcomes and Workload 155

Student-Centred Education With the conceptual frameworks in place at the general level as well as the subject-specific level, further steps could be set. A distinction can be made here again between policy-making at the European and national level and the Tuning initiative. With the qualifications framework for the EHEA defined, countries were asked from 2005 onward to turn their national descriptions of qualifications into National Qualifications Frameworks (NQF), according to the European format. For developing these, a ten-step approach was designed (Bologna Follow-Up Group 2007). Some countries proved to be faster than others. In 2015, twenty-four countries had a certified framework in place. Also, in the setting of the EQF for LLL initiative, countries were asked to develop their NQF for LLL. Although the European Commission initiative was meant primarily for European Union member countries, in February 2015, thirty-eight countries had or were developing their framework according to the European Union model (Cedefop 2015). At the same time, it was understood that a toolbox of conceptual frameworks would not be sufficient to get the required reform of higher education in place. It was gradually learned and accepted that a paradigm shift would be required regarding the way teaching and learning is offered, as promoted by Tuning since 2002, and stressed by the European Commission since the late 1990s. The London Communiqué of 2007 (European Commission, Education and Culture 2007), resulting from a meeting of the Bologna minister of education, reported “increasing awareness that a significant outcome of the process will be a move toward student-centred higher education and away from teacher driven provision” (Conference of European Ministers Responsible for Higher Education 2007, 2). The communiqué of the next ministerial meeting in Leuven-Louvain-la-Neuve (Conference of European Ministers Responsible for Higher Education 2009) devoted—as a follow-up—a whole paragraph to the topic of “student-centred learning and the teaching mission of higher education” (3). In the document, the ministers defined student-centred learning as empowering individual learners, requiring “new approaches to teaching and learning, effective support and guidance structures and a curriculum focused more clearly on the learner in all three cycles…. Academics, in close cooperation with student and employer representatives, will continue to develop learning outcomes and international reference points for a growing number of subject areas” (3). By including this wording not

156

Robert Wagenaar

only was credit given to the work done by Tuning and many European Union-supported thematic network programs (projects covering large numbers of academics from different countries in a particular academic field), which had taken the Tuning approach on board, but it also reflected the understanding that reform required the commitment and involvement of the whole academic and supporting community, not only the leadership of higher education institutions. In relation to the above, higher education institutions were asked “to pay particular attention to improving the teaching quality of their study programs at all levels” (Conference of European Ministers Responsible for Higher Education 2009). In retrospect the first six to seven years following the signing of the Bologna Declaration can be defined as a phase of construction, development, and refinement of the (transparency) toolbox, and the years to follow should become the period of implementation. Already in the Bergen meeting in 2005, it was foreseen that the original deadline of 2010 to establish the EHEA would not be met. More fundamental reforms such as these take time. From 2005–2010, when the ten-year anniversary of the Bologna Declaration was celebrated, a tremendous pile of papers and reports were produced documenting the progress of the Bologna Process, summarizing the outcomes of related seminars, steering direction through position papers, showing its contribution, and the like. In particular, the progress reports became substantial and detailed, thanks also to the services provided by the European Commission, such as Eurydice, EUROSTAT, and eurostudent.eu. The years immediately before the celebration activities, and in particular those since then, show little progress regarding the implementation of the original aims and the instruments developed: the new ECTS model based on workload and learning outcomes, quality assurance mechanisms, the implementation of qualification frameworks, and recognition policies. In particular countries that joined the process later than 1999 proved to be lagging behind. This led to deep frustration among part of the (associated) members of the Bologna Follow-Up Group (Bologna Follow-Up Group 2014). With the many progress reports in mind produced by the European University Association—the so-called Trends reports—the European Student Union (publishing their report Bologna with Student Eyes), and the European Commission, as well as The Bologna Independent Assessment (Center for Higher Education Policy Studies 2010). Tuning was inspired with support of the EC to commence a more in-depth study regarding the implementation of

International Student Mobility Based on Learning Outcomes and Workload 157

the modernization of higher education programs and in particular the application of the learning outcomes approach. This study not only confirmed the findings of the progress reports but found that the actual situation was even more disturbing (European Commission, Education, Culture and Audiovisual Executive Agency 2015; European University Association 2015). Good practices were found, but in general the actual implementation of the student-centred approach proved not to proceed beyond a discourse on the paradigm shift, and the research team had to conclude that there was no certainty this shift will actually be achieved. A worrying disconnect was observed between the various tiers of the higher education sector, ranging from ministers to students, regarding the actual penetration of the student-centred approach and the education experience of the students. Failure to engage with and convince academic staff about the necessity and advantages of this paradigm shift was noted. Teaching staff are struggling to adjust to the new concepts, challenged by being no longer “knowledge owners” but rather learning facilitators. The vast majority of staff members had not undertaken professional development for higher-education teaching, due to the absence of informed trainers, but most of all because no budget had been made available for this purpose. This last observation summarizes the lack of progress in a nutshell: New approaches simply do not land when there is no existing knowledge about pedagogical approaches and strategies and methods to learn, teach, and assess. This resulted in the overall conclusion that new strategies had to be applied for making political and Tuning ambitions to reform higher education programs a reality (Birtwistle, Brown, and Wagenaar 2016). New Strategies From its start, the Tuning strategy had been to provide the groundwork for making degree programs more relevant and of higher quality, but to keep away from quality assurance and accreditation activities. Due to the lack of progress made and inspired by the failure of the feasibility study Assessment of Higher Education Learning Outcomes (AHELO), for which Tuning developed the conceptual frameworks for economics and engineering, it changed its policy (OECD 2011a, 2011b; Tremblay, Lalancette, and Roseveare 2012). In consultation with the European Commission it was decided to set up a feasibility study that would allow for comparing the outcomes of learning in a transnational context.

158

Robert Wagenaar

This would offer evidence of successful practices, to serve as a trigger for change. It was determined that AHELO failed because it chose a top-down approach focusing on the effectiveness of educational systems and was not able to enthuse the academic staff and students. To optimize chances of success, Tuning opted for a grassroots initiative, following its tradition. It also decided to involve in its new initiative nearly all European university networks and European higher education associations to guarantee (critical) commitment and support. In January 2016, the CALOHEE project was launched, involving seventy academics and six student representatives from nearly as many higher education institutions (CALOHEE, n.d.). Like all Tuning projects, CALOHEE is co-financed by the European Commission and the higher education institutions involved. Its ultimate mission is to find a reliable and sustainable approach for diagnostic assessments in a European context. To make this a reality, a multi-step approach has been designed to develop a multidimensional instrument that does justice to the different missions and profiles of higher education institutions in Europe. As in ECTS, five subject areas have been selected which represent the five main academic sectors: engineering (civil engineering), natural sciences (physics), social sciences (education sciences and teacher training), humanities (history), and healthcare (nursing). As a first step, the five groups were asked to update the existing documents defining the reference points for the design and delivery of degree programs and to align their descriptors to the merger of the two overarching European qualifications frameworks discussed above. The merger introduced the concept of “dimensions”; a dimension is a constructive key element that defines a subject area, and each subject area is based on a multiple set of dimensions. A second step is the development of so-called assessment frameworks, which are based on the schemes of (learning outcomes) descriptors according to the Tuning merger of qualifications frameworks. These should offer a robust basis for a measurement instrument that is multidimensional, by holding four parameters or categories—knowledge: theory and methodology; application of knowledge and skills; employability; and civic, social, and cultural engagement. For the latter category CALOHEE has developed a separate conceptual framework to be embedded in the subject area ones (CALOHEE 2017). Assessment frameworks are much more detailed than subject area frameworks and should offer well-formulated learning outcomes that

International Student Mobility Based on Learning Outcomes and Workload 159

can be assessed (Lokhoff, et al. 2010). They should provide a rather complete menu of all identified and agreed-upon items that can be learned, from which an individual institution and program can choose based on its profile and mission, while respecting the irreducible core of the academic field. Each assessment framework will offer a transparent reference for both academic staff and students of what might be expected from a qualification or degree. The system should be inspirational for reform and enhancement of existing and new study programs, most of all because all descriptors in the assessment framework are underpinned with examples of good practice of how the identified learning outcomes can be taught, learned, and assessed by means of state-ofthe-art approaches and methodologies (Wagenaar 2018a). It should also serve as the basis for developing sophisticated test items and, ultimately, actual transnational comparative assessments, which are meant to be diagnostic for various stakeholders, the higher education institutions themselves and students to begin with: Where does a program perform well, and where is room for further improvement given type, profile, and mission? These assessments should be “the proof is in the eating of the pudding” and ideally replace the existing rather bureaucratic quality assurance procedures in the years to come. In June 2016 the CALOHEE approach was made official European Commission policy (European Commission 2016). The first set of Tuning-CALOHEE subject area qualifications reference frameworks and related assessment reference frameworks have been published in the autumn of 2018. (Wagenaar, 2018b). In January 2020, the CALOHEE initiative was scheduled to be continued for another three years. Besides the CALOHEE project, Tuning has recently proposed a second strategy to close the gap between political aims and reality, by making academic staff more knowledgeable by setting up a staff development program that makes use of a mixture of online instruction and local teamwork activities. It awaits EC support to roll out this new initiative European-wide (International Tuning Academy 2017). Conclusion Over thirty years, a set of conceptual instruments and tools have been developed for implementing the learning outcomes approach as an instrument for transparency, comparability and compatibility, and transfer and recognition of studies, as well as the enhancement of degree programs in general. Enhancement is to be understood as alignment

160

Robert Wagenaar

with the needs of society by finding a better balance between knowledge acquisition, understanding and application of knowledge, and the development of subject-specific and generic skills and competences. The predominant reason to develop the extensive ERASMUS transfer scheme for both students and academic staff was (and is) to promote the European integration process. As a result of developing the indispensable components of a common credit system, it also became a key instrument through ECTS for the modernization of higher education. Currently ECTS is seen by the signatories of the Bologna Declaration and the European Commission as the main tool for reform. It is perceived as a driver for making the paradigm change to student-centred and active learning. This is seen as conditional for preparing students better for society, particularly improving their employability and thus contributing to the economy, welfare, and sustainability of Europe and the individual nation states. However, the present state of affairs shows there is still a long way to go to restructure teaching and learning according to the new paradigm. This will require bolder steps, such as the development of detailed subject-area-based assessment frameworks to strengthen societal trust and confidence about the performance of higher education institutions and their staff and the quality and relevance of their study programs. But these frameworks will also offer input for the reform of individual degree programs. Moreover, it will be absolutely necessary to get serious state-of-the-art staff development in place. This will require additional funding at both European and national levels. However, most of all it will require that academics accept that in the twenty-first century they should not be knowledge experts, but rather learner facilitators. References Adonnino, P. 1985. A People’s Europe: Reports from the Ad Hoc Committee. Bulletin of the European Communities: Supplement 7/85. Luxembourg City, Luxembourg: Office for Official Publications of the European Communities. Anderson, L. W., and D. R. Krathwohl, eds. 2000. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York, NY: Pearson. Barr, R. B., and J. Tagg. 1995. “From Teaching to Learning: A New Paradigm for Undergraduate Education.” Change, 27: 12–25. Birtwistle, T., C. Brown, and R. Wagenaar. 2016, May. “A Long Way to

International Student Mobility Based on Learning Outcomes and Workload 161

Go…: A Study on the Implementation of the Learning-Outcomes Based Approach in the European Union.” Tuning Journal for Higher Education, 3 (2): 429–463. Bloom, B. S., M. D. Engelhart, E. J. Furst, W. H. Hill, and D. R. Krathwohl. 1956. Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook I: Cognitive Domain. New York, NY: David McKay. Bologna Declaration. 1999. Joint Declaration of the European Ministers of Education. https://www.eurashe.eu/library/bologna_1999_bologna-declaration-pdf/ Bologna Follow-Up Group. 2007. National Qualifications Frameworks Development and Certification: Report from Bologna Working Group on Qualifications Frameworks. http://media.ehea.info/file/WG_ Qualifications_frameworks/40/0/QualificationsFrameworks_report2007_581400.pdf ———. 2014. The Bologna Process Revisited: The Future of the European Higher Education Area. European Higher Education Area, document BFUG_LV_IS_43_4. http://www.ehea.info/media.ehea.info/ file/2015_Yerevan/71/1/Bologna_Process_Revisited_Future_of_ the_EHEA_Final_613711.pdf Bologna Working Group on Qualifications Frameworks. 2005, February. A Framework for Qualifications of the European Higher Education Area. Copenhagen, Denmark. CALOHEE. 2016. CALOHEE Working Paper: Common Structure for Subject Area Based Qualifications Frameworks (Reference Points for the Design and Delivery of Degree Programmes in ...): A proposal. Groningen, The Netherlands: Author. https://www.calohee.eu/wp-content/ uploads/2016/06/CALOHEE-Working-Paper.pdf ———. 2017. CALOHEE Working Paper for Civic, Social and Cultural Engagement. Groningen, The Netherlands: Author. https://www.calohee.eu/working-papers/ ———. n.d. What is CALOHEE? https://www.calohee.eu Cedefop. 2015. Overview of National Qualifications Frameworks: Development in Europe. Brussels, Belgium: Author. http://www.cedefop.europa.eu/en/publications-and-resources/publications/8606 Center for Higher Education Policy Studies. 2010. The Bologna Process Independent Assessment: The First Decade of Working on the European Higher Education Area. Volume 1. Detailed Assessment Report. https://publications.europa.eu/en/publication-detail/-/publication/6226c3e4-34ef-4142-8801-7aeefb97021f

162

Robert Wagenaar

Commission of the European Communities. 1985. A People’s Europe: Report from the Ad Hoc Committee. Bulletin of the European Communities, Supplement 7/1985. Luxembourg: Author. http://repositori. uji.es/xmlui/bitstream/handle/10234/49877/Suplemento7-85en. pdf?sequence=1 Conference of European Ministers Responsible for Higher Education. 2003, September. Realising the European Higher Education Area. https:// enqa.eu/wp-content/uploads/2013/03/BerlinCommunique1.pdf ———. 2007, May. Towards the European Higher Education Area: Responding to Challenges in a Globalised World. http://www.ehea.info/media. ehea.info/file/2007_London/69/7/2007_London_Communique_ English_588697.pdf ———. 2009, April. The Bologna Process 2020—The European Higher Education Area in the New Decade. https://www.eurashe.eu/library/bologna_2009_leuven-communique-pdf/ Corbett, A. 2005. Universities and the Europe of Knowledge: Ideas, Institutions and Policy Entrepreneurship in European Union Higher Education Policy, 1955–2005. London, UK: Palgrave MacMillan. Council of Europe. n.d. Common European Framework of Reference for Languages: Learning, Teaching Assessment. Strasbourg, France: Council of Europe, Modern Languages Division. https://www.coe.int/t/dg4/ linguistic/source/framework_en.pdf Euroculture. n.d. Programme. https://www.euroculturemaster.eu/programme-outline/learning-outcomes European Association for Quality Assurance in Higher Education. 2005. Standards and Guidelines for Quality Assurance in the European Higher Education Area. Brussels, Belgium: Author. ———. 2015. Standards and Guidelines for Quality Assurance in the European Higher Education Area. Brussels. European Commission. 1991. Memorandum on Higher Education in the European Community. Luxembourg: Author. ———. 1997. Communication from the European Commission: Towards a Europe of Knowledge. Luxembourg City, Luxembourg: Author ———. 2004. ECTS Users’ Guide. Brussels, Belgium: Author. ———. 2006. The History of European Cooperation in Education and Training: Europe in the Making—An Example. Luxembourg City, Luxembourg: Author. ———. 2008. The European Qualifications Framework for Life Long Learning. Luxembourg City, Luxembourg: Author. ———. 2010. New Skills for New Jobs: Action Now. A Report by the Expert

International Student Mobility Based on Learning Outcomes and Workload 163

Group on New Skills for New Jobs Prepared for the European Commission. Brussels, Belgium: Author. ———. 2014. The Erasmus Impact Study. Effects of Mobility on the Skills and Employability of Students and the Internationalization of Higher Education Institutions. Luxembourg City, Luxembourg: Author. ———. 2015a. Erasmus Facts, Figures & Trends. The European Union Support for Student and Staff Exchanges and University Cooperation in 2013– 2014. Luxembourg City, Luxembourg: Author. ———. 2015b. ECTS Users’ Guide 2015. Publications Office of the European Union. Luxembourg. https://ec.europa.eu/education/ects/ users-guide/docs/ects-users-guide_en.pdf ———. 2016. Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions. A New Skills Agenda for Europe. Working Together to Strengthen Human Capital, Employability and Competitiveness. Strasbourg, France: Author. ———. 2017. Erasmus + International Credit Mobility. Handbook for Higher Education Institutions. Version 2.0 – November 2017. Brussels, Belgium: Author https://ec.europa.eu/programmes/erasmus-plus/ sites/erasmusplus2/files/international-credit-mobility-handbook_ en.pdf European Commission, Education and Culture. 2007, May. Towards the European Higher Education Area: Responding to Challenges in a Globalised World. http://www.ehea.info/media.ehea.info/file/2007_London/76/4/20070517_EuropeanCommission_note_588764.pdf European Commission, Education, Culture and Audiovisual Executive Agency. 2015. The European Higher Education Area in 2015: Bologna Process Implementation Report. Publications Office of the European Union. Luxembourg. http://eacea.ec.europa.eu/education/eurydice/documents/thematic_reports/182EN.pdf European Parliament. 2006. “Recommendation of the European Parliament and of the Council on Key Competences for Lifelong Learning.” Official Journal of the European Union, 10-18. European Student Union. 2015. Bologna with Student Eyes 2015. Time to Meet the Expectations from 1999. Brussels, Belgium: Author. European Union Association. 2010. Trends 2010: A Decade of Change in European Higher Education. Brussels, Belgium: Author. European University Association. 2015. Trends 2015: Learning and Teaching in European Universities. Brussels, Belgium: Author. Gaston, P. L. 2010. The Challenge of Bologna: What United States Higher

164

Robert Wagenaar

Education Has to Learn from Europe, and Why It Matters that We Learn It. Sterling, VA: Stylus Publishing. González, J., and R. Wagenaar. 2003. Tuning Educational Structures in Europe. Final Report Phase One. Bilbao, Spain: University of Deusto. ———. 2006. Tuning Educational Structures in Europe. Universities’ Contribution to the Bologna Process. An Introduction. Bilbao, Spain: University of Deusto. International Tuning Academy. 2017. Detailed Project Description Erasmus + Forward-Looking Cooperation Projects. Professional Learning and Technology for University Modules (PLATFORUM). https://eacea. ec.europa.eu/erasmus-plus/funding/forward-looking-cooperation-projects-2017-eacea412016_en Joint Quality Initiative. 2004. Shared “Dublin” Descriptors for Short Cycle, First Cycle, Second Cycle and Third Cycle Awards. A Report from a Joint Quality Initiative informal group. http:// www.jointquality.nl/ Leegwater, M. 2015. “Joint Quality Initiative – The Origin of the Dublin Descriptors – Short History.” http://ecahe.eu/assets/uploads/2016/01/Joint-Quality-Initiative-the-origin-of-the-Dublin-descriptors-short-history.pdf Lokhoff, J. B., K. Wegewijs, R. Durkin, J. Wagenaar, A. K. Gonzalez, L. Isaacs, Dona della Rose, and M. Gobbi. 2010. A Tuning Guide to Formulating Degree Programme Profiles. Including Programme Competences and Programme Learning Outcomes. Bilbao, Spain: University of Deusto. Lorenz, E., and A. Valeyre. 2006. “Organisational Forms and Innovative Performance: A Comparison of the EU-15.” In How Europe’s Economies Learn: Coordinating Competing Models, edited by E. Lorenz, B. and A. Lundvall, 50–69. Oxford University Press. Navarro, R. L. 2017. Integrated European Project. http://iessantarosadelima.com/wp-content/uploads/2018/01/INTEGRATED-EUROPEAN-PROJECT.-The-Core-curriculum-2.pdf OECD. 2011a. Tuning-AHELO Conceptual Framework of Expected and Desired Learning Outcomes in Economics. OECD Education Working Papers, No. 59. Paris, France: OECD Publishing. ———. 2011b. Tuning-AHELO Conceptual Framework of Expected and Desired Learning Outcomes in Engineering. OECD Education Working Papers, No. 60. Paris, France: OECD Publishing. Peschar, H. 2012. “The Decline of an Academic Oligarchy. The Bologna Process and Humboldt’s Last Warriors.” In European Higher Education at the Crossroads: Between the Bologna Process and National Reforms, edited by A. Curaj et al. Dordrecht, The Netherlands: Springer.

International Student Mobility Based on Learning Outcomes and Workload 165

Ravinet, P. 2005. “The Sorbonne Meeting and Declaration: Actors, Shared Vision and Europeanisation.” In The Bologna Process and the Shaping of the Future Knowledge Societies: Conference Report from the Third Conference on Knowledge and Politics, the University of Bergen, edited by T. Halvorsen and A. Nyhagen, May 18–20, 2005, 187– 204. http://euredocs.sciences-po.fr/en/conference/2005/ravinet. pdf Sorbonne Joint Declaration. 1998. Joint Declaration on Harmonisation of the Architecture of the European Higher Education System by the Four Ministers in Charge for France, Germany, Italy and the United Kingdom. Paris, France: The Sorbonne. Tagg, J. 2003. The Learning Paradigm College. San Francisco, CA: JosseyBass Inc. Task Force Human Resources, Education, Training and Youth. 1993. Evaluation of the Pilot Phase of the European Community Course Credit Transfer System. Luxembourg: Community Research and Development Information Service. Trembley, K., D. Lalancette, and D. Roseveare. 2012. Assessment of Higher Education Learning Outcomes. Paris, France: OECD Publishing. http://www.oecd.org/education/skills-beyond-school/AHELOFSReportVolume1.pdf Tuning Educational Structures in Europe. n.d. “Subject Area Brochures.” http://www.unideusto.org/tuningeu/publications/subject-area-brochures.html University of Groningen. 2000. “SOCRATES – Action 6.1 Application Form for Transnational Cooperation Programmes Project Proposal Tuning Educational Structures.” Groningen, Germany: Author. University of Utrecht. n.d. “Aim, Learning Outcome and Labels.” Accessed January 31, 2019. http://studyguidelifesciences.nl/startyour-programme/aim-learning-outcomes-and-labels#learning-outcomes Wagenaar, R. 2006. “An Introduction to the European Credit Transfer and Accumulation System (ECTS).” In EUA Bologna Handbook, Making Bologna Work, edited by E. Froment, J. Kohler, L. Purser, and L. Wilson, 1–22. Berlin, Germany: European University Association. ———. 2013. “Columbus’ Egg? Qualifications Frameworks, Sectoral Profiles and Degree Programme Profiles in Higher Education.” Tuning Journal for Higher Education, 1 (1): 71–103. ———. 2018a. “What Do We Know – What Should We Know? Measuring and Comparing Achievements of Learning in European Higher

166

Robert Wagenaar

Education: Initiating the New CALOHEE Approach.” In Assessment of Learning Outcomes in Higher Education: Cross National Comparisons and Perspectives, edited by O. Zlatin-Troitschanskaia, H. Anand Pant, M. Toepper, C. Lautenbach, and C. Kuhn. Wiesbaden, Germany: Springer. ———. 2018b. Tuning-CALOHEE Assessment Reference Frameworks for Civil Engineering, Teacher Education, History, Nursing, Physics, edited by R. Wagenaar. Groningen. https://www.calohee.eu Westerheyden, D. F., and M. Leegwater. 2003. Working on the European Dimension of Quality. Report of the Conference on Quality Assurance in Higher Education as Part of the Bologna Process, Amsterdam, 12–13 March 2002. Zoetermeer, The Netherlands: Ministry of Education, Culture and Sciences.

7

Learning Outcomes, Progression, and Qualifications: Considerations for Vocational and Higher Education in the United Kingdom Chris Winch

Introduction Learning-outcomes-based qualifications have spread across the world over the last thirty years, so one might think that there cannot be anything that is seriously wrong with them; and indeed, there must be a great deal right if they have become so widespread. We need, therefore, to briefly survey the career of at least one system of outcomes-based qualifications before we can arrive at a judgment.1 Having done this, we will look in more detail at the concept of a qualification and go on to examine the underlying design philosophy of outcomes-based qualifications. However, the terminology in this area can be confusing, so it is best first to explain the way in which critical terminology is used in this study. Learning outcome: knowledge or skill acquired through a process of learning. In its “pure” form a learning outcome can be defined inde1. See Allais (2014) for further examples, including South Africa and Australia. Learning Outcomes, Academic Credit, and Student Mobility, edited by Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon. Montréal and Kingston: McGill-Queen’s University Press, Queen’s Policy Studies Series. © 2020 The School of Policy Studies, Queen’s University at Kingston. All rights reserved.

168

Chris Winch

pendently of any cognitive hierarchy, any other learning outcomes, or both. If it is dependent on other learning outcomes in a cognitive hierarchy and is tied to the achievement of a particular curriculum, it is a standard. An outcomes-based qualification is one based on the achievement of learning outcomes in the sense above. Competence: This term means different things in different countries, but regarding outcomes-based qualifications, it means the ability to carry out a (usually) narrowly defined task to a threshold level of competence. This can be described as the ability to achieve a learning outcome. Thus, Cedefop (the European Union Vocational Education Agency) describes a competence as “the ability to apply learning outcomes adequately in a defined context” (European Centre for the Development of Vocational Training 2008, 47). Competency, by contrast, is described in European documents as a degree of independence from managerial supervision. It is thus not straightforwardly a type of skill (Brockmann, Clarke, and Winch 2009, 789). A threshold is an adequate level of ability to apply a learning outcome. It does not carry any further evaluative implication. It can be seen from these brief definitions that there is the potential for ambiguity in their application. This creates the potential for conflict but also for reduction of conflict. Thus, there is sufficient latitude in the use of the term “learning outcome” for it to be used to denote a standard as well as a descriptor independent of a cognitive hierarchy or of other learning outcomes. This “constructive ambiguity” in terminology allows for a measure of agreement on documentation among parties with different interests, for example in the context of the construction of European vocational education and training (VET) policy tools. We will start with England. National Vocational Qualifications (NVQs) were introduced in England and Wales in 1986. They apply to the professional sector and are most commonly found at the lower levels of the cognitive hierarchy (Levels 1 and 2). They arose from a perceived need to make qualifications more aligned with the needs of employers than the existing input-based qualifications. It should be noted, however, that it was the government, not the bulk of employers, who were the driving force behind their design and implementation, despite some rhetorical support from employers. England and Wales already had a well-developed system of vocational qualifications at Levels 2 and 3 before their introduction, but these were thought to be outmoded in their design philosophy. Although NVQs were envisaged to cover Levels 1 to 8 (lower-secondary to doctoral), in practice their focus lay on Levels 2 and 3, that is, semi-skilled and technical work respectively.

Learning Outcomes, Progression, and Qualifications 169

In practice, the NVQ came to be predominantly a Level 2 qualification. The levels in question are specified by the integrated national qualification framework for England, which covers both academic and professional qualifications. The framework proclaims these to be equivalent, although in practice the equivalence is nominal for the lower levels, due to the absence of theoretical underpinning knowledge, particularly in NVQs. A Level 1 NVQ is thus nominally equivalent to an upper elementary/lower secondary academic qualification, a Level 2 to higher grade 16+ qualifications, a Level 3 to 18+ academic qualifications. Levels 4 and 5 represent sub-degree qualifications. The absence of underpinning knowledge, particularly at Levels 1 and 2, means that there is a lack of permeability between NVQ Levels 1 and 2 on the one hand and Level 3 on the other and a fortiori between NVQ Levels 1, 2, and 3+ non-NVQ qualifications, which are more knowledge-based. Although national qualification frameworks were partly devised to improve the ability to move from lower- to higher-level qualifications, the permeability of qualifications remains restricted. A qualification is permeable if it has characteristics that allow its holder to progress to the achievement of a qualification placed higher on a cognitive hierarchy scale. One of the most important of these characteristics is the possession of knowledge and know-how (specifically literacy and numeracy in the latter case) that facilitate more advanced study. Lower-level NVQs, with their reliance on the ability to perform narrow tasks with little underpinning knowledge required, are relatively impermeable. This is one reason why they have lost favour with the government, which like other European governments is anxious to promote social and economic mobility through qualification progression.2 NVQs exemplify a “full-blooded” version of the outcomes-based design philosophy outlined above. Thus independence, specifiability, and hierarchy are all features of the NVQ design philosophy (see discussion below). They are strongly vocational in orientation (Stanton 2008), meaning that they are designed to fit the holder for competence in the workplace or to accredit competent workplace practice. The term “competence” here is understood as the “ability to perform the specified task.” A typical NVQ will be constructed from a set of occupational standards, which taken together constitute an occupation. It should be 2. Hence the European Qualification Framework (EQF), which is designed so that national qualification frameworks can be integrated with each other through structures which are isomorphic with the EQF and hence with each other.

170

Chris Winch

noted that “occupation” is meant here as an employer-defined concatenation of tasks. The occupational standards are then translated into behavioural manifestations of the standards. Someone who can conform to these standards is said to be competent in the relevant task. Someone who is competent in all the tasks associated with all the standards within the occupation is competent in that occupation. The NVQ does not normally specify the knowledge needed to be competent.3 Learning outcomes are nothing more than specifications of the tasks that are needed for competent performance. Thus, learning outcomes are constructed through employer-determined sets of tasks, which are bundled into job descriptions.4 These tasks are then translated into learning outcomes. Thus, if the tasks are, for example, those associated with maintenance of hygiene in restaurant kitchens, they can be translated into task-based outcome statements for each task or small set of associated tasks. NVQs exist in a qualification ecology with a variety of more traditional qualifications and the great majority exist for Level 2 and Level 3.5 They can be awarded on the basis of satisfactory workplace performance. NVQ design philosophy has pervaded a whole range of qualifications, but the approach, although extensively used, has never been fully accepted. Thus, the term “competence” within the NVQ context is quite restricted, referring to the behavioural manifestation of workplace tasks. It is important to note that this English sense of “competence” does not correspond to the French compétence, which includes within it the idea of applying knowledge to practice, or the German Kompetenz, which also includes the application of knowledge to practice, together with a civic and moral dimension to action within a holistic occupational framework. Table 7.1, adapted from Brockmann, Clarke, and Winch (2009), summarizes the situation. It is important to appreciate that the term “competence” in the NVQ sense has a specific meaning that cannot readily be translated into cognate terms in other languages. The NVQ has another feature arising from the rigidity of the design framework. The ideal task specification is context independent—the 3. As we shall see, this specification is difficult to adhere to in practice. 4. On this narrow notion of “occupation,” occupations are little more than generalized job descriptions. For a contrast with a richer concept of occupation, see Hanf (2011) and Winch (2013b). 5. Lower and upper secondary exits, respectively.

Learning Outcomes, Progression, and Qualifications 171

Table 7.1 National Conceptions of Competence

National Conceptions of Competence

Integrated Concepts of Competence

Discrete Conceptions of Competence (1)

Discrete Conceptions of Competence (2)

For example, the German Handlungskompetenz, which includes a social, moral, and civic dimension.

French and Dutch conceptions, which, although task-based, assume integration of knowledge, skill, and attitude in practice.

“English” competence, which entails task performance to an acceptable standard. This may, but need not, involve the application of underpinning knowledge.

Note: Adapted from Brockmann, Clarke, and Winch. (2009).

implicit context is the work situation. The description can stand independently so long as the context is invariant or nearly invariant. However, when work and the work context are more complex, the situation is more difficult to specify so neatly. The first problem is that the learning outcome may be quite broad and will need some more specific criteria for assessment that involve breaking down the learning outcome into subcomponents. The second concerns the range over which the relevant tasks are specified. As the contextual range broadens, it is necessary to specify the range of circumstances in which the competence is exercised through what are called “range statements.” An NVQ document will often include both assessment criteria (ACs; detailing of learning outcomes) and range statements (specification of contextual application). However, the idea of levels and thus of hierarchy was central to the NVQ design philosophy. Because occupational competence is hierarchical and at higher levels presupposes not only transversal abilities but also underpinning theoretical knowledge (see discussion below), it was never feasible to suppose that one could have a non-hierarchical system of outcomes-based qualifications. How then would it be possible to reconcile hierarchy with independence and specifiability? The answer is that it can be done, but only at a price. It is quite possible, for example, to

172

Chris Winch

achieve an NVQ Level 2 qualification and then go on to achieve a Level 3 qualification, but only if the two are conceptualized and assessed as distinct from each other. Although one may “progress” from Level 2 to Level 3; it is not possible to build upon the Level 2 qualification to attain the Level 3 one. And, of course, it is possible to achieve Level 3 without having first achieved Level 2. This means, however, that the Level 3 holder will not necessarily possess Level 2 competencies. The way around this problem would be to require that someone acquire a Level 2 qualification before they go on to acquire Level 3. Admittedly, this would be a step back from the criterion of independence, which means that each outcome is specified independently of any other (see discussion below). However, progression from a Level 2 NVQ to a Level 3 one is in practice limited (Wolf 2011). The reason is that, unlike Level 2 academic qualifications, which are designed to allow progression, NVQs are solely concerned with workplace performance, and if that requires little academic or theoretical knowledge the NVQ Level 2 holders do not have the intellectual prerequisites for progression to Level 3 unless they have acquired them from another source. The NVQ, with its practice-based outcomes, tends to isolate holders with lower-level qualifications at low levels of responsibility and know-how, with limited opportunities for progression.6 It is worth looking at examples of an NVQ at Level 2 to see how the design philosophy works out in practice. One problem here is that although learning-outcomes-based qualifications based on the design philosophy described above are very common, there appears to be a reluctance post-2010 to describe them as NVQs, although they are.7 It should also be noted that the learning outcomes design philosophy has spread into occupational qualifications that were never NVQs, supplanting the content-based earlier design philosophy. We will also see that in practice it is difficult, if not impossible, to specify learning outcomes in such highly behavioural terms that they can be directly assessed. The example is the City and Guilds (2015) Level 2 Diploma in Health and Social Care (Adults) for England. We will note a few features of this qual6. It should also be noted that in the United Kingdom economy there tends to be a dearth of technician-level jobs (Payne and Keep 2011), which means that a Level 3 qualification is often a route into the lower tiers of management. 7. The NVQ Regulatory Framework was discontinued in 2015, although it is still possible to call qualifications “NVQs.”

Learning Outcomes, Progression, and Qualifications 173

ification: First, the requirements for prior knowledge are “finessed” by pointing out that those who have taken a lower-level qualification that contained knowledge requirements will not be required to take these again unless there are doubts about the candidate’s possession of such knowledge during in situ performance (21). Note, however, that there is no official progression, since the lower-level qualification is thought to cover the same ground as the Level 2 qualification, even though knowhow need not be manifested procedurally in the lower-level qualification. Second, the qualification is very largely based on workplace performance, although the mandatory off-the-job element has been recently increased to five hours per week (24). Note however that this competence-based qualification is designed to be assessed in the candidates’ workplace as they undertake their normal role (29). Note also that although knowledge and understanding should mainly be inferred from practice, it may be necessary for the assessor to question the candidate to probe knowledge and understanding further (28). Further examination of one of the ACs associated with learning outcomes—“Understand what is required for competence in own work role”—suggests that candidates need to be able to describe their work role and to identify standards that influence the way the role is carried out (38). We can see, therefore, that the original design philosophy has been modified considerably. Although key features have been modified to give the qualification the flexibility that it needs, one may wonder whether it has also failed to overcome the shortcomings that are associated with this type of qualification. Three areas, in particular, may be identified. First, knowledge and understanding are to be inferred from practice except when doubt arises. However, observing behaviour on a single occasion can never be enough to confirm know-how, since attributions of know-how depend on the ability to act appropriately in a range of similar but varying contexts (Hornsby 2011). Second, if the propositional knowledge that shows understanding of the rationale for action only requires observation of one action, then it cannot be enough, either, for the candidate to demonstrate underlying knowledge and understanding if, for example, they are unable to explain or justify their actions. Third, without a curriculum, it is not possible to obtain an understanding of how elements of know-how and propositional knowledge fit together into a coherent whole. One might to some extent be able to infer this from learning outcomes, ACs, range statements, and so on, but one can never be sure that critical gaps are not going to be found. These issues have

174

Chris Winch

potentially negative implications for the possibility of prior learning assessment, advanced standing, and meaningful credit transfer. Despite the original idea that learning outcomes should be directly assessed, the complexity of even quite simple occupations has made this implausible. Furthermore, since it is unrealistic to expect most occupations not to be conducted with at least some measure of knowledge and understanding, it is necessary to include non-behavioural learning outcomes within the more behavioural ones. These two requirements have necessitated the introduction of ACs to provide greater precision to the qualification subcomponents, undermining the specifiability of the learning outcomes themselves. It is now acknowledged that NVQs cannot provide the relevant, high-quality workplace-based qualifications that were originally intended. Two recent reviews of vocational qualifications (Richard 2012; Whitehead 2013) have failed to endorse the design principles underlying NVQs, following the critique made by the Wolf Report (Wolf 2011). As well as failing to provide high quality, the following problems beset NVQs: • poor labour market value, in some cases providing negative returns on previously acquired qualifications (BIS 2011); • the proliferation of qualifications and sub-qualifications, many of which were never used, including at Levels 2 and 3; • very little uptake above Level 4; and • concern that, in focusing on the workplace practice of today, they fail to prepare learners for the abilities that will be needed in the future. Finally, new regulations for the approval of qualifications introduced in 2015 and 2016 (see Ofqual 2018) signal the abandonment of the NVQ design philosophy, although the term “learning outcome” continues to be used, more in the sense of the “standard” described above. Instead, a new criterion of Total Qualification Time was introduced, which sets out the “inputs” (usually supervised activities and time taken to do them successfully) that are necessary to achieve a qualification. It is no longer possible to describe a qualification independently of the study time and content needed to achieve it, nor is it possible to specify the qualification without locating it within a hierarchy of cognitive levels with their own content, which needs to be mastered before an attempt at a higher level can be made. Thus, the radical design philosophy of the NVQ has finally been abandoned as unviable.

Learning Outcomes, Progression, and Qualifications 175

Learning Outcomes in Europe: ECTS, EQF, and ECVET Since 2004, the European Union, in a sustained attempt to develop a European-wide labour market and transparent and transferable vocational qualifications, has developed a range of instruments based on learning outcomes to facilitate these objectives. I will focus on the European Qualifications Framework (EQF) and the European Credit System for Vocational Education and Training (ECVET), as a means of transferring credit between different qualifications in different countries.8 ECTS Before doing so, it is necessary by way of contrast to describe the European Credit Transfer and Accumulation System (ECTS), introduced by the European Union in 1989. ECTS applies only to higher education (Level 4 and above) and is not predicated on learning outcomes but on credit points based on hours of study. It is organized in three main cycles of bachelor’s, master’s, and doctoral levels. It is “input-based” in the sense that it does not incorporate the design features of independence and specifiability described above, although it is hierarchical. ECTS was designed for higher education qualifications that had not used the great majority of learning outcomes design principles. ECTS is relatively unproblematic to operate as it is based on clearly understood and operable principles. Under ECTS credit is given for prior learning and credit can be transferred from one institution to another, building toward the final qualification at each of the three higher education cycles. Transferable credit is expressed in credit points, which are a function of study hours. It is easy to see how the idea of epistemic ascent can be employed to distinguish levels within a hierarchy of qualifications and thus to (a) align part qualifications at the same level, or (b) to indicate the order of progression in moving from, for example, the bachelor’s to the master’s to the doctoral level. Thus, in a natural science program it would be possible to classify replicative experimentation at the bachelor’s level, the selection of a course of action from a body of natural science theory in a professional master’s qualification, and contribution to original research at the doctoral level.

8. Space renders it difficult to discuss European Skills, Competences and Occupations (ESCO), which has affinities with earlier NVQ-like attempts to relate tasks to qualifications. See Clarke and Winch (2015) for more on this.

176

Chris Winch

EQF The EQF (see European Commission, n.d.) is a learning outcomes framework that was intended to be output-based and designed along the principles of independence, specifiability, and hierarchy described above. It encompasses both vocational and general educational qualifications and covers upper primary (Level 1) to doctoral levels (Level 8). It is a “meta-framework” in the sense that it is designed to compare not only qualifications from each of the twenty-eight European Union countries but also national qualification frameworks, which it is hoped will be designed to be compatible with the EQF. Although ostensibly designed along output-based lines, in practice the “learning outcomes” of the EQF can also be read as standards. This became necessary at an early stage of the introduction of the EQF due to the reluctance of some social partners (notably German trade unions, among others) to endorse the strongly output-based interpretation of learning outcomes (Méhaut and Winch 2012). Furthermore, labour market implementation of the EQF involves detailed engagement with vocational curricula relevant to occupations and hence to engagement with content (Brockmann, Clarke, and Winch 2010). It is not true to say, therefore, that the EQF is a purely learning-outcomes-based approach, despite its designers’ intentions. The realities of European politics and the practicalities of adapting the EQF to sectoral and occupational requirements have seen to that. This move parallels the abandonment of NVQs in England. ECVET ECVET (see European Credit System for Vocational Education and Training, n.d.) is designed to allow for the allocation and transfer of credit for vocational education and training across national boundaries. It is supposed to be based on a learning outcomes approach which eschews “input-based” measures such as numbers of hours of study like those used by, for example, ECTS. However, designing a credit transfer system that does not make use of input-based measures has proved to be an impossible job. The reason is not hard to see. Credit is accumulated through learning, whether formal, informal, or non-formal; this must, therefore, have taken time to occur. If credit transfer, especially across national boundaries, is to be perceived as trustworthy, then there must be a common measure of the time taken to achieve the credits, even if the time taken is a proxy for the effort expended. Thus, if Learning Outcome 1 at Level 3 in a plumbing qualification

Learning Outcomes, Progression, and Qualifications 177

is to be awarded X level of credit in all twenty-eight European Union countries, then there must be a way of ensuring that the achievement of Learning Outcomes 1 in Poland is equivalent to its achievement in, say, France. For that to happen, there must be agreement on the length of time normally taken to master Learning Outcome 1, by whatever means.9 Otherwise, it will not be possible to maintain trust in the equivalence of qualifications and qualification parts awarded in different jurisdictions. This means in effect that assessment of Learning Outcome 1 must be as rigorous in Poland as it is in France, and that in turn means that ACs and assessment instruments have at least approximate equivalence. We find, then, that engagement with the content underlying Learning Outcome 1 in terms of various kinds of knowledge (including know-how) has to be specified and an estimation of the time taken to acquire it must be agreed upon. In practice, therefore, ECVET will have to operate on similar principles to ECTS—that is, based on hours of learning acquisition using a temporal measure as a proxy, together with a level distinction.10 If this is the case, then the way is open to award credit points for part qualifications, which can either be used to complete qualifications at the same level or to complete a part qualification at a given level as a prelude to completing it at a higher level. This can be done by decomposing an overall qualification into modules or even sub-modules. Once again, the general idea of epistemic ascent can be used (possibly in a more granular form) to indicate levels of increasing cognitive difficulty. But credit accumulation and transfer will only work if it is quantified in terms of effort expended and/or cognitive achievement or some proxy for effort expended to acquire the necessary know-how, together with a specification of the level of cognitive difficulty. For example, at Level 7 in a social science discipline one would expect the ability to carry out replicative research and an original investigation under guidance, while at Level 8 one would expect the ability to carry out lightly supervised original research. The procedure then would be to assign a time taken for the effort to bear fruit, together with criteria setting out levels of cognitive difficulty and independent working. Part 9. This could vary according to whether learning is formal, non-formal, or informal, but the principle remains the same. 10. It is theoretically possible to assign credit for transfer without assigning credit points, but in practice it is hard to see how a reliable system for the quantification of credit could be avoided if one were to maintain trust in the system.

178

Chris Winch

qualifications could be quantified using credit points at the appropriate level. Tracking back from the proxy measure (i.e., credit points), we find that effort expended is related to notional time spent on learning, the degree of independent working, and extent and difficulty of content, and hence “input” considerations are unavoidable.11 What Is a Qualification? The award of educational qualifications is customary as a means of guaranteeing the knowledge, ability, or understanding of an individual. The guarantee is given to society in general,12 to employers or educators of the individual, and to the holder of the qualification. The award of a qualification is usually a high-stakes matter, as the holding of a relevant qualification is a gate through which the holder must pass to gain access to education and employment. It is also a guarantee to clients and society that the holder can practice safely, honestly, and competently. What we understand by what lies behind this guarantee is, therefore, of the greatest importance. Perhaps the most critical feature of qualifications is what they assume, but do not (and cannot) state. If we are to investigate the probity of any qualification, therefore, we cannot rest content with what they explicitly guarantee, but must also attend to the implications of that guarantee, which are usually unstated but need to be understood by all interested parties. To deepen our understanding of this point, it will be helpful to look more closely at the more general epistemic and educational background of qualifications. Even if a qualification is not directly tied to a formal educational process, it will be conceptually tied to learning on the part of the holder, since the qualification guarantees that the holder has, at some point, learned the things that the qualification claims that he or she has learned. The notion of content, consisting of knowledge, ability, and understanding, is thus built into the very idea of an educational or professional qualification. Where the qualification is directly tied to educational processes, this content is usually associated with a curriculum, or prescribed content 11. The 2016 qualification regulations are explicit about formal learning time and formal assessment procedures. Although time is still a proxy, qualification providers will be expected to specify the content that will be taught and assessed within that time envelope. 12. There may well be significant national or even regional variation in the geographical and political extent of this guarantee.

Learning Outcomes, Progression, and Qualifications 179

(Barrow 1976), which, in turn, is related to a set of aims (White 1982). Usually, there is a set of approved forms of pedagogy through which learning is enabled. Whether and to what extent a student or apprentice has learned what is prescribed in the curriculum is determined by assessment, both formal and informal. At some stage after formal assessment, the candidate is awarded a qualification. It is worth dwelling on the concept of prescribed content because this is a crucial issue for our understanding of both curriculum-led and non-curriculum-led learning. In most curricula (we can take a state-constructed national curriculum as an example), three main aspects of the content are covered: 1. The type of epistemic attribute to be developed. This is usually to be understood in terms of the traditional epistemological distinction between knowledge that, knowledge how and acquaintance knowledge or some combination of all of these. 2. The range of content covered. This could involve the academic or other kinds of subjects and the range of sub-content within these topics, the range of abilities to be developed, the varieties of acquaintance to be made, or a combination of some or all three of these. 3. Progression in depth of knowledge, understanding, or practice of the content prescribed. In most curricula, it is assumed that not only is there an interdependence in content, but that acquisition of some content depends on the prior acquisition of previous content. This imposes constraints on the sequencing of learning. This critical point is one that we shall have cause to return to. Credit transfer needs to be arranged so that credits are classed at different levels of cognitive difficulty, and there needs to be explicit statements as to (a) how many credits can be awarded at any given level, and (b) what the minimum level of credits are for transfer to learning at the next highest credit level. A question now arises as to the extent to which a qualification that is not formally tied to a curriculum can dispense with some or all of these requirements. The issue of the viability of outcomes-based qualifications depends critically on whether or not the third requirement can be dispensed with, or at least modified so that a curriculum is not a prerequisite of awarding a qualification. Before we can answer this, we need to look more closely at “traditional” content-based qualification routes.

180

Chris Winch

In the “traditional” content-based approach to the construction of a qualification, a curriculum, as part of a formal educational process leading to a qualification—whether academic or professional—is usually constructed along the following lines: 1.

Aims are set out for the desired end-point for a qualification holder. These will usually include the epistemic attributes that are to be developed.

2.

The content range will be specified.

3.

Although progression will be expected, it is often implicit in the way in which content range is specified. Thus, there may be some content that can be acquired independently of other content, but the acquisition of some content will be dependent on the prior acquisition of some other. This is usually indicated both through the sequencing of the content and the use of concepts which have been introduced at some prior point in the setting out of the content.

The last point is particularly important, as it suggests that progression may be compromised if, for example, content B depends on the prior learning of content A and content A is not learned sufficiently well. This has implications both for pedagogy and for assessment, and it is relevant to the construction of credit pathways within an award. Pedagogically, it is important that teachers and lecturers have a good grasp of what students have and have not learned. If they cannot do this, then their ability to maintain the progression of their students is compromised. The practice of integrating assessment of pupil progress as part of a recursive procedure for adjusting content to current understanding is a critical part of the diagnostic techniques that teachers require to ensure progress through the curriculum. It is particularly important when understanding the content is dependent on a prior understanding of other content. As Flew (1976) argued, the ability to do this is simply a mark of a teacher who takes his or her work seriously. Nowadays we call it “formative assessment.” However, it is also quite likely that more formal approaches to assessment will be required to provide evidence that a student is ready to progress to a further stage. Thus, the content of the curriculum can be distilled into standards, which a student must achieve to be deemed to have reached a certain level of understanding. Until recently, the En-

Learning Outcomes, Progression, and Qualifications 181

glish national curriculum made use of attainment targets at a series of stages in each subject area to show what needed to be known or understood before the next stage could be accessed. Attainment targets are not in themselves assessment instruments, but they can be used to develop them to perform this task. Thus, from the 2010 version of the English national curriculum at Level 2 for writing we find: • Pupils write one or two short sentences, following a model, and fill in the words on a simple form. • They label items and write familiar short phrases correctly. • When they write familiar words from memory, their spelling may be approximate (Qualifications and Curriculum Development Agency 2010). Assessment instruments can be designed to gauge such understanding, as the attainment targets themselves are written at too high a level of generality to serve as assessment instruments. It should be noted, however, that superficially they could look very much like learning outcomes, albeit rather general ones. It should also be noted that their use presupposes the existence of a curriculum from whose content they have been derived, making them quite different from outcome-based statements. Outline of Epistemic Ascent We thus need to take a closer look at the claim that a curriculum can be progressive in the way in which it introduces students to knowledge, ability, and understanding, and how these relate to each other. Recall that understanding of some content often presupposes the understanding of some previously acquired content. But what does this mean? We need to distinguish three distinct, but closely related human epistemic capacities, all three of which are usually developed to some degree or other within a curriculum, whether it be academic or professional (Winch 2013a). Knowledge by acquaintance. This concerns what we know through sensory experience, for example through observation in astronomy, through a field trip in geography, through examining a document in history, or through observation of an experiment in physics. Knowledge by acquaintance is rightly considered to be important in developing students’ understanding of a subject, by acquainting them with the significant properties of the matter that they are studying, allowing

182

Chris Winch

them to develop know-how and to handle and manipulate material, and allowing them to see first-hand connections between different phenomena. Propositional knowledge. This is knowledge that something is the case, whether it be an empirical proposition concerning the birth of some historically significant individual, a mathematical formula, or a scientific law. Although each of these three have different logical properties, it makes sense to say that they are propositions that can be known. We should note, as it will be important to what follows, that propositions invariably hang together in inferential patterns, such that one can be inferred from the other. While some of these connections are formal, depending on the meaning of trans-subject logical operators, others are material, depending on conceptual connections to a large extent internal to a subject matter.13 Two Kinds of Inference Within a Knowledge Field As an example of the first case, familiar to students of science, the student should appreciate that if a proposition framed as a hypothesis implies an experimental prediction and an experiment fails to realize that prediction, then, other things being equal, one should reject the hypothesis. This is an example of a logical law known to logicians as modus tollens.14 The second case can be seen in the relationship between concepts. Thus, if I claim that Canada is a federal state, I imply that its component states have some degree of political autonomy, as this is what is meant by saying that a state is federal. Brandom (1998) describes this kind of inference: “The kind of inference whose correctnesses essentially involve the conceptual contents of its premises and conclusions may be called, following Sellars, ‘material inference’” (97). Acquisition of subject knowledge thus involves increasing conceptual grasp manifested through the ability to make, implicitly or explicitly, relevant inferences. Such ability may be assessed informally by a teacher asking students to explain the division of responsibilities between the federal Canadian state and its constituent states. Ability to find one’s way around a body of knowledge through inference is thus a form of know-how. 13. See Brandom (1998) for an explanation of this distinction. 14. I do not mean to imply that all scientific reasoning works in this way, but rather that the student of experimental science needs to be able to grasp such inferences.

Learning Outcomes, Progression, and Qualifications 183

Know-how. The ability described immediately above is a form of know-how—in this case, the ability to trace a political division of labour implied by the existence of a federal state. But we can without difficulty point to the pervasive importance of the acquisition of knowhow in all aspects of the curriculum. The ability to seek information, perform calculations, read, write, perform experiments, collect specimens, or record observations are all part of the know-how required to operate at even a fairly elementary level in a subject area, and have to be factored into curriculum design and assessment procedures. This know-how is not to be conceived of solely in terms of skills, but also in terms of higher-order abilities sometimes described as transversal, such as the ability to plan, coordinate, or evaluate.15 These may require prior skill acquisition but cannot be reduced to it. Consideration of the various kinds of epistemic ability developed within a typical curriculum should alert us to the interdependence not only of items of propositional knowledge but also of kinds of knowhow. Not only are the contents of most curricula interdependent, but they are also progressive in the sense that acquisition of one item will often be a prerequisite for the acquisition of another. This has considerable implications for our understanding of the nature of learning outcomes. First, we need to consider what is involved in having at one’s command a body of knowledge. We assume that, in the case of the knowledge relevant to academic and professional purposes, this will be systematically organized and often contain a theoretical basis consisting of central concepts within a discipline, such as cause, mass, and momentum in physics. Such concepts are themselves embedded in a propositional context, such as laws expressing the properties that flow from these concepts. These may, in turn, be so central to the discipline that they have a normative rather than an empirical force (Wittgenstein 1969) and serve as a warrant for material inferences within the subject (Toulmin 1957). Growing knowledge of a subject does not then just involve growing ability to recall a range of propositions accurately, but also finding one’s way around a subject by being able to make judgments and inferences that display an understanding of relevant conceptual connections. Subject knowledge at more than a minimal level thus involves an element of know-how, itself a form of inferential ability demonstrating an understanding of the subject matter. Someone who is an expert in a subject, therefore, is not merely able to recall a range of relevant proposi15. These should not themselves be reduced to skills (see Winch, 2013b).

184

Chris Winch

tions accurately but is better at finding his/her way around the subject through inferential ability. A similar point applies to knowledge by acquaintance. In order for someone to understand what they are experiencing, they need to have a range of concepts that will allow them to make sense of their experiences. If I look through a telescope and am told that what I am seeing is Jupiter and four of its moons, this will make little sense to me unless I understand the concepts planet and moon and how they are related. My understanding of what I see will be demonstrated in my ability to make use of my observations in discussions of and judgments about planets and moons in the context of astronomy. It is often maintained, particularly in the progressive tradition (see the discussion in the next section below), that understanding a subject or field of enquiry involves being able to carry out the kinds of activities of enquiry that are associated with it. Thus, one cannot understand experimental science if one is not able to conduct an experiment, history if one cannot examine documents, geography if one cannot conduct a field study, or one’s mother tongue if one cannot write a story. It is almost invariably the case that the teaching of subjects in contemporary classrooms in the developed world depends heavily on such activities and they are deemed essential to the understanding of central concepts within the subject. Thus, one cannot have anything more than a superficial appreciation of the experimental method unless one is able to conduct at least a simple experiment. One cannot conduct an experiment if one has no grasp of the central concepts of experimental science, such as cause and effect, control, intervention, etc. So, we can say that the ability to carry out and to understand the practical activities associated with a subject depends on prior conceptual grasp. When students are asked to carry out experiments or undertake investigations in formal educational settings, it is almost invariably the case that these activities are replicative rather than original, despite some of the more ambitious statements in some curriculum documents. Thus, a key experiment underpinning a particular law of physics may be replicated in the laboratory, geographical fieldwork will examine a particular sequence of sedimentation, or examination of historical documents will reprise earlier interpretative work undertaken on these documents. Such work can be extremely valuable in allowing students to acquire the skills necessary to carry out a physical experiment, to observe and accurately record, or to read and understand. But it is just as important to understand what such work does not do.

Learning Outcomes, Progression, and Qualifications 185

Someone researching physics, for example, typically frames and tests a hypothesis, devises a way of testing it, and evaluates the result in light of the hypothesis. To do this, she or he needs to have the ability to evaluate where there are important gaps in knowledge; and to do this, she or he will need to have a very good grasp, not only of the central concepts of the subject16 but also of the latest developments. The experimenter must have a very good sense of what is important and be able to plan an experiment, coordinate effectively with other researchers, control and modify the conduct of the experiment where necessary, communicate both with the research team and with an academic audience, and assess the extent to which the hypothesis tested has been confirmed by the experimental results. Such wide-ranging abilities are the result of deep knowledge of the subject and technical skill in running experiments. But the ability to evaluate, plan, etc. does not just involve the practice of a set of skills, but a depth of knowledge, a seriousness of purpose, and the powers of attention to make sure that planning comes to fruition, that communication occurs, and that assessment is accurate. Whether or not such abilities are possessed by the experimenter may not be determined until such a project has been carried through (Hasselberger 2014), since the exercise of relevant skills by themselves will not do it. These transversal abilities, distinguished from skills through their polyvalent nature and the great extent to which they involve deep propositional knowledge (in the sense outlined above), qualities of attention, and seriousness of purpose in relation to a goal,17 are critical to the development of such higher-order abilities as being able to conduct original research within a subject. We can conclude that there is such a thing as progression in an academic subject and that much of it can be described in terms of knowhow. We should bear in mind, however, that different institutions and awarding bodies will have different conceptions of progression that will not necessarily fit easily together with each other. Here the use of a national or international qualification framework may be useful to 16. I have used the term “subject” throughout, but in some contexts, such as this one, it refers to the discipline as opposed to disciplinary knowledge reconfigured (recontextualized) for pedagogic purposes. 17. We should also note that such an ability as planning may be manifested in different ways according to personal preference and circumstance and that the role of attention and seriousness is necessary to distinguish, for example, seeming to plan and actually planning.

186

Chris Winch

ensure that different institutional curricula and awarding bodies do not diverge in their requirements to such an extent that it is not possible to transfer learning acquired in one institution or awarding body to another. Good subject knowledge involves knowing one’s way around the subject, understanding what is important, grasping central concepts and the propositions and normative statements associated with them, and being able to readily make inferences within the subject. Progression involves a practical understanding of what is involved in knowledge acquisition and evaluation, through the ability to conduct replicative activities that practice the skills needed to carry through, for example, fieldwork, systematic observation, documentary analysis, and experimentation. Finally, the ability to make significant contributions to the advancement of knowledge within a subject requires not only deep subject knowledge, including an acute appreciation of what is not yet known, but higher-order abilities connected with project management. Whatever form of assessment for a qualification is adopted, whether it be standards-based or learning-outcomes-based, it will have to take account of the progression of expertise. The above discussion has focused primarily on qualifications in academic subjects. What are the main issues concerning progression in professional qualifications? These too often involve subject expertise, but there are some important further considerations to take into account. The most obvious is the need for a qualification holder to demonstrate occupational competence. But this requirement itself entails further considerations. First is the question as to whether this competence can be captured in skill specifications. Second is whether or not there is scope for distinguishing, within qualifications or between qualifications arranged in a hierarchy of cognitive levels, between competence and expertise (Dreyfus and Dreyfus 1996). Third is the question as to how one is to judge the ability of a candidate to apply subject knowledge to professional judgment and action. One cannot answer the first and second of these questions without addressing the third. Those occupational qualifications that rely on a systematically organized body of knowledge to serve at least as a partial basis for action and judgment need to guarantee that the qualification holder can, in fact, make such judgments and act in a way that expresses them. We need to take account of the fact that this does not just involve single actions, but also more extended agency such as project management, which in turn relies on abilities to plan, coordinate, evaluate, etc. We also know that occupations that rely on this kind of knowledge usually

Learning Outcomes, Progression, and Qualifications 187

also involve complexity and variety in the range of situations in which such judgment is exercised.18 This, in turn, means that it is largely impossible to assess competence in such a range of situations directly. Assessment, therefore, has to rely to some extent on (a) testing responses to hypothetical situations, (b) post hoc assessment of rationale for judgment, (c) observation and evaluation of extended sequences of articulated professional actions, or (d) some combination of the above three. The third of these requirements deserves some attention. Articulated autonomous action in professional contexts involves the management of a project; that is, a significant form of professional action which involves a high degree of agent independence and discretion, extended over a period of time, usually with various stages and an identifiable outcome. As such, it will involve articulated sequences of planning, communication, coordination, control, and evaluation. A consequence of the polyvalent nature of these abilities is that they cannot be reduced to a set of skills but must be assessed in the context of the full completion of the project, and may thus be manifested in different ways in different contexts.19 Since it is likely that such an assessment will involve making a judgment as to the extent that the candidate has been able to align aims, situational considerations, and options derived from theoretical considerations, some degree of dialogue with the candidate will be necessary. The first and third of these points are distal, involving considerations relevant to action but not available in the immediate professional environment, while the second, being proximal, is more susceptible to in situ assessment.20 These considerations allow us to think of cognitive progression in terms of epistemic ascent from fragmented propositional knowledge and knowledge by acquaintance to a more holistic, inferential grasp of subject matter, to an appreciation of the work involved in replicating and validating existing knowledge, to an appreciation of the work involved in creating new knowledge and rejecting older, previously accepted claims. Parallel to this is a progression in know-how from skill to transversal abilities to project management abilities. 18. The presence of this range proved to be a challenge for the learning-outcomes-based NVQs in England, since at higher levels the range of activities in which the learning outcome was operative tended to be large (see Raggatt and Williams 2002). 19. We should also allow for variation in the ways in which individuals go about planning, and so on. 20. See Keestra (2018) for a good account of how proximal and distal considerations are relevant to judgments of expertise.

188

Chris Winch

There is no reason to suppose that professional qualifications pose special difficulties for credit transfer, provided that there is a qualification framework that provides enough structure to allow transfer from one institution or awarding body to another. One way in which to do so is through the use of a “transparency tool” that sets out the possible range of knowledge, know-how, and personal attributes that a qualification holder is expected to have (Winch 2015). This is described in Table 7.2. What Are Learning Outcomes? Answering this question is not easy. This is, first, because terminology can be confusing, and second because conceptually it is not a straightforward matter to understand different formulations of the ideas behind learning outcomes. However, it is useful to consider their rise to influence historically, to see what the key ideas behind their introduction were. Allais (2014) is a useful guide here, as she successfully disentangles two distinct but related sets of ideas that have come together to form the current movement in favour of the adoption of learning outcomes. Task Specification The first is the movement to make the economy more efficient through detailed specification of tasks. This idea, first championed by Adam Smith (1776/1981) and brought to prominence in the twentieth century by Frederick Taylor (1911), is most concerned with preparing workers for the tasks they need to perform. On this view, much formal educational content is irrelevant to these needs, and at the very least it should be expunged from vocational education even if not necessarily from education for civic and personal aims.21 This approach to vocational preparation entails reducing the power of educational institutions in favour of employers, for the latter rather than the former are better able to state what is needed for the efficient performance of economic tasks. It is allied with the view of “educational producer capture,” whereby educational institutions are motivated by internal goals of remuneration and influence rather than by the needs of their “consumers.” This was a strong theme in Smith’s (1776/1981) treatment of vocational, pro21. Smith (1776/1981, 784–786) makes the case for mandatory elementary education for all the population for civic rather than economic reasons.

Contingent facts (e.g., local conditions)

Local procedures

Materials

Technical theory

Normative theory

Social science theory

Occupational capacity2

Project management ability1

Transversal abilities

Skill

Mastery of technique (skill exercised in non-operational conditions)

Know-how Each characteristic presupposes possession of the one above (apart from technique)

Attributes

Civic

Other locations

Workplace

Other locations

Social

At least one of these will be involved in knowhow above a threshold level of ability

Workplace

Individual

Personal characteristics

Liberal

1. These would typically be sequences of transversals, e.g., from planning to co-ordination to evaluation. 2. This is a key feature of qualifications in the German dual system: berufliche Handlungsfähigkeit.

Note: Adapted from Winch, (2015).

Non-systematic

Systematic

Knowledge

Vocational

Aims of Qualification

Outline of a Transparency Framework for Professional Qualifications

Table 7.2

Learning Outcomes, Progression, and Qualifications 189

190

Chris Winch

fessional, and general education. Allais (2014) correctly identifies this line of thought as being responsible, for example, for the introduction of NVQs in England and similar developments in Australia. It can also be seen in the North American occupational classification framework O*Net (see O*Net Online, n.d.). Curriculum Deconstruction The second influence identified by Allais (2014) also arose in the eighteenth century, this time with Rousseau (1762/1911), and is broadly identified as “progressivism,” also taking in such figures as John Dewey and the late twentieth century “deschoolers.”22 Like the Smith (1776/1981) tendency, the progressivists regarded traditional educational thought with disdain and were particularly suspicious of the division of formal knowledge into academic subjects.23 Progressivists are, by and large, more interested in the outcomes of learning processes rather than formal educational means of achieving them. Formal education is not necessarily the best way of harnessing the innate motivation of learners and should not expect a privileged role in doing so. Thus, Rousseau (1762/1911) emphasized the role of nature in learning and the importance of learning a practical craft. Dewey and his followers championed the project method of learning, which transcended subject boundaries and emphasized experience and practical engagement. It would be true to say that both Smithian vocationalism and progressivism placed value on the idea that desirable educational outcomes could be based on processes that did not depend on formal education. In this sense, they are both “output-oriented” philosophies of education, and both have made powerful and complementary contributions to the learning outcomes movement. Learning Outcomes and Standards We will conclude this section by considering what is distinctive about learning outcomes. The best way is to refer back to our discussion of standards above. As we saw, standards are conceptually tied to content or “inputs.” Learning outcomes, however, are not; they are inde22. It should be noted that Dewey opposed the Taylorist tendency within North American industrial training but, as Allais (2014) points out, both tendencies shared some crucial assumptions. 23. As Allais (2014) demonstrates, Dewey’s position on this question was ambiguous.

Learning Outcomes, Progression, and Qualifications 191

pendently specifiable and may arise from formal education through a curriculum but may well not.24 The two may look superficially similar, but their role is different. More confusingly, it is sometimes the case that what are called “learning outcomes” actually play the role of standards. It is important not to dwell too much on the details of terminology, but rather to examine the underlying role that these statements play in the formulation of qualifications. Key Design Features of Learning Outcomes Qualifications: Independent, Specifiable, Hierarchical What are the key design features of outcomes-based qualifications? Briefly, learning outcomes are statements of what a learner is able to know, do, or understand, which do not need to relate to a program of prior learning.25 The previous discussion will help us to make sense of these qualifications. It should be noted that these are pure design features reflected, for example, in “transformational” approaches to qualification frameworks (Allais, Raffe, and Young 2009). In reality, the situation is often messier, with studied ambiguity relating to the need to find political accommodation, for example in the evolution of the EQF. The following three features26 of learning outcomes entail that they should be usable on their own and will ascribe a certain cognitive level to those who meet their requirements: 1. Independence: A learning outcome should be understandable and usable without reference to other learning outcomes. 2. Specifiability: The formulation of a learning outcome should be sufficiently precise to avoid ambiguity about what it means (this 24. See Coles (2007) for a clear explanation of the distinction. 25. Note here that the Ontario Council on Admissions and Transfer (ONCAT) position paper (ONCAT 2015) goes on to explicitly link such statements with content (4). It is not true, therefore, to say that learning outcomes conceived in this way pay no attention to inputs. See also the British Columbia Council on Admissions and Transfer (BCCAT) document (FitzGibbon 2014), which also ties learning outcomes to collegiate experience or a process of learning (2). As we have seen, statements of standards also refer to what the candidate should possess as a result of learning, which in turn is tied to prescribed content. One can call such attributes “learning outcomes,” but they actually function like standards. This point is reinforced in the Ontario Ministry of Training, Colleges, and Universities (n.d.) specification of postsecondary program standards as being related to curricula, albeit curricula that different colleges may configure differently. 26. See Brockmann, Clarke and Winch (2008) on NVQs, for example.

192

Chris Winch

may mean that it will need some contextual specification as well). 3. Hierarchicality: Although this is not strictly necessary, a learning outcome invariably occupies a place on a cognitive hierarchy.27 Thus, learning outcomes tend to be assigned to an ordering corresponding to exit points in formal education, or to levels in qualification frameworks. It is not obvious that such requirements are internally contradictory, but further scrutiny of examples shows that they give rise to some serious problems (see Winch 2014 for a detailed example of the problem). How can we explain the potentially inconsistent features of learning outcome specification? A common root seems to be that knowledge, whether it be by acquaintance, propositional, or know-how, is ordered hierarchically in the sense of epistemic ascent outlined above. Students have to master this hierarchy as they become more expert in their subject or occupation. In traditional “input-based” qualifications hierarchical ordering is recognized in the curriculum and assessment can be made by extracting criteria at various points in the hierarchy (standards) and devising assessment instruments. There are thus strong internal links between assessment instruments, standards, and curriculum. A pure learning outcomes approach (one that adopts 1, 2, and 3 as defining criteria) attempts to sever these links and to substitute independent and specifiable learning outcomes in place of standards. But because it cannot abolish epistemic hierarchy, it finds itself with a problem of consistency. One response to these apparent difficulties is to point to existing examples, such as graded music tests, which can be entered at any level without presupposing success at a level lower in the hierarchy of expertise. The example points to an interesting aspect of some kinds of competence, including competence in playing music. No one will deny that there is a hierarchy of competence here, which is underpinned by growing concept mastery. However, it is very hard to see how someone could master the abilities and concepts needed for success at Level n, without having first mastered those at Level n-1 and lower. These are all implicitly presupposed within the assessment of competence at Level n. Why, then, it might be asked, cannot implicit learning outcomes be incorporated into any learning outcome at n, without having first to

27. See Allais’s (2014) discussion of the use of Bloom’s taxonomy, for example.

Learning Outcomes, Progression, and Qualifications 193

assess n-1, n-2, etc.?28 The problem for professional qualifications is first that there may not be the same nesting of competences within other competences as is found in the ability to play a musical instrument, as in the example above. The second point is that it is simply not an option to recognize a professional competence implicitly, particularly when it is safety critical—for example, the ability to drive a commercial vehicle safely. The problem for academic qualifications is the issue already discussed in the section on know-how above, namely that academic expertise is hierarchically ordered, and guarantees are required that candidates have mastered concepts and abilities lower in the cognitive hierarchy before they can, for example, be allowed to work at the doctoral or post-doctoral level. There may be rare exceptions, but this is generally the case. The problem of credit transfer between institutions can be addressed through a qualification framework of the kind described above. Cross-disciplinary transfer, on the other hand, will need to be handled through negotiation with bodies (either institutions or awarding bodies), which are responsible for determining relevant prerequisites for further study. If one has three mutually inconsistent criteria that define a particular entity, then the inconsistency must be eliminated by removing one or more of the criteria. It is easy to see that criterion three (hierarchicality) cannot be removed, and also that it is inconsistent with both criteria one (independence) and two (specifiability). It is inconsistent with criterion one because outcomes cannot be specified independently of any others, except perhaps at the lowest level of a cognitive hierarchy. It is inconsistent with criterion two because higher-level outcomes that presuppose lower-level ones will leave it unclear as to what extent the lower-level ones are contained in the higher-level ones. It is difficult to avoid the conclusion that criteria one and two will have to be abandoned if a coherent learning outcomes approach is to be adopted. What is left, criterion three, will be implicitly tied to content and will entail that outcome statements will be non-independent (except perhaps at the lowest level) and will implicitly incorporate outcomes at lower levels by presupposing their prior achievement.

28. Notice however that the English Qualification Authority (Ofqual 2018) does require a formal curriculum and instruction for the award of graded music certificates (70).

194

Chris Winch

Conclusion Learning-outcomes-based qualifications that use the “strong” output-based conception of a learning outcome are, despite their superficial attractiveness, flawed. The history of their usage does not inspire confidence and the most recent attempts at the European level to develop a strong learning outcome approach, which is intended to have a transformational effect on European vocational qualifications, is unlikely to succeed any more than the NVQ approach in England did. However, if “learning outcomes” are conceived of more like “standards” related to curricula and notional study hours then the design flaws of the strong version can be eliminated. But then, learning outcomes are not quite the radical approach to learning that they originally appeared to be. Where they might be of value in the Canadian context is in setting out criteria that any program at a given level will need to satisfy. They are thus a “soft” constraint on curriculum construction, not a “hard” specification of a level of achievement that can be assessed independently of any underlying content. References Allais, S. 2014. Selling Out Education: National Qualification Frameworks and the Neglect of Knowledge. Rotterdam, The Netherlands: Sense. Allais, S., D. Raffe, and M. Young. 2009. Researching NQFs: Some Conceptual Issues. Geneva, Switzerland: International Labour Office and Skills and Employability Department. Barrow, R. 1976. Commonsense and the Curriculum. London, UK: Allen & Unwin. BIS. 2011, September. BIS Research Paper Number 53: Returns to Intermediate and Low Level Vocational Qualifications. London: HMSO. https://assets.publishing.service.gov.uk/government/uploads/ system/uploads/attachment_data/file/32354/11-1282-returns-intermediate-and-low-level-vocational-qualifications.pdf Brandom, R. 1998. Making It Explicit: Reasoning, Representing, and Discursive Commitment. Cambridge, MA: Harvard University Press. Brockmann, M., L. Clarke, and C. Winch. 2008 “Knowledge, Skill, Competences: European Divergences in Vocational Education and Training (VET) – The English, German and Dutch Cases.” Oxford Review of Education, 34 (5): 547–567. ———. “Competence and Competency in the EQF and in European VET Systems.” Journal of European Industrial Training, 33 (8/9): 767–799.

Learning Outcomes, Progression, and Qualifications 195

———. 2010. Bricklaying Is More Than Flemish Bond: Bricklaying Qualifications in Europe. London, UK: Lifelong Learning Programme. https:// westminsterresearch.westminster.ac.uk/item/9040z/bricklaying-is-more-than-flemish-bond-bricklaying-qualifications-in-europe City and Guilds. 2015. Level 2 Diploma in Heath and Social Care (Adults) England. London, UK: Author. Clarke, L., and C. Winch. 2015. “Have Anglo-Saxon Concepts Really Influenced the Development of European Qualifications Policy?” Research in Comparative and International Education, 10 (4) 593–606. Coles, M. 2007. Qualifications Frameworks in Europe: Platforms for Qualifications, Integration and Reform. Brussels, Belgium: Education and Culture DG. Dreyfus, H. L., and S. E. Dreyfus. 1996. “The Relationship of Theory and Practice in the Acquisition of Skill.” In Expertise in Nursing Practice: Caring, Clinical Judgment, and Ethics, edited by P. Benner, C. Tanner, and C. Chesla, 29–47. New York, NY: Springer. European Centre for the Development of Vocational Training. 2008. Terminology of European Education and Training Policy: A Selection of 100 Key Terms. http://www.cedefop.europa.eu/EN/Files/4064_ en.pdf European Commission. (n.d.) “Descriptors Defining Levels in the European Qualifications Framework (EQF).” Accessed February 4, 2019. https://ec.europa.eu/ploteus/content/descriptors-page European Credit System for Vocational Education and Training (ECVET). n.d. “What Is ECVET?” Accessed February 4, 2019. http:// www.ecvet-secretariat.eu/en/what-is-ecvet FitzGibbon, J. 2014. Learning Outcomes and Credit Transfer: Examples, Issues, and Possibilities. A BCCAT Special Report. https://www.bccat. ca/pubs/Learning_Outcomes_and_Credit_Transfer_Feb2014.pdf Flew, A. 1976. Sociology, Equality and Education: Philosophical Essays in Defence of a Variety of Differences. London, UK: Macmillan. Hanf, G. 2011. “The Changing Relevance of the Beruf.” In Knowledge, Skills and Competence in the European Labour Market, edited by M. Brockmann, L. Clarke, and C. Winch, 50–67. Abingdon, UK: Routledge. Hasselberger, W. 2014. “Human Agency, Reasons, and Inter-Subjective Understanding.” Philosophy, 89 (1): 135–160. Hornsby, J. 2011. “Ryle’s Knowing How and Knowing How to Act.” In Knowing How: Essays on Knowledge, Mind, and Action, edited by J. Bengson, and M. Moffett, 80–100. Oxford, UK: Oxford University

196

Chris Winch

Press. Keestra, M. 2018. “Drawing on a Sculpted Space of Actions: Educating for Expertise While Avoiding a Cognitive Monster.” In Education and Expertise, edited by M. Addis and C. Winch, 75–98. Oxford, UK: Wiley. Méhaut, P., and C. Winch. 2012. “The European Qualification Framework: Skills, Competences or Knowledge?” European Educational Research Journal, 11 (3): 369–381. Ofqual. 2018. “Guidance to the General Conditions of Recognition.” London, UK: HMSO. https://www.youtube.com/user/hejinghan/ videos?app=desktop O*Net Online. n.d. “Build Your Future with O*Net Online.” Accessed February 5, 2019. https://www.onetonline.org/ Ontario Council on Articulation and Transfer (ONCAT). 2015. ONCAT Position Paper: Learning Outcomes in Credit Transfer. http://67.21.231.62/sites/default/files/oncat_positon_paper_on_ learning_outcomes.pdf Ontario Ministry of Training, Colleges and Universities. n.d. “What Does a Program Standard Contain?” Accessed February 5, 2019. http://www.tcu.gov.on.ca/pepg/audiences/colleges/progstan/ contain.html Payne, J. and J. Keep. 2011. One Step Forward, Two Steps Back? Skills Policy in England under the Coalition Government. SKOPE Research Paper, vol. 102. Cardiff, UK: SKOPE, Oxford and Cardiff Universities. http://orca.cf.ac.uk/25001/ Qualifications and Curriculum Development Agency. 2010. The National Curriculum: Level Descriptions for Subjects. https://dera.ioe. ac.uk/10747/7/1849623848_Redacted.pdf Raggatt, P., and S. Williams. 2002. Government, Market and Vocational Qualifications: An Anatomy of Policy. London, UK: Falmer. Richard, D. 2012. The Richard Review of Apprenticeships. https://dera. ioe.ac.uk/17733/1/richard-review-full.pdf Rousseau, J.-J. 1911. Emile ou de l’éducation. London, UK: Dent. (Original work published 1762). Smith, A. 1981. The Wealth of Nations. Indianapolis, IN: Liberty. (Original work published 1776). Stanton, G. 2008, June 9. Learning Matters. Presentation to the CfBT Trust. Reading, UK. Taylor, F. 1911. The Principles of Scientific Management. New York, NY: Norton.

Learning Outcomes, Progression, and Qualifications 197

Toulmin, S. 1957. The Uses of Argument. Cambridge, UK: Cambridge University Press. White, J. P. 1982. The Aims of Education Restated. London, UK: Routledge. Whitehead, G. 2013. Review of Adult Vocational Qualifications in England. https://www.gov.uk/government/uploads/system/uploads/ attachment_data/file/30390 6/review-of-adult-vocational-qualifications-in-england-final.pdf Winch, C. 2013a. “Curriculum Design and Epistemic Ascent.” Journal of Philosophy of Education, 47 (1): 128–146. ———. 2013b. “Three Different Conceptions of Know-How and Their Relevance to Professional and Vocational Education.” Journal of Philosophy of Education, 47 (2): 281–298. ———. 2014. “Are the Dual System and Qualification Frameworks Compatible with Each Other?” In The Challenges of Policy Transfer in Vocational Skills Development: National Qualifications Frameworks and the Dual Model of Vocational Training in International Cooperation, edited by M. Maurer and P. Gonon, 343–363). Bern, Switzerland: Peter Lang. ———. 2015. “Towards a Framework for Professional Curriculum Design.” Journal of Education and Work, 28 (2): 165–186. Wittgenstein, L. 1969. On Certainty. Oxford, UK: Blackwell. Wolf, A. 2011. Review of Vocational Education. London, UK: HMSO.

8

Shifting Paradigms in Postsecondary Education: Historical, Conceptual, and Theoretical Frameworks Governing Outcomes-Based Approaches to Credit Transfer Christine Arnold, Mary Wilson, Michael Potter, and Leesa Wheelahan

Introduction The problematics and possibilities of employing learning outcomes as a tool to inform credit transfer are rooted in the conflicting origins and rationales for outcomes-based education and the development of student pathways, which are based on several theoretical and conceptual considerations and movements. Affordability, accountability, and assessment concerns from policymakers and institutions resulted in substantial shifts within postsecondary education (Watt 1999). International reforms in postsecondary education, particularly in the 1980s and 1990s, fuelled outcomes-based and standards-based reforms in Anglo -Saxon countries as features of the student-centred teaching and assessment movement (Tremblay, Lalancette, and Roseveare 2012). Some of the foundations for this movement can be traced to the interest in proLearning Outcomes, Academic Credit, and Student Mobility, edited by Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon. Montréal and Kingston: McGill-Queen’s University Press, Queen’s Policy Studies Series. © 2020 The School of Policy Studies, Queen’s University at Kingston. All rights reserved.

200

Christine Arnold, Mary Wilson, Michael Potter, and Leesa Wheelahan

grammed and mastery learning of the 1960s and 1970s (Watt 1999). In this chapter we delineate the historical foundations that govern outcomes-based education, including early adopters and the two contributing movements: (1) the economic, market-driven, and managerial movement in the postsecondary education industry; and (2) the student-centred movement among educational scholars and practitioners. Following this, we focus our attention on the overarching theoretical and conceptual constructs that have assisted in guiding our reading of the international scholars’ contributions. We explore three constructs to analyze outcomes-based approaches to credit transfer, which include alternatives for policy development (creation, adoption, and/or adaptation), conflicting logics that govern systems and institutions, and conceptualizations of learning progression. More specifically, we investigate considerations of policy learning versus borrowing, the relative influences of intrinsic and institutional logics, and the nature of pedagogic discourse that can be used to understand these historical origins and their effects on policy, practice, and pedagogy. These constructs all have foundations in the two movements described and aid in situating the critical questions and considerations for examining the intersection of learning outcomes and transfer policies and practices, and considering the possibilities and problematics of employing learning outcomes as a tool for transfer. Finally, this chapter discusses implications for Ontario and provides context for the subsequent two chapters, discerning systems-oriented and institutional lenses through which to examine the contributions of learning outcomes in the facilitation of more effective credit transfer. Early Adopters and Movements Early adopters of 1980s and 1990s outcomes-based and standards-based reforms included Australia, New Zealand, the United Kingdom (including, in this instance, England, Northern Ireland, and Wales), Scotland, and South Africa (Allais et al. 2009). The emergence of outcomes-based education reforms and qualifications frameworks in these countries helped to shape their early diffusion in policy throughout the world (Raffe 2011a). A steady shift from traditional institution-based qualifications to outcomes-based qualifications occurred and received varying levels of support in each of these countries (Allais, Raffe, and Young 2009). This steady shift, discussed in the literature, was the consequence of two different movements, as noted above: the first movement an eco-

Shifting Paradigms in Postsecondary Education 201

nomic, market-driven, and managerial movement in the postsecondary education industry; the second is learning-centred and student-centred discourses among educational scholars and practitioners. Economic, Market-Driven, and Managerial Within Anglo-Saxon countries, outcomes-based and standards-based reforms were promoted as an impetus for economic reform due to their “delivery of measurable cognitive and affective outcomes” (Watt 1999, 3). With neoliberalism came pressures to increase market mechanisms and measurability in postsecondary education (Allais 2007). The application of neoliberal principles of privatization in public postsecondary institutions and goods brought with it product-centric rather than process-centric accountability and assessment, reduced reliance on public funding, and pressure to diversify revenue streams (Dobbins, Knill, and Vögtle 2011; Hood 1995; Lebeau 2012; Marginson and Van der Wende 2007; Parker and Gould 1999; Power and Laughlin 1992; Tremblay, Lalancette, and Roseveare 2012; Watt 1999). The increased use of such New Public Management methods in postsecondary education paved the way for market mechanisms and principles that have manifested in privatization (Dobbins, Knill, and Vögtle 2011; Tremblay, Lalancette, and Roseveare 2012). The term “New Public Management” was coined in the late 1980s to convey a renewed focus on the importance of management and production in public service delivery and includes the following principles: performance standards, output controls, decentralization, competition, private-sector management, and cost reduction (Hood 1991). Power and Laughlin (1992) describe this as the move toward “accountingization,” which includes public accountability and administration. These forces require evidence of program quality within postsecondary institutions to demonstrate the effective use of taxpayers’ contributions (Allais 2007). The application of competencies and learning outcomes were introduced to assist with this measurement agenda because they emphasize outcomes that can be measured according to accountability criteria. The drive for economic reform, which remains as a significant influence, seeks cost efficiency, contract research, competitive grants and funding, performance-based reporting and funding, and institutional rankings. The various metrics that Ontario colleges and universities are now obliged to gather and report on provide clear evidence of the increased transparency of the system, but also of its increasing ac-

202

Christine Arnold, Mary Wilson, Michael Potter, and Leesa Wheelahan

countingization and managerialism. The required metrics can be quite successful in steering institutions toward compliance with government policy, but can also be met with various degrees of buy-in and action. In Ontario, policies that seek differentiation and standardization both encourage and detract from recent performance and quality metrics such as the Strategic Mandate Agreements, Multi-Year Accountability Agreements, National Survey of Student Engagement, and Key Performance Indicators. The obligation to collect and report on these metrics creates additional administrative expense and load—both for the data collection process and the exercise of parsing the data for various required reports. The terms accountability and assessment fill conversations, processes, and policies of both governments and postsecondary institutions and as such many countries have come to include postsecondary education in performance-based reporting and funding. The assessment of various facets of postsecondary education has appeared to be an antidote for all perceived ills in the education system (Burke 2003). Colleges and universities have been asked to identify the knowledge and skills that students should possess upon graduating to assess their achievement and improve institutional performance (Shafritz and Russell 2000). While performance reporting can be used for many purposes, one important function aims to increase information symmetry between students (consumers) and institutions (providers). The information that is made available via public reporting is information that may not otherwise be available. With increased information about institutions it is assumed that students can make informed decisions about their education and better understand both its net economic worth and relative social benefit (Hook 2004; Lang 2004). Problems of asymmetric information arise locally and globally when students are crossing borders or moving between institutions, as they might struggle with their “consumption” choices if they are not provided with transparent measures of quality (Akerlof 1970; Douglas and Edelstein 2009; Nelson 1970; Spence 1973; Tremblay, Lalancette, and Roseveare 2012). Student-Centred Many researchers have credited Rogers and Freiberg’s (1994) book Freedom to Learn as the origin of student-centred learning; in it they discuss applying their client-centred approach to counselling to educational contexts. Further, they critique the traditional transmission model of teaching that is heavily moved forward by expertise-driven consider-

Shifting Paradigms in Postsecondary Education 203

ations. It has been noted that the student-centred learning movement focuses on the whole student learning, and is much more concerned with the process than the product and emphasizes the benefits of experiential learning (Tangney 2014). The holistic developmental process underpins much of their ideology, a facet that has been explicitly noted by Kember (1997) in his meta-analysis as distinct in some interpretations of a student-centred, learning-oriented conception. Various facets of student-centred learning discussed throughout the literature include the holistic development process, intellectual development, engagement in learning, discovery learning, building understanding, and reflecting on understanding (Kember 1997; Perkins 1999; Richardson 2003; Rogers and Frieberg 1994). Generally, students are seen as “active participators rather than passive receivers” (Tangney 2014, 267). Barr and Tagg (1995) identified a paradigm shift occurring in postsecondary education, moving from an instruction paradigm to a learning paradigm. Student-centred education includes methods of teaching, learning, and assessment that develop student autonomy by placing learning at the centre of all interactions, artifacts, and structures (Boyer 1990), focusing on the knowledge, skills, attitudes, and values that students should possess as lifelong learners (Young and Paterson 2007). The outcomes-based and standards-based reforms have used the discourse of student-centredness, concentrating on making manifest the ways in which various programs intentionally cultivate specific student capacities. Tam (2014) has argued that “sharpening the focus onto student learning outcomes goes beyond mere tinkering with traditional structures and methods; it constitutes a paradigm shift in educational philosophy and practice” (2). The focus on learning outcomes is believed to be student-centred because it focuses on the student and their learning needs rather than traditional pedagogic “teacher-centred” approaches; emphasizes what students can demonstrate they have learnt; and facilitates credit transfer and pathways by focusing on the outcomes of learning rather than syllabi. All in all, it is reasoned that these multiple benefits are possible because they enable a focus on the outcomes of learning, which recognizes that these outcomes can be achieved in multiple ways. Neither a Borrower nor a Lender Be The first of the three constructs we are exploring to analyze outcomes-based approaches for credit transfer involves alternatives for

204

Christine Arnold, Mary Wilson, Michael Potter, and Leesa Wheelahan

policy development (creation, adoption, and/or adaptation). Given the influence of historical and contextual factors in the intersection and use of learning outcomes for credit transfer, it is illuminating to study the experiences and lessons garnered from other jurisdictions around the world to understand our own context better through comparison. It is advantageous to explore the relative merits and risks of borrowing policy from related international efforts in contrast to learning from policy implementation in other jurisdictions. Policy Development Policies have been implemented in many countries to reform credentials and introduce qualifications frameworks, and to use these to promote student mobility, pathways between credentials, and credit transfer. These reforms have coalesced into common policies and discourses intended to address multiple perceived problems. These efforts can be examined according to the economic, market-driven, and managerial perspective, as well as the student-centred movements discussed above. These two movements are demarcated by the sorts of problems they are intended to address and the communities for whom they are salient. The first movement is characterized as one which • emerged primarily from industrial and corporate stakeholders, policymakers, and postsecondary education administrators; • tends to focus on credentials and credit transfer as part of a broader effort to improve transparency and accountability. Such policies seek to address skills deficits and cultivate transversal skills, create an open and flexible system to draw more students in at multiple points of life, respond to labour market demands, and remedy social stratification and economic inequities. The second movement is characterized as one which • is championed more predominantly by faculty members, educational developers, and postsecondary education administrators; • tends to focus on the relations among credentials and credit transfer systems and seeks to build a continuity of learning, recognition for prior learning, and a more coherent, barrier-free educational experience for students. Although there is some overlap between these movements and interchange between their proponents—especially regarding out-

Shifting Paradigms in Postsecondary Education 205

comes-based qualifications frameworks—there are also tensions, as goals and motivations are in some cases quite distinct between them. The valuable consequence of efforts to bridge such tensions successfully is evident in the historical evolution of some of the more compelling projects inspired by these movements, and these historical examples give credence to the wisdom of examining contextual factors when weighing the merits of approaches to policy implementation. For example, the Scottish Credit and Qualifications Framework (SCQF), which has evolved into a communicative, descriptive outcomes-based meta-framework of qualifications, has become an effective tool for supporting credit transfer and articulation (Scottish Credit and Qualifications Framework 2018). Raffe, Gallacher, and Toman (2008) note that time, process, and the cultivation of trust among voluntary partners have been essential components contributing to the utility of the framework for advancing policy. The SCQF developed incrementally over an extended period and engaged numerous stakeholders in its development and coordination, successfully brokering interchange between the proponents of different movements. Similarly, the Tuning Measuring and Comparing Achievements in Learning Outcomes in Higher Education in Europe (CALOHEE) project is a promising exploration of the potential to compare student learning outcomes by domain or subject area while respecting the diversity of missions, orientations, and profiles of European degree programs (CALOHEE 2018). The project is in accordance with the fundamental policy direction for cooperation and coordination of higher education emerging out of the European Commission (European Commission 2017). In the United States, the Western Regional Education Compact codifies a policy commitment to improving student access, credit recognition, inter-institutional cooperation and innovation. This policy spawned the American Western Interstate Commission on Higher Education Interstate Passport program—an initiative that demonstrates it is possible for partnering institutions to agree to grant block transfer of lower level general education credits across jurisdictions (Western Interstate Commission for Higher Education 2018). It is tempting when reading about these relatively successful international policy directions and tools for Ontario to consider simply adopting them wholesale; however, caution is warranted. Rather than the simple adoption of effective reforms, which tend to be treated as “quick fixes,” Raffe (2011b) has recommended a policy learning approach that uses international experience to enrich policy analysis. With all due

206

Christine Arnold, Mary Wilson, Michael Potter, and Leesa Wheelahan

consideration for the inherent contextual differences that exist among countries and educational systems, there is much to be gathered from a consideration of reform efforts and policy implementations in other jurisdictions to analyze the potential of learning outcomes as a tool for transfer in Ontario. However, there is sufficient difference among educational, economic, and social systems that it would be folly to endeavour simply to adopt a policy, framework, or tool unaltered from another jurisdiction. Rather than policy borrowing—which refers to a tendency to apply “off the rack” models from other countries without significant adaptation, as though there are best practices that can be universally applied—there is opportunity for greater wisdom in policy learning. Learning from Successes and Failures Policy learning begins “with an analysis of the situation facing the country in question, prior to deciding…what approach to implementation might be appropriate” (Allais et al. 2009, 3). The purpose of policy learning is to study one’s education system by contrasting it with other systems, identifying common trends and pressures, clarifying alternative policy strategies, and identifying practical issues likely to be raised by each strategy (Raffe 2007). The aim is to assist policymakers in devising their own country-specific solutions (Grootings 2007) that account for the social and political factors that determine, at least in part, their development, acceptance, and utility (Raffe 2011b). Rather than attempt to import another nation’s policies and frameworks into the Ontario system wholesale, an effort should be made at committed study of the evolution of those systems in different jurisdictions to compare, contrast, and learn—from successes and failures—and then to determine which elements might be valuable. Parsing out the unique influences of historical, socio-economic, system-design, and ideological differences is key to deriving useful insights from the analysis of international policy reform efforts associated with learning outcomes as a tool for transfer. Through consideration of particular challenges and opportunities within unique jurisdictions insights may be gleaned regarding how issues were met by policies, strategies, and processes, as well as how policy goals and barriers emerged and were defined. Matters such as whether rapid and massive efforts at change were chosen or if incrementalism was the preferred course of action, whether there was broad-based consultation and training or if the approach taken was primarily top-down or bottom-up, and

Shifting Paradigms in Postsecondary Education 207

whether plans were envisioned for research into the sustainability of reform efforts are all worthy of analysis. Fruitful reflection on productive methods for analyzing the applicability, transferability, and utility of various international policies and associated practices may also yield insights for Ontario. As an example of the importance of attending to the influence of socio-cultural context on the viability of educational reforms, consider Allais’s (see chapter 3) account of the co-option of a nominally outcomes-based framework to redress the lasting social ills and divisions caused by apartheid policy in South Africa. Apartheid policies were acutely entrenched in the design of the entire education system prior to 1994, inspiring policymakers to reform the system through radical, rapid, and top-down revisions to qualifications and the development of a qualifications framework, involving access and equity improvement efforts—all in the service of labour-market demands to drive economic advancement. The urgency of the work resulted in attempts to import policy frameworks and instruments from other jurisdictions, most notably Australia, and force their implementation through government edicts that did not allow for adequate consideration of how—and notably whether—the reforms could address the problems they were being used to solve. The complications created by this hurried top-down importation were compounded by decisions to eschew broad consultation, learning from collaborative effort, building trust, and planning for sustained support of change processes. Overall, the framework was not successful at the system level in South Africa and this “hard” model was overturned. Similarly, Ontario can draw insights from a comparative consideration of reform efforts in Australia. As Noonan (see chapter 5) notes, there have been periods in Australia where credit transfer in postsecondary education was not the main focus. However, there has been renewed interest in recent years, with the Australian Government, as part of an initial work plan, requesting that the Higher Education Standards Panel consider whether credit transfer policies and standards are adequately meeting the needs of students and institutions (Australian Department of Education and Training 2018). The Australian approach suggests that programs using learning outcomes can contribute to the establishment of credit transfer and that “differences in how learning outcomes are defined, qualifications designed, and assessment undertaken does not constitute an insurmountable barrier to credit transfer between the sectors” (Noonan, see chapter 5, 128). Established in

208

Christine Arnold, Mary Wilson, Michael Potter, and Leesa Wheelahan

1995 (and reformed in 2011), the Australian Qualifications Framework (AQF) had, as one of its aims, the facilitation of pathways within educational sectors, and between educational sectors and labour markets (Wheelahan 2011). However, these efforts have not been as successful as initially envisaged in policy, although there are agreements and policies in place that facilitate pathways between geographically proximal institutions, perhaps as the result of attention paid to cultivating mutual understanding and trust. In Australia, the AQF has had the greatest impact in the vocational education and training (VET) sector, and greater attention has been paid to system unification and rationalization/ improvement of VET qualifications, leading to a problematic adoption of competency-based credentials (Wheelahan 2011). This result has further complicated efforts to improve access and mobility, deepened suspicions about quality, and reinforced pre-existing divides between higher education and VET. Policy borrowing from the United Kingdom National Vocational Qualifications model led to an instrumentalist and narrow focus on competencies. This “tended to downplay the importance of underpinning knowledge and a holistic view of the ‘craft concept’” (Allais 2014, 61), more common in the German and Austrian systems where there is greater esteem for vocational education. Examinations of reform movements and tools in the United States also benefit from a consideration of historically rooted and contextually informed interests and responses to movements. Bragg (see chapter 4) outlines how in the United States—although there has not yet been enough effort—political will has been exerted and concentrated on addressing inequities in system access along race, gender, and class lines. There has been limited policy or practice work considering the potential for learning outcomes to serve as a tool for credit transfer, as states have been focused on improving credit attainment and graduation rates. Mobility and access partnerships were developed as a central responsibility of the two-year college system to address class and racial divisions and provide greater opportunity to disadvantaged groups. Although interest in system unification began to circulate with Parnell’s (1985) report The Neglected Majority and negotiated numbers of transfer students increased in the 1980s, tracks between vocational and academic programs remain distinct and esteem remains hierarchical. Power historically rests with receiving rather than sending institutions. The United States is just now starting to expand full baccalaureate offerings in their college system, moving from a “student deficit perspective that characterizes students as a problem to be fixed” to a consideration of

Shifting Paradigms in Postsecondary Education 209

system level and institutional level factors that influence student credit transfer and accumulation results (Bragg, see chapter 4, 95). This transference endeavours to ensure, through initiatives such as Credit Where Credit is Due, the granting of associate’s degree credentials to students where warranted, so that they can progress to higher qualifications. Respect for the unique influences of forces at play in policy formation and implementation within and across jurisdictions is advisable and should serve to subdue enthusiasm for simple policy borrowing. While there are certainly lessons to be learned from a consideration of outcomes-based efforts to reform credit transfer internationally, such lessons must be examined referentially and contextually. There are three significant domains of contextually defined complexity to consider in the creation, adoption, or adaptation of frameworks or policy tools intended to draw together outcomes-based disciplines and credentials across vocational and academic postsecondary education to promote credit transfer and accumulation. All three must be attended to successfully for system reform and the application of frameworks to take root: • epistemological difficulties of reconciling differences in curriculum, teaching, learning, and assessment when trying to connect academic and vocational learning; • political interactions between those with competing interests; and • institutional barriers entrenched in policies, practices, resources, and operational processes. (Raffe, Howieson, and Tinklin 2007) These difficulties—epistemological, political, and institutional—are explained through Raffe’s (2003) concepts of intrinsic and institutional logics. Respecting the Influence of Intrinsic and Institutional Logics The second of the three constructs we are exploring to analyze outcomes-based approaches for credit transfer is conflicting logics that govern systems and institutions. When moving from an examination of policy and the mechanisms crafted to implement and sustain reforms outlined in policy, respect for the dynamics of forces at play within the systems and institutions that function as the sites of reform implementation is warranted. Overarching system-level policies that may appear to be abundantly rational and desirable on their merits may prove to be fraught with challenges in implementation as a consequence of pre-existing or emergent countervailing priorities, policies, practices,

210

Christine Arnold, Mary Wilson, Michael Potter, and Leesa Wheelahan

and processes at the level of the individual institutions. Identifying and distinguishing between intrinsic and institutional logics reveals the complexities that surround successful reforms. Engendering inclusivity, trust, and transparency in process requires clarity of intended direction and goal-setting, along with shared commitments to devoting time, resources, and training. Educational Reforms Raffe (2003) and Young (2003) describe the intrinsic logic of educational reforms as the political rationale for policy objectives, and institutional logic as the actual context in which the reform is implemented. Simply stated, influential policies and practices may make a great deal of sense and resonate with people when considered abstractly, but may mask troubling assumptions and requirements that cause them to founder when instantiated by institutions functioning within real material and operational conditions. Consideration of the insights shared throughout this book indicates that there are problems in failing to attend to a potential disconnect between the intrinsic logics of an outcomes-based credit transfer reform policy or tool, and the institutional logics of the settings in which the policy or tool is to be put into practice. Just as systemlevel policies and tools are born of particular influences and directed toward particular aims, postsecondary institutions have their own histories, priorities, motivations, identities, values, and practices. When considering reforms and change within a postsecondary education system, it became evident from the international scholars’ work that while intrinsic logics may have seemed appealing when introducing outcomes-based and standards-based reforms, institutional logics need to be examined and expressed. While seamlessness, boundary minimization, credit transfer, and student mobility objectives are valuable and provide the intrinsic rationales for reforms seeking to achieve these objectives, institutional logics can undermine these objectives when situational implementation is not properly considered. For example, in Australia the Australian Qualifications Council in 2009 put considerable work into trying to overcome institutional problems regarding different types and levels of funding, different curriculum models, different institutional priorities, program structures, and hierarchical relations between colleges and universities (Banikoff, Symonds, and Doolette 2009; Brown et al. 2009; Wheelahan 2009). Arguably, these efforts have had limited success (Noonan, see chapter 5).

Shifting Paradigms in Postsecondary Education 211

Similarly, in recent years, Ontario has implemented governmental policy aimed at achieving access improvements, quality enhancements, and sustainability through system differentiation (Ministry of Training, Colleges and Universities 2013). The provincial government has plotted the achievement of these reforms through the establishment of policy directions and consultations, and the evolution of policy drivers. These include strategic mandate agreements, increased reliance on accountability instruments, changes to governmental processes, introduction of funding mechanisms (particularly through opportunities supported via arm’s-length agencies such as the Ontario Council on Articulation and Transfer, eCampus Ontario, and the Higher Education Quality Council of Ontario), and targeted initiatives such as the Learning Opportunities Fund/Career-Ready Fund. The need for postsecondary education system reform in the province of Ontario is premised on evidence of demographic decline, rising costs, a drift of colleges and universities toward homogeneity (Jonker and Hicks 2016), and competition for declining investments. Arguments for a policy of differentiation make sense. However, as Milian, Davies, and Zarifa (2016) have noted, forces such as local market demand, an isomorphic tendency toward perceived elitism, and status-seeking behaviour have made it challenging for colleges and universities to identify, declare, and pursue strategies of differentiation in the face of entrenched systems, structures, and cultures that mitigate against institutional change. Outcomes-based qualifications frameworks and the use of learning outcomes as a tool for credit transfer may possess compelling intrinsic logics. It is reasonable to think that learning outcomes statements have the possibility of expressing clear and understandable descriptions of what learners should be able to do as a consequence of deliberate education through programs of study leading to credentials. It is also reasonable to assume, then, that the use of learning outcomes statements can serve as a mechanism for building a common language for identifying credit transfer opportunities and generating transparency to advance mutual understanding and trust. A qualifications framework that is outcomes-based and unifies both vocational and academic credentials may have the potential to help various stakeholders compare credentials, promote greater parity of esteem between vocational and academic educational tracks, and establish levels and equivalencies, thus highlighting pathways to success. Such assumptions are the founding principles of reform efforts and although they may be rational, they are not always borne out in implementation at the institutional level.

212

Christine Arnold, Mary Wilson, Michael Potter, and Leesa Wheelahan

Shared Knowledge, Understanding, and Mutual Trust At the institutional level, the varying intrinsic logics of reforms become more operational and different considerations come into play. For example, definitions of learning outcomes are themselves contested, poorly informed, or unreasoned, and frequently incompatible around the world, as well as within national and regional systems and fields of study. Definitions of these concepts are not universal, absolute, or fixed. They are the currency of exchange in meaning-making but are only capable of indirectly conveying culturally mediated and mutable partial understandings. Discourse may be incapable of encapsulating all of the learning that exists within fields, vocations, disciplines, or professions at any given level of qualification at any given point in time. The contingent nature of this discourse can be somewhat problematic when crafting the intrinsic logic of a reform policy or tool; however, the implications of differences in discourse quickly become less abstract and more immediately troublesome when consideration moves from the abstract realm of intrinsic logics to the practical level of institutional logics. At the level of institutional logics, the precision of the language of learning outcomes and the extent of shared knowledge of their meaning is highly influential in addressing questions of credit allocation. Several contributors to this volume provide insights into the consequential differences between the nature of discourse as it resides within intrinsic and institutional logics. Winch (see chapter 7) analyzes the ways in which epistemic capacities within complementary and interdependent forms of knowledge—knowledge how, knowledge of, and acquaintance knowledge—colour the significance of the learning outcomes statements in operation within and across vocational and academic programs and can serve to complicate considerations of credit allocation. Learning outcomes may not give appropriate recognition of these different forms of knowledge. Allais (see chapter 3) also provides cautionary notes on the ways in which outcome statements housed within received policies are open to different interpretations by educational practitioners on the ground and can serve to promote confusion among educators within institutions and, in the case of South Africa, result in subversion and rejection of the intrinsic logic of reform. Similar respect for the influence of discourse and its functions across intrinsic and institutional logics is warranted in Ontario. The intrinsic logics of reform policies and tools are also susceptible to being shaped by entrenched inequities in access and mobility

Shifting Paradigms in Postsecondary Education 213

at the institutional level, and long-held assumptions about the quality of outcomes-based approaches on historical reputation. Wagenaar (see chapter 6) describes the advantages realized by the European Higher Education Area’s Bologna Process, which introduced Tuning Educational Structures in Europe in 2000. The project sought out and linked influential intrinsic political objectives to better manage the effect of institutional logics that might serve to undermine the reform effort. His analysis demonstrates that it is possible to establish significant highorder agreement among educators as to what competent graduates of specific credentials should know, do, and be. However, there is also significant evolution within fields of study and considerable difference of opinion regarding content, teaching approaches, assessment measures, and essential learning experiences for students—differences of opinion rarely influenced by scholarship. Institutional consideration should also be given to attributes of educational purpose, rigour, breadth and depth, the nature of learner effort, the sites of learning, time on task for achievement of the outcome at the desired level of capability, and the nature of quality assurance mechanisms. Generalizing from the insights provided by international contributions to this volume, shared knowledge, understanding, and mutual trust appear to be highly necessary for the implementation and sustainability of outcomes-based credit transfer reforms. Ontario possesses its own entrenched inequities and long-held assumptions regarding college and university education that are worthy of consideration. In Ontario, there is both separation and overlap in the nature and focus of credentials offered across the system. Colleges and universities both offer vocational programs spanning undergraduate and post-graduate study, with universities possessing proprietorship over academic programs. While Ontario aspires to high rates of participation in all forms of postsecondary education, colleges have been tasked historically with greater responsibility for providing access to first-generation and otherwise disadvantaged learners. This responsibility, which is arguably a core component of the colleges’ mandate, contributes to suspicions about the calibre of students in applied, vocational, and technological fields designated for credit transfer. There are also deeply entrenched differences, assumptions, and engrained power dynamics between colleges, universities, the public, government, and employers that cannot be resolved unilaterally through the intrinsic logics of reforms intended to improve student outcomes and mobility. Attention must also be paid to institutional logics—for instance, the different curricular orientation

214

Christine Arnold, Mary Wilson, Michael Potter, and Leesa Wheelahan

between colleges and universities in Ontario. Many first undergraduate degrees in Ontario’s universities are broad-based liberal arts degrees, whereas colleges must provide credentials that are oriented to the labour market. This difference in curricular orientation makes it difficult, but not impossible, to build credit transfer between the sectors. Pedagogic Devices Shaping the Structure of Curriculum and Content The last of the three constructs we are exploring to analyze outcomes-based approaches for credit transfer is conceptualizations of learning progression. The long-entrenched difference of esteem between vocational and academic learning is a complex problem that nearly every credit transfer framework considered in this volume attempts to address. In this regard, the structure of the curriculum is just as important as its content and the rules/principles by which knowledge is converted into curriculum and discourse. Learning outcomes that focus on workplace tasks, roles, and requirements can make it difficult for students to recognize and gain access to abstract theoretical knowledge needed for higher levels of education. Therefore, it is productive to cultivate a broad-based comprehension of how meaning and understanding regarding crucial disciplinary concepts are built and negotiated, which we investigate through Bernstein’s (2000) concept of the pedagogic device. Parity of Esteem Vocational and academic programs have never been neatly comparable. Although we have long passed the point where inclusion of vocational programs in colleges and universities was itself controversial, using the same qualifications framework and learning outcome statements for both vocational and academic credentials demonstrates similarities and differences. Some of the differences that complicate efforts to improve parity of esteem have long, historical roots in system design, educational and political ideologies, and cultural beliefs about the role and nature of colleges and universities. Parity of esteem conversations arise when student mobility and the structural components of curriculum and content are deliberated. Arguments are offered that we should raise the status of vocational qualifications as if it were merely an act of political will. However, the problems associated with the differences in status between the sectors arise from a complex range of issues, including the fact that universities tend to

Shifting Paradigms in Postsecondary Education 215

prepare students for higher-status occupations than do colleges, which is signalled through the more applied orientation of college credentials. Curricula for the professions place more emphasis on the theoretical knowledge that underpins practice in their fields and signal the development of more autonomous practitioners compared to those with vocational qualifications (Allais, see chapter 3; Allais and Nathan 2014). Arguably, qualifications between colleges and universities should serve three broad purposes (Gallacher, Ingram, and Reeve 2012; Wheelahan and Moodie 2017), and lifelong learning policies are predicated on achieving these purposes: 1. In the labour market, qualifications help guide entry into and progression in the workforce. 2. In education, qualifications provide students with the knowledge and skills needed to transition into a higher level of study in their field or in a related field. 3. In society, qualifications widen access to postsecondary education and support social mobility by supporting disadvantaged students to enter higher level studies and occupations, and by ensuring they gain the attitudes and values needed for full participation as citizens. All three are significant for supporting educational and occupational progression, strengthening the links between qualifications and jobs, and supporting social inclusion and mobility (Deissinger et al. 2013). However, qualifications differ in the ways they serve these purposes, largely due to how they are used in the labour market (Cedefop 2013). Not all qualifications will fulfil all three functions in the same way and to the same extent, and it is important that there are variations within a postsecondary system for students. For example, educational pathways in nursing are supported by the strong occupational pathway from practical to registered nurse in Ontario, while graduates from credentials in business or social science often undertake a second credential in a different field to help them in the labour market (Wheelahan and Moodie 2017). The design of pathways needs to reflect the way in which students use credentials to mediate their access to the labour market or further study and to participate as citizens in society. Classification of Knowledge The ways in which curricula are structured can support or undermine

216

Christine Arnold, Mary Wilson, Michael Potter, and Leesa Wheelahan

credit transfer, depending on how they enable students to access the theoretical knowledge needed to undertake pathways to higher level credentials. Bernstein’s (2000) concept of the pedagogic device is useful in analyzing the way in which the broader context helps to shape underpinning assumptions in pedagogic discourse. The pedagogic device refers to an ensemble of rules or principles in which knowledge is converted into curriculum and pedagogic discourse (Singh 2002). This helps us to see that learning outcomes are not specifications of the intended outcomes of learning as objective and neutral statements that can be understood independently of the broader context in which they are situated. And it helps us to understand why credit transfer is not a simple matter of comparing learning outcomes. The key aspect of Bernstein’s (2000) pedagogic device discussed here is the “principle of recontextualization” and the way in which theoretical knowledge is classified and framed in curriculum. The content of a curriculum always reflects the outcome of a contest over “what matters,” as does the structure of a curriculum. Identifying the principle of recontextualization that is used to structure a curriculum, in this case learning outcomes, illustrates the relations of power shaping curriculum and how different types of knowledge and skills are privileged. Bernsteinian scholars argue that a key principle shaping education is a narrow version of human capital theory. This leads to the relative neglect of theoretical knowledge in curriculum through privileging specific knowledge and skills required for work, by using this as the organizing framework for the structuring of knowledge in the curriculum (Allais, see chapter 3; Gamble 2016; Wheelahan 2012; Young 2006). The classification of knowledge refers to the ways in which knowledge is defined in different fields and how these fields are distinguished and insulated from each other. Different principles are used to classify knowledge in vocational and academic programs. The principle of classification in broad-based liberal arts degrees is an orientation to a theoretical body of knowledge, whereas in vocational programs, particularly in those that emphasize alignment of curriculum with the requirements of the job, it is utility for work. As a general rule, the “pure” academic disciplines are strongly classified, so that the boundaries between sociology and psychology, or between chemistry and biology, are maintained. Strongly classified knowledge powerfully signals the boundaries that demarcate one body of knowledge from another and the criteria used to judge knowledge claims within the field. Weakly classified knowledge focuses on problems in practice as the cohering

Shifting Paradigms in Postsecondary Education 217

principle of curriculum, which may make it difficult for students to recognize the different disciplinary or applied disciplinary systems of knowledge they are using, and how knowledge claims are evaluated in that field. Framing refers to the locus of control over the selection, pacing, sequencing, and evaluation of knowledge (Bernstein 2000). Strongly framed knowledge is knowledge in which students have little or no control over the selection of knowledge in the curriculum and its pacing, sequencing, and evaluation, whereas with weakly framed knowledge students have greater (apparent) control over their own learning process. If the selection of knowledge is strongly framed, this means that the teacher, curriculum body, or, in some cases, the professional bodies that accredit programs, have selected the knowledge “that matters” in the curriculum, and often the way in which it is paced, sequenced, and evaluated. Weak framing is apparently student-centred, so that students control the sequence and pace of learning, and possibly aspects of evaluation. However, this relies on students having an appreciation of what comes “before” in order to understand what comes “after” in entering into and acquiring bodies of knowledge. This appreciation can be particularly difficult for students who have come from disadvantaged backgrounds and have not had the same access to disciplinary knowledge and the “know-how” required to access and work within different bodies of knowledge. Bernsteinian scholars argue that strongly framed sequencing and evaluation provides students from disadvantaged backgrounds with the “recognition” rules they need to identify and work within different bodies of knowledge, while providing students with greater control over the pace of learning gives them more opportunity to successfully learn and demonstrate that they can meet the assessment requirements (Muller 2004). This discussion demonstrates that it is not enough to ensure that we get the content right for curriculum to be progressive and support social justice and social inclusion. The way that curriculum is structured also matters. Learning outcomes have the potential to result in knowledge that is weakly framed, where theoretical knowledge is “delocated” from the system of meaning in which it is embedded and rendered as contextually specific applications of knowledge. In such circumstances, students are not easily able to enter the disciplinary system of meaning and are not able to engage with the criteria and methods within disciplines for judging knowledge claims. This is a crucially important

218

Christine Arnold, Mary Wilson, Michael Potter, and Leesa Wheelahan

difference in the design of learning outcomes and programs of instruction, and one that can significantly complicate credit transfer decisions. Connecting Concepts, Knowledge Claims, and Broad Learning Outcomes Winch (see chapter 7) employs the idea of the “epistemic ascent” to show that, in accessing knowledge, students need access to propositional knowledge, and to the inferential ability to make connections between concepts and acquaintance with the objects to which the concepts refer. Linking concepts with their objects while at the same time neglecting students’ access to the inferential ability to make connections between and judgments about knowledge results in contextually specific applications of knowledge, but not access to disciplinary systems of meaning or the criteria used within the discipline to judge the veracity of knowledge claims. Access to disciplinary systems of meaning requires access to procedural knowledge within the discipline in connecting concepts and in using the methods appropriate to the discipline to make judgments about knowledge claims. Allais and Winch (see chapters 3 and 7, respectively) argue that using learning outcomes as the core framework for quality assurance, evaluation of institutions, and funding is to invest far too much in the concept and risks narrowing learning by measuring that which can be measured, rather than that which is important. Broad learning outcomes that are a component of curriculum can support learning and be one element of a rich policy framework that relies on different components to ensure the standards and quality of learning. Both argue that it strains the concept of learning outcomes too much to expect them to play a key role in the governance and quality assurance of postsecondary education systems, and to be the main organizing principle of curriculum (see also Lassnigg 2012). Implications for Ontario Learning outcomes hold potential as a tool for credit transfer, and in considering this instrument we need to address some fundamental questions about the state of postsecondary education in Ontario, the nature of qualifications, the role of outcome statements, and the shared aims and interests in improving credit transfer and accumulation. Ontario will need to learn from other jurisdictions, and to consider who, when, and how to involve contributors, and what training and devel-

Shifting Paradigms in Postsecondary Education 219

opment are needed for all. It is important to identify the essential funding, support, and incentives needed to sustain change. Allais writes that, in South Africa, “learning outcomes were perceived as a mechanism that could transcend the specificities of different contexts and integrate across this fragmentation to create a coherent, nationally integrated education system” (Allais, see chapter 3, 219). The idea of integration was seen as a way of overcoming the dichotomy of academic and everyday knowledge, the divide between general academic and vocational education, the distinction between mental and manual training and between theory and practice, divisions between natural and social sciences in the curriculum, and the distinction between pure and applied research (Motala 2001). Allais argues that this was an unrealistic expectation for one instrument, such as learning outcomes, especially in a chaotic postsecondary education system without the involvement of staff possessing educational expertise. There is ample logical, conceptual, and empirical support for the use of learning outcomes as part of broader efforts to increase the student-centredness of postsecondary education programs. Yet, as the contributions to this volume demonstrate—and as this chapter has reinforced—it is far too easy for learning outcomes to be overextended for purposes they cannot serve, in the service of agendas to which they are inappropriate, and thus become a convenient scapegoat for failures that, in hindsight, were substantially inevitable. Ontario has varying contextual conditions and different opportunities for proceeding with improvements to our credit transfer system and therefore we should consider our current strengths and weaknesses and set clear objectives and purposes for the use of learning outcomes for credit transfer. Before determining policies and practices for the systematic use of learning outcomes as a tool for credit transfer, we can learn from the international scholars’ experiences the importance of familiarizing ourselves well with our own context, assumptions, and blind spots. Broad-based engagement in reform efforts, as well as in brokering credit transfer agreements and establishing pathways, appear to be essential to the success of student mobility initiatives. Time, process, shared goals, and clear identification of benefits are components of building and maintaining the trust relationships that undergird high-quality and high-use pathways.

220

Christine Arnold, Mary Wilson, Michael Potter, and Leesa Wheelahan

References Akerlof, G. A. 1970. “The Market for Lemons: Quality Uncertainty and the Market Mechanism.” Quarterly Journal of Economics, 84 (3): 488–500. Allais, S. 2007. “Education Service Delivery: The Disastrous Case of Outcomes-Based Qualifications Frameworks.” Progress in Development Studies, 7 (1): 65–78. ———. 2014. Selling Out Education: National Qualifications Frameworks and the Neglect of Knowledge. Rotterdam, The Netherlands: Sense Publishers. Allais, S., and O. Nathan. 2014. “Skills? What Skills? Jobs? What Jobs? An Overview of Studies Examining Relationships between Education and Training and Labour Markets.” In Education, Economy, and Society, edited by E. Motala and S. Vally, 103–124. Johannesburg, South Africa: Unisa Press. Allais, S., D. Raffe, R. Strathdee, L. Wheelahan, and M. Young. 2009. Learning From the First Qualifications Frameworks. Geneva, Switzerland: International Labour Office, & Skills and Employability Department. http://www.ilo.org/employment/Whatwedo/Publications/working-papers/WCM_041902/lang--en/index.htm Allais, S., D. Raffe, and M. Young. 2009. Researching NQFs: Some Conceptual Issues. Geneva, Switzerland: International Labour Office, & Skills and Employability Department. http://www.ilo.org/wcmsp5/groups/public/---ed_emp/---ifp_skills/documents/publication/wcms_119307.pdf Australian Department of Education and Training. 2018. “Appointment of the Higher Education Standards Panel.” https://ministers.education.gov.au/birmingham/appointment-higher-education-standards-panel Bannikoff, K., R. Symonds, and A. Doolette. 2009. Building Better Connected Learning through Improved Student Pathways: Pathways Project Report. Canberra: Australian Qualifications Framework Council. http://www.voced.edu.au/content/ngv%3A51148 Barr, R. B., and J. Tagg. 1995. “From Teaching to Learning: A New Paradigm for Undergraduate Education.” Change: The Magazine of Higher Learning, 27 (6): 12–26. Bernstein, B. 2000. Pedagogy, Symbolic Control and Identity. (2nd dd.). Oxford, UK: Rowman & Littlefield Publishers. Boyer, E. L. 1990. Scholarship Reconsidered: Priorities of the Professoriate,

Shifting Paradigms in Postsecondary Education 221

Stanford, CA: The Carnegie Foundation for the Advancement of Teaching. Brown, J., G. Burke, M. Long, and P. Noonan. 2009. Funding for Tertiary Education and Training: Impacts on Student Movement between the Sectors. http://www.voced.edu.au/content/ngv%3A51146 Burke, J. 2003. “The New Accountability for Public Higher Education from Regulation to Results.” Research in University Evaluation, 3: 66–87. CALOHEE. 2018. “Objectives of the CALOHEE Feasibility Study.” https://www.calohee.eu/main-objectives/ Cedefop. 2013. The Role of Qualifications in Governing Occupations and Professions. Publications Office of the European Union. http://www. cedefop.europa.eu/EN/publications/21922.aspx Deissinger, T., J. Aff, A. Fuller, and C. Helms. 2013. Hybrid Qualifications: Structures and Problems in the Context of European VET Policy. Bern, Switzerland: Peter Lang. Dobbins, M., C. Knill, and E. M. Vögtle. 2011. “An Analytical Framework for the Cross-Country Comparison of Higher Education Governance.” Higher Education, 62 (5): 665–683. Douglas, J. A., and R. Edelstein. 2009. “The Global Competition for Talent: The Rapidly Changing Market for International Students and the Need for a Strategic Approach in the United States.” Research and occasional papers series. Berkeley, CA: Centre for Studies in Higher Education, University of California. European Commission. 2017. Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions on a Renewed EU Agenda for Higher Education. https://eur-lex.europa.eu/legal-content/EN/ TXT/?uri=CELEX:52017DC0247 Gallacher, J., R. Ingram, and F. Reeve. 2012. “Are Vocational Qualifications Vocational?” In The Future of Vocational Education and Training in a Changing World, edited by P. Matthias, 1–5). Wiesbaden, Germany: Springer VS. Gamble, J. 2016. “From Labour Market to Labour Process: Finding a Basis for Curriculum in TVET.” International Journal of Training Research, 14 (3): 215–229. Grootings, P. 2007. “Discussing National Qualifications Frameworks: Facilitating Policy Learning in practice.” In ETF Yearbook 2007, edited by S. Nielsen and M. Nikolovska, 7–40. Turin, Italy: European Training Foundation.

222

Christine Arnold, Mary Wilson, Michael Potter, and Leesa Wheelahan

Hood, C. 1991, Spring. “A Public Management for All Seasons?” Public Administration, 69: 3–19. ———. 1995. “The ‘New Public Management’ in the 1980s: Variations on a Theme.” Accounting, Organizations and Society, 20 (2–3): 93–109. Hook, R. 2004. Accountability in Ontario’s Public Colleges. Association of Colleges of Applied Arts and Technology of Ontario. Toronto. Jonker, L., and M. Hicks. 2016. The Differentiation of the Ontario University System: Where Are We Now and Where Should We Go? Toronto, ON: Higher Education Quality Council of Ontario. Kember, D. 1997. “A Reconceptualisation of the Research into University Academics’ Conceptions of Teaching.” Learning and Instruction, 7 (3): 255–275. Lang, D. W. 2004. The Political Economy of Performance Funding. Paper presented to the Taking Public Universities Seriously Conference, Toronto. Lassnigg, L. 2012. “Lost in Translation: Learning Outcomes and the Governance of Education.” Journal of Education and Work, 25 (3): 299–330. Lebeau, Y. 2012. “Who Shall Pay for the Public Good? Comparative Trends in the Funding Crisis of Public Higher Education.” Journal of Comparative and International Education, 42 (1): 137–157. Marginson, S., and M. Van der Wende. 2007. “Globalisation and Higher Education.” Paris, France: Organisation for Economic Co-operation and Development. http://www.oecd.org/education/ research/37552729.pdf Milian, R. P., S. Davies, and D. Zarifa. 2016. “Barriers to Differentiation: Applying Organizational Studies to Ontario Higher Education.” The Canadian Journal of Higher Education, 46 (1): 19–37. Ministry of Training, Colleges and Universities. 2013. Ontario’s Differentiation Policy Framework for Postsecondary Education. Toronto, ON: Queen’s Printer for Ontario. http://www.tcu.gov.on.ca/pepg/publications/PolicyFramework_PostSec.pdf Motala, E. 2001. “Policy Analysis Capacity in South Africa.” In Implementing Education Policies: The South African Experience, edited by Y. Sayed and J. Jansen, 271–292. Cape Town, South Africa: University of Cape Town Press. Muller, J. 2004. “Introduction: The Possibilities of Basil Bernstein.” In Reading Bernstein, Researching Bernstein, edited by J. Muller, B. Davies, and A. Morias, 1–14. London, UK: RoutledgeFalmer. Nelson, P. 1970. “Information and Consumer Behaviour.” Journal of

Shifting Paradigms in Postsecondary Education 223

Political Economy, 78 (2): 311–329. Parker, L., and G. Gould. 1999. “Changing Public Sector Accountability: Critiquing New Directions.” Accounting Forum, 23 (2): 109–135. Parnell, D. 1985. The Neglected Majority. Washington, DC: American Association of Community and Junior Colleges. Perkins, D. 1999. “The Many Faces of Constructivism.” Educational Leadership, 57 (3): 6–11. Power, M., and R. Laughlin. 1992. “Critical Theory and Accounting.” In Critical Management Studies, edited by N. Alveson and H. Willmott, 115–135. London, UK: Sage. Raffe, D. 2003. “‘Simplicity Itself’: The Creation of the Scottish Credit and Qualifications Framework.” Journal of Education and Work, 16 (3): 239–257. ———. 2007. “Making Haste Slowly: The Evolution of a Unified Qualifications Framework in Scotland.” European Journal of Education, 42 (4): 485–502. ———. 2011a. “National Qualifications Frameworks: What Can Be Learnt from the International Experience?” Journal of Contemporary Educational Studies/Sodobna Pedagogika, 4: 66–80. ———. 2011b. “Are ‘Communications Frameworks’ More Successful? Policy Learning from the Scottish Credit and Qualifications Framework.” Journal of Education and Work, 24 (3–4): 283–302. Raffe, D., J. Gallacher, and N. Toman. 2008. “The Scottish Credit and Qualifications Framework: Lessons for the EQF.” European Journal of Vocational Training, 42 (1): 59–69. Raffe, D., C. Howieson, and T. Tinklin. 2007. “The Impact of a Unified Curriculum and Qualifications System: The Higher Still Reform of Post-16 Education in Scotland.” British Educational Research Journal, 33 (4): 479–508. Richardson, V. 2003. “Constructivist Pedagogy.” Teachers College Record, 105 (9): 1623–1640. Rogers, C., and J. Freiberg. 1994. Freedom to Learn (3rd ed.). New York, NY: Macmillan College. Scottish Credit and Qualifications Framework (SCQF). 2018. About the Framework. http://scqf.org.uk/the-framework/about-the-framework/ Shafritz, J. M., and E. W. Russell. 2000. Introducing Public Administration (2nd ed.). New York, NY: Addison Wesley Longman. Singh, P. 2002. “Pedagogising Knowledge: Bernstein’s Theory of the Pedagogic Device.” British Journal of Sociology of Education, 23 (4): 571–582.

224

Christine Arnold, Mary Wilson, Michael Potter, and Leesa Wheelahan

Spence, A. M. 1973. “Job Market Signalling.” Quarterly Journal of Economics, 87 (3): 355–374. Tam, M. 2014. “Outcomes-Based Approach to Quality Assessment and Curriculum Improvement in Higher Education.” Quality Assurance in Education, 22 (2): 158–168. Tangney, S. 2014. “Student-Centred Learning: A Humanist Perspective.” Teaching in Higher Education, 19 (3): 266–275. Tremblay, K., D. Lalancette, and D. Roseveare. 2012. Assessment of Higher Education Learning Outcomes. Paris, France: Organisation for Economic Co-operation and Development. http://www.oecd.org/ education/skills-beyond-school/AHELOFSReportVolume1.pdf Watt, M. G. 1999. “Standards-Based Education in the United States of America: What Are Its Implications for Curriculum Reform in Australia?” https://pdfs.semanticscholar.org/9a1e/ ba830d9e7009833a6d3124795929c3887b11.pdf Western Interstate Commission for Higher Education (WICHE). 2018. “Interstate Passport.” http://interstatepassport.wiche.edu Wheelahan, L. 2009. Programs and Pathways. Adelaide: Australian Qualifications Framework Council. https://www.voced.edu.au/ content/ngv%3A51116 ———. 2011. “From Old to New: The Australian Qualifications Framework.” Journal of Education and Work, 24 (3–4): 323–342. ———. 2012. Why Knowledge Matters in Curriculum: A Social Realist Argument. London, UK: Routledge. Wheelahan, L., and G. Moodie. 2017. “Vocational Education Qualifications’ Roles in Pathways to Work in Liberal Market Economies.” Journal of Vocational Education and Training. 69 (1): 10–27. Young, L. E., and B. L. Paterson. 2007. Teaching Nursing: Developing a Student-Centered Learning Environment. Philadelphia, PA: Lippincott Williams & Wilkins. Young, M. 2003. “Comparing Approaches to the Role of Qualifications in the Promotion of Lifelong Learning.” European Journal of Education, 38 (2): 199–211. ———. 2006. “Conceptualising Vocational Knowledge: Some Theoretical Considerations.” In Knowledge, Curriculum and Qualifications for South African Further Education, edited by M. Young and J. Gamble, 104–124. Cape Town, South Africa: Human Sciences Research Council.

9

The Overarching System: Structures, Policies, and Mechanisms Impacting Learning Outcomes and Credit Transfer Mary Catharine Lennon, Paola Borin, Brian Frank, and Krista Pearson

Introduction The previous chapter established broad historical, conceptual, and theoretical considerations for interrogating the possibilities of using learning outcomes as a tool for credit transfer. The authors recognized the consequences of tensions between more neoliberal and learner-centred influences in shaping the nature and uses of learning outcomes and credit transfer in postsecondary education and a case was made for the merits of aiming for policy learning rather than policy borrowing. This chapter builds on these notions by delving deeper into systemic structures, policies, and mechanisms that both foster and impede the utility of learning outcomes for credit transfer. Through the lens of the Ontario landscape, this chapter comparatively assesses system-level structures leveraging the insights of the international scholars in this publication. It is structured into three main themes to explore the influence of policy and its regulatory and guiding frameworks. The first explores the goals and explicit or implicit purposes of policy activities, the second examines the role of qualifications frameworks and quality Learning Outcomes, Academic Credit, and Student Mobility, edited by Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon. Montréal and Kingston: McGill-Queen’s University Press, Queen’s Policy Studies Series. © 2020 The School of Policy Studies, Queen’s University at Kingston. All rights reserved.

226

Mary Catharine Lennon, Paola Borin, Brian Frank, and Krista Pearson

assurance as a means to implement policies in learning outcomes and credit transfer, and, finally, the role of organization and leadership that can impact or influence the policy landscape of learning outcomes and credit transfer is considered in the third section, which highlights the complexity of establishing a fully integrated system in Ontario. System-Level Goals and Purposes of Learning Outcomes and Credit Transfer Policies The global rise of accountability, quality assurance, and focus on access have been drivers of change in postsecondary education policies around the world in recent decades, resulting in pressure to develop and/or integrate policies centred on learning outcomes and expected to facilitate mobility and credit transfer (Lennon 2016). Within these global trends, each region or nation grapples with unique social, economic, and cultural challenges, as well as issues of system design that guide the way in which policies are developed. The goals of any credit transfer or learning outcomes policy are directly tied to the unique circumstances of the system, including political and social issues and inherited models of postsecondary education provision. Because policies are basically a “formulated response to a problem” (Inwood 2004, 207), there are a variety of elements that influence any policy’s purpose and short- and long-term goals. These goals and purposes are meant to ensure accountability for public expenditures, reduction of public expenditures, and quality of educational programs, and to provide solutions to diverse and complex social, structural, and economic problems. Each of the chapters presented by the international scholars clearly describes the policies present in their jurisdictions as well as the systems and structures that support or constrain the achievement of policy directives. In this way, credit transfer and learning outcomes policies are established to facilitate the process whereby the transfer and outcomes are identified, defined, and applied in a predictive, consistent, and jurisdictionally relevant manner. Generally, however, the following policy purposes have been identified as a common point of intersection for learning outcomes and credit transfer policies: improve student success (Banta, Jones, and Black 2009; Banta and Blaich 2010; Hattie 2009a, 2009b); foster institutional and programmatic improvement in coordination and curriculum development (Lennon 2016); support-system-level coordination, to find precisely where the similarities and differences lie (Lennon, Frank, Lenton, Mad-

The Overarching System 227

sen, Omri, and Turner 2014; Lokhoff et al. 2010); expand coordination in student progression; and facilitate credit transfer and articulation agreements (Allais 2010). Credit transfer and student mobility policies facilitate elements including access and recognition of previous postsecondary education for further education (Wheelahan, Moodie, Lennon, Brijmohan, and Lavigne 2016); support equity and promote social inclusion by providing opportunities for disadvantaged groups (Arnold, Wheelahan, Moodie, Beaulieu, and Taylor-Cline 2018; Raffe 1998; Young 2001); and increase the number of workers with higher-level tertiary qualifications (Finnie 2004; Frenette 2014; Organisation for Economic Co-operation and Development 2014; Walters 2009). In addition, similar to learning outcomes, the purpose of system-level policy coordination and collaboration is to enable and support student success. In examining the ways in which learning outcomes and credit transfer may work together cohesively, it has been argued that learning outcomes more readily “translate the aims of a course or programme of study into a set of competencies” (Roberts 2008, 4), making it easier to give credit for learning acquired in another institution, removing barriers to student mobility, and supporting lifelong learning. Transparent learning outcomes also support employability and improve the ability of students to articulate their capacities. Clearly articulated learning outcomes also aid students in recognizing the applicability and transferability of their credentials to a variety of employment options, and in possessing the language to describe their skills and relevancy (European Commission 2014). Learning outcomes that are easy to understand and specific can also be used to inform both the education and labour market about mismatches (see, for example, Allen and de Weert 2007; Lennon 2010; Miner 2010). It is necessary to acknowledge that learning outcomes can be constituted and wielded by stakeholders—consciously and unconsciously, as well as overtly and covertly—to serve particular purposes, contexts, and audiences. The following examples demonstrate the double-edged nature of learning outcomes, as they are deployed as a tool for particular purposes. Learning outcomes can be used as a mechanism to facilitate social justice, democratization, and bridge vocational and academic qualifications (Lennon et al. 2014; Lokhoff et al. 2010; Roberts 2008). They can also be articulated to function as gatekeepers defining the boundaries of disciplines and vocations, substantiating hierarchies by differentiating programs and institutions from competitors, and restricting access to forms of knowledge, skills, and values and their

228

Mary Catharine Lennon, Paola Borin, Brian Frank, and Krista Pearson

concomitant social benefits (Kennepohl 2016; Melton 1996; Powell 2011; Wheelahan 2012). Despite the various and varied goals and purposes of learning outcomes and credit transfer policies, research suggests that some goals are more successful than others at being achieved. In a study, Lennon (2016) determined that system-level policies on learning outcomes were most likely to be successful if they focused on teaching and learning, and least likely when they focused on issues of system design and credit transfer. Policy success was evaluated through a number of means, but the findings of the study determined that many learning outcomes policies in the realm of quality assurance fail for simple policy reasons: A policy can be misdirected, such that it is “formulated to achieve an unattainable goal” (Lennon 2016, 169). A policy can also be misaligned, such that the strategy employed could never achieve the stated goals (for example, wanting learning outcomes to be useful for employers and the labour market but trying to achieve that goal through writing course curricula). Finally, a policy can be misapplied, such that basic implementation issues, such as faculty buy-in, hinder success. Each of these policy issues is identifiable in the international authors’ chapters in this publication. For example, in response to periods of great social division, two international jurisdictions employed learning outcomes and/or credit transfer as a means to improve social justice, in order to assist historically underserved populations and to improve social mobility. This strategy was applied both in South Africa following the end of apartheid, and in the 1960s in the United States following the raised awareness of social division. In South Africa, as Allais (see chapter 3) states, Education policy had been central to apartheid policy; education was therefore at the centre of much of the struggle against apartheid. Apartheid institutionalized social and political fragmentation.… Learning outcomes were perceived as a mechanism that could transcend the specificities of different contexts and integrate this fragmentation to create a coherent, nationally integrated education system. (57–58)

In actuality, however, the policy was misdirected, as the gulf between these societal goals and realities is too far to bridge with one or two education policies, no matter their scope. Learning outcomes are only statements of intended learner achievement, and a credit transfer policy alone will never be able to right the wrongs of social and historical in-

The Overarching System 229

justices. Because the goals were set at such a high level, they were hard to align with activities aimed at implementing the goals. For example, although the South African Qualifications Authority (SAQA) was seen as the vehicle to transform a disparate education and training system, the majority of the postsecondary education system did not ultimately offer the qualifications and unit standards that were developed to make this possible. The qualifications were therefore not awarded. As a further example of the challenges of the South African experience, the policies were misapplied, as the unit standards and new outcomes-based qualifications were overly prescriptive, featuring highly granular outcomes and narrowly specified assessment criteria that made them quite unusable by educators across sectors. In another international example, Wagenaar describes the aspiration for democratization and the establishment of new shared identities in the European context, where learning outcomes and credit transfer were tied to ambitions of unification within the European Higher Education Area. As Wagenaar (see chapter 6) explains: In 1984 the European Council…published a call to “strengthen and promote the European identity and image both for its citizens and the rest of the world.”… As a follow-up to this call a high-level ad hoc committee, which represented the heads of state, was established on “People’s Europe,” chaired by the Italian Pietro Adonnino. This committee was given the assignment to come up with concrete initiatives within half a year “involving the citizens of Europe more determinedly in the construction of the Community.” (143)

This goal was supported by a number of aligned and targeted policy activities: In 1998 the Sorbonne Declaration represented a joint declaration on harmonization of the architecture of European higher education in several ways. The document primarily introduced a two-cycle structure comparable to the Anglo-Saxon bachelor–master model, which aimed to make European higher education more streamlined and competitive (Wagenaar, see chapter 6). This resulted in the introduction of the Bologna Process and the development of the European Credit Transfer and Accumulation System (ECTS), which shifted from a transfer system (based on workload) into a transfer and accumulation system. The European story is one of policy success. The overarching goals were supported by clear policy activities, and the funding and political support ensured that issues of implementation have been minor. It should be noted, however, that the goals, lofty as they were, sought

230

Mary Catharine Lennon, Paola Borin, Brian Frank, and Krista Pearson

regional cohesion within the university and vocational education settings independently—there has yet to be any substantial work on the integration of the two sectors at the European level, and it continues to be a challenge for nations. Australia’s clearly directed goal of bridging vocational and academic qualifications has met with various levels of success. Learning outcomes are defined, expressed, and applied differently between the vocational education and training (VET) and higher education sectors, and while the Australian Qualifications Framework has worked to promote credit transfer between institutions, it has not been successful in fully bridging the gaps created by several structural and curricular variations between the sectors. The reason for the failure can be considered to be issues of implementation, as Noonan states that the primary barriers identified were “funding and accountability, attitudes and culture, administrative issues, curriculum and qualification design, assessment and lack of resources” (Noonan, see chapter 5, 117). In Ontario, despite a number of initiatives that have taken place over the past twenty years, there have been no well-defined directives on learning outcomes or credit transfer policies in a way that clearly articulates the goals, strategies, or measurable outcomes desired from the activities. Pressures for increased accountability along with access and accessibility for underrepresented groups have been the driving force behind many Government of Ontario policies (including those involving learning outcomes and credit transfer policies) since the Rae Report (Rae 2005), and the subsequent “Reaching Higher” strategy announced in the 2005 budget (Government of Ontario 2005). However, without clear objectives, it is left to policymakers, agencies, institutions, observers, and critics to surmise the ultimate goals and make educated guesses on how those goals might be achieved. Arguably, because Ontario has not had a policy window, or crisis, to spur change, learning outcomes and credit transfer activities have been implemented in an ad hoc, incremental manner (Lennon 2014). Because of this approach, it is impossible to say with any certainty the primary focus of the Government of Ontario in this realm. For example, while accountability and access might be construed as the primary goals for learning outcomes and credit transfer, no clear link is articulated by the government. Nonetheless, the next section, which reviews qualifications frameworks and quality assurance activities, will discuss the challenges and successes of Ontario’s policy landscape and policies which help to tease out the implicit or “working” goals of the activities.

The Overarching System 231

The Role of Qualifications Frameworks and Quality Assurance Activities As discussed in chapter 1, postsecondary education in Ontario was built as a “binary” (Skolnik 2016) or “tracked” system (Tuck 2007; Young 2001), where integration of college and university sectors was not intended, and therefore distinct relationships, funding agreements, operating guidelines, and oversight relationships were developed. The most significant shift in the Ontario system came with the introduction of the Post-Secondary Education Choice and Excellence Act (PSECE) in 2000 (Government of Ontario 2000), which opened the system to private and out-of-province degree providers and also established a model for the colleges to seek consent from the Government of Ontario to offer a limited number of applied baccalaureate degrees. As part of this policy initiative, the Ontario Qualifications Framework (OQF) was developed. The first of its kind in Canada, and one of the first globally, its introduction was a critical component of the PSECE, as it outlined which providers were able to offer which credentials, as well as establishing the precise competencies associated with each credential type. The OQF was vital to the PSECE policy, as it assured that students were being provided the level of education that matched the nomenclature of the credential, so that an Ontario awarded degree would be of equivalent quality regardless of the provider.1 A qualifications framework, whether national, regional, or disciplinary in focus, is cast in accordance with the policy goals that informed its creation and its evolutions. A qualifications framework serves as an instrument for the development, classification and recognition of skills, knowledge and competencies along a continuum of agreed levels. It is a way of structuring existing and new qualifications, which are described by learning outcomes, i.e., clear statements of what the learner must know or be able to do whether learned in a classroom, on-the-job or less formally. (Tuck 2007, v)

A qualifications framework indicates the comparability of different qualifications and, in some cases, can show how one can progress from one level to another, within and across occupations or industrial sectors (Tuck 2007). 1. Also introduced at this time was the first government-appointed quality assurance agency, the Postsecondary Education Quality Assessment Board, which was mandated with providing recommendations on the quality of degrees offered by “non-traditional” providers by evaluating the programs against the Ontario Qualifications Framework.

232

Mary Catharine Lennon, Paola Borin, Brian Frank, and Krista Pearson

Most jurisdictions presented in this publication have qualifications frameworks in place, and Allais, Noonan, Wagenaar, and Winch (see chapters 3, 5, 6, and 7) acknowledge the role of qualifications frameworks as a representation of system design, as well as a help or hindrance for the negotiation and reform of system design. For the purpose of system design, the very presence of a qualifications framework indicates a level of governmental oversight and captures a representation of governmental policy regarding learning outcomes and program pathways for students. Because of their purpose and design, qualifications frameworks function as rich resources for analyzing system design and the assumptions, restrictions, and possibilities regarding learning outcomes and credit transfer. More than 150 countries are now in the process of either establishing or implementing and refining qualifications frameworks (European Centre for Development of Vocational Training 2017), making them particularly meaningful for comparative analysis. Raffe (2011) has described three different types of qualifications frameworks within a continuum (see Table 9.1). Qualifications frameworks can also represent tracked, linked, or unified systems (Tuck 2007; Young 2001). Tracked systems maintain separations between the pathways for students in vocational education and those for students in academic education. Linked systems represent the tracked nature of vocational and academic qualifications, but also indicate where commonalities exist and where crossover pathways between the two tracks are possible. Unified systems bring all system-level vocational and academic qualifications together into a single framework. As such, qualifications frameworks represent the relative organization of intended learning outcomes to be achieved at each level of qualification within a system and capture, explicitly or implicitly, the proclivity of a system toward credit transfer. Qualifications frameworks can also be characterized as existing along a continuum of tight or loose. Tight frameworks “exert a strong control over the way qualifications are designed and quality assured and insist on common rules and procedures for all qualifications. Loose frameworks are more likely to be based on general principles, to accept that there are valid differences between types of learning or education/training sectors, and to work with the grain of the education and training institution practice which preceded the introduction of the framework” (Tuck 2007, 22). A looser framework enables communication of social policy intentions, whereas a tighter framework tends to prescribe regulation of the system. The nature of the qualifications system, whether tracked, linked, or unified,

The Overarching System 233

Table 9.1 Typology of Qualifications Frameworks Type of NQF

Communications

Reforming

Transformational

Starting Point

Existing ETa System

Existing ET System

Future ET System

Purpose

To increase transparency; to provide a tool for rationalizing system, increasing coherence, and facilitating access, transfer, and progression

To achieve specific reforms, e.g., fill gaps, enhance quality, extend access transfer and progression; to provide a tool for rationalizing system and increasing coherence

To transform ET and lead development of a new system

Design

Loose, varies across sub-frameworks; outcomes used as a common reference point

Tighter, but varies across sub-frameworks; outcomes used as a common reference point

Tight, central specification imposed more uniformly; outcomes used to drive change

Leadership and Control

Voluntary “bottom-up”; ET institutions share leadership; substantial decision-making at the level of sub-framework

Compulsory “top-down”; led by central agency/ government ET institutions as key partners; control may vary across sub-frameworks

Compulsory “top-down”; led by central agency/government ET institutions among partners; centralized control

Expected Role in Change

Tool for change; requires complementary drivers to ensure the tool is used

Drives specific changes; requires complementary drivers for other impacts

Expected to drive the transformation of the system

Note: Raffe, D. (2011). Although not explicitly defined in Raffe’s work, ET is assumed to refer to education and training. a

234

Mary Catharine Lennon, Paola Borin, Brian Frank, and Krista Pearson

and tight or loose, is a good indicator of system design and the intended purposes, functions, and interactions of institutions within the system for the provision of the credentials. As the nature of a qualifications framework is primarily to denote the differences rather than the similarities of programming, qualifications frameworks are not necessarily inherently supportive of transfer policies. In fact, Noonan suggests that the existence of the Australian Qualifications Framework “largely replicated (and perhaps even entrenched) the differing approaches between the VET and higher education sectors to qualification development and design” (see chapter 5, 126). Nonetheless, the Australian Qualifications Framework includes a “Pathways Policy,” which supports institutions in ensuring students can maximize the credit they are awarded for previous education. In South Africa, the SAQA is the cornerstone of a system intended to mitigate the social inequalities of apartheid by creating “a coherent, nationally integrated education system” (Allais, see chapter 3, 58). The outcomes, level descriptors, and attendant unit standards of SAQA, while aiming to be meaningful across vocational and academic contexts, have, according to Allais (see chapter 3), led to problems of interpretation and consequent over-specification. The chapters by international scholars make it apparent that qualifications frameworks are not the same in all jurisdictions, and the way outcomes and credit transfer operate within them may limit their effectiveness as a policy mechanism. The greatest contrast exists between the National Qualifications Frameworks in Europe and the Degree Qualifications Profile (DQP) in the United States. Institutions across the European Union have all agreed to adhere to a common model for their national qualifications frameworks and therein achieve the statements of the overarching European Qualifications Framework (EQF). This ensures that transfer can occur simply, because the recognition of learning outcomes is implicit with all countries mapping to the EQF. The American DQP is designed to reflect “the diversity of American higher education while also establishing curricular and evaluation benchmarks that would contribute to curriculum improvement within and across institutions” (Bragg, see chapter 4, 103). However, the DQP stands alone as a recommendation for demarcation between three levels of credentials and is not consistently referenced across state systems. Bragg, in chapter 4, describes transfer policy in the United States as largely aimed at degree completion based on attainment and recognition (upon transfer) of the associate’s degree by four-year universities.

The Overarching System 235

The DQP does not fulfil the role of policy mechanism so much as it acts as a guide. Nevertheless, in chapter 4 she highlights initiatives that use outcomes, in some cases referenced to the DQP, to optimize transfer of general education between institutions. Analysis of the OQF reveals Ontario’s movement from a more tracked and tighter model to a more linked and loose system over time. However the design and content of the OQF itself, and its patterns of use by system actors, serves as evidence of a system in tension and undergoing evolution, complicating the prospective use of learning outcomes for credit transfer. The OQF is fairly detailed. It describes the overall design and intended purpose of qualifications, expectations regarding preparation for employment and further study, typical duration, the providers, the input measure of admission requirements, and the output measure of student learning outcomes as differentiating characteristics of vocational and academic credentials offered by the colleges and universities from short-cycle certificates to doctoral programs. The OQF does not include any provision, such as a blanket statement from the Government of Ontario, on how students or institutions should navigate transfer within or between the different credentials. This is perhaps because it was designed at the outset as a communications tool to articulate what the various credentials entailed and what institutions were able to provide them. Since its inception in 2001, the OQF has undergone two updates: once in 2007, and again in 2017, both of which made changes to reflect governmental decisions. In 2007 the degree-level expectations used by both Ontario’s public universities and the college sector’s Postsecondary Education Quality Assurance Board (PEQAB) were incorporated into the OQF, and in 2018, Indigenous Institutes were included as possible providers of diploma and degrees (Postsecondary Education Quality Assurance Board 2010). It should also be noted that postsecondary providers in Ontario interact with the framework in quite different ways, largely due to the quality assurance mechanisms in Ontario, where different bodies are responsible for (1) public colleges, (2) public universities established by a government act, and (3) Indigenous institutions, and (4) all other degree providers. The quality of certificate and diploma programs at colleges is overseen by the Ontario College Quality Assurance Service (OCQAS). While the OCQAS’s practices do consider the OQF, many programs in the colleges follow the provincially established program standards, which adhere to the OQF but never explicitly link to it or

236

Mary Catharine Lennon, Paola Borin, Brian Frank, and Krista Pearson

acknowledge it as a guiding framework. In accordance with the provincial legislative acts under which each was established, the province’s universities follow their own internal academic governance processes, which observe the guidelines of the quality assurance framework developed by the Ontario Universities’ Council on Quality Assurance for undergraduate and graduate qualifications. Although the degree-level expectations set out in the OQF are shared in the Quality Council expectations, the Quality Council does not explicitly acknowledge the OQF. In 2018, the newly established Indigenous Advanced Education and Skills Council developed a new quality assurance framework and benchmarks suitable for its members. Steered by the Indigenous Institutes Quality Assessment Board (IIQAB), the Framework was developed in collaboration with World Indigenous Nations Higher Education Consortium (WINHEC) to support the new, unique and complementary pillar of Ontario’s postsecondary education and training system. Finally, degree offerings by the colleges, as well as private providers, out-of-province providers, and some provincial universities seeking to grant degrees outside of their powers as constituted in their founding act,2 fall under the jurisdiction of PEQAB, the only agency that clearly requires adherence to the OQF as the measure against which programs are evaluated. Thus, as a result of governance differentiation within the Ontario landscape, three of the four quality assurance bodies do not explicitly acknowledge the OQF as a guide to the development and quality assurance of programming. This difference in governance and quality assurance processes complicates communication and recognition of the credentials. Impact of Organization and Leadership on Learning Outcomes and Credit Transfer at the System Level The previous sections of this chapter have shed light on how the purposes and diverse goals of postsecondary systems are addressed by policies and mechanisms that commonly embed transfer and learn2. As an example of the type of approvals by PEQAB outside of college degrees, new universities, such as University of Ontario Institute of Technology (2002) and the Algoma University (2008), were referred to PEQAB for a defined period of time as a part of the provincial oversight of new university charters.

The Overarching System 237

ing outcomes as a means of achieving these goals. Policy mechanisms shape—by constraining or enabling—the possibilities for learning outcomes to serve as a tool for credit transfer. All too often these levers do not link or complement the many elements of the system that must interact and support one another in pursuit of efficient credit transfer. The section that follows considers how organization and leadership within a system also play a key role in the ways learning outcomes and credit transfer are shaped and implemented. We explore possibilities, taken from examples in practice in diverse jurisdictions that involve how learning outcomes and credit transfer are interconnected in the policy development sphere, so as to enable policy mechanisms to function more effectively in the furtherance of system policy goals aimed at improved student mobility. From the literature and careful reading of the international scholars’ discussions of their jurisdictions, the factors understood to impact how well outcomes and transfer are integrated include issues of leadership; for example, governance decision-making related to the formulation or deployment of policies and policy mechanisms that use learning outcomes or credit transfer. These mechanisms can operate on a spectrum that runs from top-down, to centralized or consultative, to bottom-up or grassroots. Similarly, the organizational structures of a system that influence the extent of autonomy or compartmentalization between various elements of the system can determine the alignment of learning outcomes with credit transfer policies. There is no right or wrong model for leadership and organization; however, the jurisdictional examples do indicate costs and benefits associated with different models. While Ontario and other jurisdictions have benefited from a willingness and capacity for diverse actors within the system to work together to mitigate disconnects, leadership—both at the ministerial and local levels—can advance or limit the use of learning outcomes and credit transfer. Noonan (see chapter 5), in referencing the Australian study into credit transfer, Giving Credit Where Credit is Due, identifies institutional leadership as one of the primary enablers of credit transfer. He outlines the policy leadership of then-minister for employment education and training, John Dawkins, who in 1989 brought attention to the high cost (to institutions) of neglecting mechanisms to enable student progression to higher levels of education and thus career attainment. Dawkins proposed important measures, most notably those that put the onus on receiving institutions to demonstrate why credit cannot be granted, and the linking of funding to availability of credit transfer.

238

Mary Catharine Lennon, Paola Borin, Brian Frank, and Krista Pearson

Unfortunately, these recommendations were not pursued but were instead replaced with guidelines developed by senior institutional providers to steer institutional actions. Noonan explains that this softened approach unified government policy respecting credit transfer and enabled greater emphasis on a mechanism supporting national vocational education standards advocated by employer bodies. As a result, Australia’s system followed a trajectory wherein strong boundaries between vocational and academic education were established and credit transfer diminished. Europe, on the other hand, has benefitted from strong regional leadership that has fostered projects across the European Union and taken the long view of politics and policy. The European Union recognized the value of supporting mobility of faculty, students, and workers as a part of the formation of a cohesive European Union. As a result, to enable movement across nations, learning outcomes and transfer policies are ingrained in all levels of higher education, from the ministries that are responsible for demonstrating to the European Union how they are achieving their goals, to the institutions that are responsible, to the quality assurance agencies for adherence to the national qualifications frameworks. The seeds planted by leadership and nurtured in collaboration were established early and have taken nearly forty years to be actualized in such a way that all activities are built upon one another—from high-level national qualifications statements to programs. The Tuning project, and the subsequent Measuring and Comparing Achievements of Learning Outcomes in Higher Education in Europe (CALOHEE) project, for example, have further ensured that transparency of learning outcomes and student achievement are central to the European postsecondary education system and thus support and interconnect across policy mechanisms to streamline outcomes-based transfer (Wagenaar, see chapter 6). This demonstrates that strong leadership need not result in top-down approaches to decision-making. The European Union has chosen decentralized and collaborative options that have enabled it to model and work within outcomes-based frameworks and Tuning reference points. The European Union has been able to work with diverse institutional mandates to sustain the engagement of stakeholders in the development and use of widely agreed-upon learning outcomes to create and compare programs whose credit can transfer across national, sectoral, and institutional boundaries. Learning outcomes, it should be pointed out here, are not the only tools or measures integrated into the ECTS.

The Overarching System 239

The ECTS has deployed the same collaborative approach to negotiate and implement a common measure of student workload to complement learning outcomes in the facilitation of credit transfer. Thus, we see in the European Union example a system that collaboratively fosters consistency and a degree of trust to bridge autonomous national systems. This does not mean that the ECTS is without flaws. Wagenaar in chapter 6 points to organizational and implementation challenges where the pivotal yet autonomous role of faculty in the adoption and use of learning outcomes impedes buy-in and trust—a limitation that may be mitigated by training. The systems in South Africa and Australia share commonalities. The South African system resulted from the rapid adoption of policies and outcomes-based mechanisms that sought to integrate vocational and academic education under the overarching embrace of knowledge-in-action and critical cross-field outcomes. According to Allais’s critique in chapter 3, the system suffers from its top-down approach. While the its rhetoric is one of democratization and collaboration, Allais points out that in practice a large number of standards were registered on the South African qualifications framework that were highly detailed and written in such a way that “people felt alienated by the new terminology and structures” (see chapter 3, 56). This top-down approach lacked consultation and the engagement and training of those expected to implement the new processes, and resulted in learning outcomes that were difficult to use, and for which funding and training were not provided to support implementation. As such, a high proportion of unit standards were ultimately never used, rendering them useless as a means to assure equivalency of programming and as a tool to support credit transfer. This is similar to issues in the Australian system as discussed by Noonan (see chapter 5), where the VET and higher education sectors have different approaches to qualification development and design, and the VET sector’s focus on employment skills through the “training package” approach is not being recognized by the universities. The entire system has been called “fractured” by Noonan (2016) and Fowler (2017) and has been a significant challenge to credit transfer policies. In the highly decentralized United States postsecondary system, learning outcomes and credit transfer activities are driven in large part by non-governmental agencies with distinct and targeted goals, which are siloed into those focused on transfer and those focused on learning outcomes. The lack of integration, as well as a lack of high-level gov-

240

Mary Catharine Lennon, Paola Borin, Brian Frank, and Krista Pearson

ernance, has led to a number of disjointed—though very high-quality—activities. For example, the United States does not have a national qualifications framework to which all credentials must align. Instead, the DQP, which uses learning outcomes as its foundation, was developed through wide consultation supported by arm’s-length research bodies and serves largely to guide curriculum development, although the rhetoric surrounding it asserts its aim to enhance credit transfer (Lumina Foundation 2014). Despite the fact that use of the DQP is entirely voluntary, it is presumed that its application in the design of qualifications and programs will result in greater consistency, and hence improved mobility and transfer pathways. Quality assurance and public accreditation at the state level are joined in the United States. These ensure the quality of provision in both private and public institutions and also guarantee the outcomes of graduates. Accreditation of programs in the professions, such as engineering, is layered on this—as it is in most jurisdictions. The stated outcomes required for accreditation in some professions may be similar or different from those of the related degree program in how they are structured and expressed, resulting in the possible need to map the two. Collaboration and innovation in outcomes and credit transfer in the United States is driven in large part by independent, privately funded organizations, such as the Lumina, Mozilla, and Gates foundations, that have encouraged experimentation and research into outcomes-based approaches, relatively few of which are aimed at credit transfer (Bragg, see chapter 4). This fractured approach has resulted in a relatively low level of integration of learning outcomes into the credit transfer process, as is noted by Bragg. The Government of Ontario’s primary leadership in policy activities in the realm of learning outcomes and credit transfer is through the OQF as a core reference in the design of credentials by institutions. While it is arguably being used by colleges and universities, the institutions and quality assurance agencies do not directly tie activities to the OQF (with the exception of programs under the umbrella of PEQAB’s quality assurance). To be trusted, qualifications need to both reflect the realities of type and demands for progression within the system, particularly when relating applied and theoretical studies. Belief in the necessity of qualification frameworks and subsequent alignment of programs with them requires clarity about how and by whom such frameworks are developed and maintained. So, while the Government of Ontario has provided the OQF as a tool, it has stopped short of fully

The Overarching System 241

implementing it by not encouraging buy-in from the various sectors. The Government of Ontario does support a rich environment of inter-institutional collaboration, some of which has been fostered, funded, and enabled by quasi-governmental bodies such as the Higher Education Quality Council of Ontario (HEQCO) and the Ontario Council on Articulation and Transfer (ONCAT). As a result of their efforts, faculty, staff, and researchers have been encouraged to work across sectoral boundaries to develop and evaluate models for bilateral and multilateral outcomes-based transfer agreements, to analyze existing patterns of student transfer and recommend methods. Suggestions have included using learning outcomes and consideration of models best suited to meeting the needs of the system as well as students. However, few of the activities and research of these agencies have been taken on by the Government of Ontario and implemented in a wide-scale manner to support system-wide change. Implications for Ontario This chapter has provided considerable detail about Ontario’s postsecondary education system structure, organization, and policies, in an effort to highlight the influence of structural constraints as critical to understanding “success” or “failure” of learning outcomes and credit transfer policies. Recognizing this, Ontario should consider policy activities that take into consideration the existing landscape and governance structures, as well as the quality and qualifications framework mechanisms, and work within it to achieve policy success. There is an opportunity for clearer goals that align with policy activities, provided there is adequate support and funding for implementation. The nature of the shifting, yet still a somewhat binary or tracked system of the college and university sectors, is a critical variable when considering the constraints and realities for transfer credit and learning outcomes in Ontario. Moreover, collective commitment and trust in existing quality mechanisms built upon respect for the distinct needs of system sectors, as well as common acceptance and use of the OQF, are actions that hold promise for improvement of outcomes-based approaches and credit transfer within and beyond provincial jurisdiction. Establish Clear and Reasonable Goals, Aligned Activities, and Support System-Wide Implementation While it may seem obvious, establishing clear long-term goals and rea-

242

Mary Catharine Lennon, Paola Borin, Brian Frank, and Krista Pearson

sonable short-term targets for system-wide use of learning outcomes in programming and transfer credit strategy is necessary for change. This would involve setting and aligning strategies and activities to meet those targets and buttressing implementation with proper support and funding. This is the crux (and most confounding aspect) of policymaking. More than ten years ago, the Government of Ontario recognized that they did not have enough information to develop clear strategies for understanding or improving educational quality or issues of credit transfer, and so they developed HEQCO and subsequently ONCAT to provide evidence-based research to inform their decisions. Despite good work from both agencies, to date the Government of Ontario has not developed any system-wide strategy to further embed learning outcomes into programming or to move past the traditional bilateral or multilateral transfer activities. An example of a provincial initiative to align high-level expectations in Ontario is the Ontario Tuning project, initiated by the HEQCO, which developed disciplinary sector-level learning outcomes for diploma and degree programs (Lennon 2014). Though the primary purpose was to support identifying and measuring learning outcomes, the work required aligning and comparing relationships between expectations in diploma and degree programs and was intended to support discussions about credit transfer and student mobility. The absence of more of these common understandings leaves institutions without sufficient basis to fully understand the compatibility and variance between programs and credentials. Work Within the Structural Parameters Jurisdictions with binary or tracked systems—where vocational education and academic education are run independently—require different transfer policy approaches than integrated systems. As has been noted multiple times, the challenges of having learning outcomes developed within the different sectors for different purposes or through different means does not support a common understanding, use of language, or respect for content. In Ontario, the differences between sectors are further compounded by how learning credits are earned, how many hours constitute a credit, how those credits are translated via transcripts, and how the transcripts are evaluated for determining credit transfer. The issue of credit transfer fundamentally lies in documenting and interpreting the nature of the learning that has taken place. It involves

The Overarching System 243

identifying, recognizing, and supporting readiness to perform at the required levels. Operationalizing this at the system level is not without challenges. It may be useful to take the perspective that fundamental differences in the missions of institution types are not necessarily a problem for credit transfer, but rather that the problem lies in the lack of translation of education. Clarifying this could make a considerable contribution for moving policy discussion forward. Trust in the Foundations of the System and Quality Assurance Mechanisms Trust among the many components and actors of a system fundamentally affects the capacity for outcomes-based transfer between qualifications and part qualifications. For learning outcomes to be a significant tool for credit transfer, all system actors need to be confident in the underlying frameworks and the mechanisms that support and ensure the achievement of outcomes. Similarly, the issue of who creates and controls policies in learning outcomes and credit transfer has a significant impact, which can shape the nature and the strength of a policy. The relationship between primary stakeholders also influences policies. For example, a system with high levels of trust between sectors will have distinctly different policies than one with strained relationships, as seen in Australia and South Africa. While these structural features are largely outside of the control of any one policy, they have a significant influence on problem identification and policy development. It is within these broader constructs that actual learning outcomes and transfer policies are developed based on a number of choices: the goal, focus, relationship to other policies, etc. While these choices are somewhat discretionary, they are strongly guided by context. Quality assurance processes can help to build trust. Whether or not an institution or program has a full understanding of the curriculum of a program, there should be trust that the program and institution are operating at a certain level and that the level they are achieving is of sufficient quality to pass quality assurance evaluations. To not trust the programming suggests a mistrust of the quality assurance mechanisms in place. The strong presence of the European Quality Assurance Agency highlights the importance of its role in the European context, where national quality assurance agencies are held to the highest standards as to how they assure the quality of their institutions and programs, so that the quality of the programming from another country cannot be

244

Mary Catharine Lennon, Paola Borin, Brian Frank, and Krista Pearson

questioned. The fact that Ontario has four different quality assurance agencies is not necessarily the root of the problem. The issue is that there is a lack of consistency in acknowledging the connection among the agencies and, subsequently, the application of the OQF by these agencies. References Allais, S. 2010. The Implementation and Impact of Qualifications Frameworks: Report of a Study in 16 Countries. Geneva, Switzerland: International Labour Office. Allen, J., and E. de Weert. 2007. “What Do Educational Mismatches Tell Us about Skill Mismatches? A Cross-Country Analysis.” European Journal of Education, 42 (1): 59–73. Arnold, C. H., L. Wheelahan, G. Moodie, J. Beaulieu, and J.-C. Taylor-Cline. 2018. “Mapping the Typology of Transition Systems in a Liberal Market Economy: The Case of Canada.” Journal of Education and Work, 31 (2): 125–143. Banta, T. W., and C. Blaich. 2010. “Closing the Assessment Loop.” Change: The Magazine of Higher Learning, 43 (1): 22–27. Banta, T. W., E. A. Jones, and K. E. Black. 2009. Designing Effective Assessment: Principles and Profiles of Good Practice. San Francisco, CA: JosseyBass Inc. European Centre for Development of Vocational Training (Cedefop). 2017. Global Inventory of Regional and National Qualifications Frameworks. Volume I: Thematic Chapters. UNESCO Institute for Lifelong Learning. http://www.cedefop.europa.eu/en/publications-and-resources/publications/2221 European Commission. 2014. About Europass. https://europass.cedefop.europa.eu/en/about.iehtml Finnie, R. 2004. “The School-to-Work Transition of Canadian Post-Secondary Graduates: A Dynamic Analysis.” Journal of Higher Education Policy and Management 26 (1): 35–58. Fowler, C. 2017. The Boundaries and Connections between the VET and Higher Education Sectors: “Confused, Contested and Collaborative.” Adelaide, Australia: National Centre for Vocational Education Research. https://www.ncver.edu.au/__data/assets/pdf_file/0027/158706/ Boundaries_-and-_connections_between_VET_-and-_higher_ed_ sectors.pdf Frenette, M. 2014. An Investment of a Lifetime? The Long-Term Labour Mar-

The Overarching System 245

ket Premiums Associated with a Postsecondary Education. Ottawa, ON: Statistics Canada. Government of Ontario. 2000, c. 36. Post-Secondary Education Choice and Excellence Act. https://www.ontario.ca/laws/statute/00p36 ———. (2005). Reaching Higher: The McGuinty Government Plan for Post-Secondary Education. http://www.fin.gov.on.ca/en/budget/ontariobudgets/2005/pdf/bke1.Pdf Hattie, J. A. C. 2009a. “The Black Box of Tertiary Assessment: An Impending Revolution.” In Tertiary Assessment and Higher Education Student Outcomes: Policy, Practice and Research, edited by L. H. Meyer, M. Rees, R. B. Fletcher, P. M. Johnston, and H. Anderson, 259–275. Wellington, NZ: Ako Aotearoa. ———. 2009b. Visible Learning: A Synthesis of Over 800 Meta-Analyses Relating to Achievement. London, UK: Routledge. Inwood, G. J. 2004. Understanding Canadian Public Administration: An Introduction to Theory and Practice (3rd ed.). Toronto, ON: Pearson Canada. Kennepohl, D. K. 2016. “Incorporating Learning Outcomes in Transfer Credit: The Way Forward for Campus Alberta?” Canadian Journal of Higher Education, 46 (2): 148–164. Lennon, M. C. 2010. A Fine Balance: Supporting Skills and Competency Development. Toronto, ON: Higher Education Quality Council of Ontario. ———. 2014. “Incremental Steps towards a Competency-Based Education System in Ontario.” Tuning Journal for Higher Education, 2 (1): 59–89. ———. 2016. In Search of Quality: Evaluating the Impact of Learning Outcomes Policies in Higher Education Regulation. (Doctoral dissertation). University of Toronto/Ontario Institute for Studies in Education (OISE). Lennon, M. C., B. Frank, R. Lenton, K. Madsen, A. Omri, and R. Turner. 2014. Tuning: Identifying and Measuring Sector-Based Learning Outcomes in Postsecondary Education. Toronto, ON: Higher Education Quality Council of Ontario. Lokhoff, J., B. Wegewijs, K. Durkin, R. Wagenaar, J. Gonzalez, A. K. Isaacs, … M. Gobbi. 2010. A Guide to Formulating Degree Programme Profiles Including Programme Competences and Programme Learning Outcomes. Bilbao, Spain: Universidad de Deusto. Lumina Foundation. 2014. Degree Qualifications Profile. https://www. luminafoundation.org/files/resources/dqp.pdf

246

Mary Catharine Lennon, Paola Borin, Brian Frank, and Krista Pearson

Melton, R. 1996. “Learning Outcomes for Higher Education: Some Key Issues.” British Journal of Educational Studies, 44 (4): 409–425. Miner, R. 2010. People without Jobs, Jobs without People: Canada’s Labour Market Future. Toronto, ON: Miner and Miner Consulting. http:// www.minerandminer.ca/data/CO_PWJ_Brochure_CAN_2010_ v4.pdf Noonan, P. 2016. Response to Commonwealth Government’s Discussion Paper: Redesigning VET FEE-HELP. Melbourne, Australia: Mitchell Institute. http://www.mitchellinstitute.org.au/papers/response-to-commonwealth-governments-discussion-paper-redesigning-vet-fee-help/ Organisation for Economic Co-operation and Development. 2014. How Canada Compares. Paris, France: OECD. http://skills.oecd.org/informationbycountry/canada.html Postsecondary Education Quality Assurance Board. 2010. Report on Recommendations of the PEQAB Self-Study. http://www.peqab.ca/Publications/recommendations.pdf Powell, J. P. 2011. “Outcomes Assessment: Conceptual and Other Problems.” AAUP Journal of Academic Freedom, 22 (2): 1–25. Rae, B. 2005. Ontario: A Leader in Learning, Report & Recommendations. Toronto, ON: Ministry of Training, Colleges and Universities. Raffe, D. 1998. Conclusion: “Where Are Pathways Going? Conceptual and Methodological Lessons from the Pathways Study in Pathways in Vocational and Technical Education and Training.” In Pathways and Participation in Vocational and Technical Education and Training, Organisation for Economic Co-operation and Development, 375–394. Paris, France: OECD. ———. 2011. “Are ‘Communications Frameworks’ More Successful? Policy Learning from the Scottish Credit and Qualifications Framework.” Journal of Education and Work, 24 (3–4): 283–302. Roberts, G. 2008, November. Learning Outcomes Based Higher Education and Employability. Paper presented at Employability: The Employers’ Perspective and its Implications conference, Luxembourg. Skolnik, M. L. 2016. Postsecondary System Design and Governance: A Background Paper Prepared for the Ministry of Advanced Education Campus 20/20 Project Regarding the Future of British Columbia’s Postsecondary Education System. Victoria, BC: Ministry of Advanced Education. Tuck, R. 2007. An Introductory Guide to NQFs: Conceptual and Practical Issues for Policy-Makers. Geneva, Switzerland: International Labour Office. https://www.ilo.org/wcmsp5/groups/public/@ed_emp/@

The Overarching System 247

ifp_skills/documents/instructionalmaterial/wcms_103623.pdf Walters, D. 2009. “A Comparison of the Labour Market Outcomes of Postsecondary Graduates of Various Levels and Fields over a Four-Cohort Period.” Canadian Journal of Sociology, 29 (1): 1–27. Wheelahan, L., G. Moodie, M. C. Lennon, A. Brijmohan, and E. Lavigne. 2016. Student Mobility in Ontario: A Framework and Decision Making Tool for Building Better Pathways. Toronto, ON: Centre for the Study of Canadian and International Higher Education, Ontario Institute for Studies in Education, University of Toronto. Young, M. 2001. The Role of National Qualifications Frameworks in Promoting Lifelong Learning (Discussion paper). Paris, France: OECD. http://www.oecd.org/edu/innovation-education/34376318.pdf

10

Adoption, Adaptation, and Implementation: The Complexities of Using Learning Outcomes to Advance Credit Transfer in Institutional Contexts Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers

Introduction The previous two chapters provide conceptual and systems-oriented lenses through which to examine contributions and challenges in the use of learning outcomes to advance credit transfer. In chapter 8, the social, economic, and pedagogical impetus for outcomes-based and standards-based educational reform are explored, along with the tradeoffs between policy aims and the practical realities of postsecondary education delivery. Chapter 9 moves these notions into an examination of how systems construct goals, policies, and mechanisms such as qualifications frameworks that seek to balance accountability and quality with access, equity, and lifelong learning. This chapter explores these challenges through an investigation of how educational providers adapt learning outcomes that are designed to serve both system goals and institutional requirements (Adam 2004; Hubball and Gold 2007). A proposed typology of outcomes distinguishes between how outcomes are formulated and what they are meant to Learning Outcomes, Academic Credit, and Student Mobility, edited by Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon. Montréal and Kingston: McGill-Queen’s University Press, Queen’s Policy Studies Series. © 2020 The School of Policy Studies, Queen’s University at Kingston. All rights reserved.

250

Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers

do. It demonstrates how outcomes, at the institutional level, are primarily connected to the learning process (see chapters 3, 6, and 7). It further functions as a basis for examination of how diverse institutional players and functions leverage system-wide outcomes frameworks and standards to formulate outcomes relevant to teaching and learning, as well as administrative tasks including decisions surrounding credit transfer. The range of purposes to which outcomes are put within institutions suggests that their effectiveness depends upon alignment, common language, and faculty/staff buy-in (Allais, Raffe, and Young 2009; Fitzgibbon 2014; Hubball and Gold 2007). The lens of the educational provider pursued in this chapter also probes related factors, such as the linkages between outcomes, quality assurance, faculty development, and divisions between educational sectors that impact both trust and the capacity of outcomes to function effectively as a tool for credit transfer. Hence, outcomes are framed here as among a range of “currencies” or measures of exchange such as time, credit weight, content, and curriculum that may complement outcomes in determining credit transfer. Finally, this chapter looks at significant initiatives in Ontario for success factors that may impact the prevalence and efficiency of credit transfer between Ontario postsecondary institutions. Outcomes in Context In Ontario, learning outcomes are embedded in the structure of postsecondary education such that their use cascades from system-wide qualifications frameworks through to classroom learning. Outcomes used in this array of contexts are responding to both neoliberal and student-centred pressures (see chapter 8) and serve multiple purposes (see chapter 9), leading them to develop different characteristics and capacities. Outcomes, while functioning as institution-based practices as well as instruments of system goals and policy such as seamless student mobility, are challenged to align with and deliver on needs that are often at variance with one another. Havnes and Prøitz (2016) predicted that outcomes that are more in line with system needs would tend to be pre-defined and closed-ended, while those that connect more closely to the learner would require greater latitude of description and even open-endedness, features that make learning outcomes, especially at the institutional level, problematic as a tool for credit transfer. Table 10.1 acts as a heuristic to differentiate amongst types of

Institutional Outcomes

Values

Accreditation Requirements

Tuning Reference Points

Program Standards

Standards

Qualification Frameworks Degree Level Expectation

Frameworks

Functions Types

Institution

System

System

Required

Spans Sectors

Typology of Learning Outcomes

Table 10.1

Minimum or Average Achievement

Individual Level or Stakeholder Student Progression in Input Assessment Complexity

continued

Broad Cate- Discipline- Learning gories of Specific Process and Learning Learning Context

Adoption, Adaptation, and Implementation 251

+

Institution

Institution

Required

Spans Sectors

Minimum or Average Achievement

*

Individual Level or Stakeholder Student Progression in Input Assessment Complexity

Broad Cate- Discipline- Learning gories of Specific Process and Learning Learning Context

* Ontario Colleges require Program Advisory Committees to ensure consultation with local employers. + Where vocational and professional regulation and/or accreditation applies, courses may be required to comply with these.

Unit or Course Outcomes

Effects

Program Outcomes

Profiles

Functions Types

Typology of Learning Outcomes, continued

Table 10.1

252 Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers

Adoption, Adaptation, and Implementation 253

learning outcomes at both the system and institutional levels (vertical axis), based upon their characteristics of formulation and function (horizontal axis). It enables, for instance, comparison of learning outcomes within a qualifications framework to those within a course by virtue of whether each is a requirement, and whether each is used in vocational as well as academic sectors. This comparison reveals that qualifications frameworks typically require compliance, span sectors, define average achievement, and are based on broad categories rather than discipline-specific knowledge. This contrasts strongly to course or unit outcomes, which are typically designed by faculty, demonstrated as achieved through individual student assessment, and embedded in the classroom learning processes. The types of learning outcomes laid out here are distinct from another. Qualifications frameworks describe differences between credentials, often using descriptors that signify levels in a progression and focus on broad categories of knowledge, skill, and ability. Degree-level expectations in Ontario connect to qualifications frameworks and do much the same in describing different degree types. Standards form the next outcome type in the cascade. These include program standards written as outcomes that express a minimum of learning to be achieved in programs of similar kinds across the system. In Ontario, program standards extend to standards for (overarching) employability skills and general education. The Tuning system can be seen as another form of standard. In Europe, Tuning reference points are written by academics from across the system to define outcomes for cognate disciplines and broad occupational fields. Professional accreditation standards define requirements of regulated occupations. At the institution level, there are three types of outcomes. The first are outcomes that represent values held by individual educational providers and are defined as overarching educational mandates, priorities, and emphases that all programs within an institution seek to fulfil. Next are profiles that define the general and specific knowledge, skills, and abilities that can be reliably expected of graduates from a program. Finally, the effects of learning are defined in course outcomes. These indicate what a learner is expected to know, be, and do at the end of an intentional unit of study. Course outcomes engage with elements, activities, and thresholds in the learning process and their demonstration through assessment. It is evident that institutional outcomes, while referenced to systems,

254

Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers

are written within academia rather than at the system level and operate closer to the sites of learning where curriculum, sequence of learning, course design, and individual learner assessment are hallmarks (Adam 2004; Carter, Coyle, and Leslie 2011; Havnes and Prøitz 2016). An emphasis on the learning process is critical to the formulation of outcomes at the institutional level, a point that is argued by Allais (see chapter 3) and Winch (see chapter 7), as well as Arnold, Potter, Wheelahan and Wilson (see chapter 8), who challenge how outcomes connect to learning progression, authentic classroom teaching practices, and sound pedagogy. Table 10.1 also points to differences in how categories of knowledge (broad, transferable, transversal, and subject-specific) may play out differently in system and institutional outcomes. While systems produce outcomes that describe the expected knowledge, skills, and abilities of graduates in terms of frameworks and standards, institutions must actually develop and support unique approaches to the ways that system-level outcomes are employed and determine the most appropriate mix, emphasis, and translation of these in local programs, curricula, and courses. This process of interpretation leads to more variability in learning and sequencing that is less easily compared for credit transfer. Allais (see chapter 3; Allais 2012) reminds us that we cannot assume that an outcomes-based standard represents “some ‘sameness’ across a wide range of knowledge areas and learning experiences” (Allais 2012, 10). She also argues that learning outcomes that are meaningful in the classroom must be derived “within the logic and emphasis specific to the content of the knowledge field or practice” (Allais, see chapter 3, 71). Winch (see chapter 7) points to the dependency of outcomes on content and context, a feature he says undermines the claim that outcomes can stand alone without reference to inputs such as curriculum, subject matter, and time on task. Moreover, the insights from Allais, Noonan, and Winch (see chapters 3, 5, and 7), as well as those presented in chapter 8 regarding strongly and weakly classified and framed knowledge, problematize learning outcomes and the way they may separate theoretical and applied knowledge. In Europe, the Tuning process maps the essential core of disciplines and occupational fields by defining competences and outcomes, as well as recommended approaches to teaching, learning, and assessment that are built upon common understanding among faculty and subject area experts (see chapter 6; European Commission 2010). This provides a system-wide resource for institutional program and course design that

Adoption, Adaptation, and Implementation 255

promotes greater alignment of outcomes between systems and institutions, which enables increased comparability between programs (Lokhoff et al. 2010; Lennon et al. 2014). Learning Outcomes Across Spheres of Work in Institutions Learning outcomes—generated at both the system and institution levels—are used throughout postsecondary institutions to inform functions such as academic regulation, records, quality assurance, program design, curriculum development, faculty development, course design, delivery, assessment, and student learning, as well as in the determination of credit transfer and the development of credit transfer agreements (Hubball and Gold 2007). Table 10.2 matches outcome types to institutional spheres of work. Learning outcomes—whose types and interdependencies are examined above—are expected to be effective in a wide array of functions, ranging from the creation of broad institutional mandates, to connecting programs with qualifications, to the design, alignment, and delivery of programs, courses, and assessments. These outcomes are also expected to guide the learning choices of students, and ultimately to inform credit transfer (FitzGibbon 2014). This suggests that their effectiveness is to some degree connected to common language and broad recognition of distinct categories of learning (McFarlane and Brumwell 2016). Allais (2012) questions the transparency of learning outcomes, asserting that agreement on descriptors is problematic due to the diversity of terminologies. She also notes that system-wide outcomes and the institutional-level outcomes that reference them are weakly embedded in curriculum, and that there is an inevitable need for clarification and precision. Moreover, she argues (see chapter 3) that learning outcomes created outside of the field of knowledge and learning processes are poor benchmarks and guides for classroom learning. Her views, as well as those of Winch (see chapter 7), suggest that it is unlikely that outcomes alone can provide the means for effective comparisons of different programs. It is also worth noting that Allais (see chapter 3) and Wagenaar (see chapter 6), as well as others (Hussey and Smith 2002), point to limitations in faculty buy-in and regard for the value of learning outcomes as factors that impact the effectiveness of outcomes as a tool for credit transfer. Thus, we see that while outcomes may act as a useful measure for

Qualification frameworks and degree level expectations define level and broad categories of knowledge, skill and ability required for qualifications.

Frameworks

Align credentials and academic regulations with qualification levels. Use these as criteria for quality assurance.

Interpret qualifications and levels into credentials. Apply descriptors to graduate profiles and translate these to credentials and subject-specific programs.

Administration Programs

Learning Outcomes: Who Uses Them and How

Table 10.2

Interpret frameworks to inform curriculum development, course design, teaching, and assessments.

Faculty Clarify scope and level of achievement required for specific qualifications. Guide choices of what qualification to pursue.

Students

continued

Inform admission and/or block transfer of credits from one qualification to another, typically at a successive level.

Credit Transfer

256 Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers

Institution outcomes define overarching educational mandate, priorities, and emphases.

Values

Program standards,* Tuning reference points, and accreditation or professional requirements define benchmarks for qualifications.

Standards

Guide policies and priorities in areas of emphasis and/ or differentiation. Inform the development of internal/ external local partnerships.

Inform quality assurance process and compliance with external requirements. Use as criteria for quality assurance.

Inform and guide deployment of resources, design or modification of programs, and methods of delivery.

Interpret standards in the design and development of programs and curriculum.

Administration Programs

Inform and guide choices and commitments to pursue innovations in curriculum and pedagogy.

Design curriculum, courses, and assessments to standards, requirements, and/or reference points.

Faculty

Learning Outcomes: Who Uses Them and How, continued

Table 10.2

Inform choices amongst institutions Assess nature and scope of learning opportunities and experiences.

Understand the shape and boundaries of the program and field. Assess employment relevance of programs and courses.

Students

continued

Contribute broad basis for comparison of programs by administrators, program leaders, and faculty.

Credit Transfer

Adoption, Adaptation, and Implementation 257

Program outcomes describe emphases, curriculum, and learning enhancements in specialized and broad areas of knowledge, skill, and ability.

Profiles

Provide measures of academic quality.

Provide basis of curriculum design. Define program sequences, requirements, prerequisites, concentrations, and majors.

Administration Programs

Inform engagement in curriculum design. Guide development of new and modified courses. Facilitate alignment with all levels of LOs.

Faculty

Learning Outcomes: Who Uses Them and How, continued

Table 10.2

Clarify program structure and requirements. Guide choices in the selection of programs and concentrations.

Students

continued

Provide information (alongside inputs) to inform transfer agreements; guaranteed admission and the granting of specific transfer credit upon completion of whole credential by administrators, program leaders, and faculty.

Credit Transfer

258 Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers

Provide measures of academic quality. Establish basis for assessment, allocation of marks, and accumulation of credits.

Provide basis for alignment of courses with curriculum and program aims. Enable planning and support for course delivery.

Provide basis for alignment of pedagogy, content, activities, assignments, and assessment. Share in descriptions and syllabus.

Faculty Clarify understanding of course learning expectations and requirements. Guide course selection.

Students

Provide information (alongside inputs) to inform transfer agreements; determine course equivalency and/or specific transfer credit upon completion of whole or part credential by administrators, program leaders and faculty.

Credit Transfer

*In Ontario, program standards extend to standards for employability (overarching), skills, and general education.

Course outcomes describe elements, activities, thresholds in the learning process and their demonstration through assessments

Effects

Administration Programs

Learning Outcomes: Who Uses Them and How, continued

Table 10.2

Adoption, Adaptation, and Implementation 259

260

Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers

achieved learning, for the purpose of determining credit transfer they can and perhaps should be complemented by input measures such as time, curriculum, content, and assessment. Taken together, such measures or “currencies” add significant and multiple dimensions to the enterprise of calculating credit transfer. Alternatives or complements to learning outcomes as a tool for credit transfer are explored more fully later in this chapter. Additional Factors Affecting Institutions, Learning Outcomes, and Credit Transfer There is much to be considered beyond the types and uses of learning outcomes. The following considerations link outcomes and credit transfer to the purposes of postsecondary education, institutional cultures, and processes of implementation. Mechanisms and Tools that Enable Institutions to Use Learning Outcomes Multiple mechanisms exist to represent achieved outcomes and learning progression within institutions. In various ways, they collect, structure, and support the analysis of data related to curriculum and pedagogical practices. Some curriculum development tools may also support credit transfer. These fit into two groups: those that support the mapping and visualization of learning progression, and those that promote consistency of outcomes language. Curriculum mapping that uses learning outcomes is an increasingly common practice in Ontario and elsewhere, and is acknowledged by both Bragg and Noonan (see chapters 4 and 5) as a means to find possible pathways for credit transfer through the identification of equivalencies between programs or the courses within them. Curriculum maps that track and compare learning outcomes chart course sequences and contributions to the progression of student learning throughout a program of study (Kopera-Frye, Mahaffy, and Svare 2008). One example of a mapping tool is the Course Outline Mapping and Management System used by eight Ontario colleges. It is a web-based application that allows faculty and administrators to create, edit, approve, and store course outlines. It enables users to map course curriculum (e.g., course learning requirements and embedded knowledge and skills) to program outcomes and provides reports in the form of curriculum maps (Algonquin College 2018). Alternative tools populate curriculum maps by mining course syllabi across programs to facilitate the

Adoption, Adaptation, and Implementation 261

collection of data needed to describe how learning progresses within a program. An example of this is the uOSyllabus tool at the University of Ottawa (Milot and Gauthier 2016). Other promising mapping and visualization tools have been developed and piloted in Ontario. These include a natural language processing tool used to automate the identification of similarities between course outlines by tagging and extracting learning outcomes, thus reducing the time and human resources required to determine transfer credit equivalency (Lakehead University 2016); a web-based tool that is being developed to visualize outcomes in curriculum and the content of programs, enabling users to identify learning gaps important in the consideration of bridge programs and equivalency (Zakani et al. 2016); and a web-based tool that maps specialized and highly diverse college and university video game programs against a set of competencies and outcomes for the entire field, using a Tuning process that engages educators and practitioners in creating reference points for comparison of programs and to recommend bridging options (Bridge et al. 2016). Commonality of language plays an important part in the functionality of all of these tools and their underlying taxonomies, thus providing a degree of consistency in how outcomes are written and applied. Noonan (see chapter 5) discusses how learning taxonomies support shared understanding of outcome terminology across broad and field-specific knowledge, skills, and abilities. Taxonomies such as Bloom’s (Anderson et al. 2001) and SOLO (Biggs and Collis 2014) are used in these tools, as well as in curriculum and course development contexts, to distinguish multiple dimensions of learning and the exigencies of cognate disciplines (Allais 2012). Relevance to Work Preparation for “entry into and progression in the workforce” is acknowledged in chapter 8 as among the three main purposes of postsecondary education. Comprehensive outcomes-based approaches facilitate connections between learning and work (Allais, Raffe, and Young 2009; European Centre for Development of Vocational Training 2017; Lennon 2014) and, by extension, credit transfer. They do so through the development of programs and curriculum that effectively leverage the following: • regulation of vocational and professional programs; • mechanisms such as Tuning and consultation for understanding the needs of employers in unregulated fields; and

262

Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers

• recognition of non-formal, experiential learning and informal, prior learning that has taken place during employment. We have seen in previous chapters how the links between postsecondary education and the requirements and expectations of employers have resulted in vocational as well as professional programs defined in whole or in part by accreditation or employer-defined requirements. Noonan (see chapter 5), Wagenaar (see chapter 6), and others argue that enumeration of employer requirements, especially in broad occupational sectors such as nursing and engineering—whether through accreditation standards or Tuning—and their alignment with learning outcomes can be facilitating factors in credit transfer (Wheelahan 2000; Zakani et al. 2016). The Tuning process, involving collaboration among educators and employers to define the essential core of a discipline or field (European Commission 2010), is used to help build programs that can be more easily compared and mapped for credit transfer (Lennon et al. 2014). However, Allais, Noonan, and Winch (see chapters 4, 5, and 7) all caution that where outcomes narrowly reflect work requirements, they may limit student learning and the autonomy of institutions to set programs that benefit learners in broader ways. In Ontario, unregulated vocational or academic fields do not require formal employer codification of work requirements to guide program development. However, in the college sector, informal employer input into the outcomes of programs is the norm. At the system level, employers are consulted on college program standards (Ministry of Training, Colleges and Universities 2003). At the institutional level, college program advisory committees engage with local employers on the fit between curriculum and employer needs (Ministry of Training, Colleges and Universities 2003), and colleges also reference their outcomes to annual key performance indicator surveys of employers (Ministry of Advanced Education and Skills Development 2017). Experiential and work integrated learning (WIL) is a frequent component of vocational and professional programs, where such learning is manifest in internships, co-op placements, field or service learning, apprenticeships, or other forms of non-formal learning that are oriented in diverse ways to curriculum (UNESCO Institute for Lifelong Learning 2002). WIL “is as much about learning as it is about work” (Sattler 2011, 94). In this context, learning outcomes are negotiated between the institution, the student, and the employer, but in most cases—with the exception of some co-op placements—they may not actually earn credit

Adoption, Adaptation, and Implementation 263

that will be included in their academic transcripts (Sattler 2011). WIL activities promote learning in line with the outcomes valued by employers, such as independence, personal responsibility, work ethic, professionalism, communication skills, interpersonal and teamwork skills, problem solving, creative thinking, analytical capabilities, and technological literacy (Aon Hewitt 2016; Turcotte, Nichols, and Philipps 2016), which poses challenges for assessment. Learning outcomes related to such skills and abilities must be clearly identified in curriculum and courses and assessed in some manner in order to earn credit and facilitate credit transfer. Similar challenges are found in recognizing prior learning in the workplace. Here too, the existence of clear program and course outcomes is critical to how work experience is evaluated for the awarding of credit (Kennepohl 2016; Weingarten 2013). Recognition of prior learning, as well as of older formal learning, is determined in a somewhat inconsistent manner by institutions, a practice that narrows the scope of credit transfer. While colleges typically assess prior learning for credit, universities rarely do. Neither is consistent in their award of credit for courses completed in the more distant past. Leveraging Quality Assurance and Faculty Development Wagenaar (see chapter 6) suggests that faculty are central players in producing meaningful outcomes and using them in the design of programs, curricula, and courses that are ultimately compared for credit transfer. He further asserts that faculty, as designers and facilitators of learning, must be encouraged to engage with outcomes-based practices more deeply. He cites lack of training as a significant reason for implementation problems in Europe. The services associated with faculty development and quality assurance at colleges and universities share a dependence on learning outcomes that form the backbone of teaching, learning, and assessment (Fink 2013), as well as of curricular alignment and mapping and course sequencing (Diamond 2011). These services counter what Allais (2007) calls “designing-down,” or mere conformity with system-generated outcomes. Given that learning experiences in courses and programs in Ontario postsecondary education are designed, communicated, and evaluated using learning outcomes, it is logical to provide authentic and ongoing guidance and support for faculty in their formulation and use of outcomes. Such faculty development must address the inevitable

264

Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers

and profound challenges that exist in engaging instructors in the use of learning outcomes beyond the mere ticking of boxes. Such work must be sensitive to the possibilities for content-referenced outcomes that are embodied in individual teaching practices, as well as the conceptual depth and breadth of a discipline. Such guidance must strive to recognize and ameliorate concerns among faculty regarding administrative imposition (see chapter 3; Allais 2007). Greater emphasis on faculty development will inevitably impact how faculty carry out what is often a determinative role in credit equivalency and the identification of transfer pathways between programs. Comprehensive training and sustained facilitation that fosters inclusive and genuine discourse within the institution can ensure a consistent understanding and use of learning outcomes and other inputs that may factor into credit transfer decisions, especially where turnover of responsibility among faculty and staff may impede the predictability and reliability of transfer practices. Noonan (see chapter 5) addresses and expands upon these problems, suggesting that consistent, coherent, widely used, and well understood approaches to developing and assessing learning outcomes must span what are called in Australia VET and higher education sectors, if outcomes are to be successful as tools for improving credit transfer. Trust and Collaboration Trust is widely viewed as critical to the establishment of transfer pathways, minimizing cost and other frictions in their development (Lennon et al. 2016). FitzGibbon (2014), in his analysis of learning outcomes and credit transfer, argues that the willingness of one institution to grant credit for learning in another institution is often founded in “trust in the quality of the sending institution. This trust can be derived from knowledge of the institution due to proximity, or belief in the accreditation or accountability processes that the institution undergoes” (15). Such willingness to view courses or whole credentials as equivalent can be impeded by vague perceptions of the esteem afforded programs and institutions, which may or may not be borne out by experience, facts, or evidence of student success. Challenges to the establishment and maintenance of trust and the fostering of collaborations that use learning outcomes to advance credit transfer exist throughout the complex array of institutional functions. Building capacity in this arena at the institutional level suggests there is a need to

Adoption, Adaptation, and Implementation 265

• connect the implementation of learning outcomes across academic and administrative divides; • consider educational inputs along with learning outcomes; and • encourage grassroots engagement in the formulation and uses of learning outcomes relevant to credit transfer through the establishment of continuous platforms and forums. A significant challenge to trust can be traced to what Biggs (2001) defines as a struggle between “establishing the optimal conditions for learning…[and] administrative convenience” (234). The ability of learning outcomes to define student achievement, and also to support the determination of credit equivalency, relies on communication and cooperation between faculty and administrative staff, and assumes that they are equally committed to the coherent and consistent formulation and use of outcomes. Wagenaar (2013) goes further to say that “it is of crucial importance that the instruments which facilitate (inter)national mobility and recognition are owned by the academic staff and used on a daily basis” (86). He drives home the need to regularly engage faculty and staff in understanding and connecting the uses of outcomes in support of curriculum, teaching, and learning, as well as credit transfer. Such an enterprise, as we see in Bragg’s examination of reverse credit transfer practice in the United States (see chapter 4), also requires sophisticated and resource-intensive data and information-sharing systems to support the effective use of outcomes in credit transfer. Allais, Noonan, Wagenaar, and Winch (see chapters 3, 5, 6, and 7) all remind us that differences at the program and faculty level in perception, experience, training, and conceptualization of learning can impede commitments to outcomes approaches. It is worth noting that these scholars have, in the previous chapters, argued that educational inputs such as content, context, time, student workload, and other “currencies” should be considered alongside outcomes when comparing programs and courses for credit transfer. It is useful to reflect upon the recent experiences in both the United States and Ontario that demonstrate the success of grassroots approaches where successful collaboration between institutions effectively leverages learning outcomes and builds upon faculty-driven research, experimentation, and innovation. Bragg (see chapter 4) points to the American Association of Colleges and Universities’ Quality Collaboratives Initiative that fosters partnerships between postsecondary institutions and engages decision makers, faculty, support staff, and others

266

Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers

in the identification and implementation of methods to improve the transfer process and enhance student learning. In Ontario, the Higher Education Quality Council of Ontario’s Tuning project is an example of pan-institution collaboration and trust-building among faculty and administrators from six colleges and universities (Lennon et al. 2014). Like its counterpart in Europe, this multi-year initiative was a “bottom-up process by those who are ‘on the ground’ to articulate learning outcomes that are relevant, appropriate and usable” (Lennon et al. 2014, 3). Divisions between Educational Sectors According to Noonan and Winch (see chapters 5 and 7), outcomes play out differently in vocational and academic education. Allais, Noonan, and Winch (see chapters 3, 5, and 7) all address how pre-set, narrow standards of vocational workplace competence and performance generated at the system level distort learning outcomes and limit the pedagogical autonomy of institutions. Winch (see chapter 7), in his examination of the attributes of and progression in learning, illustrates how narrowly framed vocational outcomes are incompatible with academic learning, and by extension credit transfer. To mitigate against such barriers, Winch’s argument for greater integration of underpinning and inferential ability (know-how) is augmented by Noonan’s (see chapter 5) call for tighter linkages between occupations and career structures that span both vocational and academic education, especially in light of the fact that universities readily accommodate professional and regulatory accreditation not entirely dissimilar from workplace competence standards. Ontario colleges and universities share a qualifications framework (see chapter 9; Ministry of Training, Colleges and Universities 2009) based, in large part, upon the university degree-level expectations (DLEs; Ontario Council of Academic Vice Presidents, n.d.). However, Ontario’s college programs, apart from degrees, are bound to systemdefined standards for programs as well as for employability skills and general education (Ministry of Training, Colleges and Universities 2003). By contrast, Ontario universities have greater autonomy in the creation of programs with unique dimensions, curricula and courses that need only reference DLEs. These disparities greatly influence credit transfer. The program approval processes in the college sector, which now requires the description of different points of entry into the program, and thus takes credit transfer more effectively into account (Ministry of Training, Colleges and Universities 2012), is an appealing model.

Adoption, Adaptation, and Implementation 267

Evolving Transfer Patterns Changes in the behaviour of students regarding transfer and mobility warrant examination when considering measures to facilitate their movement that are largely in the hands of institutions. Bragg’s (see chapter 4) description of student transfer patterns in the United States underscores what is also the case in Ontario (Lennon et al. 2016): Traditional progression scenarios—wherein students move in a linear, typically vertical pattern of transfer through successively more advanced credentials—are diminishing and lateral patterns are increasingly prevalent. Bragg’s analysis of student mobility in the United States points to a high rate of movement between multiple institutions during the course of study toward a single credential. This understanding of student patterns of mobility is complemented by the analysis of trends in Australia that show students moving from university to VET programs in more lateral approaches to the pursuit of credentials (Wheelahan 2000). Bragg recommends a holistic approach to student transfer that includes the facilitation of lateral transfer to enable students to move between institutions within a similar or dissimilar credential, or reverse transfer to enable students to convert four-year degree credits to college credit. Research at the Centre for the Study of Canadian and International Higher Education indicates that “current flows of students and established pathway agreements are not aligned” and that there may be greater benefit in establishing “pathways in complementary fields of education based on an analysis of data on student flows between qualifications, institutions and fields of education” (Lennon et al. 2016, 43). This suggests a growing need for multilateral agreements that make provision for lateral transfer and find alignments—perhaps through examination of learning outcomes—between disciplinary and occupational fields. Winch (see chapter 7) advocates for the decomposition of qualifications into modules and sub-modules to enable transfer for students with partial credentials. This modularization of higher education into units for accumulation and transfer of credit is already happening in Europe (see chapter 6). Breaking down credentials into transferable parts can be of value in Ontario, where an increasing number of students transfer mid-stream in a credential and from diploma to diploma, degree to degree, diploma to degree, or degree to diploma (Fallon 2015). Previous chapters offer some encouraging insights that may signal opportunities for more practical development and implementation of

268

Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers

transfer initiatives going forward. Bragg (see chapter 4) offers the concept of guided pathways, where advising, institutional systems, and data align to make transfer seamless and easy for students, with minimal loss of credit and maximum progression, along with the development of more intentional partnerships. The reliance on pre-arranged articulation, joint, or 2+2 agreements between institutions, whether bilateral or multilateral, is becoming increasingly common in Ontario. Several studies (Ontario Council on Articulation and Transfer 2013) suggest that students who transfer within a prearranged agreement are more likely to be successful post-transfer than students who transfer on a case-by-case basis. Agreements are also usually between highaffinity programs, which are easier to map for comparisons of courses, outcomes, and assessments. That said, anecdotal evidence suggests that learning outcomes are not systematically or consistently used in working out equivalencies, gaps, and potential bridges between programs. The role that outcomes play in the Tuning process’s meta-profiles and sectoral frameworks contributes significantly in Europe to efforts to span diverse programs, especially across the divide that often exists between vocational and academic education (Wagenaar 2013, 86). These frameworks create learning outcomes for groupings of related fields or occupational sectors such as healthcare, engineering, or the creative arts. Vocational or academic programs can be mapped to these outcomes in order to identify commonality and gaps necessary for the transfer of credit from one related area of study to another. Credit Transfer Currencies The discourse above outlines how different types of outcomes, especially those used at the institutional level, connect to practices related to administration, curriculum development, teaching and learning, and the comparison of programs and courses. This section expands upon some of the practical realities of credit transfer in Ontario discussed in chapter 2. Hallmarks of Ontario Transfer Practice Credit transfer in Ontario (see chapter 2) is generally awarded to two types of applicants: • students seeking individual assessment of credit based on course equivalency; or

Adoption, Adaptation, and Implementation 269

• students transferring within a prearranged transfer agreement in which blocks of courses may be transferred for credit. In both cases, outcomes, as well as other information, play a role. There is considerable variability in how and by whom transfer credit decisions are made in institutions (Fallon 2015). Moreover, Ontario transfer practices are in some instances vague and confusing for students (Arnold 2015). These practices leave the onus on the student to sort and compare programs and curriculum and, when transferring, to source and supply documents that define achievement and verify coursework so as to enable receiving institutions to make transfer credit decisions. Studies of student transfer in Ontario indicate that the majority of students do not understand the level of transfer credit they are granted, nor are they informed with respect to the process and reasons for the award or rejection of transfer credit (Usher and Jarvey 2012). However, institutions are making progress to improve student transfer by establishing transfer offices and providing student advisement closer to students’ areas of study, and many institutions are posting more clear and comprehensive information on their websites (McLoy et al. 2017; Trick 2016). Additionally, ongoing improvements to the ONTransfer website offer a significant resource to students making choices about transfer (ONTransfer, n.d.). Where pathways are developed through bilateral transfer agreements, the typical practice is to examine standards-based college diplomas against the requirements of individual university bachelor’s degrees (Fallon 2015). Learning outcomes, which may be more or less comparable across educational sectors, are typically mapped from one program to another in the process of negotiating transfer of a block of courses for credit. Other factors—particularly educational inputs such as time spent in learning, assessments, or resulting grades (FitzGibbon 2014)—may emerge in this process and are often given consideration in the calculus of credit transfer. These factors change and evolve depending on circumstances, creating tension and inconsistency between which currencies (outcomes or inputs) take precedence. Currencies in Credit Transfer beyond Learning Outcomes Time and workload. Time is an important currency often factored into credit transfer. “Seat time” (Johnstone, Ewell, and Paulson 2010), or the duration of study for a particular concept or knowledge unit, course, program, or credential, can be useful measures through which to com-

270

Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers

pare learning for credit transfer. In Ontario and other jurisdictions, credit hours are understood as a proxy for learning and, by extension, an input currency for credit transfer. How those credit hours are determined, and their relevance to the actual learning outcomes realized, is not always clear (Laitinen 2012). Moreover, the fact that they are used to determine costs suggests an evolution of the credit hour away from actual hours of learning. Despite this, credit hours are frequently considered in credit transfer, especially in Ontario. Time is also associated with workload. The European Credit Transfer and Accumulation System (ECTS) awards credit points for programs, courses, and sections of courses based on estimations of student workload in which one year of credit is defined as 1,800 hours or sixty ECTS credits, which may be distributed among classroom contact, laboratory work, placements, or self-study (see chapter 6). ECTS workload credits complement the assessment of competence based on defined learning outcomes (European Commission 2015). Winch (see chapter 7) suggests that the application of ECTS credit points in the European Commission Vocational Education and Training System is a means to break up qualifications into modules and sub-modules for ease of comparison and transferability. Credit transfer, in these examples, only works where there is a quantification of time as a commonly accepted proxy measure for effort expected and/or cognitive achievement. Content, curriculum, and assessment. Wagenaar (see chapter 6) discusses the advances made in Europe in considering content and assessment as an important input currency. His review of Tuning provides an overview of its process, which engages academics in the development of collective outcomes-based understanding of the essential core of disciplines while also developing resources such as recommended student workload; appropriate approaches to teaching, learning, and assessment; and quality enhancements meant to assist in the design of curriculum and courses that make “study programs compatible and comparable” (Wagenaar, see chapter 6, 147). Winch (see chapter 7) suggests that the evolving conceptualization of learning as achieved competence across a range of outcomes and skills is in line with the definition of credentials based upon curriculum and progression informed by a set of overarching learning outcomes. Wagenaar (see chapter 6) also addresses the Measuring and Comparing Achievements of Learning Outcomes in Higher Education in Europe (CALOHEE) project (Erasmus Programme of the European Union 2017). Its comprehensive

Adoption, Adaptation, and Implementation 271

approach is applied to five occupational sectors (engineering, education, history, nursing, and physics) and divides learning into four competency categories, enabling it to go beyond descriptions of learning outcomes to define competency based on application and assessment. The CALOHEE recommendations provide insights on how to create currencies that truly measure learning and allow institutions to compare that learning. Allais and Winch (see chapters 3 and 7) both advocate for learning outcomes at the course level that better reflect the learning process and the logic of the discipline—a consideration that is in line with the principles of CALOHEE. Allais (see chapter 3) focuses on the need for learning outcomes designed from the classroom up and embedded in the learning process to mitigate against problems of excessive granularity that occur when interpreting top-down standards. Grading and GPA. Another currency that is considered in granting credit transfer is course marks and grade point average (GPA). When attempting to earn transfer credit, students generally have to provide evidence of their graded achievement. In a multi-institution study of transfer practices in Ontario (Arnold 2015), an understanding of overall GPA and course grades was cited as one of the challenges in the evaluation of credit for transfer, given that Ontario institutions use different scales, with colleges generally using letter grades and universities numeric scales. Other Information Considered in Credit Transfer Admission standards. Variability of admission standards is another area of tension or uncertainty between colleges and universities in Ontario when it comes to credit transfer. In general, college programs provide relatively open access to applicants seeking postsecondary education. That said, college programs that are classified as “oversubscribed” may impose higher admission standards. Many universities have modest admission standards for general undergraduate admission, but the actual admission cut-offs for popular or selective programs can be much higher. This can also vary based on the state of demographic growth or decline in the secondary school population and the number of international applicants seeking admission to a particular institution. Therefore, it is difficult to know how to advise a transfer student regarding the need to meet certain standards for general or specific program admissions.

272

Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers

Reach-back and bridging. Prerequisite secondary school courses can become a barrier to students seeking to transfer credit between postsecondary programs with particular requirements. This “reach-back” into secondary school requirements can overlook where outcomes of a completed postsecondary program exceed the prerequisite. Bridging courses that compensate for real or perceived gaps in courses or learning units are often seen as a better way to address gaps between programs and curricula to ensure the required learning outcomes of the receiving institution are met when students transfer. A comprehensive study of bridging in the Ontario context (Lakehead University 2016) showed that academic bridging is not always useful or necessary, but instead provides productive orientation, socialization, and mentorship for transfer students entering a new postsecondary culture. Titling and duration. The way programs are titled, and their duration of study, may also confound comparability. The Ontario Qualifications Framework (Ministry of Training, Colleges and Universities 2009) provides basic titling for credentials such as the diploma, advanced diploma, bachelor’s degree, and honours bachelor’s degree, and suggests the average length of programs awarding these credentials. Despite this, titling and duration can vary widely between institutions and among programs. The Ontario Credential Validation Service for Ontario colleges (Ontario College Quality Assurance Service, n.d.) attempts to formulate a consensus on titling protocols. University titling for undergraduate degrees is more open to variation, and it can be challenging for students and institutions to determine which credentials to compare for transfer. Student records. A recent study of alternatives to traditional student records (e.g., transcripts and diplomas) traces how advances in technology and a greater focus on learning outcomes are resulting in innovation and new formats for “comprehensive student records” that provide complementary information about individual student attainment. (Duklas and Bridge 2017). These records enable institutions, program administrators, and faculty (and employers) to examine what types of learning, experiences, and assessments underpin static and often opaque transcript and diploma information, such as course titles, dates, credits, and marks. Examples include the European Diploma Supplement (European Commission 2015) and others in the United States, such as badges or micro-credentials that recognize cognitive or skills learning, and e-transcripts and online diplomas that link to detailed

Adoption, Adaptation, and Implementation 273

course outcomes, artifacts that demonstrate achievement, and contextualization for grades and credit weight. Implications for Ontario We have seen in the example of the ECTS presented by Wagenaar (see chapter 6) that a long-term commitment to the improvement of credit transfer and student mobility is necessary in an education environment that places a high value on student-centred learning, social and economic equity, and opportunity. Many jurisdictions, like Ontario, are developing and testing new practices, and in Ontario much of this work is occurring at local institutions and among groups of institutions that leverage “on the ground” experience. These efforts have been encouraged, funded, and shared through the Ontario Council on Articulation and Transfer. Much of this work explores links between programs such as those described in chapter 1 that are designed to cross educational sectors. It is undergirded by research conducted by the Higher Education Quality Council of Ontario, the Centre for the Study of Canadian and International Higher Education, and elsewhere that examines statistical, theoretical, and practical aspects of learning outcomes and credit transfer in Ontario. There is much work still to do, particularly in meta-analysis of and building connections between what are often disconnected initiatives. Nevertheless, it is clear that the practical application of new outcomes-based approaches to credit transfer as it relates to institutions depends on collaboration, trust-building, and understanding that arise from project-based exploration that involves the comparison of common learning outcomes in the course of developing pathway agreements and partnerships. Multi-Institution Pilot Projects The pursuit of several ambitious large-scale transfer agreements offers insight for the future of student mobility in Ontario. These agreements leverage comparison of learning outcomes and broad engagement of faculty and staff across multiple colleges and universities to enable reciprocity of credits in the curriculum of related disciplines and/or within broad occupational sectors. Such work is driving a shift toward a culture of transfer in Ontario (Ontario Council on Articulation and Transfer 2018b). These agreements include an unsuccessful effort to integrate—for the purpose of credit transfer—the curriculum of Ontario’s

274

Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers

college and university business programs and an ultimately successful agreement between all twenty-four of the province’s college business programs that sets a standard block of credit transfer for students moving between colleges (Ontario College Heads of Business 2016). A similar multi-institution, cross-sector agreement in the regulated field of engineering is being developed (Ontario Council on Articulation and Transfer 2018a). These schemes all involve deep engagement of faculty, academic administration, and registrarial staff in the comparison of learning outcomes and mapping of programs to determine gaps and assign equivalency. A third project focused on transferability of college general arts and science credits to related university programs such as business, humanities, and liberal arts through mapping of learning outcomes for course-to-course and course-to-program comparison (Ontario Council on Articulation and Transfer 2016). This project has resulted in 3,435 course transfers at thirty-five institutions. These and other multi-institutional projects have provided valuable insights as to the features that determine successful development and implementation of agreements that respect diverse institutional and program emphases, while seeking to use outcomes and other measures to compare and align curriculum. It is evident from these that learning outcomes play a valuable role, especially within broad occupational fields where there is a strong sense of disciplinary content and/or where professional regulation may also guide curriculum. In unregulated fields of study, the use of outcomes assists the mapping of relevant and underpinning knowledge in disciplines that enjoy affinities. From an implementation perspective, it is evident from the experience gained in these projects that colleges and universities need to view agreements as beneficial to institutional interests. Analysis of the success factors associated with the completed college-to-college business project (Ontario Council on Articulation and Transfer 2018b) revealed that negotiation of agreements requires effective communication among parties, full engagement of the registrar’s office, and participation of staff and academic faculty who are well trained for the task. This assessment also indicated that the capacity to assign credit through automation could ensure a more consistent application of agreements. Moreover, given the potential magnitude of such multi-institution credit transfer agreements that span educational sectors, a central and ongoing challenge will lie in sustainability—especially as program leaders change and curricula evolve.

Adoption, Adaptation, and Implementation 275

Tools Institutions use a multiplicity of learning-outcomes-based tools in various spheres of work (e.g., curriculum and faculty development, learning management, and quality assurance), several of which were discussed in an earlier section of this chapter. Careful evaluation of the effectiveness of their use and the efficiency they bring to the determination of pathways and credit transfer is necessary. Equally important is the embedding of reflection points regarding credit transfer in cyclical program review processes in Ontario. These represent an opportunity to leverage and integrate tools into credit transfer activities that were developed for the quality assurance process. The products and reports generated by such tools are useful to program self-study exercises, especially where analysis of the program includes learning outcomes and alignments between degree-level expectations, standards, programs, curricula, and course or unit outcomes. Moreover, self-study templates could include a section regarding current and prospective transfer pathways that refer to curriculum maps or related visualizations of learning paths to indicate entry points and relevant areas of equivalency with programs at partner institutions. Funding Among the variables limiting transfer between Ontario college and university institutions are the potential costs and uncertain benefits of transfer, including loss of revenue to the institution (Lennon et al. 2016). There is little incentive for many institutions to facilitate student mobility in light of current funding formulas and recruitment challenges. A significant factor that will impact institutional commitments to credit transfer in Ontario is the new “corridor funding” model that will soon be introduced (Ministry of Advanced Education and Skills Development 2016). After decades of steady enrolment growth, colleges and universities will have their funding allocations more or less frozen in place, regardless of whether their enrolments decline or increase. At the same time, they are now being asked to focus more on quality improvements and institutional differentiation (Levy 2017). This parallels a similar policy being implemented in Australia, where the government will “effectively end the demand-driven funding system for higher education by placing financial caps on universities for government-funded undergraduate places for 2018 and 2019 and then only providing additional funding across all universities commensurate

276

Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers

with population growth” (Noonan, see chapter 5, 135). In both cases, increasing the commitment to recognition of prior learning and credit transfer might actually represent opportunities for some institutions to both differentiate themselves and maximize revenue from their now more fixed allocations of government funding. Conclusion The perceived value of learning outcomes rests on the assumption that they can improve education overall and make programs more comparable. Equally problematic is the belief that learning outcomes effectively capture learning in a program (see chapter 3; Allais 2012). This chapter has argued that learning outcomes are of different and interdependent types and serve widely diverse functions within the system, and within institutions that necessitate interpretation and variability, which must be built into how we conceive and use outcomes for transfer. Such limitations and complexity also suggest that learning outcomes cannot stand alone as a currency for transfer but should be complemented by other measures of learning. Considering the extent to which outcomes are embedded in the Ontario system, their use in institutions must be supported by consistency and buy-in at all levels before learning outcomes can become foundational to credit transfer. References Adam, S. 2004. “An Introduction to Learning Outcomes: A Consideration of the Nature, Function and Position of Learning Outcomes in the Creation of the European Higher Education Area.” In EUA Bologna Handbook, edited by E. Froment, J. Kohler, L. Purser, and L. Wilson, (section B2.3-1). Berlin, Germany: European University Association. Algonquin College. 2018. COMMS: “An Overview.” http://www.algonquincollege.com/comms/ Allais, S. 2007. The Rise and Fall of the NQF: A Critical Analysis of the South African National Qualifications Framework (Doctoral dissertation). University of the Witwatersrand, Johannesburg, South Africa. ———. 2012. “Claims vs. Practicalities: Lessons about Using Learning Outcomes.” Journal of Education and Work, 25 (3): 331–354. Allais, S., D. Raffe, and M. Young. 2009. Researching NQFs: Some Conceptual Issues. Geneva, Switzerland: International Labour Office. Anderson, L. W., D. R. Krathwohl, P. Airasian, K. Cruikshank, R. Mayer,

Adoption, Adaptation, and Implementation 277

P. Pintrich, … M. Wittrock. 2001. A Taxonomy for Learning, Teaching and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives. New York, NY: Longman Publishing. Aon Hewitt. 2016. Developing Canada’s Future Workforce: A Survey of Large Private-Sector Employers. Ottawa, ON: Business Council of Canada. http://thebusinesscouncil.ca/wp-content/uploads/2016/03/ Developing-Canadas-Future-Workforce.pdf Arnold, C. H. 2015. Transfer Literacy: Assessing Informational Symmetries and Asymmetries (Doctoral dissertation). Ontario Institute for Studies in Education, University of Toronto. Biggs, J. 2001. “The Reflective Institution: Assuring and Enhancing the Quality of Teaching and Learning.” Higher Education, 41: 221–238. Biggs, J. B., and K. F. Collis. 2014. Evaluating the Quality of Learning: The SOLO Taxonomy (Structure of the Observed Learning Outcome). Cambridge, MA: Academic Press. Bridge, J., K. Baggot, M. Wilson, A. Hogue, J. Post, and G. Murphy. 2016. College University Pathways for Games: Game Education Matrix. Toronto, ON: Ontario Council on Articulation and Transfer. Carter, I., J. Coyle, and D. Leslie. 2011. “Easing the Transfer of Students from College to University Programs: How Can Learning Outcomes Help?” Canadian Journal of Higher Education, 41 (2): 10–27. Diamond, R. 2011. Designing and Assessing Courses and Curricula: A Practical Guide. Hoboken, NJ: John Wiley & Sons. Duklas, J., and J. Bridge. 2017. Creating a Typology for Alternative Credentials. Toronto, ON: Ontario Council on Articulation and Transfer. European Centre for Development of Vocational Training (Cedefop). 2017. Overview of National Qualifications Framework Developments in Europe. Hamburg, Germany: UNESCO Institute for Lifelong Learning. http://www.cedefop.europa.eu/en/publications-and-resources/publications/4163 European Commission. 2010. Tuning Educational Structures in Europe: A Guide to Formulating Degree Programme Profiles. Bilbao, Spain: University of Deusto. ———. 2015. ECTS Users’ Guide. Publications Office of the European Union. http://ec.europa.eu/education/ects/users-guide/index_ en.htm Erasmus Programme of the European Union. 2017. CALOHEE Working Paper: Common Structure for Subject Area Based Qualification Frameworks. https://www.calohee.eu/wp-content/uploads/2016/06/CALOHEE-Working-Paper.pdf

278

Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers

Fallon, N. 2015. Learning Outcomes in Credit Transfer: A Key Tool for Innovation in Student Mobility. Toronto, ON: Ontario Council on Articulation and Transfer. Fink, L. D. 2013. Creating Significant Learning Experiences: An Integrated Approach to Designing College Courses. Hoboken, NJ: John Wiley & Sons. FitzGibbon, J. 2014. Learning Outcomes and Credit Transfer: Examples, Issues, and Possibilities. Vancouver, BC: British Columbia Council for Articulation and Transfer. Havnes, A., and T. S. Prøitz. 2016. “Why Use Learning Outcomes in Higher Education? Exploring the Grounds for Academic Resistance and Reclaiming the Value of Unexpected Learning.” Educational Assessment, Evaluation and Accountability, 28 (3): 205–223. Hubball, H., and N. Gold. 2007. “The Scholarship of Curriculum Practice and Undergraduate Reform: Integrating Theory into Practice.” New Directions for Teaching and Learning, 112: 5–14. Hussey, T., and P. Smith. 2002. “The Trouble with Learning Outcomes.” Active Learning in Higher Education, 3 (3): 220–233. Johnstone, S. M., P. Ewell, and K. Paulson. 2002. Student Learning as Academic Currency. ACE Center for Policy Analysis. http://www.acenet. edu/bookstore/pdf/distributed-learning/distributed-learning-04. pdf Kennepohl, D. 2016. “Incorporating Learning Outcomes in Transfer Credit: The Way Forward for Campus Alberta?” Canadian Journal of Higher Education, 46 (2): 148–164. Kopera-Frye, K., J. Mahaffy, and G. M. Svare. 2008. “The Map to Curriculum Alignment and Improvement.” Collected Essays on Learning and Teaching, 1 (8): 8–14. Laitinen, A. 2012. Cracking the Credit Hour. Washington, DC: New America Foundation and Education Sector. http://www.cbenetwork.org/ sites/457/uploaded/files/Cracking_the_Credit_Hour_Sept5_0.pdf Lakehead University. 2016. Transitioning to University: Best Practices for College Transfer Bridging Courses. Toronto, ON: Ontario Council on Articulation and Transfer. Lennon, M. C. 2014. “Incremental Steps toward a Competency-Based Post-Secondary Education System in Ontario.” Tuning Journal of Higher Education, 2 (1): 59–89. Lennon, M. C., A. Brijmohan, E. Lavigne, J. Yang, G. Moodie, and L. Wheelahan. 2016. Ontario Student Mobility: Carving Paths of Desire. Toronto, ON: Centre for the Study of Canadian and International High-

Adoption, Adaptation, and Implementation 279

er Education, Ontario Institute for Studies in Education, University of Toronto. Lennon, M. C., B. Frank, J. Humphreys, R. Lenton, K. Madsen, A. Omri, and R. Turner. 2014. Tuning: Identifying and Measuring Sector-Based Learning Outcomes in Postsecondary Education. Toronto, ON: Higher Education Quality Council of Ontario. Levy, S. 2017, December 15. Next Steps in Strategic Mandate Agreements and Changes in the College and University Funding Formulas [Memorandum]. Toronto, ON: Ministry of Advanced Education and Skills Development. http://www.tcu.gov.on.ca/pepg/documents/next_ step_sma_agreements_2017.pdf Lokhoff, J., B. Wegewijs, K. Durkin, R. Wagenaar, J. Gonzalez, A. K. Issacs,…M. Gobbi. 2010. A Tuning Guide to Formulating Degree Programme Profiles. Bilbao, Spain: University of Deusto. MacFarlane, A., and S. Brumwell. 2016. The Landscape of Learning Outcomes Assessment in Canada. Toronto, ON: Higher Education Quality Council of Ontario. McLoy, U., V. Baker, Williams, K., and H. Decock. 2017. Seneca College’s Degree and Credit Transfer Office: A Profile of Users and Examination of Outcomes. Toronto, ON: Ontario Council on Articulation and Transfer. Milot, P., and G. Gauthier. 2016, October. uoSyllabus: Outil en ligne de conception de plan de cours et d’analyse curriculaire. Presented at the G3 Conference: La qualité dans les institutions de l’enseignement supérieur—Vos outils sous la loupe, Geneva, Switzerland. Ministry of Advanced Education and Skills Development. 2016. College Funding Model Consultation Paper. http://www.tcu.gov.on.ca/pepg/ Audiences/colleges/cff/fundingreform.html Ministry of Advanced Education and Skills Development. 2017. “Key Performance Indicators, Colleges and Other Public Institutions.” http://www.tcu.gov.on.ca/pepg/audiences/colleges/colindicator. html Ministry of Training, Colleges and Universities. 2003. Frameworks for Programs of Instruction. Toronto, ON: Author. http://www.tcu.gov. on.ca/pepg/documents/FrameworkforPrograms.pdf ———. 2009. Ontario Qualifications Framework. http://www.tcu.gov. on.ca/pepg/programs/oqf/ ———. 2012. Application for Program Approval Form. www.forms.ssb. gov.on.ca/mbs/ssb/forms/ssbforms.nsf/...58.../58-2678E_2012.pdf http://www.tcu.gov.on.ca/pepg/publications/MajorPolicyFrame-

280

Jean Bridge, Jovan Groen, Mary Pierce, and Richard Wiggers

work.pdf Ontario College Heads of Business. 2016. HOB ONCAT Implementation Project. Toronto, ON: Ontario Council on Articulation and Transfer. Ontario College Quality Assurance Service. n.d. “About CVS: Providing Program-Level Quality Assurance.” http://www.ocqas.org/credentials-validation-service/about-cvs/ Ontario Council of Academic Vice Presidents. n.d. “Appendix 1: OCAV’s Undergraduate and Graduate Degree Level Expectations.” In Quality Assurance Framework. Toronto, ON: Ontario Universities Council on Quality Assurance. http://oucqa.ca/wp-content/uploads/2013/06/APPENDIX-1.pdf Ontario Council on Articulation and Transfer. 2013. Summary of ONCAT-Funded Pathways and Transfer Research. Toronto, ON: Ontario Council on Articulation and Transfer. ———. 2016. Pathways for Early Childhood Leadership. Toronto, ON: Ontario Council on Articulation and Transfer. ———. 2018b. Research on Current System-Level Business Transfer Agreements: Awareness, Usage and Maintenance/Sustainability. Toronto, ON: Ontario Council on Articulation and Transfer. ONTransfer. n.d. “ONTransfer.ca by ONCAT.” http://www.ontransfer. ca Sattler, P. 2011. Work Integrated Learning in Ontario’s Postsecondary Sector. Toronto, ON: Higher Education Quality Council of Ontario. Trick, D. 2016. The Cost of Recruiting and Admitting Transfer Students: Results of a Survey of Ontario Colleges and Universities. Toronto, ON: Ontario Council on Articulation and Transfer. Turcotte, J. F., L. Nichols, and L. Philipps. 2016. Maximizing Opportunity, Mitigating Risk: Aligning Law, Policy and Practice to Strengthen Work-Integrated Learning in Ontario. Toronto, ON: Higher Education Quality Council of Ontario. UNESCO Institute for Lifelong Learning. 2002. UNESCO Guidelines for the Recognition, Validation and Accreditation of the Outcomes of Non-Formal and Informal Learning. http://unesdoc.unesco.org/images/0021/002163/216360e.pdf Usher, A., and P. Jarvey. 2012. Student Experiences in Credit Transfer at Ontario Colleges. Toronto, ON: Colleges Ontario & Higher Education Strategy Associates. https://www.collegesontario.org/research/external-reports/Credit%20Transfer%20Summary%20Report.pdf Wagenaar, R. 2013. “Columbus’ Egg? Qualifications Frameworks, Sectoral Profiles and Degree Programme Profiles in Higher Education.”

Adoption, Adaptation, and Implementation 281

Tuning Journal for Higher Education, 1 (1): 71–103. Weingarten, H. 2013. “Learning Outcomes: Lessons Learned and the Next Big Thing.” (Blog post). It’sNotAcademic. http://blog-en.heqco.ca/2013/05/harvey-p-weingarten-learning-outcomes-lessonslearned-and-the-next-big-thing/ Wheelahan, L. 2000. Bridging the Divide: Developing the Institutional Structures that Most Effectively Deliver Cross-Sectoral Education and Training. Hamburg, Germany: UNESCO Institute for Lifelong Learning. Zakani, S., B. Frank, R. Turner, and J. Kaupp. 2016. Framework for Transferability between Engineering and Technology Programs. Toronto, ON: Ontario Council on Articulation and Transfer.

11

Summary of Findings and Recommendations Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

Introduction The research and perspectives presented in this publication have offered many insights about the use of learning outcomes as a means to improve credit transfer. The preceding pages have given voice to those who are cautiously optimistic that outcomes-based approaches—especially combined with other evidence and measures of student achievement—can deliver value in improving credit transfer (see chapters 4 and 6). They have also provided a forum for scholars who find learning outcomes deficient in their conceptualization and application, both as benchmarks of learning progression and as reliable measures for the determination of credit transfer, especially across the divides that often exist between vocational and academic education, disciplinary traditions, and generic and specialized knowledge (see chapters 3, 5, and 7). It is clear from the previous chapters that there are significant differences among jurisdictions in terms of the drivers and evolution of systems. Each distinct set of needs and problems has spurred a wide array of approaches at the systems, governmental, and institutional levels that shape how learning outcomes may influence credit transfer. This chapter reviews the critical factors that inform and intersect with one Learning Outcomes, Academic Credit, and Student Mobility, edited by Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon. Montréal and Kingston: McGill-Queen’s University Press, Queen’s Policy Studies Series. © 2020 The School of Policy Studies, Queen’s University at Kingston. All rights reserved.

284

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

another in complex ways to affect how Ontario’s students are facilitated in their access to and progression within postsecondary education. It provides an integrated series of recommendations drawn from the analysis and commentary aggregated in this publication. Taken as a whole, these recommendations aim to inform how Ontario could and should address the use of learning outcomes as a tool for credit transfer. We conclude that if outcomes-based approaches are to offer genuine promise in improving credit transfer in Ontario, their use must rest upon awareness and reconciliation of numerous complex considerations and on commitment to building deep and sustainable engagement among all stakeholders, so as to create and align policies and mechanisms that will better support the needs and aspirations of students. Contextual Conditions and Considerations It is important for those working to develop, implement, and assess the benefits of outcomes-based approaches to credit transfer to think broadly and holistically. The challenge of assessing the role of learning outcomes as a tool for credit transfer is that which Remington (2013) describes as a “wicked problem” where circumstances, understandings, and pressures within complex systems are diverse, fluid, and ever-evolving, and where the search for straightforward answers or easily applied solutions is futile. Instead, it is more likely that there may be a number of simultaneous and rewarding options worth pursuing. The conditions necessary for undertaking such work involve genuine engagement of the many stakeholders in examining trade-offs, envisioning future scenarios, and crafting transition and sustainability strategies wherein “local and expert knowledge is incorporated into problem-solving processes and solutions that have ‘sticking power’” (Remington 2013, 73). Therefore, before we explore recommendations pertaining to the use of learning outcomes for credit transfer that may be applied in the province of Ontario, we wish to highlight the following circumstances and considerations, which have factored greatly in our deliberations. We believe these factors will continue to bear heavily on the development of sustainable approaches to credit transfer in Ontario. The following points outline these key factors: • Context—history, symbolism, and materiality Notions of learning outcomes and student mobility have evolved out of different, and at times overlapping, social and educational traditions. With roots in behavioural psychology, competen-

Summary of Findings and Recommendations 285

cy-based education, and standards-based reforms, learning outcomes speak to the entwined paradigms of student-centredness and neoliberalism (see chapters 3, 4, 5, 6, 8, and 9) that have shaped contemporary postsecondary education. Developments in access to higher education, credential attainment, and student mobility have sought in diverse ways to adapt to this complex terrain (see chapters 4, 8, and 9). Hence, we see systems respond differently to such competing imperatives depending upon their evolving social priorities (see chapters 3, 4, 5, 6, and 9). System policies and practices that may, to varying degrees, use outcomes-based approaches to improve teaching and learning, as well as credit transfer, also transmit symbolic meaning as to what is valued in the system (see chapters 8 and 10). Finally, how learning outcomes themselves are defined, expressed, and deployed throughout the entirety of the system is based on shared understandings and the capacity to communicate meaning and to inform qualifications frameworks, quality assurance requirements, curriculum design, as well as the delivery of courses that embed the logics of specific disciplinary and occupational fields (see chapter 10). The vicissitudes of contexts also inform the material realities of operations within education systems and institutions—influencing the degree of readiness for engagement, as well as the availability of resources and tools for deployment toward projects connected to the use of learning outcomes as a tool to inform credit transfer (see chapter 10). • Pedagogy—discourses, devices, and practices Learning outcomes are fundamentally situated within the domain of teaching and learning, where conceptualization of learning, the means by which learning progresses, and the choices that educational instructors and educational developers make to facilitate learning are negotiated (see chapters 3, 8, and 10). Learning cannot be simply reduced to qualification descriptors, graduate profiles, or program standards (see chapter 8) and must strive to authentically represent and relate many categories and preconditions of knowledge and learning, as well as learning progression (see chapters 3, 5, 7, 8, and 10). Care must be taken to address consciously the discourses that underlie and colour outcomes (see chapter 8). Among these are the myriad ways in which learning in vocations and regulated fields is focused on transition into the

286

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

labour market, as opposed to the development of knowledge and skill needed for higher-level study in the field and/or for improved social and economic mobility and citizenship. Such considerations indicate that the structure of curriculum—its content, as well as the rules by which knowledge is classified, framed, and converted into meaning—is profoundly important in building and negotiating learning pathways and credit transfer. • Policy—concept, practice, and process Higher education is an integral mechanism for the achievement of government social goals, linked inevitably to overarching cultural values such as equity and social/economic inclusion (see chapters 8 and 9). Explicit and implicit choices in the policy realm dictate how each jurisdiction responds to its unique political circumstances, social challenges, and inherited models of postsecondary education provision (see chapter 9). Common policy purposes that lie at the intersection of learning outcomes and credit transfer address student success, coordination in program and curriculum development, differentiation and alignment within the system, coordination of student progression, and equitable access to advanced learning (see chapter 9). Yet outcomes-based policies can be a double-edged sword in so far as they may reinforce hierarchies and boundaries of disciplines and vocations, as well as in how they may restrict access to forms of knowledge and social benefits (see chapter 9). The success of learning outcomes policies is not assured but is more likely when their implementation is focused on teaching and learning rather than system reform (see chapter 9). Policy choices and activities, including funding and implementation, benefit from clarity of goals as well as leadership that is mindful of how top-down, centralized, consultative, or grassroots approaches influence the formulation, alignment, and widespread support necessary for long-term sustainability of policy (see chapter 9). Inter-institutional collaboration across sector boundaries, which promotes evaluation and development of well-researched and innovative strategies and practices, can be encouraged and facilitated by government and non-governmental agencies (see chapters 9 and 10). • Structure—frameworks, tools, and mechanisms Outcome-based credit transfer relies on complementary and interconnected elements of system design to define and enable com-

Summary of Findings and Recommendations 287

parison of learning across educational sectors, institutions, and programs. Qualifications frameworks, which exist in most jurisdictions, typically use learning outcomes to structure the development, classification, and recognition of skills, knowledge, and abilities along a continuum of progressively advancing levels and credentials (see chapters 9 and 10). Qualifications frameworks provide a fundamental instrument for student mobility across a system. Tools such as undergraduate degree-level expectations (UDLEs) and diploma program standards, as well as taxonomies and rubrics, are essential means by which disciplinary and broadly transferable knowledge, skill, and ability can be expressed in commonly agreed-upon terms that underlie equivalency for credit transfer. In addition, Tuning as a method for establishing shared reference points for disciplines, and mechanisms such as quality assurance reporting, provide positive and constructive mechanisms that contribute to trust in the consistency and quality of institutions, programs, and credentials, and ultimately in credit transfer (see chapters 9 and 10). All of these elements both constrain and enable how learning outcomes serve student mobility. Where they are aligned well with clear policy goals, they enable the many parts of the system to operate in concert and more effectively benefit the necessary range of stakeholders. • Stakeholders—roles, responsibilities, and motivations Effective planning, operation, and delivery of diverse types of postsecondary education across a complex ecosystem requires rationalization and synergy with respect to who does what, how, and for what reasons (see chapters 9 and 10). Students are the primary stakeholders and beneficiaries of the system, although the larger community and employers derive nearly equal benefit from a well-functioning system of postsecondary education (see chapter 8). We have learned that leadership has greater impact on student mobility when it is informed and influenced by stakeholders, and where organizational structures have addressed the benefits and limitations of autonomy and compartmentalization (see chapter 9). These cautions pertain to both systems and institutions. Disconnects between the ministerial and local levels can advance or limit the extent to which and ways in which learning outcomes are used to improve credit transfer (see chapter 9). As manifested in institutions, learning outcomes must reference

288

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

system-level outcomes-based frameworks, standards, and quality assurance requirements, while also being formulated for direct teaching and learning purposes, and ultimately for use by program leaders and administrators who determine credit transfer arrangements (see chapter 10). Such complexity in functional tasks pertaining to outcomes and transfer suggests the need for clarity, collaboration, and training. • Power—actors, dynamics, and trust Actors within the postsecondary system are many and diverse. While it might seem that influence impacting the system cascades directly from governments to institutions, it is more the case that—in well-functioning systems—goals, policies, and their implementations are the result of cycles of listening, attention to results, multivalent leadership, and widespread action on the part of all who participate. Therefore administrators, faculty members, professional staff, and students all affect how the system evolves and functions as much as do governments, policymakers, arm’slength agencies, and academic leaders (see chapters 9 and 10). In resource-constrained systems, collaboration among all players in the system mitigates the many challenges that exist in simultaneously meeting accountability requirements and investing the time and resources required to ensure the provision of high-quality programs and services to students (see chapters 9 and 10). There are many nuanced dynamics that determine how priorities and policies are set (or not) in the system, and what actors hold and share the levers. Awareness of and transparency in these is central to managing their operations for the purpose of meeting core goals (see chapter 9). This is especially true in light of the logics applied by governing systems and institutions, wherein overarching and well-rationalized system policies may run afoul of countervailing priorities, processes, and practices in individual institutions (see chapter 8). Similar tensions arise in how systems and institutions manage risk in the structuring of programs, curriculum, and transfer pathways so as to protect students and ensure credential attainment. Such measures may, under the guise of orderly learning progression, set up confusing or overly complex barriers and requirements that limit student access to pathways of their choice (see chapter 10). Related dynamics pertain to the problematics of trust and inequities in esteem that exist between vocational and

Summary of Findings and Recommendations 289

academic sectors and among institutions. These obstacles may be reinforced by the use of learning outcomes that are not compatible between sectors (see chapter 8). Moreover, top-down imposition of outcomes and transfer models can stifle trust and the capacity for agency and creativity by those on the ground who must implement outcomes-related policy and ensure its mechanisms are carried out (see chapters 3, 6, 8, and 10). That said, development and use of learning outcomes at multiple sites, especially in institutions, can foster growth in trust across critical boundaries (see chapters 3, 4, and 10). Recommendations Suggestions for how learning outcomes can be used as effective tools for improving credit transfer have emerged out of the investment in rigorous critical inquiry by many policy leaders, scholars, educators, and administrators from across government, quality assurance agencies, arm’s-length research organizations, and institutions that deliver postsecondary education. This publication has assembled and examined a large and multidimensional body of literature and analysis and considered a diverse array of initiatives, innovations, practices, and experiences. The recommendations that follow draw upon our analysis of conceptual and theoretical frameworks, systems, and institutions, and are directly referenced to lessons learned from the invited international scholars. They include core themes that repeatedly arise in these many analyses that dig deeply into diverse contexts and circumstances. What follows are a series of broadly framed principles that are meant to guide Ontario as it continues to consider its approach to the use of learning outcomes in pursuit of improvements to credit transfer. These are broad recommendations that focus on potentials and options rather than providing specific prescriptions for the future. 1. Learning outcomes, as with any system construct, should not be overloaded conceptually with unrealistic expectations that they alone can make postsecondary education accessible, equitable, cost-effective, and time-efficient. There is an impulse within the Ontario system, and within other higher education jurisdictions examined in this publication, to think of learning outcomes as a potential silver bullet capable of reducing complexities to fit a single-solution framework (see chapter 8). For example, in South Africa, “learning outcomes were perceived as a mechanism

290

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

that could transcend the specificities of different contexts and integrate this fragmentation to create a coherent, nationally integrated education system” (Allais, see chapter 3, 58) and to bridge between vocational and academic knowledge, as well as across disciplines and areas of pure and applied research. However, as Allais (see chapter 3) notes, “when too much weight is placed on learning outcomes as a policy mechanism, in the absence of specialized and well-trained communities of trust who share a good understanding of their knowledge fields, the tendency to over-specification is inevitable,” (69–70) and ultimately, “over-reliance on learning outcomes collapses in on itself eventually, which means it cannot assist.” (66). It is necessary to be moderate in our expectations of what learning outcomes can do for credit transfer. Learning outcomes can inform credit transfer decisions and can serve very effectively in building a common understanding of credentials, but are not sufficiently meaningful to be used in isolation (see chapters 3, 5, 7, 8, and 10). Learning outcomes provide useful but ultimately inadequate information in the absence of commonly pursued standards and trust, or an extensive, shared familiarity between the sending and receiving institutions of factors influencing the depth, breadth, progression, and sophistication of learning. Moreover, learning outcomes serve a plethora of purposes in postsecondary systems and institutions—in Ontario and around the world—by virtue of the fact that they are formulated and operate to classify qualifications, define standards, specify programs, and inform the design of curriculum and courses. At the same time, they provide a means for quality and accountability (see chapter 10). These diverse instantiations of learning outcomes are also tasked with addressing progression in transversal as well as discipline-specific knowledge, in both vocational and academic contexts. Hence, we see learning outcomes as multifaceted, working for many purposes and masters, and using descriptors, language, and taxonomies that are not always easily legible across educational sectors, disciplines, and occupational fields (see chapters 3, 5, 8, and 10). When learning outcomes are used for credit transfer, high levels of intricacy and variability are encountered, not the least of which is the very nature of learning progression and the capacity of the system to prepare students for increasingly more advanced levels of knowledge (see chapters 7 and 8). There are many dimensions of practical application to consider as well. For example:

Summary of Findings and Recommendations 291

• How do those individuals engaged in structuring transfer agreements factor in and bridge the diversity of ways in which outcomes are expressed in vocations, professions, and academic fields of study? • How can outcomes be reliably and consistently used in the determination of equivalency of programs, courses, and more granular units of learning? • How can academics, administrators, and registrars work with these measures? • Is it sufficient to inform students of the likely gaps and deficits in learning that they are most likely to encounter as they transfer and the risks to academic success that they face, or do we wish to safeguard against such risks as fully as possible before students embark on new academic pathways? • Should qualifications be broken down into learning modules or micro-credentials so as to better serve the interests of students seeking to transfer credit from a partially completed credential? • Might it be possible to improve student records or transcripts by embedding learning outcomes? • Considering the provincial commitment to experiential learning and the persistent limitations faced in awarding credit for prior learning, can learning outcomes contribute to the recognition of such learning for academic credit? These are among the pressing questions that deserve greater investigation, and for which there are many answers. Looking beyond learning outcomes, it is clear that more types of information are required or are already commonly used to support credit transfer, such as • • • • • • • •

admissions standards content teaching methods learning activities time and/or workload measures assessment strategies learning tools sites and facilities.

Consideration of time invested in learning tasks is among the most important factors that may complement learning outcomes and inform

292

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

credit transfer decisions (see chapters 3, 6, 7 and 10). With regard to time on task, as Winch (see chapter 7) comments, “credit is accumulated through learning, whether formal, informal or non-formal; this must, therefore, have taken time to occur. If credit transfer, especially across national boundaries, is to be perceived as trustworthy, then there must be a common measure of the time taken to achieve the credits, even if the time taken is a proxy for the effort expended.” (see chapter 7, 176). Currently, Ontario postsecondary institutions do not share a common definition or standard for credit conversion in relation to time. Ontario institutions may not wish to move toward a common standard, but investigations into methods for conversion and equation would be productive. The most developed conception of credit, based on workload and time, is held within the European Credit Transfer System (ECTS) model, in which sixty ECTS credits are associated with the student workload of a full-time academic year of formal learning necessary to achieve the intended learning outcomes. This system provides the ability to equate credit as a proxy for time spent in learning with reference to a common standard and may serve to contribute meaningfully to the basis of credit transfer decisions. If there are to be reforms to Ontario’s system of credit transfer that emphasize more meaningful formulation of learning outcomes, as well as measures of educational input such as those outlined above, these will benefit first from collective agreement on such questions as why student mobility matters and the nature of the improvements to credit transfer the system should aim to achieve, and—only once these have been asked—what policies and aligned mechanisms and activities should be pursued to accomplish these goals. Ontario will be well served by an examination of the implications and feasibility of credit models that include other educational input measures, while also pursuing continued exploration of the theoretical and practical aspects of using learning outcomes as a tool for credit transfer (see chapters 6, 7, 8, 9, and 10). 2. The leveraging of learning outcomes for credit transfer cannot be accomplished uniformly or quickly but must be built over time through purposeful policy work and sustained implementation efforts at all levels. Policy learning is preferable to policy borrowing (see chapter 8), and it is an activity that must flow from the careful weighing of possibilities rather than from the rapid adoption of solutions that others have used. It is unlikely that one jurisdiction can fully model itself on another, be-

Summary of Findings and Recommendations 293

cause each will have a unique set of circumstances, constraints, and drivers (see chapters 8 and 9). Like any other jurisdiction, Ontario will need to address any differences between the intrinsic logic of an intended intervention and the institutional logics that might influence or mitigate the successful implementation of that solution (see chapter 8). Attending carefully to the contextual differences of jurisdictions takes time and concentrated effort. Allais’s (see chapter 3) account of the highly problematic efforts to revolutionize the South African postsecondary system through the rapid, extreme, and top-down application of borrowed learning outcomes frameworks, tools, and policies lends particular caution. Similarly, Noonan’s account of the undesirable consequences of the introduction of competency-based training modules in the Australian vocational education and training (VET) sector for credit transfer and mobility gives credence to the notion that such policy reform efforts demand greater proactive weighing of possibilities and attention to possible consequences. Noonan (see chapter 5) points out that the benefits of credit transfer will only be realized if “a more consistent and coherent approach to defining, developing, and assessing learning outcomes between the higher education and VET sectors is developed, understood, and widely used” (136). In general, the quick and radical implementations of reform efforts involving learning outcomes and credit transfer have not proven to be productive. In contrast, Wagenaar’s (see chapter 6) exploration of the lengthy, focused, and broadly inclusive efforts of the European Union to employ learning outcomes in the incremental and evolutionary development of an integrated European system of higher education offers greater promise for establishing sustainable structures that have resulted in transparent, fair, and consistent pathways and credit transfer arrangements for students. Reform that leverages the potential of learning outcomes to improve credit transfer will require, as has been argued in chapter 9, clarity of policy purposes and a conscious fit between context, purpose, and activities constructed on a foundation of careful and inclusive deliberation. Ontario will benefit from a committed effort to define shared provincial transfer goals as well as recognition of the structural and other barriers that impede student mobility. Problems and attendant solutions related to the use of learning outcomes as a tool to improve credit transfer should continue to be examined based upon research and broad consultation, followed by the establishment of relevant and clear policy objectives that consider all stakeholders. Such work obviously cannot be completed overnight, as it requires careful

294

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

consideration of local circumstances and evolving conceptualizations of qualifications and credentials. We must also contend with differences in teaching, learning, and assessment across fields of academic and vocational study, as well as the complexities of attitudes and capacities of diverse actors throughout the system (see chapters 3, 5, 6, and 8). Ontario’s postsecondary education system has evolved largely through incremental change influenced over time by changes in governing parties, as well as policy priorities that are often contingent on shifting financial circumstances. Incremental change has many virtues, not the least of which is the avoidance of radical steps and a respect for historic precedents. Such caution, however, may lead to weak and ambiguous policy that can be exploited by institutions and stakeholders. Ontario, while maintaining a steady and long-term commitment to the improvement of student equity, access, and mobility, as well as efficiency and quality, has not sought to take bold steps to enhance credit transfer and mobility. Instead, the province has focused its efforts largely on project-based activities of arm’s-length bodies and institutions, while pursuing modest government policies such as the inclusion of learning outcomes in new program proposals and strategic mandate agreements. Nevertheless, lessons garnered from other jurisdictions reviewed in this publication point to the virtue of extending such already established commitments to ever more ambitious efforts to further advance credit transfer and pathway opportunities for students. Consistent government policy and commitment to advancing the system of credit transfer is warranted and welcome. However, in Ontario, incremental and conflicting approaches have prevented the establishment of clear goals regarding student mobility and led to inconsistent policy approaches to credit transfer and student mobility. An example of a problematic policy tool is evident in the shortcomings of the Ontario Qualifications Framework (OQF). Unlike other qualifications frameworks, it designates institutional type with credentials. As well, it utilizes UDLEs as the basis of comparison for credentials rather than also referencing college program standards, which include vocational learning outcomes, essential employability skills, and general education requirements. This privileging of university learning outcomes and orphaning of college diploma program standards in the credentials framework overlooks how fundamental program standards have become to the organization of colleges and feeds an odd disjuncture between guiding qualifications documents for the colleges and universities. One of the motivating purposes for permitting the Ontario

Summary of Findings and Recommendations 295

college sector to grant honours baccalaureate degrees was to address difficulties in establishing good transfer pathways into degrees for diploma holders (Skolnik 2012), yet college degree students continue to encounter transfer and progression barriers between vocationally and academically focused programming, stemming in part from system differentiation policies that seek to distinguish applied, vocational, and academic education. These conflicting approaches have sent deeply mixed signals about the system, and have set up unnecessary impediments to credit transfer while perpetuating differentials of esteem between colleges and universities. These deficiencies, as suggested in chapters 1 and 9, can be ameliorated through further development of the OQF to use outcomes that enable linkages between educational sectors, reflect all credentials, and better integrate credit transfer. There exists an opportunity now to recast the OQF as that which Raffe (2011) refers to as a communications framework that “seeks to bring about change indirectly, by providing tools to support incremental reforms and by making the system transparent and thereby facilitating rationalization and coherence” (283). Such efforts will necessitate resolution as to whether the Ontario postsecondary education system should revert to a tracked model or continue to seek opportunities to become more linked and loose over time, thus clarifying the logic by which both universities and colleges grant a range of credentials within the system (see chapter 9). Continued support for the work of the Ontario Council on Articulation and Transfer (ONCAT) and the ongoing engagement of its member institutions in advancing research, policy, and practice on credit transfer and pathways will be essential to maintaining Ontario’s momentum in improving our system of credit transfer. Critical factors in the sustainability of reform reside in the continued pursuit of platforms that support research and the exploration of concepts and practices that expand our capacity to use outcomes and other measures to determine credit transfer. However, broader and more all-encompassing initiatives are needed as well. Sustainability for both discrete case studies and more comprehensive initiatives will require deliberate and earnest efforts to build trust and openness on many levels. This includes trust across social and economic spheres in the quality and nature of credentials (see chapters 6 and 9), trust and understanding between institutions of higher education, and openness among those responsible for the establishment and achievement of system goals and policies. The development of trust is conducive to

296

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

drawing on a broad base of research and practical experience and to leveraging guiding insights into student learning (see chapters 3, 4, 6, 7, 8, and 10), patterns of desire and mobility (see chapters 1, 4, 9, and 10), labour market demand and gaps (see chapters 3, 5, 6, 8, 9, and 10), and organizational and administrative practices, constraints, and opportunities (see chapters 3, 4, 5, 8, 9, and 10). This trust-building takes time and focus and involves enabling policies and forums for exchange and information-sharing, as well as training all actors to achieve coherence and alignment in the use of learning outcomes that span many boundaries—particularly those pertaining to educational and occupational sectors (see chapters 3, 6, and 10). Financial and moral incentives and opportunities for engaging in this work will benefit from being combined and consistently maintained in the search for improvements on behalf of Ontario’s students. 3. A wide diversity of stakeholders must be sustainably engaged and supported in the effective understanding and use of learning outcomes and credit transfer. Postsecondary education, like all other complex systems, is an assemblage of interconnected entities, actors, principles, policies, and processes that form an evolving but unitary whole. Chapters 9 and 10 make it clear that in postsecondary education there are many stakeholders, with a diversity of perspectives, needs, and ambitions. A well-functioning system finds ways to sustainably engage and support representatives of every element of the system in developing common goals, as well as collective understandings and practices. The coherent use of learning outcomes is among the many and most complex challenges for a postsecondary educational system. Hence, it is necessary to establish mechanisms within which to share meaningfully structured information, forums for ongoing collaboration, and means for continued training that can make the use of learning outcomes a truly collective endeavour. Commentary provided in chapters 9 and 10 indicates that it is not rational to assume that the actors who operate in the postsecondary education sector will be capable of intuiting or gleaning all that matters of the intricacies and complexities of learning outcomes, and their potential contributions to credit transfer and pathways development, without the benefit of ongoing support and training and the presence of vibrant networks. The importance of professional development and training opportunities for policymakers, administrators, advisors, and faculty

Summary of Findings and Recommendations 297

members has been discussed by Allais, Bragg, Noonan, and Wagenaar (see chapters 3, 4, 5, and 6). To make this possible, the province of Ontario would be well advised to engage systematically with the vast array of actors who are compartmentalized in policy leadership, system design, accountability frameworks, institutional leadership and administration, scholarship, and teaching to build structures for ongoing exchange and collective activity. Wagenaar (see chapter 6) provides a glimpse into the decades of methodical, collective, and consultative work of European commissions, joint ministerial declarations, action programs, and pilots to test and ultimately establish internationally agreed-upon principles of trust, transparency, and combined use of credit and learning outcomes as measures of student achievement. This work engaged critical processes such as Tuning and the development of tools such as consistently formatted course catalogues and transcripts that led to the Erasmus Mobility Scheme and ultimately the ECTS (see chapter 6). The European Union’s long-term approach to structured reform engaged diverse actors within and across the boundaries of national systems, and endures today in the quest for continuous improvement embodied in the Measuring and Comparing Achievements of Learning Outcomes in Higher Education in Europe (CALOHEE) initiative (see chapters 1, 6, 9, and 10). In the United States, Bragg (see chapter 4) also calls attention to the value of engaging diverse actors and taking the time to build trust relationships, while reminding us that current research indicates that some factors “that appear to contribute to institutional pairings…that yield favourable results are top leadership commitment to transfer, faculty and staff engagement and support for various aspects of the transfer function, and close geographic proximity of institutions” (105). Ontario can exploit similarly favourable factors by challenging assumptions regarding the silos within which these many actors work and by building better bridges between them to support improved understanding of the myriad of system-, institution-, and program-level considerations that must be taken into account when establishing enhanced transfer opportunities. Engaging a diversity of actors over time in the evolution and use of learning outcomes for management of learning, delivery of learning, and credit transfer is conducive to building shared principles and practices, as well as more clearly identifying the tensions to be addressed. As individuals transition into and out of roles in the system and new methods and tools are explored, a healthy system will necessitate ongoing and constant engagement, training, and support.

298

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

ONCAT plays an essential role in such work, and focuses its efforts on • supporting projects aimed at creating and expanding credit transfer pathways; • enabling research that improves understanding of the student transfer experience; • developing better credit transfer data collection and management systems; • hosting annual Student Pathways in Education conferences that bring together credit transfer experts to share and promote best practices; and • connecting students to transfer opportunities through ONTransfer.ca. (Ontario Council on Articulation and Transfer, n.d.) ONCAT has established a range of mechanisms through which those engaged in transfer work in the province of Ontario connect and learn from one another and conduct research into policies and practices. Among these initiatives are discipline-focused transfer committees, a transfer advisor group, a student transfer expert panel, a learning outcomes advisory group, and a faculty fellows program. These are important undertakings for the province, but Ontario would also benefit from the provision of centrally organized and structured professional development opportunities that offer education in the theoretical foundations of student transfer and mobility, as well as its intersections with outcomes-based credential models, so as to build greater appreciation amongst those using learning outcomes. Similarly, educational opportunities focused on system-, institutional-, and role-specific policies and practices, tools, and frameworks and quality assurance mechanisms that permit and limit credit transfer in Ontario are necessary. Institutions, professional and disciplinary associations and networks, and individuals also have a part to play in instigating and perpetuating professional development opportunities and focused work on the advancement of research into learning outcomes as a tool to inform transfer. 4. Recognize the differences in construction and progression of knowledge between disciplines and fields, as well as vocational and academic education, and factor this understanding into how outcomes are used in credit transfer to enable spanning of boundaries. There are systemic divisions between sectors, disciplines, and fields

Summary of Findings and Recommendations 299

that exist in postsecondary education, and we must acknowledge what they are and how they influence and affect each other. As Noonan (see chapter 5) cautions, “the differences between the sectors in relation to qualification design and assessment are long-standing” (117), and we would be ill-advised to ignore this history. Many in this publication have stressed the fact that there are differences between how learning outcomes are formulated within and across the vocations and in academic fields that hinder how outcomes can be used for progression and boundary spanning. This should not be underestimated or oversimplified. Even highly similar and aligned programs of study can be structured very differently in the ordering of courses or the depth, breadth, approach, or progression of knowledge, skills, and habits of mind cultivated intentionally term over term. It is a puzzle, and one that often complicates credit transfer for students as they transition from one program of study to another. Students who lack adequate prior experience and understanding of knowledge, skills, and habits of mind essential for successful progression within a program of study may struggle unduly and find themselves getting behind in their coursework following transfer. While this is not likely to be entirely preventable, students will benefit from efforts to build and illuminate pathways for their progression with as much detail as possible to aid in their navigation and avoid unbridgeable gaps. Comprehensive qualifications frameworks that feature learning outcomes and level descriptors for relating and differentiating credentials can help to identify how students can move between qualifications successfully. However, as Winch (see chapter 7) has noted, “the permeability of qualifications remains restricted. A qualification is permeable if it has characteristics that allow its holder to progress to the achievement of a qualification placed higher on a cognitive hierarchy scale.” (169) and because learning outcomes statements as defined within programs of study are typically intended to be independent, specifiable, and hierarchical. For this purpose, the use of range statements—which can aid in describing the scope, or contexts and circumstances, in which a student who has successfully achieved a learning outcome should be able to apply their knowledge or skill—can add meaningful detail that can help sending and receiving institutions to understand how a given learning outcome is situated within a program of study. Ontario may benefit from further investigating the possible application and utility of range statements. Such efforts may be linked to Tuning of disciplines and occupational fields that are not already subject to professional

300

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

regulation to define necessary reference points and levels of progression, something that has considerable precedent in Europe, the United States, and elsewhere (see chapters 4 and 7). Similarly, strategies for examining and comparing the assessment of learning outcomes—the rigour, application, sites, and complexity of assessments in particular—may assist in building an understanding of the basis of credit transfer for learners. Such analysis may help to elucidate where student learning may reside along a continuum of epistemic ascent, as described by Winch (see chapter 7), “from fragmented propositional knowledge and knowledge by acquaintance to a more holistic, inferential grasp of subject matter, to an appreciation of the work involved in replicating and validating existing knowledge, to an appreciation of the work involved in creating new knowledge and rejecting older, previously accepted claims. Parallel to this is a progression in know-how from skill to transversal abilities to project management abilities.” (187–188). Unearthing and documenting such patterns would logically contribute to the development of more robust credit transfer opportunities. As such, a more nuanced appreciation of learning progression may also suggest the “decomposing an overall qualification into modules or even sub-modules” (Winch, see chapter 7, 177) to enable transfer with part-qualifications. 5. Learning outcomes must be authentic to and embodied in teaching and learning processes and not externally imposed or narrowly specified. System-generated outcomes, competencies, and standards will not adequately serve the Ontario system if they reflect only the expediencies of top-down practices, workplace requirements, or highly specified definitions of atomized performances of competencies within narrowly defined vocations. If there is to be an alignment between system-wide classification of qualifications, definition of standards and program curriculum, and course expectations, outcomes must be as authentic and as meaningful in practice as they are in theory. Outcomes must deeply connect to the learning process and have relevance to fields of study. This conceptualization of learning outcomes is best achieved when the outcomes are generated and embedded in context, within “a body of knowledge, skills and expertise, or a field of learning” (Allais, see chapter 3, 72) and developed by educators active in the discipline or profession, working in concert, and with a keen awareness of what there is to be learned and how all engaged students can learn successfully. Learning outcomes evolve as disciplines and vocations do, and

Summary of Findings and Recommendations 301

are animated by curricula and derive their meaning and value in relation to the program of study and its contextual realization: “without a curriculum, it is not possible to obtain an understanding of how elements of know-how and propositional knowledge fit together into a coherent whole” (Winch, see chapter 7, 173). Wagenaar’s explanation of the Tuning process that underpins the ECTS and related investigation of Tuning by the Higher Education Quality Council of Ontario (HEQCO; see chapter 10) may provide a useful model for how Ontario academics and those closest to teaching and learning can collectively map and recommend competencies and outcomes to be achieved in particular disciplines and occupational fields so as to improve the comparability of learning in programs. The aim was that through “a common—well-accepted—methodology for program design, delivery, and enhancement, as well as a shared set of documents in place as reference points… recognition of studies would be significantly improved” (Wagenaar, see chapter 6, 148). The process and engagement of faculty in trust-building that emerges from Tuning in the European context holds as much promise for Ontario. We do not advise the abstraction of learning outcomes from the situation of their cultivation and acquisition or from their direct relevance to the field of study if they are to serve as a reliable and meaningful tool for informing credit transfer decisions. Nor would we suggest that Tuning processes or the collaborative development of range statements are one-shot operations. Vocations and disciplines are not static and fixed, so logically, any consideration of outcome statements for the purpose of informing credit transfer must also remain dynamic and iterative. Logically, learning outcomes that are imposed by external authorities, such as governmental or employer agencies, and are too narrowly specified as fixed, detailed standards of competencies that are to be achieved within structured packages of education, are flawed (see chapters 3, 5, and 7). Learning outcomes which do not derive, in some significant ways, from the expert knowledge and experience of faculty members who teach, research and practice in the field are less likely to be embraced and implemented by educators—particularly if they do not resonate meaningfully with the priorities of educators and their lived experiences of the classroom. In the South African experience, we find evidence of the disaffection of educators that resulted from the adoption of “a cosmetic process” (Allais, see chapter 3, 61) by which institutions inserted learning outcomes into their program curricula as a compliance activity. Instructors were also ill-prepared and ill-sup-

302

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

ported in subsequent efforts to inculcate learning outcomes statements into curriculum, and were not provided with adequate time to interpret and think deeply about how best to weave in the received learning outcomes. Ultimately, the effort to establish outcomes-based qualifications in South Africa was ill-planned, proved impractical, and failed, leaving any subsequent opportunity to leverage learning outcomes as a tool to inform credit transfer at an impasse. Wagenaar (see chapter 6) also acknowledges difficulties in the European Union that stem, in his view, from lack of faculty commitment, engagement, and training in the use of learning outcomes, despite efforts to inject collaboration and choice in the process. Faculty resistance to the use of outcomes has at times hampered implementation of the ECTS. These experiences form cautionary tales for Ontario and should serve to maintain and deepen efforts to invest time, resources, and expertise in the development of well-trained, interprofessional postsecondary education communities committed to trust-building, knowledge-building, and the advancement of multifaceted approaches to improving credit transfer that draw upon learning outcomes as one meaning-making source of evidence of curriculum intent and student learning among others. Conclusion This publication, particularly in chapters 2, 9, and 10, has delineated how Ontario has built a postsecondary education system upon a foundation that supports optimal access, equity, and accountability, and where learning excellence is also a priority. The commentary provided in these chapters has outlined what has been accomplished over time in terms of strengthening the system and adapting it to enable colleges and universities to structure and share a qualifications framework, and to evolve mechanisms for quality assurance that support student-centred curriculum and pedagogies that rely on learning outcomes and, at the same time, guide credit transfer. Previous chapters have also outlined the development of diverse, if ad hoc, means and models for transfer of credit between institutions and within and between the two sectors. Many of the recent advances in credit transfer and the use of learning outcomes rest heavily upon the work of ONCAT and HEQCO to provide analysis of data and exploration of concepts. These agencies have also supported examination of policy options and experimentation in practical mechanisms that directly facilitate credit transfer. Taken as a whole, this work has expand-

Summary of Findings and Recommendations 303

ed our understanding of how the postsecondary education system can deliver the flexibility in credit transfer necessary to support student mobility and success in a social and economic environment that demands lifelong learning. Significant work remains to be done to draw together the many strands of these activities and initiatives so that they may inform a collectively determined path forward. The time is ripe for a structured review of how and in what way Ontario will position its postsecondary sectors and institutions in order to offer more effective and transparent ways for students to move more seamlessly between sectors and programs, and to complete or accumulate new credentials that will enable them to gain greater access to evolving work opportunities. Improvement to student mobility calls for greater intention and clarity regarding the extent to which colleges are enabled to grant degrees, as well as continued investment in the development of transfer arrangements that span college and university sectors (Skolnik 2012). A coming together of many actors throughout the system is necessary to define clear goals. Ontario would benefit from the pursuit of focused, coherent reforms that transcend isolated, one-off increments of change and are capable of buffering the system from the temptations of radical or poorly conceptualized or formulated initiatives. Such clarity and principled commitment will enable Ontario to define and take future next steps that fully leverage the work in which it has so generously invested in the hopes of creating an optimal system of credit transfer. Many of these steps will build on the groundwork that has already been established to weigh new options, run scenarios, and consider implications in the process of determining the components necessary for a policy-rich environment capable of supporting structures and interrelationships that buttress a robust system of credit accumulation and recognition. This means building a broad-based and deeply engaged community around reform activities that can reconcile the intrinsic logic that frames and structures the larger system with the on-the-ground institutional logic necessary for pedagogical and administrative implementation and practice. It means making collectively informed decisions and building consistently toward a future where students, not systems, are the priority.

304

Christine Arnold, Mary Wilson, Jean Bridge, and Mary Catharine Lennon

References Ontario Council on Articulation and Transfer. n.d. Welcome to the Ontario Council on Articulation and Transfer. http://www.oncat.ca/ Raffe, D. 2011. “Are ‘Communications Frameworks’ More Successful? Policy Learning from the Scottish Credit and Qualifications Framework.” Journal of Education and Work, 24 (3–4): 283–302. Remington, S. 2013. Sustainable World: Approaches to Analyzing and Resolving Wicked Problems. Dubuque, IA: Kendall Hunt. https:// he.kendallhunt.com/sites/default/files/uploadedFiles/Kendall_Hunt/Content/Higher_Education/Uploads/Remington_Doucette_1e_Ch2.pdf Skolnik, M. L. 2012. College Baccalaureate Degrees and the Diversification of Baccalaureate Production in Ontario. https://www.researchgate. net/publication/267685148_College_Baccalaureate_Degrees_and_ the_Diversification_Of_Baccalaureate_Production_in_Ontario