Best Practices for Administering Online Programs [1 ed.] 2020019620, 2020019621, 9780367349738, 9780367349745, 9780429329081

Table of contents :
Cover
Half Title
Series Page
Title Page
Copyright Page
Dedication
Contents
Acknowledgments
List of
Figures and Tables
Introduction
Faculty
Administrators
Designers
Support Staff
About This Book
Chapter 1: Getting Started Online
Reasons for Creating Online Programs
Making the Case for Developing an Online Program
Opportunities and Challenges of Online Education
Advantages of Online Technology in Face-to-Face and Blended Courses
Defining the Program and Target Audience
Assessing Program Demand
Overview of the Course Development Process
The Benefits of Having a Digital Strategy
Identifying Faculty
Choosing a Learning Management System
Summary Checklist
References
Chapter 2: Faculty Considerations for Building an Online Program
Helping Faculty Transition to Teaching Online
Preparing Faculty to Develop Online Courses
Mentoring Faculty to Teach Online
Faculty Compensation Models
The Benefits of Designating Faculty Coordinators
The Benefits of Employing Course Facilitators
Recruiting, Developing, and Evaluating Facilitators
Summary Checklist
References
Chapter 3: Issues That Online Courses Must Address
Choosing Technologies
Fostering Academic Integrity
Scaling Online Courses
Accommodating Students with Disabilities
Marketing Online Programs
Developing Marketing Collateral
Student Data Security and Privacy
Summary Checklist
References
Chapter 4: Program Definition and Development
Defining Program Goals
Selecting a Course Scheduling Model
Selecting a Course Development Model
Organizational Design to Support Faculty Who Teach Online
Selecting Modes of Instruction
Interaction Models
Selecting Synchronous Teaching Technology
Developing a Marketing Plan
Enrollment Forecasting
Summary Checklist
References
Chapter 5: Program Administration
Hiring and Training Enrollment Advisors
Processing Admissions
Developing a Recruitment Pipeline
Developing a Student Retention Process
Defining and Administering Contact Hours
Defining and Administering Course Standards
Identifying Lower-Cost Actions to Improve Program Quality
Identifying Moderate-Cost Actions to Improve Program Quality
Identifying Higher-Cost Actions to Improve Program Quality
Evaluating the Program
Updating Marketing Collateral
Managing the Program throughout Its Lifecycle
Summary Checklist
References
Chapter 6: Fundamentals of Course Design and Development
Starting a New Course
Using Design Standards and Award Rubrics
Designing for Meaningful Interaction
Defining Goals and Learning Objectives
Creating a Course Map
Creating Assessments
The Roles of Assessments in Courses
Planning Learning Activities
Designing for Students with Varied Preparation
Aligning the Content by Topic
Leveling Workload
Developing and Implementing Content
Making Videos
Pre-launch Tasks
Addressing Delayed Development
Building Courses to Facilitate Updates
Summary Checklist
References
Chapter 7: Advanced Course Design
Hiring Designers, Support Staff, and Videographers
Planning Course Development
Holding a Development Kickoff Meeting
Motivating Faculty
Building for Universal Design
Avoiding Multiple Copies of Content
Deciding What Technologies to Use
Synchronous Sessions
Telepresence
Integrating Multimedia Content, Simulators, and Virtual Labs
Managing Production Tasks
Copy-editing
Pre-launch Designer Tasks
Performing Faculty QA
Final Preparation for Launch
Summary Checklist
References
Chapter 8: When a Course Is Underway
Guiding Students’ Course Preparation
Support Staff Responsibilities
QA Tasks for Support Staff
Pre-launch Tasks for Support Staff
Course Launch Kickoff Meeting
Weekly Faculty, Facilitator, and Support Staff Meetings
Setting Up Student Groups
Orientation Course
Welcome Messages
Administering Email and Announcements
Tasks Performed by Support Staff While a Course Is Running
Contacting Students Who Have Not Logged In
Building a Learning Community
Regularly Scheduled Synchronous Sessions
Outreach Summarizing Prior and Coming Weeks
Study Groups
Calling Students
Students at Risk
Wrapping Up
Administering Proctored Exams
Winding Down the Course
Building Community Beyond the Classroom
Summary Checklist
References
Chapter 9: Addressing Unexpected Developments
Technical Problems and Technical Support
LMS Outage or Slowness
Faculty or Facilitator Illness or Absence
Course Design or Implementation Problems
Students Having the Wrong Textbook or No Textbook
Students in Crisis
Challenging Unexpectedly Strong Students
Helping Underprepared Students
Students Having Difficulty with Assignments or Assessments
Student Complaints
Late Submissions
Content Errors or Omissions
Misalignment of Course Content
Uneven Student Workload
Compromised Academic Integrity
Summary Checklist
References
Chapter 10: Administering Course Revisions
Planning Course Updates
Estimating Course Revision Effort
Course and Faculty Teaching Evaluations
Curriculum Review
Software Updates, New Technologies, and Pedagogy Advancements
Administering Asynchronous Discussion Revisions
Internal and External Reviews
Outcome Analysis
Program Evaluation
Summary Checklist
References
Glossary
Index

Citation preview

Best Practices for Administering Online Programs

Best Practices for Administering Online Programs is a practical volume for university teams seeking to manage effective online programs. Defining, designing, implementing, and updating online courses is a highly collaborative effort, particularly with limited resources and expanding student enrollment. This book unites the efforts of program directors, supervisors, department chairs, participating faculty, instructional designers, IT specialists, and support staff toward a common goal: affordable, accessible, and scalable online learning. Readers will find guidelines for fostering quality, faculty skills, academic integrity, learning objectives, course improvement, and more. Daniel Hillman is Associate Director of Instructional Design in the Office of Distance Education at Metropolitan College of Boston University, USA. Robert Schudy is Associate Professor Emeritus of Computer Science at Metropolitan College of Boston University, USA. Anatoly Temkin is Chair of the Computer Science Department at Metropolitan College of Boston University, USA.

Best Practices in Online Teaching and Learning Series Editor Susan Ko

For a full list of titles in this series, please visit: https://www.routledge.com/ Best-Practices-in-Online-Teaching-and-Learning/book-series/BPOTL Best Practices for Teaching with Emerging Technologies by Michelle Pacansky-Brock Best Practices in Online Program Development: Teaching and Learning in Higher Education by Elliot King and Neil Alperstein Best Practices for Flipping the College Classroom edited by Julee B. Waldrop and Melody A. Bowdon Best Practices in Engaging Online Learners through Active and Experiential Learning Strategies by Stephanie Smith Budhai and Ke’Anna Brown Skipwith Best Practices for Teaching with Emerging Technologies, 2e by Michelle Pacansky-Brock Best Practices in Planning Strategically for Online Educational Programs by Elliott King and Neil Alperstein Best Practices in Designing Courses with Open Educational Resources by Olena Zhadko and Susan Ko Best Practices for Administering Online Programs by Daniel Hillman, Robert Schudy, and Anatoly Temkin

Best Practices for Administering Online Programs

Daniel Hillman Robert Schudy Anatoly Temkin

First published 2021 by Routledge 52 Vanderbilt Avenue, New York, NY 10017 and by Routledge 2 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN Routledge is an imprint of the Taylor & Francis Group, an informa business © 2021 Taylor & Francis The right of Daniel Hillman, Robert Schudy, and Anatoly Temkin to be identified as authors of this work has been asserted by them in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this book may be reprinted or reproduced or utilized in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging-in-Publication Data Names: Hillman, Daniel, author. | Schudy, Robert, author. | Temkin, Anatoly, author. Title: Best practices for administering online programs / Daniel Hillman, Robert Schudy, Anatoly Temkin. Description: New York, NY : Routledge, 2021. | Series: Best practices in online teaching and learning | Includes bibliographical references and index. Identifiers: LCCN 2020019620 (print) | LCCN 2020019621 (ebook) | ISBN 9780367349738 (hardback) | ISBN 9780367349745 (paperback) | ISBN 9780429329081 (ebook) Subjects: LCSH: Web-based instruction--Study and teaching. | Web-based instruction--Design. Classification: LCC LB1044.87 .H56 2021 (print) | LCC LB1044.87 (ebook) | DDC 371.33/44678--dc23 LC record available at https://lccn.loc.gov/2020019620 LC ebook record available at https://lccn.loc.gov/2020019621 ISBN: 978-0-367-34973-8 (hbk) ISBN: 978-0-367-34974-5 (pbk) ISBN: 978-0-429-32908-1 (ebk) Typeset in Bembo by SPi Global, India

To our families and colleagues

Contents

Acknowledgments List of Figures and Tables Introduction

xii xiii 1

Faculty  1 Administrators  1 Designers  2 Support Staff  2 About This Book  3 1 Getting Started Online

5

Reasons for Creating Online Programs  5 Making the Case for Developing an Online Program  6 Opportunities and Challenges of Online Education  7 Advantages of Online Technology in Face-to-Face and Blended Courses  8 Defining the Program and Target Audience  9 Assessing Program Demand  11 Overview of the Course Development Process  12 The Benefits of Having a Digital Strategy  13 Identifying Faculty  14 Choosing a Learning Management System  15 Summary Checklist  17 References  18 2 Faculty Considerations for Building an Online Program Helping Faculty Transition to Teaching Online  19 Preparing Faculty to Develop Online Courses  22

19

viii Contents

Mentoring Faculty to Teach Online  24 Faculty Compensation Models  26 The Benefits of Designating Faculty Coordinators  27 The Benefits of Employing Course Facilitators  28 Recruiting, Developing, and Evaluating Facilitators  29 Summary Checklist  30 References  31 3 Issues That Online Courses Must Address

32

Choosing Technologies  32 Fostering Academic Integrity  34 Scaling Online Courses  38 Accommodating Students with Disabilities  41 Marketing Online Programs  42 Developing Marketing Collateral  47 Student Data Security and Privacy  49 Summary Checklist  49 References  49 4 Program Definition and Development

51

Defining Program Goals  51 Selecting a Course Scheduling Model  52 Selecting a Course Development Model  55 Organizational Design to Support Faculty Who Teach Online  57 Selecting Modes of Instruction  58 Interaction Models  60 Selecting Synchronous Teaching Technology  62 Developing a Marketing Plan  62 Enrollment Forecasting  63 Summary Checklist  65 References  65 5

Program Administration Hiring and Training Enrollment Advisors  66 Processing Admissions  67 Developing a Recruitment Pipeline  71 Developing a Student Retention Process  72 Defining and Administering Contact Hours  72

66

Contents ix

Defining and Administering Course Standards  74 Identifying Lower-Cost Actions to Improve Program Quality  76 Identifying Moderate-Cost Actions to Improve Program Quality  77 Identifying Higher-Cost Actions to Improve Program Quality  78 Evaluating the Program  79 Updating Marketing Collateral  80 Managing the Program throughout Its Lifecycle  80 Summary Checklist  83 References  83 6 Fundamentals of Course Design and Development

84

Starting a New Course  85 Using Design Standards and Award Rubrics  85 Designing for Meaningful Interaction  86 Defining Goals and Learning Objectives  90 Creating a Course Map  91 Creating Assessments  94 The Roles of Assessments in Courses  95 Planning Learning Activities  100 Designing for Students with Varied Preparation  102 Aligning the Content by Topic  103 Leveling Workload  104 Developing and Implementing Content  105 Making Videos  106 Pre-launch Tasks  107 Addressing Delayed Development  108 Building Courses to Facilitate Updates  109 Summary Checklist  112 References  113 7 Advanced Course Design Hiring Designers, Support Staff, and Videographers  115 Planning Course Development  119 Holding a Development Kickoff Meeting  121 Motivating Faculty  121 Building for Universal Design  124 Avoiding Multiple Copies of Content  126 Deciding What Technologies to Use  127

114

x Contents

Synchronous Sessions  133 Telepresence  136 Integrating Multimedia Content, Simulators, and Virtual Labs  137 Managing Production Tasks  139 Copy-editing  141 Pre-launch Designer Tasks  142 Performing Faculty QA  143 Final Preparation for Launch  143 Summary Checklist  143 References  144 8 When a Course Is Underway

145

Guiding Students’ Course Preparation  146 Support Staff Responsibilities  147 QA Tasks for Support Staff  148 Pre-launch Tasks for Support Staff  149 Course Launch Kickoff Meeting  150 Weekly Faculty, Facilitator, and Support Staff Meetings  151 Setting Up Student Groups  151 Orientation Course  152 Welcome Messages  152 Administering Email and Announcements  153 Tasks Performed by Support Staff While a Course Is Running  154 Contacting Students Who Have Not Logged In  155 Building a Learning Community  156 Regularly Scheduled Synchronous Sessions  156 Outreach Summarizing Prior and Coming Weeks  158 Study Groups  158 Calling Students  159 Students at Risk  159 Wrapping Up  160 Administering Proctored Exams  161 Winding Down the Course  163 Building Community Beyond the Classroom  164 Summary Checklist  164 References  165 9 Addressing Unexpected Developments Technical Problems and Technical Support  167 LMS Outage or Slowness  168

166

Contents xi

Faculty or Facilitator Illness or Absence  168 Course Design or Implementation Problems  169 Students Having the Wrong Textbook or No Textbook  169 Students in Crisis  170 Challenging Unexpectedly Strong Students  170 Helping Underprepared Students  171 Students Having Difficulty with Assignments or Assessments  172 Student Complaints  173 Late Submissions  174 Content Errors or Omissions  174 Misalignment of Course Content  175 Uneven Student Workload  175 Compromised Academic Integrity  176 Summary Checklist  176 References  177 10 Administering Course Revisions

178

Planning Course Updates  178 Estimating Course Revision Effort  180 Course and Faculty Teaching Evaluations  182 Curriculum Review  184 Software Updates, New Technologies, and Pedagogy Advancements  185 Administering Asynchronous Discussion Revisions  185 Internal and External Reviews  185 Outcome Analysis  186 Program Evaluation  187 Summary Checklist  187 References 188 Glossary Index

189 197

Acknowledgments

We wish to thank the following people: Claudia HCQ Sorsby, for her comprehensive and thoughtful reviews of all drafts of this book, and for pointing out instances where what we wrote was not what we meant. Harold Zacks, for asking the questions that inspired us. The leadership of Metropolitan College, for initiating and supporting our online programs: former Dean Jay Halfond, Dean Tanya Zlateva, and Associate Dean Lou Chitkushev. Susan Ko for her advice and for including us in the Best Practices series. Dan Schwartz at Routledge for guiding us through the editorial process. Thanks to the following people who provided us with their thoughtful comments: Andrew Abrahamson, Thomas Cavanagh, Jackie Corcoran, Susan Diesel, Jim Frey, Elena Garofoli, Summer Garrard, Robert Haley, Alyssa Kariofyllis, Lawrence Ragan, Fawn Thompson, Ed Wozniak, and Jenny Young.

Figures and Tables

Figures 4.1 Schedules with different models 6.1 Test Yourself report for multiple-answer question 6.2 Inline formative assessment displaying correct answers and feedback 7.1 Technical knowledge and diplomacy ratios 7.2 Example development schedule of a seven-week course

53 98 111 115 120

Tables 5.1 5.2 6.1 7.1

Transition course decisions Goals and standards Example module map Implementing content with and without designers and editors

70 76 93 120

Introduction

We wrote this book to provide administrators, faculty, designers, and support staff with practical guidance for creating and running successful online programs. This book describes what is involved in administering every stage of developing and running an online program, from conceptual definition to revision. Many aspects of the administration of online programs are not obvious. We have observed programs that failed because program administrators did not foresee and address critical risks. We have distilled our experience and the experience of others into pragmatic advice. This book will help everyone involved in online education understand the process of creating and teaching successful programs. To describe the processes and responsibilities, we have identified four roles: administrator, faculty, designer, and support. One person may perform more than one role; for example, a faculty member may also be an administrator. In addition, more than one person may perform the same role; for example, more than one faculty may develop and teach a course. Over the course of the book, we demonstrate how people in the different roles work together as a team. Our definitions of the roles are as follows.

Faculty Faculty create and teach courses. They have expertise in the subject matter and its pedagogy, but they may not know much about designing and developing online courses, the support needed when they are running, or their administration. Reading this book will help faculty successfully develop and teach online courses.

Administrators Administrators here refer to those directly involved in supervising programs. This can include program directors and administrators, as well as department chairs. Primary responsibilities of administrators include developing the

2 Introduction

organizational structure and providing the resources needed to develop and deliver programs. Program- and department-level administrators are usually faculty. At primary and secondary schools, administrators may include head teachers, department chairs, headmasters, and principals. We do not explicitly address the governance roles played by high-level administrators, such as chancellors, provosts, presidents, superintendents, or school boards, although much of the material in this book will help them understand what is needed to support program-level administrators. Reading this book will help administrators understand the resources required, gain knowledge of successful practices in program and course development, and learn how to address the unique challenges in bringing traditional face-to-face programs online.

Designers Designers help faculty implement course content and pedagogic ideas. In this book, they include everyone who helps organize, guide, and perform course development, such as instructional designers, videographers, illustrators, animators, and editors. At most institutions, designers act as faculty technical support and trainers, assisting them with the technical knowledge and skills to put their courses online. We describe what needs to be done by whoever is covering that role, even if an institution may not have people dedicated to the role. Reading this book will help designers understand administrative, faculty, and support roles of developing and running programs, and how they enable and drive the overall design and implementation process.

Support Staff Support staff assist faculty and students while courses are running.We like to use a NASA analogy to describe the relationship between designers and support staff—the designers are like Cape Canaveral staff, getting the rocket ready to launch, while the support staff are like the Mission Control Center staff in Houston, who take over after liftoff and during the space mission. Support staff help courses run smoothly, just as designers help create them. Support staff offer a point of contact for students’ non-academic issues, such as helping with the learning management system (LMS), submitting assignments, using the online library, and other less technical issues. Faculty support includes responding to their questions about using the LMS, helping with routine operational tasks such as identifying students who are falling behind, and identifying inappropriate discussion posts. Support staff may also participate in synchronous sessions, assisting faculty by setting up software, solving issues when a course is underway, and managing recordings. Support staff can significantly improve student satisfaction and retention.

Introduction 3

About This Book This book has 10 chapters, which follow the progression of program development from conceptual definition to revision. The first three chapters cover strategic considerations. Chapter 1 – Getting started online covers reasons for creating online programs, their opportunities, and challenges. It provides an overview of program and target audience, advantages of online programs, and how to assess their demand. It goes on to describe considerations in developing or expanding online programs, such as the benefits of having a digital strategy, transitioning the institution, programs, and faculty to online education, the course development process, and choosing an LMS. Chapter 2 – Faculty considerations for building an online program describes how faculty can adapt to online teaching by training in the technology and receiving mentoring from experienced peers. This chapter also describes the benefits of having faculty coordinators and course facilitators, as well as the recruitment, training, and compensation of faculty and facilitators. Chapter 3 – Issues that online courses must address deals with important challenges in online education which, when addressed, can help programs grow in quality and scale. Topics covered include choosing the technologies and marketing programs. This chapter also covers studentrelated issues such as academic integrity, accommodating students with disabilities, and data security and privacy. The next four chapters cover how to develop or improve a program for online and course development. Chapter 4 – Program definition and development describes different models of scheduling, development, instruction, interaction, and synchronous teaching. Administrators need to understand them and select the ones that are most appropriate for their programs. Online programs should be defined with a clear understanding of how they will compete in online markets, and how they will be marketed. Chapter 5 – Program administration covers the many administrative tasks that are essential for online programs. Administrative responsibilities include the development of a recruitment pipeline, assuring program quality, reviewing marketing collateral, evaluating the program, and managing the program lifecycle. Chapter 6 – Fundamentals of course design and development describes how adopting good overall design practices before creating

4 Introduction

courses enhances student learning and participation, ensures that technology will work for all students, and minimizes the maintenance effort. Ideally, course development begins with setting learning objectives, developing a course map, and creating assessments and learning activities. Course production should also address alignment, workload leveling, and standardization of tasks for course uniformity. Chapter 7 – Advanced course design deals with the issues involved in administering course development when coordinating a team. Quality assurance processes may involve people other than the primary course developers, such as editors, support staff, and facilitators. When designers and other staff are available, it is feasible to take advantage of more advanced techniques such as universal design and using the same content across courses. The final three chapters address when a course is running and revisions. Chapter 8 – When a course is underway describes the many activities that occur in preparation for the launch of an online course, while a course is running, and shortly after it ends. The material is organized in sections corresponding to the phases of course execution: pre-launch, the first week, the middle weeks, the last week, and wrapping up. Chapter 9 – Addressing unexpected developments is about unexpected occurrences while a course is running. This chapter offers tips on how to address issues that are identified when a course is underway, such as technical problems, student issues, and problems that are discovered with the course. Chapter 10 – Administering course revisions describes how to determine when a course needs to be updated, how to plan the updates, and how to administer those revisions. It also covers curriculum and program review and evaluation. We have also included a glossary to make the book more approachable. First instances of terms listed in the glossary are presented in boldface.

Chapter 1

Getting Started Online

This chapter is written primarily for administrators and faculty who are considering developing online programs, in order to help stakeholders understand important decision criteria. It may also be used by administrators and faculty whose institutions have been offering online programs and who want to better organize and administer their online offerings, and who may wish to better integrate them with their more traditional, face-to-face courses. This chapter covers: • • • • • • • • • • • •

Reasons for creating online programs. Making the case for developing an online program. Opportunities and challenges of online education. Advantages of online technology in face-to-face and blended courses. Defining the program and target audience. Assessing program demand. Overview of the course development process. The benefits of having a digital strategy. Identifying faculty. Choosing a learning management system. Choosing a transition approach. Identifying further resources needed for online programs.

Developing an online program is a major commitment, so proposed online programs should be carefully evaluated. Existing programs may also benefit from this strategic thinking. Key considerations include the market for the program, the use of online content in face-to-face and blended courses, and the role of the program in the institution’s digital strategy. More tactical considerations include choosing the scheduling, course development, and instructional models; these are covered in Chapter 4.

Reasons for Creating Online Programs Institutions create online programs for many reasons. One is to enable students to take courses who cannot come to campus. Another is to improve retention by

6  Getting Started Online

allowing students to continue when they move or travel. Both of these enhance enrollments and expand the institution’s geographic reach. Online courses can also support face-to-face programs by enabling more flexible study and by providing course content in multiple formats. Done well, they improve educational outcomes for many students. Sometimes institutions create online programs because competing institutions have them, and they seek to enhance their own institution’s reputation. In many institutions, online education has grown organically. At first it was led by a few tech enthusiasts and early adopters, who built websites to support their face-to-face courses; some built websites that supported fully online courses. In the early days, this was technically challenging, because there were no tools to support online courses. Two innovations changed this: the development of learning management systems (LMSs), and the tailoring of content management systems. With these tools, the development of online courses is now much easier. In parallel with this, there was a dramatic increase in the acceptance of web-based online courses, which prompted many schools to create them.

Making the Case for Developing an Online Program Online educational technology is used successfully in conjunction with face-toface instruction, and academic leaders who are aware of this are more inclined to support purely online education (Schrum, Galizio, & Ledesma, 2011). Many institutions’ administrators and faculty believe that entering online markets will improve enrollments, so the leadership of these institutions may also be inclined to support online education. Properly done, online education can support diversity of student interests, readiness, and approaches to learning better than face-to-face education, and an institution may be able to use this to make programs more appealing to broader audiences and markets. Building a high-quality online program requires a serious investment of faculty time and other resources.Therefore, best practice is to evaluate the costs and benefits and document them in a formal proposal. This proposal may be developed by an institution’s faculty and/or administrators, and supported by accreditors, business, or academic partners. It should contain: • • • • • • • • •

a conceptual description of the proposed program estimates of the costs of developing and offering the program how it fits in with or supports the institution’s digital strategy how the program will be marketed how it will be administered how faculty will be identified to develop and teach the courses market assessment an analysis of competing programs enrollment projections

Getting Started Online  7

• • •

relationships to existing and planned programs instructional design, media, and other resource needs the contributions that the proposed program will make to the institution and its students’ success.

The proposal should be presented to an institution’s administrative leadership, because the program development process requires supervision and advocacy. When an institution has strong leadership and initiative, the programs are more likely to be successful (Portugal, 2006). Strong leadership provides financial resources, a supportive digital strategy, and encouragement to faculty and other stakeholders. While some programs can succeed without support from leadership, they are typically more difficult to launch and sustain.

Opportunities and Challenges of Online Education Many faculty who have only taught face to face think of each lecture as a performance of scripted material, like a play, rather than a collection of scenes that can be prerecorded and edited, like a movie. One advantage of online education is that each part of a course can take the form or use the technology that is the best for teaching that particular material. For example, a course in accounting can explain a point in text, use an animation to illustrate and reinforce the point, and then include an embedded spreadsheet to have the students try it on their own, all on the same webpage. To allow time for each element to be presented in the form that suits it best, whether that is text, video, or something else, faculty must prepare the material well in advance, like a cinematographer setting up different camera angles and lighting to help tell the story. And, like making a movie, only the best takes are included in the final result. This does lead to a challenge of developing online courses, which is that course material must be completed in advance, ideally before the course launches. This can be difficult for faculty, who are unaccustomed to such schedules. Students expect online course content to be of good textbook quality, and they have limited tolerance for typos, grammar errors, poor explanations, and other defects that could have been identified by editors and other faculty. Best practice is for course development to be completed several weeks before launch, to allow enough time to correct minor errors. (For more on editing and quality assurance, see Chapter 7.) If changes are to be made while a course is running, students are more likely to be patient with imperfections if material is presented as the latest advances in the field or has been posted in response to student requests. A major advantage of online courses is that they can address the diversity of student approaches to learning better than face-to-face courses (Mestre, 2006). Some students learn better visually, while others learn better by listening, and yet others from interacting with the material. Online courses with both asynchronous and synchronous

8  Getting Started Online

components can support all these diverse approaches while at the same time providing more opportunities for discussion than in the face-to-face classroom. For example, students in an online course can read the lecture material, ask faculty questions in synchronous sessions (known at Boston University as Live Classrooms or Live Offices, depending on whether they are course-wide or for a subgroup), and discuss the material with their classmates in asynchronous discussion forums and synchronous sessions. Faculty who teach online can provide recorded videos with both closed captions and transcripts, meeting the needs of students with disabilities and those who benefit from being able to choose instructional approaches. A particular challenge with online education is that even faculty who may have excellent communication skills face to face need to adapt to the different communication styles that are needed to teach online effectively. Online communications, such as email, chat, and texting, require different skills. Chief among them is the ability of faculty to show that they care about their students’ success, even at a distance (Lammers & Gillaspy, 2013). In a classroom, faculty can convey this through encouragement, smiles, and other body language, but it is a far more challenging task online. Although teleconferencing is becoming easier as the technology improves, text-based asynchronous interactions, such as discussions, still form the core of online teaching. Multimedia conferencing tools can help faculty who are less skilled in text-based asynchronous communications convey this sense of caring, because their students will be able to speak with them and see them, while faculty who are more comfortable with asynchronous modes of communication can use technologies like email and texting to establish a strong sense of connection with their students (Glazier, 2016).

Advantages of Online Technology in Face-to-Face and Blended Courses Fortunately, online course content and digital technology can greatly improve the quality of traditional face-to-face, blended, and flipped courses. Online technologies can help faculty monitor and address student preparation, improve engagement, support diverse instructional approaches and diversity of student pace, and allow unlimited review of course material. Faculty can also take advantage of the opportunity to offer extra material that may not be in textbooks. Online technology can help students prepare for the face-to-face component of flipped classrooms by studying and taking assessments that help faculty monitor student preparation. For example, students can be required to complete online tutorial readiness assessments before face-to-face sessions, and faculty can learn from them which topics the students have mastered, and where they need additional work in class. This can best be done in a face-to-face environment if the assessment is implemented digitally and automatically graded. Without the fast grading of the digital implementation, faculty will not have the quiz results until the class is over.

Getting Started Online  9

Online course technology can significantly improve student learning in courses taught in both face-to-face and blended formats by improving engagement, because each student can interact with the content independently, and communicate with other students and faculty (Tang & Byrne, 2007). Online technologies allow students to have many conversations with faculty and other students; instead of being limited by classroom time, students can continue a discussion, or ask deeper questions to pursue an individual interest. With a greater sense of privacy, students can also ask “dumb” questions they might have been embarrassed to ask in a group (Harrington and Loffredo, 2010; Sullivan, 2002). Online, students can spend variable amounts of time on different topics, as their preparation and interests dictate. Students may need different explanations or examples, or additional review; this is easier for individual students to do online, particularly if the material is presented in different ways with many examples, but it does not work well in the physical classroom. It is easier for online students interested in advanced material to study it more deeply (see Chapter 9). Developing an online course requires more up-front faculty effort than developing a face-to-face course, but faculty are rewarded when they are able to use their online course content in their face-to-face courses, as online courses provide additional ways for students to interact with and learn the material. The more effort that faculty invest up front in developing a course, by providing numerous examples, formative assessments, remedial content, and advanced material, the less time they will have to spend later answering questions and helping struggling or advanced students. For example, when online quizzes include extensive feedback, students are less likely to use class time to ask questions about them. (For more on motivating faculty, see Chapter 7.) Integrating online content in face-to-face courses also enables faculty to include content not adequately covered in available textbooks.This helps faculty teach courses with rapidly evolving subject matter, such as technology or neuroscience, in which the latest content is of the greatest interest. Online teaching objects, such as interactive multimedia, may even be superior to traditional faceto-face teaching methods for some subject matter and some students. For example, an online simulation can make it possible (and safe) for students to gain a qualitative sense of what is involved in controlling a nuclear reactor, alongside a class in which they learn the theory of nuclear reactor control.

Defining the Program and Target Audience The next step in defining a new online program is the creation of a specification document based on the original proposal. It sets out the program’s vision statement, academic content, target audience, marketing strategy, projected enrollments, its relationship to other programs, and other program features. This document may also identify the faculty who will further define and develop the program.

10  Getting Started Online

It is important to design strong, forward-looking academic programs, so the knowledge that students acquire will have lasting value. This is done by basing the program design on analysis of current and future educational needs. We attempt to forecast how our students’ needs will evolve in the years ahead by tracking trends and evolving business practices. The vision statement describes the students expected to be interested in the program, how it would serve them, and the value it would provide to both the students and the institution. It may also include a description of how the program is expected to evolve.

Thomas Cavanagh, vice provost for digital learning, University of Central Florida The vision should align with the institutional mission and the stated strategic objectives. A program that is looking to provide access for community college transfers will have a different strategy than one focused on professional master’s degrees. Start from the goals and work backwards. Part of that process is ensuring that the goals are achievable with current resources.While revenue and expenses are a necessary part of the planning process, mission orientation and quality must be an equal focus. Programs are usually created by faculty or administrators. There is typically an outline of how they will be developed and managed and who will teach the courses. If the projected online program is based on an existing face-to-face program, the document should describe how they are related, including how students could take both face-to-face and online courses. Online markets are more competitive than local markets, because online programs compete with programs from everywhere, so programs should be developed in a way that addresses competing programs. (For more on market assessment, see Chapter 3.) Legislation and regulations may limit the target audience for a program. In the United States there are many evolving federal and state laws and regulations that apply to online education, to protect students from unscrupulous programs and to protect in-state schools from outside competition. For example, many states currently require that online programs obtain authorization from their departments of education before programs can be offered to residents of that state. Happily, this is being addressed by the National Council for State Authorization Reciprocity Agreements (NC-SARA), which establishes national standards for postsecondary distance education offerings across states and territories (https:// nc-sara.org/). Additionally, the United States Department of Education has many regulations governing online programs. We do not cover all specific state or federal regulations here, because many are evolving rapidly. We recommend that regulations be addressed at the institutional level.

Getting Started Online  11

However, there are some federal laws that must be kept in mind.Administrators must ensure that the online programs that they are responsible for are fully compliant with the Americans with Disabilities Act (ADA), and follow good disability accommodation practices. Policies and procedures should be defined that ensure that faculty, staff, and administrators have received training in both the ADA and the institution’s policies and procedures regarding students with disabilities. Admissions information should include messaging that students with disabilities should contact the institution’s Office of Disability Services sufficiently in advance of their first course so that accommodations can be identified. (For more on accommodating students with disabilities, see Chapter 3.) Many accommodations, such as closed captions for videos, should be built into courses as they are developed. Best practice is to build all courses using universal design principles that help all students, including those with disabilities. (For more on universal design, see Chapter 7.)

Assessing Program Demand In this section we describe the market assessments needed to support the decision to launch an online program. In Chapter 3, we describe how to plan and implement the marketing that will help bring students to the program, including competitive landscape analysis. In Chapter 4, we describe marketing plans, and in Chapter 5, we describe how to update the program’s marketing collateral. Online programs can significantly affect an institution’s overall reputation. Consequently, it is essential for the institution’s leadership (e.g., deans, provosts, or principals) to assess the likelihood of success before making the decision to commit to an online program. Analysis of market demand can also help existing programs that are struggling with low enrollments by identifying program changes that might make them more relevant and attractive to prospective students (Ke & Kwak, 2013). One of the ways in which online markets differ from face-to-face ones is that a geographically dispersed, sparse market can support an online program when it would not support a face-to-face program. However, because there are fewer geographic restrictions in online markets, they are more competitive, and competition forces players to differentiate themselves. What this means for those defining online programs is that market analysis and market differentiation are more important for online programs than face-to-face programs, and that niche programs are often successful online, when they might not be on campus. (For more on market analysis, see Chapter 3.) Academic programs are essentially differentiated by their cost, reputation, or content, both face to face and online. An institution may find a less crowded market by adjusting the cost.The reputation of the institution may allow its programs to set themselves apart by offering higher-quality courses than their competitors. Online markets can be extraordinarily differentiated in terms of content, based on subject matter and regional differences in laws, customs, languages, businesses, and

12  Getting Started Online

niche markets. Examples include law courses specific to each state, baseball analytics, aquaculture, oyster and alligator farming, and Native American languages.

Overview of the Course Development Process In many institutions, faculty are expected to shoulder virtually all of the work involved in online course development. If, in addition to their subject expertise, faculty are knowledgeable about online education theory, skilled in multimedia production and integration, and have enough time, they may be able to produce high-quality content on their own, but this is not a common combination. For programs with resources, several kinds of support can be helpful to faculty. Best practice is to provide online faculty with mentors, designers, and faculty and student support staff to help them develop and teach their courses (for more on mentoring, see later in this chapter). Faculty who are new to online development may have never worked with instructional designers and development staff, so it is important that they meet with them early in course development to learn how such support can help, and take full advantage of their experience. It is often useful to think about developing courses in two overall steps. First, define the course content; second, design the best way to teach each component of the content. The first part is done by faculty, while the second part might be done by faculty alone, or faculty working together with a designer. More specifically, however, course development involves multiple stages. Instructional designers help faculty implement their material online; then, after a course has run, they also play an important role in revising and improving the course (for more on this process, see Chapter 10). During development, designers help faculty construct the overall design of their courses, assist with the technical implementation, and may also coordinate the other staff (for details on this process, see Chapter 7). At the same time, faculty can benefit from the assistance of production staff, such as videographers, animators, and illustrators, to help them plan and create their original content and put it online. Next, editors and instructional designers may work with the faculty to devise ways to make the content more accessible, memorable, and comprehensible to students. Minor revisions could, for example, involve content being organized in a table, displayed as an image or a series of images, or illustrated as an animation or video. A large program may also have a multimedia specialist who helps faculty with conferencing tools, recording studios, and audio and video devices. For example, staff can edit videos recorded by faculty and help them produce videos that are short enough that students will actually watch them. Another advantage to having staff edit videos is that they can produce more consistent results, such as consistently adding closed captions, because they would know the program’s accessibility requirements and style. (For more on accessibility and course production, see Chapter 7.)

Getting Started Online  13

The Benefits of Having a Digital Strategy Regardless of the reasons for deciding to create online programs, an institution should start by creating a digital strategy that provides context for them. A digital strategy defines the policies and procedures for how an institution uses digital technology in education, and how it is funded and managed. All institutions use digital technology, but not all have a digital strategy. Creating a digital strategy will shape an institution’s thinking by documenting how things are done and, ideally, will provide insight into how to do them better. An institution should have a digital strategy covering online education before developing online courses. A digital strategy should help faculty and administration understand what is permitted and encouraged, and how to develop and offer online courses within the organization (Garrison & Kanuka, 2004).To be effective, a digital strategy must be broadly embraced within the institution. Strategies are often rejected by faculty and other stakeholders because they do not address all of their needs and they were not involved in defining them. The best approach is to define the digital strategy at the middle levels of the institution, with extensive consultation with governance level and faculty. Different programs may have somewhat different digital strategies; the institutional strategy should provide a framework for programs. The strategy should also define how digital instruction is governed and funded. It should define the infrastructure that will be provided for the development and offering of online courses, and how it will be managed. A detailed digital strategy may also define policies specific to online programs, such as: • • •

academic and budget approvals required for online courses how course development will be funded and managed faculty compensation for online course development and teaching.

Lawrence Ragan, retired director, Penn State’s World Campus Two primary things that institutions have come to recognize in terms of digital learning are that digital learning is an integral, crucial, critical piece of the education framework today, and that for digital learning to be successful you must have a confident and competent faculty. You must provide the services and the development preparation for those faculty members. It’s absolutely critical that we prepare our faculty for success in this new space, so resources have become much more available for faculty development services. Sometimes it exists within centers for excellence in teaching and learning; sometimes it’s created as a brand-new entity for faculty development in the digital learning space. But the recognition is you don’t just turn over the course development environment and say, “Okay faculty, go forth and digitize.” There’s more to it.

14  Getting Started Online

A well-defined digital strategy may encourage faculty to use digital technology in their face-to-face courses and to develop online courses, and it should also address how online courses interoperate with face-to-face courses. For example, it may specify that online courses must meet all the requirements of the corresponding face-to-face courses, so that students can mix online and face-to-face courses to learn the same content. It may also address roles played by online noncredit courses; for example, how to count noncredit math courses that help students get ready for programs that require math. It should specify how online courses and programs appear on students’ transcripts, and define how unconventional courses, such as Massive Open Online Courses (MOOCs), fit into the institution’s overall online portfolio. For example, a digital strategy may use MOOCs as teaser courses to recruit students and help them prepare. (For more on MOOCs and MOOCish courses, see Chapter 4.) A digital strategy should address how quality assurance will be implemented for online courses and who is responsible for it. One part of quality assurance is fundamentally technical, such as maintaining standards of clear English and ensuring the functionality of the instructional design and infrastructure. The other part of quality assurance is academic, and includes faculty qualifications, appropriateness of the subject matter for the program, appropriate level of subject matter for the courses, and rigorous academic integrity. Academic quality assurance is arguably more important online than in face-to-face courses, because online courses are more widely visible and can have a greater impact on the reputation of the institution. Strategy can be driven by program requirements. For example, if there is an online chemistry program that includes residential requirements for lab work, then the digital strategy may provide a framework for online programs that include on-campus activities. King and Alperstein (2018) cover planning for online programs well. One of their main theses is that strategic planning should be an ongoing activity of administering online programs, unlike institutional planning, which is done every five years or so. Creating and updating these plans typically consumes a lot of senior management time. Unfortunately, these tend to be so high-level and stale as to not be useful. What is needed is ongoing strategic planning, as an integral part of operational planning, as it often reveals issues. If these issues are addressed in the operational context, as updates to strategic plans, they are more likely to be grounded in reality and useful, as well as timely. An ideal digital strategy provides a framework within which online programs can plan.

Identifying Faculty Even the best-developed online courses are better when good faculty teach them. The specification document is used to identify the faculty who will develop and teach the program. If an institution is bringing a face-to-face program online, ideally the institution will already have the faculty needed, and

Getting Started Online  15

those faculty are interested in developing and teaching the program online. (For more on motivating faculty to teach online, see Chapter 7.) If an institution does not have the faculty they need for the online program, they can either work with current faculty or hire additional faculty. If they prefer to work with current faculty, they should consider recruiting from other departments. For example, if a program needs faculty for a statistics course in a psychology department, the administrator may reach out to the math department. If an institution is hiring new faculty for a program that they wish to develop, it is an important opportunity to hire those skilled in online development and teaching. Such faculty may be particularly helpful in mentoring existing faculty who are new to online education. Faculty who use technology to teach face-to-face as well as blended courses are more likely to have good online teaching skills. Prior successful online development and teaching experience are the best predictors of success. Some faculty may be good at developing online courses, but not as good at teaching them, because they lack online communication skills. A large department with many faculty with expertise in the same area may be able to assign different faculty to develop and teach courses.

Choosing a Learning Management System The most visible element of online education is the technical structure used to offer the courses. In the early days of web-based online education, faculty who wanted to put their course material online needed to have website development skills. Courses were often offered piecemeal, by the more technically adept faculty, and students were often confronted with a variety of tools and interfaces. However, that has changed with the development of learning management systems, or LMSs. An LMS provides a basic scaffold, or system, that allows anyone to make course materials available online easily, even without web development skills. Now, an LMS saves everyone from having to know how to build a website, and it can provide support for tests, homework, grading, multimedia conferencing, discussions, video creation and sharing, and other components of modern online courses. While it is still possible to develop courses without an LMS, doing so requires much more effort. The framework for choosing, funding, and administering LMSs should be defined in the institution’s digital strategy. If an institution has an LMS that is used to support face-to-face instruction, but may not have all of the functionality needed to support online instruction, then the LMS may need to be replaced, updated, or have additional functionality provided by servers linked from the legacy LMS. (For more on the challenges of using technologies see Chapter 3; for more on deciding what technologies to use, see Chapter 7.) Other kinds of software systems are used for online programs, and may be bundled with an LMS. One of these is a learning content management system (LCMS), which is an organized repository for the many files that make up courses. File management for courses is complex and labor intensive, and these specialized

16  Getting Started Online

file-management systems have been developed to make this easier. Programs that share elements across courses, such as help pages and institutional policies, should use an LCMS. When choosing an LCMS, administrators should consider the cost of the software, compatibility with the LMS, and the cost of training. Ideally, programs will also have a video server specially designed to stream video content. Institutions with limited resources can use YouTube or a server which is not optimized for video streaming. Since videos are a significant part of many online courses, the LMS or a separate video server should be tested with large video files. Hardware and staff support for synchronous sessions should also be tested, to ensure their quality. The choice of an LMS will involve compromises. Some of them are easy to use, but the design tradeoffs to make an LMS easier to use usually involve eliminating or hiding advanced features. It is not necessary for an LMS to have all the functionality desired for all specific educational needs, because an LMS can be extended by integrating third-party extensions. An LMS can be chosen based on its core functions and available extensions. We evaluated several LMSs for online programs and found that those which were easier to use lacked some features, such as the ability to organize courses by separate learning modules, which we need. Another tradeoff is the cost of the LMS and its associated support.When selecting an LMS, program administrators should evaluate the cost of technical support, which may be provided internally or from a vendor. Other considerations include whether the LMS permits access controls so that only registered users can access the courses, whether it can easily integrate with the institution’s student information system, and how well the LMS protects the privacy of student data. The LMS should also support granting access to users from outside the institution, such as guest lecturers or academic accreditors. One of the most valuable features of an LMS and associated extensions is the ability to monitor student activity and progress, including: • • • • • •

students who are not participating in the course learning activities how much students use different learning resources students who are submitting assignments late students who are at academic risk in individual courses students who are at academic risk in programs uniformity of grading across sections and facilitators.

Many LMSs provide these functions, but they may cost extra. Every LMS has minimum technical resource requirements.While some departments or colleges are large enough to administer their own LMS, most departments and colleges are too small to do this well.There are several advantages to contracting the hosting of the LMS to companies specializing in providing this service: •

Vendors manage many LMS instances and will simply do a better job than system administrators who spend only a fraction of their time on the LMS.

Getting Started Online  17

• •

•

The institution does not have to make a long-term commitment to particular hardware or resources, which may be inappropriate as programs and available software evolve. Because of economies of scale, LMS hosting services are usually able to provide the service at a lower cost than an institution itself can. Large institutions with extensive IT infrastructure may find it more cost effective to provide this service in-house, but most institutions will find it better and less expensive to procure this service from commercial vendors. If an institution is procuring such a service, they need to be very careful that the service-level agreements include guarantees of uptime, response latency, and timely software updates. Cloud-hosted LMS deployments scale up or down easily with changes in usage.

To help decide which LMS will be best for an institution’s needs, administration, in consultation with faculty and system administrators, should determine how well it will work with their production model. (For more on production models, see Chapter 4.) When choosing an LMS, program administrators also need to consider the amount of training that will be necessary for both faculty and staff, along with the associated costs.

Jim Frey, learning design director The number one most important thing to keep in mind when choosing technologies is that you are designing a complete experience for your learner. The experience includes the obvious elements such as content, media, assignments, and assessments. But it also is hugely dependent on the ecosystem you are creating, which is shaped by your technology decisions.

Summary Checklist ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑

Document the reasons for developing an online program. Make the business and academic cases for developing the program. Identify the challenges in developing the program. Decide how to use online content in face-to-face courses. Define the program concept and target audience. Assess program demand. Define the course development process. Develop institutional and program digital strategies. Identify faculty who will develop and teach the courses. Choose and administer an LMS. Decide how to make the transition to online. Identify resources needed, as well as those currently available.

18  Getting Started Online

References Garrison, D. R., & Kanuka, H. (2004). Blended learning: Uncovering its transformative potential in higher education. The Internet and Higher Education, 7(2), 95–105. Glazier, R. A. (2016). Building rapport to improve retention and success in online classes. Journal of Political Science Education, 12(4), 437–456. Harrington, R., & Loffredo, D. A. (2010). MBTI personality type and other factors that relate to preference for online versus face-to-face instruction. The Internet and Higher Education, 13, 89–95. Ke, F., & Kwak, D. (2013). Constructs of student-centered online learning on learning satisfaction of a diverse online student body: A structural equation modeling approach. Journal of Educational Computing Research, 48(1), 97–122. King, E., & Alperstein, N. (2018). Best Practices in Planning Strategically for Online Educational Programs. New York: Routledge. Lammers, W. J., & Gillaspy Jr, J. A. (2013). Brief measure of student-instructor rapport predicts student success in online courses. International Journal for the Scholarship of Teaching and Learning, 7(2), 1–13. ERIC Document Reproduction Service No. EJ1135165. Mestre, L. (2006). Accommodating diverse learning styles in an online environment. Reference & User Services Quarterly, 46(2), 27–32. Portugal, L. (2006). Emerging leadership roles in distance education: Current state of affairs and forecasting future trends. Academic Leadership:The Online Journal, 4(3), 1–12. Schrum, L., Galizio, L. M., & Ledesma, P. (2011). Educational leadership and technology integration: An investigation into preparation, experiences, and roles. Journal of School Leadership, 21(2), 241–261. Sullivan, P. (2002). “It’s easier to be yourself when you are invisible”: Female college students discuss their online classroom experiences. Innovative Higher Education, 27(2), 129–144. Tang, M., & Byrne, R. (2007). Regular versus online versus blended: A qualitative description of the advantages of the electronic modes and a quantitative evaluation. International Journal on E-learning, 6(2), 257–266.

Chapter 2

Faculty Considerations for Building an Online Program

This chapter is written primarily for administrators who are considering launching or have already launched online programs and for the designers and faculty who develop and teach courses in these programs. This chapter covers: • • • • • • •

Helping faculty transition to teaching online. Preparing faculty to develop online courses. Mentoring faculty to teach online. Faculty compensation models. The benefits of designating faculty coordinators. The benefits of employing course facilitators. Recruiting, developing, and evaluating facilitators.

Faculty who teach face to face may find that adapting to online teaching presents some new challenges, including learning technology as well as more preparation and anticipation of student needs. It is very helpful to have experienced faculty mentor their peers who are new to teaching online. This chapter also discusses the benefits of having faculty coordinators and course facilitators, as well as the recruitment, training, and compensation of faculty and facilitators.

Helping Faculty Transition to Teaching Online The process of learning to teach a particular subject well has always required mastery of the subject matter, knowledge of students’ background, optimal topic sequencing, and good examples. All of these may be specific to the topic, so general teaching skills are only a place to start. Learning to teach a subject well may require years of practice. Mastery of the subject matter ensures faculty know what is important to teach or emphasize, know important applications, are able to answer student questions, and have the ability to explain the relationships between topics. We have worked with hundreds of faculty at Boston University as they made the transition from teaching on campus to also teaching online. Occasionally, faculty may assume that teaching online is the same as teaching face to face, and

20  Faculty Considerations for an Online Program

they expect to do nothing more than present the same lectures to a webcam, which will disappoint students. More commonly, though, nowadays most faculty are aware that teaching online is different from teaching in the face-to-face classroom, and they may be concerned that they will have to spend a great deal of time learning to teach online, and perhaps not be able to teach as well as they do in the classroom. It can be helpful for them to recognize that the patterns of instruction that they are familiar with are the same for face to face and online, though the ways in which they are implemented differ (Hillman, 1999). Both involve cycles of presenting material, determining whether students understand it, and then either presenting the content differently, such as by providing an example, or moving on (Mehan, 1978; Sinclair & Coulthard, 1975). However, developing an online course before seeing and hearing students can be challenging for faculty, because there is no immediate feedback to let them know how well the students are understanding. Such courses can be equally challenging for the students, because they have no immediate way to communicate confusion. Therefore, other mechanisms must be used to accomplish this sort of continuing evaluation. For example, embedding formative evaluations in the material tests students’ understanding of a topic, and can give them both immediate feedback and additional explanations. Reviewing the evaluation data can also provide faculty with a clear view of which sections students understood and where they had trouble. (For more on formative evaluation, see Chapter 6.)

Fawn Thompson, lead instructional designer at Boston University’s Office of Distance Education In my experience, reluctant behavior often stems from one of two sources: The faculty is vastly overburdened, or has deep concerns that have not been properly addressed by anyone in the process thus far. While I can’t do much about busy schedules (other than be respectful of their time), I can discuss and investigate concerns, and/or possibly point faculty to other resources. For example, an instructor who was concerned about student privacy simply needed me to explain in greater detail how specific tools in the LMS would work and demonstrate how she could double-check student settings. Another instructor had serious concerns about how his small-group discussions would translate to the online environment; I connected him with another faculty member who had a great deal of experience leading similar synchronous discussions, so that she could explain her process and share her experiences with him. I also followed up with these faculty to ensure that I answered any remaining questions.

Faculty Considerations for an Online Program  21

Fundamentally, online courses must be developed in a web-compatible format, and teaching well online requires faculty to make extensive use of an LMS, at a minimum. Most faculty who teach only face to face have limited experience with this kind of development, so to develop a course and teach it online successfully, faculty need to learn how to use the LMS and other tools, and they will learn faster and better if training is available. Online education works best if faculty take advantage of new teaching methods enabled by technology (Holmes, Tracy, Painter, Oestreich, & Park, 2015). Best practice is to begin the preparation of faculty with a dedicated online orientation course that covers the basics of online teaching and how to use the LMS (Vaill & Testori, 2012). This introductory course should use the same design template as the other courses in the program, to help faculty learn their design and navigation. After this orientation, faculty development should be tailored to their teaching background. There are many advantages to offering multiple tiers of faculty training, taught by faculty who are more experienced in online education. Full-time faculty are presumed to be committed teachers with good teaching skills. Part-time or adjunct faculty may be skilled in the subject matter, but they may benefit from additional training to develop their teaching; in other cases, part-time faculty may have extensive teaching experience. A typical faculty development plan includes enrolling them in a course that is being taught by a skilled instructor, preferably one of the courses he or she will be teaching. If staffing allows, the new faculty may co-teach their first online course with an experienced, full-time faculty member. The content of such training will depend upon whether faculty will be responsible for developing their own courses from scratch, working with an established course template, or working in a course that has already been created by other faculty.This training should include core components which are always used in online courses, such as creating and editing announcements, tests, and discussions, and working with the grade book. Later courses can be offered that are tailored to individual faculty needs, and may cover topics such as creating videos, advanced test options, and using the LMS to measure assessment quality and difficulty by analyzing the statistics of student scores on individual graded items and how they correlate with other measures of their performance to improve assessments (item analysis). Program administrators need to identify the skills that faculty need to teach online effectively; the skills will depend on whether designers or support staff are available. (For more on online course development and design, see Chapters 6 and 7.) In all cases, though, when introducing faculty who are new to online teaching, it is helpful to have them teach courses in full-length terms as opposed to shorter terms. This allows more time and reduces their anxiety about having to get everything right the first time. (For more on scheduling models and term lengths, see Chapter 4.)

22  Faculty Considerations for an Online Program

One of the best ways to help faculty develop an online course is to have them teach it first in a blended format. Blended courses include both face-to-face and online components. Faculty can gradually develop online components of a blended course as they teach it in the familiar and forgiving face-to-face format, making the transition to online much easier than developing and teaching online courses from scratch. Teaching face to face helps faculty learn which explanations and examples work and which do not; this also helps them debug the assignments and assessments, fine-tune the learning objectives, determine optimal sequencing, and integrate other pedagogy that easily transfers to online. While teaching face to face, faculty can also identify areas where students become confused. This lets them know where online students may benefit from embedded formative evaluations or additional explanations, guidance, or examples. Faculty can also gain practice in the use of various technologies, such as creating slides for use in the classroom, and uploading syllabi and course materials to course websites. As a bonus, face-to-face experience takes away some of the apprehension that many faculty feel when teaching a new course, and gives them more confidence when they are developing and teaching online (Ulmer, Watson, & Derby, 2007). It also provides reassurance that if they have difficulties in the online portion, they will still be meeting their students face to face and will have an opportunity to recover. If institutions provide faculty with the same development support for blended courses as for fully online courses, it helps faculty make the transition to purely online teaching.This is especially helpful for institutions where faculty have little or no development support for face-to-face courses. Once all components needed for online are added to a blended course, the course can be offered entirely online. Faculty do need to identify the technologies that are best for their disciplines and the ways they want to teach. For example, music education courses require strong audio, while mathematics courses require ways to format and render symbols. This approach helps faculty adjust to online formats, gives them time to develop their courses, and helps their institution ensure the quality of their courses. When faculty have extensive experience developing and teaching online, they can develop successful online courses without first teaching them face to face.

Preparing Faculty to Develop Online Courses Few faculty have formal training in systematic course development. Even faculty with years of face-to-face experience can benefit from learning how to build courses in a systematic fashion, and this is particularly important for faculty developing online courses. Having a defined system helps ensure that all content generated is specifically related to the goals of the course, that evaluation actually measures whether students meet the course objectives, and that there is a map or checklist of all course content elements. It also addresses the consistency of

Faculty Considerations for an Online Program  23

courses across the program, making it easier for students to transition from course to course. None of this is unique to online education, but, because communication in online courses can be more difficult than in face-to-face courses, and development is more expensive, it is even more important to get the development done right the first time to avoid rework. (For a detailed discussion of course development, see Chapters 6 and 7.) Developing the asynchronous content of online courses requires planning. Faculty who have not yet taught online may not realize how much this changes their approach to course development and teaching. How administrators make faculty aware of these differences, and the time and support that is available to help them, can have a big impact on faculty’s willingness to develop and teach online. Major differences include: • •

It takes significantly more time to develop online courses, and administrators should be aware of this. This is a major effort for faculty, and they should be given enough time for course development. Developing online courses requires much more design than face-to-face courses. For example, an online course requires layout and navigation of a course website, whereas a face-to-face course may primarily have a textbook and lecture slides. If a program has standard course designs, new faculty can use one of them for their courses.

Faculty who are accustomed to teaching face to face may be unaccustomed to the scheduling demands of online course preparation. In programs with the resources to provide designers and other support, scheduling and coordination demands become important. For example, faculty who are used to doing everything by themselves just in time for their face-to-face lectures may resent having to prepare their lectures in advance to allow time to implement them in the LMS, or for others to do so. Faculty will usually come to understand and accept these schedule and coordination demands when they realize that once the course is set up online the first time, they will be able to use it again with only minor updates. If they are working with other staff, they may also be more accepting once they learn how the development staff can improve the quality of the course. It is helpful for faculty to recognize that they do not need to learn all of these new technologies to succeed. Most faculty need only a few easily learned skills to reap huge dividends. One approach that can make this simpler is to have different faculty learn different parts of the LMS and related technologies and share their knowledge. For example, one faculty member may be most interested in learning how to use the LMS grade book, while another is interested in learning about automatically graded assessments, another is interested in adding videos, and another in math formatting and symbols. Having faculty share their skills with other faculty reduces the demands on their time.This community effort should be ongoing, incorporating new faculty. Assistance from a faculty learning community can be invaluable when faculty

24  Faculty Considerations for an Online Program

are learning how to develop and teach well online. For example, a business professor who has lectured for years on the dynamics of global competition may have an excellent sense of how to cover the material in face-to-face lectures, but have difficulty adapting that experience to the design of an interactive business simulation. If faculty are available who have successfully developed and taught online courses, it can be very helpful for new online faculty to meet with these experienced faculty and have access to the courses they have developed. Indeed, providing success stories can support and inspire faculty as they develop their own courses and teach them online.

Lawrence Ragan, retired director, Penn State’s World Campus I think institutions have been more strategic in how they approach the design, development, and delivery of faculty development programs. There has been a shift in the mindset of the faculty members to digital learning and what it means to them.Ten to fifteen years ago, you’d be hard pressed to find a faculty member who had a degree online, and you’d be hard pressed to find a bunch of people who’d actually taken an online course, and you’d have fewer faculty who had actually taught an online course. Obviously over time we now are getting faculty into the academy who have either taken an online course, perhaps their whole degree has been or largely has been online, or they’ve participated in blended learning programs.

Mentoring Faculty to Teach Online Having faculty with extensive online teaching experience mentor faculty who are new to online is one of the best ways to help them learn. It can improve the quality of their teaching, reduce their anxiety, and speed their learning (McKenzie, Ozkan, & Layton, 2006). Best practice is to give experienced faculty mentors full access to the course content to review all aspects of course development and teaching. Mentors can identify areas for improvement in course content and pedagogy. They can also help with more difficult issues, such as academic integrity and accommodating student disabilities. (For more on academic integrity and student disabilities, see Chapter 3.) Informal mentoring can include: •

Program directors, area coordinators, and other faculty answering questions when brought up by faculty new to online. Examples of formal mentoring include:

• •

Documenting best practices in teaching online. Having program directors conduct webinars on teaching online for all faculty and facilitators.

Faculty Considerations for an Online Program  25

• • • •

Enrolling program directors, chairs, or experienced online faculty in every course, so they can advise faculty who are new to online. Having the director of disability services guide faculty in how to accommodate students who disclose disabilities while a course is running. Designating lead facilitators, who are faculty in training, and having their faculty assign them increasing responsibilities over several terms, until they are ready. Having a dean or principal provide guidance to faculty when student misconduct is suspected.

One way that mentors can help is by guiding faculty in conducting synchronous sessions and producing videos. For example, faculty need to know how to appropriately light their faces when they are using webcams.This is not difficult, because it can be done with table and floor lamps, but faculty need some training in how to position the lamps to have their faces uniformly illuminated, without creepy shadows where their eyes should be. If faculty are making many recordings with video, administrators should consider providing a quiet space with good lighting to act as a recording studio. If a program does not yet have faculty with online experience, it can use faculty who have experience implementing face-to-face courses in an LMS. Or, if a program has faculty who enjoy working with technology, they can be encouraged to experiment and support their less experienced or tech-savvy colleagues. Because faculty are curious and active learners, this process can quickly bring them up to speed. Instructional designers can also serve as mentors for faculty in many aspects of course design and online pedagogy. For example, an instructional designer may suggest to faculty that a textual explanation would be easier to understand if it were supplemented with a diagram; designers may also help develop online tools, such as formative assessments, or ways to improve student engagement. Most faculty are hired because of their academic credentials and expertise, but they may have limited experience with course development, teaching, or online technologies. Designers benefit from extensive education in course design, online pedagogy, and online technology, and they can share this expertise with faculty during the course development process. (For more on how designers can help in course definition, development, and design, see Chapter 7.)

Lawrence Ragan, retired director, Penn State’s World Campus We’re trying to break down the barriers between online and face-to-face teaching, to create equality, to send a message that good teaching is good teaching. Let’s help you get new skill sets for today’s teaching and learning

26  Faculty Considerations for an Online Program

environment, whether that’s in the classroom, online, in a blended program— that’s all we want to do. When faculty develop and teach online they’re surrounded by instructional designers and they’re getting faculty development experiences. Guess what happens? Face-to-face teaching improves. What a win–win! Faculty talk to their colleagues who are using these really cool new techniques in the classroom.They’re saying, “Where did you learn that?” “That was a technique I picked up online and now I’m using it in my classroom.” I would say to faculty members, it’s not because you’re teaching online that you’ve become a better instructor, it’s because you’re learning pedagogical techniques and approaches and for the first time, maybe in your career, somebody is asking you, “What are you trying to do in this class? Let me understand your objectives.” Nobody’s ever asked that before; someone handed him a syllabus and said, “Go teach Psych 2.” While faculty do not usually get much feedback from students while they are teaching online courses, they often receive a great deal of feedback in student evaluations near the end of the course.We have found that students’ evaluations of online courses tend to be significantly more detailed than evaluations for face-to-face courses, because online students have more time to think about their comments and express themselves, and because they can type and edit their feedback instead of having to write it by hand. Because of this, online students’ evaluations often provide more useful feedback for improving a course than face-to-face evaluations. These more detailed evaluations have shown that students highly value prompt feedback and answers to their questions. Faculty mentors emphasize the importance of this communication and teach techniques for accomplishing it, such as: •

developing assessments with feedback and configuring them for immediate release • creating forums where students can post their questions and anyone can answer • scheduling frequent synchronous sessions with faculty, including office hours • grading assignments promptly.

Faculty Compensation Models Persuading faculty to develop online courses is not always easy (Bender, 2005): First, because of the technical expertise needed, and second, because more effort is required to develop online courses than face-to-face courses. Our program

Faculty Considerations for an Online Program  27

dealt with this by compensating faculty with course releases or additional payments. While it may be challenging for administrators to motivate faculty in general to develop online courses, it can be even more difficult to persuade tenure-track faculty to develop them, because course development usually does not count toward promotion and tenure. This reluctance is changing. Many full-time faculty now recognize that teaching online enhances their career development and improves their teaching credentials (Green, Alejandro, & Brown, 2009). Some faculty also recognize that teaching online will benefit their face-to-face teaching by helping them learn better ways of designing their courses, presenting material, and interacting with their students. Unfortunately, even many faculty who are strongly motivated to develop online courses often cannot allocate enough time for it. In this situation, the best solution is to release them from some of their regular duties so that they have enough time for online course development. Course release does not apply to part-time faculty, and so they are normally compensated financially. Compensation for initial course development and compensation for course updates varies widely across institutions and programs. Best practice is to pay faculty when the development of a course has been completed.

Lawrence Ragan, retired director, Penn State’s World Campus There’s a realization that developing skill sets and competencies in the digital learning realm has value in faculty careers. I can put it down on my résumé that I taught online; then, should I move to a new institution, it has value in the marketplace. That incentive has drawn more faculty into the desire to learn, to develop their own skill sets, so that they are going to be successful online.The faculty, I think very smartly, have figured out that this is a gift that they are being offered of faculty preparation in an online learning domain, and if they embrace that, there’s no indictment on them, it’s only an opportunity for them. I think they now fully realize this, and desire to unpack this gift. “What do you have for me? What can I take next?” Whereas 10–15 years ago, we had to create incentive models and we had to do things to draw them in. Now I hear from all of these colleagues that I interviewed the same response and that is, “The faculty get it now.” And they not only pursue faculty preparation programs, they expect them.

The Benefits of Designating Faculty Coordinators When an institution wants to go online, it will need to determine how to finetune its organizational structure to do this effectively. Having online faculty

28  Faculty Considerations for an Online Program

program coordinators is recommended, because administering online programs is more difficult than administering face-to-face programs, but they are not absolutely necessary. These coordinators are faculty or staff who have responsibility for online programs, assisting the department chair or other administrators. We have had considerable success having full-time faculty who teach online also take on the role of coordinator, in exchange for course releases. Even small online programs should consider designating such coordinators, because those who do this well contribute significantly to the faculty development, curriculum, pedagogy, policies, procedures, and online marketing. Having one faculty review all courses helps ensure that they form a coherent curriculum and meet program standards such as academic level, student workload, navigation, faculty contact hours, interactivity, nature and quality of assessments, accessibility, and academic integrity. In addition, if a program has support staff, faculty coordinators can help to coordinate and manage expectations between them and faculty. Faculty coordinators also serve key roles in marketing the programs to potential students, and ensuring that descriptions of programs are correct and that the marketing collateral, such as ads, websites, and pamphlets, protects the brands of the institution and program. (For more on marketing, see Chapters 3 and 5.)

The Benefits of Employing Course Facilitators One of the challenges of maintaining a high-quality student experience as course enrollments grow is to provide students with the individual attention they require. One way to do this is to divide the students into small groups and provide each group with a facilitator, who is responsible for grading their assignments, answering questions, and guiding discussions (for more on scalability of courses, see Chapter 3). These facilitators must be proficient in both subject matter and teaching. Facilitators in undergraduate courses may be graduate teaching assistants, while in graduate courses, facilitators should have completed at least the same level of study as the program they are facilitating and have extensive experience in the subject matter. Facilitators enable programs to provide students with guidance comparable to what a good tutor provides, at reasonable cost. They also help students learn by answering questions, providing feedback on assignments and course research projects, and providing one-onone tutoring when necessary. Facilitators may also assist faculty by leading discussions for their groups. However, sessions led by facilitators should supplement, not substitute for, sessions conducted by faculty. Facilitated courses work best when students are organized into groups, with one facilitator assigned to each group. Best practice is for the group sizes to be large enough for the group discussions to be diverse, but small enough for students and facilitators to get to know each other. In courses with students with diverse interests, it may be helpful to assign students to groups based on their interest, with a facilitator knowledgeable in those areas.

Faculty Considerations for an Online Program  29

Facilitators have a disproportionate effect on the student experience. Our own research has shown that the correlation between students’ satisfaction with their facilitator and their satisfaction with their instructor is more than 80 percent. Our faculty are responsible for selecting their facilitators from a pool of well-qualified, trained candidates, as described below.

Recruiting, Developing, and Evaluating Facilitators Most of the facilitators for our programs are graduates of our master’s degree programs. Our best facilitators have come to us in this way:They understand the material well, know how our online courses operate, and accept our high standards. When we do not have enough facilitators volunteering after graduation, we search our database of graduates with criteria such as: • • •

a concentration in the area of the relevant course at least a 3.7 GPA ‘A’ grades in all courses in the concentration.

We contact graduates who satisfy these criteria, ask them if they are interested, and request their current résumé. If they are interested, we send their transcript and résumé to the program director and department chair. The director and chair determine which courses the applicant is qualified to facilitate. We normally assign new facilitators to lower-level courses, and if they do well we assign them to more advanced courses. When a program is first introduced, there will of course not be any graduates, so we reach out to our academic and professional networks, asking if anyone knows of people who would be well-qualified to facilitate. Occasionally we have needed to post job listings on our department website and on external websites. Programs may need to reach out beyond the local area to search for faculty and facilitators. Since online facilitators can work at a distance, it can be important not to marginalize non-local faculty and facilitators by conducting face-to-face training and meetings, so a program should be prepared to offer all facilitator training and meetings online. Some of our facilitators are skilled teachers, but others need to learn to teach well, so we provide extensive training. When facilitators are first identified as candidates, they are required to take a part-time, online course in facilitating. In this course, candidates serve as facilitators, while trainers serve as their students, constantly challenging them.The candidates are graded in this course. Most pass, but some do not.The most common reason for not passing is that they did not participate enough in the course, such as failing to respond in a timely manner or not communicating well enough. Candidates who pass this training are then available to be selected as facilitators. New facilitators are often provided with a mentor, who may be a faculty

30  Faculty Considerations for an Online Program

member or an experienced facilitator. The mentor will check the new facilitator’s grading, answer their questions, and provide general guidance. This process of mentoring continues as the facilitators become more skillful. Each term we hold a short webinar on how to facilitate well, hosted by the program director and department chair, which facilitators are expected to attend. During this webinar the policies and best practices for facilitating and teaching online in general are described.The webinars are very interactive, and the facilitators and instructors ask many questions, share their experiences, and suggest techniques that other facilitators may wish to try. Facilitators are evaluated each term by their students and the course faculty. The student evaluations have specific questions about facilitators, such as how quickly they graded assignments and assessments, how effective they were in guiding discussions, how good was the quality of their feedback, and how well the students thought that their facilitator did overall. Facilitators are also evaluated by the course faculty.These evaluations are stored in a database, and are sent to faculty when they are choosing their facilitators for subsequent terms. Each term the department chair sends each facilitator their student evaluations, together with praise and/or guidance on how they can improve. If a facilitator’s overall student evaluations are well below the mean for the program, then the facilitator’s performance is reviewed by the program director or chair, and the facilitator may be placed on inactive status. An inactive facilitator may be reinstated by the department chair after remedial training. When new, part-time faculty candidates who lack a terminal degree or teaching experience inquire if they can participate in teaching our online courses, we start them as facilitators. If and when they have proven themselves as facilitators, we provide them with additional learning opportunities, such as conducting synchronous problem-solving sessions. When they have learned how to teach well online, they become eligible to be appointed as adjunct professors. This development process typically takes about three years, during which time they are mentored in the courses that they are facilitating by full-time faculty.

Summary Checklist ☑ ☑ ☑ ☑ ☑ ☑ ☑

Develop a plan for faculty transitioning to online teaching. Develop a plan for faculty transitioning to online course development. Implement mentoring for online faculty. Develop a faculty compensation model for online. Decide whether to designate faculty coordinators. Decide whether to use course facilitators. Create a facilitator recruitment, development, and evaluation plan.

Faculty Considerations for an Online Program  31

References Bender, D. M. (2005). Developing a collaborative multidisciplinary online design course. The Journal of Educators Online, 2(2), 1–12. Green, T., Alejandro, J., & Brown, A. H. (2009). The retention of experienced faculty in online distance education programs: Understanding factors that impact their involvement. International Review of Research in Open and Distance Learning, 10(3), 1–15. Hillman, D. C. A. (1999). A new method for analyzing patterns of interaction. The American Journal of Distance Education, 13(2), 37–47. Holmes, M. R., Tracy, E. M., Painter, L. L., Oestreich, T., & Park, H. (2015). Moving from flipcharts to the flipped classroom: Using technology driven teaching methods to promote active learning in foundation and advanced masters social work courses. Clinical Social Work Journal, 43(2), 215–224. McKenzie, B. K., Ozkan, B. C., & Layton, K. (2006). Tips for administrators in promoting distance programs using peer mentoring. Online Journal of Distance Learning Administration, 9(2), 1–8. Mehan, H. (1978). Structuring school structure. Harvard Educational Review, 48(1), 32–64. Sinclair, J. M., & Coulthard, R. M. (1975). Towards an analysis of discourse. Oxford: Oxford University Press. Ulmer, L. W., Watson, L. W., & Derby, D. (2007). Perceptions of higher education faculty members on the value of distance education. Quarterly Review of Distance Education, 8(1), 59–70. Vaill, A. L., & Testori, P. A. (2012). Orientation, mentoring and ongoing support: A three-tiered approach to online faculty development. Journal of Asynchronous Learning Networks, 16(2), 111–119.

Chapter 3

Issues That Online Courses Must Address

A number of important topics should be addressed before course development begins. This chapter outlines decisions that should be made about how courses will be developed and run. This chapter covers: • • • • • • •

Choosing technologies. Fostering academic integrity. Scaling online courses. Accommodating students with disabilities. Marketing online programs. Developing marketing collateral. Student data security and privacy.

When these topics are addressed properly at the beginning of program development, they lay a foundation to help programs grow in quality and scale. This chapter describes ways to design content so it engages students and helps them learn, how to build course elements that support diverse approaches to learning and students with disabilities, and how to address issues of scaling online courses and academic integrity.

Choosing Technologies An important step in designing courses is choosing the technologies that will be used. An LMS will support some technologies, allowing users to extend its capabilities, but it will also preclude the use of others. For example, one type of extension allows faculty to save time by integrating publishers’ content. Most online courses use the web, and many technologies can be embedded in websites or linked from them. Different subject matter warrants different technologies; for example, a math class needs the ability to display equations and a chemistry class needs a way to display chemical formulae. The selection of the best technologies for

Issues That Online Courses Must Address  33

learning activities depends on the pedagogy, available software, and the devices that students have. Two of the most significant technological developments in the history of education were the chalkboard, which enabled spontaneous, one-to-many communication (Anderson, Finn, & Campion, 1963), and the personal computer, which enabled low-cost, many-to-many communication (Hiltz & Turoff, 1978). Technological changes often result in new pedagogic approaches for both faculty and students. Technology influences the way content is presented and courses are developed; for example, embedded JavaScript enables simple interactive formative assessments, and HTML5 enables video without the need for additional technologies. Some of these innovations affect the way that students interact with the course content. For example, technology enabling the generation of high-­ quality voice from text interacts synergistically with smartphones, allowing students to listen to lectures while doing other things such as exercising or cooking. Multimedia conferencing introduces the spontaneous interaction and spirit of a classroom into the online environment, and also allows students to present to teammates or the whole class, which in turn permits faculty to assign team projects and presentations. Software for online education continues to evolve rapidly. There are many programs that may perform similar functions, but may differ significantly in usability and other qualities, so software selection should be revisited periodically. For example, textbook publishers are producing virtual labs and other content that can be integrated into courses. Programming-language texts are now available with websites that include teaching compilers for the languages, which in turn provide students with intelligent guidance rather than just error messages. There are also web-based programming textbooks which provide a place for students to work on assignments and then provide context-appropriate feedback that has been designed specifically for each assignment. This feedback can take the form of a hint in the context of an assignment; this usually allows students to continue learning when they might otherwise get stuck. A practical issue in producing such feedback is that there are many ways in which students fail to understand material and therefore make mistakes; identifying students’ learning needs based on their submissions is challenging. Providing good context-appropriate feedback that addresses all the common student errors requires a great deal of effort by faculty. The publishers of digital textbooks make the investment of identifying common student errors when working on the assignments in the textbooks, and provide appropriate guidance to help students identify and learn from their errors. This helps the publishers amortize the cost of developing these automated assignment graders over all textbooks sold. These digital textbooks are useful for the early parts of programming courses, but in more advanced courses, when students are learning software design and other subtler material, manual grading by faculty is indispensable.

34  Issues That Online Courses Must Address

Textbooks in many other areas also include interactive online labs, enabling students to do assignments in environments that are too complex for faculty to develop for their courses. For example, management textbooks may include interactive business simulations that allow students to understand the consequences of different management decisions. It is difficult for faculty to produce such content, because it is very time consuming and requires specialized technical skills, so it is important to evaluate offerings available from publishers and decide whether to use them. Other software helps faculty with routine tasks such as grading and ensuring academic integrity. For example, the grade books in modern LMSs can make it easy to check grading consistency and calculate weighted final grades, and the availability of originality services integrated into the LMS can help in the prevention and detection of plagiarism. (For more on this, see the next section, “Fostering academic integrity.”) One of the decisions that needs to be made at the program and occasionally the course level is the types of devices that will be supported. Some programs require that their students have specific devices, which simplifies this choice, but potentially adds to students’ costs. Other programs need to support many kinds of devices, complicating development and technical support. There are still significant differences between smartphones, tablets, and personal computers, including screen size, so if a course needs to support all of them this will need to be addressed by the instructional designers. It is wise to choose one or two supported browsers and test the courses on them, because browsers are continually evolving and it can be difficult to maintain compatibility with many of them. The simplest way to address browser compatibility is to ensure that all HTML and style sheets (CSS) validate, which increases the probability that they will render as expected in any modern browser. Similarly, complex computations are commonly embedded in HTML as Java or Python. To prevent compatibility issues between the code in a website and the versions of Java or Python installed on a student’s computer, best practice is to manage the versions that students install. This can be performed as simply as by having a list of recommended software and browser versions. More sophisticated institutions may have web pages that check students’ versions and, if necessary, recommend a solution.

Fostering Academic Integrity It is critical that academic institutions create an environment in which academic integrity is encouraged and enforced. First, it is essential for fostering a learning community in which students know no one has an unfair advantage. Second, it is important to ensure the validity and value of the credentials an institution provides. Finally, emphasizing academic integrity fosters professional and personal integrity in students. Encouraging integrity is primarily the responsibility of faculty.

Issues That Online Courses Must Address  35

It is more difficult to ensure academic integrity online than face to face. The most basic problem is confirming that the student getting credit is the one who is actually doing the work. In a face-to-face classroom faculty can see who is in the room, and they also have many opportunities to engage students in dialogue. In so doing, they get a sense of how well the students are learning the subject matter, and how each student communicates and thinks. This understanding underpins the faculty’s ability to recognize when something in an assignment, “does not sound like Student X.” When faculty teach online, it is harder to develop that understanding, and students know this—and in turn, they have a sense of being hidden behind their screens. As a result, students may be tempted to submit the work of other people as their own. Plagiarism is the most common form of academic misconduct, in both online and face-to-face courses. Helping students cheat in online courses has grown into a major business. There are dozens of “tutorial” websites that provide students with assignment and assessment solutions (Kitahara & Westfall, 2007) and even match students with “tutors” who complete the course material, or even take entire courses for them. These matching websites have made this market very efficient; the cost for a student to hire someone to take an entire course for them may be only a few hundred dollars, depending on the level of the course. These websites are designed to pop up when students search for answers to their assignments or assessments. Legitimate tutorial websites differ from cheating websites because they provide their own assignments and assessments. Some of the illegitimate websites offer solutions to problems in exchange for new graded material from the course (and serve up ads for the paid cheating services). Others also serve as brokers between students and tutors who are paid by students to do their coursework or “contract cheating” (Morris, 2018); they make money both by charging students for the solutions, and by charging both students and tutors when they make connections. Fundamentally, the best way to combat any and all attempts at cheating is to make it easier, or essentially cheaper, to learn the material than to cheat. But what can a program or institution do when any student can pay $300 to hire someone to take an online course for them? The program can make it more difficult by comparing students’ work across all the courses they have taken. The key is to make it prohibitively difficult or expensive for registered students to pay for someone to impersonate them. Institutions can also set their LMS so that each student’s image is embedded in his or her profile and is verified by proctors before exams are taken. Other security options include typing-detection software that creates student profiles when they are applying for a program or taking its first course, and in every course thereafter, randomly asks them to verify their identity by typing a phrase (Young, Davies, Jenkins, & Pflegerd, 2019). In blended courses, on-campus residency requirements make the use of student impersonators more difficult.

36  Issues That Online Courses Must Address

It may still be easy to pay someone to take one course, but it is far more difficult and expensive to find someone willing to take a series of courses or an entire degree. Fortunately, doing this amount of work to avoid schoolwork is rare. At the more common level of everyday cheating on specific assignments, there are several methods that institutions and faculty use to ensure the integrity and value of the degrees which they offer. A first method is randomization of assessments, a technique that makes sharing solutions much more difficult (Cluskey, Ehlen, & Raiborn, 2011; Cizek, 1999). Faculty can create sets of questions on the same topic and specify that the LMS randomly select questions for different students from those question sets. When assessments are implemented using randomization, it is highly unlikely that students will receive the same exams. For example, faculty can create a set of five questions on one topic and specify that each student should receive two of them, randomly selected from that question set. We have been using this technique successfully for more than a decade, and have found that it effectively prevents the sharing of assessment solutions on websites. From our experience, when students know that they are all receiving different assessments, they are less likely to post them, and if any students do post their solutions, these will be of little use to other students. Such randomization can also be used to improve the integrity of regular coursework assignments, as well as assessments. The same LMS mechanism can be used to randomly select assignment problems or essay questions, which makes it much more difficult for students to cheat. A 20-question test, with each question randomly selected from three possible options, will result in more than three billion possible combinations. As faculty gradually build up a corpus of assessment questions, such randomization becomes more powerful. A second method is based on faculty’s ability to determine when the writing style and other characteristics of a student’s submission are inconsistent with what they know and expect from that student, based on previous submissions (Heberling, 2002). Faculty in both online and face-to-face courses learn their students’ communication styles, such as vocabulary usage, length of responses, clarity of expression, and writing style. Wide variations in writing style, such as inconsistent vocabulary usage, may indicate that a student is not presenting his or her own work. Given that different authors preferentially use individual vocabularies and phrases, word-use frequency can be used to identify plagiarized material. More broadly, if a student has been writing at the 10th-grade level, and submits an assignment that is written at the graduate level, the faculty have reason to suspect that the submission may not be the student’s work. One technique for verifying that a student’s submission is their own work is to use an originality verification service such as Turnitin, or search for distinctive passages on Google. Plagiarism can often be identified by inconsistency in specific textual/wordprocessing issues. For example, when students copy and paste, it can result in inconsistent usage of single and double quotation marks, or inconsistent

Issues That Online Courses Must Address  37

placement of punctuation inside and outside quotation marks. When students copy from the web, the copied text often includes HTML tags. These tags may not be visible in a word processor, like Word, but they can be seen in a text editor, such as Notepad. A third method for fostering academic integrity is based on faculty establishing a strong relationship with their students. When students trust that their faculty care about their success in a course, they will reach out to them for help, rather than going to websites that provide easy answers (Morris, 2018). However, it is difficult for faculty to establish this level of trust when there are many students in a course. We have obtained modest scaling by providing students with facilitators, and by training facilitators to build trust with and build confidence in their students. Faculty can also help build this bridge of trust by running synchronous sessions in which they make it clear to students that their first priority is helping students succeed in the course. Unfortunately, some faculty may not know how to do this well, and concentrate more on course requirements than enabling their students’ success. One way that faculty can develop this trust is by explaining their grading criteria clearly and ensuring that students understand the grading rubric, up front. As the course progresses, engaging students in a dialogue, answering questions about the subject matter in a supportive way, and reviewing assignment requirements can also help students feel connected and stay on track. Best practice is to help students understand what their faculty expect of them and what they need to do to succeed. Faculty may even begin this dialogue before a course officially opens, by sending email or a video to their students, welcoming them to the course and guiding their preparation (see Chapter 8). Staying on track is critical, since students are more likely to cheat when they feel that they are so far behind in the course that they have no way to succeed (Wilhoit, 1994), and they tend to fall behind when they do not understand something and are unable to figure it out without help. There are a number of techniques for helping students stay on schedule. One of the most effective is to define milestone deliverables for term projects. Another is to provide them with multiple ways to get answers to their questions promptly. Discussion forums can be created for students to post questions at any time and quickly obtain guidance. Best practice is for faculty to encourage students and facilitators to answer these questions; this often results in productive dialogue and rapid response times. Requiring students to participate in synchronous sessions with telephone calls or videoconferencing meetings provides a fourth method for improving academic integrity (Born, 2003). These dialogues make students aware that faculty know exactly what they know, and this makes them less tempted to cheat. Faculty can also conduct multimedia sessions during office hours during which students can voice their questions or concerns. For these techniques to be effective, faculty need to ask questions to determine if students understand the material. There are a number of ways to do this. One of the simplest is to require students to attend synchronous assessment sessions. These can be classroom

38  Issues That Online Courses Must Address

discussions, reviews of assignment solutions, team meetings, project milestone meetings, student presentations with questions from faculty and students, and other learning activities that help faculty assure themselves that students have learned the material (McNabb & Olmstead, 2009). Best practice is to conduct meetings with recorded audio and video, to help ensure that the students who are participating are the ones who are getting credit for the course. A fifth method for ensuring academic integrity is proctored exams, in which a proctor identifies students and observes them while they are taking the exams. Online assessments can be proctored by contracting with an online proctoring service, which will require students to verify their identity by displaying photo IDs to their webcam, and will have representatives watch students and their screen while they take the assessment. This model has the advantages of not requiring student travel, not taking up scarce classroom resources, and comparatively low cost. A comprehensive proctored final exam can also help identify cheating on previously graded items. For example, if a student does very well on un-proctored tests, but fails a comprehensive proctored final exam, this indicates that the student may have had help on the un-proctored tests. Over the last decade and a half, we have used several proctoring vendors. Institutions should be aware that there are few, if any, stable, high-quality proctoring service vendors, so in the lifetime of a program it may need to change vendors. Courses should be designed so that instructions and elements specific to a vendor can be easily replaced. A sixth method for enhancing academic integrity is to require students to take at least one course on campus, because it is far more expensive for a student to get an impersonator to travel and participate in a face-to-face course than it is online. Impersonating students is also difficult in an era when most of them have a photo history on social media and other websites. For these reasons, some online programs have required that students take at least one course on campus. This is commonly a capstone course, which allows faculty to assess how well the students have mastered the material from prior courses. However, this method is becoming less common due to improved online proctoring and student preference to reduce travel. The choice of the academic integrity measures for a program depends on the subject matter, accreditation requirements, institutional preferences, and available resources. For example, a laboratory course may be designed with required on-campus sessions that include labs and exams. Similarly, a nursing program may require that students participate in a residency where their knowledge and skills are evaluated, or a program in music education can require that students demonstrate their teaching skills in a classroom.

Scaling Online Courses One of the common problems in education is scaling up the number of students in courses without sacrificing the student experience and course quality.

Issues That Online Courses Must Address  39

In online education, scalability limitations become more apparent as physical limitations such as classroom capacity are no longer relevant, so institutions may be tempted to enroll as many students as possible. The traditional approach to improving both face-to-face and online scalability while preserving most of the course quality is to add facilitators or other faculty to help the primary faculty with grading, leading group discussions, and answering student questions.This approach improves enrollment scalability with minimal quality compromise, until the number of students in the synchronous discussions becomes so large that students lose the benefit of interacting with the faculty and may feel uncomfortable participating actively. The quality of the student experience declines with increasing enrollments beyond a certain point, which varies depending on the academic level of the course, the course activities, the course design, and the subject matter (Benshoff & Gibbons, 2011). When course enrollment becomes large enough to discourage student engagement, the traditional solution, both online and face to face, is to organize students into discussion groups, and provide each group with a facilitator to guide discussions. Organizing a course into facilitated groups restores the ability of students to engage productively in discussions with their classmates and faculty. It also helps group facilitators understand their students as individuals, and thus provide them with more tailored guidance and support. Online, we have found that the optimum group size for activities such as problem-solving workshops is only three to five students, while for student presentations or synchronous sessions with instructors it is between 20 and 50. Courses designed with facilitated groups, each containing about 15 students, scale well up to about 100 students in total. Groups of this size are a good compromise between the ability of larger groups to have a variety of opinions and the ability of smaller groups to adapt to student interests and needs. Facilitators also give us more specific advantages, in terms of both practical and academic issues. For example, we have students all over the world, and we have improved our ability to support them by assigning them to groups with facilitators who are in their time zones, thus ensuring speedier feedback. Academically, we can also offer more specialized subject matter assistance; in one of our database courses there is a separate set of assignments for students who are pursuing their concentration in health informatics, and we assign these students to groups with facilitators who are health informatics experts. In this course students can also choose the database system which they wish to use, and we assign them to groups with facilitators who are experts in those systems. One aspect of a positive student experience is their ability to interact one to one with faculty. Such interaction is more important in advanced courses, because students are asked to do research on problems that may not be in a textbook or lectures. As enrollments increase, this takes more faculty time, limiting the number of students that can be taught effectively by each faculty member. We have experimented with having multiple faculty in the same course, and this improves scalability, but it has confused some students, who did not understand

40  Issues That Online Courses Must Address

which faculty to contact with which issues.When enrollments are high, we have found that it works better to offer multiple sections of a course, each with its own faculty. One way that we have been able to improve enrollment scalability in facilitated courses is by augmenting the responsibilities of facilitators.We have experimented with having a lead facilitator, an experienced facilitator who has extra responsibilities that span the entire course, such as checking grading uniformity across groups. Having a lead facilitator in a course has many advantages. If necessary, the lead facilitator of a course can assume faculty responsibilities. If a lead facilitator does very well for many terms they may be appointed as adjunct faculty. (For more on facilitator development, see Chapter 2.) One of the reasons for having a facilitator designated as a lead facilitator is to develop their teaching skills. If a lead facilitator is in training to conduct synchronous sessions and other, more labor-intensive activities, it is helpful to reduce the number of students in his or her group. We have found that if groups have fewer than about seven students, then discussions may lack sufficient diversity of perspectives; therefore we assign at least seven students to all groups. When the faculty and lead facilitator have worked well together, this arrangement has benefited the students by providing them with additional support. However, we have found that if lead facilitators are assigned fewer students, with the expectation that they would perform tasks such as helping needy students in all groups, faculty are tempted to assign them some tasks that they should perform themselves, such as hosting weekly Live Classrooms. We have also developed a method for improving online scalability by adding a facilitator at large. A facilitator at large is usually an adjunct faculty with significant teaching experience, who has not been assigned a group of students or routine grading responsibilities. When more than 75 students are enrolled in a course, we do not assign a group to the lead facilitator, who is designated a facilitator at large. We have also used a facilitator at large when faculty needed additional assistance, such as teaching an online course for the first time. The responsibilities of the facilitator at large are determined by the faculty, and are adapted to the particular needs of the course. One of the more important facilitator-at-large responsibilities is providing individual tutoring for struggling students. Assisting these students can exhaust their group facilitators, but the facilitator at large can provide them with additional, one-to-one tutoring. Facilitators at large often conduct synchronous help sessions for students. Like lead facilitators, facilitators at large are also responsible for identifying grading differences between groups, and reporting them to faculty. Facilitators at large may conduct weekly assignment solution workshops, which are synchronous sessions in which students present their graded assignments to be discussed by everyone. Students are informed that they may contact the facilitator at large with any question. This can reduce the time that students need to wait to get their questions answered, and also provides them with additional perspectives on difficult topics.

Issues That Online Courses Must Address  41

The real limiting factor for enrollment is that as the number of students increases, faculty workload also increases. In a small, traditional face-to-face course, a single faculty member does everything, including lecturing, creating and grading assessments and assignments, and leading in-class activities. This model does not scale any better online than it does face to face, because some tasks need to be performed by faculty for each student. Many teaching tasks, such as grading assignments and leading discussions, can be done by facilitators, but some tasks cannot be delegated, such as student misconduct and situations that may affect grading, so they need to be performed by faculty.

Accommodating Students with Disabilities Students can have many kinds of disabilities, including learning disabilities, mental illnesses, visual and hearing impairment, and impaired mobility. In the United States, disabilities are legally defined, and individuals with disabilities are protected by the Americans with Disabilities Act (ADA). Because of the legal implications of educating students with disabilities, and the complexities of accommodating them, larger institutions generally have an Office of Disability Services, staffed by specialists in the identification and accommodation of students with disabilities. Smaller institutions often hire consultants to provide these services. The diagnosis of disabilities is a clinical specialty, and design of appropriate accommodations is also done by specialists, so faculty must seek qualified advice from either or both when they have students who may have disabilities. This is complicated by the fact that occasionally students will claim to be disabled when they, in fact, are not. The diagnosis of an individual student’s disability should be determined by clinicians, and accommodations that fall outside universal course design practices (see Chapter 7) should be designed by disability services specialists. Because disability services normally require both clinical diagnosis and accommodation design, it can take weeks, or even months, for students who claim disabilities to receive approved accommodations. This constitutes a problem when students disclose their disability while a course is running; occasionally students with disabilities may choose to not disclose them, but then change their minds when they are in a course where their learning is impaired by their disabilities. In these situations, students may disclose their disability to faculty. When these situations arise, the institution’s Office of Disability Services should be informed immediately, and their instructions, which may include interim accommodations, should be followed. The most difficult situations arise when faculty or staff encounter behavior or otherwise observe students who appear to have disabilities, but they have not disclosed those disabilities nor gone through the formal accommodation process. In these situations, the faculty and staff can find themselves in the awkward situation of suspecting a student has a condition that they should respond to, but  they do not have the necessary medical or other applicable training.

42  Issues That Online Courses Must Address

Again, faculty should consult the institution’s Office of Disability Services or other appropriate offices if they have concerns about a student. Disability accommodations are designed by the Office of Disability Services, based on specific disability diagnoses and, potentially, on the course and its learning activities. In our experience, the most common accommodation is providing additional time for assessments. In courses with a hearing-impaired or deaf student, a course that has a learning activity that requires students to interact synchronously in groups may require changes to the way the group operates, such as including a signer.This is similar to what might be done in a face-to-face class, in which such students may be provided with signers, transcribers, and other traditional accommodation specialists. Online, just as all recordings should have closed captions, all synchronous activities should have a text window which can be used by a realtime Communication Access Realtime Translation (CART) provider. Recent versions of synchronous meeting software, such as Zoom, offer transcription of speakers within a few hours of recording. We have found that if a course has closed captioning for all prerecorded video content and CART or automated transcription of synchronous interaction, then hearing-impaired students can do very well. One of the major advantages of asynchronous online education over traditional education is that students can review the online material as often as they need to. Indeed, this can be of great help to some students with learning disabilities, who can miss material covered in a face-to-face course and be unable to catch up. Another advantage of asynchronous online education for students with intermittent medical conditions is that they can study when they are able and rest when they must. If a course has students who are visually impaired, they may need to be provided with auditory and/or tactile substitutes for vision. Screen readers are commonly used by visually impaired students to read the content of webpages aloud, which has multiple implications for course design. All text in course material, including PDF attachments, should be formatted so that it can be interpreted by screen readers; for example, headers should be differentiated from body text. All images and illustrations should also have text descriptions (alt attributes) that can be read aloud by screen-reader software. Videos should include narrative descriptions of what is happening on screen; for example, if a video shows faculty drawing a graph, the narrative should describe in detail what is drawn. When diagrams are central to teaching the subject matter, they should be in a format suitable for tactile graphics output on thermoform plastic, swell paper, or a graphic Braille display.

Marketing Online Programs Marketing of online programs is much more important than marketing of on-campus programs, for several reasons. First, brick-and-mortar institutions are

Issues That Online Courses Must Address  43

known in their local communities, which provides potential students with some awareness of the institution and its programs. Since fully online programs are not associated with a physical campus where students come to study, they have no local market where they have a competitive advantage over programs that are farther away. Second, the online marketplace is more competitive, with extensive market differentiation and direct competition with well-known institutions. Because online programs can be accessed by students everywhere, online markets can have many competing programs. Marketing for online programs can be managed in several ways. If an institution has a marketing department or organization, then it is natural for that organization to assume the responsibilities of managing the marketing of the online program.This can be challenging, because marketing of online programs requires skills that a traditional marketing department may not have. Consequently, most marketing departments outsource some of these skills. The marketing of some programs may be entirely outsourced. Online marketing has a significant effect on the brand of the program and institution, so outsourced marketing must be managed with care, lest it damage the reputation of the institution. Marketing online programs is a skilled discipline that requires personnel with expertise in online marketing, enrollment advising, website design, search engine optimization, and other areas. It can be difficult for smaller programs to assemble the necessary talent. One of the ways to address this issue is by consolidating the marketing for online programs at the institutional level, to take advantages of the economies of scale. Some institutions may find it preferable to work with a commercial online marketing vendor, particularly in the early stages of an online program. A vendor who specializes in online marketing may contribute significantly to rapid early enrollment growth. When an online program is mature, marketing may be performed effectively by an institution. Implementing marketing in-house at an academic institution has significant advantages, including making it easier for faculty to contribute to marketing campaigns, and improving control of the outreach communications and sales processes. Marketing performed in-house may reduce costs; commercial online marketing vendors sometimes charge more than 30 percent of tuition, which is much more expensive than typical on-campus marketing. When marketing is brought in-house, this reduces communication, coordination, and management costs, and improves control of marketing messaging. Institutions can also do things in-house that outsourced vendors cannot easily do, such as having program graduates and faculty speak with prospective students. It is imperative that institutions perform a competitive market analysis before launching an online program. If the analysis determines that a planned program is entering a crowded market, the institution should consider repositioning the program so that it can compete successfully in a less crowded market. As online programs have struggled to find less competitive markets, this has resulted in

44  Issues That Online Courses Must Address

much more market differentiation online. For example, programs are differentiated not only by their academic level, but also by: • • •

•

•

Student career goals. For example, one student may wish to become the IT director of a Fortune 500 company and another may wish to gain acceptance to a PhD program in computer science. Reputation of the institution. When the student earns a degree online, the reputation of the institution is very important. Student preparation. Online programs differ in the required student preparation, and what they do to help students prepare. For example, one master’s program requires that students already have a bachelor’s degree in the discipline, while another program has transition courses to help its students prepare. Theoretical vs. practical emphasis. For example, if students want to take a job right after college they will be interested in learning practical skills, whereas students who want to pursue a PhD in a STEM field will need to have a deeper understanding of the theories underlying the discipline. Cost. The price of a program may differ substantially based on the program’s development and delivery costs, as well as the extent to which it is subsidized or profitable. (A program may be subsidized by alumni, foundations, governments, or the institution itself.) Some programs may be offered below cost when they are first launched, to attract students. MOOCs provide limited opportunities for students to interact with faculty, which lowers costs. High-touch programs provide extensive faculty interaction, and they consequently have higher costs.

If there is an existing face-to-face program that is in high demand, it suggests that the program will also be in high demand online. This assumes, though, that the local region does not have some unique characteristics that make the market unrepresentative. For example, an institution in New England might offer wellregarded face-to-face courses in cranberry husbandry and lobster ranching, but that does not necessarily translate to interest online throughout the world. This also assumes that there is nothing that prevents face-to-face courses from being offered online, such as a tango lesson that requires holding one’s partner. If an institution already offers a program in a blended format, and is considering offering it fully online, reasonably accurate demand forecasts can be derived from the demand for the blended program.The content of many blended courses is mainly online, so the extrapolation of the blended market demand to a purely online demand is straightforward. This assessment can be made more accurate by interviewing students who are taking blended courses and asking their opinion about taking courses in a purely online format.There are also blended formats in which some students take face-to-face courses remotely, using technology such as video conferencing. This too suggests demand for those courses online. Similarly, the demand for a program that is similar to an existing one can be assessed simply by speaking with students in that program. For example, students

Issues That Online Courses Must Address  45

in an information technology program who work in healthcare may ask for an information technology course covering the many unique characteristics of health informatics. Some new courses or programs at our institution have originated in just this way, when students in our existing programs requested them. We followed up on these suggestions by contacting other students in the existing programs, asking them if they would be interested in the new courses or programs, if they thought that their colleagues would be interested, and if they could suggest improvements to the programs. The actual market demand can be significantly higher than estimated by this method, because it only assesses demand from current students; it nonetheless establishes a lower bound on the market demand.We normally follow up these indications of market interest with other techniques, such as determining how often people search for keywords and phrases associated with a program. Another comparatively inexpensive yet effective way to assess market demand is a non-enroll survey. These surveys are conducted by emailing students who expressed interest in a program, but did not subsequently enroll in the expected time period. Non-enroll surveys ask questions such as whether the student plans to enroll, whether he or she enrolled in another program, and what programs and courses students are interested in. We have found non-enroll surveys to be very helpful in evaluating proposed curriculum changes, such as new concentrations. Non-enroll surveys may also inform programs about students they have lost to competing programs. If a program is losing many students to competing programs, this may prompt changes to the marketing or the program to make it more competitive. Market research may reveal a specific market for a variation of a program, such as one targeted to students with certain backgrounds or career goals.When market research identifies underserved markets that are not very different from the markets that a program was planned for, it is often desirable to adapt it to those markets (Bontrager, 2004). For example, students in a technical program may express an interest in the management aspects of the field; when market assessments confirm this demand, the institution may extend the curriculum to include management subjects. If an institution has no current face-to-face program in an area, and is still considering launching an online program, it would be well advised to conduct a formal market demand assessment. Some large institutions have the resources that this approach requires, but most need to hire consultants or an online education marketing company. Online education market assessment is difficult because of sampling biases and other confounding factors, such as the inability to identify and reach prospective students to get them into the pipeline (For more on this pipeline, see Chapter 5). Competitive landscape analysis is an essential part of this assessment, because even if the market is large, established players can make it difficult to take market share. However, the results of market demand assessments and competitive landscape analyses often identify underserved or niche markets.

46  Issues That Online Courses Must Address

A word of caution Institutions need to be aware that it is important to assess the program demand, both because the enrollment demand can be less than is needed to support the program and, rarely, because the enrollment can be higher than the institution can support. A college at Boston University experienced this latter phenomenon some years ago, when they launched an online PhD program. They knew that the demand was not very large in any one geographic area, but they soon discovered that the aggregate demand was much larger than they expected, and the program filled to its enrollment cap more quickly than they expected. Unfortunately, the cap had been set too high, and faculty were strained by supervising all the doctoral theses. In time the pent-up demand was met and enrollments declined to a sustainable level.

Market analysis for an online program should begin as it is being defined, to gauge student interest in the program and its variants. Market analysis should include competitive landscape analysis, because the competitive landscape may help those who are defining the program to improve its differentiation from its competitors. A competitive landscape analysis traditionally includes a table, with one row for each competing program and columns for their key market attributes. These attributes typically include program enrollments, admissions requirements, cost, institutional ranking, and any distinguishing competitive characteristics, such as internship programs, career services, or specialized concentrations. Analysis of the competitive landscape often leads to decisions to change program focus, to differentiate the proposed program from strong competing programs. Competitive landscape analysis should produce a statement of what the program provides that is of value to students that other programs do not provide as well.This analysis also identifies key aspects of the program that can be used in marketing, such as self-paced courses. Most larger institutions have marketing departments that can prepare a competitive landscape analysis, given the description of the proposed online program. However, at smaller institutions, the administration and faculty may have to do their own research by searching online for competing programs.

Danger: Marketing websites Be aware that there are marketing websites that are paid by schools to present their programs and not others. We recommend original keyword searches, because relying solely on marketing websites could be misleading, and one might not learn about important competing programs.

Issues That Online Courses Must Address  47

Marketing budgets should be established as part of the original program proposal. If the budgets are not adequate, the institutions should consider whether they should make the other investments necessary in the program. Marketing budgets depend strongly on the marketing channels that will be used. If a program is marketed using existing institutional websites and native search, the cost can be low. If online programs are heavily marketed using online advertising, marketing budgets can exceed 30 percent of tuition revenue. Another consideration that will affect the budget includes how leads will be managed. If applicants complete their forms online and submit them to the admissions committee with no human involvement, the cost of managing the leads will be less than if enrollment advisors are hired to help the prospects complete the applications. (For more on enrollment advisors, see Chapter 5.) Furthermore, if a well-designed application pipeline is to be established, software such as a specialized customer relationship management (CRM) system may need to be procured to manage the communications. Most programs are initially marketed by companies that specialize in marketing online programs. These companies may handle all aspects of marketing. Institutions need to maintain control of the messaging of these companies to protect their brand. Marketing is usually organized into campaigns.They may be as simple as marketing a program for a semester, though campaigns usually have emphases that vary, such as test marketing in a new channel or testing new marketing collateral. One of the important activities in a campaign is evaluating its effectiveness. (For more on this, see the next section.)

Developing Marketing Collateral Once the marketing strategy has been defined and the program launch decision has been made, marketing collateral must be developed. Marketing collateral is the information and material used to communicate about a program. This includes ads that appear when certain search terms are used, webpages that are linked from ads, program and course descriptions, application forms, and videos. Marketing collateral is usually targeted to particular cohorts of prospective students. Some is designed specifically to lead prospective students to landing pages in the program website, such as links in emails, search engine ads, and links in other websites. At a minimum, the program’s website should describe the program, how to apply to it, the costs, the course schedule, information about the courses, degrees, certificates, diplomas offered, and program faculty. An effective website will systematically convince prospective students that this is the best program for them. One of the key techniques in developing these websites is testing their effectiveness. An online program requires the development of websites with many landing pages that are tailored to specific interests of prospective students, to stimulate their immediate interest. Best practice is for the links to preserve the information about the interests of the prospective student represented in the context of the link to the landing page. Different versions of web pages may be

48  Issues That Online Courses Must Address

developed and marketing analytics used to determine their effectiveness. The effectiveness of online marketing can vary significantly, so it is important to analyze and tune it to maximize cost efficiency. This analysis should ideally be followed all the way from the source of leads to the quality and number of students, though it is commonly performed only from collateral to leads. Many online lead sources can be expensive; they may include both paid and unpaid search results, purchased search engine ads, purchased email lists, and referrals. Marketing analytics identify the contributions of each source of leads, to help the marketing campaign decide how to allocate the resources between search engines, social media, and other sources. We have tried many different media, and have generally found that search engine ads are the most effective, largely because our prospective students make extensive use of search engines when looking for schools and programs.We have found ads on social media sites such as LinkedIn to be less effective, but still worthwhile. These strategies scale down well, because charges for search results or ads occur only when someone clicks on them. Faculty with administrative responsibilities should also participate in the marketing campaign by checking the correctness of the collateral. Some faculty may also appear in videos and other media to make their courses more appealing to potential students. Another effective way of encouraging students to enroll is to include testimonials from students and alumni. Because marketing is complex and expensive, its effectiveness is difficult to assess without formal evaluation methods. Typical metrics are: • • • • • • •

the number of leads identified by channel over time quality of prospects, such as their average GPAs the number of applications resulting from the leads the number of admissions resulting from the applications the number of starts resulting from the applications the number of successfully completed terms; this is related to revenue the graduation rate.

All of these metrics vary per channel and per marketing campaign or other time windows used for evaluating the effectiveness of marketing. Many of these metrics are not available for some time. For example, the graduation rate may not be known for five years after a student takes their first course.This makes marketing evaluation challenging and tends to shift the focus to the earlier metrics, such as the number of leads and starts. A number of metrics are derived from this data including: • •

The cost per lead, including purchased items such as purchased search terms and email lists, but not including other costs such as content development. The cost per start, which is higher than the cost per lead, because not all leads complete an application, not all applicants are admitted, and not all admitted applicants start.

Issues That Online Courses Must Address  49

•

The conversion rate from prospects to starts. This is important because if there are many applicants who do not start, it increases the cost of managing the pipeline.

Student Data Security and Privacy Most institutions have a student information system that serves as a repository for student grades and other private student data, and policies that govern the use of and access to that data. When courses are taught online, the grade information for each course is accessible in various places, such as the grade center of the LMS, the database that stores the data for the LMS, the web servers, and the network, so their security must be assured. One of the basic security provisions is storing student data in a well-secured database. There are additional operational considerations in managing access to online student data. Course development may involve many people who should not have access to such data, so access should be restricted as part of preparing courses for launch.They must be identified and removed from the course before it runs, or at a minimum, their privileges to access student data must be revoked. This should include removing faculty and staff who do not need access while the course is running, so that the only people besides students enrolled in the course are the relevant faculty and the necessary support and design staff. If the LMS supports it, another way to manage access is to define privileges and roles for development staff that do not allow access to student data. For example, course editors and reviewing faculty may be assigned a special role that allows them access to all of the course content, but not student submissions or grades. Access to archived courses should be controlled in the same way as access to running courses.

Summary Checklist ☑ ☑ ☑ ☑ ☑ ☑ ☑

Choose the instructional technologies. Decide how to foster academic integrity. Identify ways to scale courses and programs. Determine how to accommodate students with disabilities. Create a plan for marketing the online program. Create a plan for developing marketing collateral. Address data security and student privacy issues.

References Anderson, C., Finn, J. D., & Campion, L. E. (1963). Technology in American education 1650-1900 (United States. Office of Education. Bulletin 1962, no. 19. New media for instruction, 1). Washington: U.S. Dept. of Health, Education, and Welfare, Office of Education.

50  Issues That Online Courses Must Address Benshoff, J. M., & Gibbons, M. M. (2011). Bringing life to e-learning: Incorporating a synchronous approach to online teaching in counselor education. The Professional Counselor, 1(1), 21–28. Bontrager, B. (2004). Strategic enrollment management: Core strategies and best practices. College and University, 79(4), 9–15. Born, A. D. (2003). Teaching tip: How to reduce plagiarism. Journal of Information Systems Education, 14(3), 223–224. Cluskey, G. R., Jr., Ehlen, C. R., & Raiborn, M. H. (2011). Thwarting online exam cheating without proctor supervision. Journal of Academic and Business Ethics, 4(1) 1–7. Cizek, G. J. (1999). Cheating on tests: how to do it, detect it, and prevent it. Mahwah, NJ: Lawrence Erlbaum. Heberling, M. (2002). Maintaining academic integrity in online education. Online Journal of Distance Learning Administration, V(1), 1–6. Hiltz, S. R., & Turoff, M. (1978). The network nation: Human communication via computer. Reading, MA: Addison-Wesley. Kitahara, R. T., & Westfall, F. (2007). Promoting academic integrity in online distance learning courses. MERLOT Journal of Online Learning and Teaching, 3(3), 265–276. McNabb, L., & Olmstead, A. (2009). Communities of integrity in online courses: Faculty member beliefs and strategies. MERLOT Journal of Online Learning and Teaching, 5(2), 208–221. Morris, E. J. (2018). Academic integrity matters: five considerations for addressing contract cheating. International Journal for Educational Integrity, 14, 1–12. Wilhoit, S. (1994). Helping students avoid plagiarism. College Teaching, 42(4), 161–164. Young, J. R., Davies, R. S., Jenkins, J. L., & Pflegerd, I. (2019). Keystroke dynamics: Establishing keyprints to verify users in online courses. Computers in the Schools, 36(1),1–20.

Chapter 4

Program Definition and Development

This chapter is written primarily for faculty, administrators, and others involved in program definition and development. Designers may find this chapter useful for understanding scheduling, development, instruction, and interaction models, program marketing, and forecasting. This chapter covers: • • • • • • • • •

Defining program goals. Selecting a course scheduling model. Selecting a course development model. Organizational design to support faculty who teach online. Selecting modes of instruction. Interaction models. Selecting synchronous teaching technology. Developing a marketing plan. Enrollment forecasting.

Online programs succeed when instruction meets the needs of their students. Online instruction offers many choices, including scheduling, development, instruction, interaction, and synchronous teaching models; administrators need to understand them and select the ones that offer the best fit for the needs of their institution and their students.

Defining Program Goals A well-developed program has clear definitions of the goals, context, and sequence of courses to be taught, as well as its target audience. For example, a program in baseball analytics would have a goal of providing its graduates with both a broad knowledge of the role of analytics in the context of the game of professional baseball, including their history and development, and a specific understanding of analytic techniques and applications. The coursework would provide a foundation of statistics, algorithms, and data

52  Program Definition and Development

modeling. The target audience would be students looking for positions as baseball data analysts, executives, or agents. When developing a curriculum, faculty should be aware of the content of all the courses in the program, to try to overlap the content just enough for students to integrate the new material with their previous knowledge, but not so much that it wastes their time. For example, many psychology and economics courses use statistics, so each course might cover introductory statistics. If a curriculum is designed with an introductory statistics course that works for all social sciences, this wasteful overlap can be eliminated.

Thomas Cavanagh, vice provost for digital learning, University of Central Florida In many cases (e.g., nursing), program goals and learning objectives are determined by accreditation requirements. In cases where accreditation requirements do not exist, expert faculty teams work together to ensure that all critical elements are included in the program curriculum.

Selecting a Course Scheduling Model A scheduling model provides a temporal framework within which courses are offered. One of the critical decisions in designing a scheduling model for a program is the length of the terms. Traditional scheduling models are based on fixed-length terms; they define when the terms begin, when they end, and various dates during the term, such as the last date students can drop and get a refund, and the final exam dates. Common term lengths are a full semester (14–16 weeks), a half-semester (seven weeks or less), and an intersession (four weeks or less), with each one still covering the same amount of material, and offering the same number of credits. There are also scheduling models that are not based on fixed-length terms, such as those for self-paced courses, workshops, and many MOOCs, but they are far less common. An institution should choose the appropriate model or combination of models for particular programs. It is difficult to mix courses with different term lengths in the same program, because students may wish to take a course with a longer term concurrently with a course with a shorter term and may thereby be overloaded when the courses overlap. The digital strategy may address how the scheduling of the terms of online courses relates to the terms of face-to-face courses. For example, some institutions have digital strategies that allow for two online terms within each on-campus term. If a program has full-length terms, then students who are taking a full course load will take more courses each term than if they were half-length terms. This is illustrated in Figure 4.1. The first case in the figure illustrates how a student

Program Definition and Development  53

Figure 4.1  Schedules with different models.

takes a full course load of four courses in a 14-week term. Case 2 illustrates how a student takes the same four courses in a compressed seven-week term. Case 3 illustrates how a student takes a half load in a 14-week term. Case 4 illustrates how a student takes a half load in a seven-week term. Case 5 illustrates how a student takes a quarter load in a 14-week term, by taking only one course.There is no corresponding quarter load in a compressed seven-week term, because a single compressed course represents a half load. This can be a problem for students with limited time or learning disabilities. Case 6 illustrates what happens when a student takes a 14-week course and a seven-week course in the same 14-week term; during the second seven weeks they have twice the workload of the first seven weeks.

54  Program Definition and Development

An advantage of longer terms is that students have more time to master the material, which can be particularly important for students with learning disabilities (as noted earlier, extra time for assessments is the most common accommodation for such students; see Chapter 3). Longer terms are particularly advantageous in advanced courses or courses with large projects. Another advantage of longer terms is that students have more time to recover from illness and other events that may disrupt their studies during the term. A disadvantage of longer terms, however, is that students need to take more courses at a time, or take longer to complete their studies. Shorter terms also have advantages: • • •

•

Intense, concentrated courses may appeal to students who like to focus on one course at a time. Shorter terms allow students to pause their studies and quickly return to them. Shorter terms support smaller courses such as transition courses and prerequisites better than longer terms, because they do not require as much time commitment from students, or force them to wait as long to begin their studies. Shorter terms also work well for fine-grained courses, which are often a better fit to student needs than longer courses. For example, a traditional undergraduate course covers many topics, and only a few of those topics may be needed as prerequisites for more advanced courses. The topics needed as prerequisites can be packaged in a short course.

A common scheduling model is a “carousel,” in which courses come around periodically, like horses on an amusement park carousel, with students getting “on” and “off ” the courses. A course may occur more than once on a carousel. Carousels commonly repeat annually, with each course coming around at least once per year.This model is often used in low-enrollment programs, because only one or a few courses need to be offered each term. One advantage of carousel schedules is that it is easy to guarantee that each course will be offered at least once per year. Another advantage is that it is easy to schedule course sequences so that students can take them without breaks. However, programs with carousel scheduling may need to find ways to help students who need to skip a course that they want to take (“get off the carousel”), so that their studies are not unduly delayed while they wait for their skipped course to come around again. When enrollments in courses become too high, best practice is to offer the courses more often, to maintain desirable student/faculty ratios. This also increases students’ course choices and reduces scheduling problems. When all courses are offered in all terms, these scheduling problems no longer exist. Unfortunately, few programs have high enough enrollment to offer all courses in all terms, so it is common to offer more popular courses more frequently and

Program Definition and Development  55

less popular courses less frequently.This works particularly well in programs that have core courses and concentration electives, because core courses can be offered each term and the concentration electives less frequently.When program enrollments are high enough, the core courses can be offered every term, the higher-enrollment concentration electives several times per year, and the lowerenrollment concentration electives once per year. When course enrollments are high enough, a scheduling model can be used in which courses do not close and are updated while they run, with students progressing through the courses in cohorts, all studying the same material at the same time. For example, Boston University uses this scheduling model for a required employee laboratory-safety course. It allows students to start their studies more frequently, rather than having to wait for a course to be offered, and progress at their own pace. With high enough enrollments, the students can be organized into groups that are studying the same material, which improves learning.This design also allows struggling students to drop back to a cohort that was formed later, without having to drop the class or get an incomplete. Implementing this model in a face-to-face program would require high enrollments, many faculty, and many classrooms, which is not generally feasible for most programs. Another advantage of online technologies is that everyone in the class can receive personalized feedback on different topics at the same time; in the classroom this is not feasible. One way to take advantage of this flexibility is to allow students to progress through online courses at their own pace; however, this has the disadvantage of not supporting cohorts of students. There are many advantages to having such cohorts, including the ability to conduct discussions and synchronized activities, such as grading and release of assignments. It also allows students to build camaraderie and connection, which is far more difficult when they are all progressing independently.

Selecting a Course Development Model One of the first decisions that faculty and program administrators must make is: Who will be responsible for the instructional design, i.e., the look and feel of the online courses, and for the practical details of implementing the course content in the LMS? The academic content will usually be provided by faculty, but how that content is designed and implemented will affect budgets, staffing requirements, and the quality of the result. There are broadly three different models of course development. In the first and most popular, the faculty design, develop, and implement the content largely by themselves.The main advantage of this approach is its low cost, but it also has the advantage of enabling faculty to quickly change course content themselves. If faculty receive adequate training in the design and implementation of online courses, this approach can produce good courses.

56  Program Definition and Development

However, this approach has many disadvantages. It requires the most faculty time, which is generally a scarce resource. Consistency can become an issue, as courses produced by different faculty may have different navigation and presentation styles, so that students may be forced to waste time learning the different organizational style of each course. The course quality itself can also be undermined, depending on the faculty’s ability to develop and present their subject material online. For example, a French-language faculty may be fluent and knowledgeable, but if she designs her vocabulary exercises in such a way that students are forced to click on words just to see them, their learning will be compromised. An additional concern is that some faculty may use third-party technologies that are not supported by the institution. For example, one of our faculty liked to register domains for each of his graduate courses to hold course materials, and this was not supported by the institution. Unfortunately, he failed to notice that one of these domain registrations had expired, and the URL had been reassigned to an adult entertainment site. Naturally, this was discovered by a student. If institutions choose this development model, they should also select an LMS with a what-you-see-is-what-you-get course-editing interface. This has the advantage of making it straightforward for anyone to insert content with minimal training, which is good when the production model has the faculty building and maintaining their own courses. However, this limits an institution’s ability to customize the look and feel of its courses and extend the functionality of the LMS software. This can be especially important for presenting math or computer code, in which external software for formatting may be important. The second approach is a “publishing” model, in which faculty are treated like authors writing for a publication. They write the course content, include their sketches, notes, and ideas for media objects, and forward their work to staff members, who format the text and build media objects for the course before publishing it in the LMS. This approach usually employs a course template and a house formatting style that ensure a consistent look and feel across courses, so students do not have to waste time learning new styles, and can always find course elements in the same places. While the cost of the support staff can be significant, this form results in courses with consistently higher technical quality. The disadvantage of this approach is that pedagogic subtleties and content detail may be changed, potentially resulting in content errors and a pedagogically weaker course. This can be overcome if faculty review the material before launch, so that any weaknesses introduced during implementation can be identified and corrected. Common examples include errors in figures, incorrect typesetting of mathematical expressions, typos, and missing material. The third approach regards faculty as scarce resources who outline the course and describe the course content. Staff members write the actual text of the

Program Definition and Development  57

course, build media objects, and insert them in the LMS. This model is popular with for-profit institutions, because of its low cost. The most noteworthy weakness of this approach is that course content is written by staff who may not have a deep understanding of the subject matter or pedagogy. They may introduce weaknesses that would never have been present if the content were written by more qualified authors. This weakness can be ameliorated by having faculty with a deep understanding of the subject matter review the course, updating it as necessary. If the staff writers have a good general understanding of online pedagogy, they may produce content that is pedagogically reasonable, even though it may be somewhat incorrect. Because faculty may find it easier to correct errors than write from scratch, this approach can benefit greatly from the “stone soup” method. (For more on “stone souping” faculty, see Chapter 7.)

Organizational Design to Support Faculty Who Teach Online In most course development models, faculty require a high level of support to teach well using online technology. Most organizations have retained their legacy centers that support face-to-face education, and created new, separate offices of distance education.The primary reasons have been organizational inertia, risk reduction, and cost. Unfortunately, developing online courses requires much more time and other resources than developing face-to-face courses (McIsaac & Craft, 2003), and legacy centers may not have the resources necessary to support online development. Another problem is that the offices and centers serve significantly different faculty communities; faculty who teach primarily online have to learn to be proficient with educational technology, unlike faculty who teach only face to face. Consequently, most institutions with significant online programs have an office of distance education that is specifically responsible for assisting faculty in the design and development of online courses.This office may offer training in online technology, create course frameworks and templates for consistent design and navigation, and provide staff to design, build, edit, and support courses. At Boston University, the Office of Distance Education (ODE) supports online programs, while legacy offices support face-to-face courses. When we develop an online course, we often design the course website so that it also supports the face-to-face version of the course, which results in stronger courses and a more uniform curriculum. Admittedly, this can complicate the relationship between ODE and the legacy centers. One downside of face-to-face courses using websites is that it complicates the relationship between ODE and the legacy centers, as the latter often do not have the time or skills to make updates to courses created by the former. For example, an online course may include video conferencing that would not

58  Program Definition and Development

be used face to face, or automated learning activities designed for individual students, such as formative evaluation questions (for more on learning activities, see Chapter 6). Smaller, more agile institutions may be able to combine centers that support face-to-face education and offices that support online education. This helps them attain adequate scale to provide the diversity of services needed. For example, this allows a smaller institution to hire instructional designers and support staff. Another advantage is that this provides an integrated point of support for faculty, which is important when they are learning about online, and developing and teaching blended courses, given the large amount of overlap between the two. (For more on blended courses, see Chapter 1.) In institutions of any size, an office of distance education can take a leadership role in fostering a community that is experienced in online education, and disseminate its skills and practices throughout the institution. This can also help overcome political barriers within the institution, encouraging broader adoption of online teaching. Centralization also provides economies of scale, which can help spread workloads, such as peak loads during new program development, over more staff. Those economies of scale may also enable skilled specialists such as animators, videographers, copyeditors, and illustrators to be hired by institutions with the appropriate resources. And finally, the economies of scale may allow institutions to pursue more advanced technologies and to hire better management. Smaller institutions may be best served by contracting for many of these services.

Selecting Modes of Instruction Once a broad course development model has been chosen, faculty and designers must select the best combination of synchronous and asynchronous teaching methods for each topic. Online instruction generally includes both synchronous and asynchronous components, though there are courses that offer only one or the other. The optimal mix depends on the subject matter, faculty preferences, and student preferences. One advantage of asynchronous components is that they can reduce faculty teaching effort over many terms, although significant effort is required initially (Fishman et al., 2013). Asynchronous content can bring the knowledge of senior faculty to more students, by having them develop a course and then having junior faculty teach it. Since asynchronous components stay the same from one offering to the next, this facilitates course evaluation and updates. If a course has multiple contributors, each of them can update components covering their areas of greatest expertise. Another advantage of asynchronous content is that students can extract it and review it offline as often as they wish. Synchronous education has many good points as well. In synchronous interaction, faculty can adapt to their actual students’ needs, instead of the needs of the

Program Definition and Development  59

students they were expecting when they wrote the asynchronous course. Such adaptation may include reviewing prerequisite material if it seems inadequate or to be missing the mark, choosing different examples that resonate more with students, defining unclear terminology, and/or providing material that is more advanced. Synchronous interaction with faculty and peers motivates students, because they can communicate more naturally. Synchronous communication is also more effective at developing peer-learning communities, because of the easier communication and emotional support that it provides (Hrastinski, 2008). Synchronous communication tends to be more informal, and can provide more opportunities for faculty to interact spontaneously with students. This is particularly important when dealing with material that may be challenging for students.The ease, speed, and informality of synchronous interaction results in more ideas being explored and misunderstandings corrected, and thus is more effective for instruction and feedback (Mason & Bacsich, 1998). If the participants accept that misunderstandings are an important part of the learning process, that acceptance encourages risk taking, and speeds and strengthens learning. Finally, it improves academic integrity, because faculty can get to know how individual students comprehend the material and their communication style, which makes it harder for them to submit others’ work as their own (for more on academic integrity, see Chapter 3). One of the things that makes teaching synchronously seem easier is that an audience of a live performance tends to be more forgiving than an audience of an edited work. Students expect asynchronous material to be polished, like a textbook or film, but they expect synchronous material to be more spontaneous. Synchronous teaching allows faculty to adjust “on the fly.” Most faculty will find it easier to teach a course face to face first, before converting it to an online course, because when they teach face to face, they learn which pedagogy works, and which does not. However, no one approach is best for all students. Many students benefit from synchronous interactions with faculty, while other students learn most effectively by interacting with others via the course website, or by working independently with the course content. One example is an asynchronous course that covers all the material, combined with multimedia synchronous sessions to support the students who learn most effectively by listening, asking questions, or otherwise interacting more directly with faculty and/or their fellow students (Boling et al. 2012). Many students have practical difficulties attending scheduled synchronous activities, so best practice would be to record the synchronous sessions and provide multiple opportunities for students to interact synchronously with their faculty and peers. Best practice is for asynchronous content to cover all of the assessed material, and to also provide several hours per week for students to interact synchronously with their faculty and fellow students. Embedding synchronous activities within an asynchronous course works well pedagogically, but even with the asynchronous component, it can be challenging to schedule synchronous activities that both faculty and students

60  Program Definition and Development

can attend. This problem is compounded when a course has students in widely distributed time zones. We have addressed these challenges by conducting synchronous sessions at two widely separated times on weekends, so that students in any time zone can attend at least one session.

Interaction Models Faculty need to choose the ways that online students communicate with faculty and with each other. There are many models for this interaction, and faculty need to choose appropriate ones for their courses and subjects. High-touch online programs, such as those at Boston University, provide extensive interaction between faculty and students, both synchronously and asynchronously. Institutions with fewer resources may offer similar levels of high-touch communication between faculty and students in some courses, but they provide less assistance to faculty. Massive open online courses (MOOCs) are low-touch, providing little or no communication between students and faculty. While high-touch models are good for students, the realities of faculty workload limit the number of students that can be supported by each faculty, and this affects the cost of the program. There is a tradeoff between faculty engagement while a course is running and course development cost: Lower levels of faculty involvement when a course is running require corresponding increases in course development effort, to anticipate and address the many questions and issues that would normally be addressed by faculty during the course. Faculty and designers need to find the best methods of interaction, which can depend on the level of the course, the subject matter, the students, and the target program cost. For example, in the School of Social Work at Boston University, several graduate-level courses involve a great deal of role-playing among the students, with some students playing the social workers and others their clients; these are monitored and graded by faculty. This interaction model allows each faculty to supervise several students at a time. A more expensive model would involve the faculty role-playing the client with each student individually. In either case, a high level of synchronous interaction is necessary and course size is necessarily limited, which would be reflected in the cost of the program. It is important to put the choice of interaction models in historical perspective. Educators have long sought ways to use technology to reduce the cost per student, and thereby improve access to education. The key is using technology to enable each faculty member to teach more students. MOOCs are the most aggressive effort so far to minimize faculty involvement. Interest peaked around 2012, when major universities were jumping on MOOCs as the next revolution in education. Early MOOC development efforts were heavily subsidized by foundations, schools, and governments because of their promise to dramatically lower the cost of education.This has been followed by a major trough of disillusionment, continuing to this day, driven by the low completion rates (often below 5 percent), questionable academic integrity, variable

Program Definition and Development  61

student satisfaction, learning outcomes compromised by the lack of personalized faculty feedback, variable quality curriculum design, and the high costs of both developing high-quality MOOCs and of running them with many students. This disillusionment was furthered by the growth of a number of for-profit organizations that offered MOOCs with low academic standards. The ongoing development of MOOC courses is demonstrating ways that technology and new course designs can address these issues and improve scalability, while also providing courses that are of acceptable quality to students. For example, a friend of one of the authors is taking a MOOCish online graduate course at a well-known school. The course has more than 700 students, two dozen teaching assistants, and one professor. Assignments are graded automatically, with an appeal process that places the submission in a queue to be reviewed by a teaching assistant. The course also lacks flexibility, so advanced students are required to do assignments designed for less advanced students. Students accept these limitations because the course is inexpensive. Substantial progress has been made in technical ways to reduce faculty effort when it comes to grading and providing feedback—other than hiring more teaching assistants or facilitators. For example, syntax and other simple errors in computer programs can now be graded automatically. Progress is also being made in the application of natural language processing and artificial intelligence to the grading of text-based answers to assessment questions. Big-data technologies, such as machine learning, have been successfully applied to high-enrollment MOOCs: Faculty grading of representative student submissions is applied to grading the rest of them. Many well-known institutions developed MOOCs, but without subsidies, the MOOC model has proven difficult to sustain. High-enrollment MOOCs require many people to keep them running, so they need a revenue model to support them. Different sustainable revenue models have been tried for MOOCs. Some MOOCs have charged low tuition fees and additional fees for a certificate of completion that is based on a proctored final exam; this model provides insufficient revenue to provide a high level of facilitator support, but it can cover the hosting and proctoring costs. MOOCish courses can charge higher tuition than a traditional MOOC, and provide higher levels of faculty and facilitator support. Most MOOCs were developed as standalone experiments, not part of a well-designed curriculum, and this has also limited their appeal and academic quality. One hybrid interaction model that has succeeded is based on programs that begin with low-cost, MOOCish courses covering introductory material that does not require much faculty participation, and then continue with fullcost, advanced courses that offer higher levels of faculty support (see Chapter 7). This essentially recreates the traditional big-school face-to-face model in which introductory courses have many students, with the professors delivering lectures in large lecture halls, but rarely interacting with individual students until they reach smaller, more advanced classes. This model has been used in micro-master’s programs.

62  Program Definition and Development

Selecting Synchronous Teaching Technology The choice of technology or platform that will be used to support a program’s synchronous teaching elements should ideally be made at the institutional level. Most institutions, whatever their size, have some sort of tech support office (at Boston University it is the Department of Information Services and Technology), and they would usually be the ones to examine the various options. Occasionally, individual faculty will pick a conferencing system to use for their own courses, but this is rare. Administrators should not think that simply providing faculty with a conferencing system will make them successful in synchronous teaching. They will require support, both technical and pedagogical, and the level of resources available for this support should be made clear early in the program development process. While it is possible to run courses without synchronous components, this assumes that all of the students will be able to learn the material from the asynchronous content alone. Synchronous learning activities improve course quality because they: • • • •

Give students and faculty a way to quickly and efficiently ask and answer questions. Give faculty an additional opportunity to encourage students and learn where they are struggling. Provide another mode of interaction to accommodate diverse approaches to learning. Help build a learning community in the course.

To teach well online in a synchronous format, faculty need to be confident, not only in their mastery of the subject matter, but also in their ability to present well in these sessions, that the technology will perform for them, and that they will be well supported as they teach.

Developing a Marketing Plan Online programs need to be marketed. In Chapter 3 we described the market assessments needed to support the decision to launch an online program. In this section we describe how to plan and implement the marketing that will help bring students to the program. Most online programs benefit from a marketing plan, which is a document outlining the key considerations and assumptions upon which the marketing will be built. The marketing plan should define the target audience and how to reach it. It should also define the overall strategy, including budget and spending plan, how the marketing pipeline will be managed, and how communications will be managed (for more on this pipeline, see Chapter 5).

Program Definition and Development  63

The overall marketing strategy should define which marketing activities are outsourced, how marketing collateral will be developed, what marketing channels will be used, how the marketing budgets will be managed, and how admissions will be done. Communications is central to marketing. It begins with outreach to prospective students, continues when enrollment advisors communicate with leads, is formalized when applicants complete their applications, and, ideally, continues as students study and after they graduate. Because different people are responsible for marketing, enrollment advising, admissions, retention, and alumni, several handoffs are involved in this communication.

Enrollment Forecasting Having described in Chapter 3 how program enrollments are forecast from market data before a program is running, we now describe how more accurate operational forecasts are derived and used when a program is running. Accurate forecasts help administrators determine how many facilitators, teaching assistants, groups, or sections are needed. They also help administrators decide whether to create additional sections of high-enrollment courses, or to cancel low-enrollment courses. Many resources need to be finalized shortly before launch, so forecasts that grow in accuracy as the launch date approaches are highly desirable. The forecasting error of the algorithm described below falls to zero as the launch day approaches. The problem of accurately forecasting enrollments is similar to the problem of accurately forecasting passengers, baggage, and freight for airlines and ships, and we have adapted the most common algorithm (Curry, 1990). It is based on the observation that the percentage of students who have enrolled some number of days before the launch date is very stable for the same term from year to year. For example, if 30 percent of the students enrolled 40 days before launch in the spring term last year, then about 30 percent of the students will have enrolled 40 days before launch in the spring term this year. To forecast the launch-day enrollments for a term we extract the history of enrollments from our student information system for a reference term, which is typically the same term one year earlier. For each number of days before launch we calculate the current-to-launch-day-enrollment ratio by dividing the number of students enrolled that many days before launch by the launchday enrollment. This ratio is zero before enrollment opens, and gradually approaches one on launch day. For example, if the launch-day enrollment for all courses in a program was 1,000 and the enrollment 20 days before launch was 800, then the current-to-launch-day-enrollment ratio 20 days before launch is 0.8. We derive the launch-day forecast for the current term by dividing the current enrollment by the reference term current-to-launch-dayenrollment ratio, for the same number of days before launch. Continuing the example, if this year’s total program enrollment 20 days before launch is 900,

64  Program Definition and Development

then the forecast launch-day enrollment is 900/0.8 = 1,125. We also use this ratio to forecast individual course enrollments. For example, if the enrollment in a course 20 days before launch is 100, then the launch-day enrollment forecast for that course is 100/0.8 = 112.5. This can be expressed in mathematical terms: Let rn be the current-to-launch-day-enrollment ratio n days before launch for the reference term, and en be the enrollment n days before launch in the reference term. Then

rn =

en , e0

where e0 is the reference launch-day enrollment. Let En be the current term enrollment n days before launch, and Fn be the forecast launch-day enrollment for the current term, based on the enrollment n days before launch. Then

Fn =

En rn

Similarly, let Gn be the enrollment in a course n days before launch, and Hn be the forecast launch-day enrollment for the course, based on enrollments n days before launch. Then

Hn =

Gn rn

A number of things can be done to optimize the accuracy of the forecast: •

It is important to use enrollment histories from the same term in the academic year, because enrollment histories vary significantly for different terms. Students tend to prefer to start classes in the fall term, so they enroll earlier than in other terms, whereas students tend to enroll later for the spring term because of the winter holidays and campus shutdown in December. • It is important to use recent reference terms because student behavior evolves. A reference from one year ago provides more accuracy than one from two years ago. • In high-enrollment programs, accuracy can be improved by using separate references for entry courses, courses with prerequisites, and transition courses. This is because more experienced students tend to enroll earlier. Because there are modest differences in the time histories for different types of courses (for example, entry courses and courses with prerequisites), the accuracy of the forecast could be improved by creating separate enrollment forecasting models for the different course types. However, when histories are collected for different types of courses, there will be fewer students supporting each of the histories, which increases the forecast error.

Program Definition and Development  65

• •

This increase in the forecast error may be more important than the improvement from separating the course types, so this technique only applies to high-enrollment cohorts. In the first offering of a program, there is of course no reference term, so other forecasting methods, such as using the time history for a similar program, need to be used. This algorithm has much lower error than algorithms based on market analysis, particularly in the critical time before course launch. We have found that the time histories of different programs are similar, so the time history for an existing program can be used to provide reference enrollment data for forecasts during the first year of a new program.

Summary Checklist ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑

Define the program goals and learning objectives. Select a course scheduling model. Select a course development model. Create an organizational design to support faculty. Select an instruction model. Select an interaction model. Select a synchronous teaching model. Develop a marketing plan for the program. Forecast enrollment.

References Boling, E. C., Hough, M., Krinsky, H., Saleem, H., & Stevens, M. (2012). Cutting the distance in distance education: Perspectives on what promotes positive, online learning experiences. The Internet and Higher Education, 15(2), 118–126. Curry, R. E. (1990). Optimal airline seat allocation with fare classes nested by origins and destinations. Transportation Science, 24(3), 193–204. Fishman, B., Konstantopoulos, S., Kubitskey, B. W., Vath, R., Park, G., Johnson, H., & Edelson, D. C. (2013). Comparing the impact of online and face-to-face professional development in the context of curriculum implementation. Journal of Teacher Education, 64(5), 426–438. Hrastinski, S. (2008). Asynchronous and synchronous e-learning. Educause Quarterly, 31(4), 51–55. Mason, R., & Bacsich, P. (1998). Embedding computer conferencing into university teaching. Computers & Education, 30(3–4), 249–258. McIsaac, M. S., & Craft, E. H. (2003). Faculty development: Using distance education effectively in the classroom. Computers in the Schools, 20(3), 41–49.

Chapter 5

Program Administration

This chapter is written primarily for faculty, program administrators, and others involved in admissions, recruitment, retention, and program outcomes. This chapter covers: • • • • • • • • • • • •

Hiring and training enrollment advisors. Processing admissions. Developing a recruitment pipeline. Developing a student retention process. Administering contact hours. Defining and administering course standards. Identifying lower-cost actions to improve program quality. Identifying moderate-cost actions to improve program quality. Identifying higher-cost actions to improve program quality. Evaluating the program. Updating marketing collateral. Managing the program throughout its lifecycle.

This chapter describes the responsibilities of program-level administrators, who are typically faculty. These responsibilities usually include assisting in the development of a recruitment pipeline, ensuring course quality, reviewing marketing collateral, evaluating the program, and managing the program lifecycle. Ideally, program administrators should lead and shape their programs. Program administrators are essential to the long-term success of online programs, because they are familiar with them and their students, and what is needed for both to succeed.

Hiring and Training Enrollment Advisors Most applicants to an online program want to speak with someone who can explain the details of the program, help them decide if this is the best program for them, and even help them with the application process. These are the roles of enrollment advisors.They often encounter prospective students who are not well

Program Administration  67

prepared, and their role can then be to counsel them to take transition courses or even consider other programs. They spend much of their time on the phone, speaking with prospective students. This communication is typically aided by a customer relationship management (CRM) system. Each applicant usually speaks with one enrollment advisor, who develops a relationship with them.

Thomas Cavanagh, vice provost for digital learning, University of Central Florida We run regular, cyclical marketing campaigns focused on specific targets. Once we are contacted by a prospective student, we have contact specialists who work with them to answer any questions and help complete applications. We use a variety of software tools, at the center of which is a customer relationship management (CRM) tool.

Identifying, recruiting, and hiring good enrollment advisors is critical for marketing success. Enrollment advisors need many skills, including good telephone and email skills, excellent ability to relate to applicants, and a great deal of patience and understanding. People who thrive as enrollment advisors are typically highly extraverted and motivated by helping other people. Enrollment advisors should report to managers who understand the unique challenges of their job, which involves spending most of their work hours communicating with prospective students. Enrollment advisors need extensive training, both in enrollment advising and in the programs that they are supporting. Training in enrollment advising is best performed by their managers or experienced enrollment advisors. Training in programs is best performed by directors or faculty of the programs.This training should cover admissions requirements, courses in the program, program length, and the careers for which it prepares students. This training is usually performed in a classroom setting, and the sessions may be recorded for review and for training of later hires. If many enrollment advisors are needed for a program, then it may be worth the effort to prepare training material.

Processing Admissions In this section, we describe the importance of rigorous admissions processing for online programs, and approaches to administering it. Admissions normally involves two phases. In the first phase enrollment advisors work with applicants to help them determine if a program is a good fit for their needs, and help them complete their applications.They also prepare a file for each applicant, and when it has been completed, forward it to the academic admissions committee. In the second phase, the admissions committee evaluates the applications to determine

68  Program Administration

which applicants to accept, which to reject, and any transition coursework that may be required before admission. Enrollment advisors can serve a critical role in the admissions process by informing students of what is expected of them in a program, including prior preparation, typical workload and academic expectations, and what is required to complete the program. Best practice is for the enrollment advisors to be fulltime employees of the institution, so they can become familiar with its offerings and provide insight into its programs. We recommend that enrollment advisors go through intensive training, with frequent updates. Programs are living organisms that change frequently; in-house advisors can easily learn about the latest changes in programs immediately and can update prospective students right away. We have found that in-house advisors do a better job than enrollment advisors associated with outside marketing agencies, who may represent multiple institutions and so have a conflict of interest.

Limitations of letters of recommendation Many programs require letters of recommendation, with at least one provided by a previous faculty. This can be an obstacle for students who did not graduate recently, because they may not remember faculty who taught them many years ago; faculty may not remember students who were in their classrooms a long time ago and may not have the class records. Many programs allow these students to submit letters from people who know them professionally, such as their colleagues. These letters can provide valuable insight into students’ motivation and character, but transcripts, work history, and personal essays are better predictors of academic performance. However, when it comes to older students, undergraduate transcripts are also limited in their usefulness, because they testify to the academic abilities or motivation the students had many years ago, but may not accurately reflect their current abilities, interests, or motivation.

The faculty admissions committee examines documents provided by enrollment advisors and decides if an applicant is a good fit for the program. Enrollment decisions require a great deal of skill. The admissions committee needs to be able to evaluate transcripts from a variety of institutions and determine which courses predict subsequent success. Admissions decisions require understanding what it takes to succeed in the program and how to identify applicants who will succeed. Some applicants may not qualify prima facie, but skilled admissions committees can reliably identify which students will succeed. For example, an applicant who received their undergraduate degree

Program Administration  69

from a top music program may not superficially appear qualified for a technical program, but an admissions committee that understands music education would know to look at the student’s grades in music theory and composition courses. Similarly, a student who completed an undergraduate degree with a major in philosophy may be well qualified for graduate study in computer science, and the admissions committee needs to know that they should look at the student’s grades in highly formal philosophy courses such as symbolic logic and epistemology. As with face-to-face programs, online program administrators try to accept applicants who differ widely in academic preparation, cultural and socioeconomic backgrounds, job experience, and age. This poses significant challenges for admissions committees and program designers. There are several ways to make programs more open to students with diverse backgrounds. One of the most important ways is to have transition courses that help prepare students with backgrounds that are more appropriate for different careers or disciplines. At Boston University we have used several approaches to these transition programs. One approach is to require students to complete coursework that addresses deficiencies that the admissions committee has identified. This approach can work well, but it provides less visibility into a student’s ability when courses are taken at another institution. Available courses may not be a good fit with what the admissions committee requires of the students. Accordingly, best practice is for larger online programs to develop transition courses that are specifically designed to prepare students for the program’s entry courses. Some programs at Boston University have developed transition courses that use a MOOCish model. These courses are inexpensive, and they provide students with an opportunity to prepare and also to determine whether they want to continue studying in the program.Transition courses are most useful for people plunging into an unfamiliar field without an academic background or work experience. We have developed prerequisite undergraduate transition courses that serve as a broad introduction to the material in the entry courses. These courses help students get accustomed to the online mode of delivery, program and faculty expectations, working in groups, and communicating with faculty, facilitators, and support staff. Thousands of students have taken these courses, and those who have completed them with a passing grade of B or better have done very well in our programs. We have found that students who took these courses did much better than those who had a similar background but did not take them. One disadvantage of undergraduate transition courses for graduate programs is that they are not covered by financial aid, which puts an extra financial burden on students. Table 5.1 illustrates typical decisions made for programs with transition courses for applicants with different academic backgrounds and work experience.

70  Program Administration Table 5.1  Transition course decisions Relevant academic background

No relevant academic background

Relevant work experience

No transition courses required.

The admissions committee should evaluate the student’s theoretical preparation. If it is adequate, no transition coursework is required.

No relevant work experience

The admissions committee should evaluate the student’s prior coursework, sometimes requesting syllabi, to determine if their schoolwork covered important skills they would have learned on a job. If it is adequate, no transition coursework is required.

The admissions committee should require transition coursework, and the final admission decision should be made when that coursework has been completed.

Graduate programs that have transition courses may be able to classify their applicants into the four categories in the table. •

•

•

•

Applicants who have both relevant academic preparation and work experience. They require no transition courses and should be admitted or rejected based on their application. This is the most successful group of applicants. Applicants who do not have a relevant academic background, but have work experience. Some of them educated themselves or completed a certificate program. They may be successful in their field, but need to be prepared for the academic challenges of a program. These applicants should take transition courses. Applicants with a relevant academic background but no work experience. These applicants may be well prepared for advanced study if their undergraduate programs were rigorous and recently completed. If the applicant completed their academic studies, but has forgotten much of the material, transition courses will refresh their knowledge. Applicants with no relevant academic background and no work experience. Applicants in this category require rigorous transition courses, because they will have little understanding of the discipline or the skills they will need to  succeed in their studies. Applicants’ performance in comprehensive

Program Administration  71

transition courses helps the admissions committee decide which of them are ready. These courses also help applicants decide whether they like their new discipline and can succeed in it.

Requiring standardized exams Many programs no longer require standardized exams for admissions, though they may accept those exams to support an application. First, even if applicants receive acceptable scores, they may not be predictive of their readiness for a program. Second, it can be difficult for foreign students to take these exams, because it may require lengthy travel to exam centers where they are administered.

Developing a Recruitment Pipeline Successfully marketing an online program requires a process for creating, maintaining, and analyzing a pipeline through which prospective students progress from marketing leads to enrolled students. A formal marketing pipeline allows the institution to forecast enrollments, as well as analyze the performance of marketing collateral and enrollment advisors. This pipeline also forms a framework for dialogue between enrollment advisors and prospective students. The dialogue may take the form of telephone calls, email, text, or a face-to-face meeting, such as at a marketing event like a college fair. Enrollment advisors can thus help improve the quality of the enrollment pipeline. The pipeline typically has a number of stages, such as marketing lead, contact established, completion of application, payment of application fee, acceptance, and enrollment (Smith, 2016). Enrollment pipelines are supported by CRM systems that track the status of students, as well as supporting the dialogue between prospective students and their enrollment advisors. Prospective students become leads and enter the pipeline when they provide contact information. They advance to the next stage when they establish communication with an enrollment advisor. They move to a further stage when they pay the application fee.Then the admissions committee reviews their applications. Of the applicants, a certain percentage will be accepted, and of those, a certain percentage will register. The last stage in the pipeline occurs when students “start,” or enroll for their first courses. Marketing, retention, and development teams may continue to follow students’ progress as they move through the program, to analyze their success and how it relates to marketing factors. Enrollment advisors establish the initial relationships with students, and they guide and encourage students to enroll in their first course. If an admitted student does not enroll for their first course when expected, their enrollment advisor should reach out to them. Some students may be sitting on the fence, so

72  Program Administration

advisors should be able to answer their program-specific questions and, if needed, refer them to faculty who will be able to answer their course-specific questions. After a student has enrolled for their first course, retention responsibilities may be passed to retention specialists or advisors.

Developing a Student Retention Process Student retention begins when prospective students are admitted and continues throughout their studies. One of the key factors in retention is that students should be guided to take courses in a recommended sequence, so that they are better prepared for them. Many programs begin to advise students on recommended course sequences in the packet of information they send when students are admitted. Sometimes the standard recommended course sequence is not the best for particular students. In programs with multiple concentrations, recommended sequences should be developed for each of the concentrations. For a program to be successful, it is important that students value their experience, graduate, and contribute to the reputation of their alma mater. Retention processes serve key roles in keeping students enrolled in face-to-face and online programs, by reminding them to register for courses. If students still have not enrolled, retention advisors try to determine why and what can be done to help them continue in the program. Retention advisors should be associated with the department, because this will facilitate communication with faculty. We have found that the retention effort should start about two weeks before the term begins. Our retention advisors send an email to students, followed up by a phone call. Email is preferable to texting, because the latter is considered too informal. Best practice is to send students an email with the phone number from which the call will originate. It is also helpful to call students on the last day before the registration deadline, as many of them simply forget to register.

Defining and Administering Contact Hours Contact between faculty and students is central to teaching and learning. Academic leadership needs to establish policies for how much faculty contact students can expect. In a traditional face-to-face course, the students and faculty meet regularly, and the number of hours that they meet usually determines the number of credit hours in a course, and how much a course costs. For example, if students and faculty meet four hours per week for a 14-week course this conventionally counts as a four-credit-hour course, with a total of 56 contact hours. Counting contact hours for online and technology-enabled courses is more complicated, because faculty can interact with students in different ways (Powell, Helm, Layne, & Ice, 2012). The main difference between face-to-face and online counting of contact hours is how asynchronous learning activities are treated. Administrators who wish to establish contact-hour policies must determine how contact hours are

Program Administration  73

to be measured for the different ways in which students interact with faculty (Sturges, 2013). It is obvious that each hour of synchronous interaction between faculty and students counts as one contact hour. It is less obvious how contact hours should be counted for technology-enabled asynchronous learning activities in which students interact with material created by faculty. One emerging policy is that technology-enabled asynchronous interaction with faculty-developed material is counted as asynchronous contact time only when the student receives faculty-developed feedback that depends on student actions such as answers to questions. To be counted as asynchronous faculty interaction, the faculty-developed content must include responsive interactions with students, where the system reacts with different faculty feedback based on their responses. An example of asynchronous contact is inline formative evaluations that ask students questions and provide feedback based on their answers. Boston University’s policy is that three hours of this asynchronous technologyenabled interaction with faculty counts as one hour of contact time. Faculty contact may differ based on the learning objectives. For example, a French language course may have vocabulary development learning objectives that are well met by asynchronous activities, and spontaneous dialogue learning objectives that require synchronous faculty involvement. Contact-hour requirements may also depend on the subject matter. For example, contact-hour requirements for social work courses with extensive role-playing activities may be higher than for courses that emphasize rote learning. The number of contact hours in an online course can differ widely with course design. At one extreme are MOOCs, in which students receive little, if any, faculty attention. Because MOOCs offer very little faculty engagement, encouragement, or support, students become frustrated, lose interest, and most drop. (For more on MOOCs, see Chapter 4.) At the other extreme are “high-touch” online courses, in which faculty conduct frequent synchronous lectures and office hours, so students have extensive access to faculty. High touch is necessary to supervise student research, so it is common in advanced graduate courses. Because high-touch courses require more faculty time per student, they are more expensive to offer, and enrollments must be limited; facilitators improve scalability. Online technologies can provide many of the advantages of high-touch interaction in a scalable way. For example: • • •

Automatically graded tests provide feedback for all answers, giving students individualized tutoring, without forcing them to wait. Automatically graded practice tests do the same, but without the stress of being counted toward course grades. Inline formative questions with feedback provide students with confirmation when they have mastered the material just presented, or individualized guidance when they have not. Their scope is narrower than tests, and they are presented on the same page as the course content. There may be only one or two such questions per page.

74  Program Administration

Policies for how course contact hours are counted and course contact-hour requirements differ between institutions. Policies are currently evolving for how contact hours are counted for online and other technology-enabled courses, and this complicates the administration of contact-hour policies.

Defining and Administering Course Standards Course standards define elements that occur in all courses in a program, such as common designs, ways of assessing characteristics of courses, and how they should be implemented. These standards ensure that all courses share common features that make them easier to develop, navigate, and manage. Course standards are defined and administered at the program level. There are many different kinds of course standards: •

•

•

•

•

Academic integrity. Examples include requiring originality testing, comprehensive proctored final exams, on-campus residency requirements, and randomization of assessment questions. (For more on academic integrity, see Chapter 3.) Synchronous sessions. Standards for synchronous sessions help ensure that students have enough contact with their faculty and classmates and opportunities to ask questions. Standards help students and faculty know what is expected. Discussion forums. Programs may create standard discussion forums that are included by default in all courses. Examples of topics include introductions, graded module discussions, and course improvement suggestions. Programs may also have standards prescribing if and how discussions are graded. Assessment and assignments. Assessment standards address requirements regarding subject-matter coverage and grading weights. One common assessment standard is that each course must have a comprehensive proctored final exam. Another is that each module must have a quiz. An emerging standard at Boston University is that each content page must include a formative assessment with feedback. Contact-hour requirements. Another evolving standard at Boston University is that each hour of synchronous interaction between students and faculty counts as one contact hour, and each three hours of time that students spend interacting with online content created by faculty counts as one contact hour.

There are many benefits to having standards for all courses in a program, including: •

They make it easier for faculty and designers to create courses with uniform design, which helps students transition from course to course in a program.

Program Administration  75

• •

They make it easier to create course templates, which reduce course development time, effort, and risk. They can drive program-wide improvements.

Program standards should be appropriately flexible and easy to understand, implement, and administer. Course standards improve efficiency, but they can inhibit innovation in course design, execution, and pedagogy. For example, a program may require that at least 40 percent of grades be based on a comprehensive proctored final exam, but this may be inappropriate for a research seminar where most of the work consists of student presentations. However, as standards become more flexible, they require more administrative supervision, so the appropriate standards for a program may not be the most flexible. Care must also be taken to ensure that the design of course standards does not impair the academic freedom of the faculty. Well-designed standards can help faculty express what they want to teach by providing appropriate infrastructure for their content without restricting it. Standards should provide a baseline for innovation and improvement, without hindering those essential efforts. Defining standards involves tradeoffs between standards defined in terms of their goals, and standards defined in terms of ways to attain those goals. Standards defined in terms of ways to attain goals are easily understood and administered, because they specify details about how to do that, but they unavoidably inhibit flexibility and innovation in attaining those goals. Standards defined in terms of goals are the most flexible, but they are also the most difficult to administer, because they lack details about how they are to be attained. This is illustrated in Table 5.2 as follows: • •

• •

The first column lists program goals that may lead to a standard. The second column lists standards that would meet the goals, but unnecessarily restrict course innovation and academic freedom. Standards in this column restrict what can be done in courses in ways that may reduce quality. The third column lists more flexible standards that meet the goals without significantly restricting courses. Standards in this column are usually a good tradeoff between flexibility and ease of administration. The fourth column lists flexible standards that attain the goals, without restricting how they are attained. Standards in this column require more administrative effort.

In institutions with extensive online experience, new programs have the option of adapting standards developed by similar existing programs, so standardization may come early in the program. If a program is unable to adopt proven ­standards, it may take some time to define and implement its own (King & Alperstein, 2015).

76  Program Administration Table 5.2  Goals and standards Goal

Restrictive standard

More flexible standard

Flexible standard

Discourage cheating

Courses must have a comprehensive proctored final exam.

Courses must have a comprehensive proctored final exam or other means to ensure that students who receive credit are the ones who took the course.

Courses must incorporate a set of academic integrity measures approved by the academic integrity committee.

Optimize navigation skill transfer between courses

Course navigation must be based on the program course template (see Chapter 4).

Course navigation must be based on the program course template, except for approved deviations.

Courses must be designed so that when students learn how to navigate in one course, they do not have to learn different ways to navigate in other courses.

4-credit courses must have at least the equivalent of forty contact hours

Each course must have forty hours of faculty-led synchronous sessions.

Each course must have the equivalent of forty contact hours, counting three hours of student interaction with content created by faculty as one contact hour.

Each course must have at least the equivalent of forty contact hours, as measured by approved metrics.

Identifying Lower-Cost Actions to Improve Program Quality These first three actions are related to the course development process. Course development is easier if these activities are an integral part of the initial process, rather than retrofitted to existing programs and courses. Our experience has been that they reduce development time while improving quality. •

Design curricula carefully, with well-designed and integrated courses that meet the program’s and course’s learning objectives. This is important to any program and delivery method, but it is particularly so online, because

Program Administration  77

changes to online curricula are more expensive. Our curriculum quality process is supported by regular, formal curriculum reviews. (This process is described in Chapter 10.) • Minimize course prerequisites. Excessive prerequisites complicate course scheduling for both students and the program. One curriculum design that works well provides one introductory course for all of the advanced courses in each major area of the curriculum. Prerequisite material can be reinforced by including review material in courses. (This is described in Chapter 6.) • Include both asynchronous and synchronous course components. Asynchronous and synchronous components meet different student needs, and a good course needs both. We routinely record synchronous sessions and provide closed captioning. Faculty and administrators need to decide whether courses are to be offered in synchronous, asynchronous, or combined formats, and ensure that the course material and activities are correspondingly aligned. (This is described in more detail in Chapter 6.)

Identifying Moderate-Cost Actions to Improve Program Quality The following are investments in the people, technologies, and course and curriculum design that enhance student learning. These actions have moderate initial and recurring costs, but they will be repaid abundantly in course and program quality. •

•

•

Adopt a formal, quality-oriented course development process, because it is essential to develop high-quality courses consistently. Our process includes faculty peer review, faculty coordinator design and development supervision, technical editor review, facilitator quality assurance, formal student evaluations, and formal update planning. (These processes are described in Chapter 7.) Recruit enough facilitators to maintain low student/faculty ratios. We have a formal, multiphase facilitator recruitment and development program, and provide one facilitator for each group of at most 15 students. An LMS allows faculty to develop automatically graded tutorial assessments for recall and other lower-level questions, but automated grading of assignments and answering student questions does not come close to the quality provided by faculty for higher-level learning activities. (For more on facilitator recruitment and development, see Chapter 2.) Host courses on high-performance servers and provide technical support. Programs should assess the relative cost and performance of outsourced

78  Program Administration

hosting compared to hosting in-house. Programs should also evaluate the options for outsourced or in-house technical support. In any outsourcing arrangement, it is critical that the program carefully draft the contractual quality of service guarantees, and that those guarantees include penalties when the requirements are not met.

Identifying Higher-Cost Actions to Improve Program Quality The following contribute substantially to the quality of online courses, but they require investments that many institutions may not be willing or able to make. •

•

•

•

Include a comprehensive proctored final exam in each course to help ensure academic integrity, which is more difficult to achieve online. Proctoring of final exams involves professional proctors who remotely verify the identity of students and monitor them as they take the exam, recording their actions and their screens. Best practice is to weight proctored exams fairly heavily, typically 30 to 40 percent. In situations where students’ scores on proctored exams are much lower than their scores on un-proctored graded assignments and assessments, the program may require such students to complete additional proctored assessments covering the same material. (For more detail on final exams, see Chapter 8.) Include support staff when courses are running. Online students may feel isolated from their institution, faculty, and other students. Student-support personnel can help students with non-academic issues. For example, when students begin a program, they may be contacted by a student-support person, welcoming them and letting them know that they can contact them directly with non-academic issues such as questions about registration, schedules, how to log in to the course, and basic technical support. As courses run, the support person can send students reminders about upcoming events such as drop/add deadlines, course evaluations, and scheduling their proctored final exam. They may also perform QA on course websites before launch. (The roles of student support while courses are underway are described in more detail in Chapter 8.) Hire instructional designers, videographers, and web animators to provide the many technical skills needed to develop top-quality course websites. The primary reason we frequently mention hiring design staff is that this allows faculty to concentrate on the content rather than its implementation. Even if a program cannot hire full-time staff, it should consider hiring parttime contractors or student workers to perform these roles. (For more on course design, building courses, and hiring design staff, see Chapter 7.) Consider hiring full-time faculty. Learning to teach well online requires a significant time commitment, which may be challenging for part-time

Program Administration  79

faculty. Full-time faculty can devote more time to mastering online development and teaching. Faculty should ideally have earned at least one degree more advanced than the coursework they are teaching. Undergraduatelevel instruction should be performed by faculty with at least a master’s degree in the subject area, and teaching at the postgraduate level requires PhD-level faculty.

Evaluating the Program Online programs should be evaluated periodically to assess their academic quality, competitive landscape, market and program evolution, enrollment projections, opportunities, challenges, and any other relevant longer-term issues. Program evaluation is often conducted as a formal process, involving the faculty from the relevant departments meeting to review and evaluate the program from many perspectives (Vlachopoulos, 2016). In this review, the faculty will usually analyze the courses and their relationships within the curriculum. Additional inputs include student evaluations, peer evaluations, the topics covered, enrollment trends, outside evaluations such as accreditors, and other relevant information, such as changes in the competitive landscape. The output of a program review is usually a document or report that addresses the areas just described, as well as identifying weaknesses and opportunities in the curriculum, pedagogy, academic integrity, and instructional technology. The program review may recommend actions such as updating courses, updating the curriculum, changes to marketing, new courses, and/or changes to program administration. Occasionally a program review may recommend major changes, such as a program being combined with another program, split, terminated, or otherwise restructured. There are many ways to measure the quality of online courses and programs, including evaluating faculty, teaching, online material, students, and academic integrity. •

Metrics used for evaluating faculty include teaching experience and evaluations, degrees, publications, and/or service to the profession and community. • Metrics used for evaluating the quality of teaching include student evaluations, student surveys, the number and nature of contact hours, and peer faculty review. • Metrics used for evaluating online course content include student evaluations, peer reviews, and the number of requests for faculty help. • Metrics used for evaluating programs include outcome assessments and interviews with graduates. (See Chapter 10 for more on outcome assessments.)

80  Program Administration

• Metrics used for evaluating students include performance on proctored assessments, grade-point averages (including in prior programs), and performance in proctored capstone projects. • Metrics used for evaluating the academic integrity of programs include students’ performance on proctored assessments, observing disparities between their performance in proctored and un-proctored assessments, their ability to answer questions in synchronous sessions, and systematic testing of the academic integrity provisions.

Updating Marketing Collateral Program administrators and faculty must be involved in keeping the marketing collateral—information and material used to communicate about a program— consistent with changes in the programs and changes in the interests of prospective students. This usually requires that they review, update, or write marketing collateral. New course content is usually of great interest to prospective students, so marketing updates should be made as soon as possible. The scope of updates can vary from minor changes to creating collateral for new courses. As when developing the initial marketing collateral (see Chapter 3), faculty and administrators should work together to ensure that what is presented is relevant, current, of interest to prospective students, and correct. Many faculty find that it is easiest to update an extract of the current collateral. If faculty are too busy to write the updates, and marketing staff are available, it can work well for faculty and staff to review the collateral together in a recorded session, in which faculty describe and dictate the changes that need to be made by the marketing staff. If this approach is used, it is important that the faculty have an opportunity to review and approve the updated material before it is published. If faculty are reluctant to make the changes needed, they can usually be persuaded by having someone such as a graduate student or marketing staff write the updates and have faculty make corrections as part of an approval review. (This is a variant of the “stone soup” method described in the context of course development in Chapter 7.) The marketing collateral to be updated or created can include videos of faculty describing the courses and programs. These can be very effective if the faculty present well in video and the videos are of high quality.The same videorecording facilities can be used to produce marketing and course videos. (This is described in more detail in Chapter 6.)

Managing the Program throughout Its Lifecycle Programs and courses typically follow phases, which we describe as launch, growth, plateau, decline, and teach-out. The launch phase for a program or

Program Administration  81

course begins when it is first offered and it typically ends when courses have been offered and updated a few times, and are fully developed.The growth phase extends as long as enrollments are growing. The plateau phase begins when enrollments have stabilized, and lasts until they begin to decline. The decline phase begins when enrollments begin to decline significantly, and ends when the decision is made to restructure or terminate the program. The teach-out phase begins when the decision is made to terminate a program, and ends when the last student in the program has completed their studies. There are different issues and management techniques for the different phases. The duration of the phases can vary widely; for example, the launch phase can last from less than a year to several years, and a well-managed program with capped enrollments may stay in the plateau phase indefinitely. Launch phase is the most challenging, because faculty are learning how to teach the material online, while also updating the material. Programs may experience considerable enrollment growth during the launch phase, which adds the challenges of rapid growth, such as faculty and facilitator shortages. During launch phase, not all of the content may have been developed in its final form. The course may not have mature academic integrity features, such as random question selection from question sets. Courses are tuned during the launch phase, as feedback from students, facilitators, and faculty is used to guide the refinement of the content. During the launch phase, student workload is also assessed and adjusted across the term. (For more on workload leveling, see Chapter 6.) The growth phase begins when the program and courses are fully developed, and extends as long as enrollments are growing. One challenge in the growth phase may be finding enough additional qualified faculty, facilitators, and support staff to maintain quality as enrollments grow. Courses may need to be offered more frequently, to maintain good student-to-faculty ratios and offer students more opportunities to take classes. Online enrollments can grow very rapidly, particularly when a new program taps an unmet market need, or because of significant market events. For example, shortly after the 9/11 attacks, enrollments in our Computer Security program grew very rapidly. Additional administrative effort is required to manage rapid growth, including schedule adjustments, recruiting and developing faculty and staff, and improving scalability of course design. When enrollments are growing rapidly, it is critical that additional resources be brought to bear to maintain program quality. If an institution cannot respond quickly enough with additional resources, it may be necessary to cap enrollments to protect the reputation of the program and institution. The plateau phase begins when enrollments stabilize and it ends when enrollments are declining. This phase can last indefinitely, particularly if enrollments are capped. One of the main challenges in the plateau phase is preventing complacency and refreshing the excitement of growth and change. To do this, the administrators need to nurture a culture of innovation, always seeking to find

82  Program Administration

new ways to improve courses, identifying or creating new online technologies, and creating new courses and curricula. The plateau phase is risky, because if a sense of complacency and entitlement develops, quality and innovation may suffer, and this will ultimately result in declining reputation and enrollments. The decline phase begins when enrollments begin to decrease significantly. The most common causes of enrollment decline are increasing competition and failure to keep up with an evolving online education market. If an institution wishes to reverse falling enrollments, it should perform market analysis to understand the causes. This may reveal some combination of increased competition, a change in market focus, or a general decline in market interest. Successful follow-ups may include updating the courses and program, restructuring the program, changing marketing, and/or offering courses in additional formats. The teach-out phase begins when a decision is made to terminate a program, and it ends when the last students have completed their courses. The traditional way to implement teach-out is to stop admitting students to the program, and then to offer the courses as long as students still enrolled in the program need them to complete their studies.Technology enables additional ways to teach out online programs, including having students take face-to-face courses using telepresence. (For more on telepresence, see Chapter 7.) Courses can experience low enrollments for a number of reasons, and they are common in the launch and teach-out phases. If low-enrollment courses are scheduled less frequently, this may increase their enrollments. There is a practical limit to how infrequently low-enrollment courses can be offered, of course, because students cannot wait too long to take courses that they need. This limit is often once per year, though in some advanced programs it can be longer. Online courses can function well with low enrollments. An online course can even be used for directed studies, with only one student in the course. In courses that typically have facilitators, if enrollments are low, faculty may need to teach without their help. If enrollments are low and the decision has been made to have fewer students per faculty member, then the faculty can perform course updates as part of their teaching load (for more on course updates, see Chapter 10). Marketing also evolves through a program’s lifecycle. When a program is in the launch phase, marketing helps define and tune the program so that it attracts many students. In the early and main growth phases, the marketing effort may shift to tuning for maximum cost effectiveness. In the plateau phase, marketing may be trying to identify growth opportunities in contiguous markets. In the decline phase, marketing may no longer spend to acquire new leads, focusing on cultivating the lead database to convert the leads in the pipeline. In the teach-out phase, marketing is not active.

Program Administration  83

Summary Checklist ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑

Hire and train enrollment advisors. Decide how to process admissions. Create the recruitment pipeline. Develop a student retention process. Determine the contact-hour requirements. Define and administer course standards. Identify lower-cost actions to improve program quality. Identify moderate-cost actions to improve program quality. Identify higher-cost actions to improve program quality. Periodically evaluate the program. Update marketing collateral. Manage the program throughout its lifecycle.

References King, E., & Alperstein, N. (2015). Best practices in online program development. New York: Routledge. Powell, K., Helm, J. S., Layne, M., & Ice, P. (2012). Quantifying online learning contact hours. Administrative Issues Journal: Education, Practice, and Research, 2(2), 80–93. Smith, R. (2016). Recruiting and serving online students at a traditional university. College and University, 91(3), 67–74. Sturges, D. L. (2013). Techniques for increasing student engagement for contact hour equivalence: Online courses that are flexplace, not flextime. HETS Online Journal, 3(2), 90–121. Vlachopoulos, D. (2016). Assuring quality in e-learning course design: The roadmap. International Review of Research in Open and Distributed Learning, 17(6), 183–205.

Chapter 6

Fundamentals of Course Design and Development

This chapter is written primarily for faculty and designers. Administrators may be interested in the sections on the stages of course development, problems that can arise, and how to address them. This chapter covers: • • • • • • • • • • • • • • • •

Starting a new course. Using design standards and award rubrics. Designing for meaningful interaction. Defining goals and learning objectives. Creating a course map. Creating assessments. The roles of assessments in courses. Planning learning activities. Designing for students with varied preparation. Aligning the content by topic. Leveling workload. Developing and implementing content. Making videos. Pre-launch tasks. Addressing delayed development. Building courses to facilitate updates.

Designing and developing a good online course typically takes six months to a year, and the process must be carefully planned. Development usually involves the stages and events described below, no matter how many people are involved, but this chapter is specifically addressed to faculty who are primarily responsible for their online courses, including designing and developing content, assessments, and assignments. (For a discussion of more advanced elements that become available with more staff and greater resources, see Chapter 7.) Many institutions lack the resources to field full teams of distance education specialists. Most courses are put online by faculty working on their own, in addition to their regular course load. Fortunately, the basic principles of

Fundamentals of Course Design and Development  85

good online course design and development are not enormously different from traditional, face-to-face courses. Perhaps the biggest difference is the importance of advance preparation. Instead of being able to develop course or lecture materials as the term progresses, the asynchronous nature of most online interaction demands that the entire course be ready for the first day of the term. Careful design of asynchronous courses is also more important than for synchronous courses, because faculty have limited ability to adapt. In a face-to-face course, experienced faculty can modify their teaching to the needs of their students, going over topics again, asking questions, providing additional examples, or postponing some of the material. This same approach can be used in the synchronous components of online courses. Best practice is for online courses to include both asynchronous and synchronous components, to allow the faculty to adapt to unanticipated student needs as much as possible and to better accommodate varying student interaction styles and learning rates. Fortunately, some of the most important decisions are likely to have been made by the time faculty sits down to begin designing a course. The basic technical scaffold, the LMS, will have been chosen, and the faculty might even have developed some experience in using it for routine administrative matters. The overall program goals will be known, and the term schedule will have been set, so the faculty can now focus on building the course. Potentially, if course elements have been defined and standardized at the program level, there may be a template course that includes the universal and individualized elements expected for that program, which faculty can simply follow. If not, however, keeping certain overall design principles in mind from the beginning will make course development easier, and the courses themselves more effective, by enhancing student learning and participation, ensuring that technology will work for all students, and minimizing the maintenance effort.

Starting a New Course When faculty have an idea for a new course, they prepare a preliminary course description, which they discuss with their colleagues. If there is sufficient interest, they expand it into a complete syllabus. The syllabus is then typically reviewed by the faculty at a department meeting and sometimes by administrators before it is approved for development. This review should include how the proposed course fits into the online program curriculum. At smaller institutions this review and approval process may involve only one step. At larger institutions, it may involve several levels of review and approval.

Using Design Standards and Award Rubrics Just looking at excellent courses, for example those that have won design awards, can provide an overall sense of how things work online, but even more information

86  Fundamentals of Course Design and Development

is easily available. Committees that evaluate courses for instructional design awards generally use a two-part rubric. The first part is a checklist of elements that should be included in any good course, and the second looks for original, noteworthy elements that make the course outstanding. Faculty should take advantage of these rubrics as a systematic way to improve course quality, by using their checklists as a guide to what should be included when building or revising a course (McGahan, Jackson, & Premer, 2015). The checklist serves as a way of making sure that each course meets minimum design standards, and, ideally, goes beyond. Faculty can choose several award rubrics to use as guides, such as: • • •

Blackboard Catalyst Awards. Online Learning Consortium Awards. United States Distance Learning Association.

In Boston University’s Office of Distance Education, we collected rubrics from awards, as well as design recommendations and standards from: • • •

California State University, Chico’s Rubric for Online Instruction. The Open SUNY COTE Quality Review (OSCQR) Process and Rubric. The Quality Matters Course Design Rubric Standards.

We then compiled a list of recommended design elements and removed items that were duplicates, inapplicable, or that we disagreed with. We organized the remaining elements into two categories. The first category contains universal elements which apply to every course. For example, technical support is provided by the university’s IT office for all courses, so the section describing it should be the same for every course. The second category contains elements which vary depending on the course and/or instructor, and faculty should focus on the elements that are most applicable to their course. For example, the terminology in an introductory medical course may be defined in a course-level glossary. Best practice is to organize the award rubrics by the section of the course in which they will be used. For example, all elements that apply to the syllabus should be listed together, in the order in which they will be addressed.

Designing for Meaningful Interaction Effective teaching is based on getting students actively involved with the subject matter through reading, problem solving, discussions, projects, assessments, and synchronous telecommunications.While learning can occur passively, through reading and observation, or through rote memorization, the best way to learn to solve problems is by engaging in a sequence of progressively more challenging ones. Ideally, all activities should include feedback to correct students’ understanding, and to guide and reinforce it. In a face-to-face class, most faculty follow a

Fundamentals of Course Design and Development  87

pattern in which they cover material, then ask questions to see if the students understand it. Experienced faculty know how to respond to confused-looking students, to dead silence in the classroom, and to other signs that additional explanation is needed. When faculty or students speak in a traditional face-to-face classroom, they address the whole class, so there is a single conversation between the faculty and all students. One of the great advantages of well-designed online courses is that there can be different conversations between faculty and individual students. Some of these conversations can involve automation, in which the LMS and learning objects respond with faculty-provided feedback that depends upon student input. For example, the system may ask students a multiple-choice question, and provide them feedback that is based on their answer. Automation of this feedback allows common solutions to be offered by the software, allowing faculty more time to respond personally to novel questions. If the course contains supplemental material, such as embedded questions, context-sensitive help, advanced topics, or remedial material, students can use any or all of them, if they want to (for more on students with varied preparation, see later in this chapter). In this way, interaction with the course material can be adapted to meet the needs of a diverse student body, and be even better than in face-to-face classrooms. Traditional forms of online student engagement, such as text discussions and synchronous multimedia interactions, can be improved with careful thought and design. Text discussions can be made engaging if faculty post good, leading, open-ended questions, such as “What would you do in this situation?” or “What are the most important factors to consider in this problem?” Faculty should be cautious about summarizing, or other actions that tend to suggest that a topic has been exhausted, and instead participate in ways that stimulate and guide the discussions. For example, if faculty praises a student for having thoroughly answered a question, it shuts down discussion, but if faculty follows up with a challenging generalization of a discussion question, that stimulates discussion. Another extremely effective method of enhancing student engagement online is formative assessment questions, where students are presented with questions after some course content and the software includes feedback provided by faculty for different student responses. This immediacy is important, because feedback is more effective if it is provided as soon as possible after a student response (Carless, Salter,Yang, & Lam, 2011). We recommend that every topic or module contain a combination of both immediate-response, formative-evaluation questions and activities that involve deeper thinking, such as discussions, essays, or projects. Best practice is to include a combination of immediate, automatic responses provided by software as well as deeper, though delayed, evaluation and feedback by faculty and other students, such as from discussions and grading of assignments (Almala, 2007). Quizzing is possible with many types of learning objects, including text, graphics, videos, and both static and interactive animations. For example, video

88  Fundamentals of Course Design and Development

segments can be made more engaging with the use of in-video quizzing, in which a video pauses and students are asked questions which have to be answered correctly before continuing. Synchronous multimedia sessions can be made very engaging if the students have headsets and webcams so they can talk with and see each other. This type of multimedia engagement is particularly helpful in reducing the affective distance between students and their faculty. The other side of enhancing engagement is not about what can be added, but what should be avoided: Courses should minimize content that takes students’ attention away from the learning objectives. This may seem obvious, but it is surprisingly difficult. Countless writers have struggled with Strunk and White’s (1979) famous admonition to “Omit needless words,” and Elmore Leonard once explained his popularity to a crowd of admirers by explaining, “I try to leave out the parts that people skip” (Quote Investigator, 2019). Each element in a course should be in it for a reason. Irrelevant material, such as misleading design elements or digressive lectures, has the potential to confuse students, waste their time, and make them ask for clarification. Unproductive material can be created if faculty start writing course content before mapping their courses. (For more on course maps, see later in this chapter.) Similarly, inappropriate technologies can be distracting and waste student time (Dixson, 2015). For example, an online course should not require students to perform tasks to reveal content for the first time, such as having to hover over or click a link to reveal the definition of a vocabulary item. Students are forced to perform a task of no pedagogic value to view the content, which wastes their time; worse, students may miss the hidden content entirely. Furthermore, even such simple technological objects can create accessibility problems, as they may not be visible when studied offline, such as when printed or saved as a PDF (for more on building for accessibility, see Chapter 7). Technologies in online courses should be carefully chosen for their pedagogic value. Done properly, hyperlinks and multimedia objects can mimic techniques that faculty use with individual students in a face-to-face environment. For example, a Flemish-language dictionary might show the text for a ball, with an illustration. Clicking on the picture could play a sound of someone saying the word, just as a tutor would. This pedagogical goal could not be accomplished without the technology.

Jim Frey, learning design director You should constantly ask, “Will this help learners achieve their objectives?” Sometimes a piece of content might give you pride because it is particularly innovative, engaging, or memorable. However, you must also consider its relevance, because that ties directly back to the question about

Fundamentals of Course Design and Development  89

objectives. If what you’re working on is interesting and well-crafted but still not relevant, you will want to re-evaluate either the content or the objectives! A good rule of thumb: if you can't clearly and concisely explain how something in your course ties back to the objectives, you should do one of the following things: • • • •

Find the connections. Rework the content so that it does connect. Remove the content. Rework your objectives so the connections are clear. (Any of the above options will likely require input from/collaboration with your faculty member or subject matter expert.)

Even something as basic as typography offers both opportunities and pitfalls. Ornamentation can call attention to an object, or differentiate elements, but it is easy to get carried away. Elements like callout boxes, horizontal lines (horizontal rules), bold and italic typefaces, or dingbats (printer’s ornaments) emphasize the meaning of text or separate its contents from something else on the page. Many designers love to draw boxes around text, and will spend an inordinate amount of time debating whether those boxes should have sharp or softly rounded corners, without asking why that text needs a box. Beware of using so many decorations as to be distracting—or worse, confusing. Another common example is coloring text or text backgrounds. Color has no role unless it contributes to students’ learning. It can be used to emphasize elements, such as highlighting important elements in red. However, much like ornamentation with type, too much can be distracting and confuse the user. (For more on color usage and design issues, see Chapter 7.) Typography and design can confuse the user by communicating unintended meaning. Just as the course design should not distract or confuse, typographical information that confuses or distracts from the content should be eliminated. Header sizes, indenting, bullets, and numbers all suggest structures, but in some cases no such structure exists. For example, consider two different ways of presenting learning objectives for a biology course: • Describe changes in organisms from day to night • Describe changes in organisms from night to day

1. Describe changes in organisms from day to night 2. Describe changes in organisms from night to day

The text is identical in the two examples, but the list on the right is numbered, suggesting that the objectives are ordered according to some principle that is not specified. This introduces many questions. Do the numbers indicate a temporal

90  Fundamentals of Course Design and Development

order, priority, or importance? Are they dependent on each other? Must the first one be completed before the second can begin? This numbering will distract and confuse students and may result in them asking faculty to clarify. This will waste student and faculty time, reducing both satisfaction and learning. It is important to ensure that the design of each course element makes clear what it is, what it is for, where it begins and ends, and how to use it. Section headers should help students understand the organization of the course and how to navigate it. If an element contains remedial or advanced material that will not be addressed in an assessment, but is included to help underprepared or advanced students, this should be clearly communicated to students (see later in this chapter). For example, advanced material may be in a callout box labeled “Advanced Topic.”

Fawn Thompson, lead instructional designer at Boston University’s Office of Distance Education Many new instructors wish to add images to make their courses more aesthetically pleasing. While this is great in theory, I find that adding clip art or meaningless images can make their content look lower quality overall. I always try to work with instructors to find or create images that are relevant, reflective, informative, and/or inspiring.

Another important design principle is to minimize the time that students spend changing their cognitive context (what they are thinking about), because context changes take time, and too many context changes are disorienting. For example, formative-evaluation questions should be presented with (or as near as possible to) the material they cover. Similarly, the ease with which students can establish context is a key design objective: Replying to a discussion question is much easier if the software allows the student to see the post to which they are replying, as opposed to hiding previous posts.

Defining Goals and Learning Objectives Faculty need to describe the goals and objectives of courses, expressed in terms that can be measured, and they need to explain how students will be assessed, such as by graded assignments, projects, papers, tests, or exams. Next, they need to organize the objectives into modules to distribute the material across the length of the course, and then develop lectures and other learning activities, which should deepen students’ knowledge and provide an opportunity for them to interact with each other, share their ideas, and receive guidance when

Fundamentals of Course Design and Development  91

necessary (Woo & Reeves, 2007). Best practice is for faculty to create assessments before developing learning activities, so they can be sure that the activities prepare students for the assessments (Raible, Bennett, & Bastedo, 2016). Course goals are typically high-level descriptions of what students need to learn. Learning objectives are concrete, measurable manifestations of course goals that describe the student’s performance, conditions, and criteria (Mager, 1997). For example, a course with the goal of “understanding relational databases” may have learning objectives such as “describe the standard SQL data types,” “create database tables,” and “query databases.” Learning objectives are stated in terms of things students can do to demonstrate that they have learned the material. An abnormal psychology course may have a learning objective of “Describe at least five of the nine characteristics of narcissistic personality disorder from the DSM-IV.” If students meet all of the learning objectives, then they have met the course goals. Programs and courses are best developed within a framework defined by a hierarchy of goals and learning objectives, with explicit objectives at the program, course, module, and module-section levels.The goals of a course should support the program goals; the goals of modules should support the goals of the course; the goals of sections should support the goals of modules. Many lower-level goals may support one or more higher-level goals. For example, the goal of learning an alphabet supports goals of learning to read and write. A high-level goal that students should be able to diagram English sentences may be supported by a lower-level goal that students are able to identify the parts of speech. Some learning objective sequences are fairly obvious. For example, one must learn the alphabet before spelling, and one must learn to add numbers before learning to multiply them. Other learning objective sequences can be far from obvious, and determining the optimum learning sequence may require some experimentation. A default sequence of learning objectives should be defined to meet the needs of most students. Sometimes, though, students can meet different learning objectives in different orders. For example, visual learners may prefer to begin their study of set theory with Venn diagrams, while logical learners may prefer to begin with the axioms. The best order of the learning objectives may also depend on the students’ background. For example, a student who has mastered one Romance language and is learning another may prefer to begin with their similarities. One of the big advantages of asynchronous online study is that students can accomplish learning objectives in the order that works best for them.

Creating a Course Map Faculty who teach face-to-face courses are accustomed to organizing their courses as a sequence of class meetings, and preparing lectures, classroom activities, assignments, and assessments for each class. Designers often think

92  Fundamentals of Course Design and Development

of online courses as divided into modules, which often correspond with one or more weeks of a course. An important early development step is the creation of a document called a course map, which is a formal, module-bymodule specification of the key learning objectives, learning activities, and content components, such as readings, assessments, and assignments. It also identifies any technical components that need to be developed, such as videos. Best practice is for the faculty to produce a standard course map that includes the topics to be covered in each module, course elements, and learning activities. Unless the entire course is designed and sequenced as a whole, the content sequence may be suboptimal, limiting its effectiveness. Indeed, one of the more advanced faculty skills in teaching any subject is knowing its optimal teaching sequence.The optimum sequence may depend on students’ background, because it may be easier for them to master a topic if it is cognitively similar to something they have already studied. Good examples help students understand a topic by providing them with particularized illustrations of concepts, without introducing confusing or irrelevant detail. For example, in one of our transition courses we originally covered college algebra in two modules, but we discovered that students without mathematics backgrounds struggled to learn the material in this compressed format.We redeveloped the course, distributing the mathematics topics throughout. In another example, in a database course we originally followed the textbook’s sequencing, but we learned that it did not allow enough time for students to master Structured Query Language (SQL), so we redeveloped the course with parallel tracks on database design and SQL. Course maps are usually created as a sequence of module maps, in which each module map outlines the content of a module. An example module map appears in Table 6.1. Each section of the course map can include hyperlinks to detailed descriptions and examples. These descriptions and examples can be of great help for faculty new to online development. For example, the learning objectives portion of a course-map worksheet can contain a hyperlink to sample learning objectives from other courses. When the course map has been developed, it may be reviewed by other faculty to evaluate the course’s workload, academic integrity, completeness, and relationship to other courses. Within each module, best practice is to include a study guide to the sequence in which students should perform the module’s learning activities. For example, a module study guide may recommend that students begin by reading a textbook chapter before reading the online course material but that they can do the discussion forum, synchronous session with faculty, and exercises in any order they wish.

Fundamentals of Course Design and Development  93 Table 6.1  Example module map Item

What it is

Example

Overview

General topics and overview of content

The parts of speech and prepositional phrases

Learning objectives

Learning objectives for this module

Given a sentence, to be able to identify the parts of speech, particularly prepositions and prepositional phrases

Prerequisite(s)*

What students need to do before beginning this module

Complete modules 1 & 2

Required reading*

Book chapters and page numbers, articles, web resources, videos, etc.

• Lecture 3 • Read From the Mixed-Up Files of Mrs. Basil E. Frankweiler

Individual assignment(s)*

Synopsis of individual assignment(s)

Complete workbook, exercise 3

Group assignment(s)*

Synopsis of group assignment(s)

Challenge response assignment—students pose sentences and other students identify parts of speech, particularly prepositions and prepositional phrases

Quizzes/exams*

Number of questions, and format—multiple choice, multiple answer, essay, etc.

Quiz 3—20 multiplechoice questions

Discussion topic(s)*

Topics to be covered

• The roles of quotation in parts of speech • Claudia’s annoyance with Jamie’s misuse of prepositions in the Mixed-Up Files of Mrs. Basil E. Frankweiler

Synchronous session(s)*

Purpose, format, and host of all sessions

Review session with faculty, 7:00 p.m. ET Sunday

* Calculate estimated time to complete to determine the total student workload (see section on leveling student workload).

94  Fundamentals of Course Design and Development

Plan before writing Faculty who are new to online education often make the mistake of writing lecture content before outlining how the whole course will work together. Sadly, this often results in material being developed which is not used. For example, one of the authors developed a course without having a clear understanding of how much material would be covered. He developed too much material for the first half of the course, leaving little room for the second half. As a result of this, the material for the first half had to be rewritten. This could have been avoided if a detailed design had been developed before the writing began. An insufficiently detailed design may necessitate moving content multiple times as the structure of the course evolves. Faculty who are accustomed to a more freestyle approach in faceto-face lectures and who are unfamiliar with this process can be guided by a faculty mentor or designer.

Creating Assessments A good approach to developing a module is to begin by creating assessments that determine how well students have learned the material; then the rest of the material can be created to prepare them for the assessments. In most programs, assessments are graded, and the overall course score is defined as a weighted average of their scores and other graded activities. Assessments should cover all learning objectives. Assessments help students and faculty identify topics where students need additional study. They can also help teach the material, particularly if the assessments are automatically graded and released promptly with detailed feedback.To serve these roles, well-designed assessments ask students to apply what they have learned, rather than simply recalling material from the lessons. It is important that assessment questions be at the appropriate level of difficulty; if a question is too easy the students will not learn from it, and if it is too hard, they will not even be able to attempt to answer it. Well-designed assessments make good use of students’ time by providing them feedback on what they know and what they need to work on. Assessment questions should measure how well students have met the learning objectives. Question validity can be measured by how well the students’ scores on a question correlate with other measures of their performance on the same topic.When performance on an assessment question is not correlated with student performance, it undermines the validity of grading. If a question is poorly phrased, it may inadvertently test students’ expectations of what they expect to be asked or their language skills rather than their understanding of the subject matter. If students have a deep understanding of the subject matter which may cause them to answer differently than the faculty expect, this might result

Fundamentals of Course Design and Development  95

in a poor score even though they understand the material well. For example, a physics student who has studied Einstein’s theory of relativity might answer a question on Newton’s laws of motion quite differently from one who has not. There are methods for assessing how well questions measure students’ understanding, but only after a significant number of students have answered them. The basic technique is to analyze the statistics for individual questions, including statistics that compare students’ performance on each question with performance on the assessment as a whole. One of these techniques is to examine how well the score on a question predicts the score for the assessment as a whole.This is a useful indicator of question quality, because we expect that students who do well overall would tend to do well on any question, and if not, that question should be scrutinized. Some LMSs compute the difference between the average scores for the students who scored in the top quartile of the assessment and the students who scored in the bottom quartile. If this value is a negative number, the question should be regarded as defective and should not be counted toward the assessment score. If a question has a low, but positive value, it is not doing a good job at measuring the learning of a topic, but it will not corrupt the assessment score. Faculty usually strive for assessment questions with the highest possible number. This approach should be used with caution, because it is possible to design assessments in which the best students will score well, but slightly weaker students will fail, as their understanding is just short of what is needed to answer the questions correctly. It is also important to calibrate questions—to adjust their difficulty, so they are neither too hard nor too easy for students. If almost all of the students answer a question correctly, faculty should consider making it more difficult. If a question is at the right level of difficulty for students, they will be challenged to answer it correctly, and the grading feedback will help them understand what they have mastered and what they need to study more.

The Roles of Assessments in Courses Assessments help students learn and measure what they have learned. Formative assessments are used to reinforce and guide students’ understanding. Ideally, they are provided immediately after a student has studied something, to assess their understanding of the topic. Summative assessments are used to determine how well a student has met the learning objectives. In our online courses, we typically introduce one topic per page, in which the concept is introduced, explained, and demonstrated. At the bottom of the page, a formative assessment is offered to ensure that students understand the topic before they move on. At the end of a module, we provide an automatically graded review quiz, and then the summative assessment quiz that counts toward students’ grades. It is often helpful to include a student readiness assessment at the beginning of a course, particularly for students with weaknesses in their background. The

96  Fundamentals of Course Design and Development

results can identify areas in which prerequisite courses did not adequately prepare students, as well as areas in which both individual students and the class as a whole need additional preparation to address shortfalls. Faculty can respond to these needs in many ways, including offering synchronous lectures on the areas of weakness, conducting help sessions, adding readings, or even creating new assignments. For example, one of the authors administered a readiness assessment at the beginning of a database course, and discovered that several students lacked knowledge of object-oriented technology, so he offered an extra synchronous session to prepare those students. In our experience, students appreciate the adaptation of the course to their particular needs; the result has been lower drop rates and higher student satisfaction. The way some LMSs implement assessments works well for summative assessments but not for formative, because assessments are in a separate test area, and the context change interrupts students’ thoughts. Most LMSs are also not very good for formative evaluation because they are designed for module-level testing, while formative assessments are most effective at the concept or topic level. While a variety of tools exist for embedding questions in written lectures, they are fragmented in their abilities and have severe limitations. Many of them: • • • • •

rely on platform- or vendor-specific tools for their creation or editing require third-party plug-ins such as Flash, which limits their ability to work on mobile devices require users to go to the vendor’s website to see statistics or reports rarely give instructors the opportunity to share instructors’ reports with students, due to limited permissions have recurring license costs.

However, there are ways that faculty and designers can work around these limitations. By using the testing features of the LMS, they can create simple, inline formative assessments and link them from the content pages. For example, a course page may introduce and explain a concept, and then provide a link to an assessment where students can test their understanding of the concept. As formative assessments, these tests should be set up in an LMS to not count toward students’ grades, and students should be able to take them multiple times.These tests should be in an automatically graded format, such as multiple answer, so that students can receive immediate feedback written by the faculty. By using the item analysis feature of the LMS, faculty and designers can also use these formative assessment questions to identify areas where the learning material can be improved. We find such inline formative assessments so useful that we developed software to add this capability to the LMS.We developed Test Yourself (TY) items to allow students to see if they understand written lecture material before they have even left the topic page. Since students can test themselves on material on the same page where it is presented, the feedback can reinforce ideas, provide guidance where they need help, and suggest sources to learn more about the

Fundamentals of Course Design and Development  97

topic. We also developed TY items in such a way that they can be used on any device that can display HTML and JavaScript. To ensure that all of these TY items are useful even if the written lecture is printed or saved as a PDF, we wrote our print CSS to display the question, answer, and feedback, even if they are hidden on screen. In addition to the immediate feedback for students, reports are automatically generated which enable faculty to see each question, the question’s feedback, and the cumulative results (Figure 6.1). This figure shows a TY report for a multiple-answer item. In this example, the third and fourth answers are correct, as indicated by the checkmarks, but 26 percent of respondents incorrectly selected the first answer, which the faculty may wish to address. The report also shows that students used this item both during its module week and to review for the final exam. Like the TY questions themselves, these reports can be embedded directly in LMSs, either to be used by faculty, put on a page to be shared with students in review sessions, or a combination of both. These can indicate if students are not using the TY questions, or if they are only using them at the last minute. These results can also be used to inform synchronous discussions and improve course content. For example, if many students are selecting a particular incorrect answer, it might be because the concept was not explained well in the preceding text, or because the question itself needs revision. Faculty can use this information to improve the course. The reports give faculty a solid foundation on which to base updates to the asynchronous content. Rather than waiting for students to ask for concepts to be reviewed, the TY reports enable faculty to determine where students need additional help sooner, and then improve the asynchronous content. Faculty can share the results with students in synchronous sessions. More typical formative assessments can also help students review the module and prepare for module summative assessments. Review questions should have the same structure and content as the module summative assessments. These review questions are an integral part of the online course, and should be automatically graded to allow immediate feedback. They should be offered at the end of a module or other significant stopping point, before summative evaluations such as quizzes or exams. Review questions can also be used as a testing ground for assessment questions, because students are much more tolerant of imperfections in review questions than assessments that count toward their grades. Faculty can improve the acceptance of review question weaknesses by granting a little extra credit when students point them out. The LMS implementation of summative assessments can usually be used for module quizzes, midterm exams and other tests, and final exams. They are normally placed at the end of modules or sections of a course, such as weeks or a halfway point, or the end of a course. Because the summative assessments are all implemented in the LMS, it is convenient to migrate questions between these kinds of assessments. For example, we have found that it works well to create and

98  Fundamentals of Course Design and Development

Figure 6.1  Test Yourself report for multiple-answer question.

Fundamentals of Course Design and Development  99

tune new questions in review question sets, to promote better review questions to module quizzes, and the best questions to the final exam. (For more on validating, calibrating, and tuning assessment questions, see the previous section.) Other kinds of assessments include presentations, discussions, written assignments, interviews, and dozens of other ways of measuring how well students have met learning objectives. These are usually implemented and graded in the LMS. In the case of activities which occur outside the LMS, such as role-plays, students’ grades are entered in the LMS manually for grade calculation.

Elena Garofoli, lead instructional designer at Boston University’s Office of Distance Education The idea of starting with the end in mind is, to me, what drives the entire course development process. Only after goals and objectives have been identified, as well as the means of assessing whether or not the goals/ objectives have been achieved, should the content be determined and parsed into chunks. Content that does not directly support the course goals and objectives should be eliminated. There are two main considerations in assigning weights to assessments in a module.The first consideration is that an assessment with deep coverage of the learning objectives merits higher weight than a shorter assessment with minimal coverage. For example, material that takes 30 percent of the course to convey should count for approximately 30 percent of the final grade. In a similar way, an assignment that takes 40 hours to complete and covers much of the material should count more than an assignment that takes less time and only covers a little material. The second consideration is that students prefer that the weights for similar assessments be the same for all modules; for example, all weekly quizzes should have the same weight. In our experience, this helps students allocate their study time and reduces their anxiety. Maintaining the academic integrity of assessments is another element to be addressed at this stage. It can be strengthened by creating multiple versions of assessments and assignments for each assessed topic before a course runs, to ensure that different students will get different problems. (For more on academic integrity, see Chapter 3.) Proctored assessments should be worth enough so that students who have been cheating on other, non-proctored, assessments will not get a passing grade for the course. In our experience, the sum of proctored assessments should count for at least 30 percent of the final course grade. We have also found that if a

100  Fundamentals of Course Design and Development

single assessment counts more than 30 percent, some students will have so much anxiety that it interferes with their performance.

Planning Learning Activities Learning activities are what students do to learn the material. Their selection depends on overall learning goals, subject matter, student preparation, and available technologies. Example learning activities include reading, watching videos, interacting with multimedia objects, testing one’s knowledge with formative assessments, engaging in synchronous or asynchronous discussions, interacting in study groups with faculty and classmates, and completing assignments, summative assessments, and projects. Learning activities need to support the learning objectives, and they should be aligned with them, in the same sequence, so that students are ready for them. Student preparation also influences the choice of learning activities. For example, some learning activities may include a review of prerequisites, and these are very helpful for students who have forgotten the prerequisite material. Some learning goals emphasize that students be able to solve nontrivial problems; problem-solving assignments are natural learning activities to meet these goals. Other learning goals may be the ability to recall something, such as a formula or law of physics; these learning objectives are naturally met with exercises that ask students to recall and apply the information. Learning activities that do not support learning objectives should be eliminated. All learning activities of a module should be grounded in the learning objectives of that module. Assignments should provide practice working with the material covered in the module, while also integrating material from prior modules. Experienced faculty know that if there is a graded test or other assessment, it should be scheduled after the learning activities. Best practice is to include practice tests, so that students can learn what they have not yet mastered before they take the test that counts toward their grade. The first learning activities are usually combinations of readings, videos, animations, or other learning objects that the students use to familiarize themselves with the module’s content. Later learning activities usually ask students to apply what they have learned and assess how well they have learned it. Students may prefer different orders for these activities or just skip some of them, if they think they know the material or the activities are not compatible with their learning styles. For example, some students may prefer that activities begin with a textbased discussion of the material, a synchronous discussion with their classmates and faculty, working independently on assignments that use the subject matter, taking a practice test, or answering review questions. Learning objectives can usually be met with different learning activities: For example, children can learn multiplication using blocks, flashcards, or pencil and paper. Likewise, one activity may support multiple learning objectives. It is desirable to provide multiple learning activities for each learning objective, because

Fundamentals of Course Design and Development  101

different learning activities may support different learning styles, and because repetition reinforces learning. If the same material is covered in different ways in an asynchronous course, then students are free to select those that best meet their needs, and students can be doing different activities at the same time. For example, one student may prefer to watch a video while another may prefer to read the transcript of the video. As we have mentioned, asynchronous online education has a significant advantage over classroom learning in this regard, because the constraints of the classroom setting typically prevent different students from doing different, incompatible activities at the same time. One of the big decisions in developing learning activities is to define the mix of asynchronous and synchronous components. Many students prefer that material be covered both asynchronously and synchronously. This gives them the opportunity to study the material asynchronously and bring their questions and insights to a synchronous session with their faculty, as is done in flipped classrooms (Roehl, Reddy, & Shannon, 2013). When designing learning activities, the faculty should decide which mix of formats will be most effective. Some technologies, such as videos and animations, are fundamentally asynchronous, and it is a waste of precious synchronous class time to gather everyone just to watch a video.

Jim Frey, learning design director An important question regarding technology is:Where does your program fall on the spectrum of asynchronous vs. synchronous? In other words, will you require your learners to participate in real-time sessions? When designed well, synchronous sessions can be beneficial to learning and provide a dimension to the experience that cannot easily be replicated in the self-paced (asynchronous) portion. But synchronous learning presents challenges. Logistically, it requires learners who may be in multiple time zones to coordinate their schedules.Technically, it requires learners to have access to hardware and software that is up to the task of real-time conferencing (although this is becoming less of a concern as time goes on). Also on the technical side, synchronous sessions put greater demands on instructors, facilitators, and administrators. They require designers to put thought into how to maximize the value of that synchronous time. It is better not to have a synchronous session at all than to ask learners to participate in an activity with little or no apparent benefit.

Sometimes the choice of learning activities depends on the available technologies. For example, if an institution’s platform supports formative and summative assessments embedded in the course content, then these can be very helpful;

102  Fundamentals of Course Design and Development

unfortunately, few, if any, LMSs support this natively. User devices can constrain the learning activities; for example, a student may be easily able to take a multiplechoice test on a phone, but would have difficulty using desktop publishing software to lay out a newspaper page on the same device. Diverse instructional approaches can also have a major impact on the design of learning activities. Best practice is to meet the preferences of different students by covering each of the learning objectives with diverse activities. Some students may require significantly different mixes of learning activities; for instance, students in an introductory music class who are visual learners may benefit from seeing a visualization of the score to supplement the audio track, and students who struggle with math word problems may benefit from displaying them with diagrams. Because students can perform quite differently on assessments of different types, it is helpful to include graded assessments that make use of different modes of instruction. For example, a quiz that assesses students’ ability to understand and apply technical diagrams can include questions based on both text and diagrams. We learn and retain what we learn best when there is repeated interaction with the material, such as reading it, discussing it, applying it in assignments, and then reviewing it for an exam or a final project. Consequently, the sequence of learning activities should include review of material that covered prior learning objectives. Learning activities can be most effective when they combine activities from earlier assignments with activities that cover the current material, to help students integrate the new content with the earlier content. Lecture material that begins with review and builds on it is very effective. In some courses, the learning activities can be organized into an integrative term project that threads throughout an entire course.The main design challenge for such a project is to design a sequence of activities that aligns with the modules’ subject matter and learning objectives. For example, in a database course a term project may require identifying the entities for the first module deliverable, creating the tables for those entities in the second, and so forth, until, through the sequence of module deliverables, they have incrementally created a significant database system. Explicit review activities can be helpful, particularly when those activities are designed to help students prepare for the final exam; examples include review lectures, review problems, and practice final exams. Online technology provides additional ways for students to review. For example, students can use previous quizzes to review; and, if the assessment questions have been randomized in the LMS, they will get new questions, broadening their preparation.

Designing for Students with Varied Preparation In most courses some students will lag behind and others will already know much of the course material. A fundamental problem that faculty face is how to create courses that are challenging for everyone, including the best students, without overwhelming the weaker students. Diversity of preparation is less of a

Fundamentals of Course Design and Development  103

problem in courses with well-aligned prerequisites, but it can be a significant challenge in courses without prerequisites, or when students transfer from academic programs with different curricula. One of the ways to design courses to support this variance in preparation is to provide additional review material for less well-prepared students. We have done this in two ways. The first is to include a Module 0, which covers the prerequisite material for the course, but is not counted toward the final grade. The second approach is to incorporate Help links that cover the prerequisite material. For example, in a physics course, a section on acceleration that assumes prerequisite knowledge of forces could have a help page that covers forces. Challenging advanced students has usually been addressed by subjecting them to more work—extra reading material and assignments, which are frequently no more challenging than the regular work, simply more time-consuming. A more humane and efficient approach allows advanced students to choose if they want to challenge themselves by reading more advanced material and doing advanced assignments, in place of the defaults. Content that is identified as advanced allows anyone the opportunity to go deeper into the subject matter. We encapsulate both the review and advanced material in webpages that include formative evaluations and links for further reading.These approaches can be used to create a course that can be more challenging for all students, without overwhelming any. As is typical for online courses, this requires additional faculty effort up front, but it reduces faculty effort while a course is running to deal with struggling students and students who are not challenged enough. We have found that this “tiered” design also improves student satisfaction.

Aligning the Content by Topic Alignment means coordinating learning activities and assessments by topic, so that students are prepared for them. Ideally, one can cover all of a topic in one module. If that is not feasible, it is sufficient to break the topic into subtopics, which can be taught in different modules. Alignment is different than topic sequencing, because alignment is the coordination of activities within a topic, while topic sequencing is the ordering of topic coverage based on student readiness. Particular care should be taken to maintain alignment when moving material between modules. One way to visualize alignment is to create a table with a row for each of the topics to be covered in the course and a column for each type of object in which it will appear. For example, assume a course has a topic covered in lecture three, with readings from chapter five of the textbook. Students perform a formative assessment with an optional review question set, then are graded with quiz three, assignments three and four, and the final exam. If this topic were assessed earlier, in quiz two, the students would not be ready for it. This table reveals alignment issues and identifies solutions. For example, if an assessment question is on a topic not covered yet, one way to fix this is by moving the learning activities for this topic to an earlier module or moving the question to a later assessment.

104  Fundamentals of Course Design and Development

If the student workload for each learning activity is added to this table, then the table can also be used for workload assessment and leveling (see next section). Modern LMSs allow faculty to tag assessment and assignment problems. For example, questions about the regional variations in Malbec wines in a wine course would be tagged with the metadata identifiers “Malbec” and “regional variations.” This helps maintain alignment of the lectures, assessments, and assignments when they are updated.

Jim Frey, learning design director Providing frequent context and framing is one of the most important things faculty and designers do. Content does not teach itself; a welldesigned learning experience recognizes this and regularly orients learners to their learning journey, communicating why their path is set up the way it is. Do not assume connections among concepts are obvious; help learners see and understand them. (Or challenge learners to hypothesize on those connections and why they are being asked to follow a certain path. This brings us back to the idea of active participation!)

Leveling Workload In our experience, students learn more effectively and with less stress if their workload is reasonable and evenly distributed throughout the course. When a course is first being developed, it is helpful to use a course map that lists all of the learning activities in each week, along with an estimate of the student workload associated with each of those activities. Workload is measured by how long students are expected to take to complete assigned readings, interactive learning objects, assignments, assessments, review questions, synchronous sessions, and discussions. There are four common situations: • • • •

The workload for the course as a whole may be too low, which compromises student learning. The solutions for this may involve increasing the scope of the course and/or adding learning activities. The workload for a specific module may be too low. The solutions for this are to increase the scope of the module, or to move some of the activities from other modules where the workload is higher. The workload for the course as a whole may be too high, which can overwhelm students. The solutions for this may involve reducing the scope of the course and/or improving the efficiency of the learning assignments. The workload for a specific module may be too high.The solutions for this are to reduce the scope of the module, or to move some of the learning activities to other modules.

Fundamentals of Course Design and Development  105

Workload leveling is a multistep process. When faculty are creating the syllabus, they attempt to distribute the material evenly. As part of the initial development, faculty estimate how much time it will take students to complete each of the learning activities, and sum them by module. Faculty lecture at about 150 words per minute (Miller, Maruyama, Beaber, & Valone, 1976), so in a 50-minute class one would expect to have about 7,500 words of explanation, student response, and feedback. While this will vary by discipline, with this knowledge, the workload for students to study a lecture can be estimated before the lecture is written; counting the pages devoted to this topic in textbooks or other readings may also help. Work per page differs based on the subject matter, so corrections need to be made for specifics like equations, new vocabulary, or other, more time-consuming reading. Workloads for completing assignments can be estimated based on prior experience teaching with this assignment; if an assignment is new, the workload can be estimated based on the number of steps that the student must go through and the size of the deliverables. Most students budget about the same amount of time to prepare for each assessment; these time budgets can be obtained by asking former students. Faculty can estimate student workload by adding this preparation time to the allotted time for actually taking the assessment. The workload for discussions varies widely, based on what is expected of students and discussion types. For example, some discussions are graded based on the number of posts or follow-ups, summarizing the posts of others, or are like small research papers that require citations. After students have taken the course, workload estimates can be improved by asking them how much time they spent on each of the learning activities. Likewise, data on the time that students spend on activities such as watching videos can often be obtained from course servers. The total workload of each module is estimated as the sum of the learning activity workloads. If the workload is significantly different between modules, this may warrant moving assignments between the modules to level it. In courses with major term projects, the workload for activities such as regular assignments should be reduced near the end of the term, to allow students more time to complete their projects.

Developing and Implementing Content When faculty are satisfied with the course map, have chosen their assessments and learning activities, and the student workload has been appropriately distributed, the course development is about half done. Then, the main content development begins. Content development usually progresses in the sequence that the content will be studied, which may be based on a textbook. This phase typically takes three months to a year, depending primarily on how many faculty are contributing, and how much time faculty can allocate to the task. Tracking the level of effort per course is useful, to address faculty workload and help development stay on schedule. Workload is estimated before development starts and is updated as necessary.

106  Fundamentals of Course Design and Development

Most faculty implement content directly in the LMS. Modern LMSs support what-you-see-is-what-you-get HTML formatting without having faculty need to know or even see HTML. For example, most faculty will write their lectures in their favorite word processor, and then paste them into the LMS. Faculty can also insert photos, graphics, and videos into the course using the LMS, as well as creating assignments and assessments, and assigning their values to the course grade book. (For a discussion of how this can be done with the assistance of designers and staff, see Chapter 7.) Best practice in developing new courses is to begin with a template course (Borgemenke, Holt, & Fish, 2016). Template courses are built in the program’s LMS, with standard structure and navigation. The program’s template course includes content that is common to all courses in the program, such as styles, links to the master pages for academic integrity policies for the program, contact information for the program administrators, technical support, and help. Every universal design element that can be included in a template course should be included. This is particularly important when courses are developed by faculty without the assistance of designers, or when several people are working on the same course. (For more on universal design, see Chapter 7.)

Thomas Cavanagh, vice provost for digital learning at University of Central Florida I have worked at both an institution that used master course templates for all online courses and one that did not. Each suited the academic culture and context, as well as the programmatic requirements of each institution. There is no one-size-fits-all solution. For example, what works at a small liberal arts college may not be appropriate for a large community college system.

Making Videos There are three considerations for making successful videos, for both the course and individual modules: • •

Keep them short. We have found that viewers begin to drop off when videos are longer than about four minutes. Convey enthusiasm. One of the best uses for video is to have faculty do short introductory ones to share their enthusiasm for the subject and explain why it is important. Faculty should be mindful of students’ time by not telling them what is going to be covered, what assignments will be due, or anything they can get from the syllabus. Keeping changeable material out of the videos also eliminates the need to frequently edit or re-record them.

Fundamentals of Course Design and Development  107

•

Make them interesting. Use video as an opportunity to tell a story, provide background information, show how something works, or demonstrate a concept. For example, in a video for a psychology course, the faculty talks about Phineas Gage, the victim of a railroad construction explosion who had a three-foot iron rod driven through his head in 1848 and lived to end up in every abnormal psychology textbook. Her animated storytelling, interspersed with photos and diagrams, brings the story to life. In another example, in one of our most popular introductory videos, a faculty lectures from his kitchen as he cooks bacon. The sizzling can be heard in the background as he explains how early in the 20th century Edward Bernays used ideas from his uncle, Sigmund Freud, to convince consumers that bacon and eggs were an essential part of an American breakfast.

There are also technical considerations in producing high-quality videos. While it may seem humanizing and add “personality” to record professors in their offices, they rarely have adequate lighting and may be noisy, so such spaces are often unsuitable for making videos. Backdrops, lights, tripods, and other equipment are useful for programs as they grow. If possible, a dedicated space can serve as a studio, to save setup time and provide a convenient place to store equipment. The more a program uses videos, the more it will benefit from areas that work well as recording studios. Best practice is to use a small, quiet recording studio with lighting that illuminates the faces of the faculty well, a good camera, and a good microphone. Such a setup need not be complex or expensive; moreover, the same setup can also be used to create videos for marketing and for courses. Lavalier microphones clipped to the faculty’s clothes or worn around the neck work better than microphones fixed in space, because the volume from fixed microphones varies when faculty move their heads. For institutions with limited resources, teaching faculty how to use available lamps and headsets at home greatly improves the quality of their videos. Simple table or floor lamps in front of faculty work better than overhead room lighting, which creates unflattering facial shadows. Lighting should be warm white, as is used in homes.Video production quality can be improved by editing, by either faculty or staff.

Pre-launch Tasks The following tasks need to be performed for each course before it runs. Faculty will need to perform these tasks if their program does not have designers or other staff. All of these tasks can be performed in parallel or in any other order; we complete them about three weeks before the course start date, to allow time for QA.

108  Fundamentals of Course Design and Development

Tasks in the LMS to prepare the course for running • • • • • • • •

Upload course content. Set due dates, as well as quiz and exam dates, in the course syllabus and LMS. Verify that all mandatory program elements are present in the syllabus and course, such as links to libraries, tutoring services, and career services. Test the function of extensions to the LMS. For example, if JavaScript is used in a course to create PDF extracts of lectures, it should be tested. Check that internal links work. Check that links to external websites work, and the pages have not changed. Verify that when someone logs in as a student the course works properly. Ensure that course grade book settings are correct.

Tasks for copies of previously run courses • • • • • •

Delete old discussion posts. Delete old announcements. Delete old synchronous session recordings. Update the course calendar. Check that internal links still work. Check that links to external webpages still work, and that the pages have not changed.

Addressing Delayed Development The most common course development issue is failure to complete the work on schedule. Unless a course is being developed by faculty with a good course development track record, best practice is to not advertise the offering until the program administration has determined that it will be ready on time. We have found that paying faculty on completion of development, not before, increases the likelihood of on-time development. A number of things can be done to accelerate course development. The most basic technique is to arrange for the faculty who are writing the content to have more time to write. For example, they may receive course release. Another approach is to have several faculty work in parallel on the same course. For this to be successful, the course needs to be defined in advance, down to the detail of the course map.When multiple faculty are writing a course, additional faculty administrative effort is needed to ensure that topic sequencing is correct, that navigation and design are consistent, and that grading appropriately integrates these independently written components. If necessary, several actions can be performed to successfully offer an incompletely developed course. One is to develop the missing content while the course is running. For example, one of the authors had insufficient time to develop a course and was forced to write the last two modules while he was

Fundamentals of Course Design and Development  109

teaching the first four modules. A second option is to deliver the undeveloped material synchronously in lectures, as in a face-to-face class. Transcriptions of these synchronous lectures may be used as a starting point for creating the online content to complete the development (the stone in the “stone soup;” for more on stone-soup development, see Chapter 7). Several of our faculty have used this method to create a first draft of their online lectures from recordings of their face-to-face lectures.

Building Courses to Facilitate Updates Over the lifecycle of a course most of the effort is spent on updates, so courses should be built to minimize the effort it takes to improve them. Courses are like constantly evolving organisms, with their roots and feelers reaching into the program, the institution, and the internet. Courses should be built to make it easy to adapt them for each term. One problematic type of update is moving course content, because references to signposts such as section numbers need to be found and changed. Ideally, each course module should be built so that it does not need updating if it is moved. To accomplish this encapsulation, one should avoid references outside of the modules themselves, either explicitly or implicitly. For example, it is better to have faculty say in a video, “In this module we will…” rather than “In module five we will…”. Best practice is to avoid making references to elements or modules which came before or will come after, because if the order is changed, the references will no longer be correct. However, it is often helpful for course sections to include references to prior sections; if this is done, there should be a mechanism to identify the references so that they can be updated. For example, if modules refer to sections in other modules, and the words “see Section” are used everywhere, then it is easy to locate all of these inter-module references. One of the fundamental design principles is locality of design, which means that each item appears in only one place in a course. An example of this principle is due dates. A typical course will have multiple assessments, assignments, and other deliverables with due dates; these due dates may be sprinkled throughout a course. After the first update of the course this information will be incorrect, and the faculty must go through the entire course, finding and replacing the old dates, before the course runs again. Inevitably, at least one old due date will be missed, only to be found by a student. Best practice to prevent this from happening is to list due dates only in one place, such as the syllabus, with other parts of the course referencing that place. Locality of design also applies to assignment descriptions and grade weights. If faculty scatter assignment descriptions and their weights throughout the course, it will be difficult to find and update all of them. If the weights for some  assignments are in the modules, as in the text of the lectures, and the weights for other assignments are in the syllabus, then students may have

110  Fundamentals of Course Design and Development

difficulty determining where to look for them. Likewise, faculty will have difficulty finding all of the grade weights when they are adjusting the course grading, and they may miss some of them, resulting in inconsistencies. Best practice is to implement assignments as standalone documents which contain the complete assignment instructions, but not include due dates, grade weightings, or references to the date or term in which they were created. If this is done, the due dates or weights can be modified by changing only the syllabus. Some assignments may be given each time a course runs, sometimes for several years in a row, but students may complain if they see a creation date that shows that it has not been updated recently. Using version numbers instead of term dates can help with document version management without revealing the age of documents. Adding elements to enhance and expand a course incrementally, without requiring significant redevelopment, is a basic tenet of building to facilitate updates. Consider an online math course containing only text and illustrations, with solutions to assignments provided as documents. Faculty can enhance this by making videos of themselves working through the problems, as they do in face-to-face courses. This course enhancement does not require creating new problems and solutions; it simply offers a different way to see the problems being solved. Some elements can be added to a course to improve it without modifying any existing course content. For example, faculty can add formative assessments between paragraphs, so that students can test their understanding as they learn. Later, faculty can improve the assessment feedback with detailed explanations and suggestions. For example, questions with a single correct answer can be converted to more difficult questions that allow multiple correct answers (Figure 6.2). (For more on incremental course development and the stone-soup method, see Chapter 7.) Another incremental way to improve a course is to use information that students provide in solving assignments and assessments. For example, a common incorrect student response to a short-answer question can be used as an incorrect answer when the question is converted to an automatically graded question. Exemplary student submissions can also be used as examples of good solutions. Faculty should always obtain permission from students to use their submissions as examples. Faculty can also provide students with graded assignment solutions to help them understand what they need to do to get a good grade. In our experience, when students see samples of outstanding work from prior terms it is effective in reducing grading complaints and grade begging. In addition, we have experimented with having students create their own test questions as a study aid. While many of these questions were not good enough to be used “as is,” they are often the seeds of good questions. Over time, such methods increase the number of validated and calibrated assessment questions, enabling faculty to assemble them into topical question sets, from which the LMS can randomly select questions, improving the academic integrity of the course. One essential part of building for revision is maintaining the organization of the underlying files, deleting files and references to files that are no longer needed. For example, remove out-of-date references to JavaScript or CSS from

Fundamentals of Course Design and Development  111

Figure 6.2  Inline formative assessment displaying correct answers and feedback.

112  Fundamentals of Course Design and Development

HTML files. If a file is no longer being used in a course, such as a topic that is no longer covered, consider deleting it. Deleting unused files makes it easier to locate the files that matter when revising the course, saves server space, and makes copying future iterations of the course faster. It will also save potential errors of working on incorrect versions of a file. If a copy of a course is made for each iteration and a file is deleted that is still being used, it can always be recovered from a previous version of the course. If obsolete files must be retained, we recommend making a folder with a name like “DeleteAfter2021-10-31” and storing them there. When one returns to the course, it is obvious if the folder and its contents should be deleted. One of the most useful techniques in designing to facilitate updates is to localize the material that will soon be out of date, such as emerging technologies or politics. Such ephemeral content can be localized in synchronous sessions, and in asynchronous learning activities such as discussion questions and links to external sites about those topics. Sometimes students can be encouraged to post and report on evolving topics of general interest. If an ephemeral topic is only of interest to some students, faculty can generate discussion forums or study groups for those students.

Fawn Thompson, lead instructional designer at Boston University’s Office of Distance Education Populate your modules with content that will not become quickly dated. Consider identifying one area or tool that you will use to post timely or date-sensitive information throughout the course cycle. Many faculty with whom I work use the discussion tool to set up an area for current events. Others prefer to use a synchronous tool such as Zoom to hold interactive sessions throughout the course run.

A flexible course design can accommodate courses with different term lengths and modes of instruction. One way that we accomplish this is by using the term “module” instead of “week” throughout the course.This allows the same course to be used for different length terms, simply by changing the due dates, without the risk of confusing students when a “week” is no longer seven days. Similarly, we have used the same course website for face-to-face and online courses.

Summary Checklist Plan course development. ☑ ☑ Determine how to use design standards and award rubrics. ☑ Determine how to design for meaningful interaction.

Fundamentals of Course Design and Development  113

☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑

Define course goals and learning objectives. Create the course map. Define the course assessments. Identify the summative assessments and grade weighting. Plan student learning activities. Determine how to accommodate students with varied preparation. Align the content by topic. Level the course workload. Develop and implement the course content. Make course videos. Complete pre-launch tasks. Devise a plan to address delayed development. Develop a process that facilitates course updates.

References Almala, A. H. (2007). Review of current issues in quality e-learning environments. Distance Learning, 4(3), 23–30. Borgemenke, A. J., Holt, W. C., & Fish, W. W. (2016). Universal course shell template design and implementation to enhance student outcomes in online coursework. In A. A. Piña & J. B. Huett (Eds), Beyond the online course: Leadership perspectives on e-Learning (pp. 147–157). Charlotte, NC: Information Age Publishing, Inc. Carless, D., Salter, D., Yang, M., & Lam, J. (2011). Developing sustainable feedback practices. Studies in Higher Education, 36(4), 395–407. Dixson, M. D. (2015). Measuring student engagement in the online course: The Online Student Engagement Scale (OSE). Online Learning, 19(4), 1–15. Mager, R. F. (1997). Preparing instructional objectives: A critical tool in the development of effective instruction (3rd ed.). Atlanta: Center for Effective Performance. McGahan, S. J., Jackson, C. M., & Premer, K. (2015). Online course quality assurance: Development of a quality checklist. InSight: A Journal of Scholarly Teaching, 10, 126–140. Miller, N., Maruyama, G., Beaber, R. J., & Valone, K. (1976). Speed of speech and persuasion. Journal of Personality and Social Psychology, 34(4), 615–624. Quote Investigator. (2019, November 14). I try to leave out the parts that people skip [Blog post]. Raible, J., Bennett, L., & Bastedo, K. (2016). Writing measurable learning objectives to aid successful online course development. International Journal for the Scholarship of Technology Enhanced Learning, 1(1), 112–122. Roehl, A., Reddy, S. L., & Shannon, G. J. (2013). The flipped classroom: An opportunity to engage millennial students through active learning strategies. Journal of Family & Consumer Sciences, 105(2), 44–49. Strunk, W., Jr., & White, E. B. (1979). The elements of style (3rd ed.). New York: Macmillan. Woo, Y., & Reeves, T. C. (2007). Meaningful interaction in web-based learning: A social constructivist interpretation. Internet and Higher Education, 10, 15–25.

Chapter 7

Advanced Course Design

This chapter is written primarily for faculty and designers. It covers some of the more advanced elements and procedures that can be used when resources are available. It will also help administrators and support staff understand how courses are developed, why it takes so much time and effort, and how courses evolve over time. This chapter covers: • • • • • • • • • • • • • • •

Hiring designers, support staff, and videographers. Planning course development. Holding a development kickoff meeting. Motivating faculty. Building for universal design. Avoiding multiple copies of content. Deciding what technologies to use. Synchronous sessions. Telepresence. Integrating multimedia content, simulators, and virtual labs. Managing production tasks. Copy-editing. Pre-launch designer tasks. Performing faculty QA. Final preparation for launch.

The previous chapter described the basic course development process when only faculty are involved; most can develop written material, assessments, and assignments without assistance. Programs with more resources can provide faculty with design and pedagogic help throughout the process. Such programs may also develop more sophisticated course components, such as multimedia interactive objects and virtual labs. Best practices for course building vary, based on the size and composition of the development team, the complexity of the course, and the subject matter. This chapter includes recommendations for coordinating production and

Advanced Course Design  115

routine maintenance tasks across the course development team. It also describes additional quality assurance processes, such as reviews by editors, faculty, and facilitators.

Hiring Designers, Support Staff, and Videographers Many programs start out leaving everything up to faculty, but as they grow, they may invest in staff to improve quality and reduce faculty workload (Neely & Tucker, 2010). Programs with limited resources may start by hiring a part-time student worker with design or media production experience, rather than a fulltime employee. The order in which staff are acquired will depend on the nature of the programs and how their courses are taught. For example, a program that builds courses containing a large number of edited videos would find their resources well invested in videographers. Similarly, if a program makes extensive use of proctored exams, and faculty have to manage that process, they may want to hire someone to coordinate this effort, taking the workload off faculty and ensuring consistency. All programs will be rewarded by having specialized staff to support students and faculty, rather than relying entirely on faculty. The three most common additional staff are designers, media specialists (typically videographers), and faculty and student support staff. Although they all require technical knowledge for working with online courses and diplomatic skills for working with faculty, the ratios of these vary greatly depending on their primary role (see Figure 7.1). For most programs, the first hire should usually be an instructional designer, because a designer will help shape the structure of the courses. Many designers also have skills in areas such as multimedia objects, video, and graphics. An instructional designer should have the following skills and knowledge: • •

Ability to develop teaching material effectively. Ability to work with faculty and guide them through the instructional design process—while designers may know more than faculty about some

Figure 7.1  Technical knowledge and diplomacy ratios.

116  Advanced Course Design

• • • • • •

aspects of what is required to make a good course, they often need to accomplish this by influencing faculty. Persuading faculty to do more work to make a course better can require considerable diplomacy. (See the “stonesoup” method later in this chapter.) Ability to present material effectively—like faculty, designers need to know how to present and explain things clearly to different audiences. Ability to estimate production time—how much work can be done on a course in the time allotted. Ability to manage changing priorities and deadlines. Attention to detail and ability to check one’s own work—skilled designers check their work before declaring it complete, to minimize errors. Ability to communicate—the larger the design team, the more essential the ability to communicate successfully with faculty and teammates. Ability to apply a coherent instructional design process that incorporates all of the above elements.

Jim Frey, learning design director For me, the most important things to keep in mind when creating and maintaining a working relationship with faculty are respect and partnership. Faculty are involved in this process because they bring experience and expertise in the subject matter and in teaching it. You as a designer bring your own expertise and role to play, as well. But you are not there simply to tell faculty what to do. You are partners in this process. When each of you respects what the other brings to the table, that will create the best possible working relationship. Part of showing respect and partnership is being highly communicative about not just the what, but the why of the process. In a way, you are doing learning design for the faculty member as well as for the learners. Think about how best to communicate with the faculty you work with, according to their needs and preferences. What information do they need to know? When do they need to know it? What is the most effective method of communication? Your working relationship will be better for it.

In addition to these skills, there are two questions to ask designer candidates in interviews, to gauge how they think about design and how they would discuss it with faculty.They are related, so we expect to see a fair amount of overlap in the answers. •

The first question is: “To have students succeed, what elements are essential in every course?”

Advanced Course Design  117

•

The second question we call “the PowerPoint question.” It describes a scenario in which a faculty has been teaching a course face to face and will be putting it online with the help of a designer. For the first module to be developed, the faculty just sends the designer a PowerPoint file, says that these are the slides that have been used to teach the course, and that is all there is. What do you say?

In reply to the first question, experienced designers will mention interaction, explain its importance in education, and describe methods and learning activities that promote interaction with course content, faculty, and other students. The answer we are looking for with the second question is, “it would be better to add more.” Not just more ways of delivering the content, but also the complete text of what is said in the face-to-face classroom. When an instructor uses PowerPoint in a face-to-face course, they do not just display the slides and click through them in silence; they explain things as they go along. They ask questions. They see if students understand. They rephrase things and give different examples. The designer should emphasize that there is more going on in a faceto-face class than displaying slides, and that there are a variety of ways to simulate that interaction online.

Fawn Thompson, lead instructional designer at Boston University’s Office of Distance Education Once you find qualified instructional designers, training and mentoring them are of utmost importance. It is easy—especially with the best designers—to consider your job done once they are trained and working independently. Not so! Ignoring your direct reports will cause them to feel bored and undervalued. Mentoring without micromanaging is a tough balance, but vital to team success. Here are a few tips: •

Recognize designers’ work in ways that are most meaningful to them. Public acknowledgment of special work they have accomplished is great for many people, but if they happen to hate public attention, then an email or personal conversation noting your appreciation of key elements might be better. • Use their performance evaluations as a roadmap for career growth. Set regular meetings throughout the year to discuss development and goals, both long and short term. Ask how you can help, or offer specific resources. When possible, make connections with others in the organization to promote cross-team development.

118  Advanced Course Design

• When a team member comes to you for advice or an impromptu meeting, close or minimize whatever is on your screen and set aside any paperwork so it is clear that they have your full attention. • If someone really enjoys a specific area of instructional design, try to ensure they are involved in projects that would allow them to explore that topic further. However… • Don’t assume that because someone is good at something, they must enjoy it. They may actually hate it and do it only to please you and others on the team. Make sure that any dreaded work is equally distributed. • Find ways to share works in progress or design challenges. A few years ago, I introduced the idea of peer review to my team; we took turns sharing course challenges or ideas after every launch, and providing each other with feedback. This practice was later adopted by the larger designer team at our organization. • Share your mistakes.Wallow in them. Point out how others can learn from them. Bonus: Designers on your team will feel more comfortable coming to you when they make mistakes early on, when damage control is easier to accomplish, if they know you will consider these to be “learning experiences” rather than opportunities for chastisement.

Hiring for other specialized staff such as support and videographers is similar to designers. They, too, require the ability to manage their priorities and time, and communicate with their team members. When hiring student and faculty support staff, we look for two kinds of experience; customer service and technical. While experience working in higher education is an advantage, working in a restaurant, help desk, or other customerfacing, high-stress job also counts. Second, the applicant should have good basic technical skills. Ideally, they have experience being an online student themselves, or at least are conversant with office suite software and browsers. The interview question that we use with a potential support person is, “Suppose there’s a professor who is reluctant to do what you want them to do, such as take the LMS orientation course so they will have fewer technical-support queries. What would you say to them to get them to do it?” We are looking for an answer that shows that the applicant can work well with people and displays emotional maturity. Ideally, they can respect faculty and see their viewpoint, while remaining firm and being able to explain objectively why things would be best for everyone if they did the orientation course. Videographers generally have the fewest interactions with faculty, but spend most of their days working with technology, so they are expected to have

Advanced Course Design  119

advanced technical skills. These applicants should provide samples of work to demonstrate that they have successful experience with video and audio, recording in both studio and outside locations, and how to light a scene. In addition to answering questions about how they created the material in the samples, an interview with a videographer applicant should ask, “Describe a production problem that you encountered and how you overcame it.” Videographers should also have experience with post-production, and know editing programs well. They should be able to troubleshoot issues, such as fixing audio problems. They should know graphics programs such as Photoshop and Illustrator, how to import and export footage, and how to follow scripts.

Planning Course Development The course development plan should be created by faculty, ideally with assistance from designers. Faculty are the driving force behind course development and teaching, but instructional designers and other production staff can be very helpful to them. The perspectives of faculty and designers are complementary. Faculty have a mental model of what they want to teach, whereas designers see courses as collections of elements for teaching this material online. Much of the success of advanced instructional development depends on the dialogue and cooperation between faculty and course designers. And, as noted earlier, faculty are often not accustomed to developing a course as part of a team, so the planning process can help faculty understand the importance of the teamwork required. Faculty who are new to online teaching should consult with more experienced faculty for help in this area (for more on this, see Chapter 2). The development plan includes a list of the team members and their responsibilities, a description of how the development is to be coordinated, the deliverables, and the development schedule. It may also include procedures for determining when a course is ready to launch, course-specific quality procedures and milestones, and assessments of risks, such as schedule slippage or poor course quality. The course development schedule should allow enough time to perform quality assurance after the course is built, but before launch, if possible (Figure 7.2). Table 7.1 illustrates the main steps in implementing content, with and without designers and editors. Editors’ work is in brackets to indicate that it can be done at either point. In many programs, the editors may include peer faculty, senior facilitators, or outside experts. (For more about editors, see the section later in this chapter.) If there are instructional designers, this writing and implementation process involves a dialogue between them and the faculty, as the faculty describe and write about what they want to teach and the designers help them determine how best to teach it online. The result of this content development phase is a course website that is ready for quality assurance (for more on QA, see later in this chapter).

120  Advanced Course Design

Figure 7.2  Example development schedule of a seven-week course. Table 7.1  Implementing content with and without designers and editors Without designers Without editors With editors

1. Faculty write content 2. Faculty implement content 3. Faculty review content and make changes 1. Faculty write content 2. [Editors suggest changes for faculty approval] 3. Faculty review content and make changes 4. Faculty implement content 5. [Editors suggest changes for faculty approval] 6. [Faculty review content and make changes]

With designers

1. Faculty write content 2. Designers implement content 3. Faculty review 4. Designers implement changes 1. Faculty write content 2. [Editors suggest changes for faculty approval] 3. Faculty review content 4. Designers implement content 5. Designers implement changes 6. [Faculty review content and make changes]

Advanced Course Design  121

Fawn Thompson, lead instructional designer at Boston University’s Office of Distance Education Learning about faculty goals, listening closely and responding to their concerns, and being very clear about processes and roles is important in establishing trust with faculty. Toward the end of course development for new courses, the support administrator and I set up a virtual meeting with faculty, to celebrate the course completion and help faculty feel ready and supported as they embark on their first launch.

Holding a Development Kickoff Meeting Once a course development team has been assembled, it is recommended to hold a kickoff meeting for everyone involved, especially for those who are new to online course development. Experienced faculty or a faculty coordinator should lead. The goals of this meeting include reaching consensus on the course deliverables and the delivery schedule, including the completion dates for each course module, video, animation, and other educational media elements. We recommend that the team first produce a representative module that contains elements frequently used in other modules, rather than starting with the first module, which tends to be predominantly review and introductory material. For example, a course on social work might begin with a theory review, with later modules involving extensive role-play exercises. This gives everyone an opportunity to determine what works and what does not, and how long it will take to produce the other modules.This initial development of a representative module may result in changes to the design and the development schedule.

Motivating Faculty One of the key challenges in developing online courses is engaging faculty in their development. One of the most effective ways of motivating faculty to develop online courses is by showing them how the material developed for these courses improves students’ learning, reduces their own workload while the courses are running, and improves faculty and course evaluations. For faculty, the difficult part is finding enough time to create courses and make them better. For designers, the difficult part is getting faculty to deliver content on a reasonable schedule. Courses can always be improved, by providing additional examples, more formative evaluations, supplementary readings, links to additional resources, additional assignments, and practice assessments. Most people find it easier to react to something and improve it than to create something from scratch, so starting with less than ideal content and working to

122  Advanced Course Design

improve it can help development. We take advantage of this by something that we call stone soup development. Children all over the world learn a variant of the folk story about stone soup, in which strangers come to a town, bringing an empty cooking pot.The townspeople refuse to share their food with the strangers, so the strangers fill the pot with water, add a stone, and place it over a fire. The townspeople ask what they are doing and are told that they are making stone soup, which is delicious, and they would love to share it with the townspeople, but it needs just a bit of salt before it will be ready.The townspeople do not mind adding some salt to the pot to try this magical soup, and then some carrots, onions, celery, and so on. The result is, of course, soup, and the townies are amazed and everyone gets soup. When told to children, this is supposed to be a story about sharing and other values, but for the crafty designer, it is a how-to manual on getting course content from busy faculty. It shows how to make a good course a great course. The better courses get, the more enthusiastic faculty become, so they magically manage to find time for course improvements. For many courses, the textbook is the stone in the stone soup, or, if a course has been taught face to face, the faculty PowerPoint slides may be the stone.

Elena Garofoli, lead instructional designer at Boston University’s Office of Distance Education Ask faculty what is most important for them to accomplish in their course. Let them know you are a resource to support them in their successful teaching and their students’ learning. Realize that not all faculty will be eager to adopt every suggestion you offer and proceed accordingly. Believe that trust builds over time in faculty–instructional designer relationships. Approach each interaction with the belief that change is possible.

A related technique that our designers have found effective is to inspire faculty to provide material by supplying default text for them to approve or revise, rather than leaving a blank space to fill in or an “X” placeholder for replacement. Good filler text specifies what faculty might want to communicate and it helps them identify what may need to be changed. We have found that faculty are much more likely to improve a default paragraph than to write a whole new one. To ensure that default text can be spotted easily by faculty members or searched by computer, we strongly recommend that defaults be displayed in a consistent, highly visible style. Our designers use a text style that highlights the selected text with bold garish red type on a yellow background, so that faculty can quickly see what needs approval or revision. For example, if faculty are adding synchronous

Advanced Course Design  123

office hours to the syllabus, there should be default text. Once it is approved or modified, the instructor or designer can remove the highlighting.

A note on the dangers of placeholder content While placeholder text, images, and objects are fine for mocking up examples to see what pages of a course will look like when complete, there are dangers. Placeholder content can easily allow one to succumb to a form of banner blindness in which one subconsciously classifies items on pages as unimportant and fails to notice them. For example, the problem with putting a placeholder image of a faculty member in the syllabus, to show what the faculty information page will look like when completed, is that it allows designers and other faculty to see it multiple times, mentally note that a picture has been placed, and then forget that the picture is not the one that is supposed to be used in the final version. In a similar way, when one is skimming through a course to see what needs to be added, it is easy to mentally categorize a section containing dummy text or objects as complete, because there is something there. If placeholders are used to indicate content that needs to be added or revised, they should be highlighted as described earlier to ensure that they are easy to notice or can be searched by text style for revision or removal.

It is useful to keep a list of potential course improvements in the course developer files, so that as time allows each course gets better with each iteration. For example, courses can usually be improved with inline formative evaluations, module review questions, and interviews with practitioners in the field. Some programs may have standard or recommended course elements or course update plans, which can guide this process. (For more on what prompts course updates, see Chapter 10.) Occasionally, technology or other rapid change can render most of the content of a course obsolete. For example, a course on Egyptian hieroglyphics would have become obsolete with the discovery of the Rosetta Stone. When this occurs, faculty should examine the content of the obsolete course and identify content, if any, that can be reused. If little can be retained, redevelopment of the course from scratch may be preferable to revising the obsolete course. Redevelopment affords opportunities to incorporate improved pedagogy, content, resources, and technology, so the redeveloped course may end up being superior to an incrementally improved course. For example, an updated course may not be able to take advantage of a new textbook with improved, hands-on pedagogy based on a virtual lab provided by the publisher. Such situations

124  Advanced Course Design

are rare; normally, gradual improvements produce a superior course to a completely redeveloped course, because updates can preserve proven content.

Building for Universal Design Universal design is the creation of an environment that can be used to the greatest extent by the most people, regardless of their abilities (National Disability Authority Centre for Excellence in Universal Design, 2019). It is an approach that treats accessibility as something that benefits all users, extending everyone’s abilities. For example, closed captioning on television programs was originally developed for viewers with hearing impairments, but it has proven invaluable for watching television across a noisy bar, at home with the sound off to avoid waking sleeping family members, or when it is difficult to understand characters’ speech. In a similar way, sidewalk curb cuts and ramps were originally designed for wheelchair access, but they are also useful for strollers and wheeled luggage. It requires less effort to include universal design as part of the initial development than it does to retrofit courses to make them accessible. For example, some programs add closed captions to all videos, so it is most efficient to make that step part of the standard video production process. If the production process includes transcribing videos for closed captioning, it is not much more effort to create a transcript for inclusion in courses. Unfortunately, it is difficult for institutions that rely on their faculty to build courses to also cover universal design issues.They are dependent on the individual faculty members having enough time, knowledge, and skill to implement universal design, and are thus less likely to have courses that consistently meet diverse needs. Fortunately, if they are building courses in a modern LMS, it will generally implement what they request, using universal design principles. For example, a section title is created in an LMS by typing it in a box; the LMS takes care of the underlying HTML structure so that it is formatted correctly and is also usable by reader software for the visually impaired. Many of the options for adding elements to an LMS act similarly, which helps ensure a basic level of universal design. However, LMSs have limitations. In a similar example, images or illustrations should include alt attributes that can be read by screen readers, and many LMSs and HTML-building software do automatically add alt attributes. Unfortunately, unless someone defines those attributes, the software is forced to make up the text, which may not be pedagogically useful; for example, the software may take the filename of the image as the alt attribute. Programs with specialists, such as designers, do not have to rely on their LMSs and thus avoid these sorts of problems. They can also have more sophisticated HTML objects, such as interactive animations, and be confident that all their elements comply with universal design. Best universal design practice is to build course elements so that they have the greatest number of potential uses (Wynants & Dennis, 2018). Students have more options for using material that is presented as text, compared to video or

Advanced Course Design  125

audio. Text and video transcripts should be in a form that can be read online, printed, saved as a file, or read aloud by the user’s computer. Ideally, videos should include closed captions as well as transcripts that can be searched, downloaded, and read aloud. This allows students to read the transcript rather than watch the video, have their computer read the text file to them, or listen to the transcript while exercising, walking, or driving. To make this easier, it is recommended to make modules available as concatenated pages, so users can print or download all the text at once. Technological limitations can create situations where anyone may have difficulty using a website. For example, if a website differentiates items solely by color with similar saturations, these colors may be indistinguishable to color-blind people or when printed in black and white.This can be remedied by ensuring that the design differentiates items in multiple ways, by using labels, positioning, variations in typeface, or patterns/textures. The design can still differentiate by color as well, as long as colors with different saturations and brightness are used.This also applies to the amount of contrast between text and its background color. Again, technology can be used to make such clear design easier. Once a clear and easy-to-read design has been created, covering all the elements listed above, it can be implemented as a template and used for all course pages. An instructional designer or faculty can accomplish this easily by creating a style in CSS, including such elements as typeface, space between lines, and font size. Standardization of universal design practices can strengthen the consistency of the look and feel of the institution’s pages. It also saves the faculty or designers who are developing course pages from having to decide how to implement universal design for each course. Practitioners of universal design also acknowledge that users of online courses may not always have access to a usable internet connection, such as when they are traveling, so course material should be designed so that it can be saved for offline study. We design our formative evaluations so that students can interact with them on the LMS, but they are still helpful when printed. For example, we create questions in plain HTML and JavaScript, so that when printed, they display the questions, with the answers lower on the same page, so students can still test themselves. Unfortunately, some multimedia creation tools do not support universal design. For example, they may produce multimedia objects, such as simulations or formative assessments, which are not printable or usable offline. And some subjects simply cannot be made accessible, such as a real-time flight simulator.

Jim Frey, learning design director Universal design is a broad field of expertise and you will want at least a working knowledge of it, if not more. Your institution may have its own Universal Design for Learning (UDL) guidelines that may cover a lot of this. Familiarize yourself with the parameters you are operating within and be creative and thoughtful while staying inside those boundaries.

126  Advanced Course Design

Fawn Thompson, lead instructional designer at Boston University’s Office of Distance Education Many people talk about universal design without actually knowing what it means or learning any of the principles. I recommend resources at the National Disability Authority’s Centre for Excellence in Universal Design site; last time I checked, these were organized well and easy to understand. In fact, they do a really good job of practicing what they preach.

Avoiding Multiple Copies of Content In the last chapter, we talked about the importance of locality of design in ensuring that content was not repeated. However, some content, as such as technical support pages, must be used in every course.When duplicated content is updated, all of the copies must be found and updated in the same way or it will introduce inconsistency.This can be avoided by having one master copy of shared content, and linking to that copy from all courses that need it. For example, at our university, the Office of Distance Education is responsible for maintaining hundreds of courses, and every one of them references the same technical support page. If every course had a copy of this page, and the page were to change, then that would require hundreds of courses to be updated. So, rather than maintaining individual copies of this page, all of our courses link to the same file. When the original is modified, every course displays the updates. We have an area in the LCMS dedicated to containing master pages used by programs and courses. There is nothing special about the area, but we make every file in it readable to the public, so we do not have to worry about read permissions. The area also has directories for pages that are used by every course at the university and directories for particular programs. Consider the typical syllabus. The first pages are unique to that course—the course description, goals and learning objectives, faculty information, due dates, grading weights, and grading rubrics. Then there are pages which are the same for every course in the department or program—policies and procedures, who is responsible for what, and important scheduling dates. Finally, there are pages that are used throughout the institution, such as disability policies and technical support. These pages change over time and it is important to ensure that all courses always use the latest version. If this information in courses is linked to master files, then all courses automatically use the latest version. For example, we include instructions in each course covering how to log into the synchronous sessions. Each course is linked to common text using a few lines of JavaScript code. When switching from using Adobe Connect to Zoom, we were able to change the instructions in all our courses by updating a single file.

Advanced Course Design  127

We use similar technologies to link within courses, to ensure consistency and save course update effort. For example, syllabi have weekly study guides that list due dates and times for assignments, discussions, assessments, and synchronous sessions. Study guides are listed for all of the modules together in the syllabus, so that students can easily get an overview of the entire course; each study guide is displayed in each module, as the first page.The version that is displayed in a module is linked to the original in the syllabus. Then, when a course is updated, only the study guide in the syllabus needs to be updated and this updates the module study guides. One advantage of this approach is that all of the assignments are displayed together in the study guide, which helps faculty assess student workload and adjust it, if needed. (For more on workload leveling, see Chapter 6.) We also use master copies for discussions across groups. In courses with many students, the students are organized into groups led by facilitators. This is usually accomplished by creating separate copies of each discussion question in each group.When questions are updated, it is labor intensive to update all of the copies and difficult to ensure that every group has the updated version.This problem can be solved by creating one master copy of the discussion questions, and linking that copy to the discussions in all of the groups.This is easily done with JavaScript code.When the master copy of a discussion question is modified, the changes are reflected in the discussion forums of all of the groups.This also makes it easier for faculty to update discussion questions to address topics of current interest.

Jim Frey, learning design director Give thought to where you are going to list specific due dates and be disciplined about not putting them anywhere else. Not only will this make your course easier to update, but from a design standpoint it will be very helpful to learners.They will see and understand quickly where to go to find due dates and will appreciate consistency throughout the course.

Deciding What Technologies to Use One of the enduring opportunities of online education stems from the continual evolution and improvement of technology and its associated pedagogy. Both administrators and faculty need to recognize that it is difficult to keep up with the technology, identify where its usage is appropriate, adapt pedagogy for new technology, and update courses accordingly. Online faculty often worry about and get fatigued from having to constantly learn new tools just to be able to teach. Designers, as well as faculty, often feel that they must be familiar with all new educational software, but that feeling is generally unwarranted. Though the software available for teaching online changes frequently, the basic operations performed by that software change slowly, so it is comparatively easy to

128  Advanced Course Design

transition from one software system to another. When one looks at software in terms of the tasks it is designed to accomplish, changes are almost always slow. The internet (and ARPANET before that) has supported threaded conversations since the late 1980s (Lueg & Fisher, 2012).Videoconferencing software that includes seeing and hearing each other with a camera, sharing application and desktop views, whiteboard areas, and a window for text chat on the side has been readily available since the 1990s (Finn, Sellen, & Wilbur, 1997). Major advances in the last 20 years have been improvements in bandwidth and scalability; the development of capable, web-based LMSs; and introduction of easy-to-use multimedia conferencing. None of these provide significant new functionality, but they do improve performance and usability. And, by reducing costs dramatically, these tools became available to nearly all schools, not just a few. Faculty and designers of online programs should consider if a new tool allows them to accomplish something that could not be done before. They may learn of a new tool and how it was used effectively in other courses and programs. They realize that if they learn enough about this tool to use it in a course, that they will learn more about teaching online and be perceived as leaders. This motivates the faculty and designers to learn new tools and experiment with using them for teaching. Ideally, they would evaluate how new tools and techniques improve learning and student outcomes (Dykman & Davis, 2008); unfortunately, they do not usually have the time or other resources to do this, so new tools may be adopted just because they are new, even if they do not improve outcomes. Another consideration is whether the advantages of this new tool justify the cost of the tool, training, and support for everyone who uses it. For example, in recent years mind-mapping software had a short heyday, being hailed as a group activity that would help students demonstrate their knowledge of how concepts are related. However, in our experience, while the software helped a few students, the effort required to learn the tools and understand the diagrams distracted from learning the actual subject matter. Different faculty and designers may have significantly different attitudes toward using new technology. Some faculty and designers are eager to try every new technology to determine if there are ways they can use them in their courses, and they are willing to accept the risks. Other faculty and designers are more conservative, and want to use only proven technology.

Jim Frey, learning design director You should make every effort to anticipate the types of experiences you want your learners to benefit from, and then make sure you have a plan for facilitating all of those. For example, if you wish to use the case method as part of your program, it will be helpful for you to mock up a prototype

Advanced Course Design  129

of how you plan to set that up online, as it typically contains interaction types that differ from “traditional” online learning. A designer who works for the LMS company can probably help brainstorm that or create a proof of concept.Then you can be more confident from the start that you know how you are going to address your needs. In some cases, you may determine that you need to go outside the LMS to obtain the features and functionality you want or need.

Fawn Thompson, lead instructional designer at Boston University’s Office of Distance Education I prefer to use software that has already been thoroughly tested and proven successful before unleashing it on my faculty. I usually wait until software has developed a decent reputation before I begin to experiment with it myself. I encourage faculty to experiment with tools that are supported by the university. I give them the research I’ve completed and (when possible/ relevant) connect them to others who have used these tools. If the tool is not supported by the university, I make sure that the interested instructors know that university support will be limited or unavailable so they can choose whether or not to proceed.

The choice of technologies should depend on the subject matter, learning activities, and level of the course, not what is newest and shiniest. The best way to select technology is to evaluate how well it supports the learning activities. For example, faculty teaching a language course online need to be able to hear and correct students’ pronunciation and enable students to easily converse with each other, so they need software that supports audio conferencing and possibly recording. Faculty teaching Kanji or Chinese will need the ability to watch students create characters with the correct stroke order, whereas this would be irrelevant when teaching French or Portuguese. Whenever possible, best practice is to use the simplest, most minimal technologies that will accomplish the tasks. Simpler tools are easier to learn and easier to use, and they tend to work on the greatest number of devices and platforms. Simple tools made with HTML, CSS, and JavaScript can do many things, including changing colors and revealing solutions when clicked, in ways that are supported by all modern browsers. Many free or inexpensive tools are available that perform such elementary tasks.We now use JavaScript and CSS to adapt the

130  Advanced Course Design

material to the mode of instruction; for example, we use them to hide references to online-only elements in face-to-face courses. When building a course, one should strive for it—and everything in it, including learning objects—to be usable on as many kinds of devices as possible, from desktop computers with large monitors to small portable devices such as phones. An additional advantage of minimal tools is that they are more likely to work on older equipment and in situations where students may be restricted by slow or expensive internet connections, such as in remote rural areas or while traveling. While designers are often seduced by the desire to use all the latest, “coolest” gadgets, many students cannot afford them. Learning objects that can run on old equipment and limited bandwidth help everybody, not just those few with the latest model. For example, we designed a brief orientation course for incoming students, to introduce them to Boston and the university, which made extensive use of video. Unfortunately, many students who took this course were in rural China, with limited internet bandwidth, so they were unable to download and see the videos. We modified the course to reduce dependency on video and made use of static images and audio, which solved the problem. Video can be effective when used to complement other kinds of instruction. Consider an online math course containing only text and illustrations. It can be enhanced by adding videos of the faculty working through the problems, as they do in face-to-face courses. This course enhancement does not require creating new problems and solutions; it simply offers a different way to see the problems being solved. When solutions are offered in both text and video formats, it provides students an opportunity to choose the format that best meets their needs. Before adopting a tool, be sure it will make the course better, and be aware of potential dangers. For example, if faculty want students to share ideas via text, there are many ways to do this, including blogs, threaded discussions, and wikis. Some faculty use social media for this. Social media applications provide different ways of sharing text, images, and videos. However, a problem with trying to integrate a specific social platform into a course is that such platforms may evolve rapidly or be supplanted by their competitors. What “everyone is using” this year may be passé or unavailable next year.Tastes and trends in social media change rapidly, and faculty are likely to find themselves using last year’s fashion, and may be viewed as clueless and patronizing—adults trying to talk like cool kids. Social media may help engage students, but faculty need to provide quality control over the social media content to ensure that what students are learning is correct and appropriate for the course. Faculty also need to assure the privacy and security of their students’ data; both these issues make the use of social media sites problematic. For example, if a professor uses a special hashtag for her course in Twitter, anything posted by anyone using that tag will appear in the feed for the course.

Advanced Course Design  131

When deciding which tools to use in a course, administrators or designers should assess whether the users of the new technology will need support, and, if so, determine how that support will be provided. We begin our evaluation by going over the new tool with the relevant faculty. If they deem it useful and can use it without trouble, we move on to testing it. One way to assess whether technology will need support is to have some students try it out, and monitor how much support they need; this can also identify documentation or embedded help needs. If the institution provides technical support, designers or faculty should be sure that it is willing and able to support the new software or technology integrated into a course. If not, they must be prepared to support it themselves. If students do not have access to technical support, they will ask faculty to solve all of their technical problems; this will take a lot of faculty time, and will annoy and distract them from teaching. For example, when students in database courses were required to install Oracle or another industrial-strength database, they asked many questions of their faculty, taking many hours of their time. We solved this problem by writing detailed installation guides and providing both students and faculty with facilitators who are skilled in the tools.

Elena Garofoli, lead instructional designer at Boston University’s Office of Distance Education If faculty want to try a different tool, I ask the following questions: • • • • • • • •

Why do they want to use it? What role would the tool serve in the course? Is it accessible? If not, don’t use it. Will the tool require students to use a separate login in addition to the LMS? If yes, tread carefully or not at all. Is the tool likely to be around while the course is being run? Are there other tools that are better known and more reliable? What is the application’s competition? Is the tool supported by the institution or organization? If there is a cost associated with it, who pays for it? Does it have seamless integration with the LMS?

Program faculty, administrators, and staff should identify a standard set of tools and provide support and training for them (Aust et al., 2015). Tools in the standard toolkit should be useful in most or all courses in the program, while individual courses may make good use of tools that are not in the standard toolkit.

132  Advanced Course Design

For example, a tool that displays equations may be useful throughout a math program, while a tool that supports interactive graphing of functions may only be useful in some of the courses. When faculty or staff adopt a new tool for a course and find that it works well, it may be appropriate to share that experience throughout the program, or even more broadly. New tools can be evangelized in presentations, such as webinars, in which faculty or staff demonstrate and answer questions about them. Best practice is for faculty and staff to gain hands-on experience with the tools during the webinar, by doing small assignments. Afterwards, faculty should be provided with an opportunity to use the tools, to gain a deeper understanding and perform experiments that help them understand how the tools will work in their courses. This continuing access can be conveniently implemented as a sandbox training course that includes many tools. One of the great changes in distance education in the last few decades is that everything has become digital and many tools are based on web-based software. The advantage of this confluence of media to a common digital format is that they are all based on the same core technologies, such as HTML and JavaScript. Faculty or designers who understand these technologies have the option of creating or modifying the files that make the website run. Programs may choose to train faculty in the core web technologies and provide them with tools such as HTML editors. These HTML editors present faculty with the rendered output of the HTML and allow them to edit it, automatically changing the underlying HTML, without having to learn it. This is another example of the usefulness of designers, as it can be unreasonable to expect faculty to keep up with all these technological changes in addition to their regular responsibilities. Similarly, one way to create multimedia objects is by using software that automates many of the detailed operations, such as drag-and-drop, in-video quizzing, and animations. While this software can expedite the production and maintenance of multimedia objects, programs that are considering adopting these tools should assess whether they are cost effective and whether there are restrictions on whether the outputs of the products will run on all devices. The cost of this software and the training time needed to become proficient may create a production bottleneck. Best practice is to create objects directly in HTML and JavaScript, or other technologies that are supported by most modern browsers. Students should be able to take courses on portable devices, including cellphones, and listen to recordings while doing other things. Course design that complies with the latest web standards is compatible with many tools, including those for students with disabilities. (For more on disabilities and universal design, see the section earlier in this chapter.) Another task that designers can do for faculty is to track the evolution of internet standards and practices, so that design content does not have to be redeveloped when emerging changes become mandatory. For example, a few years ago it became apparent that Flash was not going to be supported on most

Advanced Course Design  133

battery-powered devices due to its high power consumption, so we stopped building new objects in Flash or adopting new tools that required it. We also began replacing any Flash-dependent objects in our courses with HTML and JavaScript that did not require Flash. Flash is no longer being supported in mobile environments, but none of our courses were affected.

Fawn Thompson, lead instructional designer at Boston University’s Office of Distance Education The most important factors when choosing technologies are ease of use and highest reward for all parties (not just students) on as many systems as possible. Imagine that you will get hit by a bus tomorrow and someone will have to take over your courses: What will be easiest for that person to learn and continue to implement?

Synchronous Sessions Conducting synchronous sessions requires many skills, such as how to respond to student questions, when to use slides and other visual aids, and how to handle audio feedback and other technical issues. It takes many sessions to develop and refine those skills. We have found that it is helpful to have faculty mentors conduct synchronous training sessions for facilitators and faculty new to online teaching; these sessions can serve to teach the operation of the conferencing software and how to conduct sessions, while allowing the faculty and facilitators to ask questions. Conferencing software can be complex and its operation may not be obvious. Many things can go wrong, so it is best to provide support while courses are running, which can be provided either by the vendors of the conferencing software, or in house. Programs with support staff should consider having them provide faculty and facilitator training and support. Our experience has been that this support is most effective when the support staff are enrolled in the virtual rooms. Each support person can be in many synchronous sessions, so this need not be expensive. When faculty and facilitators have successfully conducted a few sessions, they may no longer need this level of support, but it is desirable to have a staff member available to answer texts or phone calls if something happens that the faculty or facilitator are unable to handle. For synchronous teaching to be successful, the technology needs to be of sufficient quality. One of the key features is the ability of the conferencing software to support students with lower bandwidth connections and network failures. The best conferencing tools will support hundreds of students with

134  Advanced Course Design

microphones and webcams, even though some of them have poor connections. This is technically difficult for vendors, but is important for students. Administrators should evaluate conferencing software in realistic tests with the largest number of students expected, with the testers using webcams and headsets. Administrators should not assume that weaker conferencing software will be good enough because none of the students will be speaking. Effective synchronous teaching requires students be able to talk and share their webcams or computer screens. When there are many students in a synchronous session, acoustic feedback can be a serious problem, and the only way to eliminate this is if everyone has a headset. In our experience, each course should have one virtual room that is accessible to all students, facilitators, faculty, and support staff, for class-wide discussions. Facilitators should also have a virtual meeting room that is accessible to them, the students in their group, faculty, and support staff for small group discussions and facilitator office hours. If virtual rooms are used for term projects or other presentations open to all students, then they should be configured so that all students have access. Virtual rooms should be configured and tested before courses launch. Synchronous sessions are most efficient when they are less than an hour long, so students are alert and can ask questions (Martin & Parker, 2014). Synchronous interactions where students can speak and ask questions work well in courses with low to moderate enrollments, but they do not work as well in courses with high enrollments. High-enrollment face-to-face classes have long employed teaching assistants to teach smaller sessions, where students can discuss the material and ask questions. A similar technique can be used online, where facilitators conduct synchronous discussion sessions with their students. Sessions led by facilitators should supplement, not substitute for, sessions conducted by faculty. Administrators should be aware that even when facilitators are being used, faculty may still need to conduct many synchronous sessions, which may require significant preparation time. For example, open office hours allow students to ask questions related to anything in the course. Faculty should schedule them at least once per week. In high-enrollment courses many students may join the meeting, so facilitators may also need to conduct office hours to allow everyone a chance to ask questions. Open office hours should be scheduled at a time when most students can attend. The length of office hours should be about an hour, with the option to extend if many students come or some students wish to discuss something privately. These sessions are not normally recorded, to reinforce their informality. Best practice is to ask students early in the term when they would like these sessions scheduled. Topical synchronous sessions are used for introducing new material and reviewing material in response to unanticipated student needs or interests. Topical sessions may be scheduled after the course has started if:

Advanced Course Design  135

• • • • •

Students are struggling with a topic or want to discuss it in greater detail. Students ask to learn material that is not in the lectures. Important material emerges during the course. Faculty want to include material that has not yet been implemented in the lectures. Faculty want to strengthen their rapport with students.

Faculty often use PowerPoint slides or other prepared material, as they would in a classroom. If the faculty do not have appropriate slides, they need to prepare them.Topical sessions should be recorded for students to review and for students who cannot attend at the scheduled time. Topical sessions that are similar to face-to-face lectures can be a way to help faculty transition to teaching online, though they are best supplemented with substantial asynchronous content. Recordings of topical synchronous sessions can sometimes be used as a starting point for developing new asynchronous course content. If a face-to-face course needs to be brought online with insufficient time for development, some of the content can be taught in synchronous sessions. Topical synchronous sessions should be announced to students several days in advance. For example, in a social work course in which synchronous role-play sessions are essential, course sections may be scheduled at different times, to allow students to enroll in the most convenient section. Student presentation sessions are used for term projects and other forms of student presentations; in higher-enrollment courses they may be conducted by facilitators. Best practice is for faculty to attend as many of these sessions as possible, and for faculty to encourage students to attend. Faculty may wish to award some extra credit for students who attend and ask good questions.The length of these sessions depends on the material to be presented and the available time. These sessions should usually be recorded, unless the material being discussed is too sensitive, such as personal issues or proprietary business matters. Review sessions online are similar to review sessions in face-to-face courses. These are commonly scheduled on a regular basis throughout the term, and provide students opportunities to ask questions.There are several kinds of review sessions: • • • •

Sessions near the end of modules that cover their material. Assessment or assignment reviews; in advanced courses these may be conducted by faculty and students. Course milestone reviews, such as steps in a long-term project. Final exam review, which covers the entire course.

Review sessions should usually last less than an hour, except for the final exam review, and they should be recorded. Faculty may need substantial time to prepare for review sessions, particularly if they are creating slides. Preparation for

136  Advanced Course Design

the final exam review involves checking the final exam to make sure that the review is aligned with it.

Telepresence Synchronous conferencing technology can also enable students and faculty to participate in face-to-face courses when they are not physically in the classroom. This is termed telepresence, or remote learning, and it is particularly helpful when a student is ill or traveling, for remote guest lecturers, or even when the entire institution is working from home during a global pandemic. Different levels of realism are supported by different technologies, ranging from simple telephone audio to virtual reality. At Boston University we use telepresence for remote guest lecturers, and often include edited recordings in later iterations of a course. We have been using telepresence for decades with Adobe Connect, Blackboard Collaborate, Zoom, and other tools. Several of our classrooms have been modified to better support telepresence, with microphone arrays in the ceiling and cameras on the walls. This provides superior audio and video quality, but it is not essential for effective telepresence in smaller classrooms. Students in the classroom quickly become accustomed to having some of their classmates virtually present. We have found that the novelty of telepresent students increases the engagement and attention of students in the classroom. The novelty distracts the students for a few minutes, but they quickly adapt and the experience is generally pleasant for all. Faculty may initially find it challenging to attend concurrently to students present in the classroom and students who are telepresent, but they gradually learn how to divide their attention and encourage telepresent students to use the teleconferencing software to let faculty know when they have a question or comment. When faculty teach students who are physically present in the classroom, they can usually tell which students have questions or want to speak. When some students are remote, this is more difficult, and different techniques are needed for different numbers of remote students. If there are a dozen students in a classroom and only a few remote students, telepresence does not require fancy hardware. Students can be seated in a semicircle, with a microphone/speaker in the center.When there are a half-dozen or more remote students, they may inadvertently speak over other students, so the best approach is to have them type into a chat window, and to have this displayed at the back of the classroom, behind the classroom students, where faculty can easily monitor it. When there are too many remote students for faculty to easily monitor their chat comments, a facilitator is helpful, to answer some of the remote questions and identify the most pedagogically valuable comments or questions for the faculty to address for the whole class. Scaling telepresence to higher enrollments can be challenging. We have used inexpensive technology in ordinary classrooms, such as USB microphones and

Advanced Course Design  137

speakers that plug into computers. This works well in a quiet classroom when students’ seats can be arranged in a circle with a microphone in the center. In large classes, there is substantial benefit to having more than one microphone, so that students can be clearly heard by telepresent students. If remote students use a headset, everyone can hear them clearly; however, they may have difficulty hearing the students in the classroom unless there are a sufficient number of microphones. Telepresent and face-to-face students all share one audio communication channel, and this can create a bottleneck when there are many of them. When there are a large number of telepresent students, we use a facilitator, who is present in the classroom, to coordinate remote communications. The facilitator mutes and unmutes remote student microphones to suppress noise, and notifies faculty when a student has a question or has raised their virtual hand. Audio quality is important for good telepresence communication. If the instructor wears a headset or lavalier microphone it helps telepresent students understand what the instructor is saying, in spite of room noise. It also improves the quality of classroom recordings. Headsets should be worn by telepresent students to avoid acoustic feedback. Online technology can also greatly improve the robustness of face-to-face courses. Courses that provide conferencing tools work very well to support remote students in face-to-face classes, as well as students who are ill, traveling, or snowed in. Recordings of synchronous sessions are helpful for students who are behind in their studies, because they can watch them repeatedly. Conferencing tools allow remote students to take on-campus courses that are not offered online, and may also enable any student to take on-campus courses when the course scheduling works better for them. They do not require the full development of a sophisticated course website, but only modest adaptations of a traditional classroom.

Integrating Multimedia Content, Simulators, and Virtual Labs Designers can help faculty integrate digital multimedia into their courses.This can be very effective, as some kinds of material are best taught through interactive learning objects. An interactive diagram is often the best way to teach complex, interdependent material, such as system architecture. These objects make it possible for students to interact with details of components without losing the context of the relationship between the components. They also allow students to indicate their cognitive focus by mousing or hovering over, or clicking on, an object, to which the system responds by providing detail on that object, and, potentially, posing interactive questions. For example, students can be studying a complex systems diagram, and by clicking on an object in that diagram, learn about its details and relationships to other objects in the system. It is not feasible to display all of this information at once, but it can be done through multimedia.

138  Advanced Course Design

Interactive objects can also include embedded software. For example, we offer accounting courses that have embedded spreadsheets for students to solve problems, and data science courses that have embedded the R programming system. Integrating software in course websites enables students to use the same tools in the course that they would use at work, without needing to download and potentially obtain licenses for that software. When software has been embedded in course websites it can also be pre-populated with data and algorithms, so students do not waste time downloading code and data to work on their course material.This is especially useful for courses in which learning how to install the software is not a learning objective. Some courses use simulations, which permit students to input different values to observe how a simulated system responds, or to test and derive theories. Simulations in which users can select hurricanes of different strengths and buildings of different types can be very effective in teaching hurricane-resistant building design. Interactive multimedia may also allow students to modify parameters, or pose questions and accept responses. For a more advanced example, the role of sway in the collapse of buildings in earthquakes can be taught using a combination of videos of buildings in earthquakes, while interactive simulations let students make engineering changes to suppress sway. In another example, students may get a good understanding of how infectious diseases spread and how different vaccination strategies change the results with an animated, interactive simulation. Virtual labs are course assignments that are implemented using technologies that students could not generally implement themselves. We use virtual labs in our database courses, hosting the virtual labs at the university, because they include large databases that would not fit on a typical student's computer. Courses on computer security can use these virtual labs to enable students to attack and defend realistic virtual computer systems.Virtual labs let students use penetration tools in the safety of a constrained environment, when those tools could be used to attack real systems.Virtual labs also allow many students to do the same assignment concurrently without multiple physical lab setups, and they can also be hosted on cloud platforms, which reduces the platform cost. Cloud platforms that have excellent support for virtualization also reduce development effort. Interactive objects, simulators, and virtual labs all permit students to experiment with the subject matter, while the computer system in the multimedia or virtual labs provides feedback. Simulators can provide a surrogate experience when the actual experience would be expensive, dangerous, unethical, or incompatible with a home or classroom setting. The value of virtual simulations has long been known in the training of pilots, as pilots practice handling dangerous situations in flight simulators. A virtual chemistry class makes it possible for students to perform experiments that would require specialized safety equipment, such as fume hoods. Similarly, medical students can learn surgical procedures by using simulations, without endangering human subjects.

Advanced Course Design  139

Fawn Thompson, lead instructional designer at Boston University’s Office of Distance Education When designing an interactive object, you need to become a storyteller. Whether you prefer to begin on paper or in a digital format, storyboard each segment and describe the affordances [what is possible]. Consider the progression of your story. Will it be linear or nonlinear? What elements do you hope will be surprising or impactful for the participants? Will participants receive feedback on their interactions, and if so, when and how?

Managing Production Tasks It requires some effort to ensure that courses are consistent and of high quality when a program has more than one person involved in design and development. Ensuring uniformity in courses across a program requires standards, and processes that assure compliance with those standards. It is easier to obtain uniformity if developers use template courses and development task checklists. When the design elements include information that has default values, such as having assignments due Monday mornings at 6:00 a.m. Boston time, these should also be included. Not only does this ensure that all courses contain the elements, but it also saves the effort of adding them manually for each course. For example, one element that should be included in all courses informs students when faculty will conduct synchronous office hours. To ensure that this information will be both included and correct in all new courses, include it in the template course and flag it as default text that needs to be approved or revised. A task checklist ensures that designers and production staff perform the same tasks for both initial development and revision. A task checklist can also be used to monitor production to determine if it is on schedule. This, in turn, can be used to determine if resources need to be reallocated or the course rescheduled. Example tasks in the checklist may include copying courses in the LMS for each time the course runs in a term, enrolling faculty and facilitators, and changing dates in the syllabi. (For a more complete list, see the section on designer tasks below.) Checklist tasks can be standardized and coordinated in a spreadsheet or database. However, when there are more than a few designers working together on many courses, task-managing software that tracks tasks by term, course, and person can be invaluable. Such software differs from a ticketing system, because rather than being designed for one-off items that need to be

140  Advanced Course Design

fixed, a task manager allows one to set up a standard template of tasks and subtasks that can be repeated for each course. Task-management software helps coordinate development and support tasks. We began using a task-manager program nearly a decade ago to standardize our development process, and looked at more than 130 available options. The process of deciding which items to include in the task manager was instructive, as it forced us to examine all of the ways that our designers and support staff had been accomplishing tasks, and to settle on the best way to do each one. The software also helps ensure that tasks are handed off properly. For example, one task may have designers set up groups for a course in the LMS, and another task may have support staff assign students to those groups. A task manager ensures that designers and support staff receive their respective tasks, and can also notify support staff when the groups have been created by designers. Each task contains a label and instructions. For example, a task with the label “Assessments are correct” may have the description, “Each quiz or exam should have instructions with start and due dates that agree with those in the Study Guide.” For more detailed explanations of tasks, with illustrations or videos, we include in the description a link to our wiki. This saves training time for new staff and also helps document, coordinate, and preserve institutional memory. We discovered that although one can work with a standard set of tasks for courses in a program, there are many tasks that are unique to specific courses or faculty. In addition to tasks which must be performed for every course, there may be tasks at the college, program, course, faculty, or term level: • • • • •

College-based tasks: Every new student in one college needs to be sent a welcome letter from the dean, whereas another college’s dean might not send one. Program-based tasks: “Add Andrew Jones to all Arts Administration courses because he is the faculty coordinator, and give him the role of Advanced Instructor.” Course-based tasks: A math course has solution videos which need to be made unavailable until the course starts. Faculty-based tasks: Instead of “Live Classroom,” Professor Smith prefers her synchronous sessions to be labeled “Coffee and Conversation with Professor Smith.” Term-based tasks: If a course is scheduled to start on the first Tuesday in July, and that day happens to be Independence Day, then the course will start a day later.

Task-managing software should also have a way to hide or remove irrelevant tasks. For example, programs that have proctored final exams will have tasks that are not relevant for courses that do not have them.

Advanced Course Design  141

Fawn Thompson, lead instructional designer at Boston University’s Office of Distance Education I use checklists and spreadsheets (for detailed production tasks) as well as calendar reminders (for timelines and delivery). We also use shared course folders for notes that may be helpful for delivery in future, and to archive development materials. This is very important in case of later conflicts over development roles, content ownership, and editorial decisions. A faculty member once accused a designer on my team of removing content without her permission. We were able to find the document where the faculty member had requested that we remove the content in question in an earlier iteration of the course, so the issue was resolved without harming the designer’s reputation.

Copy-editing When the course content has been written, best practice is for it to be reviewed by copy editors before it is implemented. Copy-editing primarily addresses grammar, writing style, and clarity issues. Out of respect for faculty’s academic freedom, it does not address the correctness of the content, because the editors are not content experts. This editorial review can be performed by facilitators, advanced students, or professional editors. Programs that have no need for fulltime editors can hire them as contractors or freelancers. When copy editors are available, it is preferable to do copy-editing before faculty QA, because faculty may be distracted by typos and other non-content errors. If the content is developed in the order that students are expected to study it, then copy-editing can be performed incrementally as the content is developed, to reduce the editorial delay. One approach is for designers to produce an extract of the course in a format such as Microsoft Word, and then send it to an editor.The editor reviews the document and edits it, resulting in a document that identifies the editor’s comments, queries, and suggested changes. If an institution has both designers and editors, the document produced by editors can be sent first to faculty, who update it to address the issues identified by the editor, and then to designers, who make the changes. It is important to allow enough time in the course development schedule for this editing work, as it is another element that faculty who are accustomed to working on their own may not expect. If a program does not have designers, the faculty, or a student worker, will have to input the changes. If a program does not have editors, best practice is to have another faculty, graduate assistant, or an expert in the subject matter review the course content.

142  Advanced Course Design

Pre-launch Designer Tasks The following are tasks that need to be performed for each course before it runs. If a program does not have designers or staff performing designer tasks, faculty will need to perform them. All of these tasks can be performed in parallel or in any other order; we complete them about three weeks before the course start date, to allow time for support QA. Tasks for project management It is very important to track the designer level of effort per course, to address workloads across designers and prevent bottlenecks. This level of effort, measured in hours, can be stored in the task-management system. It is estimated before development starts and updated as necessary. Even a rough estimate helps to identify when a designer or team may be overwhelmed or idle. Tasks in the LMS to prepare the course for running If your program has support staff, enroll them with a Support role. This role allows staff to make changes to course material as requested by faculty, as they may be the first to be told about needed edits when a course is running. A more limited Editor or Reviewer role might be useful for editors who should have read access to the course before it runs, but should not have access to the grade book. Tasks that require faculty input/approval • • •

Delete old announcements. Some faculty create announcements in advance to be released automatically at scheduled times, so be careful not to delete them when removing the old announcements. Revise due dates in the course syllabus. If a course has run before, faculty will probably keep the same schedule of deliverables, e.g., quizzes on Sundays, but this should be confirmed. Set up due dates in the LMS.

Tasks related to designer QA • • • • • •

Test the function of extensions to the LMS implemented in JavaScript and other programming languages. For example, if JavaScript is used in a course to create PDF extracts of lectures, it should be tested. Verify that videos are captioned properly and that transcripts are attached. Verify that links in the course material are working and directing to correct sites. Notify support staff that the course is ready for support QA. Make changes based on support staff QA feedback. If editors (or staff serving as editors) are available, send copy-editing requests and revision summaries to the editorial team, to guide the editors by communicating what has changed in the course.

Advanced Course Design  143

Performing Faculty QA Faculty QA is performed by course faculty or others who have expertise in the subject matter. While this is time-consuming for faculty, it is important for successful content revision (Kidney, Cummings, & Boehm, 2007). Faculty review the course for technical correctness, whether it is up to date, and whether it dovetails well with other courses in the curriculum. For example, faculty QA of a prerequisite course should assess how well that course prepares students for subsequent courses. Faculty QA also covers the assessments in the course. Faculty QA requires access to the course website, to review interactive multimedia objects, videos, and other media content that may not be viewable in the extract. If faculty are not working with designers, or do not have access to student workers, they will need to implement the changes themselves. When the faculty QA phase is complete, the course is ready for the final phase, which is preparing it for launch.

Final Preparation for Launch The last phase of course development occurs in the final weeks before launch, when facilitators are enrolled in a course, when a course is customized for its term, and when it receives final QA. Faculty may request that specific areas of the course be checked and updated, typically because they are new or have been problematic. This may also be performed by support staff, if available; this typically includes checking external links and that all the expected course components are present. This final QA may be performed by program facilitators.

Summary Checklist ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑

Hire and develop designers and other staff. Plan course development. Hold a kickoff meeting with the development team. Motivate faculty to create and update content. Make use of universal design principles. Avoid maintaining multiple copies of content. Decide what technologies to use. Decide how to use synchronous sessions. Decide whether and how to use telepresence. Identify multimedia content and virtual labs. Manage production tasks. Copy-edit the content. Preform pre-launch tasks. Have faculty QA the course. Customize the course for the term and perform final QA.

144  Advanced Course Design

References Aust, P., Thomas, G., Powell, T., Randall, C. K., Slinger-Friedman, V., Terantino, J. et al. (2015). Advanced faculty professional development for online course building: an action research project. Internet Learning, 4(2), 103–118. Dykman, C. A., & Davis, C. K. (2008). Online education forum: part two – teaching online versus teaching conventionally. Journal of Information Systems Education, 19(2), 157–164. Finn, K. E., Sellen, A. J., & Wilbur, S. B. (1997). Video-mediated communication. Mahwah, NJ: L. Erlbaum Associates Inc. Kidney, G., Cummings, L., & Boehm, A. (2007). Toward a quality assurance approach to e-learning courses. International Journal on E-learning, 6(1), 17–30. Lueg, C., & Fisher, D. (2012). From Usenet to CoWebs: interacting with social information spaces. London: Springer Science & Business Media. Martin, F., & Parker, M. A. (2014). Use of synchronous virtual classrooms: why, who, and how? MERLOT Journal of Online Learning and Teaching, 10(2), 192–210. National Disabilty Authority Centre for Excellence in Universal Design, What is Universal Design. Retrieved November 1, 2019, from http://universaldesign.ie/ What-is-Universal-Design/. Neely, P. W., & Tucker, J. P. (2010). Unbundling faculty roles in online distance education programs. International Review of Research in Open and Distance Learning, 11(2), 20–32. Wynants, S., & Dennis, J. (2018). Professional development in an online context: opportunities and challenges from the voices of college faculty. Journal of Educators Online, 15(1), 1–13.

Chapter 8

When a Course Is Underway

This chapter is written primarily for faculty and support staff. Administrators may help to standardize policies and procedures. Designers may be involved in the creation and updating of learning objects to address issues that were not anticipated when the course was designed. This chapter covers: • • • • • • • • • • • • • • • • • • • • • •

Guiding students’ course preparation. Support staff responsibilities. QA tasks for support staff. Pre-launch tasks for support staff. Course launch kickoff meeting. Weekly faculty, facilitator, and support staff meetings. Setting up student groups. Orientation course. Welcome messages. Administering email and announcements. Tasks performed by support staff while a course is running. Contacting students who have not logged in. Building a learning community. Regularly scheduled synchronous sessions. Outreach summarizing prior and coming weeks. Study groups. Calling students. Students at risk. Wrapping up. Administering proctored exams. Winding down the course. Building community beyond the classroom.

This chapter describes activities that occur in preparation for the launch of an online course, while a course is running, and shortly after it ends. The material

146  When a Course Is Underway

is organized in sections corresponding to the phases of course execution: prelaunch, the first week, the middle weeks, the last week, and wrapping up.

Guiding Students’ Course Preparation There are many reasons for faculty to contact students before a course officially begins. The primary reason is to give students a head start on readings, assignments, technology orientations, and, if necessary, installing software used in the course. Another important reason is to begin establishing rapport between faculty and students. Establishing this sense of connection is more challenging in online education, because faculty cannot communicate their care using body language such as smiles and nods, but it is especially important online, because it reduces the emotional distance between faculty and students. (For more on student engagement, see Chapter 6.) We have tried various approaches to offer students extra preparation for some courses, such as providing students access to the entire course a week before it opened. This worked well for some students, but it required that faculty and support staff be in the course a week longer, increasing their workload. It also reduced the time available to prepare the course for launch, which interfered with efforts to improve it. We have also had faculty conduct synchronous meetings with the enrolled students before launch, but we learned that this requires substantial additional effort with only limited return. In both of these cases, few students took advantage of these extra learning opportunities.

Susan Diesel, assistant director of student and faculty support at Boston University’s Office of Distance Education There are two student groups that faculty may try to contact before classes start: Students new to a program who have not yet taken a class, and students who have taken at least one class. Students new to a program are invited to attend an online webinar, before the class begins. They are introduced to the program requirements and important contacts, and are given a chance to ask questions. Programs with a small schedule of classes each term invite all students, new and returning, to attend the webinar. All students are sent a draft of the course syllabus by faculty or the support staff before classes begin. Faculty may also reach out to students by email with an introductory message to welcome them and let them know what the course expectations are.

When a Course Is Underway  147

Support Staff Responsibilities When a course is running, helping students with issues that are not related to the subject matter distracts faculty from their primary responsibility of teaching. For example, if a student asks the faculty how to use the online library, this can take a lot of time, but it doesn’t require subject matter knowledge to answer; having someone else provide this support will save faculty time, with no loss to the student. Some student issues are indeed related to the content, but more often they are course-specific technical problems that the program’s faculty is not prepared to answer. For example, a student may have difficulty uploading an assignment. Occasionally, faculty benefit from support when their courses are running, such as configuring a student’s quiz score so that it does not count in the quiz average. In most places, designers provide technical support to faculty throughout course development and while the course is running. Institutions that have separate support staff use them to provide technical support to students when courses are running. Unfortunately, it is relatively rare for institutions to have such dedicated support staff, so students turn to the general technical support of the institution, or to their faculty. What is needed is to have people enrolled in the course who are familiar with the issues that both students and faculty encounter. Support functions can be allocated in many different ways. Best practice is to have support staff in all online courses. If a program has support staff, faculty coordinators can help to coordinate and manage expectations between them and faculty. If a program does not have the resources, support staff functions can be performed by IT staff, facilitators, or designers. If enrollments and faculty workload are low enough, faculty often perform these functions. In some programs, the support staff roles and technical support roles may be combined. Support roles may also be filled by designers, but the frequent interruptions of these roles may make it difficult for them to deal with complex design problems. We believe that the benefits of support staff outweigh their cost. Each staff member can support multiple courses in the same term. Best practice is for support staff to work with designers to set up courses, including enrolling students, assigning them to groups, and performing quality assurance checks to be sure that courses are configured properly and are ready for launch. When necessary, support staff may assist faculty with the LMS and other educational technology. While the course is running, they help identify students who are falling behind. In high-touch programs, support staff provide a first line of contact for students. Early in a course, all new students in a program may be contacted by their support person by phone to personally ensure that everything is going well for them; this helps build that all-important sense of connection.The support person may also send email announcements to prepare students for milestones such as course launch, final exams, and course evaluations. Support staff often refer students to people throughout the institution, including advisors, financial aid, libraries, and department administrators. Support staff may also monitor class discussions to identify student concerns that may not have been brought to the faculty’s attention, so that they can be

148  When a Course Is Underway

addressed quickly. (See later sections in this chapter for more details on support staff tasks.) When resources are not available for dedicated support staff, institutions may be able to find creative ways to provide their functions, perhaps with teaching assistants or student workers. Faculty at smaller institutions, or ones with fewer resources, may be expected to learn the relevant technologies on their own.

QA Tasks for Support Staff In Chapter 6, we discussed QA tasks that must be performed by faculty before a course runs. If designers and support staff are available, these tasks are performed by designers and checked by support staff. All of these tasks can be done in parallel or in any other order; we complete them between 24 and 12 days before the course start date. (For tasks that are specific to the design and construction of courses, see Chapters 6 and 7.) Here at Boston University, the following items are checked by support staff before each course launches: •

Each quiz or exam in the LMS includes instructions and the correct start and due dates for the term. They agree with the ones in the syllabus. • Assignments and assessments in the LMS all have instructions, deadlines, and correct settings.Verify that files associated with the assignments can be downloaded. As with the quizzes and exams, ensure that due dates agree with those in the syllabus. • Any other due dates in the LMS calendar are updated to reflect the current course iteration. • Discussion topics are consistent across all groups. • The technical support page appears in the appropriate place. • The course is configured so that students cannot access it before the official start date and time. • The course start and end dates are correct in the LMS. • The course title and semester header are correct in the LMS. • The LMS grade-book column settings for graded items are correct. Columns are hidden or visible to students according to faculty preferences. If the grade book contains a column for facilitator names, those names populate that column. • The Help link exists in the navigation area and the correct documents are linked, such as tutorials and links to the LMS orientation. • All learning objects, such as inline formative evaluations, display and execute correctly. • Synchronous session instructions are embedded and correct. • External and internal message tools work. • The Navigation panel has the complete list of course tools and links: Home Page,Announcements,Assessments,Assignments, Calendar, Class Discussion, External Email, Internal Messages, Live Classrooms/Offices, Live Classroom Recordings, My Grades, Printable Lectures, and Help.

When a Course Is Underway  149

• The syllabus contains the following pages: Course Description, Faculty, Study Guide, Course Resources, Grading Information, Roles and Responsibilities, Academic Conduct Policy, Disability Services, Netiquette, Registration Information and Important Dates, and Technical Support. • Videos work. Downloads operate correctly and open in separate windows. Transcripts of all videos are contained in the course. Closed-caption button works and captions are in place. • Page concatenation works for printing every page of a module and for creating extracts for offline study. • Alt attributes exist and are useful.

Pre-launch Tasks for Support Staff In addition to QA work, support staff perform a number of tasks to prepare courses for launch. QA tasks are related to preparing the course content, while pre-launch tasks are related to enrolling and preparing students and faculty for the course. In programs with fewer resources, these pre-launch tasks may be done entirely by faculty or designers. Again, many of these tasks can be done in parallel or in any other order; we complete them in the three to four weeks before the course starts. The following tasks are performed by support staff before each course launches: • • • • • • •

• • • •

Sending students email with book information for their course four weeks prior to the course’s launch. If the bookstore website does not contain the book information for the course, include the ISBNs. Training faculty and facilitators to use the LMS, as needed. Enrolling new instructors in the LMS orientation course. Sending faculty all appropriate guides, offering synchronous session training. Enrolling students; this may be performed automatically by the institution if the LMS is integrated with the student information system. Emailing students the syllabus or a link to it one week before launch. Compiling a list of new students to be sent welcome packets. Welcome packets contain welcome letters from the program, student ID card, and branded “swag” such as a travel mug with the institution logo or a T-shirt. Sending new students a reminder to attend the LMS orientation course. Sending students email to remind them of the course URL and recommended browsers, and about when the course opens, ordering textbooks, paying tuition, financial aid, and important course dates. This email also provides contact information for the campus bookstore and student accounting services, and the phone number of the support person sending the reminder. Setting up group discussions. Enrolling students randomly into groups. Enrolling faculty and support staff in all groups. Enrolling facilitators into their groups.

150  When a Course Is Underway

• •

Creating or updating synchronous session accounts for faculty and facilitators. Attending the faculty and facilitator kickoff meeting (see kickoff meeting section below).

Course Launch Kickoff Meeting A week or two before launch, faculty should host an online kickoff meeting for facilitators, support staff, and others involved in the update or execution of the course. Best practice is to create a synchronous, virtual meeting room in the course, known as a faculty forum; it should be accessible only by faculty, facilitators, and support staff. Faculty will use this forum while the course is running to coordinate their activities in weekly meetings with facilitators. This restricted, synchronous forum can provide an effective place for faculty to discuss course-related issues in the weekly calls. In addition, a related, restricted asynchronous discussion forum should also be created, to provide a convenient place to post course-related resources, such as assignment solutions and exemplary submissions. In the kickoff meeting faculty communicate their expectations and describe planned course elements, such as additional synchronous sessions. The faculty also describe updates made for the current running of the course, any known issues, and any updates planned while a course is running. They establish when synchronous sessions will be held, what facilitators are expected to do, grading expectations, and how late submissions and student misconduct are to be handled. If there are new technologies or design features, faculty teach facilitators how to use them in the course. This meeting typically lasts less than an hour. Best practice is for the faculty to prepare and post slides for this meeting, so that facilitators have a record of what is expected. When facilitators have become comfortable with a course that has not changed significantly, kickoff meetings may not be needed; any issues can be dealt with in the weekly calls.

Alyssa Kariofyllis, lead online engagement manager at Boston University’s Office of Distance Education Faculty hold kickoff calls before the semester begins to discuss materials that will be covered during the semester, provide an overview of assignments and assessments, and establish grading expectations, including what to value when grading, turnaround time, and amount of feedback. General expectations around availability, the frequency with which they should be checking the course, and other requirements are explained in the facilitator contract. Any issues during the semester (students “disappearing,” grading discrepancies, students submitting late work, etc.) are typically discussed with the entire teaching team, so that all facilitators approach similar problems consistently.

When a Course Is Underway  151

Weekly Faculty, Facilitator, and Support Staff Meetings Best practice is for faculty to conduct weekly synchronous meetings with facilitators and support staff, using the faculty forum. These meetings, which should last less than an hour, provide a weekly point of coordination for the teaching team and support staff. Topics discussed often include students who are struggling, assignments or assessments that students are struggling with, areas where students need additional synchronous support, any faculty or facilitator illnesses or absences, and any issues with the course content. Much of the discussion in the weekly meetings centers on grading, including grading standards, rubrics, plagiarism, late submissions, and grading uniformity. It is helpful if the synchronous conferencing service supports telephoning in, because facilitators may not always have computer access.

Alyssa Kariofyllis, lead online engagement manager at Boston University’s Office of Distance Education Most of the communication between faculty, support staff, and facilitators takes place during weekly conference calls or via email. Our LMS includes space in the grade center for notes that are not available to students, which could be a good space for faculty and facilitators to keep track of student progress behind the scenes. We previously built private discussion boards where teaching teams could communicate, but we found everyone preferred email because it provides an instant notification, which results in a quicker turnaround for assisting students.

Setting Up Student Groups In courses that have facilitated groups, one of the most important pre-launch activities is assigning students to their groups. Once the students are assigned to groups, their facilitators can reach out to them by email and post welcome messages. In our programs, support staff assist faculty by doing the work of assigning students to groups. It is very helpful if the LMS is linked to the institution’s student information system, because this integrates student data, such as email addresses and those who are adding or dropping the course. A common practice is to assign students to groups randomly, to minimize the number of students taking the same courses being in the same groups. Alternatively, students may be assigned to groups with facilitators with particular expertise in different aspects of a course. As in previous examples, students in a database course who are pursuing a concentration in health

152  When a Course Is Underway

informatics will be assigned to a group with a facilitator who is expert in health informatics databases. In some courses, students can choose software tools, such as database management systems, and students are assigned to groups with facilitators with expertise in those tools. This process of assigning students to groups based on their individual learning goals has proven effective in supporting a diversity of learning goals without increasing costs or the number of courses offered.

Orientation Course We created an orientation course for new students to experience elements of a typical course online. This course allows them to get used to the navigation and practice typical tasks they will be required to use in their courses, such as uploading assignments, taking tests, and participating in discussions. The orientation course also includes tips on how to be a successful student, the time commitments involved and how to manage their time, and provides links to other university services available to them (Bozarth, Chapman, & LaMonica, 2004; Ludwig-Hardman & Dunlap, 2003). The link to this orientation is sent to students in the support staff welcome letter when they enroll in their first course. Programs without support staff can include it in introductory program or faculty welcome letters.

Welcome Messages When faculty first email their students, the goal is to establish a learning community (Conrad, 2002) by welcoming them to the course and reminding them what courses or other prerequisites they should have completed in preparation. Welcome email should be sent out sufficiently in advance so that if students do not think the course is the best fit for them, they still have time to drop it and enroll in another. This email typically includes the syllabus and a list of course readings, so students can get an early start. Welcome email may also include the first lectures of the course, particularly when there is no textbook. For courses in which students are required to install software, these emails provide detailed instructions on downloading and installing the software. The faculty may also email instructions for assignments to help students get started. Students often reply to welcome email, and this provides an opportunity to establish communications and rapport, as well as addressing individual issues, such as waiving prerequisites for well-qualified students and providing disability accommodations. After students have been assigned to groups, but before they are given access to the course, best practice is for faculty to post welcome messages or videos for their students. Ideally, faculty will have created video introductions to the course and modules as part of course development. (For more on video introductions, see Chapters 3

When a Course Is Underway  153

and  6.) Facilitators should also send their groups welcome messages. Faculty may provide facilitators with guidance and sample welcome emails and posts; this is especially helpful for new facilitators. A welcome message may cover the following: • • • • • • •

Welcoming students to the course. This is another opportunity to establish relationships and dialogue with the students. Introducing the faculty. List education, teaching experience, and some humanizing details, such as hobbies. Prerequisites. This is one of the last times to caution students who may not be prepared. Textbooks, required and recommended for the course. This should be a repeat of the information that has already been sent by support staff. The role of the course in the program. Setting expectations of the effort required to succeed in the course. Grading policies and when students can expect feedback and grades.

Administering Email and Announcements Administrators should guide their faculty in the use of email and announcements while courses are running. Most LMSs implement an internal email system. Internal email has many advantages, including reliable transmission, privacy, and automatic archiving with the course. If an LMS email is sent it will be reliably transmitted to the addressees’ mailboxes; this is in contrast to ordinary external (SMTP) email, which may be lost, blocked by firewalls, or refused by the recipients’ email clients or vendors. Internal email is more private than external email, because the only systems that have access are those associated with the LMS, in contrast to the many servers and networks that have access to external email. LMS email is also archived with the LMS, so there is a permanent, auditable record of the communication. Institutions and programs should define policies regarding email communication in online courses. These policies should define preferred means of communicating for different kinds of information. For example, the policy may specify that external email may be used before a course opens, and any academic communication after a course opens should be via LMS email. Announcements serve a different role than email. They appear on the course’s homepage, or in a specific Announcements section, where they are visible to all students, while email is sent only to a specific list of recipients. An advantage of announcements is that they can be written in advance and scheduled for release while the course is running. For example, announcements welcoming students to the course and reminding them to submit course evaluations can be written once and reused for each iteration of the course. Since students see fewer announcements than emails, and they are only from faculty, students are less likely to overlook them.

154  When a Course Is Underway

Alyssa Kariofyllis, lead online engagement manager at Boston University’s Office of Distance Education Instructors often post Live Classroom or due date reminders in course announcements. Some instructors will post weekly updates that focus on upcoming topics or weekly retrospectives that summarize what students wrote in their discussion posts. Some instructors will mention specific student content across groups, which students seem to appreciate. Students have also responded positively to instructors posting links to articles about current events that relate to the content students are studying.

Tasks Performed by Support Staff While a Course Is Running Support staff perform a number of tasks to launch the course and support it while it is running. In programs with fewer resources these tasks may be done entirely by faculty or designers. Some of the tasks are repeated weekly while the course is running. These tasks are typically performed at the times indicated below: Launch day • • • • •

Ensuring the course is available to students in the LMS. Sending “Welcome to Group X” message to students. This introduces the facilitator, and briefly describes how to use the email and internal messaging features of the LMS. Removing students who have dropped the course from the LMS course roster; this may be automated if the LMS is synchronized with the institution’s student information system. Reminding faculty to forward letters from the Office of Disability Services (ODS) to support staff regarding students who have been granted disability accommodations. Creating synchronous session accounts for new students.

First week after the start of the course •

Creating the student attendance report for faculty.

When a Course Is Underway  155

Second week after the start of the course • •

Sending final student rosters to faculty and facilitators. Emailing or calling students taking their first online course to check on how they are doing, and reminding them to register for the following semester.

Every week after the start of the course •

Posting links to synchronous session recordings after a session, if the LMS and conferencing software do not do this automatically. Verify that the recordings are smooth and working correctly.

Three to two weeks before the end of the course • •

Disabling access to the course for withdrawn students. Uploading the link to the student course evaluation tool.

Between two and one weeks before the end of the course • • •

Reminding students that course evaluations are available and the last day they will be accepted. Reminding faculty to ask students to complete course evaluations. Ensuring that final exam settings are correct, if this was not done before the course launched.

The last week before the end of the course •

Conducting the wrap-up meeting with faculty and facilitators.

Contacting Students Who Have Not Logged In In the first two days of the course, faculty and/or support staff should reach out to every student who has not logged into the course, reminding them to log in. If a program has support staff, contacting these students may be one of their responsibilities. If a program has facilitators, but no support staff, then it may be a part of the facilitators’ responsibilities to contact students in their group who have not logged in. Otherwise, faculty need to contact them. This contact is typically initiated by email, with follow-up phone calls if there is no response.

156  When a Course Is Underway

Susan Diesel, assistant director of student and faculty support at Boston University’s Office of Distance Education The short duration [seven weeks] of our courses means we don’t want to wait to get in touch with a student who has not logged in. We reach out to students who have not logged into the course after the first day to make sure they know the course has started. If we get no reply, we ask their facilitator to try to reach the student.

Building a Learning Community At the beginning of the course, students learn what is expected of them and what they will be doing. Key information such as the learning objectives, textbook, and general course organization should be in the syllabus, but important details, such as the effort required to complete the assignments, are best covered in an introductory synchronous session. Such sessions also give faculty an opportunity to hear their students’ questions and concerns about the course. In advanced courses, faculty may also want to learn about special topics that students would like to cover. Again, although all this information can be communicated in a written format, synchronous multimedia interaction provides a familiarity that is difficult to attain through text. These synchronous sessions help students get to know their facilitators and classmates, and to establish dialogues that build trust and understanding between students and faculty, which is beneficial throughout the course (Clark, Strudler, & Grove, 2015). Ideally, faculty strive to help students understand that they want them to succeed and will be available to help them. In our experience, this humanizing effort reduces student anxiety and improves student engagement and satisfaction (Shea, Li, & Pickett, 2006). Study groups or project teams are also created or finalized during the first week. A common set of discussion forum topics can also be useful in building a learning community. There are many benefits to having some course components used across all courses in a program, such as student-introduction discussion posts in which students introduce themselves and describe what they want to learn from the course. Students appreciate these common discussion topics, because they provide a familiar way to become involved in community-building dialogue.

Regularly Scheduled Synchronous Sessions There are many different ways in which faculty can use synchronous sessions. One is online office hours, where faculty are in their virtual offices waiting for

When a Course Is Underway  157

students to come with their questions. Another is a traditional lecture format with slides, such as faculty would use in a face-to-face class, in which students and faculty can discuss the course content. Another is an interactive review session, in which faculty discuss the subject matter and ask their students questions, like a face-to-face class.This is often used to prepare students for assessments and assignments. Such synchronous sessions allow students to ask questions, discuss the material with faculty, and explore some topics more deeply than they are covered in the asynchronous content. In some advanced courses, regularly scheduled synchronous sessions allow students to present their work and have everyone discuss it. For this to work well, the assignments need to be advanced enough and flexible enough so that there is not just one correct answer. These assignment-solution workshops need to be scheduled long enough after the assignment due dates so that almost all of the students will have completed the assignment. Because the solution workshops thoroughly cover the solutions, policies need to be in place so that assignments submitted after the presentations do not count toward students’ grades. There are many advantages to scheduling synchronous sessions at the same days and times throughout the term. When sessions are scheduled in a regular rhythm, students can more effectively adapt their studies to them. For example, students will know that they can get their questions answered at a particular time. Regular scheduling is simpler and makes it easier to coordinate the days and times of the sessions. Regular scheduling also makes it easier to integrate the synchronous sessions with the students’ learning activities during the week. For example, a regularly scheduled assignment-solution workshop is naturally scheduled at the same time each week after the assignment is due. However, there are also advantages to scheduling some synchronous sessions that are not in a regular weekly rhythm. For example, it is very helpful to have a final exam review session shortly before the final exam. Synchronous sessions are most effective if students have headsets or other means to participate with voice and possibly video. In our experience, headsets are necessary for synchronous sessions, to prevent acoustic feedback. Headsets are now inexpensive and they can be used for all courses.

Susan Diesel, assistant director of student and faculty support at Boston University’s Office of Distance Education Faculty use synchronous sessions in a variety of ways. Traditional lectures with screen share, question and answer sessions, office hours, guest speakers, and group presentations are all common.

158  When a Course Is Underway

Outreach Summarizing Prior and Coming Weeks Best practice is for faculty to reach out to their students at the end of each module, summarizing what was covered in the previous module and introducing the next module or exam. This outreach can be accomplished using the announcement facility of the LMS, email, video, a synchronous session, or any combination of the above. These periodic interactions reassure students that faculty are involved in the course. If the interactions are synchronous, this provides an additional opportunity for faculty to establish dialogues with their students, learn about their concerns, and get feedback on previous modules. In large courses it is nearly impossible for faculty to participate in all discussions, but some faculty engagement with students is very important. For example, if students create posts introducing themselves, faculty or facilitators should reply to these posts to try to find common ground and help form a sense of community. If significant issues arise or a facilitator asks for help, faculty participation in discussions can be essential. For example, if a student posts a question on a topic that is more advanced or specialized that the facilitator can handle, then faculty will need to respond. Occasionally discussion posts may contain material that identifies problems in the course, is offensive, or otherwise needs faculty action; in these cases, facilitators or support staff can bring them to faculty attention.

Study Groups One of the ways that students form lasting connections is by studying together. Ideally, courses are designed so that students study and work on problems together. This poses academic challenges because students may be submitting solutions that are not entirely their own work. (For more on building courses that support both learning communities and academic integrity, see Chapter 3.) Study groups can be instrumental for student learning, because they allow them to engage with their peers.We have found that it works best if each study group or project team has its own virtual meeting room.These rooms allow the members to communicate, coordinate their activities, share their work products, organize their deliverables, and present their results. These rooms may be created by designers or faculty before the course begins.They may also be created while a course is underway for groups of students who discover that they share interests in special topics or are in the same locales. It is beneficial to identify these groups early in the course and create virtual study rooms or other infrastructure to support them. If an institution does not have the infrastructure for these groups, social media groups that allow restricted enrollments may support many of these needs. Outside social media has risks, however; the ability of faculty to monitor it and correct any errors of content may be compromised (see other issues with outside social media in Chapter 7). Study groups require some level of monitoring for inappropriate activities, which may also be difficult to handle in uncontrolled social media.

When a Course Is Underway  159

Calling Students We have found that one of the best ways to reduce students’ anxiety is for faculty to speak with them one on one, but this is generally done in only the highesttouch programs. Traditionally, it was done by telephone, but now it can also be done with synchronous conferencing software. This especially benefits students who are not doing well in a course, because a friendly conversation may reduce their stress, boost their motivation, and help them improve their study habits (Croxton, 2014). Students truly appreciate faculty speaking with them, because they can talk about academic issues and also ask career-related and other questions. Faculty also benefit from these calls, which give them a chance to get to know their students, how they feel about the course and program, and the challenges they face.These calls help faculty provide a human touch to online courses. Before faculty can phone students, they need phone numbers that their students will answer. Institutional records may have obsolete numbers or numbers that students do not answer. We overcome this by asking students the numbers at which they would like to be called, their city and state, and the best time to be called. A questionnaire to collect this information can be created in the LMS, or faculty can ask students to email the information to them. (We also use the city-level information to post a world map showing where students are located, to help them organize face-to-face study groups. We do not identify who is in a city, but merely that someone is there.) In facilitated courses, each of our facilitators collects this data for their group and provides it as a table to the faculty. Many students never answer the phone, or only answer when the phone call is preceded by a text. Faculty should be aware of this and let students know both that they will be calling and the number from which they will call.

Students at Risk During the middle weeks of the course, students study the readings and other material, and work on their assignments, assessments, and projects to build knowledge and prepare for the final exam or other end-of-term activities and deliverables. It is important that the middle-week activities include enough assessments, so that faculty can identify students at risk early enough to take effective corrective actions. For this reason, best practice is to include an assessment in each module. Students can be academically at risk for many reasons (Wladis, Conway, & Hachey, 2016). For example, they may be spending a lot of time on another course.They may be under-prepared for a course.They may be dealing with life, health, or work crises that make studying more challenging. They may be struggling with a learning or other disability. They may have difficulties because of inadequate language proficiency or the faculty’s wicked Boston accent. Students at risk can be identified either when they disclose that they are having difficulty or when they do not perform well on assessments. They can also be identified by facilitators, support staff, or even automatically by some LMSs,

160  When a Course Is Underway

which compute student risk indicators from student scores in the course grade book. (More sophisticated systems track students’ grades across all courses, as well as information from the student information system, to get a wider picture of a student’s risk profile.) Once students at risk have been identified, faculty should reach out to them to give them an opportunity to describe what is causing their difficulties in the course. We have found that speaking one on one works best, because the informal format makes students more comfortable. It is good to begin this dialogue by asking the student if there is anything the faculty can do to help with the course. Often students’ replies will lead to straightforward academic solutions, such as allowing additional time to submit work. Occasionally students may disclose personal information in this dialogue. If this disclosure involves disabilities the faculty should refer them to the institution’s disability services. (For more on accommodating students with disabilities, see Chapter 3.) Disclosures of health or many other kinds of personal information should be regarded as privileged and confidential, and handled in accordance with applicable laws and the institution’s policies.

Susan Diesel, assistant director of student and faculty support at Boston University’s Office of Distance Education Students are identified as at risk if they show a lack of participation or logins to the online course environment, failure to complete assigned work, poor performance on assignments and assessments, or any combination of these. We respond by checking in with the student to see if there are any external circumstances affecting the student’s ability to participate in the class. We may recommend resources to the student to help them academically or personally.

Wrapping Up In the last week or two of a typical online course, students synthesize the material, complete their term projects, and prepare for the final exam or other end-ofcourse assessments. As with face-to-face courses, faculty may wish to have a lower assignment workload near the end of the course, to allow students more time to complete projects and study for the final exam. It is helpful if faculty conduct synchronous final exam review sessions sometime in the three days before the final exam. These sessions should be recorded so that students can review them. In courses with term projects, it is advantageous to have students present their term projects in synchronous sessions attended by their faculty and classmates.

When a Course Is Underway  161

This affords them an opportunity to refine their presentation skills. It also strengthens the academic integrity of the course by discouraging students from presenting other people’s work as their own, because students know that their faculty and classmates will ask them questions.

Administering Proctored Exams There are several ways to ensure that students who get credit for courses are the ones who took them (see Chapter 3). The traditional way is to assess their knowledge in the classroom, including quizzes, exams, student presentations, and extensive dialogue between students and faculty. This can be done online with fully synchronous courses, but in our experience this format does not work as well pedagogically as online courses designed with substantial asynchronous components. Another way to assess student knowledge is to require some faceto-face attendance for every course, with comprehensive assessments and presentations. This technique, while effective, is expensive for students because of the travel, so it is only used in a few online PhD programs. A third way is to have synchronous activities, such as the term project presentations listed above, in which other students, facilitators, and faculty ask them probing questions. This helps ensure the integrity of term projects, if not the entire course, so it should be combined with other methods. We have tried many ways to improve academic integrity. Over the last two decades, a number of businesses have emerged to provide proctoring services for online programs. Some of them have required students to go to local testing centers, while others have allowed students to take exams from their homes or places of work. In our experience, one of the problems with such proctoring centers is that their computers and firewalls have not been sufficiently up to date to support our online exam technologies. We have evaluated many of these services and have found that almost none of them meet our academic requirements. These include not requiring students to travel, the ability to support longer sessions for students with authorized disability accommodations, full recording of students and their computers, positive student identification, the ability to handle unexpected events, support for makeup exams, and means to survey and monitor the rooms where students take the exams. Our experience has been that proctoring services need to provide human proctors, who can handle situations such as a fire alarm going off or a student needing to go to the bathroom. As the market has matured, the services that have survived have human proctors observing a dozen or more students via webcam, with variable support from artificial intelligence systems. For example, some of the services have used AI to track student eye movements during an exam, and flag moments when students look away from their monitor, which might indicate cheating. We expect that over time more of the proctoring will be automated, and this will reduce the cost.

162  When a Course Is Underway

We have found that a multi-step process is needed to evaluate proctoring vendors. The vendor’s offerings and cost are evaluated against the program’s proctoring requirements. A few of these services have required students to purchase specialized proctoring hardware, such as 360° cameras and fingerprint readers, while others require only webcams with microphones. If a service does not support makeup exams, they may be disqualified. Vendors who meet all of the requirements may advance to testing and evaluation in a pilot. Testing may involve having faculty, staff, and potentially students taking an exam to see how well the proctoring works; this may involve attempts to cheat to see if the proctoring catches it. Services that pass the testing hurdle proceed to operational evaluation, with a real final exam in one course.This process may seem elaborate, but we have found it necessary, because we have identified critical shortcomings at all of these stages. Proctoring services charge a fee for each exam. This makes it expensive to proctor quizzes or other smaller assessments. Consequently, many online programs use comprehensive proctored final exams rather than proctored quizzes or midterms to obtain good integrity at minimal cost. The cost of proctoring is typically covered by tuition. We have found that comprehensive proctored final exams are effective and relatively inexpensive. It improves academic integrity if all students in a course take the exam during the same time period, because this makes it more difficult for students to share their knowledge of the exam with classmates. However, we have found that exam windows should be long enough to allow students some scheduling flexibility, including at least one weekend day to accommodate working students, and to spread the proctoring workload. Strong randomization can help address this conflict, as it ensures that students will have different questions (for more on academic integrity, see Chapter 3). One of the considerations in evaluating proctoring services is their ability to handle peak workloads, such as the Saturday of an exam window. Only students can identify the best time to take their exams, and only proctoring services know when they have available testing times, so it is best to have students enroll via websites provided by the services. We have achieved the best results when support staff verify that all students have enrolled, and contact students who have not. In larger programs, this task may be performed by support staff specializing in exam coordination; in smaller programs, this may be performed by support staff in addition to their other duties, or by faculty. Student support is needed while students are taking exams to address issues such as problems with students’ computers or networks. Some of the technical issues can be resolved by the service, and some by the program’s technical support, but some require academic rulings, such as when a student runs out of time because of technical issues or needs to reschedule their exam. We have found that it is very helpful if students are provided with a phone number that they can call to reach this support, any time during the exam window.

When a Course Is Underway  163

Rarely, a student situation may preclude the use of a proctoring service. This most commonly occurs when students are in locations with inadequate bandwidth for a proctoring service; in these situations, students should propose alternative proctors, which must be approved by the program. Sometimes a student may need to take a different final exam than the other students. Some proctoring services support this. If the service does not, faculty may conduct an oral final exam. While this method can work very well, it takes a lot of faculty time and thus does not scale.

Winding Down the Course The course may not end immediately after the final exam has been completed. Some students may have incompletes, while others may wish to go over their final exam or review another student’s project. Therefore, we make a number of non-required activities for students available after they take the final exam. We have found that keeping the course open for two weeks after the final exam closes is sufficient to allow students to review the material and continue communicating with their classmates. During this time, students will often exchange contact information to stay in touch with their peers. Students should also be able to download their graded projects for their records and e-portfolios. We have found that recordings of project presentations can be useful for students, and we keep these recordings available until the last student with an incomplete in the course has finished. After students have taken the final exam, they may have more time and can also provide feedback about the final exam, so this is a good time for faculty to solicit feedback on the course and ways to improve it. When faculty are seeking guidance or recommendations for course updates, they can create surveys for students to complete after the final exam. (For more on planning course updates, see Chapter 10.) These can help faculty adjust emphasis on course topics, identify new topics, and better understand how well different learning objects and activities worked for students. Best practice is for faculty to acknowledge the conclusion of the course by sending a wrap-up announcement or email which congratulates students on their accomplishments and wishes them well in their studies. Faculty can still communicate with students, including answering questions about the final exam, or helping students finish any incomplete work before the next term starts. Support staff perform a number of tasks to wrap up the course after it closes. In smaller programs these tasks may be done entirely by faculty or designers.The day the course ends, they send email to students announcing the course closing and reminding them that it will become unavailable two weeks after the end date. The also check for students with incomplete grades and ensure that they have access to the course through their completion deadline. Between the day the course ends and three weeks later, they run LMS reports of student activity, for later analysis.

164  When a Course Is Underway

Alyssa Kariofyllis, lead online engagement manager at Boston University’s Office of Distance Education Some faculty send end-of-semester messages that summarize what students have learned and how it may be useful to them in future courses and their careers. Faculty and facilitators will often have a final Live Classroom session to answer any lingering questions from students. Depending on the exam format, faculty and facilitators may also provide detailed feedback on student answers.

Building Community Beyond the Classroom Relationships that persist beyond the end of the course can be very valuable. In lower levels of education, these connections help students develop social skills. In higher levels of education, they can foster career development. It is usually desirable to encourage students to share phone numbers, email addresses, and other contact information. One way to support this ongoing learning community is to create a website for continuing students and graduates. The group site can be sustained with posts by staff, faculty, and other students to share information about ongoing learning opportunities, job postings, and get-togethers.

Summary Checklist ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑ ☑

Let students know how to prepare for the course and program. Describe support staff responsibilities. Execute support staff QA tasks. Execute support staff pre-launch tasks. Conduct a kickoff meeting for the faculty, facilitators, and support. Conduct weekly faculty, facilitator, and support staff meetings. Set up student groups. Enroll new students in the LMS orientation course. Post welcome messages from the teaching team and support staff. Create policies for email and announcements. Document the tasks to be performed by support staff. Contact students who have not logged in. Create the infrastructure for a learning community. Create policies for scheduled synchronous sessions. Create policies about recapping previous modules and introducing upcoming modules.

When a Course Is Underway  165

☑ ☑ ☑ ☑ ☑

Create study groups. Consider calling students. Identify students at risk and reach out to them. Provide infrastructure for student presentations. Administer proctored final exams or other comprehensive summative assessments. Keep the course open for a week or more after the final exam, so that stu☑ dents can continue their learning and communicate with each other. ☑ Provide infrastructure for students to maintain their learning community.

References Bozarth, J., Chapman, D. D., & LaMonica, L. (2004). Preparing for distance learning: designing an online student orientation course. Journal of Educational Technology & Society, 7(1), 87–106. Clark, C., Strudler, N., & Grove, K. (2015). Comparing asynchronous and synchronous video vs. text based discussions in an online teacher education course. Online Learning, 19(3), 48–69. Conrad, D. L. (2002). Engagement, excitement, anxiety, and fear: learners’ experiences of starting an online course. The American Journal of Distance Education, 16(4) 205–226. Croxton, R. A. (2014). The role of interactivity in student satisfaction and persistence in online learning. MERLOT Journal of Online Learning and Teaching, 10(2), 314–325. Ludwig-Hardman, S., & Dunlap, J. C. (2003). Learner support services for online students: scaffolding for success. The International Review of Research in Open and Distributed Learning, 4(1). Shea, P., Li, C. S., & Pickett, A. (2006). A study of teaching presence and student sense of learning community in fully online and web-enhanced college courses. The Internet and Higher Education, 9(3), 175–190. Wladis, C., Conway, K. M., & Hachey, A. C. (2016). Assessing readiness for online education—research models for identifying students at risk. Online Learning, 20(3), 97–109.

Chapter 9

Addressing Unexpected Developments

This chapter is primarily of interest to faculty, because they are responsible for coordinating responses to most unexpected developments. Administrators, designers, and support staff may also be involved in resolving some unexpected issues. This chapter provides best practices for addressing: • • • • • • • • • • • • • • •

Technical problems and technical support. LMS outage or slowness. Faculty or facilitator illness or absence. Course design or implementation problems. Students having the wrong textbook or no textbook. Students in crisis. Challenging unexpectedly strong students. Helping underprepared students. Students having difficulty with assignments or assessments. Student complaints. Late submissions. Content errors or omissions. Misalignment of course content. Uneven student workload. Compromised academic integrity.

Many unexpected things can occur while a course is running. Some of them also occur in face-to-face courses; others, such as LMS outage or slowness, are unique to online courses or are more important online. This chapter offers tips on how to address issues that are identified while a course is underway. Some problems, such as unclear writing, mismatch between the level of the course and the students, and poor pedagogy or design, can be addressed in part while a course is running, but fully addressing them requires more extensive course updates.

Addressing Unexpected Developments  167

Technical Problems and Technical Support While technology enables online education, it can also cause a variety of problems. It is desirable to have technical support dedicated to online programs, because personnel who specialize in online support will have the skills to diagnose and address the complex issues that can arise due to interactions between LMSs, students’ internet service providers, students’ computers, and browsers (Haugsbakken, Nykvist, & Lysne, 2019). Dedicated online support can help students troubleshoot their problems with the LMS, or this technical support may be provided by the vendors of LMSs and other services. We have experimented with outsourcing technical support to LMS vendors; unfortunately, when we did so, our students in technology programs were disappointed that the support personnel were unable to handle difficult problems.We have found that in-house technical support works well for programs with more resources, because they can afford to hire people who have both good technical skills and knowledge of the academic programs. Still, programs with fewer resources may need to outsource technical support. When faculty become skilled and confident using technology, they are better able to handle their own technical problems (Zheng, Wang, Doll, Deng, & Williams, 2018). They know that technology provides them many ways to teach, and they know how to adapt their teaching when some technologies fail. Similarly, experienced faculty know that if a synchronous technology breaks or is not working for some students, they can create documents or videos and send students links to them.

Lawrence Ragan, retired director at Penn State’s World Campus When I was doing faculty development, I always positioned myself in a service mode. I would say to them, the last thing we want is for you to fail. We’re going to be here, we’re going to be behind you, you’re not going to look bad, because that was the fear.Those days the fear was, “Oh my goodness what if the internet crashes or something.” I said, “That’s probably not going to happen, but we’re going to be here.” And once they knew we had their back, and we had strategies to accommodate for these things, they became much more comfortable, much more flexible and willing. They figured out where their added value was, which is in that content domain. As a faculty member, when you get over the fear of failure, when you get over the fear that you’re going to break something, that it’s going to collapse somehow, once you bridge that point, you’re pretty good. When things go wrong for faculty members, when they are comfortable with technology, they have more capabilities, and can have a more seamless flow between the synchronous and asynchronous pieces.

168  Addressing Unexpected Developments

LMS Outage or Slowness A server may be functional but too slow to support good student interaction. This can arise because the server is overloaded, because of technical problems with the server, or because of network problems. Server slowness is most common and likely to cause issues in the courses near the end of term, because of increased server load due to students uploading large assignments and taking their final exams. Software and systems are available to monitor server performance, and this can help promptly identify slowness and outages. Programs should also include information for students that tells them who to contact if they experience these problems. When students cannot access the LMS or the LMS is unusably slow, they feel cut off from the course.Technically sophisticated students will be concerned, but less proficient students may panic. If this occurs, faculty and support staff should reach out to students via external email, reassuring them that the institution is addressing the problem and that they will be taken care of (Tallent-Runnels et al., 2005). If the outage is expected to last more than a few hours, it may be helpful for faculty to email the students study materials such as course extracts and assignments, and to conduct extra synchronous sessions, if possible. Faculty can also extend deadlines and allow students to submit their work late without penalty. If the outage is expected to last for a day or longer, it is usually helpful to create a temporary group for the class in an external social or professional media service such as Google Groups or Facebook, so that students can ask questions and stay in touch with their peers and faculty. Sometimes a multimedia messaging system used in the course may still be functioning, even though the LMS is not. In other situations, a new multimedia forum may need to be created until the server is again functional.

Faculty or Facilitator Illness or Absence Illness or a personal crisis may prevent faculty or facilitators from performing their duties; therefore, we design our courses so that substitutes can easily assume their roles. One good practice is to have reserve faculty and facilitators identified who can be quickly enrolled in the courses, if needed. For example, either an area coordinator or an adjunct faculty member acting as a facilitator may be fully qualified to substitute for a faculty member who falls ill. If a facilitator is ill or otherwise gets behind in their grading, it is helpful if courses are designed so that any facilitator or faculty can grade any item or respond to any discussion post. In courses with many facilitators, it is helpful to have a designated reserve facilitator who can assume the responsibilities on short notice. Reserve facilitators should ideally be more experienced, because if a situation warrants the services of a reserve, additional effort and skills may be needed. Reserve facilitators need not add to the cost of a program, because they may already be group facilitators in the course, who can take on the additional responsibilities if called upon.

Addressing Unexpected Developments  169

The same facilitator can serve the reserve role in more than one course if the probability that he or she will be activated is low. It may be appropriate to enroll more than one reserve facilitator, if there are many unproven facilitators in a course. For example, in one of the authors’ courses there were many new facilitators, and several of them needed to be replaced early in the term.

Course Design or Implementation Problems Occasionally technical or design problems with a course may not be detected by the QA processes and only be identified while a course is running. In courses with links to external websites, the most common problem is a broken link. Faculty usually need to be involved to identify an appropriate substitute web link, if one can be found. If no substitute link can be found, faculty may need to create additional content to replace the linked material. For example, one of our courses had about 100 links to a government web server. While the course was running, the server was updated, and all of the links broke. For this reason, we recommend minimizing the use of external links for critical material. Rarely, problems may be identified in the design of the course, such as broken internal links or material that was not updated. These problems usually need to be addressed by faculty and designers, or faculty alone.

Students Having the Wrong Textbook or No Textbook Occasionally a student may have the wrong edition of a textbook, or the wrong textbook altogether. This can occur when students consult an old syllabus that they found on a search engine cache, and order the book required by an old version of the course, or when they order the newest edition online before the course has been updated to that edition. Students with limited financial resources sometimes ask faculty if they can use an earlier edition of a text, as it is usually much less expensive, particularly if purchased used.When a new edition becomes available shortly before a course launches, we anticipate that some students may have already bought the older edition and some may have the new edition, so best practice is to provide references in the online materials for both editions. These dual references can be created by facilitators or faculty. If the new edition of the text covers material that is not in the older edition, lecture material should be added to cover these topics, assuming that they are relevant. Sometimes a student does not receive the textbook in time for the start of the course. This can occur when a student does not order it in time, or when they order it from a deep discount web vendor who may take many weeks to deliver. We try to prevent this by guiding students to purchase early, and from a reputable online vendor who has the book in stock. Many institutions have arrangements with their affiliated bookstores that can handle sending books internationally with guaranteed on-time delivery, or send a digital version.

170  Addressing Unexpected Developments

We have found that when courses are repeatedly updated and improved, they tend to cover the material from a textbook in ways that are more interactive and easier to understand, so courses become less dependent on the textbook. This can eventually result in courses with all of the course material covered in the online websites. When this occurs, textbooks may become optional, and mainly serve to help students study before the courses open.

Students in Crisis Many things can happen to students that impair their ability to study, including illness, family issues, natural disasters, or job crises. If an institution has specialists to help students in crisis, faculty or administrators may refer such students to them. Faculty deal with these situations most commonly by affording students additional time to complete their work. If this additional time extends beyond the end of the term, special procedures, such as incompletes, are used, just as they are used in face-to-face classes. When a student receives an incomplete in a facilitated course, the student’s late submissions are best graded by their regular facilitator, to maintain continuity. When courses have students in many geographic locations, they are more likely to have students affected by hurricanes, earthquakes, or other natural disasters. Our students have undergone lengthy power and internet outages, and have had their homes flooded in hurricanes and destroyed in wildfires. If only a few students in a course are affected by such an event, it can be handled using the incomplete procedures described above. If many students in a class are affected by a disaster, programs should consider organizing the affected students into facilitated groups that extend past the end of the term, to provide the same level of support and interaction that they would have received during the term. Online formats that include significant asynchronous components work well for incompletes, directed study, and other study models, particularly when students have access to recordings of any synchronous components and access to faculty to answer their questions.

Challenging Unexpectedly Strong Students Some students in a course may be better prepared than the course was designed to accommodate. To keep these students challenged and help them learn, it is important to give them replacement work that is more advanced, rather than just giving them more of the same level of work. If the regular assignments or problems are too easy for advanced students, faculty can create advanced assignments for these students. If the advanced assignments cover all of the relevant areas of the regular assignments, best practice is for the students to do the advanced assignments instead of the regular assignments. For example, in a course taught by one of the authors, there is an assignment to create a data warehouse that handles structured data. The professor created a second, more

Addressing Unexpected Developments  171

advanced, assignment that included both structured and unstructured data. In another example, a language course could have an assignment requiring translation of a poem, and a more advanced assignment would require the poem to be translated in a way that preserves its connotations and effect. Experienced faculty will often have a number of these challenging assignments in reserve, and they can let advanced students select the ones that they find most interesting. They may also be willing to share them with less experienced faculty, to help ease their workload. Advanced students also need more advanced lecture material. This can be provided during development time as Advanced Topic webpages in the online lectures, and while a course runs, as more challenging readings, identified as being based on student interests. If many advanced students are interested in an advanced topic, faculty can recommend advanced readings, more challenging topics, extra-credit work that is more challenging, or even conduct a synchronous session on it. This approach works well online, but is far more difficult to implement in face-to-face courses. If there are many advanced students in a course, it is beneficial to create extracredit, advanced discussion forums. If participation in these activities is graded as a few percent extra credit, faculty should not be concerned that this will distort the grading, because stronger students will already earn A grades. Eager, but less advanced students who participate can learn from these advanced activities and earn appropriate extra credit. Best practice in these situations is to offer advanced students a list of topics to discuss, ask if there are any other topics they would like to cover, and whether they would like to discuss them asynchronously or synchronously. For example, in a course on book production, some advanced students may be interested in how fore-edge books are produced. Advanced discussions can be synchronous, particularly if there are no appropriate readings. Faculty can announce an optional, advanced synchronous session and make it available to the entire class. Alternatively, if students are interested in advanced topics for which faculty can provide readings, asynchronous discussion questions can direct students to research a topic in the advanced readings, and share what they have learned.

Helping Underprepared Students When teaching a course, faculty may discover that some students are not as academically prepared as expected. Students may be underprepared in only a few areas, or more generally underprepared. Under-preparation commonly occurs when students have taken weak prerequisite courses, or have simply forgotten what they have previously learned. Broadly speaking, there are two techniques for addressing under-p­reparation. One is to require students to take prerequisite courses. If the under-preparation affects many students in courses with prerequisites, this indicates that the prerequisite courses may need to be strengthened. When many students have been found to be

172  Addressing Unexpected Developments

underprepared in a specific topic area, we analyze their prior coursework to identify and correct weaknesses. For example, students in an advanced painting course may struggle with color mixing, which should have been covered in the introductory course, and this can be corrected by strengthening the coverage of color mixing in the introductory course. In programs that have transition courses, we have found it helpful to include the material that underprepared students may be missing; our experience has been that when transition courses are tuned in this way, students who complete them are well prepared. The other method for addressing under-preparation is to provide additional material in the courses, just as with the more advanced students. Since one of the great advantages of asynchronous online education is that students can be studying different topics at the same time, faculty and designers can take advantage of this by introducing Help links or buttons which open webpages covering prerequisite material, or by including mouseovers or links that present definitions of terms and phrases that students may have forgotten. In courses with prerequisites, we have found it advantageous to include a Module 0 that is a review of prerequisite material. (For more on Module 0, see Chapter 6.)

Students Having Difficulty with Assignments or Assessments Students may have difficulty with assignments or assessments for various reasons: because of limited English-language comprehension skills or insufficient knowledge of the material, because they are unclearly written, or some combination of the above. When students have difficulty with an assignment, a good first step is to ask them which part of it they need help and are struggling with. If many students have difficulty understanding a particular assignment, occasionally the best approach is to reformulate it to make it easier for students to understand, or to embed hints, examples, or guidance in the material. For instance, an assignment might remind students that it builds on an example from the online lecture. Sometimes faculty will learn that students are having difficulty with assignments or assessments because the online course material has not prepared them sufficiently. In these situations, faculty can add content to the course to help students prepare, such as offering synchronous sessions focused on topics the students are struggling with. Faculty can also make some advanced assignments and assessments fall due later in the course, allow students to resubmit them, or make them optional and extra credit. If students understand an assignment, but have difficulty working on it, one of the most effective ways to address this is for the student and faculty to discuss ways to approach it. In facilitated courses, the best approach is for a student to meet with his or her facilitator in the facilitator’s virtual meeting room. In many cases, simply starting from the beginning and re-reading the assignment will clear up the student’s confusion.

Addressing Unexpected Developments  173

If many students are having difficulty completing an assignment, this indicates that the students are not adequately prepared for the assignment, or that something may be wrong with the assignment itself. In these situations, the best approach is often for the faculty to create another assignment that they think that the students are ready for, or to adjust the original assignment. The new assignment should bridge the preparation gap, enabling the students to complete the original assignment. Another approach is for the faculty to provide example solutions of similar problems, crafted so that the solutions guide the students over the parts of the assignment that they are having difficulty with, without revealing much of the solution. If students are having difficulty solving a complex problem that takes multiple steps to work out, it is often helpful to break the problem into a sequence of simpler problems. A related approach, which works well for humanities courses, is to factor the problem into subproblems. For example, a question involving analysis of the history of a civilization can be factored into analysis of its government, economy, military, literature, and culture.

Student Complaints Students may complain about academic issues, including grading, online lectures, synchronous sessions, recordings, assessments, assignments, and faculty teaching style. Students may also complain about the technology and design used to deliver the course, such as confusing navigation, slow response of some components, audio and video quality, and difficulty playing or downloading recordings. We also get feedback regarding the technology and design in the student evaluations. Online students need to know who to contact with their complaints, so online courses should provide students with contact information for the faculty and staff who are most appropriate to address for specific types of issues. Online programs need some organized process for communicating complaints to the people who can most effectively address them. For example, complaints about the content or grading should be communicated to the faculty, complaints about faculty should be communicated to the program director or department chair, and complaints about the technology should be communicated to the faculty and support. Online courses should also include an escalation sequence—who to contact first, who to contact second if that does not work, and so forth. If the students know the appropriate escalation sequence, they are less likely to contact the highest levels of the university administration until they have exhausted previous options. Student complaints can be communicated in many ways: via email, social media, by telephone, or in person. Student support staff or faculty may also notice student complaints when monitoring online discussions. How these complaints are communicated and managed depends on the urgency of the issues. For example, a problem that is preventing a student from finishing a final exam should usually be communicated by a phone call, while a spelling error in a lecture may be communicated via email.

174  Addressing Unexpected Developments

Late Submissions Occasionally students may be unable to complete some deliverables on time. The way faculty handle these situations makes a great deal of difference to students. Fortunately, faculty have more ways to handle late submissions online than in face-to-face classes. Online courses support private communication between faculty and students who submit late. Asynchronous formats provide faculty and students with additional options when a student falls behind. For example, if a student falls behind in their study of asynchronous material, it is still available to them later. This is different from a face-to-face course in which the class simply moves on, sometimes leaving the student behind. Students who fall behind on their assignments can usually submit them later than their classmates without compromising the integrity of submissions. As with face-to-face teaching, best practice is to not penalize students for late submissions when they have communicated legitimate reasons to their faculty. When late submissions are due to transient causes, such as power or internet outages, students who fall behind can usually catch up. Some circumstances, such as a protracted illness, may prevent students from completing their assignments for a long period, during which assignment submissions of other students may have become available. In these situations, it is desirable to have a second, parallel set of assignments.When students cannot catch up during the term, then predominantly asynchronous courses can continue to support their learning after the end of the term, which makes incompletes more manageable online than in a traditional classroom.

Content Errors or Omissions Many things can cause errors or omissions in the content of a course, ranging from typos and poor writing to changes in the subject area. It is important that errors be corrected promptly to minimize the risks of student confusion, and to address negative feelings that students may have about the course.When students identify content errors or omissions, best practice is to acknowledge this contribution to the course, often by awarding extra credit. If erroneous, unclear, or ambiguous text or figures are detected while a course is running, the best approach is to promptly inform students and correct the errors. If the meaning was clear in the uncorrected content, then it is not necessary to notify students. If, however, the errors have confused students, due dates may need to be adjusted. Similarly, when content omissions are identified while a course is running, it is usually appropriate for faculty to cover the material in a synchronous session, and it may be appropriate to add lecture or other content. Rarely, regrading may be necessary. In most disciplines, ongoing research, or social, political, or cultural changes may make some of the content of a course obsolete.This obsolescence may take

Addressing Unexpected Developments  175

the form of changes to topics and their significance, irrelevance of legacy topics, or introduction of new topics. For example, a course on surveillance laws may need to be updated substantially to incorporate material on video, audio, motion sensors, and drones. If current research or events become important but are not covered in a course, best practice is to add the material, presented as emerging research or relevant current events, and use it to seed discussions in synchronous sessions. Sometimes additional content will need to be added to prepare students for new material. For example, students might not have fully understood the importance of the fall of the Soviet Union without some background on the history of the Cold War. New course content presented as the latest developments always generates student interest. The synchronous presentation of this material imparts a sense of currency and immediacy that helps students feel more engaged with the course and their faculty.

Misalignment of Course Content Sometimes misalignments between course elements are only detected when a course is underway, usually by students. This occurs most commonly when they need to understand multiple concepts to solve a problem, but one of the concepts has not been covered yet. Faculty reliably identify major concepts needed to solve problems, but may miss minor concepts, particularly if they are regarded as prerequisite for the course. Misalignment is most common in the first offering of a course and after extensive reorganizations or updates. Alignment problems can make it difficult and frustrating for students to learn the material, especially when they are asked to do an assignment or assessment that involves material that they have not covered yet. Minor alignment issues can be dealt with when a course is running by simple means such as changing due dates. For example, if an assignment requires content that is covered in a future module, this can be addressed while the course is being taught by making the assignment due later in the course. Larger-scale alignment problems are more serious, but they are correspondingly easier to identify, and are usually corrected before the course runs. (For more on alignment during course development, see Chapter 6.)

Uneven Student Workload Sometimes faculty discover that the student workload is uneven when the course is already underway, especially when students complain that the workload is too high. Student workload assessments can be improved on the fly by including surveys in the course, asking students to provide their assessment of their workload for each module and assignment, and suggestions for adjusting it. If serious workload problems are identified, they can be addressed by delaying due dates of deliverables in high-workload modules, making some deliverables

176  Addressing Unexpected Developments

extra credit, or reducing the scope of some assignments. For example, if an assignment has multiple problems, one or more of them can be made extra credit or due later as a separate assignment. In courses with term projects, the workload of students who are behind can be reduced during the term by having them finish their projects after the end of the term. Allowing students to complete a term project after the term should be reserved for those who have been disadvantaged by illness or other life events, so that this additional time does not give them an unfair advantage. When course workload problems have been identified, a course update should be planned to correct them.

Compromised Academic Integrity The academic integrity of a course can be compromised when it is running. For example, problems or solutions to assessments or assignments may become available to students who have not yet submitted their work. One way that faculty can address this is to examine the students’ scores and submissions on the compromised assessments or assignments to determine if the posted solutions were used. Often posted solutions contain distinctive features or errors that facilitate this. If the scores are consistent with the students’ earlier performance, and there is no evidence that the posted solutions were used, faculty may choose to let the scores stand. When such problems are identified, faculty should create new assessments or assignments, schedule a course development update, and initiate disciplinary actions. Fully developed courses may have a large number of graded items, so if some are compromised faculty may ignore them, and still have enough scores for calculating students’ grades. If faculty are not comfortable counting a student’s compromised submission, an approach that works well is to have the student complete an alternate assessment or assignment on the spot. (For more on academic integrity, see Chapter 3.)

Summary Checklist If technical issues arise, adjust to the situation while they are being fixed. ☑ ☑ Monitor LMS performance. ☑ If an instructor or facilitator is unable to perform their duties, provide a substitute. ☑ Ensure that someone—faculty or designers—is available to correct problems. ☑ If students have the wrong edition of a text, provide alternate readings. ☑ If students are in crisis, consider contacting appropriate medical or other specialists. ☑ If a course has unexpectedly weak or strong students, create additional content for them.

Addressing Unexpected Developments  177

☑ If some students are not adequately prepared for some of the material, and are struggling, consider offering synchronous sessions on that material. ☑ If assignments or assessments are not a good fit for the current students, consider creating alternate assignments or assessments. ☑ If students complain, listen to them and address their concerns. ☑ If students submit deliverables late, and are falling behind, ask if there is anything you can do to help. ☑ If typos or other minor bugs are identified, correct them as soon as possible. ☑ If there are misalignments, consider deferring deadlines until the material has been adequately covered, and schedule an update. ☑ If students complain that the workload in a module is too high, consider deferring some of the deliverables to a later module or making them optional, and schedule a development update to level the workload. ☑ Develop academic integrity policies and procedures. ☑ If some of the assessments or assignment solutions become available, consider creating new assessments or assignments, and schedule a development update to improve the course’s academic integrity.

References Haugsbakken, H., Nykvist, S., & Lysne, D. A. (2019). The need to focus on digital pedagogy for online learning. European Journal of Education, 2(3), 25–31. Tallent-Runnels, M. K., Lan,W.Y., Fryer,W.,Thomas, J. A., Cooper, S., & Wang, K. (2005). The relationship between problems with technology and graduate students’ evaluations of online teaching. Internet and Higher Education, 8, 167–174. Zheng,Y.,Wang, J., Doll,W., Deng, X., & Williams, M. (2018).The impact of organisational support, technical support, and self-efficacy on faculty perceived benefits of using learning management system. Behaviour & Information Technology, 37(4), 311–319.

Chapter 10

Administering Course Revisions

This chapter is primarily for faculty, administrators, and designers who may be involved in identifying needs for course updates, and planning those updates. Support staff may find this chapter useful for understanding the course update process. This chapter covers: • • • • • • • • •

Planning course updates. Estimating course revision effort. Course and faculty teaching evaluations. Curriculum review. Software updates, new technologies, and pedagogy advancements. Administering asynchronous discussion revisions. Internal and external reviews. Outcome analysis. Program evaluation.

A course is like a living organism that needs care, feeding, and periodic checkups to stay healthy. Over time, the quality of online courses and programs is largely determined by the effectiveness of the processes that are used to update the courses and curricula. Careful planning of course updates can address issues identified in previous course offerings, and improve the coordination of courses within a curriculum.

Planning Course Updates The foundations of course update planning include faculty reviews, student evaluations, and, occasionally, accreditors’ and others’ feedback. Best practice is for faculty to review the overall curricula and learning outcomes at least once per year, and to work with designers to identify ways to use advancements in online technologies and pedagogy to teach the material most effectively (Herman, 2012). We have learned that we need to update our courses at least once per year, for several reasons: to incorporate student feedback and our own

Administering Course Revisions  179

experience teaching the course; to remain current with evolving subject matter; to maintain rigor and academic integrity; and to keep up with the latest online pedagogy (Chao, Saj, & Tessier, 2006). Courses can contain different types of asynchronous elements, such as modules, quizzes, exams, and discussions. Element types have different life expectancies, and their revision should be administered accordingly. For example, a final exam may need to be revised each time a course is offered, to maintain academic integrity, while an interactive graphic on a stable topic may require no revisions for many years. Some course elements, such as modules and assignments, are needed for the first offering of a course, while others, such as large assessment question banks, are best developed incrementally through revisions. Periodic updates of the course assessments and assignments help maintain academic integrity and rigor, by making prior term submissions no longer useful. Faculty and administrators need to be aware that students may gain access to other students’ work in many ways. For example, we have identified students submitting term projects done by other students who had posted them online as part of their digital portfolio. Because writing entirely new assessments and assignments is time-consuming for faculty, we add new questions to question sets each time the course runs, and use the LMS features that support random selection of assessment and assignment questions, so that students receive different assignment and assessment problems. We have found that to maintain academic integrity, it is enough to have three questions in each question set (for more on this, see Chapter 3). To ensure uniform coverage of the subject matter there needs to be a separate question set for each topic. Faculty should analyze the statistics for each of the questions and adjust their difficulty, so that the difficulty of the assessments is not significantly affected by the random selection of questions. Faculty should also retire questions when they are no longer relevant. Courses may also need to be revised to take advantage of improvements in online pedagogy. Pedagogy that is dependent on technology can evolve rapidly (Johnson, Aragon, Shaik, & Palma-Rivas, 2000). For example, the advent of handheld devices, such as tablets and smartphones, drove the development of new tools that are compatible with and take advantage of the capabilities of these devices. We make extensive use of formative assessments and other engaging assessment elements that can be used on these devices.We code them in JavaScript, so that any modern browser or device can use them. The additional investment in development time and manpower needed to create the JavaScript items and other technologies is abundantly repaid by improved student learning and, in turn, a reduced need for faculty to help struggling students. One of the ways to make updating more efficient is to design courses in a modular fashion, so that future changes will only need to be made in as few modules as possible.This can be achieved by organizing the modules so that each topic is covered in only one module. If new subject matter is just emerging, it can be introduced initially as an advanced topic. When it becomes mainstream,

180  Administering Course Revisions

the subject matter can be integrated into the main content. For example, we created a webpage on Spark (a big data-management system) as a new development in our advanced database management course when Spark was emerging, and moved it to the main course lecture pages when Spark went mainstream. We also use several sources for identifying the areas in our courses which would benefit the most from updates. For example, the most useful source in graduate courses is student opinion about topics they would like to see added or dropped. We obtain this guidance primarily from conversations with our students, feedback in the student evaluations, and online surveys. We strive to learn from our students what they think should be covered, how it should be covered, and what is no longer relevant. We also carefully analyze our own experience in teaching the course, as well as reports from students and facilitators, to check the student workload for each module and assignment, and identify areas where students seem to need additional help, for example, through interactive learning objects or short topical videos. We conduct course update planning meetings after each online course has run. These meetings should include the faculty, the program coordinator, and the department chair. They consider student evaluations, survey results, course update logs, assessment and assignment grade statistics, and any other reviews by faculty or accreditors. These meetings produce a course update plan with what is to be changed, who is going to do it, and how much effort it will take.

Jim Frey, learning design director Do not trust your future self to remember the details of decisions you made during the design process. The update may not come for several months or more, and you will probably have millions of unrelated issues on your mind in between. In addition, it is entirely possible that someone else will be the one performing a future update. So, you should create a course update checklist as part of the design process. This will require minimal additional effort and will make the update process simple and efficient, while reducing the risk that you will forget something and leave outdated information in the course the next time it runs.

Estimating Course Revision Effort It is important to estimate the level of effort required for revisions before they begin, so that the scope can be adjusted to ensure that the work is completed on time.These estimates are also useful for budgeting effort and compensation.Topdown or bottom-up methods can be used to forecast this effort; best practice is to use both methods and reconcile the results.

Administering Course Revisions  181

A top-down method is similar to creating an outline before starting to write. One simple example is to estimate the percentage of the content that needs to be revised, and multiply this by the level of effort that was required to develop the legacy content. This works because the level of effort required for many kinds of course revisions tends to scale with the level of effort that was required to produce the legacy content originally. Another top-down method is based on the level of effort for a prior update. For example, if the prior update had about the same scope as the current update, and the same faculty are doing the update, then the level of effort will be about the same. However, neither of these methods accounts well for unanticipated changes to legacy content, such as a criminal law course in a state that just legalized cannabis. A bottom-up method is similar to writing without an outline. One such approach is to estimate the level of effort needed to accomplish revision tasks, such as developing new quiz questions and the course content to teach them, and then aggregate those detailed efforts. For example, if the course update plan includes analyzing and updating the weekly quizzes, then the effort for this can be estimated by multiplying the effort per question by the number of questions. Top-down and bottom-up methods have different weaknesses. Top-down methods suffer from lack of detail, so they tend to underestimate the effort to integrate small changes, while at the same time they can overestimate the effort needed to make changes that don’t require significant integration. Bottom-up methods do well on the details, but do not incorporate information such as experience with similar types of changes in other courses. Accurate estimation depends upon the accuracy and detail of the update plan. It is also useful to estimate the percentage of course revision expected, per year. The percentage can vary widely, based primarily on the stability of the domain of the content. For example, a well-developed mathematics course may need only 10–20 percent revision per year, to incorporate new material and maintain academic integrity, while political science courses on current events may require up to 100 percent revision each iteration. These annual revision estimates are useful for budgeting and staff resource planning. One common revision is to adapt a course to a new edition of a textbook. Sometimes the new edition will have a different topic ordering, which can require reordering of online content. When content is reordered, care must be taken to align assessments, assignments, and other learning activities. Often reordering requires updating lectures and other content, so that earlier learning activities prepare students well for later activities. If the new text edition involves reordering of content, administrators should expect that faculty will need to modify some of the legacy content. Less commonly, a textbook may be replaced. The most common reason to choose a new textbook is because the new text covers the material in a better way, or the material is more up to date. Changing the main text for a course usually requires reorganizing the course so that it follows the organization of the

182  Administering Course Revisions

new text; modules will need to be updated, assignments adjusted, assessments updated, and discussions revised. We have found that changing a main textbook usually requires about half the level of effort that was originally required to develop the legacy content. Often revisions to a course require changes to material that is not new or modified, to maintain the alignment within modules. For example, if a topic in a module is replaced with a new topic, or the topic material is updated, then changes are likely to be required in the module assignments and assessments, as well as comprehensive assessments, such as a final exam. Common revisions to a course include adding material, deleting material, moving material between modules, and changing the order of modules. When course revisions involve moving material between modules, substantial effort may be required to restore alignment. These changes require analysis of the alignment of elements within modules, and adjustments, so administrators should expect and support these tasks. Net content additions or deletions may change student workload, so its analysis and adjustment should also be taken into consideration. Changing the module order is usually done to improve the overall learning experience, but it may disrupt the learning sequence, so again, administrators should support learning sequence analysis and any necessary adjustments. Design that minimizes inter-module dependency simplifies module order revisions, but it can be difficult or impossible to eliminate some learning sequence dependencies. Administrators should be aware of this, and be prepared if portions of reordered modules need to be moved again.

Course and Faculty Teaching Evaluations At the postsecondary level, each time an online course is offered, students have the opportunity to evaluate it. Student evaluation survey questions usually request responses as both text and selection on a scale of one-to-five. These evaluations can provide valuable insight on students’ perceptions of the quality of courses and ways to improve them. Faculty should look for patterns and common themes in student evaluations and use them in course update planning (Berk, 2013). Often there is general agreement among the students regarding how well some aspects of the course work for them (Marsh, 1982). In the numeric evaluations, we use a low standard deviation as an indicator to identify a pattern of agreement. We manually aggregate text feedback and look for students saying similar things. We give more credence to the substance of student comments when many students have similar feedback. It takes some skill to interpret student evaluation results, because they often respond inconsistently to the same aspect of the course; some students may like a particular module, assignment, or faculty, while others in the same course may

Administering Course Revisions  183

dislike them. The difficulty in understanding students’ likes and dislikes lies in understanding exactly why they like or dislike course elements, and how that insight can be used to improve the course. For example, different students may like and dislike synchronous review sessions. This may be due to learning and interaction preferences of particular students, so it should be stated that those sessions are optional. Our research has informed our interpretation of the evaluations. For example, we have found there is a high correlation between facilitator and faculty evaluations, so usually poor or mediocre faculty evaluations can be attributed to underperforming facilitators. Similarly, our research has shown that there is a high correlation between satisfaction with courses and the amount of time that faculty spend interacting synchronously with their students (Eom & Ashill, 2016). Given that, our faculty experimented with increasing the frequency of optional interactive synchronous presentations, but they learned from the endof-course survey that many students felt that they could have learned most of the material faster asynchronously. The faculty eventually found that the best practice was offering frequent, optional online office hours during which students could ask questions, as well as having synchronous presentations to cover difficult topics. It is often difficult to compare online and on-campus student evaluations, even for the same course. They can be very different, even for the same faculty teaching the same course, with identical assessments and assignments. Some faculty have widely differing online and face-to-face evaluations, while for other faculty they are almost identical. Initially, one might think that the primary reason for this difference is that our online courses are seven weeks long whereas all of our face-to-face courses are 14 weeks long, so students are under more pressure with the shorter timeframe. However, our analysis of student comments reveals that in addition to some students feeling the time pressure, they also feel that some faculty have difficulty communicating to their online students that they care about their success in the course, even though they do this well face to face (see Chapter 2). When the student evaluations identify a problem with a course or faculty, our program directors, department chairs, and other academic administrators may examine the archived course to understand what happened and how to improve the course. Sometimes, a meeting between the faculty and chair needs to be arranged to discuss the course outcome. For example, if the student evaluations include multiple complaints that faculty was not available synchronously, did not log into the course frequently, or did not reply to student posts, this can be verified in the archived course and appropriate administrative actions may be taken. Student evaluations are useful in comparing courses, because the questions are general and always the same. However, this lack of specificity limits the utility of student evaluations in assessing the satisfaction and effectiveness of particular

184  Administering Course Revisions

learning objects, activities, and other elements specific to a course.When faculty do not understand how well particular course elements work for the students, they can create surveys that ask students to provide feedback on just those elements. This is helpful when a course is first offered or new elements are introduced (Anderson, Cain, & Bird, 2005).

Curriculum Review Developing and maintaining high-quality curricula is difficult, particularly in rapidly changing disciplines such as computer science, information technology, or biochemistry. When the subject matter changes, courses must be updated frequently, and the curricula in turn also need to be revised. Reviews of curricula should be scheduled at least once per year for rapidly changing subject areas. Best practice is for the review to be conducted at a meeting of faculty and domain experts who systematically evaluate the curricula and courses, and recommend changes. The quality of the curricula can also be assessed by surveying and interviewing graduates.These surveys and interviews tell faculty how well programs meet student needs (Spiel, Schober, & Reimann, 2006). They may identify updates to be made to courses, courses that need to be created, and courses that are no longer useful. Faculty can also conduct formal outcome analysis to evaluate how the curricula helped students succeed in their careers. Faculty also review all of the courses, degrees, concentrations, and certificates in the program. Each review addresses: • • • • • • • • • •

The quality of the design and the appropriateness of the educational technology choices. How completely and well each course covers its subject material, including the latest developments in the field. How well the material is aligned in the lectures, assessments, assignments, and other learning activities. The quality and coverage of assessments. The effectiveness of academic integrity measures. Student evaluations. Peer reviews by specialists. External evaluations, such as from accrediting bodies. Contact hours, as measured in archived courses. Student workload.

The decisions made during the reviews are documented and addressed in the course and curriculum update plans. Progress on the decisions is reviewed in the subsequent curriculum review.

Administering Course Revisions  185

Software Updates, New Technologies, and Pedagogy Advancements Updates in the LMS are usually upward compatible, requiring no course changes, though such changes may be desirable to take advantage of the new features. When a new kind of software, such as a different video conferencing system, is added to a course, significant course updates may be needed. Some new activities may become possible, such as virtual class meetings, role-plays, debates, small group discussions, and breakout sessions. If new software replaces legacy software, it may lack some of the previous features or have new features, and this may also require course revisions. Courses should often be updated when students gain access to new technology (Stoltenkamp, Kies, & Njenga, 2007). For example, when smart phones became ubiquitous, students could easily upload pictures and videos, allowing them to share examples and discuss them with the class. When emerging technologies such as augmented and virtual reality become widely available, they will surely inspire modifications to courses to take advantage of their capabilities. Course updates may also be initiated when faculty devise better ways to teach. Many of the reasons for updates are pedagogic improvements, such as updated examples and assignments, improved student activities, more effective engagement with faculty (Kauffman, 2015), and strengthened measures to enforce academic integrity.

Administering Asynchronous Discussion Revisions In traditional course designs, the most important course-specific discussions are graded discussions associated with modules or topics. More discussion questions can be added in revisions, and less productive questions revised or replaced. We have found it helpful to have a number of questions in reserve, hidden from students, for facilitators or faculty to reveal if additional questions are needed to sustain discussions. Revisions of course-specific discussions are the responsibility of faculty, who may create and update discussion forums and questions before a course opens, and create new forums and discussions as needed while courses run (in programs with more resources, the creation and updating of forums in the LMS may be delegated to designers).

Internal and External Reviews The primary purpose of internal reviews is to create and refine courses and curricula. Common internal reviewers are faculty who develop and shape the course and its content, and faculty and facilitators who teach the course. Ideally, faculty should review courses at each stage of their definition, development, or

186  Administering Course Revisions

revision (Fish & Wickersham, 2009). This can be as informal as looking over the course and making suggestions; this helps identify errors and typos. It is usually helpful to have someone who is not skilled in the subject matter provide feedback on typos and unclear areas, but this review is less helpful for advanced and esoteric courses, where subject matter experts are needed to provide feedback about the course content. Faculty who are skilled at teaching online may also provide feedback on the pedagogy. Faculty who know the curriculum and intended audience can also provide feedback on prerequisites, course level, and workload. Occasionally courses or programs may be reviewed by editors for clarity and style, and by external faculty and experts to verify that courses and curricula meet academic quality standards.These reviews may be conducted for accreditation. Accreditor reviewers are typically senior faculty from other institutions; we have found their feedback very helpful for improving our courses. For example, CAHIIM (the Commission on Accreditation for Health Informatics and Information Management Education) reviewed our health informatics courses and curricula and made recommendations. After we addressed their recommendations, they approved our program. Similarly, the Global Accreditation Center for Project Management Education Programs reviewed our project management curriculum, and their feedback both affirmed the design and provided guidance on making it better.

Outcome Analysis Outcome analyses are activities that assess how well educational programs meet students’ needs, program learning objectives, employer needs, and even the needs of society. Outcome analysis evaluates not only how well students master the material, but also how their lives and careers have been helped by what they learned. The results of outcome assessments may be quantitative, such as the average amount that completing a program increased the earnings of its graduates, or qualitative, for example, how happy a student is to have earned a degree. Outcome analysis is most commonly performed at the program level, but it can also be performed at the course level. Program outcome analysis can be performed in part by surveying and tracking graduates’ career progress. Program-level outcome analysis can be performed by assessing student performance in capstone courses. Our faculty review capstone term projects to assess how well our students have mastered the subject matter at the end of their programs. These capstone projects are research projects, defined and planned by students with the support of their faculty. The research projects include a formal research paper and presentation to their classmates and faculty, which is recorded.Typical research projects require that students have mastered a wide range of skills, and this allows our faculty to assess how well our programs are teaching those skills.

Administering Course Revisions  187

Program Evaluation The goals of program evaluation are to guide strategic decisions, such as whether a given program should be updated, merged with other programs, restructured, or shut down. Program evaluation should analyze the fit of the program in its current academic markets, competition in those markets, current and projected enrollments, the role of the program in supporting other programs, and the future of the program. Program evaluation is conducted periodically or when there is reason to think that an evaluation is needed. It is the strategic complement to curriculum assessment and update, which is more tactical. Program evaluation typically assesses: • • • • • • • • • •

The environment within which the program operates. The competitive landscape. Evolving market needs. Opportunities to expand into contiguous, emerging, or other markets. Analysis of the enrollment history. Enrollment projections. Curriculum evolution. Rankings. Financial analysis, including the revenue and costs for prior terms and financial projections. Academic evolution of the program.

The primary objective of this analysis is to understand how the program has evolved and can evolve over time. Program evaluation culminates in planning the future of the program. Plans may include additional investment in the program, restructuring it, and administrative or marketing changes. Thoughtful planning can often reinvigorate programs with new directions of growth by repositioning them. Program planning requires awareness of the relationship of the program to other programs, particularly competing ones, and it should be conducted within the framework of the institution’s digital strategy.

Summary Checklist Plan course updates. ☑ ☑ Estimate the course revision effort. ☑ Update accordingly if course or teaching evaluations indicate opportunities for improvement. ☑ Update courses if curriculum review identifies needed changes. ☑ Update courses if advances in technology or pedagogy justify changes. ☑ Update asynchronous discussions.

188  Administering Course Revisions

Update courses if internal or external reviews identify needed changes. ☑ ☑ Update courses if outcome analysis identifies deficiencies or opportunities. ☑ Update courses if program evaluation identifies ways that they should be changed to improve the program.

References Anderson, H. M., Cain, J., & Bird, E. (2005). Online student course evaluations: review of literature and a pilot study. American Journal of Pharmaceutical Education, 69(1), 34–43. Berk, R. A. (2013). Face-to-face versus online course evaluations: a “consumer’s guide” to seven strategies. MERLOT Journal of Online Learning and Teaching, 9(1), 140–148. Chao, T., Saj, T., & Tessier, F. (2006). Establishing a quality review for online courses. EDUCAUSE Review, 29(3), 32–40. Eom, S. B., & Ashill, N. (2016).The determinants of students’ perceived learning outcomes and satisfaction in university online education: an update. Decision Sciences Journal of Innovative Education, 14(2), 185–215. Fish, W. W., & Wickersham, L. E. (2009). Best practices for online instructors: reminders. The Quarterly Review of Distance Education, 10(3), 279–284. Herman, J. H. (2012). Faculty development programs: the frequency and variety of professional development programs available to online instructors. Journal of Asynchronous Learning Networks, 16(5), 87–106. Johnson, S. D., Aragon, S. R., Shaik, N., & Palma-Rivas, N. (2000). Comparative analysis of learner satisfaction and learning outcomes in online and face-to-face learning environments. Journal of Interactive Learning Research, 11(1), 29–49. Kauffman, H. (2015). A review of predictive factors of student success in and satisfaction with online learning. Research in Learning Technology, 23, 1–13. Marsh, H. W. (1982). Validity of students’ evaluations of college teaching: a multitrait– multimethod analysis. Journal of Educational Psychology, 74(2), 264–279. Spiel, C., Schober, B., & Reimann, R. (2006). Evaluation of curricula in higher education: challenges for evaluators. Evaluation Review, 30(4), 430–450. Stoltenkamp, J., Kies, C., & Njenga, J. (2007). Institutionalising the elearning division at the University of the Western Cape (UWC): lessons learnt. International Journal of Education and Development using Information and Communication Technology, 3(4), 143–152.

Glossary

academic integrity  Policies and procedures that aim to assure that the grades and credit that students receive for their coursework accurately represent their learning in the program. accessibility  The extent to which a course can be used by anyone, regardless of disability or impairment. See universal design. administrators  People directly involved in supervising programs, such as program administrators, program directors, department chairs, and, possibly, deans. Many of these administrators will also be faculty. At primary and secondary schools, administrators might include head teachers, department chairs, headmasters, and principals. We do not include higher-level governance and administration such as chancellors, provosts, presidents, superintendents, or school boards. Neither do we include staff administration, such as various ranks of instructional designers and support staff. affective distance  The psychological term for how closely people feel that they are emotionally tied. As affective distance decreases, trust and successful communication increase. alignment  Coordination of learning activities by topic, so that students are prepared for them and for assessments on the topic. Alignment is different than sequencing, because alignment is the coordination of activities within a topic, while sequencing is the coordination of topic coverage based on student readiness. See sequencing. alt attribute  An attribute of all HTML elements used to specify text that is to be rendered when the element cannot be used in its primary form. For example, a screen reader used by a blind student is unable to describe a picture, but can read the alt attribute that has been assigned to it. assessment  Something that helps faculty determine how well individual students have met the course learning objectives. Common assessments include quizzes, tests, exams, graded assignments, and graded discussions. assessment, formative  See formative assessment. assessment, summative  See summative assessment.

190 Glossary

asynchronous  Activities that are not performed at the same time—in contrast to synchronous interaction. asynchronous interaction  Communication that is not in real time. Examples include email, text messages, and previously recorded videos. banner blindness  When a user learns to ignore large images (banners) at the top and sides of a screen because they are usually ads. This causes them not to pay attention to similarly placed images that are not ads. blended/hybrid  Courses that include both online and face-to-face components. calibrate questions  The process of analyzing the statistics of student performance on assessment questions. Overly difficult questions may be modified so that the scores are in a desired range, and overly easy assessments may be made more difficult. carousel  A scheduling model in which courses are offered in a rotating sequence. Students take courses when they are offered, just as one might select a horse at an amusement park carousel. CSS  Cascading Style Sheet. Instructions for browsers that control how webpages are rendered, including the look and design. contact hours  In a traditional face-to-face course, the number of hours during the term in which students and faculty meet. Online, contact hours are measured as a combination of synchronous contact with faculty and asynchronous interaction with content created by faculty. course/program launch  When a course or program is opened for use by students. course map  A list of the modules of a course and their learning objectives, topics, learning objects, assessments, and assignments. curriculum  The courses that are available for students to take in a program. A curriculum may support multiple courses of study. customer relationship management (CRM) system  A software and database system that supports communication with current and future customers, or in academia, students. CRM functions include recording contact information, automatically tracking emails, phone calls, and other digital communications, and keeping records of all contacts attempted. development  The process by which courses are produced for students to take. Development usually precedes course launch, though some development may be done while a course is running. development models  Development models specify whether and how instructional designers and other staff are involved in course development, how much time is planned for developing a course, whether and how faculty are compensated, course development approval policies, and other common development policies and procedures. digital strategy  The policies and procedures for how an institution uses digital technology in education, and how it is funded and managed.

Glossary 191

editor  A person who makes the text and graphics of courses easier to understand by identifying changes for faculty. They are usually expert in English writing but not in the subject matter. engagement  A property of successful learning activities which encourages students to reflect on, work with, or feel connected to the subject matter. face-to-face interaction  People in the same room see and speak with each other, in real time, without using technology. facilitators  People who have mastered the subject matter well enough to grade assignments and help students learn the material. facilitator, lead  An experienced facilitator who has extra responsibilities that span the entire course, such as checking grading uniformity across groups. facilitator at large  A senior facilitator, often an adjunct faculty member, who is added to a high-enrollment course to assist faculty and group facilitators by helping them with high-workload students and difficult situations, and by conducting synchronous sessions as directed by the faculty. faculty  The people who have mastered the subject matter, create the course content, and teach the courses. faculty and student support staff  People who support both faculty and students while courses are running. They may also do this before courses launch and after they conclude. faculty coordinators  Faculty who are assigned administrative responsibility for courses and programs. faculty evaluations  The formal process for evaluating faculty, particularly faculty teaching. Faculty evaluations are commonly based on student evaluation, peer reviews, publications, and service. faculty forum  A synchronous virtual meeting room that is accessible to all faculty, facilitators, and support staff, but not to students. The faculty forum is used for kickoff and other meetings while the course is running. See kickoff meetings. fine-grained courses  Courses that cover fewer topics than a traditional 14-week course. flipped courses  Courses in which students study the course material using digital technology to prepare for class sessions. They spend the class time in discussions, group problem solving, group lab exercises, and other activities that work best in a face-to-face classroom, rather than listening to lectures. formative assessment/evaluation  An activity used to help students evaluate their mastery of material while they are learning, which does not count toward their grade. It should be given directly after presentation of material and provide immediate feedback, to correct misunderstandings and misconceptions. Examples include faculty asking students a question in class and providing feedback, or practice tests. grade begging  Asking for extra credit to earn a higher grade, disputing scores for graded items, and other activities that students engage in to try to improve their grades after grades have been awarded.

192 Glossary

high-touch  Courses or programs in which students have many opportunities to interact with their faculty. HTML  HyperText Markup Language. The standard format used to represent the content of pages in a web browser. It may include technologies such as CSS and JavaScript. incomplete  A grade assigned to students to allow them and their faculty additional time to complete the course. Incompletes may be offered by faculty due to student illness or other good reasons, provided that most of the coursework has been completed. instructional designer  A person who helps organize, develop, and implement instructional materials to make them more effective, usable, accessible, or available online. instructional models  The ways courses are organized and taught. Instructional models may specify how faculty interact with students and whether courses are primarily based on synchronous or asynchronous modes. item analysis  Calculating assessment quality and difficulty by examining the statistics of student scores on individual graded items and how they correlate with other measures of their performance. This can often be done within the LMS. JavaScript  A common programming language that runs in modern browsers. It is often used for animations and responses to user input. kickoff meeting  A meeting conducted early in an activity to establish the organizational framework, goals, and general scope of the activity. Kickoff meetings may be conducted at the beginning of course development and course launch. See faculty forum. landing pages  Webpages linked from online marketing collateral. Landing pages are usually part of the program marketing website. See marketing collateral. LCMS  See Learning Content Management System. lead facilitator  See facilitator, lead. learning activities  Things that students do to learn the material. These may include reading, watching videos, interacting with multimedia objects, engaging in synchronous or asynchronous discussions and study groups with faculty and classmates, and completing assignments, assessments, and projects. Learning Content Management System (LCMS)  A repository for files linked from courses, organized to support course development and content sharing. Learning Management System (LMS)  A computer software server system that hosts the websites for online courses, including lectures, assessments, and assignments. An LMS may host multimedia, video, or other services, or they may be hosted separately and linked from the LMS.

Glossary 193

Live Classroom/Office  “Live Classroom” is our term for a synchronous activity usually conducted by the faculty. “Live Office” is our term for a synchronous activity connected with a group within a course, usually by a facilitator. Live Classrooms or Live Offices can also be conducted between faculty and a single student. LMS  See Learning Management System. marketing collateral  Information and material used to communicate about a program. This includes ads that appear when certain search terms are used, webpages that may be linked from ads or native search, program and course descriptions, search engine optimization materials, printed materials, “swag,” application forms, and videos. See search engine optimization. marketing channel  A communication medium used to establish initial contact with leads. Common channels include the institution’s website, purchased search engine terms, native search, direct email, “snail mail,” and referrals. See native search. market differentiation  Specializations that support related programs within a broad market without having them compete head to head. Markets may be differentiated based on product cost, quality, target audience, geography, culture, and other ways. An example of market differentiation is programs that address the needs of students who wish to become musical performers and students who wish to teach music. mentoring  A one-on-one process in which a trusted person with more experience or knowledge in some area provides guidance to someone who is learning that area. micro-master’s  A master’s program in which the introductory courses are inexpensive and online. They reduce the cost of the master’s program and allow students to determine how well the program meets their needs before applying to the program or spending a significant amount of money. Admissions decisions may be based on student performance in the introductory courses. module  A coordinated collection of learning objects that students study together, commonly one week of studies. Although it is commonly used interchangeably with week, the generic term allows the same course to be used in different length terms. Module 0  A review module in an asynchronous online course that primes students for the course, but does not count toward students’ grades. This material may come from prerequisite courses, and, if possible, should be made available before the course launches. MOOC  (Massive Open Online Course). An online course designed to have unlimited enrollments. MOOCish courses  Courses that incorporate interaction styles and technology that were pioneered in MOOCs.

194 Glossary

mouseover  An active area of text in a website that responds to placing a mouse or cursor over it. For example, a term in an online course may respond to a mouseover with the word’s definition. multimedia [object/media]  Technologies that present one or more visual, auditory, tactile, and other media. Multimedia may be interactive, allowing students to modify variables, or presenting questions and accepting responses from students. native search  The results from a search engine when a term is entered. Not to be confused with ads which may be displayed when a search term is entered. non-enroll survey  A survey of prospective students who expressed interest in a program, but did not subsequently enroll. on-campus residency requirement  Some online programs require that students complete a portion of each course or of the program on campus, to improve academic integrity and provide students with opportunities to do labs and face-to-face group learning activities. online  Computer-mediated communication, usually accomplished through the World Wide Web. Online education usually refers to courses implemented with an LMS and offered over the web. online technologies  The many technologies used in online courses, including HTML, JavaScript, Java, streaming video, and LMSs. outcome analysis  Activities that assess how well educational programs meet students’ needs, program learning objectives, and employer needs. pipeline  A database containing prospective and enrolled students, with stages that represent their progression into and through the program. Pipelines can be used for marketing, recruiting, and admissions. See customer relationship management (CRM) system. proctored assessments   Summative evaluation in which students’ activities are monitored, either by faculty or a proctor. In-person proctoring takes the form of having a faculty or approved proctor in the same room with students as they complete the assessment. Online proctoring may be implemented using webcams, lockdown browsers, or synchronous presentations to faculty. question validity  The extent to which a question measures how well students have met the learning objectives. remote learning  See telepresence. sandbox  An environment in which people can play and learn without risk or harming the environment. Examples include course websites that are not normally accessible by students, but can be used by faculty and staff who are experimenting with new technologies and teaching techniques. search engine optimization (SEO)  The design of text for websites to maximize the probability that the website will be ranked highly by a search engine, which helps attract people to the website.

Glossary 195

sequencing  The ordering of topics in a course based on student readiness, so that students prepare for more advanced material by first studying foundational material. See alignment. start  Students who have enrolled in their first course. stone-soup development  A development process which begins with rudimentary content that motivates faculty to improve it. This is used to gently persuade busy faculty to create and improve course elements. [Named after the popular children’s story.] student evaluations  Questionnaires completed by students evaluating the course, faculty, and technologies. These are useful for evaluating course quality and planning course updates. student information system (SIS)  The computer system that manages student information, including their contact information, courses for which they registered, and their grades. study group  Students who meet in face-to-face or synchronous sessions to help each other learn. Study groups may be facilitated by teaching staff. study room  A virtual meeting room that supports study groups. A study room may include synchronous multimedia as well as asynchronous activities such as discussions. A study room may be accessible to all students in a course, to the students in a facilitated group, or only to students with an affinity, such as time zone or interest. See virtual room. summative assessment  An activity used to help students assess their mastery of material after they have learned it, and which counts toward their grade. Common examples include quizzes, tests, and exams. synchronous  Activities that are performed at the same time—in contrast to asynchronous activities. synchronous interaction  People in different locations see and speak with each other, in real time, using technology. synchronous sessions  Real-time meetings implemented using computermediated communications. task manager  A software application that records and tracks the progress of tasks. Task-manager systems support task priorities. They also allow them to be assigned to those responsible for their resolution. Unlike a ticketing system, they support templates of tasks and subtasks that can be repeated. See ticketing system. telepresence  Situations in which students or faculty participate in a classroom setting, using technology, even though they are not physically present in the classroom. Also commonly known as remote learning. ticketing system  A software application that records and tracks the progress of identified course or technical issues. Ticketing systems support issue priorities. They also allow issues to be assigned to those responsible for their resolution. See task manager.

196 Glossary

transition course(s)  Courses designed to prepare students without adequate preparation for advanced study, commonly used to prepare graduate students for study in an area that they have not studied as undergraduates. universal design  Course and software design that helps all students, including those with disabilities. See accessibility. universal element  A component that applies to every course in a program, such as technical support instructions or academic integrity policies. virtual labs  Class assignments that are implemented using technologies that students could not feasibly implement themselves or run on their own computers, such as simulations, websites with licensed software, or large data sets. virtual room  A space for synchronous meetings, implemented using technology such as Adobe Connect, Blackboard Collaborate, or Zoom. Virtual rooms may be used for synchronous class sessions led by the instructor, oneon-one sessions between the instructor and students, student presentations, study groups, or other situations where synchronous communication is best. See study room. wiki  A website that supports collaborative editing of its content by its users. Within a program, wikis are useful for describing tasks and procedures, to retain institutional memory.

Index

Page numbers in italics refer to figures, those in bold indicate tables. academic background: transition course decisions 69, 70, 70 academic freedom 75, 141 academic integrity 3, 14, 24, 34–38, 59, 61, 74, 76, 78, 81, 92, 99, 106, 110, 160; compromised 176; course revision 179, 181, 184, 185; definition 189; improvement 161–163; metrics 79 accessibility 12, 88, 124, 131, 189; see also universal design accounting course 7, 138 adjunct faculty 21, 30, 40, 168, 191 administrators 1–21, 51, 62, 63, 66, 72, 77, 80, 81, 84, 101, 114, 134, 145, 189; course revisions 178; unexpected developments 166 admissions processing 67–71 Adobe Connect 126, 136, 196 ads 35, 47, 48, 190, 193, 194 advanced course 61–62 advanced course design 4, 114–144; avoiding multiple copies of content 126–127; building for universal design 124–126; copy-editing 141; dangers of placeholder content 123; deciding which technologies to use 127–133; development kickoff meeting 12; development milestones 120; development schedule of seven-week course 120; final preparation for launch 143; hiring designers, support staff, videographers 115–119; implementing content with and

without designers 120; integrating multimedia content, simulators, and virtual labs 137–139; managing production tasks 139–141; motivating faculty 121–124; performing faculty QA 143; planning course development 119–121; planning, production, launch 120; pre-launch tasks; designer QA 142; LMS (to prepare course for running) 142; project management 142; requiring faculty input 142; synchronous sessions 133–136; telepresence 136–137 advanced student 102, 103, 170–171 advanced topic 87, 90, 171, 179 affective distance 88, 146, 189 algorithms 51, 63, 65, 138 alignment 103–104; definition 103, 189 Alperstein, N. 14 alt attribute 42, 124, 149, 189 Americans with Disabilities Act (ADA) 11, 41 animation and animators 2, 12, 58, 78, 87, 100, 101, 121, 124, 132, 138, 192 announcements 153–154 area coordinator 168 artificial intelligence 61, 161 assessments: creation 94–95; definition 189; questions 97, 99, 110; randomization 36; role in courses 95–100 assessments and assignments 74, 179, 182; grade statistics 180; quality and coverage 184

198 Index asynchronous: components 7–8, 77; definition 190; discussion revisions 185; interaction 8, 72–73, 85, 190; learning 23, 42, 62, 91, 101; teaching methods 58–60 audio 12, 22, 38, 102, 119, 125, 130, 133, 136, 137, 173, 175; conferencing 129 automatic grading 8, 23, 73, 77, 94, 95, 97, 110 award rubrics (course quality) 85–86 banner blindness 123, 190 Bernays, E. 107 Blackboard Catalyst Awards 86 Blackboard Collaborate 136, 196 blended courses 5, 15, 24, 26, 22, 44, 58, 190; advantages of online technology 8–9 blindness/visual impairment 41, 42, 124 Boston University 8, 19, 46, 55, 69, 73, 74, 130, 164; Computer Security program 81; Department of Information Services and Technology 62; Office of Distance Education (ODE) 20, 57, 86, 90, 99, 112, 117, 121, 122, 126, 129, 131, 133, 139, 141, 146, 150, 151, 154, 156, 157, 160, 164; School of Social Work 60 Braille 42 browser 34, 118, 132, 149, 167, 179, 190, 192, 194 calibrate questions 95, 110, 190 California State University: Chico’s Rubric for Online Instruction 86 camera 7, 107, 128, 136, 162 capstone course 38, 79, 186 career development 27, 118, 164 carousel 54, 190 Cascading Style Sheet (CSS) 34, 97, 110, 125, 129–130, 190, 192 Cavanagh, T. 10, 52, 67, 106 Cloud 17, 138 color 89 color-blindness 125 Commission on Accreditation for Health Informatics (CAHIIM) 186 communication 8, 15, 116, 189 Communication Access Realtime Translation (CART) 42

community beyond classroom 164 competition 81 competitive landscape 187 computer science 69, 184 computer security course 138 conferencing software 133–134 consistency 56, 115, 125, 126, 150, 182 contact hours 28, 72–73, 74, 76, 79, 184, 190 content development 105–106 context 13, 14, 33, 47, 51, 87, 90, 96, 104, 106, 137 conversation 9, 87, 117, 128, 140, 159, 180 copy-editing 58, 120, 141, 142 cost efficiency 16–17, 43, 44, 47, 48, 55, 56, 57, 60–61, 73, 75–78, 131, 132, 161–162 course content: errors or omissions 174–175; misalignment 175 course design and development 3–4, 84–113; aligning content by topic 103–104; assessments (creation/role) 94–100; content (development and implementation) 105–106; course maps 91–94; delayed development 108–109; design standards and award rubrics 85–86; goals and learning objectives 90–91; earning activities (planning) 100–102; meaningful interaction 86–90; plan before writing 94; pre-launch tasks 107–108; starting new course 85; students with varied preparation 102–103; student workload 104–105; updates 109–112; video-making 106–107 course development 3–4, 5, 23, 24, 25, 84, 110 course development model 55–57 course development process 12, 77, 99 course goals (defining) 90–91 course improvement 74, 97, 121, 122, 123, 146, 170; student evaluation surveys 182–184; see also course revision course innovation 75, 81 course launch 7, 65, 147, 148, 149, 155, 169, 192, 193; definition 190; kickoff meeting 150; “program launch” 47, 190 course map 4, 88, 91–94, 104, 108, 120, 190

Index 199 course material: unproductive or irrelevant 88–89, 92 course navigation 76 course quality 38–39 course redevelopment 123–124 course release 27, 28, 108 course revision 4, 178–188; asynchronous discussion revisions 185; course and faculty teaching evaluations 182–184; curriculum review 184; estimating course revision effort 180–182; internal and external reviews 185–186; new technologies 185; outcome analysis 186; pedagogy advancements 185; planning course updates 178–180; program evaluation 187; software updates 185 course scheduling model 52–55 course standards 74–75; defined in terms of goals 75, 76; flexibility 75, 76 course underway 4, 145–165; building community beyond classroom 164; calling students 159; course launch kickoff meeting 150; email and announcements 153–154; faculty, facilitator, and support staff weekly meetings 151; learning community 156; orientation course 152; outreach 158; proctored exams 161–163; student groups 151–152; students course preparation 146; students failure to log in 155–156; students at risk 159–160; study groups 158; support staff pre-launch tasks 149–150; support staff QA tasks 148–149; support staff responsibilities 147–148; support staff tasks 154–155; synchronous sessions 156–157; welcome messages 152–153; winding down course 163–164; wrapping up 160–161 course update checklist (Frey) 180 course updating 123, 126–127, 143, 166, 175, 178–180 course website 22, 23, 32, 47, 57, 59, 78, 138, 143, 194; Announcements section 153, 158; external links 169 curriculum 4, 45, 52, 57, 79, 81, 185, 190

curriculum design 61, 76–77 curriculum review 76; maintaining high quality 184 customer relationship management (CRM) system 47, 67, 71, 190, 194 data privacy/security 3, 16, 49 data science courses 138 data warehouse 170–171 database course 39, 92, 96, 102, 131, 138, 151, 180 databases 49, 91, 92, 139 deafness/hearing impairment 41, 42, 124 default text 122–123 designer see instructional designer design principles: locality of design 109, 126 design standards (course quality) 85–86 development (course development) 6, 12, 13, 190 development models 55–57, 190 Diesel, S. 146, 156, 157, 160 digital strategy 3, 6, 52, 187, 190; benefits 13–14, 15 digital textbooks 33 diplomatic skills 115, 115 disabilities 3, 8, 11, 24, 25, 41–42, 126, 132, 159, 160, 196; legally defined (USA) 41; student accommodations 41, 42, 54, 152, 154 discussion forums 39, 74 due dates 108, 109, 110, 126, 127, 142, 154, 157, 174, 175 economies of scale 17, 43, 58 editor 2, 4, 7, 12, 49, 191, 119, 120, 132, 141, 142, 186; definition 191 email 8, 37, 45, 47, 67, 71, 72, 117, 146–163, 168, 173, 190, 193; addresses 164; external (SMTP) 153; internal system 153; lists 48 emotional distance see affective distance engagement (student engagement) 8, 25, 39, 87, 136, 156, 191 enrollment 6, 9, 11, 28, 39–47, 61, 80–82, 134, 135, 136–137, 147, 149, 187, 193; forecasting 63–65, 71 enrollment advisors 66–67, 71; in-house vs. outside agencies 68 enthusiasm 106, 122 exams 97, 161, 179

200 Index face-to-face courses 5, 6, 7, 10, 11, 14, 117; academic integrity 35; advantages of online technology 8–9 face-to-face interaction 191 Facebook 168 facilitated course 28, 40, 159, 170, 172 facilitator at large 40, 191 facilitator (course facilitator) 3, 4, 19, 37, 39, 61, 63, 73, 77, 81, 82, 101, 119, 127–153, 183, 191; illness or absence 168–169; recruitment, development, evaluation 29–30; training 120; weekly meetings 151 facilitator, lead 25, 40, 191 faculty 1–13, 51, 84, 114, 127–149; calling students 159; career development 27; communication skills 8, 15; course revisions 178; definition 191; development plans 21; fear of failure 167; identification 14–15; illness or absence 168–169; marketing collateral (updating) 80; motivation (advance course design) 121–124; motivation to teach online 15; optimal teaching sequence 92; part-time 21, 27, 30, 78; rapport with students 146, 152; strong relationship with students (fostering academic integrity) 37; teaching online (organizational support) 57–58; unexpected developments 166; weekly meetings 151 faculty and student support staff see support staff faculty considerations for building online programs 3, 19–31; compensation models 26–27; designating faculty coordinators 27–28; development of online courses 22–24; employing course facilitators 28–30; mentoring to teach online 24–26; transition to teaching online 19–22 faculty coordinators 3, 19, 27–28, 121, 147, 191 faculty evaluations 183, 191; metrics 79 faculty forum 150, 151, 191 faculty training 120 feedback 9, 20, 26, 33, 39, 55, 61, 73, 74, 81, 86–87, 94, 96–97, 118, 139, 163

field 98 final exam 97, 99, 102, 103, 155, 157, 160, 182; administration 161–163; LMS slowness 168; oral 162; proctored 38, 61, 74, 75, 76, 78, 115, 140; review 135–136; student complaints 173 fine-grained courses 54, 191 Flash 96, 132–133 flight simulators 125, 138 flipped courses 8, 191 formative assessment 9, 25, 74, 95–97, 100, 101, 103, 110, 111, 179, 191 formative evaluation 20, 22, 90, 121, 123, 148, 191 four-credit courses 76 French language course 56, 73, 129 Freud, S. 107 Frey, J. 17, 88, 101, 104, 116, 125, 127, 128, 180 Gage, P. 107 Garofoli, E. 99, 122, 131 Global Accreditation Center for Project Management Education Programs 186 Google Groups 168 GPA 29, 48 grade begging 110, 191 grade book 21, 23, 34, 106, 108, 142, 148, 160 graded assignments 90 graded test 73, 100 grading criteria 37 grading uniformity 40 graduation rate 48 group facilitator 168 hardware 16, 17, 101, 136, 162 headset 88, 107, 134, 137, 157 health informatics 39, 152, 186 Help link 103, 148, 172 high-touch 44, 60, 73, 147, 159, 192 hyperlinks 88, 92 HyperText Markup Language 34, 97, 106, 110, 124, 125, 129, 132–133, 189; definition 192; HTML5 33 illustrator 2, 12, 58, 119 impersonation 35, 38 incomplete 55, 163, 170; definition 192 information technology 45, 184

Index 201 instruction modes 58–60 instructional designer 2, 12, 21, 25–26, 51, 58, 60, 69, 74, 78, 84, 101, 104, 114, 117, 120, 122–149; course revisions 178; definition 192; hired for advanced courses 115–119; unexpected developments 166 instructional models 5, 192 instructor 20, 21, 26, 29, 30, 39, 86, 90, 96, 101, 117, 123, 129, 137, 140, 149, 154, 192, 196 interaction 3, 117; meaningful 86–90 interaction model 60–62 internet 109, 125, 128, 132, 167; bandwidth 128, 130, 133, 163; outage 170, 174; World Wide Web 194; see also online item analysis 21, 96, 192 JavaScript 33, 97, 108, 110, 125, 126, 127, 129, 132, 133, 142, 179, 192, 194 Kariofyllis, A. 150, 151, 154, 164 kickoff meeting 121, 150, 191, 192 King, E. 14 landing pages 47, 192 leadership 7, 11, 58, 72, 128 leads 47, 48, 63, 71, 82, 193 learning activities 4, 16, 33, 58, 92, 103–104, 105, 129, 194; asynchronous and synchronous mix 101; definition 100, 192; planning 100–102; sequencing 102 learning community 152, 156 learning content management system (LCMS) 15–16, 126, 192 learning disabilities 41, 42, 53, 54, 159 learning management system (LMS) 2, 3, 5, 6, 20–36, 49, 55, 56, 77, 85, 87, 95, 96, 97, 99, 102, 104, 106, 108, 110, 124–154, 179, 185, 194; characterisation 15–17; definition 192; orientation course 118; outage or slowness 168; specialist providers 16–17; tasks to prepare advanced course for running 142; unexpected developments 167 learning objectives 92, 93, 94, 95; defining 90–91; supported by learning activities 100 learning outcomes 178

learning styles 100, 101 Leonard, E. 88 letters of recommendation 68 lighting 7, 25, 107, 119 Live Classroom 8, 40, 140, 148, 154, 164, 192–193 Live Office 8, 192–193 market analysis 10, 11, 81 market differentiation 11, 193 marketing 3, 67, 71, 80, 82; outsourcing 63 marketing channel 47, 63, 193 marketing collateral 11, 47–49, 63, 71, 192; definition 47, 193; typical metrics 48; updating 80 marketing online programs 42–47; blended market demand as indicator 44; budgets 47; competitive landscape analysis 45–46; competitive market analysis 43–44; danger; marketing websites 46; in-house 43; market de mand assessments 45; market research (revelation of specific markets) 45; marketing campaigns 47, 48; non-enroll survey 45; outsourcing 43; program differentiation 44; specialist companies 47; word of caution 46 marketing plan 62–63 Massive Open Online Course 14, 44, 52, 60, 73, 193; MOOCish courses 61, 69, 193 mathematics 14, 15, 22, 32, 56, 92, 102, 110, 130, 132, 140, 181 mentoring 3, 15, 19, 29–30, 84, 117, 193; faculty to teach online 24–26; formal vs. informal 24 micro-master’s 62, 193 microphones 107, 134, 136–137, 162 Microsoft Word 37, 141 mind-mapping software 128 module 16, 90, 91, 96, 100, 105, 109, 112, 121, 125, 179, 182; definition 193; development schedule (advanced courses) 120; graded module discussions 74; outreach 158 “Module 0” 103, 172, 193 module maps 92, 93 mouseover 172, 193

202 Index multimedia 8, 12, 37, 100, 125, 156, 168, 192, 195; conferencing 8, 15, 33, 128; content 137–139; definition 194; object 114, 132, 143, 194; synchronous sessions 87–88 music education 22, 38, 68–69, 102 National Council for State Authorization Reciprocity Agreements (USA) 10 National Disability Authority: Centre for Excellence in Universal Design 127 native search 47, 193; definition 194 niche markets 12, 45 non-enroll survey 45, 194 null value 98 nursing courses 38, 52 obsolescence: courses 123, 174–175; files 110, 112; social media 130 office hours 26, 37, 73, 123, 134, 139, 157, 183 Office of Disability Services (ODS) 11, 41–42, 154 office of distance education 57, 58 “Omit needless words” (Strunk and White) 88 on-campus residency requirement 35, 38, 74, 194 online 194; getting started 3, 5–18; see also internet online course: differences (from face-toface teaching) 23; preparing faculty to develop 22–24 online course: pre-development issues 3, 32–50; academic integrity 34–38; marketing collateral 47–49; marketing online programs 42–47; scaling online courses 34–38; student data security 49; students with disabilities 41–42; technology choice 32–34 online course content: metrics 79 online education 3, 194 Online Learning Consortium Awards 86 online orientation course 21 online proctoring service 38 online program: administering every stage 1; advantages of online technology 8–9; assessing demand 11–12; choosing LMS 15–17; cost-benefit analysis

(best practice) 6; course development process 12; defining program (specification document) 9–11; digital strategy (benefits) 13–14, 15; faculty considerations 3, 19–31; formal proposal 6–7; identifying faculty 14–15; opportunities and challenges 7–8; reasons for creating 5–6; starting 5–18; target audience 9–11 online technologies 8–9, 32–34, 194 Open SUNY COTE Quality Review (OSCQR) Process and Rubric 86 Oracle 131 outcome analysis 184, 186, 194 outsourcing 43, 63, 77, 167 pedagogy 1, 2, 22–28, 33, 56–62, 75, 79, 88, 114, 123, 124, 127, 136, 161, 166, 178–179, 186; advancement 185 peer-learning communities 59 personal computer 33, 34 Photoshop 119 pipeline 45, 63, 71, 83, 194 placeholder content 123 plagiarism 34, 35, 36–37, 151 portable devices 130, 132, 179 PowerPoint 117, 122, 135 prerequisite courses 54, 59, 64, 69, 76–77, 96, 143, 152, 153, 171–172, 186, 193 prerequisite material 100, 103, 175 primary key field 98 problem-solving 30, 39, 86, 100, 119, 130, 131, 138, 173, 175 proctored assessments 78, 99, 194 proctored final exam see final exam production tasks (advanced course design) 139–141; checklist 139; college-based 140; course-based 140; faculty-based 140; program-based 140; term-based 140; unique to specific courses or faculty 140 program administration 3, 66–83; actions to improve program quality (cost types) 75–78; admissions 67–71; contact hours 72–73, 74; course standards 74–75, 76; enrollment advisors 66–67; managing program throughout lifecycle

Index 203 80–82; marketing collateral 80; program evaluation 79; recruitment pipeline 71; student retention 72 program definition and development 3, 51–65; course development model 55–57; course scheduling model 52–55; enrollment forecasting 63–65; faculty teaching online (organizational support) 57–58; interaction models 60–62; marketing plan 62–63; modes of instruction 58–60; synchronous teaching technology 62; program goals (defining) 51–52 program evaluation 4, 79, 187 program quality 75–78 psychology course 52, 107 publishing model 56 punctuation: plagiarism detection 37 Python 34 quality assurance (QA) 4, 7, 14, 78, 107, 115, 119, 120, 130, 141, 142, 143, 147, 169 Quality Matters Course Design Rubric Standards 86 question validity 94–95, 194 quizzes 8, 9, 74, 87–88, 93, 95, 97, 99, 102, 103, 132, 140, 142, 147, 148, 161, 162, 179, 181, 189, 195 quotation marks: inconsistency of usage (plagiarism detection) 36 Ragan, L. 13, 24, 25–26, 27, 167 RAM 111 randomization 36, 74 random question selection 81 real time 42, 101, 125, 190, 191, 195 relational database 98 remedial material 9, 87, 90 remedial training 30 remote learning see telepresence reputation 6, 11, 14, 43, 44, 72, 81, 129, 141 reserve facilitator 168–169 respect 20, 116, 118, 141 retention advisors 72 review sessions 135–136 role-play 60, 73, 99, 121, 135, 185 sandbox 132, 194 scalability 61, 73, 81, 128

scaling online courses 38–41 scheduling 3, 5, 119 scheduling models 21, 190 search engine ads 47, 48 search engine optimization (SEO) 43, 194 self-paced courses 46, 52, 101 sequencing 19, 194 simulators 137–139 slides 22, 23, 117, 122, 133, 135, 150, 157 smartphones 33, 34, 179, 185 social media 38, 48, 158, 173; problem of rapid obsolescence 130 social skills 164 software 2, 15–16, 33–34, 47, 56, 67, 87, 90, 101, 102, 124, 127–128, 129, 131, 132, 138, 152, 190, 192, 195, 196; screen-readers 42; updates 185 Spark system 180 spreadsheets 7, 138, 139, 141 start 48–49, 195 statistics course 15, 52 stone-soup development 57, 80, 109, 110, 122, 195 Structured Query Language (SQL) 91, 92, 111 Strunk, W., Jr. 88 student engagement see engagement student evaluations 26, 30, 79, 195 student feedback (course updates) 178–179, 180 student groups 151–152 student information system (SIS) 16, 49, 63, 149, 151, 154, 160, 195 student retention process 72 students: career goals 44, 45; complaints 173; completion rates 61; course and faculty teaching evaluations 182–184; course preparation 146; in crisis 170; data security 49; difficulties with assignments 172–173; failure to log in 155–156; late submissions 174; “logical” vs. “visual” learners 91; ratio to faculty 54, 77, 81; at risk 159–160; strong 170–171; textbook problems 169–170; underprepared 171–172; with varied preparation (course design) 102–103; welcome messages 146, 149, 151; writing style 36

204 Index student satisfaction 2, 29, 61, 90, 96, 103, 156, 183; see also engagement student workload 81, 104–105, 127, 175–176, 184 study group 100, 158, 195 study guide 127, 140 study room 158, 195 summative assessment 95–97, 100, 101, 195 support staff (faculty and student support staff) 2, 4, 12, 58, 78, 81, 114, 133–136, 191; course revisions 178; definition 191; hired for advanced courses 115–119; pre-launch tasks 149–150; QA tasks 148–149; responsibilities (course underway) 147–148; tasks while course is running 154–155; unexpected developments 166; weekly meetings 151 surgical simulations 138 surveillance laws course 175 syllabus 126–127, 146, 148, 149, 152, 169; new course 85, 86 synchronous activities 77, 85, 96, 195 synchronous conferencing software 159 synchronous discussion 20, 39 synchronous interaction 42, 72, 134–135, 195 synchronous sessions 7–8, 16, 25, 37–38, 40, 74, 76, 79, 93, 150, 155, 195; advanced course design 133–136; regularly scheduled 156–157 synchronous teaching 3; methods 58–60; technology 62 task manager 139–140, 195 teaching assistants 28, 61, 63, 134, 148 teaching quality: metrics 79 teamwork 1, 119 technical knowledge and diplomacy ratios 115, 115 technical problems 4; saving faculty time 147; unexpected developments 167 technical support 126, 131; cost 16; in-house vs. outsourcing 167 technology 3, 4, 7; choice 32–34, 127–133, 184; inappropriate 88; new 185

teleconferencing 8, 136 telephone calls 37, 67, 71, 72, 133, 147, 149, 151, 155, 159, 162, 164, 173, 190 telepresence (remote learning) 82, 136–137, 194, 195 term lengths 21, 52–53, 53; advantages and disadvantages 54 Test Yourself (TY) 96–97, 98, 111 textbooks 7, 8, 9, 23, 39, 59, 92, 103, 105, 122, 123, 152, 153; new editions 181; replacement 181–182; student problems 169–170 Thompson, F. 20, 90, 112, 117–118, 121, 126, 129, 133, 139, 141 threaded discussions 128, 130 ticketing system 139, 195 time 6, 7, 9, 12, 14, 16, 20, 21, 22, 23, 26, 27, 32, 34, 40, 42, 45, 48, 53, 54, 57, 78, 87, 89, 92, 93n, 94, 99, 103, 105, 106, 108, 114, 116, 119, 121, 122, 124, 131, 135, 140, 141, 143, 147, 152, 160, 162, 170, 179, 183, 192 time wasting 52, 56, 88, 90, 101, 138 time zones 39, 60, 101, 195 topical sessions 134–135 topic sequencing 103, 189 training 3, 11, 16, 17, 19, 21, 22, 57, 67, 68, 117, 120, 132, 133, 140, 149 transition course 44, 54, 64, 67, 68, 69–70, 92, 172, 195 transition course decisions 70 travel 6, 38, 71, 125, 130, 136, 137, 161 trust 37, 121, 122, 156, 189, 193 tuition fees 61, 149, 162 Turnitin (verification service) 36 tutorial websites (legitimate versus illegitimate) 35 Twitter 130 typing-detection software 35 typography (opportunities and pitfalls) 89–90 unexpected developments 4, 166–177; academic integrity compromised 176; challenging strong students 170–171; content errors or omissions 174–175; course design or implementation problems 169; faculty or facilitator illness or

Index 205 absence 168–169; helping underprepared students 171–172; late submissions 174; LMS outage 168; misalignment of course content 175; student complaints 173; students having difficulty with assignments 172–173; students in crisis 170; student workload uneven 175–176; technical problems, technical support 167; textbooks (student problems) 169–170 United States Department of Education 10 United States Distance Learning Association 86 universal design 11, 106, 124–126, 196; see also accessibility Universal Design for Learning (UDL) 125 universal element 86, 196 usage inconsistency 36–37 video 7, 15, 16, 21, 23, 25, 33, 38, 80, 87–88, 93, 100, 101, 105, 106–107,

109, 121, 130, 138, 143, 152, 158, 175, 190, 194 video captions 8, 11, 12, 42, 124, 125, 142, 149 video conferencing 37, 44, 57, 128, 185 videographers 2, 12, 58, 78, 115, 118, 119 video quality 136 virtual labs 33, 114, 137–139, 196 virtual room 133–134, 196 vision statement 9–10 webcams 20, 25, 38, 88, 134, 161, 162, 194 webinars 24, 30, 132, 146 welcome messages 152–153, 154 White, E. B. 88 wiki 130, 140, 196 word-use frequency 36 work experience: transition course decisions 69, 70, 70 YouTube 16 Zoom 42, 112, 126, 136, 196