Measuring Return on Investment in Training : A Practical Implementation Guide [1 ed.] 9781869223854, 9781869220198

Citation preview

Return on Investment

| IN TRAINING |

This book is intended to help human resource and training managers to obtain the necessary knowledge and skills to determine Return On Investment (ROI) in training programmes.

Money speaks sense in a language all nations understand. | Aphra Behn 1681 |

Measuring Return on Investment in Training provides a systematic and integrated approach to the theory and practice of ROI. It also provides a more holistic understanding of the correct application of ROI measurement in the workplace.

For too long, the training and development process has escaped the scrutiny of accountability. | Jack Phillips |

You will learn how to:

Both assessment and evaluation are required to determine if learning has been translated into work performance and therefore if it is making an impact on the achievement of the company’s strategic objectives and thus the bottom line. | Ken Lancaster & Robyn Wolfson |

If it can’t be measured, it can’t be managed. | Peter Drucker |

• measure ROI to determine the value of training for your company. • motivate the use of ROI for the organisation. • apply the steps to measure ROI. • calculate ROI for a training intervention. • avoid the pitfalls in ROI measurements. • provide guidelines for the correct calculation of ROI. • implement the ROI process in an organisation. • overcome resistance to ROI. • compile a ROI report for a company.

ROI can be used both to justify a planned investment and to evaluate the extent to which the desired return was achieved. | Clive Shepherd | Measuring the ROI of training programs is one way to show business leaders the value of training investments in nancial terms. | Larry Long |

| Measuring Return on Investment | IN TRAINING |

Measuring

| IN TRAINING |

A Practical Implementation Guide

Also included is a CD-ROM with spreadsheets and ROI calculations. | Marius Meyer ISBN: 1-86922-019-6

ROIcover

1

5/14/03, 8:58 AM

| Catherina Opperman | Chris Dyrbye

|

MEASURING ROI IN TRAINING

MEASURING ROI IN TRAINING A

PRACTICAL IMPLEMENTATION GUIDE

MARIUS MEYER CATHERINA OPPERMAN CHRIS DYRBYE

2003

Copyright © Knowres Publishing All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means without prior written permission from the publisher. First published in 2003. ISBN 978 1 86922 385 4 Ebook Published by Knowres Publishing (Pty) Ltd P O Box 3954 Randburg 2125 Republic of South Africa Tel: (+27 11) 706 6009  Fax: (+27 11) 706 1127 E-mail: [email protected] Website: www.kr.co.za

Printed and bound by: Creda Communications, Johannesburg Typesetting and layout: Cia Joubert [083] 297 7902 Cover design by Carike Meiring, 43 Degrees Design, [email protected]

DEDICATED TO

THE PEOPLE WE LOVE

CONTENTS

CONTENTS Foreword .................................................................................... v About the authors ................................................................... vii Preface ...................................................................................... ix Chapter 1: The Need For Roi In Training .............................................. 1 Demystifying ROI ......................................................................... 5 Why do people resist ROI? .......................................................... 8 Reasons for ROI .......................................................................... 9 Examples of ROI measurements .............................................. 11 Pitfalls to avoid ........................................................................... 15 Guidelines for implementing ROI ............................................... 16 Outline of the book ..................................................................... 17 How to use this book .................................................................. 20 Conclusion: Chapter 1 ......................................................................... 21 Chapter 2: Competency Analysis linked to Business Performance Management ........................................................................................ 23 Performance management process .......................................... 27 Step 1: Determine what data needs to be collected to conduct a gap analysis and assess competence ................................... 28 Time spent on measuring performance management as part ....... of the process of calculating the ROI in training ......................... 30 Training process sequence ........................................................ 31 Why do we need to train people? ............................................... 32 Learning aligned to the principles of the South African Qualifications Authority (SAQA) Act and the Skills Development Act .............................................................................................. 32 Objectives of the NQF ............................................................... 33 Competency analysis ................................................................ 35 The critical cross-field outcomes ............................................... 35 Emotional intelligence (quotient) (EQ) and the critical cross-field outcomes ................................................................. 36 Step 2: Identify the key performance indicators and the interpersonal skills of the job ............................................................................ 37 Step 3: Attach a weighting to the competencies ........................ 39 Criteria for the weighting of the key performance indicators .. 39 Step 4: Determine the outputs to be delivered .......................... 39 Link outputs to the balanced score card (BSC) ..................... 40 How do we define the balanced score card? ......................... 40 Customer perspective ............................................................ 41 Internal perspective ................................................................ 41 Learning and growth perspective ........................................... 41

i

MEASURING ROI IN TRAINING

Financial perspective ............................................................. 42 The scorecards are combined to measure individual, departmental and the business unit’s performance ............... 42 Step 5: Identify unit standards and link assessment criteria to output ......................................................................................... 43 Performance management and unit standards ...................... 43 Step 6: Determine the time needed to accomplish competence ............................................................................... 44 How to calculate time ............................................................. 44 Step 7: Complete the performance documents ........................ 46 Step 8: Performance review meeting ........................................ 46 360-degree multi-rater feedback ............................................ 47 Best practice suggestions ...................................................... 48 Multi-rater checklist ................................................................ 49 Perform a gap analysis by comparing the actual to the required competency level ..................................................... 50 Point scale according to organisational levels ........................ 50 Step 9: Complete the skills development matrix ....................... 52 Checklist upon completion of the performance management process .................................................................................. 59 Conclusion: Chapter 2 ......................................................................... 60 Chapter 3: Implementing Skills Development Initiatives ..................... 61 Skills implementation process ............................................................ 64 Step 1: Collate the results from the performance management process and skills audit ............................................................. 65 Step 2: Align these competencies to organisational strategies . 66 Step 3: Select learning material aligned to the principles of the NQF ........................................................................................... 67 (a) Competence ..................................................................... 67 (b) Unit standards ................................................................... 68 - Knowledge ........................................................................... 69 - Skills ..................................................................................... 69 - Behaviour/Personal expression ........................................... 69 (c) Allocation of credits ........................................................... 71 (d) Applied competence ......................................................... 71 Step 4: Decide whether internal or external material will be used ........................................................................................... 72 Step 5: Identify level descriptors when selecting material ......... 72 Step 6: Ensure the learning material covers the unit-standard criteria ........................................................................................ 73 Step 7: Use the information on the WSP and annual training report (ATR) to calculate the ROI and estimate the budget for the following year ............................................................................. 73

ii

CONTENTS

Conclusion: Chapter 3 ......................................................................... 74 Chapter 4: Assessment on pre- and post-performance ..................... 77 Why use assessment as a measurement to determine competence? ...................................................................................... 79 What is assessment? ......................................................................... 80 What is the assessment of competence? .......................................... 80 What is competence? ......................................................................... 81 Pre- and post-assessment process flow ............................................ 81 Using assessment to perform a gap analysis ..................................... 82 Assessment methods ......................................................................... 84 All evidence collected should comply with the 'VACS' criteria .... 84 Assessment tools ...................................................................... 84 Behaviour observation ............................................................ 85 Learning assignments ............................................................ 86 Feedback questionnaires ....................................................... 87 Work output records .............................................................. 87 Portfolio of evidence (POE) .................................................... 88 Guidelines for assessment ................................................................. 88 Conclusion: Chapter 4 ......................................................................... 89 Chapter 5 Calculating ROI ................................................................. 91 Overview of the ROI spreadsheet ....................................................... 93 1) Summary worksheet ............................................................ 95 2) Input assumptions worksheet ............................................... 99 3) Gap-analysis cost worksheet ............................................. 100 4) Design and development cost worksheet .......................... 101 Section 1 ........................................................................ 101 Section 2 ........................................................................ 103 Section 3 ........................................................................ 103 5) Facilitation and delegate costs worksheet .......................... 104 6) Assessment cost worksheet .............................................. 105 7) Evaluation worksheet ......................................................... 106 8) Fixed cost worksheet ......................................................... 106 9) Cost-benefit worksheet ...................................................... 108 10) Cash-flow worksheet .......................................................... 113 Conclusion: Chapter 5 ....................................................................... 115 Chapter 6: Evaluating the ROI impact on training ............................ 117 Measurement and evaluation ............................................................ 118 ROI report ......................................................................................... 119 ROI implementation process ............................................................ 123 Step 1: ROI awareness ........................................................... 124 Step 2: Strategic ROI planning ................................................ 125

iii

MEASURING ROI IN TRAINING

Step 3: GAP/performance analysis ....................................... 126 Step 4: Training plan/workplace skills plan ............................ 126 Step 5: Determine the cost of learning .................................. 126 Step 6: Conduct training ........................................................ 127 Step 7: Assessment of learning ............................................ 127 Step 8: Calculate ROI ............................................................ 127 Step 9: Evaluate training impact ............................................ 127 Step 10: Communicate ROI results ........................................ 128 Step 11: Develop and implement the training improvement plan ........................................................................... 129 The ROI review checklist .................................................................. 129 Summary .......................................................................................... 131 The way forward ................................................................................ 133 Conclusion: Chapter 6 ....................................................................... 135 Appendix: ROI resources .................................................................. 135 ROI spreadsheet – small ......................................................... 136 ROI spreadsheet – medium ..................................................... 140 ROI spreadsheet – large .......................................................... 143 ROI consolidation .............................................................................. 144 Frequently asked questions about ROI .................................... 145 ROI internet resources ............................................................ 148 Index .................................................................................................. 153 Bibliography ....................................................................................... 157 Glossary ............................................................................................ 161

iv

FOREWORD

FOREWORD I vividly recall a marketing executive of a beer company being asked whether the "huge" advertising spend was justified in terms of generating sales income and whether it could be measured. His answer left the audience with much to think about. "Half of what we spend is effective," he said, "and half of what we spend is not effective. Trouble is that we never know which half is which." Measuring the contribution of training interventions faces the same dilemma. Donald Kirkpatrick, an early pioneer in the 1970s to develop an acceptable process to evaluate training, made perfect sense when he described the stages in the evaluation of training as: 1) 2) 3) 4)

Reaction – how the trainees "liked" the training programme, Achievement of course objectives – whether the trainees could now do what the course objectives said they would be able to do. Change back on the job: Did things change or improve back on the job? The impact of this change on the bottom line of the organisation.

Steps one and two measure training efficiency. In other words, did the training programme do its job? These two steps are relatively easy to conduct with few or no contaminating factors. Steps three and four measure training effectiveness – determining whether the competencies acquired are having the desired impact on the organisation. Unfortunately, from this point on evaluation of training is by no means as simple or as clear-cut as it was in steps one and two. Step three, improvement back on the job, is impacted by extraneous factors independent of the training programme itself. For example, a heat wave sets in over the country just as the "trained" representatives return to the trade and cool drink sales soar. Was it due to the weather or their enhanced competence to sell cool drinks? And, as if that is not enough, the outcomes of step four are even more questionable. How can the outcomes of training be isolated to ROI v

MEASURING ROI IN TRAINING

when there are so many extraneous factors that impact the bottom line? For example, a rise in interest rates puts a cash strap on consumers who have less disposable income to spend on cool drinks and sales decline. Was the decline due to poor training effectiveness or would it have happened anyway … or would it have been worse if the reps were not competent? Which brings us back to the "half-and-half" dilemma of the beer marketing executive where we are reasonably confident that training made an impact on the bottom line, but we are unable to say exactly where. Accountable for every rand spent on training, management wants and is entitled to measure the success rate or ROI of its training spend but seldom makes any attempt at such a measurement simply because it is too vague and complex to measure. At best they hope, like the beer marketing executive, that half of their money spent on training hits the mark, accepting that the other half may go to waste. This kind of ambiguity is no longer acceptable in business organisations aspiring to "world-class" status. Seeking to significantly raise their levels of performance to successfully compete and survive in a global economy, pinpoint accuracy in determining the ROI on human capital development is becoming increasingly important. Whilst it satisfies shareholder needs to know that their investment is yielding a return, it enables leadership in the organisation to benchmark the effectiveness of their learning and development strategies against best practice in other organisations. It calls into question doubtful learning activities that make little or no contribution to ROI and refocuses on the high-impact learning activities that make a measurable difference. Measuring ROI for Training spells out the "how to" of Kirkpatrick's step four in a systematic and logical way with support tools that will impress the most skeptical accountant and is a must for all who are accountable for the most noble of contributions to the organisation – the development of people. Andre J Parker, M.A. cum laude HR Director, ABI 2003 vi

ABOUT THE AUTHORS

ABOUT THE AUTHORS Marius Meyer is programme manager and senior lecturer in human resource development at the Technikon of Southern Africa. He is also a consultant for the Institute for Organisation Development and Transformation. Prior to accepting a lecturing position, he was employed as a training and development practitioner. He coordinated and facilitated training and organisation development interventions in the areas of employment equity and diversity, communication, management development, performance management and quality improvement. He is a registered personnel practitioner with the South African Board for Personnel Practice and has full membership of the following professional bodies: South African Society for Quality and the Industrial Relations Association of South Africa. He is also the national director for the American Society for Training and Development Global Network South Africa. He is an expert review panel member of ASTD in the USA to assess the best training practices worldwide. Marius has a Master's degree in Human Resource Management and is a senior assessor for the South African Excellence Foundation. He is also the editor of six books including: Managing Human Resource Development: An outcomesbased approach, Organisation Development and Transformation in South Africa and ETD Practices in South Africa. Marius lectures at RAU (Rand Afrikaans University) on a part-time basis and is a moderator at Technikon Witwatersrand and Technikon Northern Gauteng. His research and consulting interest lies in the areas of management development, facilitation, change management and organisation development. He facilitated interventions for Sappi, Standard Bank, Smile, the South African Navy, Liberty Group, Matla Coal, Independent Development Trust, Department of Land Affairs, Department of Correctional Services, Western Metropolitan Local Council, Sasol, SAPS and Eskom. Contact information: Tel: (011) 471 3549 or 082 8593593 e-mail: [email protected]

Rina Opperman is a human resource development consultant and skills development facilitator heading up the People Development vii

MEASURING ROI IN TRAINING

Department of Medscheme. Catherina has an Education & Training Development Practice (ETDP) qualification from Rand Afrikaans University and a BTech degree in Human Resource Development from Technikon SA and holds the following UK City & Guilds qualifications: ETDP, D32/33; Vocational/skills assessor, D34; Internal Verifier and D36; Recognition of Prior Learning Advisor. She has been instrumental in aligning training material according to National Qualification principles in outcomes-based format and mapped to unit standards, as well as securing discretionary grant funds from the Inseta for skills development. Catherina is a qualified master assessor with the UK City & Guilds, conducting training on assessment in the workplace. She has designed, developed and facilitated various other modules. She is in the process of measuring the return on investment for the organisation through a cost-benefit analysis. Contact information: Tel: 011- 510-2497 or 082-4694885 e-mail: [email protected]

Chris Dyrbye has been involved in the banking, training, telecommunication and computer industry for the past 18 years, gaining extensive experience in operational branch banking, developing training courses, information system design and general management. He completed his Master of Business Administration (MBA) through Potchefstroom University Business School – graduating in May 2000. He has provided the following consulting activities: developed a feasibility study for an information system strategy for a Johannesburg stock exchange-listed company, which required an extensive ROI element; provided First National Bank Training Department with a feasibility study regarding their interactive technology-based training requirements; developed an e-learning ASP strategy for an SA Internet service provider, which was implemented and offered to clients as part of an e-learning/knowledge management solution; provided Absa Learning & Development Department with cost-benefit feasibility studies for various projects. Developed various tools that are used in the implementation of Absa e-business strategy with a focus on ROI in e-learning. Contact details: Cell 082 334 3563 e-mail: [email protected] viii

PREFACE

PREFACE The idea for this book originated several years ago. During the past decade, line managers in South African organisations have challenged human resource and training managers to show the value that they add. Unfortunately many human resource managers simply ignored the demand to add value, and were forced to accept the consequences – being retrenched or part of the training function being outsourced. Others are still hanging in there, and some feel that the Employment Equity Act, Skills Development Act and all the other labour-related legislation came to their rescue. After all, these laws create a lot of paper work that line management does not have time for, so human resource staff feel that they still have a role to play. The reality is that the new global world of work is a demanding and changing workplace that requires the highest level of performance and excellence. Human resource and training managers are required to become business partners by implementing people interventions that really make a difference at work. Gone are the days that companies could afford to waste thousands of rands on expensive and sometimes unnecessary training that does not make any difference in terms of business impact and profitability. In light of the above context, this book presents the human resource and training manager, as well as skills development facilitators with an opportunity to measure their impact and show the return on investment (ROI) in their training programmes. While American companies have been involved in ROI for many years, South African organisations are only now beginning to realise the importance of measuring the ROI in training. The problem, however, is that although some training managers know that they have to calculate and show ROI, they do not know how to do it, let alone where to start. This book has been written to guide the training manager through the whole ROI process by implementing a practical step-by step ROI process in a user-friendly manner. In fact, all the necessary documentation, such as ROI input and calculation spreadsheets, checklists and questionnaires are even provided on CD-ROM to make it ix

MEASURING ROI IN TRAINING

easy to determine ROI in the shortest and most effective way. Thus, it is our hope that as you read the book, you will find the knowledge and inspiration to calculate and apply the complete ROI process at your organisation. No matter in which industry you are involved, the principles and practices of ROI are universal and can be applied in any work environment. There is just one requirement to be able to measure ROI and that is that some form of training must be conducted! This book is intended to help managers to obtain the necessary knowledge and skills to determine ROI in training programmes. Specifically, we believe this book will be of particular interest to trainers, human resource managers, training managers, human resource development practitioners, skills development facilitators, training and development consultants, organisation development practitioners, line managers and staff employed at Sector Education and Training Authorities (SETAs). Although individual chapters can be read on an “as-needed basis”, we recommend that you read the book from beginning to end to have a clear picture of the holistic view of the ROI process. Since we provide a systematic and integrated approach to the theory and practice of ROI, reading the book from cover to cover will provide a more holistic understanding of the correct application of ROI measurement in the workplace. We therefore suggest that you avoid the temptation to immediately go to the spreadsheets and ROI calculations in Chapter 5, and first get to grips with the underlying principles and fundamental ROI processes and practices. You will be surprised how easy it is to measure ROI if you follow the steps in the practical way in which it is presented. We would like to thank the following people for their support, enthusiasm and inputs that assisted and inspired us to convert the idea of ROI in training into a book: Gary Taylor and Nimee Dhuloo of Medscheme, Verdi Ras of Sappi, Johan Brits of Santam, Annette Bredenkamp of First National Bank, Lex Rourke and Robin Probart of ASTD Global Network South Africa, Vicky Griffen of Nedcor, John x

PREFACE

Hefer of Global Access SA, Isabeau Koppel of ABSA and our Creator for granting us the skills and ability. In addition, we would like to thank the staff of Knowledge Resources for their hard work and dedication to make this publication a reality. We also thank our loved ones for their patience and support at the time of writing this book during the December holidays, late at night and over several weekends. This book has not been written to make bean counters out of human resource staff, but rather to make human resource staff count. Being able to measure ROI will give you the knowledge and skills to show the value of training in your organisation. Not only will it improve your own confidence as a competent human resource or training practitioner, but also your credibility with line management. In fact, it may ensure your survival with the organisation. We wish you much success as you go forth and measure ROI!

Money speaks sense in a language all nations understand. (Aphra Behn, 1681)

Marius Meyer Rina Opperman Chris Dyrbye 2003

xi

CHAPTER 1: THE NEED

FOR

ROI

IN

TRAINING

CHAPTER 1

THE NEED FOR ROI IN TRAINING

For too long, the training and development process has escaped the scrutiny of accountability. (Jack Phillips)

1

MEASURING ROI IN TRAINING

Oh no, not another training course! I can't spare my staff for three days to go on a course. This course was a real waste of time – I haven't heard anything new. My boss will never allow me to apply these radical ideas back in the office. The course is too expensive. My manager should have been on this course. These things cannot be applied in my work environment. All these statements are typical comments made by South African managers and employees about training. No wonder that training is not the most popular activity in our organisations. At the centre of all these statements is the presupposition that training does not really add value. In fact, training is often seen as a waste of time, cost and effort. Whether this is true or not is not the real answer to the problem. One should rather pose the questions: How do we know the value of training? How do we measure the effectiveness of training? It is exactly these two questions that we would like to answer in this book. We want to explore ways and means of quantifying the value of training so that you can measure the value of training in your organisation. And that is what return on investment (ROI) is all about - measuring the return on the investment your company put in training. Efforts to measure return on investment of training come from the USA, where the American Society for Training and Development has been the champion of ROI for decades already. It all started with Donald Kirkpatrick, an American training specialist who designed an evaluation system of training that addresses evaluation at four levels, i.e. reaction, learning, behaviour and results. Reaction evaluation refers to the typical evaluation forms we complete after a training course, the so-called smile-sheets in which learners indicate the extent to which they enjoyed the course, the value thereof, and other issues such as the training venue, food etc. Donald Kirkpatrick, the father of training evaluation

2

CHAPTER 1: THE NEED

FOR

ROI

IN

TRAINING

The second level refers to learning evaluation, in other words, the extent to which the learners have learned certain concepts, principles, knowledge and skills. This type of evaluation is normally done by means of tests because these enable you to assess the knowledge level of a learner. The written driver's licence test is a good example of learner evaluation. The third level of evaluation is called behaviour evaluation, and this type of evaluation seeks to determine whether people can practically apply their knowledge and skills. Good examples are the practical driver's test or behaviour evaluation of a manager who completed a management development programme. Here we ask ourselves the questions: Is there a change in behaviour? Can we measure the application of skills in the workplace? The last level according to Kirkpatrick is results evaluation. This level of evaluation attempts to measure the results in terms of the impact of the training intervention on the organisation. For example, are there fewer accidents as a result of a safety training programme? Are there fewer grievances as a result of a labour-relations training programme? Are there more sales as a result of a sales training programme? Is there an increase in productivity as a result of a training programme? What is evident from all these questions is that there must be some form of pre- and post-course measurement in order to be able to answer these questions. Reviewing the four levels of evaluation, it is clear that ROI addresses level four of Kirkpatrick's approach to evaluation. Even today the American Society for Training and Development (ASTD) uses Kirkpatrick's four levels on an annual basis when they release the ASTD state-of-the-industry reports. Although this study has been released for decades and employers have been actively encouraged to measure ROI, less than 10% of American companies actually measure the ROI of training programmes. Jack Phillips, a training management consultant is now considered to be the main expert in the field of ROI in America and in 2002 he visited South Africa to share his ideas about ROI with local training and human resource managers. A recent ASTD study revealed that training investments can yield 3

MEASURING ROI IN TRAINING

favourable financial returns for companies and their investors. It was found that an average increase of $680 in a company's training expenditures per employee generates, on average, a 6% improvement in total shareholder return. When ranked according to how much they spent on training, those companies in the top half of the study group had an average total shareholder return in the following year of 37%. From this data the message becomes clear: training pays off for businesses and investors alike, as there is a clear relation between training and bottom-line performance. ASTD Global Network South Africa conducted the first South African state of the training and human resource industry research study during 2002. A very interesting part of this study, released early in 2003, was the section on training evaluation methods. Less than 7% of training and human resource managers indicated that their organisations are using ROI to evaluate the effectiveness of training. This means that the vast majority of training managers cannot provide any tangible evidence relating to the value of their training and development programmes. A training manager in KwaZulu-Natal has recently been issued with an ultimatum by his managing director – either show tangible ROI of training within two years or look for a new job. This is the reality facing human resource development practitioners in South Africa: that they must either become business partners or change jobs! No one puts it in better perspective than the great American human resource expert, Dave Ulrich:

Human resource professionals can, should, and must become partners with other senior managers by creating value and delivering results. 4

CHAPTER 1: THE NEED

FOR

ROI

IN

TRAINING

DEMYSTIFYING ROI What exactly is this thing called ROI and how can we apply this concept in South Africa?

ROI is about measurement. As the great management guru Peter Drucker once said: "If you can't measure it, you can't manage it!" ROI is a measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme. In other words, it is the extent to which the benefits (outputs) of training exceed the costs (inputs). ROI is thus the return on your training investment, in other words what you get back for the training effort. Because the Skills Development Act and Skills Development Levies Act require employers to invest in the skills development of employees by means of training, managers are increasingly demanding not only accountability for these delays but also evidence of how training interventions benefit their companies in terms of financial impact. The pressure to measure ROI is therefore increasing in South African organisations and indeed in organisations worldwide. Measuring the return on investment of training programmes is a powerful way to show top management the value of training investments in financial terms. ROI answers the question: "For every rand invested in training, how many rands does the employer get back?" Many training departments have not made the extra effort to show the payoff of their training programmes. The good news is that ROI can be determined through a scientific and professional approach to measurement. There are great rewards for such an evaluation. Phillips (1996) mentions the example of Magnavox Electronics Systems 5

MEASURING ROI IN TRAINING

Company in California, USA. A literacy programme was introduced at the company to improve the verbal and mathematical skills of employees. The benefits of the programme in terms of business impact were reduced scrap and rework and ultimately increased productivity. These benefits were converted into a monetary value, and a 741% return on the training investment was reported! If we think of the multitude of ABET programmes we have in South Africa, what would (or could) the ROI be? Do companies measure the ROI of these and other training programmes?

There is no other workplace issue on which so much money is spent with as little accountability as training. (Brandon Hall, 1998) Perhaps a good way to start is to ask yourself whether you want to apply ROI at your organisation. Before we can share the steps in the ROI process with you, you need to determine the need for ROI at your company. The following checklist will help you to make up your mind. CHECKLIST TO DETERMINE YOUR ROI ST ATUS STA ROI SELF-ASSESSMENT QUESTIONS 1

Do you measure the rand value return on your training investment?

2

When compiling training reports to management, do you report on the ROI of your training programmes?

3

Do you calculate all the input costs of your training programmes?

4

Are your conducting post-assessments after training to determine the ROI against the preassessments?

5

When marketing your training programmes to staff, do you use ROI figures of previous courses to encourage management to release their staff? 6

Yes N o

CHAPTER 1: THE NEED

6

Do you know the steps to measure ROI of training in financial terms?

7

When assessing external providers, do you ask them to report on the ROI of their training programmes?

8

Do you know how to link unit standards to ROI?

9

Do you have software to help you to determine ROI?

FOR

ROI

IN

TRAINING

10 Is line management excited about the ROI value of your training programmes? 11 Have you or your staff been trained to measure ROI of training? 12 Have you studied the international ROI best practices released by ASTD? 13 Do you use ROI data to decide whether to conduct training internally or to outsource training? 14 Do you use ROI data to do your planning for your annual workplace skills plan? 15 Do you use ROI data to compile your training budget for the next year? 16 Do you apply the training and development criteria of the human capital component of the King Report on Corporate Governance? Quite a tall order? If you have more "no" than "yes" answers, you definitely need to put in a lot of effort to make ROI work for you. The good news is that you are not alone, as most training managers in South Africa are exactly where you are now – at the starting point to embark on the ROI journey. The bad news is that it will take some time to make it happen, but it is not as difficult as it seems. In fact, the rest of this book will tell you exactly how to apply a complete ROI process. After working through this book and applying the principles 7

MEASURING ROI IN TRAINING

and guidelines provided, you will be able to reverse your score on the above checklist.

WHY DO PEOPLE RESIST ROI? The interesting thing about ROI is that all training managers know that ROI must be shown, yet no one is doing it. The question arises: Why not? Answers to this are that most training managers and skills development facilitators: z

do not know how to calculate ROI.

z

come from the "soft" side of the business, i.e. human resources, and they often do not have a strong enough financial background to enable them to calculate financial measurements such as ROI.

z

are in a reporting mode as far as bums on seats are concerned. They believe that the SETAs will be happy if they can show how many people they have trained. Training for the sake of training is considered to be more important over the short term than training to add value.

z

make excuses that ROI is difficult to determine for certain types of training such as soft skills.

z

see ROI as being too complicated.

z

feel that it is too much effort to determine ROI.

z

are so busy with all their training programmes that there is no or little time to calculate ROI.

z

are scared that if ROI is calculated, it will show that their training adds no value and that they will be at risk of losing their jobs! 8

CHAPTER 1: THE NEED

FOR

ROICONTENTS IN TRAINING

As you can see, it is quite easy to make excuses for not measuring the ROI of training. None of these excuses will pass the test of accountability.

REASONS FOR ROI Despite the above hesitation on the part of training managers, there are a number of trends that have increased the interest and use of ROI for training programmes in South African organisations: z

Owing to the promulgation of the Skills Development Act and the Skills Development Levies Act , training budgets are growing.

z

The SETAs are championing training and development in all 25 economic sectors. While large amounts of money have been received via the levies, these monies must now be spent wisely, and ROI will help organisations to measure skills development in a tangible way.

z

Additional training and development programmes and learnerships are being implemented as part of companies' employment equity plans and workplace skills plans.

z

Senior management is exerting pressure on the training function to show accountability in terms of tangible business benefits.

z

Most training programmes have shown very little improvement in organisational performance.

z

There is an increasing emphasis on benchmarking in companies, even in the field of human resource development. Without quantifiable measurements such as ROI data, benchmarking is a fruitless exercise.

z

Many companies are moving in the direction of outsourcing some or most of their training, with the result that training practitioners are beginning to realise that measuring the ROI of training can justify their existence in the organisation. 9

MEASURING ROI IN TRAINING

z

Many organisations are now using the balanced scorecard as a tool to manage performance. ROI measurements will make it very difficult to provide tangible evidence of the learning and growth dimension of the balanced scorecard.

z

Professional training associations such as the American Society for Training and Development (ASTD) are putting pressure on their members to measure the ROI of training. In 2002 ASTD formed a chapter in South Africa called ASTD Global Network South Africa and the first South African state-of-the-trainingindustry research study conducted by the ASTD in South Africa clearly highlighted the need for measuring the ROI of training in South African companies.

z

Companies aspiring to achieve world-class status have to develop quantifiable measurements for people management and human resource development. To achieve a business excellence award such as the South African Excellence Foundation Award (see www.saef.co.za), it is imperative to show evidence of people development measurements and benchmarks as a component of overall business improvement.

z

The King Report on Corporate Governance makes it clear that accountability must also be reported in the area of human capital management. Training is an essential component of human capital management. See the summary of the human capital section of the King Report below and notice the relevance of training and development in the area of human capital management. HUMAN CAPIT AL SECTION OF THE KING REPORT CAPITAL

z

Human capital indicates the latent potential value that employees at all levels represent for a company. It has been recognised that the development of human capital serves not only the economic interests of the company itself, but also the requirements of the society within which the company operates. 10

CHAPTER 1: THE NEED

FOR

ROI

IN

TRAINING

z

Companies should disclose in their annual reports the criteria in terms of which they propose to measure human capital development and their performance in terms of such criteria.

z

Good corporate governance requires that business practice should reflect human capital development in areas such as the number of staff, with a particular focus on demographics (race, gender, people with disabilities), age, corporate training initiatives and employee development.

z

Reporting on the development of human capital is important because it provides both a public account of past performance and an indication of the future prospects of the company.

From the King criteria on human capital it is clear that training and development plays an important role in terms of good corporate governance. Measuring the ROI of training is therefore all about accountability and reporting on the results of your training programmes.

EXAMPLES OF ROI MEASUREMENTS Although the rest of the book will provide you with dozens of ROI examples, we would like you to get a nice feel for ROI right from the start. We will therefore provide you with a simple example or two to get you started. Please note that ROI measures the impact of training on organisational performance metrics such as higher productivity, better quality products/services, reduced costs, lower labour turnover, reduced absenteeism and increased market share. It is therefore essential to have these metrics in place. If you do not have them yet, start with them immediately. Without measures, statistics and figures, you simply cannot determine ROI. Let us say that the ROI from a sales training programme is calculated at 233%. This means that for every rand spent, there has been a return of R2,33 in net benefit, after all costs have been factored in. 11

MEASURING ROI IN TRAINING

The exact form of that benefit depends on the objectives of the learning programme. For example, if the training addressed selling skills, then one key benefit should be an increase in sales. A large insurance company in Cape Town calculated the return on investment of their sales training and determined that the ROI of the sales course was nearly 1 000%. What a great ROI ratio! We bet you that this company does not have problems in getting their training budgets approved. After all, isn't this exactly the type of information line management is interested in? ROI is a key financial metric of the value of training investments and expenditures. It is a ratio of net benefits to costs, expressed as a percentage. The formula can be expressed as: [(monetary benefits – cost of the training) ÷ cost of the training] x 100

For example, evaluation research determines that there is a 10% increase in the number of sales following the implementation of a selling skills training programme for an organisation's sales personnel. Other data from the organisation's financial system reveal that each one per cent increase in sales is equal to increased annual revenue of R25 000. Further, it is known that the training programme cost R75 000. For this example, ROI is calculated as: [(R250 000 – R75 000) ÷ 75 000] x 100 = 233% This means that for every R1 invested in the training programme, the company realised a net benefit of R2,33 in the form of increased revenue from additional sales. Not a bad ROI ratio at all! Source: Adapted from Long (1999)

As you can see from the above example, considerable effort is required to collect the evaluation data needed to calculate ROI. This information provides evidence of the degree to which a training programme is affecting financial business results. In other words, ROI data provide 12

CHAPTER 1: THE NEED

FOR

ROI

IN

TRAINING

summary information about the value of training and development interventions to the organisation. Evaluating training programmes in terms of ROI enables HRD practitioners to provide line management with hard evidence about the direct financial impact of training investments. Only after a complete evaluation process, using all four Kirkpatrick levels but emphasising level 4, can HRD managers show the value they are adding to the organisation. Your ability as an HRD manager or training consultant to calculate and report on ROI to line management will have a major impact on your credibility in your organisation or the organisations you are consulting with. A typical ROI process is as follows: ROI PROCESS Evaluation planning

Develop objectives of solution Develop evaluation plans and baseline data

Data collection

Collect data during solution implementation Collect data after solution implementation

Data analysis

Isolate effects of solution Convert data to monetary value Capture costs of solution

Calculate ROI

Source: Adapted from Phillips (1999)

13

Reporting

Generate impact study

Identify intangible measures

MEASURING ROI IN TRAINING

Calculating ROI requires a process model such as the one indicated above. The various elements of evaluation – such as the objectives of evaluation, the different levels of evaluation and the design instruments for evaluation – form the specific data-collection plans. Data collection tools such as questionnaires, customer surveys, performance data, etc. can be used to collect data for the evaluation. Data must be collected before and after the training intervention in order to determine its impact. After the data has been collected, it must be converted into monetary values. According to Phillips (1996), this requires a direct conversion of hard data such as quantity, quality, cost or time. This is a much easier task for learning programmes where the data is apparent, such as technical training. For "soft" data the process is more complicated because it is not always possible to provide a direct measurement for a soft-skills programme, such as a conflict-management course. However, some techniques can be used to place values on improvements, such as the following: z z z z z z

Historical costs Supervisor estimation Management estimation Expert opinion Participant studies External studies.

The next step is to calculate the costs for the programme. The ROI formula is the annual net programme benefits divided by programme costs, where the net benefits are the monetary value of the benefits minus the costs of the programme. The simplified ROI formula is as follows:

ROI (%) = benefits – costs x 100 costs

14

CHAPTER 1: THE NEED

FOR

ROI

IN

TRAINING

Calculating the ROI ratio will answer this important question: Did the financial value of the training results exceed the cost for the training programme? If there is a positive ROI, the training practitioner can clearly show the financial benefits of the training intervention. If not, he or she is in trouble! Although the above framework provides a simplified version of the ROI process, we will explore a more comprehensive and holistic approach to ROI. Our approach to ROI will be different from the American practice in the sense that we will be focusing on integrating ROI with performance management as well as key components of the South African skills development legislation and NQF requirements such as unit standards, learning outcomes and credits. This approach will ensure that we integrate international best practice with unique South African circumstances so that the needs of local employers can be met.

PITFALLS TO AVOID All new training systems have potential pitfalls. ROI is no exception. However, to make ROI work, the correct application of the ROI process and methodology is of utmost importance. It is therefore important to avoid the following pitfalls when embarking on a ROI project: z

While training managers are often excited and even over-enthusiastic to show the results of ROI, they must realise that there is no quick fix to ROI. The whole scientific and professional approach to ROI measurement must be correctly applied.

z

Be careful not to compare apples with pears. The correct comparisons must be made using the correct data for the correct purpose. It may not make sense to compare the ROI of a technical training programme with that of a leadership development programme. Compare apples with apples!

z

Trying to implement ROI without the proper knowledge of the science and practice of ROI may be disastrous. Get the neces15

MEASURING ROI IN TRAINING

sary training and exposure for the correct application of ROI. z

A lack of measures will make ROI very difficult if not impossible to determine.

z

It is possible to manipulate ROI data in order to show a good ROI for the sake of protecting your self-interest or position. Perhaps a strong code of ethics may alleviate this problem.

z

The one-person show must be avoided. Remember that ROI can help you to become a real business partner and that will not be achieved if you try to implement ROI on your own. Work with line management and involve them in the process.

z

Another pitfall to avoid is to fail to consider the effect of nontraining activities on your ROI measurement. Take the example of two learners who attended the same module. Back at the workplace, the one employee has a very supportive supervisor as opposed to the non-supportive supervisor of the other employee. This could have an adverse impact on the ROI calculation. The environment needs to be conducive to learning to have a positive impact on ROI. A negative impact on ROI cannot therefore always be attributed to training adding no value.

GUIDELINES FOR IMPLEMENTING ROI Implementing a ROI exercise in a company is usually a major paradigm shift on the part of training practitioners. Most of them would previously have focused on reaction evaluation and learning evaluation. Very few would have done a behaviour evaluation, and there is even less chance that they would have done any ROI projects! In the light of this, it is important to follow guidelines for the effective implementation of a ROI exercise: z

Build capacity for ROI by doing thorough research and benchmarking.

z

Train training staff to understand ROI. 16

CHAPTER 1: THE NEED

FOR

ROI

IN

TRAINING

z

Quantify information before the training in order to obtain a baseline (e.g. number of accidents, customer complaints, etc.).

z

Set specific targets for evaluation at all levels (Kirkpatrick).

z

Decide on specific measurements.

z

Allocate resources for ROI.

z

Integrate ROI with performance management to ensure that skills gaps are addressed.

z

Develop a culture of measurement and accountability among training staff.

z

Start with only one course as a pilot programme to practise ROI skills.

z

Focus on individual programmes to simplify the process.

z

Communicate results to training staff and the whole organisation.

z

Celebrate ROI success stories in a very visible way.

z

Design improvement plans for training programmes in order to increase ROI.

z

Once ROI results are available, use the data to market future learning programmes.

OUTLINE OF THE BOOK While our first chapter attempted to put ROI in perspective by clarifying the need for ROI, the remainder of this book will guide you through the full application of ROI in the workplace. Whether you are a manager, consultant, human resource practitioner, training professional or skills development facilitator, you will see that the practical approach adopted will make it easy for you to internalise the content in your own environment. 17

MEASURING ROI IN TRAINING

Chapter 2 will explore the crucial link between performance management and ROI. In fact, we will argue that the need for a training programme should be identified in your performance management system, especially if you use the balanced scorecard approach to performance management. This chapter will provide you with an overview of the process to flow from performance management and job weighting to linking competencies to a job description according to key performance indicators. Chapter 2 will indicate the use of performance measurement as an organisational tool to develop and measure current competencies against desired competencies as the baseline for measuring ROI. We explore the important link between key performance indicators and the unit standards as a crucial component of ROI. In Chapter 3 we shall look at how unit standards can be broken down to equate to the time needed to learn and achieve competency according to the NQF credits. Identified skills are aligned to the strategic needs of the organisation and included in the Workplace Skills Plan. Chapter 4 will answer the question of how assessment should be done to establish pre- and post-performance, in other words how narrowing the skills gaps have an influence on the ROI measurement. We will determine the cost of assessment versus the value added by demonstrating current performance to determine what the rate of ROI is in relation to the transfer of learning. In Chapter 5 we shall discuss the input costs and benefits needed for ROI. Here we will indicate how training costs such as the gap analysis, design and development, facilitation costs, assessment, evaluation, and fixed costs and benefits should be calculated. The last chapter will conclude with an outline of the process to measure the transfer of learning when calculating the ROI as a result of the impact of training. Here we will attach a rand value to all the processes followed in the previous chapters. Evaluation is done by establishing input, the processes followed as well as the value of the outputs. 18

CHAPTER 1: THE NEED

FOR

ROI

IN

TRAINING

The ROI analysis process will be determined in the following areas of impact: z z z z z z

Conducting a gap analysis The design and development of learning material Facilitation and learning fees Assessment of outcomes Evaluation of the changed behaviour Fixed costs.

The structure of the book is based on the steps according to the following process flow that will be used in a step-by step process to measure and calculate the ROI in training. The same process is linked to the electronic spreadsheets that are discussed in detail in Chapter 5 with step-by-step explanations. Return on Investment Process Flow 2. Performance Management z Gap analysis z Skills audit z Job description z KPIs z Competencies z CCFOs z BSC

1 . The need for ROI z Strategic objectives z Legislation z Benefits z Resistance

Cost 6 . Evaluation z Measurement z Cost of learning z Reports z Review

Learner

3. Skills Implementation z Unit standards z Time frame z Credits z Budgeting z WSP

Benefit

5 . Calculate ROI z Gap analysis z Design z Facilitation z Assessment z Evaluation z Fixed costs

4. Assessment z Pre- and post z Criteria z Change of behaviour z Competence z Assessment tools

19

MEASURING ROI IN TRAINING

HOW TO USE THIS BOOK This book is not a story about ROI. If you expected a story, go back to the bookshop and get one! Rather, this is a practical hands-on book that will help you to apply the principles and practice of ROI at work. Read through each chapter to grasp the principles and guidelines and make notes of some of the important issues you would like to work on. A good idea could be to have a notebook ready to start drawing up your own plan for your organisation as you work through the book. We realise that each company is unique. Moreover, each training programme is unique. While the principles and basic ROI methodology is generic and can be applied at any workplace, we do recognise that some level of customisation will be required by the reader. However, we also realise that human resource and training managers in South Africa are busy people who implement a number of training programmes simultaneously. To save time and effort, we have included a CD-ROM that you can use with some of the most important spreadsheets to make it easy for you to simply insert your own training costs so that you can make ROI calculations with the click of a few buttons! Furthermore, if you need online help, you will be able to access the ROI website for more information and user-friendly tools to find the answers to all your ROI needs.

www.roionline.co.za

20

CHAPTER 1: THE NEED

FOR

ROI

IN

TRAINING

CONCLUSION To calculate ROI for training is much easier than you think. A simple process approach can be used. What is important is that training practitioners must commit themselves to collecting all the necessary data – information that can be measured. There are, however, important pitfalls to avoid and guidelines to follow. Once the ROI results are available, companies will be able to determine the financial value of training. This process can assist them to make important decisions about various training interventions, such as the value of leadership development, internal vs. external training, e-learning vs. traditional training etc. ROI provides training managers with an opportunity to measure what they do and to make better decisions based on facts and quantifiable information.

21

MEASURING ROI IN TRAINING

22

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

CHAPTER 2

COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

If it can't be measured, it can't be managed. (Peter Drucker)

23

MEASURING ROI IN TRAINING

Why use performance management to measure ROI? Intellectual capital is arguably the most, or one of an organisation's most valuable assets. The development of employees is vital to the survival and growth of the organisation. The development and growth of employees should be seen as a process that is the responsibility of the individual and management aligned to the initiatives of the Human Resources department and the achievement of strategic goals for the organisation. The return on investment (ROI) is measured to establish the benefit to the individual and the organisation. A rand value is attached to the time and effort spent on the transfer of learning to measure the impact as a result of the changed behaviour. Measurement is done to establish the extent to which the organisation will benefit due to the changed behaviour after learning has taken place. Measuring the impact of training will aid the organisation to realise the value of the employee due to the increased intellectual capital as well as the benefits of the money spent on the training budget. A scientific process will be followed to measure the impact on the business by measuring the value of the outputs delivered in terms of training expenditure and benefits. Due to the impact of legislation in the form of the SAQA Act, Skills Development Act, Skills Development Levies Act and the Employment Equity Act, organisations are expected to take ownership of skills development in the workplace. Training budgets have increased as a result of the skills requirements to ensure industry ownership. In spite of a lot being said and written on ROI, very few organisations actually measure the ROI in training. The process is perceived to be too complicated and to require too much effort, especially in terms of measuring life skills, e.g. communication skills. In order to measure ROI, data needs to be collected that could be evaluated to calculate the ROI in training programmes. The information is used as evidence to determine the extent to which the financial results affect the bottom line as a result of training. 24

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

The process of measuring ROI starts from the analysis of the employees' job roles in order to source, design, develop and facilitate the learning modules, assessment and evaluation of the impact as a result of the training completed. The job role is analysed as a collection of tasks that are performed. Part of the performance improvement strategy is the assessment and evaluation of current competencies. Performance management is the first step in the process from identifying the skills gap to actually measuring the return on investment as a result of the changed behaviour. A paradigm shift needs to take place, as the culture of the organisation does not always allow for new methods of doing things. The intention to measure ROI should be communicated to the rest of the organisation, and not be seen as an effort by HR, conducted in isolation to the training initiatives. Once the ROI is measured on one specific programme, and the results made known to the organisation, other modules can follow until the ROI has been calculated on all the modules in the training department. Performance management will be used as an organisational tool to measure and monitor competencies as the basis for developing future performance through measuring the benefit to the organisation as well as the individual. Performance management allows for a twoway, non-threatening communication process between employees and management. Performance management enables learners to see the connection between individual aspirations linked to organisational effectiveness. Learners are able to take control of their own learning as they know what is expected of them as well as managers who take accountability and control of the process of learning new skills and acquiring new knowledge. The process is used to generate appropriate solutions and responses to the learners' developmental needs. Performance management is often the only form of feedback both to 25

MEASURING ROI IN TRAINING

learners and management regarding the status of the learners' level of competence. The purpose of appraisal feedback is the development of the learner. To enable the learner to achieve the desired competencies, the learning requirements need to be identified. This entails job-specific competencies as well as interpersonal skills reflecting generic abilities that are embedded in the critical cross-field outcomes (CCFOs); communication skills, problem-solving skills and other related skills. Individual objectives are discussed between learners and their managers to set specific targets to be reached by a certain date. The targets are 'SMART', i.e. specific, measurable, achievable, realistic, and time-specific. Identifying goals and objectives is part of achieving organisational effectiveness. Change needs to be brought about by achieving change in individual behaviour that in turn will affect organisational performance. Each individual will have a personalised career plan that allows sufficient time for achieving the objectives identified by both parties. Performance can be monitored through regular performance review meetings that will allow for extra time for employees who do not perform at the desired level of competence, or have not reached the desired outcomes. Under-performance can be detected and corrective action can be taken in a pro-active manner. Superior performance can be identified and rewarded. A top-down approach is needed to instil a performance culture where measurable outputs that are linked to the strategic objectives of the organisation are identified. Performance management is used as part of the criteria for performance remuneration. Reward should be based on the outputs of the job according to the level and grade of the job in the organisation, and not according to job titles that vary from organisation to organisation. In a 21st-century organisation the emphasis is on measuring individual performance aligned to business performance that flows from the business strategy and is aligned to the strategy. 26

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

The holistic view on performance management includes identifying learning opportunities, training and development facilitation, assessment of competence and individual career pathing, Through rewarding high performers, high-calibre staff is not only attracted, but can be retained by the organisation as well. By pro-actively planning for training needs, the organisation is able to more accurately plan for estimated expenses in terms of the training budget. Individual responsibility is a requirement to enable the organisation to become a learning organisation where learning is seen as an ongoing process and not as a once-off event which is solely the responsibility of management. Through this process the learning culture of the organisation can be reinforced.

PERFORMANCE MANAGEMENT PROCESS A systematic process of performance management is needed to analyse gaps in performance and competencies. The following steps can be used to implement performance management as a tool to analyse gaps in performance. The performance management process will be conducted according to the process flow on the next page:

27

MEASURING ROI IN TRAINING

PERFORMANCE MANAGEMENT PROCESS FLOW Step 1: Determine what data needs to be collected to conduct a gap analysis Step 2: Identify key performance indicators of the job Identify interpersonal skills of the job Step 3: Attach a weighting to competence

Step 4: Determine outputs to be delivered Step 5: Identify unit standards and link assessment criteria to output Step 6: Determine the time needed to accomplish competence Step 7: Performance management document is completed by the appraisee and appraiser Step 8: Performance management review meeting

Step 9: Compile skills development matrix The steps can now be analysed as follows:

STEP 1: DETERMINE

WHAT DATA NEEDS TO BE

COLLECTED TO CONDUCT A GAP ANALYSIS AND ASSESS COMPETENCE

It is essential that all managers and employees are prepared to support the effective implementation of performance management. Please answer the following questions to gauge your readiness to use the first step in the ROI process, in other words to conduct a competency analysis to determine potential gaps in performance. 28

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

1

Performance management self-assessment questions Do you conduct performance management meetings quarterly or at least bi-annually?

2

Do you have detailed job descriptions for all employees?

3

Have the key performance indicators been identified for all job roles?

4

Have the competencies been weighted?

5

Have measurable objectives been identified?

6

Do you regularly assess performance competencies?

7

Are the competencies aligned to business needs?

8

Are the objectives aligned to the organisation's bottom line?

9

Are you aware of which processes and activities add value to the bottom line?

10 Do you measure the individuals' contribution to the bottom line? 11 Do you view performance management as a strategy to develop and retain talent? 12 Does the culture of your organisation support performance management? 13 Does each individual have a personal development plan? 14 Have you benchmarked trends on business performance management? 15 Is your department ready to assess performance management? 16 Is your organisation ready to assess performance management? 17 Have you evaluated the value added to the corporate score card through training interventions when assessing performance management? 29

Yes N o

MEASURING ROI IN TRAINING

Did you have many “no” answers? We will guide you in a step-bystep process which will enable you to use performance management to measure, monitor and manage performance. Now that you have determined your score, the performance improvement process can begin. Determine what data needs to be collected to implement the first step in the process of determining the learner's competencies. Various methods are available in the form of a needs analysis, a skills audit, surveys, review meetings or feedback from various sources.

TIME SPENT ON MEASURING PERFORMANCE MANAGEMENT AS PART OF THE PROCESS OF CALCULATING THE ROI IN TRAINING Whether performance management is done for the first time or at regular intervals, the time spent should be recorded to enable the accurate measuring of ROI in training. Attach a time to each activity to calculate the total time spent on the performance management process to determine the current competency of the employee. Activity

Time started

Identify key performance indicators and interpersonal skills of the job Attach a weighting to competencies Determine outputs to be delivered Identify unit standards or training modules and link assessment criteria to outputs identified Performance management document completed by appraisee and appraiser

30

Time Total time completed spent

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE CONTENTS MANAGEMENT

Appraiser/appraisee performance review meeting to finalise rating score Assessment of outcomes after time spent on learning Feedback after assessment

TRAINING PROCESS SEQUENCE The steps explained are covered in detail in Chapter 5 where an opportunity is provided to input all the relevant data according to the following sequence and breakdown of the various activities: Gap analysis

Design & development

Needs analysis Review meetings Skills audit Surveys Learning material

Software

e-Learning

Facilitation fees

Trainer cost

Assessment

Assessor cost

Evaluation Fixed costs

Learner cost Moderation cost Performance Floor space Equipment Maintenance Internet

31

Development cost Purchase price Outcomes-based conversion Mapped to unit standards Material duplication Licensing fee Development cost Purchase price Licensing fee Web development Convert material Design exercises & activities Hourly fee Travel costs Accommodation costs Hourly fee pre-assess Hourly fee post-assess Hourly rate Hourly rate Time spent Building depreciation Rental of floor space Depreciation rental Equipment & building Monthly fee

MEASURING ROI IN TRAINING

WHY DO WE NEED

TO TRAIN PEOPLE?

The National Qualifications Framework and training legislation drives local trends in human resource development and national HRD needs. One of the purposes of the Skills Development Act is to increase the levels of investment in education and training in the labour market and to improve the return on that investment. It is the responsibility of each and every training manager, human resources manager or skills development facilitator to ensure compliance to national legislation. Compliance to legislation is part of the key performance areas of your job. Measuring the ROI in training should also be part of the key performance areas of your job. To train and not measure the ROI is a waste of time and the organisation's money. The credibility of line managers is at stake if the ROI in training is not measured.

LEARNING ALIGNED TO THE PRINCIPLES OF THE SOUTH AFRICAN QUALIFICATIONS AUTHORITY (SAQA) ACT AND THE SKILLS DEVELOPMENT ACT The SAQA Act was promulgated in 1995 and is responsible for overseeing the development and implementation of the National Qualifications Framework (NQF). SAQA is a statutory body appointed by the Minister of Education in consultation with the Minister of Labour. SAQA is the highest qualifications authority in South Africa. It is responsible for establishing National Standards Bodies (NSBs), and the delineation of NSBs into sub-fields, each with a Standards Generating Body (SGB). The main functions of SAQA are: Standard setting and developing: z z z

Register and accredit bodies responsible for establishing education and training standards or qualifications. Register national standards and qualifications. Monitor the observation of rules of combinations for qualifications. 32

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

z

Ensure that the standards registered are internationally comparable.

Quality assurance: z z

Register and accredit Education and Training Quality Assurance (ETQA) bodies. Audit the quality of education and training.

To add value to the organisation, we need not only to train employees, but also to ensure the training delivered is according to the quality criteria as required by SAQA when we want to accurately measure the ROI in training.

OBJECTIVES

OF THE

NQF

These are to: 1. 2. 3. 4. 5.

create an integrated national framework of learning achievements. facilitate access to, and mobility and progression within education, training and career paths. enhance the quality of education and training. accelerate the redress of past unfair discrimination in education, training and employment opportunities. contribute to the full personal development of each learner, and the social and economic development of the nation.

In view of the above we can only but start by providing learning opportunities to the individual, and ensuring we meet strategic objectives. It is evident from the objectives of the NQF that the measurement of ROI is needed to determine whether the objectives of training have been achieved. The following table illustrates the steps that need to be followed from determining the job role for a specific position through all the steps of identifying KPAs, weighting competencies, determining outputs, sourcing training material, pre- and post-assessment, evaluation and calculating the cost of learning. 33

MEASURING ROI IN TRAINING

ranslate performance to rand value ROI – T Translate Step

Item

1

Job description

2

Identify: Key performance indicators (KPIs) Critical cross-field outcomes (CCFOs)

Action Determine the job role for e.g. Contact Centre Agent Determine what the KPIs are: Log and receive calls Provide solutions Knowledge of products and systems Administrative duties Use computerised systems

z z z z z

3

Weight competencies Attach a weighting to the competencies

4

Determine outputs

5

Allocate unit Link job competencies to the assessment standards or training criteria per output modules

6

Assessment: pre- and Determine the gap between current post and desired performance

7

Establish the learning/training time period

8

Conduct performance Performance management document management completed by appraisee and appraiser

9

Performance management review meeting

Allocate performance according to a 5-point scale, or a points-per-job scale. Feedback could be obtained from 360degree feedback and formal assessment

10

Assessment of outcomes

Formative and summative assessment

Determine the outputs per job roles

Allocate time to reach competency per each outcome, or combined outcomes per credit value. Determine the training time period. i.e. 160 hours training = 9 600 minutes. Therefore 12 outcomes = 800 minutes each, or 13 hours and 13 minutes per outcome

34

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

Compare the actual to the required competency level

Required vs. Actual Performance Level Required

Actual

KPI’s

Use computer systems

Administrative duties

Knowledge of products & systems

Provide solutions

5 4 3 2 1 0 Log & receive calls

Level

11

12

Calculate the cost of learning

Calculate ROI by doing a cost-benefit analysis by making use of Excel spreadsheets

13

Evaluate ROI

Attach a rand value to the achievement of strategic and individual objectives

COMPETENCY ANALYSIS A gap analysis needs to be done to determine the current level of learning before the performance management review can be done to implement a performance improvement strategy and accurately measure the ROI in training. To be able to assess the learner's competency level, each job is profiled in terms of individually weighted key performance indicators as agreed upon by the employee and his/ her manager with a view to linking performance to outputs. The indicators are linked to work-related competencies and the critical crossfield outcomes from the relevant unit standards or work processes. We need to not only identify job-related competencies, but also generic abilities that are represented in the following critical cross-field outcomes as provided by SAQA:

THE CRITICAL CROSS-FIELD OUTCOMES Critical cross-field outcomes are generic and cut across all areas of learning. They are not specific to one subject or field.

35

MEASURING ROI IN TRAINING

They include: z

z z z z z z

z

identifying and solving problems in which responses display that responsible decisions using critical and creative thinking have been made. working effectively with others as a member of a team, group, organisation, and community. organising and managing oneself and one's activities responsibly and effectively. communicating effectively using visual, mathematical and or/ language skills in the modes of oral and/or written persuasion. collecting, analysing, organising and critically evaluating information. using science and technology effectively and critically, showing responsibility towards the environment and health of others. demonstrating an understanding of the world as a set of related systems by realising that problem-solving contexts do not exist in isolation. contributing to the full development of each learner and the social and economic development of the society at large by making it the underlying intention of any learning programme to make an individual aware of the importance of: - reflecting and exploring a variety of strategies to learn more effectively. - participating as a responsible citizen in the life of local, national and global communities. - being culturally and aesthetically sensitive across a range of social contexts. - exploring education and career opportunities. - developing entrepreneurial opportunities.

EMOTIONAL INTELLIGENCE (QUOTIENT) (EQ) AND THE CRITICAL CROSS-FIELD OUTCOMES Emotional intelligence is used as a source of human energy to connect with people and attune us to our own and other people's emotions. Emotional self-awareness is a pre-requisite for effective selfmanagement that in turn enhances broader social skills. 36

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

Emotional competencies are learned abilities that demonstrate that we have the potential to become skilled at relationship management and social awareness. Self-knowledge and awareness are the most essential and fundamental elements required to develop emotional abilities. "Fortunately, scientists now consider EQ a learnable intelligence, one that can be developed at any time and any age" (Goleman, 1995). Organisations need to leverage their intellectual – and emotional capital for sustainable business advantage. As far as ROI is concerned, the following conclusion is quoted from Cost and Management Accounting, November 1998: "The most enduring benefit is that the more abstract the intangible investment, the more difficult it is to copy: perhaps the most intangible investment of all is a corporate culture which encourages employees to embody their skills and innovation within processes to deliver products and services that delight their customers, which is a key objective of knowledge management; succeed in this and the cash and market value added will most likely flow in abundance." Emotional intelligence forms part of the critical cross-field outcomes where the focus is on generic abilities that focus on intra-personal and inter-personal skills.

STEP 2: IDENTIFY

THE KEY PERFORMANCE INDICATORS AND OF THE JOB

THE INTERPERSONAL SKILLS

The second step is to identify the key performance indicators of the job, e.g. to log a call or to respond to a query as part of the job of a call centre agent. Certain interpersonal skills are needed to be able to perform these functions, e.g. communication skills. The KPIs identified for a contact centre agent are: z z z

Log and receive calls. Provide solutions. Knowledge of products and systems. 37

MEASURING ROI IN TRAINING

z z

Administrative duties. Ability to use computerised systems.

In the next table the following process was followed: z z z z z z

Each performance dimension of the job is plotted on a spreadsheet. A weighting is attached according to importance of the function. The appraiser rates the employee individually. The appraisee rates him/herself individually. During the review meeting an agreed rating is allocated. The rating is calculated according to the employee’s overall contribution to the organisation.

Key performance dimension

Weighting Appraiser Appraisee Agreed Calculated rating rating

Log and receive calls

5

Provide solutions

3

Knowledge of products and systems

5

Administrative duties

4

Use computerised systems

5

Communication skills

4

Conflict resolution

3

Plan, organise and control

3

Time management

4

Problem-solving skills

3

Team work

3

Decision-making skills

3

Social and economic awareness

2

Exploring education & career opportunities

2

Cultural and aesthetic awareness

2

38

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

STEP 3: ATTACH

A WEIGHTING TO THE COMPETENCIES

Once the key performance indicators are identified and linked to the competency requirements of the job specification, a weighting is attached to the competencies. The performance indicators provide standards against which to measure the job outputs. The employee and manager agree on the outcomes before the performance appraisal process can continue. Once each job linked to the job title has been weighted according to outputs the performance appraisal can commence. The job outputs are weighted according to the overall contribution to organisational effectiveness and importance per output.

CRITERIA FOR THE WEIGHTING OF THE KEY PERFORMANCE INDICATORS

The job-related key performance indicators as well as the critical crossfield outcomes that cover interpersonal skills are weighted according to the following criteria: 0 1 2 3 4 5

= = = = = =

not applicable insignificant moderately important important very important critical

STEP 4: DETERMINE

THE OUTPUTS TO BE DELIVERED

The outputs need to be measurable, e.g. number of calls responded to, or number of queries resolved within an allocated time frame. Examples include the folowing: z z z

Calls not answered by a computerised system to be answered within three rings. 95% of all telephone calls to be answered. All telephone calls answered to be logged on the computerised system. 39

MEASURING ROI IN TRAINING

z z z z

Respond to customer queries within 24 hours. All queries not responded to on first contact to be referred to the support office. Should the system be off-line, calls received to be recorded manually. Daily, weekly and monthly status output reports to be printed at the required intervals.

You can use a performance management system such as the balanced scorecard approach to determine the outputs to be delivered. An example of an output could be to answer 160 phone calls per day.

LINK OUTPUTS TO THE BALANCED SCORE CARD (BSC) Link the organisational strategy aligned to the vision, mission and values of the organisation to the balanced score card processes, namely; learning and growth, internal processes, customer perspective and financial objectives. The emphasis is on converting the overall organisational goals into specific objectives for organisational units and individual members. We have to ask ourselves what our principle objectives are, what products and services we offer, who our clients are and in which market we are competing. What do we need to deliver to meet stakeholder satisfaction?

HOW

DO WE DEFINE THE BALANCED SCORE CARD?

This is a performance management instrument that is used to translate our strategic objectives (business plans) into measurable targets for the organisation as a whole, for all business units, teams and individuals through a balanced approach. We need to ask ourselves the following questions for each of the four categories: z z

At business unit level: Who are our stakeholders and what do they expect from us? Department and team level: Who are our stakeholders and what do they expect from us? What contribution does the team or department have to make to achieve organisational effectiveness? 40

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

z

Individual level: How can I contribute to the objectives of the team and business unit through my contribution?

CUSTOMER PERSPECTIVE z z

What products and/or services do we need to deliver for the organisation to achieve its financial objectives? What do our customers expect from us and what are their values?

Customer measures can be determined by the outcomes of internal processes, but can also be drivers of financial measures.

INTERNAL PERSPECTIVE z z z z

What business systems and processes must the organisation improve on or excel at to deliver on customer-service objectives? Which internal processes do we need to improve on or excel at to satisfy our shareholders and customers? What infrastructure do we need in place that can add value? What practices and processes must be revised or improved on? What can I improve on or excel at to contribute more effectively to overall organisational effectiveness?

LEARNING AND GROWTH PERSPECTIVE z

z z

How do we identify learning and growth opportunities and requirements to assist our employees to learn and develop their skills to respond to future challenges? How do we ensure that we continuously grow and stay competitive through the development of our people? How do we create a culture of learning and growth to become a learning organisation where learning forms part of our everyday lives and is not seen as a once-off event?

The learning and growth dimension of the balanced scorecard is one of the areas where managers struggle to obtain quantifiable information to use as measurements. ROI measurement assists in this process as different ratios can be used as part of ROI calculations. 41

MEASURING ROI IN TRAINING

FINANCIAL PERSPECTIVE z

z

What financial objectives should the organisation set for itself to stay competitive and ensure long-term profitable, sustainable growth? How can the business unit, team and the individual contribute to achieving the financial targets set by the organisation?

THE SCORECARDS ARE COMBINED TO MEASURE INDIVIDUAL, DEPARTMENTAL AND THE BUSINESS UNIT’S PERFORMANCE From the input of these perspectives the organisational strategy can be both refined and operationalised to the benefit of all stakeholders. The organisation's vision and strategic objectives can be coherently communicated to all employees. Having identified the divisional and departmental drivers and key deliverables through a balanced score card approach, the personal score card for individuals can then be formulated. This reflects direct contribution to overall organisational effectiveness through measurable outputs. The outputs identified are transferred to the performance management appraisal system that forms part of individual career pathing. Individual assessment of performance against targets is conducted and a performance rating is recorded. Performance ratings are used as part of the criteria to determine remuneration increases. The combined individual scorecards become the measure for the team's performance. The combined scorecards from the various departments form part of the measure for the business unit's performance. Management by objectives (MBO) has similar objectives to the balanced score card, but the power of the balanced scorecard lies in dissecting the organisational strategy into four important components.

42

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

STEP 5: IDENTIFY

UNIT STANDARDS AND LINK

ASSESSMENT CRITERIA TO OUTPUT

The fifth step deals with the identification of unit standards and the link to relevant assessment criteria.

PERFORMANCE MANAGEMENT AND UNIT STANDARDS The following unit standard is an example against which to measure competence: Unit Standard Number: 2.7 Title of Unit Standard: Meeting performance standards within a contact centre context NQF Level: 3 Credit Value: 16 Field: Business, Commerce & Management Studies Specific Outcomes

Assessment Criteria

1

Using a computerised system

1.1 Relevant data is input accurately 1.2 Data is retrieved accurately 1.3 All logging procedures are correctly used as per company-specific systems 1.4 All industry-specific security and confidentiality requirements are complied with 1.5 Specific statistical data is understood (range: may include but not be limited to quantity, duration, wrap, hold and dropped contacts)

2

Responding to calls or other forms of communication (range: inbound or outbound)

2.1 Company-specific time targets are met (range: may include but not be limited to quantity, duration, wrap, hold and dropped calls) 2.2 Calls and customers are responded to in the appropriate language or medium. 2.3 Customers' requests are understood 2.4 Customers' requests are responded to according to company-specific procedures 2.5 Other forms of communication are considered and applied using standard industry practices

43

MEASURING ROI IN TRAINING

3

Following up customer queries

3.1 Electronic communication systems are utilised effectively (range: includes but is not limited to faxing, e-mailing, printing, photocopying, telephone) 3.2 Manual administrative tools are accurately completed as per company-specific procedures and time frames

In the above example the outcomes were equally divided by the time available. This will not always be the case; different outcomes will require a different time allocation per outcome.

STEP 6: DETERMINE

THE TIME NEEDED TO ACCOMPLISH COMPETENCE

HOW TO CALCULATE TIME For instance, 1 credit equals 10 notional hours. Therefore, 16 credits taken from our example above, would amount to 160 notional hours of learning. If you multiply the 160 hours by 60 minutes per hour you get 9 600 minutes which is the total number of minutes of training. If you divide the 9 600 minutes by the 12 outcomes, this will result in 800 minutes per outcome. Furthermore, by dividing 800 minutes by 60 minutes per hour you arrive at 13,33 hours, i.e. 13 hours and 13 minutes learning per outcome. The learning time required per outcome is allocated according to the specific assessment criteria for each outocome as illustrated on the following table:

44

45

Specific Outcome 3

Assessment Criteria

800 minutes

800 minutes

800 minutes

800 minutes

800 minutes

800 minutes

Manual administrative tools are accurately completed as per company- specific procedures and time frames

Customers’ requests are responded to according to companyspecific procedures

Customers’ requests are understood

800 minutes

800 minutes

Calls and customers are responded to in the appropriate language or medium

800 minutes

All industryspecific security and confidentiality requirements are complied with

Assessment Criteria

All logging procedures are correctly used as per company- specific systems

Assessment Criteria

Company- specific time targets are met (range: may include but not be limited to quantity, duration, wrap, hold and dropped calls)

800 minutes

Relevant data is input Data is retrieved accurately accurately

Assessment Criteria

Electronic communication systems are utilised effectively Following up (range: includes but is customer queries not limited to faxing, emailing, printing, photocopying, telephone)

Responding to calls

Specific Outcome 2

Using a computerised system

Specific Outcome 1

Learning Outcome

800 minutes

Other forms of communication are considered and applied using standard industry practices

800 minutes

Specific statistical data is understood (range: may include but not be limited to quantity, duration, wrap, hold and dropped calls)

Assessment Criteria

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

MEASURING ROI IN TRAINING

STEP 7: COMPLETE

THE PERFORMANCE DOCUMENTS

The next step is to complete performance documents. Allocate the current performance competence according to the following five-point rating scale, linked to the required performance weighting for the specific job: 1

=

2

=

3

=

4

=

5

=

Poor – unacceptable. This indicates that major improvements are needed as the results are only partially achieved and the speed of work and error rate is significantly below expectations. Below expectation – room for improvement. Results are achieved, but not to the expected standards. Speed of work and error rate does not meet output requirements. Good – effective level of performance meets expectations according to standards. Error rate and outputs are on agreed level. Very good – performance is above expectations and results are constantly achieved. Excellent – high level of performance, consistently exceeds expectations and achieves outstanding performance.

STEP 8: PERFORMANCE

REVIEW MEETING

Once the performance documents have been completed, a performance review meeting is conducted. An analysis is done using the job title as the starting point on the performance management developmental plan. The performance appraisal document is accessed by the appraisee using his/her unique employee number. Upon entering the employee number, the appraisee's job description will be displayed linked to the employee's employee number. This information is centralised in the organisation's database. Once the employee number and job title correspond, the process can proceed when the employee verifies his/her personal details: name, surname and e-mail address that will be used for confirmation once 46

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

the process is complete. The appraisee now has the opportunity to evaluate his/her own performance over the past six months, or a shorter period, depending on the intervals at which the organisation conducts the performance management review process. Once completed, the employee has the opportunity to review the weighting allocated with the option to make changes. Upon confirmation the system will generate an appraisee code to the appraisee that will be used during the review meeting with the relevant manager. The manager will be notified by the system that the appraisee has conducted the appraisal. The manager accesses the system using his/her own employee number and rates the appraisee. The score allocated by the appraisee is not known to the manager to avoid influencing the manager's decision. Once the manager has completed the appraisal document, the system will generate a notification and unique notification number to both parties informing them of the appraisal review meeting. The appraisal document cannot be reviewed prior to the appraisal meeting. The ratings allocated can be printed by the appraisee or appraiser from the “Report” section using their own employee number as means to access the system to be able to print either the appraisee report or appraisal report.

360-DEGREE MULTI-RATER FEEDBACK 360-degree multi-rater feedback systems have become an established tool to provide individuals with a clear view on their performance. The 360-degree assessment tool uses multiple raters to assess the competency of a single individual. Opinions are gathered from the person being assessed, peers, subordinates and their manager. Feedback from an external source, for example, a customer or supplier could also be obtained. The combined opinions are grouped together to provide an overall picture covering different aspects of the employee's performance. 47

MEASURING ROI IN TRAINING

The benefits of a 360-degree feedback system include the following: z z z z z z z

z z

Individuals receive feedback from a range of people at different levels. Confidentiality of ratings encourage feedback providers to give honest feedback. Feedback is provided against a competency standard or questionnaire. It reveals information that would not normally be captured. It can provide a strong incentive for self-directed learning and development. It is often top-level managers' only source of feedback It allows each member of the team to assess the performance of everyone else in the team. This makes members more responsible for their own development and for achieving the goals that were set for the team. The inclusion of multiple raters reduces the effect of individual bias. Greater accuracy increases the validity of the assessment.

BEST PRACTICE SUGGESTIONS These include the following recommendations: z z z z

z z

The purpose and process of multi-rater feedback must be well communicated to ensure optimal buy-in to the process. Employees should be able to contribute to the design of the feedback system. Appraisal feedback should be used for development purposes and not as punitive measures. Multiple ratings should be used for development purposes and not as the only measure to determine salary increases or incentive bonuses. The rating may be biased if remuneration is linked to the process. Participation by the raters should be voluntary. Only observable aspects of the ratee's job should be focused on.

48

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

MULTI-RATER CHECKLIST The multiple raters rate the employee on the same criteria against the job competencies and outputs. The checklist is compiled according to the same criteria that were used to rate the employee's key performance dimensions aligned to the weighted job competencies: Key performance

Excellent VeryGood Good

√

Log and receive calls

√

Provide solutions Knowledge of products and systems

√ √

Administrative duties Use computerised systems Communication skills Conflict resolution Plan, organise and control Time management Problem-solving skills Team work Decision-making skills Social and economic awareness Exploring education and career opportunities Cultural and aesthetic awareness

49

Below Poor expectation

MEASURING ROI IN TRAINING

THE PERSONAL DEVELOPMENT PLAN (PDP) During the appraisal review meeting the personal development plan will be completed by the appraisee's manager. Both parties will be able to view the individual's performance ratings. Based on the performance ratings allocated, developmental needs will be identified. The needs identified are linked to the training modules available internally or externally. The requirements will be populated on a PDP skills matrix according to business-specific skills, interpersonal competencies covering the critical cross-field outcomes, technical and leadership development needs.

PERFORM A GAP ANALYSIS BY COMPARING THE ACTUAL TO THE REQUIRED COMPETENCY LEVEL

Plot the current competency levels according to the specific outcomes against the required competency levels on a graph. The difference between the two levels is an indication of the gap in performance.

Level

Required vs. Actual Performance Level 5 4.5 4 3.5 3 2.5 2 1.5 1 0.5 0 Log & Receive Calls

Provide Solutions

Knowledge of Products & Systems

Administrative Duties

Use Computer Sytems

KPA's

POINT SCALE ACCORDING TO ORGANISATIONAL LEVELS Another option is to allocate points per job level according to the requirements of the job. The learner is then assessed according to a point system and matched against the required point scale that replaces the 1-5 scale that was used in the previous example. 50

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

Level

Description

Points

8

Group executive

320

7

Divisional executive

280

6

General management

240

5

Professional specialist

200

4

Technical specialist

160

3

Junior management

120

2

Clerical/administrative

80

1

Elementary skilled

40

Some organisations prefer the score to increase progressively from the elementary level towards the top level. In the example below the difference between the two top jobs is 60 points compared to the 40 points difference in the above example.

Level

Description

Points

8

Group executive

320

7

Divisional executive

260

6

General management

210

5

Professional specialist

170

4

Technical specialist

130

3

Junior management

110

2

Clerical/administrative

80

1

Elementary skilled

40

51

.

MEASURING ROI IN TRAINING

Required vs. Actual Performance Level Required

Actual

80

Level

60 40 20 0 Log & Receive Calls

Provide Solutions

Knowledge of Products & Systems

Administrative Duties

Use Computer Sytems

KPI's

Once the individual's performance level has been identified, it has to be compared to the organisational level required for the specific job. The next step would be to plot the learner's competence level against the required competency level for the specific organisation. These differences will highlight the gap in performance that was identified. This gap will be used as the source to allocate the training requirements on the individual's skills development matrix as explained below. Training will be delivered according to the skills needs that were identified using the gap analysis. During the performance review meeting, the required skills and competencies according to the job description for the specific position are identified and agreed upon by the employee and his/her manager. This process forms part of the performance management process that automatically generates the skills development matrix. The necessary training that will aid in the development of the learner's competencies is indicated on the skills development matrix together with the outputs and the date by which the learner should be able to perform at the required level.

STEP 9: COMPLETE

THE SKILLS DEVELOPMENT MATRIX

So far we have identified the KPIs and interpersonal skills necessary for the contact centre agent. After that we identified the relevant unit standard against which the learner will be assessed. The next step is to complete the skills development matrix. In this 52

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

example the competencies are allocated according to interpersonal skills and legislation, business-specific skills, technical, and leadership or supervisory skills. The outputs were identified and a time allocated by when the learner should be able to demonstrate competency. This matrix is used to source the learning interventions and material and estimate the training needs for the year ahead. The next step is to allocate the unit standards according to the same categories for interpersonal skills and legislation, business-specific skills, technical, and leadership or supervisory skills. The measurable outputs are indicated as well as the time by when the learner should be ready to demonstrate competence. If your organisation makes use of the balanced score card approach you can further allocate the unit standards according to the areas of customer service, internal processes, learning and growth, and the financial aspect. These activities are allocated to the customer service (C) area, aligned to the operational objectives (C1.1), and strategic objectives (C1) as per the balanced score card example on page 58. Additional learning is indicated under each area in addition to the required unit standards. The additional learning is indicated as (a), (b), etc. In many organisations the individual skills development matrix is an automatic process where the document is automatically completed once the computerised performance management document is completed. In the following examples the first document is the skills development matrix that contains the KPIs and interpersonal skills for the specific job. The second example is the matrix according to the balanced score card approach. The third example is an extract of the actual balanced score card displaying the operational and strategic objectives. 53

54

Cultural awareness

Education & career

Social & economic awareness

Decision making

Team work

Problem solving

Time management

Plan, organise, control

Conflict resolution

Communication skills

Diversity

Planning skills

Communication

Oracle

Business writing MS Word

Administrative duties

Computerised systems

Health benefits

Problem solving

MedQ system

Technical

Product & system knowledge

Provide solutions

Example 1: Skills development matrix Interpersonal BusinessJob Title and Description Skills & specific Legislation Skills Log & receive calls Leadership/ Managerial/ Supervisory

100% error-free letter

95% calls answered

Output 28th February

Date to be achieved by

MEASURING ROI IN TRAINING

55

2.2 Calls and customers are responded to in the appropriate language or medium

a) Knowledge of range of products & services to be able to respond to customer queries

2.4 Customers’ requests are responded to according to companyspecific procedures

2.3 Customers’ requests are understood

Customer Service

Interpersonal Businessskills specific skills

1. Using a Customer computer- Service ised system 2.1 Company2. Responding specific time to calls or targets are other forms met (range: of communimay include cation but not be (range limited to inbound or quantity, outbound) duration, wrap, hold and dropped calls)

Job title and functions Customer Service

Measurement

Date to be achieved by

28 th February a) Calls not answered by a computerised system, to be a) Knowledge of a) Team leaders to monitor answered computerised daily statissystem to log within 3 rings tics for the all calls 28 th February members of b) 95% of all received the team he/ telephone calls Internal answered she is Processes responsible 28 th February for c) All telephone calls answered 1.1 Relevant data Internal to be logged on is input Processes the computeraccurately ised system b) Liaise with 1.2 Data is reother depart- d) Respond to 28 th February trieved accumental customer rately managers queries within 24 hours

Customer Service

T echnical/ Leadership/ Technical/ computer skills managerial

Training matrix for a contact centre agent according to the BSC approach

Example 2: Balanced score card matrix

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

3. Following up on customers’ queries

Job title and functions

56

a) Effective communication between departments and divisions by following company procedures

Internal Processes

2.5 Other forms of communication are considered and applied using standard industry practices 2.6 Answer calls according to companyspecific greeting

Technical/ Leadership/ computer skills managerial

Measurement

Date to be achieved by

Internal Processes

e) All queries not 28 th February Learning & 1.3 All logging responded to on procedures are Growth first contact, to correctly used c) Take be referred to 1.4 All industryas per comresponsibilthe support specific pany-specific ity for own office security and systems development f) Should the confidential28 th February system be offity require- Learning & Growth line, calls ments are received to be complied recorded with b) Benchmark manually 1.5 Specific for best g) Daily, weekly 28 th February statistical practice and monthly data is against two status output understood other reports to be 3.1 Electronic organisations printed at the communicarequired tion systems intervals are utilised effectively (fax, e-mail, printing, photocopying, telephone)

Interpersonal Businessskills specific skills

MEASURING ROI IN TRAINING

Job title and functions

Technical/ Leadership/ computer skills managerial

57 a) Awareness of the job requirements re number of calls to be answered and the impact of lost calls on the cost centre and the organisation

Financial

Learning & Communicate Growth effectively in a second language Cross-skilling

2.7 Close calls in b) Adhere to 3.2 Manual accordance security administrawith the procedures tive tools are context of the regarding accurately call with a call confidentialcompleted as centre ity and per companypasswords specific procedures Learning & and time Growth frames

Interpersonal Businessspecific skills skills h) Meeting financial targets set for department to stay within budget

Measurement

Ongoing

Date to be achieved by

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

C1.1 T o provide z Calls not answered by a computTo appropriate, erised system to be answered effective and within 3 rings integrated z 95% of all telephone calls customer-care answered solutions and z All telephone calls answered to support to our be logged on the computerised customers system z Respond to customer queries z Provide within 24 hours appropriate, z All queries not responded to on effective and first contact to be referred to the integrated support office contact centre z Should the system be off-line, solutions and calls received to be recorded support to our manually customers z Daily, weekly and monthly status output reports to be printed at the required intervals

C1 To provide valueadded customer service and support

Activities

Operational objective

N O Strategic objective

Example 3: Actual balanced score card

58 Daily

Daily

Daily

Daily

Daily

z

Should the system be off-line, calls received to be recorded manually z Daily, weekly and monthly status output reports to be printed at the required intervals

1) Calls not answered by a computerised system, to be answered within 3 rings 2) 95% of all telephone calls answered 3) 100% of telephone calls answered to be logged on the computerised system 4) Respond to 100% of customer queries within 24 hours 5) 100% of queries not responded to on first contact, to be referred to the support office

T arget Measurement Target date

100%

95%

99%

93%

Three rings

Progress to date

MEASURING ROI IN TRAINING

CHAPTER 2: COMPETENCY ANALYSIS LINKED TO BUSINESS PERFORMANCE MANAGEMENT

CHECKLIST

UPON COMPLETION OF THE PERFORMANCE MANAGEMENT PROCESS

From the above discussion it is clear that the entire performance management process is needed to determine competency gaps. Please answer the following questions regarding the performance management process that was followed: Yes

Activity

No

Data has been collected from the skills audit, feedback and other sources Key performance indicators have been identified Interpersonal skills have been identified Competencies have been weighted Outputs have been determined Training requirements have been identified, and linked to outputs identified Time needed to accomplish competence has been determined Performance management document completed by appraisee Performance management document completed by appraiser Appraiser/appraisee performance review feedback meeting has been conducted to finalise rating score Skills development matrix has been printed for the personnel file If you have answered “yes” to all of the above questions, you can proceed to the next step in the process flow to measure the ROI in training. 59

MEASURING ROI IN TRAINING

In the next chapter we will look at identifying unit standards and/or training modules to assist the learner in demonstrating competence.

CONCLUSION In this chapter we had a look at the role of performance management as a tool to enable us to accurately measure the ROI in training. The steps were followed from performing a gap analysis to determining the current competency level, attaching a weight to the various job functions, determining the measurable outputs, identifying relevant unit standards or exit-level outcomes and the time needed to achieve these outcomes. Once the learner has been assessed, an individual career plan can be compiled and future targets set. The aim of performance management is not only to measure, monitor and manage performance, but also to foster a culture of life-long learning where learning forms part of day-to-day duties. These steps have to be followed in sequence to be able to accurately calculate the ROI to determine whether a change in behaviour has taken place as a result of the learning attended. Through the allocation of individual competencies according to the requirements of the balanced score card, the individual's contribution to the various areas of the business is measurable. Also measurable is the changed business performance that will have an impact on the bottom line. This is what training is all about: to bring about change in individual behaviour and performance and, as a result, increase the business performance for the organisation.

60

CHAPTER 3: IMPLEMENTING SKILLS DEVELOPMENT INITIATIVES

CHAPTER 3

IMPLEMENTING SKILLS DEVELOPMENT INITIATIVES

But skill will bring success. (Ecclesiastes 10:10)

61

MEASURING ROI IN TRAINING

In the previous chapter we had a look at the role of performance management in conducting a gap analysis, identifying the key performance indicators of a job, the competencies and the critical crossfield outcomes required for this specific job. We now need to identify the specific training modules that will assist in developing these skills to measure the ROI in training to establish whether the training that has been delivered has added any value to the individual's development and whether the benefits associated with training exceed the costs. Following the steps suggested in this chapter will result in a more realistic and effective ROI measurement. The human resource manager and skills development facilitator of the organisation are responsible for developing and submitting the workplace skills plan (WSP) for the organisation. The WSP is really an extract of the business plan for the organisation. We cannot plan any training interventions in isolation to the strategic imperatives of the organisation. The WSP is one of the requirements of the Sector Education and Training Authority (SETA) to enable an organisation to receive the maximum levy refund. The WSP is the formal document that is used to report on the training needs that were identified, the employee profile and the annual skills priorities, and how these are aligned to the skills needs for the sector and the national skills needs. For the organisation to be profitable, the best systems, structures, strategic enablers and processes may be available. However, the most important “s” is the staff that is required to be competent to actually implement the vision and mission of the organisation. What skills and leadership qualities do these employees need to execute the organisation's strategy? There are many ways to reach the top of an oak tree: start climbing, piggy-back on a squirrel, make friends with a big bird, sit on an acorn, use a ladder … We have identified which of these skills are needed to acquire new clients and retain existing ones, to sell the company's products and to develop new products and systems so that employees perform to the optimum of their abilities. However, what finances do we have available to have staff members trained and developed? It is all very well to perform a 62

CHAPTER 3: IMPLEMENTING SKILLS DEVELOPMENT INITIATIVES

gap analysis. If the organisation has not budgeted sufficiently for training purposes, we might as well forget about identifying the developmental needs as we will not be able to satisfy the needs of our stakeholders. The number of employees that attended training during the previous year can be used as a guideline to budget for the training expenses for the year ahead. Please complete the following questionnaire to establish whether a learning intervention will assist in narrowing the current skills gaps: Skills development self-assessment questions 1 Will training assist the learner to achieve the desired performance that has been described? 2 Will the learning intervention enable the learner to produce the desired outcomes? 3 Do you know what the stakeholders expect from you? 4 Do the business processes support training? 5 Will the training identified support the strategy of the organisation? 6 Do you know what business needs will be satisfied by the training needs that were identified? 7 Do you know what time frame is available to narrow the gap in performance to meet stakeholder satisfaction? 8 Does the training material cover the skills, knowledge and behavioural aspects of training? 9 Have the assessment criteria been identified? 10 Does the learning material cover all the outcomes according to the relevant unit standards or job-specific requirements? 11 Have arrangements been made to allow the learner sufficient time for learning? 12 Has a mentor been identified to support the learner during the learning process? 13 Will the learning have an economic impact on the organisation? 14 Has the learning material been implemented on a pilot group? 15 Does the learning material include enough exercises and activities to allow the learner to apply the new learning?

63

Yes

No

MEASURING ROI IN TRAINING

If you have answered “yes” to the above questions, you are following the correct steps to add value to the organisation through learning interventions. If you have answered “no” to a number of questions, do not despair. We will assist you to identify the correct learning material to achieve the outcomes that were identified. It is critical that the correct learning material is identified and sourced to ensure that the maximum return on the organisation's investment in training is achieved. The material has to assist the learner to demonstrate applied competence by actually applying the principles of learning to perform whole work roles in a work environment.

SKILLS IMPLEMENTATION PROCESS The example used in Chapter 2 was of a contact centre agent who is required to log and receive calls, provide solutions, have knowledge of products and systems, perform administrative duties and be able to use a computerised system. We selected the relevant unit standards to cover these competencies as per this example. In this chapter we will guide you to follow the steps discussed below when selecting your learning material. This will ensure that you align your learning to the goals and strategies of the organisation. It will also help you to avoid the pitfall of delivering training in isolation to other organisational processes where people development is perceived to be the function of the human resources department alone without input from line and senior management.

64

CHAPTER 3: IMPLEMENTING SKILLS DEVELOPMENT INITIATIVES

Skills implementation process flow Step 1: Collate the results from the performance-management process Step 2: Ensure these competency requirements are aligned to the organisation’s strategy, vision and mission Step 3: Select learning material, i.e. paper-based or e-learning Step 4: Decide whether the material can be bought “off-the-shelf”, or designed and developed in-house, or current material can be converted to an outcomes-based format and mapped to unit standards Step 5: Identify the level at which the material will be used by using the NQF level descriptors Step 6: Ensure the material has enough exercises and activities to allow the development of skills, knowledge and behaviour Step 7: Use the information on the WSP and annual training report (ATR) to calculate the ROI and estimate the budget for the following year

STEP 1: COLLATE

THE RESULTS FROM THE PERFORMANCE MANAGEMENT PROCESS AND SKILLS AUDIT

The first step is to collate the results from the performance management process as discussed in Chapter 2. The following is an extract of the skills that were identified at Medscheme from the results of a skills audit and the performance-management process: Skills development priorities: z z z

Customer-care skills Business literacy Telephone technique -

Communication skills Team dynamics Problem-solving skills

z z z

Computer literacy Business writing skills Life skills which cover the critical cross-field outcomes, e.g.

- Conflict resolution - Time management

65

- Diversity - Self-actualisation

MEASURING ROI IN TRAINING

STEP 2: ALIGN

THESE COMPETENCIES TO

ORGANISATIONAL STRATEGIES

The competencies that were identified are necessary to "live" the organisation's core values and code of ethics and include: z z z

excellent service. professionalism. building and maintaining customer relationships, aligned to Medscheme's philosophy of nurturing their people and encouraging employees to take responsibility for their own learning.

Competent employees will be able to assist Medscheme to achieve its strategic objectives which are to: z z

z z

vigorously pursue excellent service through system enhancement/ integration, product consolidation and people development. enhance human performance by equipping learners with the skills and knowledge to apply work and generic skills in a work and social context. align the organisation to enhance relationships with all its stakeholders. invest in human capital which will ultimately maximise performance that will result in accelerated business performance and effectiveness, and contribute to the social and economic development of the organisation and the sector.

The individual competencies are aligned to the organisational requirements and measured according to the balanced score card measurement of learning and growth, customer satisfaction, internal processes and the financial impact on the organisation. From the above example it is clear that individual development needs are aligned to the needs of the organisation and will result in more skilled employees to the benefit of the sector and the nation at large. Now that we know what skills are required for employees to demonstrate competence in their job function, we can source the relevant learning material. 66

CHAPTER 3: IMPLEMENTING SKILLS DEVELOPMENT INITIATIVES

Remember to record the time spent on the design and development of the material to enable us to accurately calculate the ROI in training! The steps on how to calculate the ROI are explained in detail in Chapter 5 and include the conversion of the material to outcomes-based format as well as the mapping of material to unit standards, or the conversion of paper-based material to e-learning.

STEP 3: SELECT

LEARNING MATERIAL ALIGNED TO THE

PRINCIPLES OF THE

NQF

When selecting the learning material you have a number of options available: You could use unit standards that are available on SAQA's website: www.saqa.org.za and have material developed to cover these learning outcomes. You could purchase an "off-the-shelf" module, or you can use the current material that is available in your organisation and have this material mapped to unit standards to allow the learner to become familiar with the assessment criteria and the outcomes that have to be delivered. The learning material that was selected must assist the learner to achieve the competence as described in the applicable unit standard. The following criteria are needed to align learning material to the principles of the NQF:

(A) COMPETENCE In the workplace an important shift has taken place towards outcomesbased thinking. Rather than focusing on performance the focus has shifted to the competence underpinning performance through the assessment of competence displaying the achievement of outcomes. Competence describes what the learner knows and can do, how well the learner can do it, and the accompanying attitudes, values and behaviour. Unit standards are the tools used to assess competence and 67

MEASURING ROI IN TRAINING

contain descriptions of the outcomes to be achieved. The outcomes should be attributable to one person, create evidence of learning, be assessable, measurable and verifiable. Specific outcomes should not describe a specific method, procedure, task or steps in a process. The recent change that has taken place in organisations over the past few years requires the employee to be able to apply critical thinking skills, information evaluation and self-management skills. These skills are contained in the critical cross-field outcomes that must also be embedded in the design of the learning material.

(B) UNIT STANDARDS The following definition of a "unit standard" was given in the National Standards Generating Body (NSB) regulations: '"Unit Standard" means registered statements of desired education and training outcomes and their associated assessment criteria together with administrative and other information as specified in these regulations.' Unit standards are made up of three basic components, namely knowledge (what the learner knows), skills (what the learner can demonstrate), and the underlying attitudes, values and behaviour, also referred to as KSA. This competence is also referred to as head (knowledge), hand (skills) and heart (values), or the ABC of learning – attitude, bodily movement and cognition. Source the material aligned to the desired level of learning to achieve competence covering knowledge, skills and attitude. What competence do we want the learner to demonstrate? This will determine the content and level at which our material should be pitched.

68

CHAPTER 3: IMPLEMENTING SKILLS DEVELOPMENT INITIATIVES

KNOWLEDGE The core of the model is that of knowledge or content. The learner has to provide evidence that demonstrates that learning has taken place. Content or knowledge items may include the following: Concepts

Facts

Terms

Dates

Names

Theories

Definitions

Principles

Vocabulary

Events

Rules

In general, it is the educator's function to manage the facilitation of knowledge through direct or indirect instruction. Assessment aims to demonstrate that learners have internalised the knowledge and are able to recall what they have learned.

SKILLS The second component of learning is that of establishing what skills have been learned and practised through various tasks that require learners to demonstrate their skills. These skills may be either psychomotor which requires learners to do something physically, or they may be intellectual such as problem solving or working in groups. For the purpose of instruction we can distinguish between three kinds of skill-based tasks: z z

z

Discipline-based relating to the specific area of learning, General or cross-curricular skills which are not limited to tasks in any special area of learning, but are needed in many different areas. Communication skills that are general but are especially important in almost all areas of learning.

ATTITUDES/VALUES/BEHAVIOUR The third element of real understanding looks at how the learner takes personal ownership of the learning by creating or expressing a 69

MEASURING ROI IN TRAINING

personal perspective. When learners can express their own opinions or attitudes about what they have learned, they provide convincing evidence of real understanding. We can identify three kinds of personal expression: z z

z

Logical deductions when learners draw personal conclusions, make decisions or predictions, and express opinions and beliefs. Attitudes and values when learners behave in ways that reflect what they believe to be correct or desirable in the context or society of which they are a part. Morals and ethics are evident in what learners believe are the right, honest, fair and honourable ways of conducting themselves.

For example, if a learners' competence is gauged at NQF level 3, and the learning material is pitched at NQF level 5, the desired outcomes will not be achieved and the learner will have difficulty in applying the concepts in practice. This could have a negative impact on ROI in training, due to the time wasted, as the learner will have to repeat the same training course. One must also consider the financial implications as new material will have to be bought that is pitched at the correct level of learning. Unit standards

Applied competence

Learning material

Knowledge Skills Attitudes

ROI

Credits This model of understanding, like many similar models, aims to ensure that curricula take a balanced perspective with regard to learning. Learning programme designers, in planning learning 70

CHAPTER 3: IMPLEMENTING SKILLS DEVELOPMENT INITIATIVES

programmes, may use this model to ensure that activities include all the components for a holistic and complete approach to learning that is relevant, useful, and integrates skills, knowledge and values.

(C) ALLOCATION OF CREDITS One (1) credit indicates 10 hours of notional time. This is the average time a learner should need to achieve competence and includes contact time, reading, and time on the job. The number of credits is therefore an indication of the quantity and depth required for completion of a unit standard. The aspect of credit allocation is an important element of ROI measurement. Once you have determined the amount of time needed to reach the desired level of competence, you can determine the cost of the time.

(D) APPLIED COMPETENCE Although SAQA's concept of "applied competence" has three separate components – foundational, practical and reflexive competence – the description of learning currently used retains this concept in its integrated form. "Applied competence is the overarching term for three interconnected kinds of competence: Practical competence is the demonstrated ability in an authentic context to: consider a range of possibilities for action; make considered decisions about which possibility to follow; and perform the chosen action. It is grounded in foundational competence where the learner demonstrates an understanding of the knowledge and thinking that underpins the action taken, and integrated through reflexive competence in which the learner demonstrates ability to integrate or connect performances and decision-making with understanding and with an ability to adapt to change and unforeseen circumstances and to explain the reasons behind these adaptations." Norms and Standards for Educators (Government Gazette No. 20844, Feb. 2000: 10):

What is important to realise is that learning in itself does not have an impact on ROI, because learning usually takes place in a classroom. ROI only becomes applicable when the learner goes back to the workplace and implements improvements that can be measured. 71

MEASURING ROI IN TRAINING

STEP 4: DECIDE

WHETHER INTERNAL OR EXTERNAL

MATERIAL WILL BE USED

Decide whether the material can be bought "off-the-shelf", or designed and developed in-house, or current material can be converted to an outcomes-based format and mapped to unit standards. Typically, you could do an ROI analysis using information from the external supplier with regard to the training material to be used as well as the cost thereof. The same ROI analysis must be done using internal design and development costs. By comparing the two figures you can determine the "better" ROI for the particular project. In addition, current material can also be converted to outcomes-based format and mapped to unit standards incorporating the relevant assessment criteria. The cost to perform this function is included in the ROI template you will find on the CD-ROM at the back of this book.

STEP 5: IDENTIFY

LEVEL DESCRIPTORS WHEN SELECTING MATERIAL

We recommend you use the level descriptors of the NQF when sourcing material to assist you in identifying material that is pitched at the correct level of learning. If the material is not suitable for the learners' level of understanding, the purchase of the material will be a waste of money that will have a negative impact when you calculate the ROI. Level descriptors attempt to describe the nature of the learning achievement, its complexity and relative demand at each level of a qualifications framework. Level descriptors are broad, generic, qualitative statements against which specific learning outcomes can be compared and located. Thus, sets of level descriptors can be used in a general way to determine the pegging of qualifications and standards on a framework. Because they describe learning across domains, disciplines, fields and learning pathways, level descriptors must be general and at the same time specific enough to serve their purpose. They must provide a clear understanding of the meaning of learning attainment corresponding to each level on the NQF. 72

CHAPTER 3: IMPLEMENTING SKILLS DEVELOPMENT INITIATIVES

It is important to recognise that the NQF levels and their level descriptors can only serve as reference points for specific outcomes achieved by specific qualifications. The following example of level descriptors is pitched at NQF level 3 as indicated on the contact centre agent's unit standards: z z z z z z z

Well-developed range of skills. Apply knowledge with understanding in a number of areas. Compare and interpret available information. Operate in a number of contexts, some non-routine. Significant choice from a wide range of procedures. Significant responsibility for quality and quantity of own output under general supervision and quality checking. May be responsible for output of others.

STEP 6: ENSURE

THE LEARNING MATERIAL COVERS THE

UNIT-STANDARD CRITERIA

The next step is to use the outcomes of the learning material to ensure all the assessment criteria are covered. If this is not done, it will be a total waste of time and money if the learner did not achieve the desired outcomes, resulting in a negative ROI ratio. In the next chapter we will have a look at the role of assessment to establish the learner's competence and calculate the ROI in training as a result of the impact of changed behaviour.

STEP 7: USE

THE INFORMATION ON THE

ANNUAL TRAINING REPORT

ROI

(ATR)

WSP

AND

TO CALCULATE THE

AND ESTIMATE THE BUDGET FOR THE FOLLOWING YEAR

The following example is an extract of the annual training report (ATR) for Medscheme that compares the number of learners who attended learning with the number of learners that was estimated would attend learning. "Technician" is the term used by the SETA according to the 73

MEASURING ROI IN TRAINING

guidelines for the Workplace Skills Plan. At Medscheme, contact-centre agents are classified as technical specialists under this level because of the technical background and knowledge of systems, scheme rules and legislation with which they have to be familiar. This information is used to calculate the ROI in training and to estimate the number of learners who will attend training during the following year. This information enables us to estimate more accurately the amount required for the training budget for the following year.

CONCLUSION In this chapter we had a look at the skills implementation process. The learning requirements were identified and aligned to the strategic goals of the business plan. We explored the options of using unit standards as a method of instruction. The learning material has to cover the components of knowledge, skills and behaviour. It is important to determine the level of learning at which the learning material should be pitched to ensure the desired change in behaviour as well as a transfer of learning can take place. The importance of sourcing the correct material to calculate the ROI in training to effect a positive outcome was explored. The learner has to demonstrate that a transfer of learning has taken place through demonstrating competence. This judgement decision can be made once the learner has been assessed for competence. In the next chapter we will cover the principles and application of assessment.

74

75 75

Computer Literacy Computer Literacy Computer Literacy Life Skills

Management

I I P P/I

P

Matching Skills to Job I Profiles I Presentation Skills Implementation Total Planning Total

Total Total Total Total

Total

Total Total

Total

Job Title

Performance Development Total Management Matching Skills to Job Total Profiles Presentation Skills Total

Computer Literacy Web Design Web Design Generic Skills

Business Writing Skills

Business Writing Skills

P/I

I

P P/I

Skill Priority

Technicians

Application of Legislation Legislation Principles Business Literacy Business Administration Business Literacy Induction

Key

Education Training Intervention

Key P Planning grant information only I Implementation grant information only P/I Both planning an implementation grant information

R 0.00

R 0.00 R5 675.00 R 0.00 R 940.00 R 0.00 R2 994.00 R 661.00 R 0.00 R6 162.00 R 0.00

R1 149.00

External

R865 841.00 R40 837.00 R18 454.00 R 870.00 R1257 057.00 R59 288.00 R 0.00 R 0.00

R 0.00

R 0.00 R120 317.00 R 0.00 R19 930.00 R 0.00 R63 480.00 R14 025.00 R 0.00 R130 651.00 R 0.00

R24 359.00

Internal

Cost

44

M

141 7 265 168

0 41 44 6 30 19 6 8 40 42

5

28

1032 18 1438 386

95

1 122 95 21 81 67 13 25 137 89

F

0

D

L540725738

Total for all races

B3 - EDUCATION AND TRAINING INTERVENTIONS REQUIRED TO ACHIEVE SKILLS PRIORITIES

0 0 0 0

0 0 0 0 0 0 0 0 0 0

0

CHAPTER 3: IMPLEMENTING SKILLS DEVELOPMENT INITIATIVES

MEASURING ROI IN TRAINING

76

CHAPTER 4: ASSESSMENT ON PRE- AND POST-PERFORMANCE

CHAPTER 4

ASSESSMENT

ON

PRE-

AND

POST-PERFORMANCE

Both assessment and evaluation are required to determine if learning has been translated into work performance and therefore if it is making an impact on the achievement of the company’s strategic objectives and thus the bottom line. (Ken Lancaster & Robyn Wolfson)

77

MEASURING ROI IN TRAINING

One of the key and fundamental principles of ROI is accountability. Having invested the organisation's funds in training, the training section must be able to answer the question: "OK, we have spent R35 000 on a training course for 12 staff members, let us say on ecommerce. What is the return on this investment?" Many other questions could emanate from this central question, such as: z z z z z z

Do the employees understand the principles of e-commerce? Do they have the competence to go and implement e-commerce in their departments? What additional resources or training do they need to improve their level of competence? Will it be possible to measure the implementation of the content of the training in the work environment? Did the training course meet the needs of the organisation? Do we need to send more people on the training programme to optimise the impact of the intervention?

None of these questions can be answered if we have not done a proper assessment of the situation as well as the competence level of the learners before and after the training programme. It is therefore essential to do a proper pre- and post-assessment of competence and the impact of the training programme. It is like going on a diet. Before the diet, you're weight was 85 kg, after a month you reduced your weight to 74 kg, and after three months you achieved your target weight of 66 kg. There is a pre- and post-assessment that makes it possible for you to determine the level of improvement. Assessment of training follows exactly the same principle. There is a pre-situation in terms of the performance of an employee. Let us take the example of the call centre agent we mentioned in Chapter 2. The performance standard for the call centre agent is to provide a client with a satisfactory answer to a query within 24 hours. Before the call centre training programme, the agent is able to provide feedback after 36 hours (average time logged for client feedback). Now the company sends the agent on a training programme that covers detailed 78

CHAPTER 4: ASSESSMENT ON PRE- AND POST-PERFORMANCE

information about the products and services of the company, customer satisfaction, communication and listening skills, time management skills etc. The employee has been equipped with knowledge and skills to handle queries in a more effective and efficient manner so that the response time is reduced. The post-assessment (let us say two months after the training) reveals that the new average response time in which the employee provides feedback is 21 hours PRE-TRAINING PERFORMANCE

TRAINING

36 hours

POST-TRAINING PERFORMANCE

21 hours

From this example it is clear that training could play a very significant role in improving employee performance. However, training is not a panacea for all problems in an organisation. There could be other reasons for under-performance, such as inadequate resources, negative attitudes, poor human relations, safety problems and many other factors. In addition, not all training programmes deliver the desired performance, and not all training programmes are good. In the above example there may even have been another call centre agent from the same team as the agent who achieved the 21-hour post-training performance, but who maintains the 36 hours rating despite attending the same training. If that is the case the reasons for non-improvement must be investigated. These could include a poor-quality training programme, an unsuitable employee being send on the training, the employee not having the necessary resources, or many other factors having an impact on employee performance.

WHY USE ASSESSMENT AS A MEASUREMENT TO DETERMINE COMPETENCE? Reasons for this include the following: z

Assessment is done within statutory and organisational frameworks to create an integrated national framework of learning 79

MEASURING ROI IN TRAINING

z

z z z z

achievements. The learner is accredited for learning achieved in a formal and non-formal way at the workplace or through community learning. It enables the learner to develop his/her full potential and provides opportunities for future learning. Assessment contributes to the social and economic upliftment of the nation as a whole large through learning advancement. Effective relationships are maintained and clear boundaries are set of what is expected. Learners have the opportunity to develop themselves and improve the quality of service offered. Education, training and development are seen as positive aspects of learning. Assessment contributes to mobility and progression within education, training and career paths.

WHAT IS ASSESSMENT? Assessment is the process of identifying what an individual knows and can do. An individual is assessed to obtain some measure of his or her current abilities. Those skills and that knowledge are then equated with specific standards, courses and qualification requirements. The learner is assessed against those standards and requirements and credited for learning in the appropriate manner. Learners are not measured against one another or against the average obtained by other learners for a specific module.

WHAT IS THE ASSESSMENT OF COMPETENCE? The assessment of competence is the process of making judgements about an individual's competence through matching evidence collected to the appropriate standards or outcomes. The assessment guidelines must incorporate the assessment of each specific outcome, all the assessment criteria, and include the relevant critical cross-field outcomes

80

CHAPTER 4: ASSESSMENT ON PRE- AND POST-PERFORMANCE

WHAT IS COMPETENCE? Competence is being able to perform whole work roles to the standards or outcomes expected in employment in real working environments, and not only in a training situation. Competence is about the ability to work with your hands (skills), knowledge (mind), and heart (attitudes, values or behaviour) The assessment process that is followed will be done according to the following steps:

PRE- AND POST-ASSESSMENT PROCESS FLOW Step 1: Establish the current level of performance through a gap analysis Step 2 2: Pre-assessment meeting with assessor and learner Step 3: Agree on the competence the learner has to demonstrate, and the learning time Step 4: Identify the assessment criteria Step 5: Assessor designs the assessment tools to assess knowledge, skills and attitudes, values or behaviour Step 6: Formal assessment Step 7: Assessor collects evidence through observation, questionnaires, assignments, work output records, role-plays, simulations, video recordings, etc. Step 8: Assessor completes the assessment documentation and provides feedback to learner 81

MEASURING ROI IN TRAINING

Once you have established the current level of performance by means of a gap analysis, a pre-assessment meeting has to be arranged between the assessor and learner. During this meeting both parties have to agree on the competence that has to be demonstrated, as well as the time required to achieve the desired level of competence. This can only be done if the particular assessment criteria have been identified. The assessor has to design the appropriate assessment tools that can be used to assess knowledge, skills and values, attitudes or behaviour in accordance with the pre-determined outcomes to be demonstrated. The assessor then has to conduct a formal assessment in the workplace through observation, questionnaires, video recordings, etc. to make the correct judgement decision. Once enough evidence has been collected, an assessment decision is made and all the relevant documents are completed. The remainder of this chapter will provide information on the appropriate assessment tools, methods and procedures to be followed when conducting formal workplace assessment.

USING ASSESSMENT TO PEFORM A GAP ANALYSIS What is clear from the previous example is that assessment starts with a gap analysis. In other words, the point of departure is a particular performance problem or improvement opportunity that can be addressed by means of training. The gap in performance can also be determined by assessing current competencies against the desired post-training competencies. Current performance

GAP

Future performance

There are various ways in which this gap analysis can be conducted. The following are some of the methods: z

A training needs analysis to determine the competencies, knowledge and skills an employee needs to perform his/her work effectively. 82

CHAPTER 4: ASSESSMENT ON PRE- AND POST-PERFORMANCE

z z z

A performance management system to ascertain the areas for improvement based on employee performance gaps (Chapter 2). A competency analysis of employee performance in accordance with registered unit standards and assessment criteria (Chapter 3). Benchmarking with other organisations or professional standards bodies to determine the gap between your own performance and that of best practices in a particular field of work.

The most comprehensive way to do a performance analysis is to use a combination of these and other methods as this will make it possible for an organisation to align its approach with the overall business strategy of the particular company. Furthermore, it is essential to link behaviour to performance management by specifically addressing the skills gaps for which the cost was incurred to transfer learning to the workplace. The next chapter addresses the various cost items to determine the overall costs of a particular training programme. Assessment will now assist you to determine the ROI in training. The importance of proper and correct assessment cannot be overemphasised. In fact, trying to go for short cuts or implementing assessment in a haphazard manner may distort your ROI figures and present management with inaccurate data. Not only is this unethical in terms of corporate governance and business ethics, but it may also lead to management taking wrong decisions based on incorrect data. This could be disastrous for the effective functioning of a training department and the organisation as a whole. It is therefore essential that you ensure that assessment is done in the correct manner. Following the guidelines in this book will assist you but, in the spirit of lifelong learning, it could be worthwhile to register as an assessor with a SETA or complete a recognised human resource management or development qualification at a higher education institution. Assessment will become an essential part of the ETD practitioner's role in achieving the objectives of the NQF.

83

MEASURING ROI IN TRAINING

ASSESSMENT METHODS There are different methods that can be used to collect evidence of the learner's competence. In line with the NQF principles of fair and accurate assessment, it is essential that the right assessment methods or a combination of methods be used in order to be able to make correct assessment decisions. Learners should be assessed for a combination of knowledge, skills, and attitudes, values or behaviour. In addition to these specific outcomes, the employee is assessed against critical outcomes as represented by the critical cross-field outcomes.

ALL

EVIDENCE COLLECTED SHOULD COMPLY WITH THE

'VACS'

CRITERIA

These are: z z z z

Valid: the assessment is done against the appropriate, relevant unit standard or learning material. Authentic: the evidence presented is the work of the learner who is assessed. Current: current competence should be demonstrated and the documentation must be dated. Sufficient: enough evidence needs to be collected to make the correct decision.

ASSESSMENT

TOOLS

The first step in the assessment process is to design a suitable assessment tool. In order to assess skills, an observation checklist should be designed according to the assessment criteria of the unit standards or the work-specific requirements. To assess the knowledge component of competence, the learner will have to answer questions either verbally or in a written format. A set of model answers should be available. The same unit standard is used to design a set of questions and compile the model answers. The attitudes, values or behaviour component of competence is evident in the manner in which the agent answers a call. The employee may 84

CHAPTER 4: ASSESSMENT ON PRE- AND POST-PERFORMANCE

be perfectly competent to answer the required number of calls in a given time but in a very abrupt or unsympathetic manner. The following are some assessment methods that can be used:

BEHAVIOUR OBSERVATION It is important to observe the learner's behaviour when he or she returns to the workplace. To observe the learner only during formal training will not present a true reflection of competence. For example, the contact centre agent attended training on systems as well as communication skills and conflict resolution. Dealing with an irate customer in a real-life situation requires different skills from those demonstrated in a simulated situation. The same unit standard that was used in Chapter 2 will be used to design the observation checklist. The assessment criteria as per the unit standards are transferred to a checklist to record whether the learner is competent or not yet competent. In the table below, the '2.1' refers to the assessment criteria on the unit standard as indicated in Chapter 2. 'Ccfo' refers to the critical cross-field outcomes. Remember to use the time record-keeping sheet from Chapter 2 to record the time spent on assessment to ensure the ROI is calculated correctly! Assessment criteria

Competent Not yet competent

Company-specific time targets are met 2.1

√

Calls and customers are responded to in the appropriate language or medium 2.2

√

Customers' requests are understood

2.3

Decision-making skills

ccfo

√

Cultural awareness

ccfo

√

85

√

MEASURING ROI IN TRAINING

However, it will be insufficient to measure ROI by using behaviour observation only. An assessor may assess a learner and declare the learner competent, but that does not mean that ROI will be achieved in terms of the monetary value of a training programme. ROI will feature when the contact centre agent actually implements certain concepts and practices that result in tangible improvement. For instance, if the agent applies the principles of the module on communication skills in his or her work environment and communicates more effectively, this will result in business improvements such as more telephone call queries that are resolved in a shorter time the first time, without having to refer the problem to the support office. This data can be used to measure ROI in terms of financial improvements in the organisation.

LEARNING ASSIGNMENTS During some learning programmes, facilitators provide learners with assignments that must be submitted after completion of the learning programme. This approach is currently very popular as it follows the NQF assessment guidelines, especially with regard to the proper assessment of learners before they can be declared competent. These assignments are normally in written format. For example, when a learning programme on problem-solving skills for managers is conducted the learner is expected to go back to the workplace and complete an assignment that relates to his or her work environment. Example During 2001, the Institute for Organisation Development and Transformation presented a management development programme for managers of one of South Africa's top global companies, Sappi. The assignment consisted of an individual component as well as a group assignment to be completed by a team of four to five delegates. One group of engineers decided to tackle one of their production problems in the plant as their practical scenario to practise their problem analysis skills. The group functioned so well and they were so excited about the solutions they generated, that they decided to implement their assignment in the plant. This resulted in major improvements in the workshop processes. In fact, the immediate spin-off as a result of the implementation of the assignment was a cost saving of R63 000. 86

CHAPTER 4: ASSESSMENT ON PRE- AND POST-PERFORMANCE

Now that is what ROI is all about: improving performance back in the workplace by measuring the result of the training intervention. Imagine if every group on a training progamme could go back to work and effect R60 000 in savings. Then training will suddenly become a major profit booster as opposed to the current school of thought that regards training merely as a cost item. To make this happen we need to create a learning culture in an organisation - that is a work environment where everyone takes responsibility for learning and the application of learning in the workplace.

FEEDBACK QUESTIONNAIRES A questionnaire is a very useful assessment method to assess learners in the learning and work environment. While questionnaires have often been used to assess the perceptions and feelings of staff, it can now also be used to obtain statistical data that can be used for ROI purposes. For example, using a customer-satisfaction questionnaire can provide feedback directly from the customers about the performance of employees after they have completed a customer-satisfaction training programme. However, a mere statement that 72% of customers feel that staff treat them well is not enough. This type of data is useful for the measurement of customer satisfaction, but to calculate the ROI requires quantifiable measurements in terms of rands and cents, such as increased business that is generated as a result of the customer-satisfaction training programme. That is the real value of ROI measurement - to be able to show better business results that can be measured as opposed to mere perception data.

WORK OUTPUT RECORDS One of the best methods of assessment is to use work output records, in other words the results of improved on-the-job performance. For example, if a salesperson sells more products after a training programme, this could give a clear indication of the ROI for the training programme. Once again, the need for proper measurements is of paramount importance. Useful measurements would be increased sales, decrease in defects, increased orders from clients, decrease in accidents, decrease in rework etc. All these types of work-output records are measurable, in other words they can be "counted" so that 87

MEASURING ROI IN TRAINING

the rand value can be determined and the ROI thus calculated as indicated in Chapter 5.

PORTFOLIO OF

EVIDENCE

(POE)

Aligned to the principles of the NQF to foster a culture of lifelong learning, employees collect evidence of learning and accumulate this evidence in a portfolio. A portfolio is a structured and detailed record of competence. The learner takes the portfolio with him/her when transferring between departments, jobs or organisations to prove competence in a specific field.

GUIDELINES FOR ASSESSMENT The proper use of assessment is not only an important aspect of the NQF but also forms the basis for the effective calculation of ROI. The following guidelines can be used to ensure that assessment is done correctly in order to be able to show ROI: z z

z z

z

z z z

Determine the objective of the training intervention and the role assessment can play to achieve the goals of training. Ensure that assessors are properly trained to conduct assessment so that they will not allow personal or other factors to influence the assessment process. Establish clear measures for assessment in both the pre- and post-assessment phase of evaluation. Ensure that assessment is based on clear assessment criteria or standards even if unit standards are not available for the particular learning programme. Develop a code of ethics to promote the ethical use of assessment so that the assessment process and results will not be manipulated or distorted by individuals with self-interest. Communicate the results of assessment to learners and line managers. Compare apples with apples so that the assessment will be reliable and valid for the particular situation. Form and build constructive relationships with line managers 88

CHAPTER 4: ASSESSMENT ON PRE- AND POST-PERFORMANCE

z z z z

z z

and learners as their support for the whole assessment process is crucial. Include all the necessary cost items as indicated in Chapter 5. Record the results of all assessments done. Double-check the input and output data for accuracy. If there is any doubt about the assessment process or results, make use of the services of an independent expert or consultant to verify ROI data. Maintain a database of ROI results to make comparisons and identify opportunities for improvement. Compile a comprehensive report on ROI and the whole assessment process and results.

CONCLUSION This chapter highlights the importance of proper assessment for the calculation of ROI in organisations. Assessment plays an important role in showing the legitimacy of training and ensuring that it is meaningful to the organisation in both qualitative and quantitative terms. Of particular importance are evaluation systems that focus on the ROI of training interventions. Further, we demonstrated the role of assessment as the driving mechanism behind the successful implementation of the NQF. Indeed, without proper assessment, the ROI could not be consistently and effectively calculated and the goal of quality training that adds value to the organisation would not be achieved. It is therefore essential for organisations and training managers to ensure that assessment is an integral part of their people development interventions and that it fulfils the functions of promoting effective training, providing the necessary accountability and facilitating the implementation of the NQF objectives in South Africa.

89

MEASURING ROI IN TRAINING

90

CHAPTER 5: CALCULATING ROI

CHAPTER 5

CALCULATING ROI

ROI can be used both to justify a planned investment and to evaluate the extent to which the desired return was achieved. (Clive Shepherd)

91

MEASURING ROI IN TRAINING

In this chapter we are going to have a practical look at calculating the ROI on your learning intervention or your training programme. Once we have determined the ROI, we can establish how the learning, once implemented, will affect the organisation with regard to the return on investment. We need to determine the following: z z z

What is the monetary value that learning can add to an organisation? How can we quantify the value of a training programme in monetary terms? How can you motivate the development or purchase of a new training programme?

These are questions we shall try to answer by taking you step by step through an easy, practical process that will enable you to address financial issues regarding your training programme. In order to determine the return on your investment, in this case your learning investment, you will need to keep in mind that any investment you make is about time and money. If you can determine the amount of time a process will take and you know the cost of that time, you can determine the return on that investment you have made over a specific period. So far, we have covered many of the theoretical principles underlying return on investment (ROI). It must be remembered that ROI is a financial measurement, so in order to get practical we will need to awaken your ability to work with figures. It is really very easy. You will require little time to gather and capture some data regarding your organisation and the relevant training programme. The spreadsheet will do the actual calculation. The more realistic the data you capture, the more realistic the result. It is therefore up to you to put in the effect – the result will astound you! We will now look at a practical process to measure the return on your training investment, in other words what you get back for the training effort. 92

CHAPTER 5: CALCULATING ROI

It is important to remember that when building an ROI model for your training programme you need to look at two main phases: z

z

Phase I: Design an initial costing estimate to motivate the development or purchase of a training programme. This will provide an estimated ROI on the training programme. Phase II: Once the training programme has been implemented, you need to establish whether there has actually been an ROI on the training programme. This is done by going through the process again with actual data.

In order for you to calculate the ROI for your training programme we have designed a comprehensive Microsoft EXCEL spreadsheet that is available on the CD Rom accompanying this book and also on the Web page linked to this book – www.roionline.co.za. In order to access this spreadsheet you will be required to have Microsoft EXCEL software loaded on you computer. You may customise the spreadsheet to suit your specific requirements and make it unique when determining the ROI of your learning training programmes.

OVERVIEW OF THE ROI SPREADSHEET The spreadsheet and its functionality will be extensively covered in this chapter. Make sure you have access to a computer with Microsoft EXCEL software! Once you have opened the spreadsheet in Microsoft EXCEL you will find a number of worksheet tabs at the bottom of your screen. Click on the tab to view the respective worksheet.

93

MEASURING ROI IN TRAINING

1.

Summary

2.

Input Assumptions

Input data

3.

Gap Analysis

Input data

4.

Design & Develop

Input data

5.

Facilitate & Delegate

Input data

6.

Assessment

Input data

7.

Evaluation

Input data

8.

Fixed Costs

Input data

9.

Benefit

Input data

Review data

10. Cash Flow

Review data

Each of the worksheets contains information relating to the specific topic, all of which is required to be completed in order to arrive at a realistic ROI for your learning intervention. The spreadsheet has been designed to motivate the development or purchase of a new learning intervention, be it instructor-led in a classroom, television-based, or technology-based (for example, Internet-based learning) or a combination of any media you choose, or the purchase of an existing training programme. While working through the various worksheets complete only those which are applicable to your situation situation. The suggested process flow to complete the worksheets is shown above. The worksheets "Summary" and "Cash Flow" require no input; they represent the result of the calculation. Let's have a more detailed look at each of the worksheets that make up the total spreadsheet: 94

CHAPTER 5: CALCULATING ROI

1. SUMMARY What must I do?

WORKSHEET

Use the result of your input to motivate the learning intervention. You don't need to input any data on this worksheet.

We will start at the end – the result of all your hard "data capturing". This worksheet provides a summary of all the various "cost-related worksheets" – no input of any data is required on this worksheet. You may view the information, print it, or include it in you proposals. You will see under the top section a number of financial ROI-related results from the calculations. Let's have a look at these ratios: z

ROI: This is a percentage return you will get from your training investment over the respective period, normally calculated over three years. You can also determine the ROI over one and two years respectively by completing the appropriate fields in the Input Spreadsheet, but more on that later. ROI equals the present value of your accumulated net benefits (gross benefits less ongoing costs) over a certain time period, divided by your initial costs. It is expressed as a percentage over a specific period of time. We have used three years as the most common time span since the training programme might become obsolete after this time. This is all you need to know about the calculation – the rate will be calculated automatically! Let's have a look at an example:

If the initial cost for your training programme is R10 000, your annual benefits less annual costs are constant at R5 000 for the next three years, and the interest rate is 10%, your three-year ROI would be: (R 5 000 ÷ (1 + 0.1) + R 5 000 ÷ (1 +0.1)² + R 5 000 ÷ (1 + 0.1)³) ÷ R 10 000 = 124% (the spreadsheet will do this calculation for you) 95

MEASURING ROI IN TRAINING

A positive percentage means you can go ahead with the project. If the percentage is negative, the investment in the development/purchase and presentation of this training programme should not go ahead. While ROI tells you what percentage return you will get over a specified period of time, it does not tell you anything about the magnitude of the project. For example, a 128% return may seem attractive initially, but would you rather have a 128% return on a R10 000 project or a 60% return on a R300 000 investment? That is why you will want to know the net present value. z

Net present value (NPV): This provides you with a rand value of your expected return and therefore indicates the magnitude of your project. It is calculated by adding the present value of the net benefits for each year over a specified period of time and then subtracting the initial costs of the project.

If we take the same new training programme example, the NPV would equal: R 5 000 ÷ (1 + 0.1) + R 5 000 ÷ (1 + 0.1)² + R 5 000 ÷ (1 + 0.1)³ – R 10 000 = R2 213 A positive NPV means that the project generates a profit (present value of cash inflow is greater than the present value of cash outflow), while a negative NPV means that the project generates a loss. When using this method, the organisation should determine a minimum acceptable rate of return, which is usually the organisation's cost of capital. The great thing about NPV is that it tells you about the rand value of your savings; the downside is that it doesn't tell you when savings will occur – this is when we determine the payback period. 96

CHAPTER 5: CALCULATING ROI

z

Payback period: An easy payback period is used to find out how long it will take for an investment to show a profit. This is important when time and cash flow is an issue. It is the time it takes for your project to recoup the funds spent. Normally it is expressed in years or months.

So if we use the same new training programme example as before, your payback period is: R 10 000 ÷ R 5 000 = 2 years Payback is very easy to calculate but it doesn't tell you about the magnitude of your savings, or even how your investment performs after your benefits equal the initial costs. The fourth element of our calculation now comes into play – internal rate of return. z

Internal rate of return (IRR): This is the most sophisticated of the above metrics and is used to analyse multi-year investments. IRR equals the percentage rate by which you have to discount the net benefits for your time period until the point that they equal the initial costs. IRR is closely related to net present value (NPV). The rate of return calculated by IRR is the discount rate you would need to apply to your benefits to obtain a net present value of zero. IRR takes into consideration the time value of money and the timing of cash flows over the period of a project. When using this method, the organisation should determine a minimum acceptable rate of return below which projects are not accepted. 97

MEASURING ROI IN TRAINING

If we use the same new training programme example as before, the IRR would equal 23%. This gives an NPV of: (R 5 000 ÷ 1.23 + R 5 000 ÷ 1.23² + R 5 000 ÷ 1.23³) – R10 000 = 0 (Luckily you have the spreadsheet that will work this out for you.) What does this mean? The IRR (23%) is the interest rate that discounts the future cash inflows of R 5 000 for three years to a present value of R 10 000. This means that the training programme would yield an investment return of 23%. Is this rate acceptable to management so that the project can go ahead? Each of these financial measures has its own strengths and weaknesses. Different organisations will place varying amounts of emphasis on each of the different metrics. To get a clear and complete picture of a prospective investment, you will benefit from having access to all four of these measures measures. Your financial manager should be satisfied when you present all four ratios. z

Learning cost per delegate – This calculation provides you with an estimated amount that it will cost the organisation to run the training programme per learner. It includes all the costs associated with running the training programme over the respective period. The only element excluded is the productivity loss of the learner while active during the training programme. This is considered as a non-tangible cost and can't be directly charged. More regarding this topic will be discussed under delegate costs.

Information regarding each of the various worksheets is summarised – the data has been automatically transferred from the respective worksheets. At the bottom of the worksheet – you will need to scroll structure again providdown – you will find a year-on-year cost structure,

ing the information from the various worksheets but this time represented over a three-year period.

98

CHAPTER 5: CALCULATING ROI

This will give you a good indication of what initial costs will need to be invested and also the associated costs to operate (run) the training programme over the respective period. Each of the various elements will be discussed in the respective worksheets.

2. INPUT What must I do?

ASSUMPTIONS WORKSHEET

Read the questions and input data in the applicable blue boxes.

This worksheet provides a vehicle to capture some of the assumptions that will be used throughout the calculations that are specific to your organisation and learning intervention. Let's have a look at the various questions that are asked. Question

Comment

1) Number of expected delegates to do this training programme per annum?

This data has an impact on most calculations in the spreadsheet. Be realistic over three years.

2) What is the number of productive weeks per annum?

52 weeks in the year – deduct allocation for annual and sick leave.

3) What is the number of productive hours per week?

Number of hours per week less lunch and time off other than leave.

4) What is the annual organisational turnover per employee?

This is calculated in two parts – predict figure over three years.

a) Number of employees in the organisation?

Provide the total number employed in the organisation.

b) Annual turnover for the organisation?

Provide the total turnover for the organisation – sales.

5) What is the average annual employee salary?

Human Resources should be able to provide this – get one figure.

6) What are the other average annual employee costs?

These are all other costs associated with employing a staff member, excluding salary – contact HR department.

7) What is the current interest rate?

Check the current bank prime rate – cost of money.

99

MEASURING ROI IN TRAINING

8) How many training programmes are currently offered?

This enables you to apportion some of the costs across a number of training programmes.

9) How many of those training programs use computers?

Computer-related training programmes can be apportioned with the activity of other functionalities of the computer.

Result

From the calculations above:

10) The number of productive hours per employee per annum are:

This figure is determined by multiplying (2) and (3) above.

11) The employee turnover per hour is:

Turnover (4b) divided by employees (4a) divided by productive hours (10).

12) The employee cost per hour is:

Average annual salary (5) plus other costs (6) divided by productive hours (10).

The data from this worksheet is used in all the worksheets. Try and be as realistic as possible so that you will have a more effective ROI analysis.

3. GAP What must I do?

ANALYSIS COST WORKSHEET

Read the questions and input data in the applicable blue boxes.

When you start developing a new training programme you will initially need to perform a gap analysis. This will involve a process of conducting a needs analysis and performance management appraisal, which was covered in Chapter 2. Let's have a look at the various questions that are asked. Comment

Question

1) What is the hourly rate for the The internal hourly rate is used as a default – if an external party is going to do person doing the gap analysis? the analysis, input the external hourly rate. 2) How many hours will it take to Estimate the number of hours the gap analysis will take. do the gap analysis? 3) How many hours will it take to If you use interviews give a time estimate do the one-on-one interviews? of how long this will take. 4) How many hours will it take to The amount of time a skills audit will take. do the skills audit? 5) How many hours will it take to Total amount of time this will take includdo the surveys? ing collation.

100

CHAPTER 5: CALCULATING ROI

The calculation in the spreadsheet will provide the total time it will take to do the gap analysis. From the above questions you will have a good idea of the time it will take to do the gap analysis. This information is automatically transferred to the summary and used in the various ROI calculations.

4. DESIGN

AND DEVELOPMENT COST WORKSHEET

What must I do? Read the questions and input data in the applicable blue boxes. The next worksheet covers the area of designing and developing the training programme. If you are purchasing an existing training programme complete only the applicable fields. A summary of this worksheet is displayed on top of the screen, which is divided into three sections: 1. 2. 3.

Learning material Software Technology development.

Let's have a look at the various questions that are asked.

SECTION 1 Learning Material Questions

Comment

Design and development of learning material

The gap analysis is done, the phases of development continue.

1) How long will it take to design a onehour training programme?

Estimate the design time for one-hour student time intervention.

2) How long will it take to develop a onehour training programme?

Remember, using different media takes longer to develop.

3) How long will it take to pilot and make changes to the training programme?

Make sure the training programme runs the way it is intended to.

The ratio time taken to develop one hour of student activity can be determined. For an instructor-led training programme the ratio might only be 1:25, for a TVbased training programme the ratio might be 1:50 and for a technology-based training programme the ratio might be 1:100.

101

MEASURING ROI IN TRAINING

4) What is the estimated total length (learning time) of this training programme?

Calculate the number of days i.e. one day = 8 hours. If the training programme is shorter than one day put in the number of hours, e.g. 4 hours.

5) What is the estimated cost per hour for the design and development?

The internal hourly rate is used as a default – if an external party is going to do design and development, input the external hourly rate.

Questions

Comment

“Off-the-shelf” training material

In this case you wish to purchase material that is already developed.

1) What is the cost to purchase the “off-the-shelf” training programme

This is the once-off initial cost to purchase the training programme.

2) What is the annual licensing fee per user?

If there is an annual licensing fee provide the amount per delegate.

Outcomes-based conversion

Convert existing material

1) How many outcomes need to be converted?

The actual number of outcomes to be converted.

2) How many hours will it take to convert one outcome?

The time on average that it will take to convert one outcome.

3) What is the conversion cost per outcome?

The internal hourly rate is used as a default – if an external party is going to convert the outcomes, input the external hourly rate.

Unit standard conversion

Convert existing material.

1) How many unit standards need to be converted?

The actual number of unit standards to be converted.

2) How many hours will it take to convert one unit standard?

The time on average that it will take to convert one unit standard.

3) What is the conversion cost per unit standard?

The internal hourly rate is used as a default – if an external party is going to convert the outcomes, input the external hourly rate.

Question

Comment

Material duplication

Once the training programme has been developed what are the training programme material costs?

1) What are the number of books/CDs/ files/units per user?

How many e.g. books will each delegate receive?

102

CHAPTER 5: CALCULATING ROI

2) What is the combined price per unit?

The delegate might receive a combination of different material. What is the cost for one set?

3) What other material duplication elements are included?

Include any other duplication costs, units and respective cost.

SECTION 2 Software for technology-based learning Questions

Comment

1) What is the annual softwarelicensing fee per user?

If you are motivating the development of a technology-based training programme, there is normally an annual licensing fee per delegate.

2) How many training programmes per annum will be developed?

Estimate the number of technologybased training programmes that will be developed per annum.

3) What other software development costs are included?

Include any other costs linked to the training programme, unit and cost.

4) What is the initial once-off software investment?

Include, if there is an initial investment for the software.

SECTION 3 Technology-based development Questions

Comment

1) What is the hourly rate for the person doing development of the training programme?

The internal hourly rate is used as a default – if an external party is going to develop the training programme, input the external hourly rate.

2) How many hours of learning have to be developed?

This is learner time spent learning with the technology.

3) What other costs are associated with technology-based learning?

Provide any other cost linked to technology development.

From the above questions you can determine the time and costs to develop/purchase the training programme. This information is automatically transferred to the summary and used in the various ROI calculations. 103

MEASURING ROI IN TRAINING

5. FACILITATION

AND DELEGATE COSTS WORKSHEET

What must I do? Read the questions and input data in the applicable blue boxes. The training programme has been developed and now it is time to present it. This worksheet determines the facilitation (trainers' costs) and delegate costs to run one training programme. A summary of this worksheet is displayed on top of the screen. Let's have a look at the various questions that are asked. Trainers' costs Questions

Comment

1) What is the annual salary of the trainers who will be presenting this training program?

Provide salary package for each trainer who will be presenting this training programme.

2) What is the average cost of transporting one trainer to the training venue?

Provide the transport costs incurred by one trainer.

3) What is the average cost of one night’s accommodation in a hotel?

Provide the cost for one night for one trainer. If the training programme doesn’t require overnight accommodation, leave this field as 0 (zero).

4) What is the daily allowance per trainer per day?

Provide the allowance for one trainer per day. If not applicable leave as 0 (zero).

5) How many times per annum do you expect to present the training programme?

Estimate the number of times the training programme will be presented.

Delegate costs Questions

Comment

1) What is the average cost of transporting one delegate to the training venue?

Provide the transport costs incurred by one delegate.

2) What is the average cost of one night’s accommodation in a hotel

Provide the cost for one night for one delegate. If the training programme doesn’t require overnight accommodation, leave this field as 0 (zero).

104

CHAPTER 5: CALCULATING ROI

3) What is the daily allowance per delegate per day?

Provide the allowance for one delegate per day. If not applicable leave as 0 (zero).

From the above questions you can establish the costs to present one training programme. This information is automatically transferred to the summary and used in the various ROI calculations.

6. ASSESSMENT What must I do?

COST WORKSHEET

Read the questions and input data in the applicable blue boxes.

Once the training programme has been presented, the delegates need to be assessed in order to establish whether they have understood the material covered and can demonstrate competence. Let's have a look at the various questions that are asked. Questions

Comment

1) What is the assessor’s rate per hour?

The internal hourly rate is used as a default – if an external party is going to assess the delegates, input the external hourly rate.

2) Pre-assessment: How long will the assessor take to assess each delegate?

Provide the estimated time it will take to assess one delegate.

3) Post-assessment: How long will the assessor take to assess each delegate?

Provide the estimated time it will take to assess one delegate.

4) What is the moderator’s rate per hour?

The internal hourly rate is used as a default – if an external party is going to moderate the delegates, input the external hourly rate.

5) How long will the moderator take to moderate each delegate’s assessment?

Provide the estimated time it will take to moderate one delegate.

From the above questions you can establish the cost to do the assessment for one training programme. This information is automatically transferred to the summary and used in the various ROI calculations. 105

MEASURING ROI IN TRAINING

7. EVALUATION What must I do?

WORKSHEET

Read the questions and input data in the applicable blue boxes.

The assessment has been completed and the evaluation of the delegate usually takes place back at the workplace. Let's have a look at the various questions that are asked. Questions

Comment

1) What is the rate per hour of the supervisor or the person who will do the evaluation?

The internal hourly rate is used as a default – if an external party is going to evaluate the delegates, input the external hourly rate.

2) How many hours will it take to evaluate the delegate’s behaviour/performance?

Provide an estimated time it will take to evaluate one delegate.

3) How many hours will it take to confirm the delegate development plan?

Provide an estimated time it will take to complete one delegate development plan.

From the above questions you can establish the cost to do the evaluation for one training programme. This information is automatically transferred to the summary and used in the various ROI calculations.

8. FIXED What must I do?

COST WORKSHEET

Read the questions and input data in the applicable blue boxes.

Fixed costs can be seen as costs that exist irrespective of whether a training programme is developed or presented. These are costs incurred by the organisation and usually include elements such as rent, repayment costs of equipment and maintenance of such equipment. A summary of this worksheet is displayed on top of the screen. Let's have a look at the various questions that are asked in the worksheet in order to get to a fixed-cost figure related to this training programme. 106

CHAPTER 5: CALCULATING ROI

Floor space Questions

Comment

1) What is the annual cost for floor space rental?

This is the cost of floor space in the training centre and is usually represented as cost per square metre.

2) What is the total floor space that is linked to this training programme?

Estimate how much floor space in square metres will be used to present this training programme, i.e. the classrooms used, one in each province, the office space used to develop and administrate the training programme, etc.

3) What annual percentage of floor space will be used for this training programme?

From the above figure, what annual percentage of floor space will be used exclusively for this training programme?

4) What other annual costs are linked to rental of space?

Provide an annual figure of other costs linked to rental of floor space.

Equipment – computers Questions

Comment

1) How many computers are used for learning in the organisation?

This will provide a figure of the repayment of computer equipment that is linked to training. Include computers that are used by support staff.

2) What is the period over which computer equipment is depreciated?

This varies per organisation but is usually 36 months.

3) What is the hardware cost per computer?

The computer department should give you this cost. This figure will be depreciated over the period specified.

4) What is the software cost per computer?

The computer department should be able to supply this cost. This figure will be depreciated over the period specified. Include software linked to the computer only – not the learning software – that is covered under “Development worksheet”.

5) What is the rental cost per computer per annum?

If the organisation rents its hardware and software, provide the annual rental rate per computer.

6) What is the percentage (%) time that the computer is used for learning?

From the above information what percentage of the computer time is used for the learning activity?

107

MEASURING ROI IN TRAINING

Maintenance – computers Questions

Comment

1) What is the annual maintenance cost per computer?

The computer department should give you this cost.

2) What other annual costs are associated with each computer/workstation?

Provide any other annual costs linked to the computer.

Equipment – other Questions

Comment

1) How many TV sets have been purchased for learning?

Provide the number of TV sets used in the training environment.

2) What is the cost per TV set purchased?

Provide the purchase price.

3) What other equipment has to be purchased for training?

Provide the cost and the number of units for any other equipment linked to training, e.g. overhead projectors.

Other fixed costs Questions

Comment

1) What other annual fixed costs are linked to the training department?

Provide any other fixed cost you can associate with training, e.g. annual salaries of support staff, stationery, etc.

From the above questions you can establish the fixed cost for one training programme. This information is automatically transferred to the summary and used in the various ROI calculations.

9. COST-BENEFIT What must I do?

WORKSHEET

Read the questions and input data in the appropriate blue boxes.

The final input worksheet covers the benefits derived from implementing the learning intervention. It really covers the area of measuring the productivity impact on the organisation when implementing a training programme. 108

CHAPTER 5: CALCULATING ROI

It must be remembered that determining the benefits derived from a training programme involves estimating the impact the training programme will have on the organisation in terms of the below-mentioned elements as a result of the learning. It is very difficult to determine the actual benefits due to many uncertain internal and external factors that can affect the outcome of the benefits. You need to take the estimated benefits' values at face value in order to obtain a realistic ROI analysis. Speak to as many expert people as possible in your organisation who can assist you with answering the questions that are linked to determining the benefits that may result from the training programme being implemented. Benefits can be divided into four elements: z z z z

Time Personnel Operations Revenue

Let's have a look at each of these elements separately and then discuss the questions that need to be answered. Time saving: Estimate the time saved as a result of the changed behaviour of the delegate after having completed the training programme. In order to do this, estimate the average time that the delegate currently spends on outcomes/unit standards covered in the training programme. If the delegates are not spending any time on achieving competence linked to unit standards, estimate the time they currently spend on acquiring competence related to other jobspecific outcomes. Then estimate what percentage of this time will be saved as a result of implementing the competencies of the unit standards covered in the training programme. For example: 30% time spent on handling queries on the phone and 50% saving due to implementing competencies acquired from this training programme. Input 30% under time and 50% under saving.

109

MEASURING ROI IN TRAINING

Another example: Time saving

% time

% saved

Saving/year/ delegate

Handling queries

15%

10%

R 2 250

Outcome 2

2%

10%

R 300

Outcome 3

10%

10%

R 1 500

Outcome 4

5%

10%

R 750

Outcome 5

3%

10%

R 450

Outcome 6

8%

10%

R 1 200

Outcome 7

4%

10%

R 600

Other

10%

10%

R 1 500 R 8 550

Additional efficiency discount Effective saving per year Delegates trained

50% Year 1

Year 2

R 4 275 000

R 3 847 500

1 000

800

Year 3 R 3 420 000 600

By changing the various values you can determine a realistic timesaving benefit for each of the applicable outcomes/unit standards. The spreadsheet has provided you with seven outcomes that can be measured. Replace the outcome with the name of your outcome/unit standard. The "Other" field can be replaced with any other timesaving element that might be applicable to your organisation. The annual time-saving benefit per delegate for each outcome is calculated by taking the annual productive hours multiplied by the organisational cost per delegate, multiplied by the two time time--saving percentages. Once all the unit standard time-saving benefits have been determined the total figure is again discounted by a percentage. The reason for discounting this figure again is because not all time saved can be directly linked 110

CHAPTER 5: CALCULATING ROI

as efficient time saving, i.e. not all time saved is used efficiently to generate increased productivity productivity.. The benefit of time saving for the three years is also shown. The figure provides the rand value of time saved by the delegate doing the training programme which, if used efficiently, can be used to generate additional income for the organisation. For example, if a task that used to take one hour takes 45 minutes after the training programme, there is a time saving of 15 minutes, which can be used productively towards generating additional income. A monetary value has been given to the 15-minutes time saving which has a direct benefit when calculating the ROI of the training programme. Personnel saving: By implementing this training programme, the productivity improvement of the delegates, time saving and operational savings can lead to a saving of personnel. This will obviously differ from organisation to organisation. The cost of employees is generally the highest expense of an organisation. If a saving, due to a reduction in personnel or not having to employ a staff member can be achieved by implementing this training programme, it can be attributed as a benefit in your ROI analysis. By answering the following four questions you can calculate an estimate regarding personnel cost saving that this training programme can provide to the organisation. Let's have a look at the questions. Questions

Comment

1) How many additional staff placements can be avoided due to implementing this training programme?

Calculate an estimate based on the number of delegates that will complete the training programme during each year.

2) How many staff members can be redeployed due to implementing this training programme?

Due to better efficiency the same activity can be completed by fewer employees – move the additional persons to other functions within the organisation.

3) How many temporary/seasonal staff will not be required due to implementing this training programme?

Provide the estimated number that will not be required during the year.

4) What is the value of additional payroll benefits?

Provide any other employee “pay- related” figures.

111

MEASURING ROI IN TRAINING

In order to calculate the personnel cost saving, the number of employees "saved" as determined by the questions above is calculated by the organisation's cost of employing the respective employees. This is a direct saving to the organisation and can be attributed to a benefit of doing the ROI analysis. Operational savings: This cost looks at the direct operational saving that this training programme can offer the organisation. Each training programme might have different savings. For example, if a technology-based learning option is chosen as the delivery method for this training programme, a huge saving will be achieved on transport and accommodation as the delegates will not have to leave the work environment to effect learning; it will be delivered to their place of work through, for example, the Internet or television-based instruction. Another training programme might use technology to deliver the pre-work over its existing computer system, thereby reducing the cost of material duplication, for example, printing of learning material. Analyse the elements associated with developing and delivering this training programme and determine any appropriate operations savings. Questions

Comment

1) What is the estimated saving per user for transport and accommodation?

Include air tickets, car hire, petrol, hotel, meals and allowances.

2) What is the saving on printing costs?

Estimated total cost per year.

3) What is the value of other operational savings?

Provide any other savings linked to operational initiatives of this training programme.

Revenue impact: Revenue generation can be substantial. This is probably the most difficult benefit to determine – it really is a "constructive guess" because there are so many internal-external factors linked to revenue generation in an organisation. The implementation of this training programme can impact revenue generation, and create ideas and opportunities for new products and services. For example, a sales person can, due to the benefits of the competences, make ten additional presentations resulting in an extra sale per month. 112

CHAPTER 5: CALCULATING ROI

It is suggested that without getting too scientific and over-analysing the situation, you provide an educated estimate in response to the following three questions. Ask a few "expert" people in your organisation who might provide some additional weight to your answers. Questions

Comment

1) What is the expected revenue increase that can result from this training programme?

Provide an annual figure and include all delegates that will complete the training programme. For example, if one additional sale is generated per delegate, what is the impact?

2) What is the expected revenue increase resulting from new products and services?

A lot of new ideas are usually generated during learning. Provide an estimate of only one or two opportunities to generate additional income.

3) What is the organisation’s profit margin?

This can be provided by the financial department. Check out your organisation’s financial statements.

The various benefits have been calculated. These are automatically transferred to the summary worksheet and used in the calculation of the ROI analysis. If you can't estimate any "monetary-convertible" benefits, the ROI calculation is not possible. In this case, you will need to ask yourself whether providing the learning intervention is worth your while. You can now return to the summary worksheet and check the result of your input from the worksheets. What are your ROI, your NPV, the payback period and the IRR? It will be interesting to see the cost per delegate.

10. CASH-FLOW

WORKSHEET

What must I do? Use the result of your input to motivate the learning intervention. You don't need to input any data on this worksheet.

113

MEASURING ROI IN TRAINING

The last worksheet that we are going to look at is the cash-flow statement that is generated from this training programme. The information is generated from the various "input" worksheets and can provide you with an overview of how the "bank account" will benefit or not benefit from implementing this training programme. The cash-flow statement is viewed over three years, with an initial cost or investment that has to be made to enable the training programme to be developed and become operational. The bank balance at the beginning of the period will be zero and become negative as you pay for the cost of the training programme. During the three years the monetary return on the investment (your initial cost) will be realised and the bank balance will return to positive figures, as long as your cost of operating the training programme is not more than the benefits achieved. Let's have a look at an example of the cash-flow statement. Initial cost Bank balance at beginning of the year Receipts Time saving Personnel saving Operations saving Revenue impact Total Payments Gap analysis Design and Development Licence fees Trainers' costs Delegate costs Assessment Evaluation Fixed costs Other Total Surplus/shortfall Bank balance at end of the year

Year 1

Year 2

Year 3

R0

(R694 511)

R2 055 134

R5 026 192

R0 R0 R0 R0 R0

R4 275 000 R3 847 500 R1 500 000 R1 650 000 R1 000 000 R900 000 R300 000 R600 000 R7 075 000 R6 997 500

R3 420 000 R1 800 000 R800 000 R1 000 000 R7 020 000

R5 707 R638 804 R50 000 R0 R0 R0 R0 R0 R0 R694 51 1 511 (R694 51 1) 511)

R0 R0 R0 R0 R0 R0 R0 R45 000 R40 000 R220 870 R 220 870 R 220 870 R3 439 130 R3 095 217 R2 751 304 R244 565 R244 565 R244 565 R400 000 R400 000 R400 000 R20 790 R20 790 R20 790 R0 R0 R0 R4 325 355 R4 026 442 R3 677 529 R2 749 645 R2 971 058 R3 342 471

(R694 51 1) 511)

R2 055 134 R5 026 192 R8 368 663

114

CHAPTER 5: CALCULATING ROI

Across the top of the cash-flow statement are the headings that describe the various elements: the initial cost (investment) made and the years under review. Under the first section, you will the see "beginning of the year bank balance" and the various receipts (income) as determined by your benefit worksheet. The next section shows the payments that have to be made, which are the results from your "input" worksheets. Next you will see the surplus or shortfall for the year and then the "end of the year bank balance". The cash-flow statement provides you with an excellent tool to motivate the development or purchase of your learning intervention. Your financial manager will immediately understand the various elements that you are providing and the financial impact the training programme will have on the organisation.

CONCLUSION In this chapter you have learned to calculate the return on investment from the training programme you are planning to develop, purchase or have already implemented. We have provided a simple process to follow when calculating the ROI, by answering various questions and using an EXCEL spreadsheet to co-ordinate the data and convert it into easy-to-understand information. The training manager now has a tool that can be quickly used to motivate training initiatives in an effective manner and can be easily understood by both financial and non-financial persons. It is important to remember that ROI is a financial measure that can be used to establish whether the training initiative is financially viable. This chapter covered a short explanation of the various "input" spreadsheets that form the core of the ROI calculations. Only the 115

MEASURING ROI IN TRAINING

fields that are applicable to your situation are required to be completed and used to calculate the ROI unique to your training initiative. The "output" from the worksheets is displayed in the "Summary" and the "Cash flow" worksheets, providing you with sufficient financial information to motivate your training initiative.

116

CHAPTER 6: EVALUATING THE ROI IMPACT ON TRAINING

CHAPTER 6

EVALUATING THE ROI IMPACT ON TRAINING

Measuring the ROI of training programs is one way to show business leaders the value of training investments in financial terms. (Larry Long)

117

MEASURING ROI IN TRAINING

You have completed calculating the ROI of your training programme. What do you do now? This chapter will provide you with a practical plan which you can use in the workplace. It is a suggested process flow with hints and advice to follow when using the ROI information. The process flow includes a typical report that can be used to motivate your training initiative using ROI, a checklist to follow when you are using ROI in your organisation, some relevant information, and a discussion on the complete ROI implementation process in an organisation.

MEASUREMENT AND EVALUATION The numerous definitions, descriptions and concepts on evaluation lead to confusion in the field. For the purpose of achieving the desired ROI results, it is crucial to grasp these concepts as you are required to implement both measurement and evaluation as part of the overall ROI implementation process. It is important to note the differences between evaluation and measurement: Measurement can be regarded as the gathering of data and the allocation of a numerical value to it. Congruence exists around the following aspects of evaluation: 1. 2.

Measurement comes before evaluation and a value judgement is made regarding the merit and worth of the programme. Evaluation is done with the specific aim of initiating certain actions. M EAS U R E M E N T ROI

Preassessment

Training

Postassessment

Improvement

Evaluation

118

CHAPTER 6: EVALUATING THE ROI IMPACT ON TRAINING

Evaluation can take place without measurement, but if we want to make sound judgements about an organisational operation, then hard data is necessary. Evaluation revolves around the process of making a value-assessment on the data gathered. Evaluating training can be regarded as a systematic examination: that is measurement, value-assessment and feedback of interventions that occur as part of training. Evaluation is the process of appraising something carefully to determine its value. In training, evaluation involves judging the worth of planned learning experiences i.e. how effective the particular training programme has been. There are a number of good reasons for conducting evaluations. Basically they are all about judging the impact of training interventions on individuals, groups and the organisation. People who were involved in the needs-analysis process should also be involved in evaluation. The reason is simple enough: needs analysis raises expectations about what changes should occur, while evaluation furnishes feedback on the relative success of actions taken to satisfy those expectations.

ROI REPORT Well done! You have worked through the spreadsheet in Microsoft EXCEL, which was covered in Chapter 5. What now? You are left with five important results from your calculations. These figures include the percentage ROI, the rand value NPV, payback period to recoup you investment, the IRR and the cost per learner. These results would typically be derived from the analysis of one training programme, which is either currently being used in your organisation or which still has to be developed or purchased. You obviously have a number of training programmes that you wish to analyse. Included on the CD Rom at the back of the book and on the website www.roionline.co.za you will find another spreadsheet that 119

MEASURING ROI IN TRAINING

can be used to determine an overall ROI result of all your training programmes. Use this spreadsheet to input the results of the five calculations above from all your training programmes. You can now answer the questions that we asked in a previous chapter: z z

What is the monetary value that learning can add to an organisation? How can we quantify the value of a training programme in monetary terms?

You now have a strong tool that will assist you with motivating the bottom-line value that training can add to the organisation. In order to communicate this message to management and the organisation as a whole, you need to use as many mediums and methods as possible. We suggest you record your findings in a report. A basic outline of a short report to motivate the development of a new training programme would typically include the following: 1. 2. 3. 4. 5. 6.

Introduction. Recommendation with short action statement. Justification of that action with benefits. Any further explanation/motivation. Conclusion – offer your assistance. Appendix with detailed information.

An example of a short report to justify the development of a new programme could look like this: New "Introduction to Call Centre" training programme for ABC (Pty) Ltd Introduction For a number of years ABC has been providing an introductory training programme for call centre operators using various internal and external resources. Due to technological advances made by ABC 120

CHAPTER 6: EVALUATING THE ROI IMPACT ON TRAINING

and an employee's attitude change towards technology, the need has been identified to provide training through various mediums, one being the computer. ABC call centre operators are using computer terminals in their daily activities to assist ABC clients. In conjunction with ABC's training strategy, this training programme will provide knowledge to the learner accessed via the terminal, on demand, right at the workplace. Recommendation It is recommended that the course be developed internally. With ABC's newly acquired e-learning system, development and implementation can be rapidly deployed. The training programme should have pre-reading designed on the system, a half-day workshop at regional offices, with assessments on the system. The supervisors will be used for the assessment of competence. After the training programme has been completed, the learning objects designed on the system can be used as "on demand" knowledge available to the employee. Cost of training programme The training programme will be presented to 2 400 delegates over the next three years. The cost of the training programme can be divided into various elements: z z z z z z z

Gap analysis – R13 043 Design and development – R310 435 Facilitation cost – R365 217 over a period of three years Delegate costs, including productivity loss – R 5 165 696 over a period of three years Assessment and evaluation – R1 396 565 over three years Fixed costs - R 654 620 over 3 years Cost per delegate – R 1 460

Further details and a cost projection over three years may be found in the Appendix. 121

MEASURING ROI IN TRAINING

Benefits There are sufficient skills internally to develop this programme. While developing the training programme, content can be adjusted to adhere to current NQF requirements. Current learning objectives must be aligned to unit standards, which in turn are seen as learning objects in ABC's newly acquired e-learning system. The return on the investment over a three-year period, i.e. the life expectancy of the training programme is 291% (ROI) (ROI). The rand value, or net present value, over three years, of this investment is R531 965 (NPV). (NPV) The initial investment of R323 478 will be returned within the first 10 months months. The internal rate-of-return percentage is 112% (IRR) (IRR). Conclusion From the above ROI figures it is clear that the project should go ahead. A more detailed breakdown of the project is being compiled in a formal training proposal as per the ABC standard. I wish to confirm my commitment to this project and if management wishes to proceed with this project I will gladly assist. Abner T shabalala Tshabalala Group Skills Development Manager: ABC Every report that uses elements of ROI information is very unique to each organisation. The training representative now has the financial tools to assist in motivating the implementation of training interventions and has the ability to calculate the ROI before an investment is made. The ability to measure the actual ROI after the training programme has been operating for a period is another important tool that the training representative now has in his or her control.

122

CHAPTER 6: EVALUATING THE ROI IMPACT ON TRAINING

ROI IMPLEMENTATION PROCESS Having read the previous chapters you should have a clear understanding of the different phases of implementing ROI. However, to make it even easier for you to go ahead and apply the principles and practices of ROI in your work environment, we are providing you with a step-by-step implementation process that you can adapt to meet your own needs in your organisation. Below you will find a figure explaining the ROI implementation process that is followed by an explanation of each one of the 11 steps you need to follow. ROI IMPLEMENT ATION PROCESS IMPLEMENTA

z

ROI Awareness Original ROI focus z ROI training

ROI

Strategic ROI Planning Strategic business plan z Training measurement

Training Improvement Plan

Gap/Performance Analysis z Training needs analysis

Communicate Roi Results

Training Plan/ Workplace Skills Plan

Evaluate Training Impact

z

Determine Cost Of Learning z Input costs z Facilitation, assessment etc.

z

Conduct Training Training sessions

Calculate ROI ROI ratios: NPV, IRR

Assessment Of Learning

z

123

MEASURING ROI IN TRAINING

STEP 1: ROI

AWARENESS

The first step in the ROI implementation process is that an awareness needs to be created about the importance of ROI. This usually happens when a training manager realises the importance of measuring ROI for training. Sometimes training managers are pressurised by line managers to show the value of training, and this also leads to an awareness of ROI. Other ways of becoming aware of the need for ROI is to attend seminars and conferences where training managers and consultants from different industries discuss the need and importance of ROI. Training managers who have attended the annual international conferences of ASTD in the USA will know how much emphasis is being put on ROI at these conferences. Furthermore, reading books about ROI such as this one could also be part of the process of creating and improving awareness of ROI. Visiting the ROI website (www.roionline.co.za) is another way of increasing awareness of ROI. Moreover, conference organisations such as Knowres Conferences are also organising workshops to increase awareness of ROI. Staff members of the training department should also be trained so that they can acquire the necessary knowledge and skills to measure ROI. Remember that training practitioners have not been trained to measure ROI – they see their role in terms of "running courses" and not measuring the impact of training. So training and even retraining is crucial for training staff to become ROI champions. Whatever method is used to create the initial awareness of ROI for training, it is essential that the momentum be maintained. Of course ROI is a great concept, but many great ideas die when there is no follow-through and implementation. Forming constructive relationships with line managers is a crucial element of ROI awareness. If line management can experience real commitment on the part of the training department to measure ROI, they will become more supportive of training, and indeed the training department. Having their support will not only make it easier to implement the training. They may even make a more direct contribution towards ensuring that the maximum ROI is achieved not only by supporting the transfer of 124

CHAPTER 6: EVALUATING THE ROI IMPACT ON TRAINING

learning to the learner but also by motivating the learner to apply the newly acquired knowledge in the workplace which will ultimately result in improved performance.

STEP 2: STRATEGIC ROI

PLANNING

The second step in the ROI implementation process is to conduct a strategic ROI planning session. This session is of paramount importance as ROI will fundamentally change the way training is delivered and measured. A commitment to ROI means that training is not conducted for the sake of training anymore, but to add value. This paradigm shift in the approach towards training requires a new strategic focus – an emphasis on delivering training that improves service delivery and services and, ultimately, productivity in the organisation. In fact, it means that training is not run like a "department" anymore, but more like a business that must control costs on the one hand, and make profits on the other. The better the ROI, the better the organisation (training department) is performing. A strategic ROI planning session should be conducted by involving the entire training staff and some line managers. If you invite some line managers, they will in all likelihood be very impressed with the business focus of the training department, and will therefore play a constructive role in providing the necessary support and resources to help with the overall ROI implementation process. The strategic planning session should include a clear role clarification of the various role players. For example, staff responsible for the design of learning programmes can provide specific inputs into the design cost of training programmes. Training administration staff can provide inputs on all the logistical costs associated with training. Not only will this inclusive approach assist the training manager to gain maximum support for ROI implementation, s/he will also be able to make his or her staff more cost sensitive so that they will be less likely to waste unnecessary resources on training that is not necessarily cost-effective. An inclusive approach will also lead to an environment conducive to suggestions for more effective and efficient training. 125

MEASURING ROI IN TRAINING

The outcome of the ROI strategic planning session should be a rollout plan for the implementation of ROI – in other words, what must happen, by whom, when and how. Using the following steps as a basic framework will definitely assist in the compilation of the strategic ROI plan.

STEP 3: GAP/PERFORMANCE

ANALYSIS

Once there is clarity on the overall ROI strategy, the process of implementation can continue. The training process starts with an analysis of a performance problem or development opportunity identified during the gap analysis. The gap analysis clearly indicates the discrepancy between ideal performance and current performance. A proper training-needs analysis using all the necessary analysis methods, such as skills audits, questionnaires, or focus groups can be used to ascertain the exact training need. The more accurately the gaps are identified, the more accurate the eventual ROI calculation will be. The process of performance management can also be used as a system to analyse performance problems or development opportunities. This aspect has been discussed in great depth in Chapter 2.

STEP 4: TRAINING

PLAN/WORKPLACE SKILLS PLAN

Your training plan or workplace skills plan, in accordance with the Skills Development Act, indicates clearly how you plan to fill the skills gaps identified under Step 3. Major skills development priorities are outlined, as well as the learning programmes that are planned to improve the skills level of the workforce.

STEP 5: DETERMINE

THE COST OF LEARNING

The fifth step in the ROI implementation process is to determine the cost of learning. This means that you identify all the different cost items needed to execute the training or workplace skills plan as indicated under Step 4. All the cost items have been described in Chapter 5. Use your spreadsheets described in this book which are available 126

CHAPTER 6: EVALUATING THE ROI IMPACT ON TRAINING

on the CD-ROM or the website to save some time and effort by identifying the cost items relevant to the training you conduct.

STEP 6: CONDUCT

TRAINING

We are now at the step most trainers like, and that is to present training to a group of learners. This can either occur on-the-job, in a classroom or by means of e-learning, depending on the preference of the particular organisation. Training programmes are conducted to transfer knowledge and skills to learners so that the necessary skills gaps can be addressed as discussed under Step 2.

STEP 7: ASSESSMENT

OF LEARNING

When training is completed, the important questions to answer are: z z z

Did the training make a difference? Have learners reached the required levels of competence? Are they doing their jobs better?

These questions can only be answered if assessment has been done to determine whether learners can be declared competent.

STEP 8: CALCULATE

ROI

An assessment of learner competence is followed up with the most important measurement of the ROI cycle, and that is to calculate your ROI ratios. ROI is presented as a percentage. For example, let us say that the ROI of a learning programme is determined at 77% which means the training added value to the organisation. Chapter 5 describes the process of calculating ROI.

STEP 9: EVALUATE

TRAINING IMPACT

Now you sit with a number of ratios, and you ask yourself the question: So what? What are you going to do now? Evaluating the impact of training means taking the results of the ROI calculations and deciding how these can be used by the organisation. An ROI report must be written 127

MEASURING ROI IN TRAINING

in which the ROI figures are interpreted and conclusions drawn, as well as recommendations made to management as indicated earlier in this chapter. Evaluating training impact will have both a positive and negative side to it. Achieving good ROI ratios is great, but when ROI is not so favourable, the process of evaluation assists the training manager to make decisions about the improvement of particular training programmes. Careful analysis is needed to determine the root causes of why some learning programmes don't achieve the desired results while others do. Specific action plans for improvement can then be determined and implemented.

STEP 10: COMMUNICATE

ROI RESULTS

The whole process of measuring ROI is such a rewarding experience that once the ROI results are available, they must be communicated. Of course, positive ROI figures must be celebrated, something we do not do enough in South African organisations. Various communication media can be used to communicate ROI results: z

z z z z z z z z z z

Write memoranda to line managers thanking them for their support in terms of making staff available and then give them the ROI figures! Put ROI results on the intranet for all staff to see. Use training meetings to share ROI results and lessons learned. Inform the skills development facilitator of ROI results. Report ROI results in the company newsletter and on notice boards. Award prizes for the best ROI figures reported. Indicate ROI figures on brochures used when marketing future training programmes. Write articles about ROI success stories and lessons. Send e-mails to learners informing them of ROI data. Include ROI data in training reports. Do presentations to line managers on ROI. 128

CHAPTER 6: EVALUATING THE ROI IMPACT ON TRAINING

STEP 11: DEVELOP

AND IMPLEMENT THE TRAINING IM-

PROVEMENT PLAN

As indicated previously, adopting the ROI mindset requires a business approach to the whole training and development process. And business is all about performance and productivity. Using the ROI process can help training staff to identify opportunities for improvements to training programmes. Programmes with a low or negative ROI ratio can be redesigned, improved or scrapped. ROI measurement is, however, so useful that doing another ROI calculation will assist a training team to determine whether the programme has improved. The ROI has to be calculated after every training programme and the results compared to previous ROI results.

THE ROI REVIEW CHECKLIST As you can now see, ROI is not so difficult. In fact, anyone can do it. Use the following checklist as a self-check instrument to review yourself in terms of the application of the ROI process. The ROI Review Checklist Item

ROI Review Item

1

Have you created sufficient awareness of ROI in the organisation?

2

Have the benefits of ROI been clearly communicated to training staff?

3

Do training staff have the necessary knowledge and skills to measure ROI?

4

Have you held a strategic ROI session to plan the rollout of ROI?

5

Have you incorporated ROI into the strategic HRD plan of the organisation?

6

Have you established good relationships with line managers to support you with ROI implementation?

129

Done

√

MEASURING ROI IN TRAINING

7

Have you done a proper gap analysis to determine gaps in employee performance?

8

Have you used appropriate training needs analysis methods?

9

Do you have an integrated performance-management system to help you identify performance problems?

10

Does your training or workplace skills plan clearly indicate the learning programmes needed to address these skills gaps?

11

Have you identified all the cost items of training?

12

Have you done assessment to compare pre- and post-performance?

13

Have you calculated the ROI ratio of your training programmes?

14

Have you compiled an ROI report?

15

Have you communicated the ROI ratios to management and learners?

16

Have you evaluated the training programme as a result of your ROI results?

17

Can you differentiate between the ROI results of different training programmes?

18

Have you developed a database with ROI measurements?

19

Have you double-checked your ROI calculations for accuracy?

20

Have you implemented improvements to your training plan as a result of the ROI process?

If you have answered "yes" to all of the above questions, you have followed all the steps of the ROI process that are required to accurately calculate and report on ROI. However, if all your questions weren't answered positively, it means you have to go back to the relevant chapter where the particular item is discussed in detail to ensure that you have followed all the steps correctly. Moreover, after every ROI calculation, this checklist can be used as a "self-assessment" tool.

130

CHAPTER 6: EVALUATING THE ROI IMPACT ON TRAINING

SUMMARY The field of ROI is constantly changing, with new measures and tools being developed. There is no right or wrong ROI measurement. You need to take the measures and tools that are applicable to your organisation and determine the most realistic ROI for yourself. This book does not profess that this method of calculating ROI is foolproof. The authors have tried to make the process of determining ROI easier from the training manager's point of view. The more ROI tools you use, the better the result of your analysis. From the discussions in this book it can be stated that by keeping the calculation simple it is easier to motivate your training initiative. Chapter 2 looked at the competence-analysis link to business performance management as a tool to enable you to accurately measure the ROI in training. The steps were followed from performing a gap analysis to determining the current competency level, attaching a weight to the various job functions, determining the measurable outputs, identifying relevant unit standards or exit-level outcomes and the time needed to achieve these outcomes. Once the learner has been assessed, an individual career plan can be compiled and future targets set. The aim of performance management is not only to measure, monitor and manage performance, but also to foster a culture of lifelong learning where learning forms part of the employee's day-to-day duties. In Chapter 3 we looked at the skills implementation process. The learning requirements were identified and aligned to the strategic goals of the business plan. We explored the options of using unit standards as a method of instruction. The learning material has to cover the components of knowledge, skills and behaviour. It is important to determine the level of learning at which the learning material should be pitched to ensure that the desired change in behaviour can take place as well as a transfer of learning. The learner has to demonstrate that a transfer of learning has taken place through demonstrating competence. This judgement decision can be made once the learner has been assessed for competence.

131

MEASURING ROI IN TRAINING

In Chapter 4 we covered the importance of proper assessment for the calculation of ROI in organisations. Assessment plays an important role in showing the legitimacy of training and ensuring that it is meaningful to the organisation in both qualitative and quantitative terms. Of particular importance are evaluation systems that focus on the ROI of training interventions. Further, we demonstrated the role of assessment as the driving mechanism behind the successful implementation of the NQF. Indeed, without proper assessment, the ROI cannot be consistently and effectively calculated and the goal of quality training that adds value to the organisation will not be achieved. It is therefore essential for organisations and training managers to ensure that assessment is an integral part of their people development interventions, that it fulfils the functions of promoting effective training, and that it provides the necessary accountability and facilitates the implementation of the NQF objectives in South Africa. Chapter 5 covered the practical side of determining the cost of learning. The questions that are answered include: z z z

What is the monetary value that learning can add to your organisation? How can you quantify the value of a training programme in monetary terms? How can you motivate the development or purchase of a new training programme?

ROI is about time and money. By determining the amount of time a process will take and the cost of that time, you can determine the return on that investment over a specific period. The measures discussed in this book include: z z

ROI which is the percentage return you will get from your training investment over the respective period. Net present value (NPV) which provides you with a rand value of your expected return and therefore indicates the magnitude of your project. 132

CHAPTER 6: EVALUATING THE ROI IMPACT ON TRAINING

z z

Payback period which is the period during which an investment will show a profit. Internal rate of return (IRR) which equals the percentage rate by which you have to discount the net benefits for your time period until the point when they equal the initial costs.

Each of these financial measures has its own strengths and weaknesses. Different organisations will place varying amounts of emphasis on each of the different metrics. To get a clear and complete picture of a prospective investment, you will benefit from having access to all four of these measures. Chapter 6 covered various discussions regarding the evaluation of the impact on training and the measurement of training effectiveness. A typical ROI report and checklist has been supplied to implement ROI processes in your organisation.

THE WAY FORWARD Now that you have been guided through the full process of measuring your ROI, you can explore our Internet website www.roionline.co.za linked to this book. You are welcome to visit this site and among other features share your ROI experiences with us and others in the learning and training field. The site will contain: z z z z z z z z

the latest updates to the ROI spreadsheets discussed in this book, which you can download. checklists which can also be downloaded. details on the latest ROI-related news. various learning and training-related information. details on workshops and conferences where you can learn more on ROI-related information. details of other ROI books on the market. discussion forums. any ROI-related issues that can help to make your day a successful one. 133

MEASURING ROI IN TRAINING

This will be your site which you can use to share information regarding financial issues linked to training, lessons learned, success stories, areas for improvement, challenges, and to learn from one another. Let's try and make your life easier when it comes to discussing financial issues in the training world. See you on www.roionline.co.za

CONCLUSION Business is about making a difference, adding value to the economy and society, and profiting through this process. The training representative in an organisation needs to understand how the provision of training can financially benefit the organisation and how to communicate the rand value generated as a result of the training initiative to the rest of the organisation. This book attempts to provide you with a logical process to implement and measure the ROI in training, enabling the South African training and development community to benefit from the ROI process.

134

APPENDIX

APPENDIX

ROI RESOURCES

135

MEASURING ROI

IN

TRAINING

In this Appendix we are going to have a brief look at the spreadsheets that are available on the CD Rom accompanying this book. There are many variables to consider when calculating ROI and every project/training intervention situation is different. Therefore different spreadsheets need to be used. The spreadsheets have been designed in Microsoft EXCEL. In order to access this spreadsheet you will be required to have Microsoft Excel software loaded on your computer. You may customise the spreadsheets to suit your specific requirements and make them unique to your situation in order to determine the ROI of your learning training programmes. The spreadsheets that are on the CD Rom are: 1) 2) 3) 4)

ROI spreadsheet – small ROI spreadsheet – medium ROI spreadsheet – large ROI consolidation

ROI SPREADSHEET - SMALL This spreadsheet can be used to quickly determine the ROI of your training initiative. In the spreadsheet there are a number of worksheets which you can access by selecting the appropriate option below: z z

Micro: This is an example of an ROI analysis that was done at Medscheme in the call centre environment. Lite: Can be used to quickly determine the ROI, NPV, IRR, payback period and cost per learner if the respective data is available. Complete the applicable fields marked in blue. The respective costs and benefits must already have been determined before the data can be input into this spreadsheet.

136

APPENDIX

Complete fields in grey

Year 1

ROI – Lite

Year 2

Year 3

118%

266%

435%

ROI

R 16 647

R 157 592

R 317 756

NPV

36%

125%

155%

IRR Payback (months)

8.80 R 2 750

Cost per learner 16% Discount rate

1 000

Delegates Initial Invest Costs GAP analysis

Year 1

Year 2

Year 3

R 10 000 R 100 000

Design & develop

R 200 000

R 180 000

R 160 000

Facilitator

R 300 000

R 300 000

R 300 000

Delegate

R 50 000

R 50 000

R 50 000

Evaluation

R 50 000

R 50 000

R 50 000

Assessment

R 300 000

R 300 000

R 300 000

R 900 000

R 880 000

R 860 000

Benefits

R 400 000

R 400 000

R 400 000

Time saving

R 100 000

R 100 000

R 100 000

Personnel saving

R 300 000

R 300 000

R 300 000

Operational saving

R 200 000

R 200 000

R 200 000

R 50 000

R 100 000

R 150 000

Fixed cost

R 110 000

Total

Revenue impact

R 1 050 000

Other savings

R 1 100 R 1 150 000 000

Total R 1 050 000

Cash flow

R 1 150 000

Benefits

R 110 000

R 900 000

R 1 100

R 860 000

less costs

-R 110 000

R 150 000

000

R 290 000

R 880 000

137

R 220 000

MEASURING ROI

z

IN

TRAINING

Outsource: This spreadsheet can be used when there is no initial cost in the respective training intervention. You are therefore calculating the ROI of a going cost spread over a period less the net benefits derived from that investment. The respective costs and benefits must also already have been determined before you can input the data into this spreadsheet.

Complete fields in grey Course 1

ROI – Outsource ROI

Course 2

Course 3

16%

48%

-1%

R 439 655

R 868 966

R -32 759

R 5 600

R 4 500

R 6 667

500

400

600

R 5 000

R 4 000

R 6 000

NPV Cost per learner 16% Discount rate Delegates per course Turnover per learner per hour

R 100 R 50 1 840

Cost per learner per hour Hours worked per learner per annum Costs Cost per learner/ course

R 2 500 000

Course cost

R 300 000

Other costs Total Benefits Time saving Personnel saving Operational saving

R 2 800 000 % Improve-

R 1 600 R 3 600 000 000

R 400 000

R 200 000 R 4 000 000 R 1 800

R 2 300 000

000 R 2 760 000

5%

R 460 000

R 552 000

1%

R 300 000

R 1 840

R 300 000

R 200 000

000

R 200 000

ment

138

APPENDIX

R 50 000

Revenue impact

R 3 310 000

Other savings

R 368 000

R 150 000

R 300 000 R 3 962 000 R 200 000

Total Cash flow

R 3 310 000

R 100 000 R 3 962 000

Benefits

R 2 800 000

R 2 808 R 4 000 000

R 510 000

less costs z

000

-R 38 000

In-house In-house: Provides an ROI calculation when using internal resources. This is a quick method to determine R 2the 808 ROI regarding a number of learners doing a number of courses per 000 annum. The respective costs and benefits must also already 1 800 have been determined before you can inputRthe data into this spreadsheet. 000

Complete fields in grey ROI – Classroom training (in-house) Units Number of learners Number of courses per year per learner

Invest

R 1 008 Year 1

000 Year 2

Year 3

500 2

Course length (hours)

16

Learners per session

20

Employee rand cost per hour

R 50

Facilitator rand per hour

R 75

R60 000

R60 000

R60 000

Classroom rand cost per day R 100

R10 000

R10 000

R10 000

R800 000 R800 000 R800 000

Course development rand cost per hour R11 000 R352 000 Total cost

-R352 000

R870 000 R870 000 R870 000

Benefits Time saving

R400 000 R400 000 R400 000

Personnel saving

R100 000 R100 000 R100 000

Operational saving

R300 000 R300 000 R300 000

Revenue impact

R200 000 R200 000 R200 000 R50 000 R100 000 R150 000

Other saving

R1 050 000 R1 100 000 R1 150 000

NPV at discount rate of

R-148 276 R353 448 R291 379 44% 169% 100%

16%

ROI

139

MEASURING ROI

z

IN

TRAINING

External: Additional options are provided to include external resources. The respective costs and benefits must already have been determined before you can input the data into this spreadsheet.

The “small spreadsheet” provides you with a quick way – on one page – to determine an ROI result from your training initiative.

ROI SPREADSHEET - MEDIUM The medium spreadsheet provides you, through a number of worksheets and questions, to input more data than was required by the “small spreadsheet”. A number of worksheets, as displayed at the bottom of the spreadsheet, require you to first input data in the “Input Assumption” worksheet and then in the respective worksheets as you progress through the training initiative process. The only two worksheets where no input is required are the “Summary” and the “Cash flow” worksheets. An example of the summary worksheet follows: Cost summary – ROI for course Year 1 ROI

211%

NPV

R 265 654

Payback

Year 2

Year 3 388%

539%

R 690 390 R 1 052 691

4.91 months

period IRR Learning cost

219%

145%

236%

R 1 062

per delegate Cost Gap analysis

R 3 000

Needs analysis

R 3 000

One-on-one interviews Skills audit

R 3 000

Surveys

R 3 000

140

Unit

APPENDIX

Total Design and

Learning

development

material

R 12 000 Design & development R 96 000 Purchase price of existing courseware

R 20 000

Licensing fee

R 10 000

Outcomes-based conver-

R0

sion

R0 R 140 000

R 140

Material duplication

R0

R0

Licensing fee

R0

Development cost

R0

Initial investment

R0

Development

R0

Unit standard conversion Software

e-Learning

Total Facilitation

Trainer/s

fees

cost

Other development costs

R 266 000 R 1 565

Trainer's salary/fee R0 Accommodation cost Travel costs

Total

Daily allowance

Delegate cost

Assessment

Evaluation

R0 R 200 R 1 765 Various

Lost “productivity”

R0

Accommodation cost

R0

Travel costs

R 200

Daily allowance

R 200

Total

R 100

Assessor

Fee pre-assessment

cost

Fee post-assessment Fee

Moderation cost Total

R 10

Behaviour/performance

141

R 100 R 100 R 300 R 200

MEASURING ROI

IN

TRAINING

Evaluator cost Fixed costs

Total Floor space Equipment

R 200 R 400

Development plan

R 1 225 Rental of floor space

R0

Hardware & software

R0

Other equipment R 1 875

Other fixed Year-on-year Gap analysis Design and development Licence fees Facilitation fees

R 3 100

costs

Total Initial cost

Year 2

Year 3

Year 1

R 12 000 R 256 000

R 8 000

R 44 130

R 35 304

R0

R 10 000

Delegate cost

R 9 000

R 52 957

– lost productivity

R 1 440 000 R 1 280 000

Delegate cost

R 1 600 000

– other

R 180 000

R 160 000

Assessment

R 200 000

R 270 000

R 240 000

Evaluation

R 300 000

R 360 000

R 320 000

Fixed costs

R 400 000

R 3 100

R 3 100

T otals Totals

R 3 100 R 278 000

R 2 556 057

R 2 306 230 R 2 046 404

This spreadsheet will typically be used when motivating the development or purchase of a new training intervention. Each of the phases have been detailed, with the various costing elements that need to be identified. Complete the spreadsheet, starting with the “Input Assumption” and progressing to the “Benefits” worksheet. An example of the “Input Assumption” worksheet is as follows:

142

APPENDIX

Assumptions

Year 1

Year 2

Year 3

1 000

900

800

1) Number of expected delegates to do this course per annum? 2) Number of employees in the organisation?

2 000

3) What is the current interest rate?

16%

Result The number of productive hours per employee per annum are:

1 840

The employee turnover per hour (mandatory)

R 200

The employee cost per hour (mandatory)

R 100

ROI SPREADSHEET - LARGE The major difference between the medium and the large spreadsheet is that the 'Input assumption' worksheet requires you to provide more detailed data with regard to the organisational information. See copy of worksheet below: Assumptions

Year 1

Year 2

Year 3

1) Number of expected delegates to do this course per annum?

1 000

900

800

2 000

2 000

2) What is the number of productive weeks per annum?

46

3) What is the number of productive hours per week?

40

4) What is the annual organisational turnover per employee? a) Number of employees in the organisation? b) Annual turnover for the organisation?

2 000

R300 000 R320 000 R340 000 000

5) What is the average annual employee salary? R 70 000 6) What are the other average annual employee

143

000

000

R 50 000 R 60 000

MEASURING ROI

IN

TRAINING

costs?

R 30 000 R 30 000 R 30 000

7) What is the current interest rate?

16%

8) How many courses are currently offered?

20

9) How many of those courses use computers? 0 Result 10) The number of productive hours per employee per annum are:

1 840

11) The employee turnover per hour is:

R 82

R 87

R 92

12) The employee cost per hour is:

R 43

R 49

R 54

Use this spreadsheet when you have access to the applicable information. The more information that is available, the more realistic the ROI calculations will be.

ROI CONSOLIDATION This spreadsheet enables you to input the result from all the ROI calculations you have made and then obtain an average ROI measurement from all the information you have provided. Below is an example: Organisation Name: Date: Compiled by: ROI results for all training programmes in organisation Total results from programmes below

251%

R 398 754 7 40

2 ROI (rands)

2 NPV

129%

R 1,261

Number of programmes included Training programme name

2 2 Payback IRR period (months)

1 L earning Learning cost per delegate

Induction to Organisation 291%

R 531 965 9.88

112%

R 1 460

Microsoft Word

R 265 543 4.91

145%

R 1 062

211%

144

APPENDIX

Once completed you will have an average of all the ROI calculations for the organisation. You can use this information in your reports to management when motivating training initiatives or when reporting on training that has taken place.

FREQUENTLY

ASKED QUESTIONS ABOUT

ROI

These are some of the frequently asked questions posed to us during ROI training sessions, conferences and consulting sessions: Q:

What is ROI?

A: ROI is a measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme. In other words, it is the extent to which the benefits (outputs) of training exceed the costs (inputs). Q:

Why is it essential to determine the ROI of training programmes?

A:

Calculating ROI gives the training manager an opportunity to measure the financial value of a training programme. This helps the training manager to improve his or her credibility in the organisation by being able to talk the language of line management in showing the tangible benefits of training.

Q:

If ROI is such an important concept, why are so few training managers actually doing it?

A:

Training managers realise the importance of ROI, but they do not know how to measure and calculate ROI. Training managers are more interested in the number of courses they run than in the value these courses add to the organisation.

Q:

What are some of the pitfalls regarding ROI?

145

MEASURING ROI

IN

TRAINING

A:

The most important pitfall is to compare apples with pears! Make sure that you measure what you are supposed to measure, and that you know all the various cost and benefit items so that you can calculate an accurate and realistic ROI.

Q:

What is the relevance of performance management for ROI?

A:

Performance management is of paramount importance for ROI. In fact a well-developed performance management system that includes clear measures forms the basis of any ROI system. Identifying performance gaps and addressing these gaps by means of training can provide useful information for ROI measurement.

Q:

How can ROI support the balanced scorecard?

A:

ROI provides information that can be used as the learning and growth component of the balanced scorecard. By indicating specific training measures, tangible data is created that can be used to implement the balanced scorecard.

Q:

What is the difference between the ROI ratio and NPV?

A:

The ROI ratio indicates the percentage return on the investment (eg.150%), while NPV gives you the rand value of the return on the training investment (eg. R50 000).

Q:

What is a good ROI ratio?

A:

There is no rule pertaining to what a good ROI ratio is. Each training course is different and different ROI ratios are therefore expected. However, for certain types of training, such as sales training, ROI ratios of more than 500% have been reported. In general, ROI ratios between 0% and 5 000% have been achieved.

Q:

If a low ROI, let us say 10%, has been reported, does it mean that the training was bad? 146

APPENDIX

A:

Not necessarily. A low ROI cannot always be attributed to ineffective training. There can be many factors in the work environment which have an impact on ROI, e.g. support provided by supervisors and colleagues, the culture of the organisation, market conditions etc.

Q:

Is it difficult to calculate ROI?

A:

Not at all, in fact, it is quite easy. All you need to do is some form of a pre- and post-measurement of performance, as well as determining all your costs and the financial benefits of training.

Q:

Should ROI be calculated before or after training?

A:

Before and after training. ROI can be forecast before training in order to determine whether it is indeed worthwhile to continue with the training. ROI should also be calculated after training in order to determine the real costs and benefits of the training.

Q:

Can ROI be calculated for soft-skills training?

A:

While it is certainly more difficult to calculate ROI for soft skills training than for technical or other training, it is indeed possible to calculate ROI for soft-skills training. The challenge is to determine the cost items and benefits of training. The solution does not lie in the 'skills', but in the outputs achieved in terms of the financial benefits as a result of the training. For example, when evaluating the ROI of a diversity training programme, consider all the potential financial benefits that can be measured, e.g. more business from diverse markets, less grievances relating to diversity, higher employee retention, less absenteeism etc.

147

MEASURING ROI

IN

TRAINING

ROI

INTERNET RESOURCES

American Society for Training and Development (ASTD): www.astd.org ASTD Global Network South Africa: www.astd.co.za ASTD ROI Network: www.roi.astd.org Learnativity: www.learnativity.com ROI in Training and Performance Improvement: www.maintenanceresources.com ROI Online: www.roionline.co.za School for Champions: www.schoolforchampions.com NCVER Research Forums: www.ncver.edu.au Educational Resources Information Center: www.ericacve.org Medscheme case study Medscheme places a high premium on their relationship with their customers and has a strong focus on a sound customer care culture of delivering excellent service. The first exercise they embarked on to measure ROI was for a group of 900 call centre agents at the Florida branch with the aim of training to deliver and maintain excellent service to customers. Pre-assessments were done on the number of calls answered, the number of calls resolved, the number of calls that couldn't be resolved on first contact, and voice assessments according to some of the following criteria; correct greeting, correct method of addressing the caller, non-use of jargon, etc. Medscheme sourced the services of Service Quality Institute (SQI) who customised their material to suit the needs of Medscheme. The material is written in the outcomes-based format and aligned 148

APPENDIX

to the principles of the National Qualifications Framework (NQF), as well as being mapped to unit standards. The outcomes that the learners need to be competent in were identified by doing a gap analysis through pre- and post measurement and are covered in the learning material. The costs and benefits were calculated on the following basis: z

z

z

Contact centre agents' costs include salary, floor space, telephone and p.c. costs divided by the number of telephone calls that are answered to determine the cost per call to the organisation. An average salary for the department was used. All costs associated with training were listed as per the spreadsheet; example facilitation, assessment, lost of productivity, etc. Benefits were calculated on the saving due to the increased output (number of calls). The calculated saving of 9c per call equates to a saving of R407.00 per learner per month.

The cost of the material was a once-off fee of R270,000. which includes the licensing rights that allow the organisation to use the module other than for the 900 call centre agents who attended training over a period of one year. The calculation is based on the use of the material over a period of three years. This equates to R100.00 per learner per module. Post assessments were done that indicated that the agents were able to answer more calls, resolve more call queries, as well as a reduction in the error rate of the calls answered. The increased number of calls that were answered reflects the new cost of the agent to the organisation. The cost to the organisation for answering a telephone call is now lower due to the training that was attended which brought about a positive change in behavior. In order to gain the maximum benefit of the ROI process, the learners' needs are to be identified and recorded on a Personal Development Plan (PDP), that should form part of the Perfor149

MEASURING ROI

IN

TRAINING

mance Management process to enable the learner to take control of his/her own learning, and their manager that can now record progress and identify further areas of development after assessment. The ROI in training was calculated according to the following method: Benefits - costs ÷ original cost in the training investment expressed as a %. An ROI of 521% was achieved. The cost of learning per learner was recovered over a period less than three months. Please note that averages were used in this example to protect business intelligence.

150

151 -

90 000 90 000

Less costs

Total

Benefits

Total costs

Other costs - printing

Sub total

Other costs

90 000

318

Delegate Productivity loss

30

845

Sub total

407

1 193

30

1 163

-

1 163

409

Assessment

Sub total

159

Evaluation

100 18

90 000

Per delegate 159

Month

Facilitator

Design & Develop

GAP Analysis

Number of delegates to attend

ROI

ROI - Micro

-

70 783

107 411

36 628

107 411

2 700

104 711

-

104 711

28 636

76 075

36 818

14 318

1 620

9 000

14 318

90

469 020

1 728 655

2 197 674

1 728 655

27 000

1 701 655

1 701 655

Benefit 1 learner, 1 month * 6 month's benefit

286 364 7,000/22

1 415 291

736 364 12,000/22/8*2

286 364

16 200 12,000/22=545*30=16,350/900=18

90 000 270,000/3 years

286 364 Salary 14,000/22/8*2

Year 1

900

Per month Year 1 Explanation for 90 delegates -79% 521%

APPENDIX

152

Benefit 900 learners per year

36 628

0.09

Benefit 90 learners per month

1.54

406.98

4 550

Output after training (calls)

1.63

7 000.00

Cost

Per month for 90 delegates Benefits

Benefit due to training

190

Increased output after training

20

4 590

Number of output after training

Error rate after training

4 300

50

4 400

Total calls before training

Error rate on Output (calls)

Number of Monthly Output (calls)

Monthly Salary

Units

Month

407 benefit * 90 learners

4,550* 9c saving = 407.

7,000/4,550 = 1.54 per call

4,590-20-20

Error calls use twice as much time to resolve 4,590 calls include 20 incorrectly answered

7,000/4,300 = 1.63 per call

Includes floor space, p.c. & telephone 4,400 include 50 calls incorrectly answered

Explanation

366 279 407. benefit * 900 learners

Year 1

IN

Productivity Calculation

ROI - Micro

MEASURING ROI TRAINING

BIBLIOGRAPHY American Society for Training and Development. 2002. ASTD state of the industry report. Alexandra: ASTD. ASTD see American Society for Training and Development. ASTD Global Network South Africa. 2003. The 2002 ASTD state of the South African training industry report. Johannesburg. Bassi, L.J., Ludwig, J., McMurrer, D.P. & Van Buren, M. 2000. Profiting from learning: Do firms' investments in education and training pay off? ASTD Research White Paper. Alexandra: ASTD. Cost and Management Accounting. November 1998. Fitz-ens, J. 2000. The ROI of Human Capital: Measuring the Economic Value of Employee Performance. New York: Amacom. Goleman, D. 1995. Emotional Intelligence. New York: Bantam. Holcomb, J. 1998. Training Evaluation Made Easy: Make Your Training Worth Every Penny. London: Kogan Page. Jackson, S.E. & Schuler, R.S. 2000. Managing human resources: A partnership perspective. Cincinnati: South-Western. Kirkpatrick, D.L. 1998. Evaluating Training Programs: The Four Levels. 2nd ed. San Francisco: Berrett-Koehler. Long, L.N. 1999. ROI: Capturing the big picture. Technical Training, November/December: 31-33. Merwin, S. 1992. Evaluation: 10 Significant Ways for Measuring and Improving Training Impact. San Francisco: Berrett-Koehler. Meyer, M. (ed.) 2002. Managing human resource development: An outcomes-based approach. 2nd ed. Durban: LexisNexis-Butterworths. 153

MEASURING ROI IN TRAINING

Meyer, M., Mabaso, J. & Lancaster, K. 2001. ETD practices in South Africa. Durban: Butterworths. Phillips, J.J. 1996. Measuring ROI. Technical and Skills Training, April: 10-13. Phillips, J.J. (ed). 1994. In Action: Measuring Return on Investment. Alexandra: ASTD. Phillips, J.J. 1997. Return on investment. Houston: Gulf. Phillips, J.J. 1999. ROI process. Birmingham, Ala.: The Jack Phillips Center for Research. Phillips, J.J. 1997. Handbook of Training Evaluation and Measurement Methods. 3rd ed. Houston: Gulf. RSA 1995. South African Qualifications Authority Act (Act No 58 of 1995) Government Notice No 1521. Government Gazette 364 (16725) (4 October). Pretoria: Government Printer. RSA 1998a. Regulations under the South African Qualifications Authority Act (Act No 58 of 1995) (short title: National Standards Bodies Regulations, 1998). Government Gazette 393 (18787) (28 March). Pretoria: Government Printer. RSA 1998c. Skills Development Act (Act No 97 of 1998). Government Gazette 401 (19420) (2 November). Pretoria: Government Printer. RSA 1999a. Skills Development Levies Act (Act No 9 of 1999). Government Gazette 406 (19984) (30 April). Pretoria: Government Printer. RSA 1999b. Commencement of the Skills Development Act 1998 (Act No 97 of 1998). Government Gazette 411 (20442): 2 (7 September). Pretoria: Government Printer.

154

BIBLIOGRAPHY

RSA 1999c. Regulations Regarding the Establishment of Sector Education and Training Authorities (SETAs). Government Gazette 411 (29442): 4-7 (7 September). Pretoria: Government Printer. RSA 1999d. List and Scope of Coverage of Sector Education and Training Authorities (SETAs). Government Gazette 411 (20442): 1224 (7 September). Pretoria: Government Printer. RSA 2000. 10 Norms and Standards for Educators. Government Gazette 20844 (February). Pretoria: Government Printer. Rule, K.E. 1998. The "three Rs" of ROI. Technical Training, May/ June: 26-29. Russ-Eft, D. & Preskill, H. 2001. Evaluation in Organizations: A systematic approach to enhancing learning, performance and change. Cambridge, MA: Perseus. SAQA 1998. Criteria for the Generation and Evaluation of Standards and Qualifications on the National Qualifications Framework. SAQA Bulletin 2 (1) (August). Pretoria: South African Qualifications Authority. SAQA 1999. Draft 5: Criteria and Guidelines for ETQAs. Unpublished paper. Pretoria: South African Qualifications Authority. Shepherd, C. 2002. Assessing the ROI of training [online]. Available on the Internet at: http://www.fastrak-consulting.co.uk (24 January 2002).

155

MEASURING ROI IN TRAINING

156

GLOSSARY Assessment is the process of measuring applied competence by identifying and interpreting information about a learner’s achievement and observing and evaluating the demonstration of such competence. It is the process of identifying what an individual knows and can do. Balanced scorecard is an integrated performance-management system that provides an opportunity to measure business performance at four levels: financial performance, customer perspective, internal processes, as well as learning and growth as direct measures to achieve the overall goals of the company. It is used to translate our strategic objectives (business plans) into measurable targets for the organisation as a whole, and for all business units, teams and individuals through a balanced approach. Competence is being able to perform whole work roles to the standards or outcomes expected in employment in real working environments and not only in a training situation. Competence is about the ability to work with your hands (skills), knowledge (mind), and heart (attitudes, values or behaviour) Credits indicate the weight assigned to unit standards and determine the average time it should take to acquire competence. The number of credits is therefore an indication of the quantity and depth required for completion of a unit standard. Critical cross-field outcomes are generic abilities that cut across all areas of learning and include interpersonal skills such as communication and problem-solving skills. Emotional quotient (EQ) is the intelligence that is used as a source of human energy to connect with people and attune us to the emotions of ourselves and other people. It is the learned abilities that demonstrate that we have the potential to become skilled at relationship management and social awareness.

157

MEASURING ROI IN TRAINING

Gap analysis is the difference between the learner’s competence level and the required competency level for the specific job and organisation. The difference is the gap in performance that was identified. Internal rate of return (IRR) is used to analyse multi-year investments. IRR equals the percentage rate by which you have to discount the net benefits for your time period until the point that they equal the initial costs. Key performance indicators (KPI) are the crucial, job-specific requirements that the learner needs to be able to perform for certain job roles. Knowledge or content is what the learner knows. Knowledge items may include the following: concepts, facts, definitions, theories, principles and rules. Levels are the positions on the National Qualifications Framework where national unit standards are registered and qualifications awarded. Levels are arranged to indicate increasing complexity in learning and to facilitate meaningful progression routes along career and learning pathways. Moderation is the process of ensuring that assessors are assessing work according to agreed standards, and that there is consistency among different assessors. National Qualifications Framework (NQF) is an integrated framework comprised of registered standards, units and qualifications at eight levels of education and training. Net present value (NPV) provides you with a rand value of your expected return and therefore indicates the magnitude of your project. The ratio is calculated by adding the present value of the net benefits for each year over a specified period of time and then subtracting the initial costs of the project.

158

GLOSSARY

Notional hours represent the average time a learner should need to achieve competence and include contact time, reading and time on the job required for completion of a unit standard. One (1) credit indicates 10 hours of notional time. Outcome is a detailed statement in a standard or qualification of the applied competence and indicates the end product of a learning process, in other words, what the learner is able to do. Payback period is used to find out how long it will take for an investment to show a profit. It is the time it takes for your project to recoup the funds spent. Personal development plan (PDP) contains the development needs for the individual and the job-specific and generic abilities. Performance management is the tool used as a measurement to link individual performance to the achievement of organisational effectiveness.. Return on investment (ROI) is a measure of the monetary benefits obtained by an organisation over a specified period in return for a given investment in a learning programme. In other words, it is the extent to which the benefits (outputs) of training exceed the costs (inputs). ROI equals the present value of your accumulated net benefits (gross benefits less ongoing costs) over a certain time period, divided by your initial costs. Skills are learned and practised through various tasks that require learners to demonstrate their skills. These skills may be either psychomotor, which requires learners to do something physically, or they may be intellectual such as problem-solving or working in groups.. South African Qualifications Authority (SAQA) is the statutory body responsible for ensuring the development and implementation of a National Qualifications Framework which contributes to the full development of each learner and to the social and economic development of the nation at large. 159

MEASURING ROI IN TRAINING

Skills development matrix is the document that contains all the skills-development requirements for the individual.. Specific outcomes are statements of what the learners are capable of knowing and doing at the end of a learning experience in a particular field of learning. Standard is a detailed description of a component of applied competence, serving as a criterion against which someone may be assessed for entering or exiting a learning programme, occupational level or profession. Standards generating bodies (SGBs) are the bodies which generate the standards for specified sub-fields before submitting such standards to the relevant National Standards Body. Standard-setting is the national process of developing the standards against which people may be assessed in determining their eligibility for a qualification. Training entails a programme that assists the learner in acquiring specific skills that he or she can effectively apply to a very specific job or task. Training needs analysis is the process of identifying an issue or problem, collecting, analysing and interpreting data, and using the information obtained to select or design an appropriate training intervention to address the issue or problem. Unit standards are nationally agreed and internationally comparable statements of the specific outcomes to be achieved by an individual in order to obtain credit for the unit, together with administrative and other information as prescribed in the format agreed by the South African Qualifications Authority.

160

161

MEASURING ROI IN TRAINING

162

INDEX

INDEX A American Society for Training and Development (ASTD) – 3, 7, 10 ASTD Global Network South Africa – 4, 10 Annual training report (ATR) – 65, 73 Assessment – 6, 18, 19, 25, 28 - 31, 33 - 34, 42 - 45, 47 - 48, 63, 67 - 69, 73 - 74, 77 - 94, 105 - 106, 114, 119, 121, 123, 127, 130, 132 B Balanced score card (BSC) – 19, 40, 42, 52, 53, 60, 66 Benefits – gross, operations, personnel, revenue, time saving, budget – 5, 6, 9, 12, 14, 15, 18, 19, 24, 48, 54, 62, 95, 97, 108 114, 120, 122, 129, 133 C Calculate – 6, 8, 12 - 14, 19, 21, 24, 30, 35, 44, 60, 65, 67, 72 - 74, 87, 93, 97, 102, 111, 112, 115, 116 Career plan – 26, 60, 131 Cash flow – 94, 97, 113, 114, 115 Computer – 34, 35, 38, 39, 43, 50, 53 - 58, 64, 65, 75, 93, 100, 107, 108, 112, 121 Communicate – 17, 25, 42, 48, 57, 86, 88, 120, 123, 128, 129, 130, 134 Competence – 19, 26, 27, 28, 43, 44, 46, 51, 52, 59, 60, 64, 66, 67, 68, 70, 71, 73, 74, 78, 79, 80, 81, 82, 84, 85, 88, 105, 109, 112, 121, 127, 131 Corporate culture – 37 Costs – delegate, fixed, initial, learning, ongoing, trainers – 5, 6, 11 - 14, 18 - 20, 31, 62, 72, 83, 94 - 100, 102 - 108, 112, 114, 121, 123, 125, 133 Credits – 15, 18, 19, 44, 70, 71 Critical cross-field outcomes (CCFO) – 19, 26, 34, 35, 36, 37, 39, 50, 62, 65, 68, 84, 85 D Delegate – 94, 98 - 99, 102 - 106, 109 - 114, 121 Design & development – 18, 19, 31, 72, 101, 102, 114, 121 E Emotional quotient (EQ) – 37 163

MEASURING ROI IN TRAINING

Equipment – 31, 106, 107, 108 Education and Training Quality Assurance (ETQA) – 33 Evaluation – 2 - 5, 12 - 14, 16 - 19, 25, 31, 33, 68, 77, 88, 89, 94, 106, 114, 118, 119, 121, 128, 132, 133 F Facilitation – 18, 19, 27, 31, 69, 104, 121, 123 Feedback, 360-degree – 34, 47, 48 Financial, measures – 41, 98, 133 Fixed costs – 18, 19, 31, 94, 106, 108, 114, 121 Floor space – 31, 107 G Gap analysis – 19, 27, 28, 31, 35, 50, 52, 60, 62, 63, 81, 82, 93, 100, 101, 114, 121, 126, 130, 131 I Implement – 15, 16, 20, 27, 30, 35, 62, 78, 86, 118, 124, 129, 134 Implementation – 12, 13, 16, 28, 32, 64, 74, 75, 78, 86, 89, 112, 118, 122 - 126, 129, 131, 132 Input worksheet – 108 Internal rate of return (IRR) – 97, 98, 113, 119, 122, 123, 133 Internet, site, based – 19, 21, 26, 31, 50, 65, 67, 69, 72, 83, 88, 94, 101 - 103, 111, 112, 133, 134 Instructor-led – 94, 101 J Job description – 18, 19, 34, 46, 52, Job roles – 25, 29, 33, 34 K Key performance indicators (KPI) – 18, 27, 29, 30, 34, 35, 37, 39, 50, 52, 53, 59 King Report – 7, 10 Knowledge – 3, 15, 25, 34, 35, 37, 38, 49, 50, 54, 55, 63 - 66, 68 71, 73, 74, 79 - 82, 84, 121, 124, 125, 127, 129, 131 L Learning, costs, intervention, material – 19, 31, 63 - 68, 70, 73, 74, 84, 92, 94, 95, 98, 99, 101, 108, 113, 115, 131 Learning organisation – 27, 41 Level descriptors – 65, 72, 73 Learning programme – 5, 36, 70, 86, 88, 127 M Material duplication – 31, 102, 103, 112 164

INDEX

Measure, measurement – 2, 3, 5 - 7, 9 - 11, 15 - 19, 21, 24, 25, 29, 30, 32, 33, 35, 39, 41 - 43, 48, 55 - 60, 62, 71, 78, 79, 80, 86, 87, 92, 115, 118, 119, 122 - 124, 127, 129, 131, 133, 134 Moderation – 31 Moderator – 105 Money – 6, 9, 24, 32, 72, 73, 92, 97, 99, 132 N National Qualifications Framework (NQF) – 15, 18, 32, 33, 43, 65, 67, 70, 72, 73, 83, 84, 86, 88, 89, 122, 132, Needs analysis – 30, 31, 82, 100, 119, 123, 126, 130 Net present value (NPV) – 96 - 98, 113, 119, 122, 123, 132 Non-tangible cost – 98 Notional hours – 44 O Off-the-shelf material – 65, 67, 72, 102 Organisation – 2, 3, 5, 6, 9, 12, 13, 17, 18, 20, 24, - 27, 29, 32, 33, 36, 38, 40 - 42, 46, 47, 51, 52, 57, 60, 62 - 64, 66, 67, 78, 79, 83, 86, 87, 89, 92, 96 - 99, 106 - 113, 115, 118 - 120, 122, 123, 125, 127, 129, 131 - 133 Outcome – 34, 44, 45, 74, 80, 102, 109 - 111, 126 Outcomes-based – 31, 65, 67, 72, 102 P Payback period – 96, 97, 113, 119, 133 Percentage – 12, 95 - 97, 107, 109, 110, 119, 127, 132, 133 Performance management – 15, 17, 18, 23 - 30, 33 - 35, 37, 39 43, 45 - 47, 49, 51 - 53, 59, 60, 62, 64, 65, 83, 100, 126, 130, 131 Personal development plan (PDP) – 50 Point scale – 34, 50 Present value – 95 - 98, 122, 132 Process flow – 19, 27, 59, 64, 81, 94, 118 Productivity – 3, 6, 11, 98, 108, 111, 121, 125, 129 Q Quality – 11, 14, 33, 73, 79, 80, 89, 132 R Recommendation – 120, 121 Return on investment – 2, 5, 12, 19, 24, 92 ROI, checklist, ROIONLINE, report – 6, 8, 20, 49, 59, 84, 85, 93, 118, 119, 124, 127, 129, 130, 133, 134 165

MEASURING ROI IN TRAINING

S South African Excellence Foundation (SAEF) – 10 South African Qualifications Authority (SAQA) – 24, 32, 33, 35, 67, 71 Saving – 86, 109 - 112, 114 Skills – 3, 5 - 9, 12, 14, 15, 17 - 19, 24 - 28, 30 - 32, 36 - 39, 41, 49, 50, 52 - 57, 59, 61 - 71, 73 - 75, 79 - 86, 100, 122 - 124, 126 - 131 Skills audit – 19, 30, 34, 59, 65, 100 Skills Development Act – 5, 9, 24, 32, 126 Skills Development Levies Act – 5, 9, 24 Skills development matrix – 28, 52, 53, 59 Software – 7, 31, 93, 101, 103, 107 Specific outcome – 80 Standard – 32, 43, 45, 48, 52, 56, 67, 68, 71, 73, 78, 84, 85, 102, 110, 122 Standard setting – 32 Strategic – 19, 24, 26, 33, 35, 40, 42, 53, 58, 62, 66, 74, 77, 123, 125, 126, 129, 131 Summary, worksheet – 10, 13, 93, 95, 98 - 101, 103 - 108, 113 115, 131 T Time, period – 37 - 39, 43 - 45, 49, 52, 54, 55, 57, 59, 60, 63, 65, 67, 70 - 73, 78, 79, 81, 82, 85, 86, 95 - 107, 109 - 111, 114, 127, 131 - 133 Technology-based – 94, 101, 103, 112 Television-based – 94, 112 Training, programme – 3, 11, 12, 15, 18, 20, 78, 79, 83, 86, 87, 92 - 109, 111 - 115, 118 - 122, 129, 130, 132 U Unit standard – 43, 52, 67, 68, 71, 73, 84, 85, 102, 110 V Value – 2, 4 - 8, 10, 12 - 16, 18, 21, 24, 29, 33 - 35, 37, 41, 43, 58, 62, 64, 86 - 89, 92, 95 - 98, 109, 111, 112, 117 - 120, 122, 124, 125, 127, 132, 134 W Workplace Skills Plan – 7, 18, 62, 74, 123, 126, 130

166