Behavioral Competencies Of Digital Professionals: Understanding The Role Of Emotional Intelligence 3030335771, 9783030335779, 9783030335786

Table of contents :
Foreword......Page 5
Preface......Page 9
Acknowledgments......Page 11
Contents......Page 13
About the authors......Page 14
List of Figures......Page 17
List of Tables......Page 18
Chapter 1: The Organizational Challenges of Big Data......Page 19
1.1 Competing in a Data-Driven Age......Page 20
1.2 Data Science: How to Extract Value from Big Data......Page 24
1.3 Organizational Challenges for Leveraging Big Data......Page 28
1.4 The Emotional Intelligence Side of Big Data......Page 31
1.5 The Aim of This Book and Its Structure......Page 32
References......Page 34
Chapter 2: How Big Data Creates New Job Opportunities: Skill Profiles of Emerging Professional Roles......Page 38
2.1 The Market for Big Data Jobs: Trends and Skills Shortage......Page 39
2.2 Professional Roles in the Data Science Field......Page 40
2.3 Data Scientist: Is It Still the Twenty-First century’s Sexiest Job?......Page 45
2.4 Data Analyst Versus Business Analyst......Page 50
2.5 Future Challenges for the Big Data Profession......Page 52
References......Page 54
Chapter 3: Emotional and Social Intelligence Competencies in the Digital Era......Page 57
3.1 Emotional Intelligence and Data Driven Organizations......Page 58
3.2 Competency-Based Approach and Emotional Intelligence: The Importance in the Workplace......Page 59
3.3 Emotional Intelligence and Work Environment......Page 62
3.4 The Behavioral Competencies Necessary in Today’s Workplace: The Development of a Competency Framework......Page 63
3.5 The Application of a Behavioral Competency Framework to a Changing Digitalized World......Page 72
References......Page 73
Chapter 4: When Hard Skills Are Not Enough: Behavioral Competencies of Data Scientists and Data Analysts......Page 79
4.1 Emotional Intelligence and Behavioral Competencies of Big Data Professionals......Page 80
4.2 Data Scientists and Data Analysts in the Italian Context: An Empirical Study......Page 82
4.3 The Behavioral Competency Profiles of Data Scientists and Data Analysts......Page 85
4.4 Action Competencies......Page 86
4.5 Social Competencies......Page 91
4.6 Awareness Competencies......Page 93
4.7 Cognitive Competencies......Page 94
4.8 Exploratory Competencies......Page 96
4.9 Organizational Action Competencies......Page 97
4.10 The Competency Profile of Data Scientists and Data Analysts: Concluding Remarks......Page 99
References......Page 101
Chapter 5: Managing Big Data Professionals through a Competency-Based Approach......Page 104
5.1 Behavioral Competencies as a Decisive Factor in Hiring Outstanding Big Data Professionals......Page 105
5.2 Developing Behavioral Competencies in Data Science and Big Data Academic Programs......Page 107
5.3 Introducing Wake-Up Calls in Data Science and Big Data Educational Programs......Page 110
5.4 How to Make Hiring more Emotional and Successful......Page 116
References......Page 118
Index......Page 122

Citation preview

Behavioral Competencies of Digital Professionals Understanding the Role of Emotional Intelligence Sara Bonesso Elena Bruni Fabrizio Gerli

Behavioral Competencies of Digital Professionals “I encounter many data scientists and analysts whose sole focus is solving analytical problems and developing accurate models. They are not very effective in their roles because they can’t build trust and interact effectively with people. They all need to read this excellent book and adopt its recommendations!”. —Thomas H. Davenport, Distinguished Professor, Babson College, Research Fellow, MIT Initiative on the Digital Economy, Author of Competing on Analytics and The AI Advantage “Big data, Digital disruption, new jobs and competencies: we are familiar with the big picture but we are not equipped to have a practical and helpful framework to guide us. Sure, technical skills will remain necessary but are not sufficient. This book provides a compelling, credible and sound narrative to de-code complexity by developing a set of competencies (action, social, awareness, cognitive, exploration and organizational) supported by emotional intelligence. A must read for Leaders and HR practitioners, for the intellectual curious eager to understand that Human Beings will have to remain central to Human Development”. —Paolo Gallo, Author, Executive Coach, former CHRO at World Economic Forum, World Bank and European Bank, www.paologallo.net

Sara Bonesso • Elena Bruni Fabrizio Gerli

Behavioral Competencies of Digital Professionals Understanding the Role of Emotional Intelligence

Sara Bonesso Department of Management Ca’ Foscari University of Venice Venice, Italy Fabrizio Gerli Department of Management Ca’ Foscari University of Venice Venice, Italy

Elena Bruni Department of Management Ca’ Foscari University of Venice Venice, Italy Department of Business and Management LUISS Guido Carli University Rome, Italy

ISBN 978-3-030-33577-9    ISBN 978-3-030-33578-6 (eBook) https://doi.org/10.1007/978-3-030-33578-6 © The Editor(s) (if applicable) and The Author(s), under exclusive licence to Springer Nature Switzerland AG 2020 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Palgrave Pivot imprint is published by the registered company Springer Nature Switzerland AG. The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Foreword

Titles of books are a major challenge. I am seriously bad at it. For example, I titled my first book of research on competencies predicting outstanding and superior performance in the private and public sector, The Competent Manager. If I had known that same year, 1982, In Search of Excellence would sell 4 million copies in its first couple of years, I could have used that title and invited readers to a more exciting experience. After all, that was exactly what I was describing with a voluminous set of data. The German publishers all but insured that our 2002 international best seller, Primal Leadership, would not even make up the advance in Germany by calling it The Emotional Leader. Titles are a tough choice. Don’t be misled by the somewhat boring title of Sara Bonesso’s, Elena Bruni’s, and Fabrizio Gerli’s latest book, Behavioral Competencies of Digital Professionals. They have written the Rosetta Stone of the digital mind and lifestyle! It is an exciting book about how professionals in our digital age can navigate the interpersonal and conceptual domain of their subordinates and colleagues, competitors and hard technology to adapt, innovate and perform better than others. Working with technology and digital transformations is not an individual effort, it is a team sport. Without others, no one will buy your goods or services, no one will remain working with you, and your great ideas will be relegated to the trash heap of things that “could have been.” Let me explain two underlying discoveries that illustrate how important their work about behavioral and emotional intelligence competencies are to digital work. The first discovery comes to us from neuroscience. Professor Anthony Jack’s opposing domains theory and work on opposing v

vi 

FOREWORD

poles of reasoning has shown that among the many neural networks, two are particularly important to our work: the Analytic Network (formally called the Task Positive Network); and the Empathic Network (formally called the Default Mode Network). The Analytic Network enables us to solve problems, make decisions and focus our perceptual and mental work. Any time we engage in analytic work with abstractions, like building an information system or securing a system from cybercrime, or numerical work, like analyzing financial data, we use the Analytic Network. When companies place major emphasis on the financial performance, metrics or goals, they activate the AN repeatedly. People who go into financial, analytic, software and digital work, often have a disposition to engage in such activities. People who have IQs above the normal range are also disposed to be analytic and think in abstractions. The Empathic Network enables us to be open to new ideas, scan the environment for observations of trends and patterns, be open to people and emotions. We need this network when interacting with others, understanding them or learning and innovating. When organizations emphasize staying in touch with customers, patient experience, understanding your staff, they emphasize the EN. Sadly, these two neural networks suppress each other! Yes, activating one suppresses the other. Activating one repeatedly, suppresses the other repeatedly. In fact, activating one repeatedly on top of a possible pre-­ disposition to engage that network over the other is a recipe for narrow minded approaches to anything! Often, the appeal of digital work is greater to people more comfortable with the AN than the EN – and the nature of the work feeds that predisposition and over-emphasizes the AN over the EN. In the past, many scholars and consultants have discussed the differences in management or leadership styles and approach of those that are task versus people oriented. This is further exaggerated by people claiming rational versus emotional difference sin approaches to thinking. The underlying causes of these distinctions are these two networks. Both the AN and EN are cognitive processes. Both involve reasoning. But they base the reasoning on different stimuli. Digital work invites a lopsided activation at work which can easily contagiously spread to how people live their lives. This new book on behavioral competencies and in particular EI helps the reader orient themselves toward a more effective balance of the AN

 FOREWORD 

vii

and EN.  It helps to establish the empirical basis for working with both neural networks. It explains and shines a light on how the intricate combination and integration of a person’s cognitive and emotional competencies results in outstanding performance. But the person is not static. The second major discovery is about change. The last 25 years of medical and behavioral research has shown us that humans are malleable from how we act with others to our DNA. Yes, we do affect our genetics in two major ways. First, we have experiences, consume certain foods and manage our moods to turn our genes on and off. Geneticists call it gene expression. Second, our life experiences (and even before we are born our birth Mother’s experiences) actually can change out genetic make-up in profound ways. All of that brings us back to the point that not only does our body renew itself (or die) all of the time at the cellular levels but our spirit and what excites us also changes (life and career cycle changes). We now know that adult humans can create new neurons from stem cells in parts of our brain. It is called neurogenesis. We also know that “annoying” stress episodes can cause hormones to enter our bloodstream that inhibit or stop this neurogenesis. We know that the deluge of annoying stress, not to mention acute stress that bombard us daily cause a deterioration in our cognitive, emotional and perceptual capabilities  – the effect of activating our Sympathetic Nervous System. Meanwhile, our bodies have the amazing capability to reverse that through another part of our autonomic nervous system, the Parasympathetic Nervous System. At the behavioral level, and in terms of the specific competencies that predict effectiveness in a wide range of leadership and professional roles in most countries of the world, we know that adults can dramatically develop these competencies. Whether you are focusing on those we call emotional, social or cognitive intelligence, they can be developed and the changes sustained over years. The published research studies of my colleagues, including Bonesso, Bruni and Gerli and myself have shown that in the past 25  years. Others have been showing this effect during the same period of time. So why do we persist in thinking that we cannot change? First, change is difficult. Most training programs in government and industry, as well as graduate education programs produce little sustained, desired change in these competencies. So we often conclude that people do not change because we have typically been so ineffective at inspiring and engaging durable or sustained changes. Second, people often focus on traits. These

viii 

FOREWORD

are the deeper, relatively stable characteristics we have, like openness or agreeableness, conscientiousness or introversion. But here as well, recent research is showing that people can change on even these deep traits. Our recent neuroimaging and hormonal studies have affirmed an approach to helping people change called coaching with compassion. That is, helping people change toward their dreams, values and calling (i.e., sense of purpose). The studies show, quite clearly, that the more typical approach of giving people feedback and trying to help of fix them does the reverse. It slows change and makes it less sustaining. It is like New Year’s Eve resolutions – the effort lasts no more than 3 weeks, if that long. This brings us back to the hope which you will experience in reading Sara, Elena and Fabrizio’s new book. You CAN change! You CAN become more effective! You CAN move closer to your dreams. This is not some naïve hope. It is based on our experiences and decades of our and colleagues published academic research studies. Richard E. Boyatzis is Distinguished University Professor, Case Western Reserve University, Co-author of the international best seller, Primal Leadership and the new book, Helping People Change. Cleveland, OH, USA

Richard E. Boyatzis

Preface

Why do some people get exceptional results? How  can we improve the performance of individuals and teams? These are some questions that probably every CEO, HR director, and manager, asks himself every day. And they are not the only ones asking these questions. Business schools, universities, teachers, and  trainers are all  asking the same questions. In addition, of course, to all those who work within organizations of every kind and want to improve themselves. To provide an answer to this kind of questions the Ca’ Foscari Competency Centre was founded in 2012 within the Ca’ Foscari University of Venice, Italy. A team of researchers works with the aim to increase the performance of people, through the development of their behavioral competencies. People think that having more technical skills allows them to obtain better results. But for more than thirty years, scientific research has taught us that although technical skills are required to perform a job, they alone do not allow to obtain an excellent performance. On the contrary, behavioral competencies, like emotional, social and cognitive competencies, are the actual determinants of an outstanding performance. Within the Ca’ Foscari Competency Centre, and in collaboration with the best scholars and research centers in the world on these topics, we develop training courses to improve these skills, tools to evaluate them and – above all – we do a lot of research to identify the most relevant ones for specific roles and for carrying out specific processes. This book seeks to answer the above questions, exploring some big data roles, which are emerging jobs extremely requested and critical for the competitiveness of organizations, and contains the results of our research ix

x 

PREFACE

on their behavioral competencies. This book is a journey into the still unexplored world of data scientists and data analysts, into the world of the problems they have to solve every day, of the situations that require their intervention and of the behaviors they adopt to address them. For those who already do these jobs, it is also a guide for evaluating themselves and comparing themselves to others, to better understand one’s own strengths and weaknesses and to learn the ways to acquire some competencies that today are not adequately possessed. For those who are approaching these jobs, it is a way to immerse themselves into the reality and understand what their real contents are. For those who are in charge of training programs, and for educational institutions, it is a tool to guide the design of new courses and curricula, which include the development of behavioral competencies and use consistent tools and methods. For all the others, it aims to be a map to move towards a better awareness of what is needed for a better performance. Venice, Italy Venice, Italy Venice, Italy

Sara Bonesso Elena Bruni Fabrizio Gerli

Acknowledgments

There are so many people who have contributed to our research over the last few years, that it is certainly impossible to name them all. Here we want to express gratitude to some of them, who have been particularly outstanding for their ideas, points of view, discussions, intellectual stimuli, encouragements and criticisms. To Richard Boyatzis we owe the inspiration and the countless opportunities to share our ideas and findings. His suggestions are of immeasurable value and his tireless support have oriented us within the line of research on behavioral competencies and have helped us to give life to the Ca’ Foscari Competency Centre, which currently helps thousands of people to perform better in their job and in their life. We express all our gratitude also to the colleagues of the GLEAD – Leadership Development Research Center of ESADE  – Ramon Llull University, and in particular to Joan Manuel Batista-Foguet and Ricard Serlavós Serra, for their continuous willingness to share methods, tools, experiences and perspectives, and to Robert Emmerling and Ferran Velasco Moreno for their support in the methodology and in the data analysis of this research. Another big thank you goes to all the colleagues at the Department of Management of the Ca’ Foscari University of Venice with whom we had the opportunity to share our research, and most especially to Anna Comacchio and Andrea Pontiggia, who have shown to believe in this research and have been a great source of encouragement and of precious discussions, and to Laura Cortellazzo, for her enthusiastic and valuable presence in all the activities of our research centre. xi

xii 

ACKNOWLEDGMENTS

We are also grateful to the Scientific Committee of the Ca’ Foscari Competency Centre, and particularly to Giovanni Giuriato, Luca Giustiniano, Paolo Legrenzi and Pia Masiero, for having oriented our lines of research and for the richness of the experiences they bring during each and every meeting. Of course, this book would not have been possible without the wonderful people who play the roles of data scientists and data analysts, who generously shared with us some moments of their job and allowed us to understand the deep meaning of their behaviors and choices, to get into their problems and to follow the decision-making processes they adopted. The regret of not being able to thank them one by one, for reasons of confidentiality, is as great as the gratitude towards them. Finally, we wish to express our utmost thankfulness to Maria Raffaella Caprioglio and to Giuseppe Venier, and to all the other marvelous people we have met in UMANA S.p.A, for cooperating in both our research and training activities and being a reliable and outstanding partner. In particular, Giulio De Biasio and Nicolò Capuzzo deserve our acknowledgment for their support in this research. And, of course, we cannot forget our families and friends, who kindly and patiently endured our physical and mental absence while we conducted this research and wrote this book. To them all we offer a hug of love. To give to Caesar what is Caesar’s, errors, inaccuracies and omissions, on the other hand, are to be charged to the authors.

Contents

1 The Organizational Challenges of Big Data  1 2 How Big Data Creates New Job Opportunities: Skill Profiles of Emerging Professional Roles 21 3 Emotional and Social Intelligence Competencies in the Digital Era 41 4 When Hard Skills Are Not Enough: Behavioral Competencies of Data Scientists and Data Analysts 63 5 Managing Big Data Professionals through a Competency-­ Based Approach 89 Index107

xiii

About the authors

Sara Bonesso  is Associate Professor of Business Organization and Human Resources Management at the Ca’ Foscari University of Venice, where she received her Ph.D. in Management. She is one of the founders and the vice-director of the Ca’ Foscari Competency Centre, a research centre aimed to improve individuals’ performance and employability through the development of behavioral competencies (www.unive.it/cfcc). She has been visiting scholar at the Industrial Performance Center  – Massachusetts Institute of Technology (Boston, USA) and at the Fraunhofer Institute for Systems and Innovation Research ISI (Karlsruhe, Germany). She is Associate Editor of Frontiers in Psychology (section Organizational Psychology) and member of the Consortium for Research on Emotional Intelligence in Organizations. Her research interests lie in the fields of Organizational Design, Organizational Behaviour and Human Resources Management. Her recent research investigates the development and the assessment of emotional and social competencies, the competency profile of big data professionals and of entrepreneurs, the impact of behavioral competencies on entrepreneurial intent, innovation, employability and career development. Her research has appeared in various journals, including Journal of Vocational Behavior, Industrial and Corporate Change, Creativity and Innovation Management, the Journal of Small Business Management, Frontiers in Organizational Psychology, Technovation, The Journal of Technology Transfer and the European Management Journal. Her recent works has been presented in several international conferences, such as AOM (Academy of Management), EURAM (European Academy of Management), BAM (British academy of management), IWHRM xv

xvi 

ABOUT THE AUTHORS

(International Workshop on Human Resources Management), ICEI (International Congress of Emotional Intelligence), and ISBE (Institute for Small Business Entrepreneurship). Elena Bruni  is a post-doc research fellow at the Ca’ Foscari University of Venice. She is also an Adjunct Professor in Human Resource Management and Organizational Design for International Companies at Ca’ Foscari University of Venice and Adjunct Professor in Organizational Design at LUISS Business School, Rome (Italy). Her research interests are related to innovation, organizational behavior and organization theory. Her doctoral thesis was focused on the analysis of the mechanisms underlying novelty generation, drawing on cognitive studies. She draws great attention on how business model emerges as a concept, by linking recent contributions from business model innovation literature and cognitive insights. Recently, she has been devoting attention on how Big Data is impacting organizations, leadership, and job profiles. Her recent works has been presented in several international conferences, such as AOM (Academy of Management), EURAM (European Academy of Management), BAM (British academy of management), and ISBE (Institute for Small Business Entrepreneurship). Fabrizio  Gerli is Associate Professor of Business Organization and Human Resources Management at the Ca’ Foscari University of Venice, where he received his Ph.D. in Management. He is also one of the founders and the Director of the Ca’ Foscari Competency Centre (a research centre aimed to improve individuals’ performance and employability through the development of behavioral competencies) and the Scientific Coordinator of post-graduate courses in Human Resources Management at Ca’ Foscari. He has been in charge of several research projects funded by the Italian Ministry of Labor, the Italian Ministry of Research, the European Institute of Public Administration and the European Social Fund. He has been visiting professor at HIBA (Higher Institute for Business Administration) in Damascus. He was also the founder and Scientific Director of the Master Program in Innovation Management and the Scientific Director of the full-time MBA at CUOA Business School, where he was also the Scientific Director of the Competency Development Department. He has also been member of the Board of Directors of Fondazione Università Ca’ Foscari. He is also Associate Editor of Frontiers in Psychology (section Organizational Psychology), member of the Editorial Board of the Engaged Management Review and peer reviewer for some management journals. In 2011 he was appointed as member of the

  ABOUT THE AUTHORS 

xvii

Consortium for Research on Emotional Intelligence in Organizations. His research embraces the fields of Human Resources Management, Organizational Design and Organizational Behaviour, with a specific interest on the development and evaluation of emotional, social and cognitive competencies, and their impact on individual and organizational performance. His research has been presented in several international conferences and workshops, such as AOM (Academy of Management), EURAM (European Academy of Management), BAM (British Academy of Management), EGOS (European Group of Organizational Studies), IWHRM (International Workshop on Human Resources Management), ICEI (International Congress of Emotional Intelligence), EUROMA (European Operations Management Association) and ISBE (Institute for Small Business Entrepreneurship) and has been published in various journals, including: Journal of Vocational Behavior, Industrial and Corporate Change,  Industrial Relations, Frontiers in Organizational Psychology, International Journal of Operations and Production Management, Creativity and Innovation Management, Journal of Small Business Management, European Management Journal, Journal of Management Development, Cross Cultural Management, International Journal of Training and Development.

List of Figures

Fig. 1.1 Fig. 3.1 Fig. 3.2 Fig. 4.1 Fig. 4.2

Google trends for the keyword “data science” (July 2019) 7 The emotional and social intelligence competency framework 45 The competency Hexagon 50 Behavioral competencies framework: the competency hexagon 70 Competencies of data scientists and data analysts according to the frequency of manifestation 71 Fig. 4.3 Competencies of data scientists according to their frequency of manifestation72 Fig. 4.4 Competencies of data analysts according to their frequency of manifestation73

xix

List of Tables

Table 1.1 Table 1.2

Classification of types of big data analytical methods 9 A representation of the Gartner’s maturity model for data and analytics10 Table 2.1 Summary of the main responsibilities of data architect, database architect, database administrator, and data engineer 27 Table 2.2 Technical skills of data scientists identified in scholarly articles 31 Table 2.3 Soft skills of data scientists identified in scholarly articles 32 Table 3.1 Examples of professional roles analyzed in terms of performance outcomes and related behavioral competencies 48 Table 3.2 Competency Hexagon: The thirty-three competencies and related definition 51 Table 5.1 A representation of the five discoveries in Intentional Change Theory94 Table 5.2 Experiential activities for developing self-awareness of the inner identity and future work self, and the necessary behavioral competency profile 96

xxi

CHAPTER 1

The Organizational Challenges of Big Data

Abstract  How can big data be leveraged to create value and what are the main barriers that prevent companies from benefiting from the full potential of data and analytics? This chapter describes the phenomenon of big data and how its use through data science is dramatically changing the basis of competition. The chapter also delves into the main organizational challenges faced by companies in extracting value from data, namely the promotion of a data-driven culture, the design of the internal and external structures, and the acquisition of the technical and behavioral skills required by big data professional roles. The aim and the structure of the book are illustrated. Shedding light on the human side of big data through the lense of emotional intelligence, the book aims to provide an in-depth understanding of the behavioral competencies that big data profiles require in order to achieve a higher performance. Keywords  Big data • Data science • Data analytics • Organizational challenges • Emotional intelligence The point is not to be dazzled by the volume of data, but rather to analyze it – to convert it into insights, innovations, and business value. (Davenport 2014: 2)

© The Author(s) 2020 S. Bonesso et al., Behavioral Competencies of Digital Professionals, https://doi.org/10.1007/978-3-030-33578-6_1

1

2 

S. BONESSO ET AL.

1.1   Competing in a Data-Driven Age Every time we send an email or a message, visit a website, tap an icon on a smartphone, or post and share comments, photos, and video in social media, we generate digital data. But even when we perform actions in the analog world we generate digital data: buying products at the supermarket, driving our connected car, taking the train or the bus, watching an on-demand movie, or simply taking a walk with our geo-localized smartphone in our pocket or using our credit card creates a huge amount of data. Data is considered as the new raw material of the twenty-first century (Berners-Lee and Shadbolt 2011), and its use through analytics is dramatically changing the basis of competition. The volume of available data has grown exponentially in recent years due to the increasing number of individuals, devices, and sensors that are connected by digital networks, along with the development of more sophisticated algorithms, and the improvements of computational power and data storage (McKinsey Global Institute 2016). Big data analysis is generating significant value across sectors, enhancing the competitiveness of companies. The more data-driven a firm is, the more value it generates in terms of knowledge, higher productivity, profit, and market value (BARC 2015; Brynjolfsson et al. 2011). Several classes of value have been associated with the use of big data (Davenport 2014; Lee 2017): • Improvements in decision-making. Companies can use sophisticated analytics and develop algorithms to optimize their decision processes, such as the automatic fine-tuning of inventories and pricing in response to real-time in-store and online sales, as well as to minimize risks. For instance, through the use of optimization techniques it is possible to identify the price of a product that is more likely to generate high profitability or the level of inventory that is more likely to avoid stock-outs (Davenport and Kim 2013). • Increase in process efficiency. The use of sensors and data analytics favors cost savings in operations and improves companies’ reaction time to issues in the supply chain, such as better demand forecasts, optimized distribution network management, transportation, and routing (Sanders 2016). For instance, in the fashion industry, the Prada Group is using Oracle technology to analyze historical data and market demands across its global retail network of 634 stores in

1  THE ORGANIZATIONAL CHALLENGES OF BIG DATA 

3

order to optimize the merchandising process and detect trends as well as for performance analysis, inventory management, and allocation. • Enhancement of customer experience. Granular data, namely detailed data for each single customer, allows organizations to implement specific market segmentations and to tailor products and services to meet specific customers’ needs. For instance, major retailers analyzing preferences and sentiments data can deliver personalized product/service recommendations and promotional offers, whereas financial companies exploiting social media data are able to assess the credit risk and financial needs of potential clients and provide new types of financial products. • Innovation of business models, products, and services. Data on customers’ purchase decisions and social feedback mechanisms can be complemented with digital payments and transaction data to delve deeper into innovation and product adoption. The use of big data can also promote the introduction of new business models in traditional industries, as in the case of Nike, which from a shoes manufacturer became a digital platform owner for data-driven fitness services, or Under Armour, which from solely a sports apparel company partnered with IBM Watson to apply artificial intelligence to create UA Record, an app that provides evidence-based coaching around sleep, fitness, and nutrition. • Improvements in customer service. Data on the same customer is integrated from multiple channels, allowing service personnel to better understand problems and address them quickly. Moreover, big data analytics can be used to monitor transactions in real time and detect fraudulent activities. Besides the economic value mentioned above, big data analysis may also generate social value, enhancing transparency, preventing frauds and crimes, responding to natural disasters. Improving national security, increasing transportation safety, and supporting the well-being of people through better education and health care (Günther et al. 2017). Organizations are still struggling to capture the full potential of big data. As underlined in the Future of Jobs Report released by the World Economic Forum (2018), by 2022 85 percent of the surveyed companies are likely to invest in user and entity big data analytics and 75 percent are likely to increase the use of Internet of Things and app- and web-enabled

4 

S. BONESSO ET AL.

markets. Likewise, machine learning and cloud computing are receiving considerable attention: respectively 73 percent and 72 percent of the surveyed companies indicated their intention to adopt these technologies. In addition to spurring the rate of technological advancement and its related adoption in organizational contexts, big data is profoundly changing the job-relevant skills requested in the labor market. Indeed, while the investment in big data technologies is becoming paramount, at least as important is to attract those professionals with the skills profile relevant to use these technologies effectively (Davenport and Patil 2012). Among the most in-demand digital professional roles to emerge are those of data analysts and scientists, artificial intelligence and machine learning specialists, and big data specialists (World Economic Forum 2018). But what is big data, and how can it be leveraged to create value? The term “big data” was coined in the mid-1990s but became widespread after 2011 (Gandomi and Haider 2015; Mishra et  al. 2017). In providing definitions of big data, academics and practitioners have tried to highlight the properties that characterize information in the digital era. Specifically, big data has been conceptualized as information assets characterized by a combination of volume, variety, and velocity (the so-called Three Vs) that creates an opportunity for organizations to gain competitive advantage in today’s digitized marketplace (Chen et al. 2012; Kwon et al. 2014; De Mauro et al. 2016). The size or the magnitude of data (“Volume”) is the first dimension that comes to mind when defining data as “big.” Currently, exabytes (1 million terabytes) or zettabytes (1000 exabytes) qualify high-volume data, even if bigger units of measure are occasionally developed, since – as the data storage capacities continue to increase  – the property “volume” is relative and varies by time. One of the most important fuels of the increased volume of data in recent years is the phenomenon of the Internet of Things (IoT), namely the pervasive presence of a variety of objects  – phones, sensors, Radio-Frequency Identification (RFID) tags, and actuators, among others – which can communicate and interact with each other, over the Internet, and can be remotely monitored and controlled. Data volume is expected to exponentially grow in the next years due to the increasing number of Internet users and the billions of connected devices and embedded systems that create, collect, and share data every day. The second dimension, velocity, refers to the frequency of generation of data and the speed at which it is analyzed. The diffusion of digital devices, like smartphones and sensors, has increased the rate of data

1  THE ORGANIZATIONAL CHALLENGES OF BIG DATA 

5

c­ reation and the need for real-time analytics. For instance, the high speed of data generated by mobile devices about geospatial location, demographics, and past buying patterns can be used to generate real-time, personalized offers to customers. In the retailing industry, the real-time modeling, manipulation, and visualization of transactional data can explain why sales declined in a particular product or category. The case of Walmart, with over 20,000 stores in 28 countries, represents an example of how companies can react to data quickly. The largest retailer in the world processes 2.5 petabytes of data every hour through its Data Café, a state-ofthe-art analytics hub, combining more than 200 internal and external data sources (transactional, economic, social media, and gas prices, among others) to come up with solutions and make fast decisions (Marr 2017). The last “V,” variety, is related to the fact that data is available in different forms. The traditional structured format, in rows and columns stored in Structured Query Language (SQL), represents a small percentage of all data. Big data is predominantly in semi-structured format, like XML files, or unstructured formats such as text, social media data, audio, and video. Over time, other dimensions have been added to better describe the concept of big data, such as veracity, which was introduced by IBM and refers to the unreliability and uncertainty that characterize some sources of data, due to their incompleteness, inaccuracy, or subjectivity. This is the case of customers’ sentiments collected in the social media that derive from individual judgments (Gandomi and Haider 2015). SAS added two further properties of big data: variability and complexity. The former underlines that the meaning of data is constantly changing, whereas the latter refers to the fact that big data can be collected from different sources, such as the following, according to George et al. (2014): • public data, held by government institutions and local communities, such as that concerning health care, education, transportation, and financial markets; • private data, held by individuals and private firms, which refers for instance to mobile phone usage, consumer transactions, and movement of company goods and resources; • data exhaust, generated as trails or information by-products resulting from all digital or online activities, that refers to storable choices, actions, and preferences, such as log files, cookies, and temporary files;

6 

S. BONESSO ET AL.

• community data, which is unstructured data, usually in text format, that can be distilled into dynamic networks in order to infer social trends. Some examples are consumer rankings and reviews or the timeline of social media sites. • self-quantification data, provided by individuals through personal actions and behaviors, such as that collected through wristbands that monitor fitness activities. The data can be uploaded in mobile device applications, tracked, and aggregated. To unlock the potential of this high volume of fast-moving and diverse data, technologies and analytics methods have made a leap forward in recent years, on the one hand, to capture, store, integrate, transform, and retrieve data (data management) and, on the other hand, to select the right model for analysis and to provide interpretations of the results (data analysis).

1.2   Data Science: How to Extract Value from Big Data The scientific body of knowledge that provides methods, processes, and systems to extract insights from data is defined as data science. It is an interdisciplinary field that combines statistics, computer science, data mining, machine learning, and analytics to understand and explain how we can generate analytical insights and prediction models from structured and unstructured big data. Ten years before the rise of the big data phenomenon, data science was defined by Cleveland as “an action plan to enlarge the technical areas of statistics” (Cleveland 2001: 21). The advent of big data and its related challenges in data management and analysis have progressively expanded the domain of data science beyond the statistics field, assuming an increasing relevance. Figure 1.1 shows a Google Trends chart that displays web searches for the term “data science,” highlighting the dramatic increase in interest in data science in correspondence with the interest in big data. As the size of data (volume) is increasing at an exponential rate, scalability represents a key aim for models and new technologies that allow the storing and processing of a growing amount of data. Since data is generated at a high rate (velocity), its value decays over time. Thus, analytics methods for streaming data are continually improved to provide timely

1  THE ORGANIZATIONAL CHALLENGES OF BIG DATA 

7

Fig. 1.1  Google trends for the keyword “data science” (July 2019)

answers to decision makers. Moreover, since big data can derive from heterogeneous sources (variety), current databases are severely susceptible to inconsistent, incomplete, and noisy data. Improvements in big data science techniques aim at guaranteeing a higher quality of the data in terms of accuracy, completeness, and consistency. Under the umbrella of data science can be included all the methods and techniques which refer to the two main processes of extracting knowledge from big data, namely data management and data analysis (Sivarajah et al. 2016). Data management encompasses the following three stages: • Data acquisition and warehousing. According to the 2017 Big Data Analytics Market Study released by Dresner Advisory Services, data warehouse optimization is considered critical or very important by 70 percent of all respondents. Big data has changed the way to capture and store multiple data formats into a single format and consolidate them in one place, promoting the development of new data storage devices and architectures. Due to their limited scalability, traditional relational database systems have been replaced by unified storage and processing environments for big data across multiple servers like Hadoop/MapReduce/Spark, which have become

8 

S. BONESSO ET AL.

s­ tandard tools for big data processing (Li et al. 2017). Contrary to a data warehouse, which retains data from operational systems and is meant to answer a pre-defined set of questions, a recent and different storage repository is represented by data lakes, a new technology which stores various data in its native, raw forms in a centralized location from different sources, regardless of its use in the immediate future, assuming that analysis will happen later, on demand (Jain 2017). • Data cleansing. Data quality represents a main issue in dealing with big data. This stage encompasses all the procedures for correcting or removing inaccurate and corrupt data. • Data integration and aggregation. Through these procedures data from separate sources is combined into meaningful and valuable information, which are then gathered and expressed in a summary form for subsequent analysis. The second subprocess, data analysis, implies the use of techniques which involve a number of different disciplines, including statistics, data mining, machine learning, neural networks, social network analysis, signal processing, pattern recognition, optimization methods, and visualization approaches (Chen and Zhang 2014). It comprises two stages; on the one hand, analysis and modeling that refers to the methods and techniques to generate and experiment with algorithms for extracting insights from data in order to get answers to specific problems, and on the other hand, data interpretation, which encompasses the techniques for visualizing and presenting data in an understandable form for decision makers. Data analytical methods, which have seen a rapid development in recent years, can be grouped into three main categories: descriptive, predictive, and prescriptive analysis (Chen and Zhang 2014; Davenport and Kim 2013). A definition of these methods and of their related techniques are provided in Table 1.1. Findings from the BARC’s BI Trend Monitor 2019 show that predictive and prescriptive analytics emerge among the most important trends. Moreover, the variety of data sources has fostered the diffusion of dedicated software and algorithms for analyzing specific types of data such as texts, videos, and social media analytics (Granville 2014). Another scientific domain, artificial intelligence and one of its subsets, machine learning, is receiving increasing attention for its application in big data analysis (McKinsey Global Institute 2016). The concept underpinning machine learning is to

1  THE ORGANIZATIONAL CHALLENGES OF BIG DATA 

9

Table 1.1  Classification of types of big data analytical methods Data analytics methods

Aim of the analysis

Techniques

Descriptive analytics

Provide insight into the past in a way that developments, patterns, and exceptions become evident, in the form of producing standard reports, ad hoc reports, and alerts. They answer to the question: What has happened? Make predictions and forecasts about future events using historical and current data. They answer to the question: What could happen?

Descriptive statistics (such as mean, median, mode, standard deviation, variance, and frequency measurement of specific events) and data-mining techniques.

Predictive analytics

Prescriptive analytics

Statistical methods and datamining techniques to identify uncovered patterns and capture relationships in data. They are categorized into two groups: Regression techniques and machine learning techniques. Quantify the effect of future decisions Business rules, algorithms, in order to advise on possible outcomes experimental design, before the decisions are actually made. optimization, machine learning, They provide recommendations and computational modeling regarding actions that will take procedures. advantage of the predictions. They answer to the question: What should we do?

give the algorithm a massive number of “experiences” (training data) and a generalized strategy for learning, then let it identify patterns, associations, and insights from the data. Through machine learning it is possible to create algorithms that “learn” from data without being explicitly programmed. A cutting-edge area of research within the machine learning domain is deep learning, which uses neural networks with many layers (hence the label “deep”) to push the boundaries of machine capabilities. The two most popular programming tools for data analysis are the open source software Python and R, which also provide deep learning and other machine learning libraries. Another important stage of data analysis is visualization. As pointed out by a survey conducted by McKinsey Global Institute (2016), visualization skills show an increasing demand. Recent advancements in visualization techniques and new software help make the results of complex data analyses understandable for decision makers and also for those who are new to analytics, enabling them to turn data into valuable insights.

10 

S. BONESSO ET AL.

1.3   Organizational Challenges for Leveraging Big Data A global survey administrated to 196 organizations by Gartner, Inc. in 2018 asked respondents to rate their organizations according to Gartner’s five levels of maturity for data and analytics (for a summary of the levels included in the model, see Table 1.2). It found that 60 percent of respondents rated themselves in the lowest three levels, despite companies’ having maintained that data management and analytics have represented a priority in recent years. The survey also showed that, despite the increasing attention devoted to advanced forms of analytics, 64 percent of firms still consider enterprise reporting and dashboards their most business-­ Table 1.2  A representation of the Gartner’s maturity model for data and analytics Level 1 Basic

Level 2 Opportunities

Level 3 Systematic

Level 4 Differentiating

Level 5 Transformational

Companies operate in silos. Data is used to generate single metric within a particular functional unit. The focus is on after-the-fact performance. Excel spreadsheets dominate, providing limited analytics.

Companies still operate in silos, with little collaboration or knowledge sharing. The data is used to measure performance and provide support for decision making through excel spreadsheets, reports, and dashboards.

Companies promote data harmonization and governance so the analytics can leverage end-to-end process data. Executives champion data and analytics. Data is used to establish visibility and performance measurement across processes. At this level companies also start to integrate external data sources, and different types of data are treated in different ways.

Companies promote the internal sharing of best practices and appoint a chief data officer. Analytics are faster and more dynamic, and data is used for ROI.

Data is crucial for business strategy. Chief data officer is member of the board of directors. Data comes from public and unstructured sources and the internet of things. Data is used to influence investments.

1  THE ORGANIZATIONAL CHALLENGES OF BIG DATA 

11

critical applications for data and analytics. In the same vein, traditional data sources such as transactional data and logs also continue to dominate, although 46 percent of organizations now report using external data (Gartner 2018). What are the main barriers that prevent companies from benefitting from the full potential of data and analytics? According to recent surveys conducted worldwide across a variety of industries, the biggest barriers companies face in extracting value from data and analytics are organizational (Gartner 2018; McKinsey Global Institute 2016). Specifically, the NewVantage Partners Executive Survey (2019), administrated to 65 Fortune 1000 leading firms, reveals that among the barriers mentioned by the companies, 95 percent stem from organizational challenges and only 5 percent are related to technology. Indeed, companies have responded to competitive pressure by making investments in technology, without implementing the necessary organizational changes. The first challenge is incorporating data and analytics into a core strategic vision. Companies need to diffuse a supportive organizational culture or mind-set to invest in data-driven initiatives (Pigni et  al. 2016). The implementation of the digital infrastructure should be tailored in line with the specific uses of the data according to the company’s strategic objectives. For this reason, companies need to appraise the value of big data in providing a competitive advantage in their business, to understand the type of problems that can be addressed by data analytics, to equip themselves with the adequate infrastructures and tools, and to change the consolidated decision-making habits, adopting a more test-and-learn culture to measure the business impact of data analytics. Companies that embrace a data-driven culture produce more innovative products and services, are more competitive, and increase productivity by 5–10 percent more than companies that do not (Davies 2016). However, among the 65 Fortune 1000 leading firms surveyed in the NewVantage Partners Executive Survey (2019), only 31 percent have created a data-driven organization and only 28.3 percent have promoted a data culture. The diffusion of a data-driven mind-set requires a high commitment from the top and middle management, who are in charge of leading the digital transformation across the different business units. This is also in line with the progressive shift from a traditional model of organizing companies to an agile one. Indeed, big data analytics is conceived as fundamental for firms aiming to adopt agile principles (Rialti et al. 2018), since transparency of information, c­ ontinuous

12 

S. BONESSO ET AL.

learning, and quick, efficient, and continuous decision making are indicated as among the main traits that distinguish agile organizations (McKinsey and Company 2017). The second related challenge refers to the design of the internal organizational structure and the external network to support data and analytics activities. A company can decide to adopt a decentralized or centralized organizational design for data management and analytics activities. In the first case, a central department for data science, thus a dedicated center of excellence, promotes economies of scale and specialization. A decentralized structure means that the single business departments are made accountable for the generation of value from data, with consequent needs to promote coordination through cross-functional practices. Interestingly, a recent McKinsey Analytics study (2018) revealed that leading companies, in comparison to laggard ones in the big data field, have structured their analytics unit by adopting a hybrid model led by a center of excellence. Moreover, companies through the design of external networks can establish partnerships with platform providers to get access to advanced tools or solutions, or with data providers to gain access to specific data sets. The design of the organizational structure of the data analytics activities is closely related to the definition of the set of competencies that the data science/analytics teams should possess to effectively conduct data acquisition and preparation, model building, and data presentation. Another challenge companies are facing is attracting and retaining appropriate big data professional roles. A 2018 search on LinkedIn Jobs, using the keyword “analytics,” resulted in 218,866 entries (Bowers et al. 2018), highlighting the shortage of qualified and competent big data specialists, such as data scientists, business analysts, data engineers, and data architects, among others (McKinsey Global Institute 2016). To respond to this market demand, in recent years there has been an exponential growth in the number of master of science graduates in analytics and data science. Educational programs in data science place more emphasis on computing, data management, and data mining, thus they are more data-centric, whereas programs in analytics are more problem-centric, including the entire spectrum of analytics (descriptive, predictive, and prescriptive) and engaging most often in client-based capstone experiences (Bowers et al. 2018). Despite the increasing academic offerings of dedicated educational programs, the shortage of analytical and data science skills continues to represent a critical constraint (LinkedIn Workforce Report 2018; McKinsey Global Institute 2016). The skills gap of big data professional roles not

1  THE ORGANIZATIONAL CHALLENGES OF BIG DATA 

13

only refers to technical/analytics competencies but also encompasses soft or behavioral skills that are considered a requisite for companies. Among the most in-demand soft skills for big data roles are business acumen, communication, interpersonal skills, curiosity, and interdisciplinary orientation (Costa and Santos 2017; Davenport and Patil 2012). Even if higher education institutions seem to have responded promptly to employers’ demand on the side of technical skills, educational programs in data science and business analytics still view personal and interpersonal competencies as a second priority (Bowers et al. 2018). Therefore, the development of such skills for big data professional roles remains a critical issue, as behavioral competencies are the major source of mismatch between companies’ requests and candidates’ skill profiles.

1.4   The Emotional Intelligence Side of Big Data In an article published in the Harvard Business Review in 2017, Beck and Libert, both practitioners in the machine learning field, maintained that the rise of artificial intelligence makes emotional intelligence more important. They argued that “skills like persuasion, social understanding, and empathy are going to become differentiators as artificial intelligence and machine learning take over our other tasks.” These skills, which are usually defined as “soft” in comparison to the technical/hard skills, refer to the broader concept of emotional intelligence and its related behavioral competencies, namely the ability to recognize, understand, and manage one’s own emotions (emotional competencies); the ability to understand other people’s concerns, feelings, and emotional states; the capacity to build and maintain positive relationships and to behave appropriately with others (social competencies); and the ability to analyze information and situations (cognitive competencies) (Boyatzis 2009; Goleman 1998). From research conducted by MIT and Deloitte involving more than 3700 business executives, managers, and analysts from 131 countries, it emerged that technical skills alone cannot guarantee the achievement of individual performance in the digital age. Instead, the study revealed that most successful companies are those that in their digital transformation efforts put their focus on soft skills, such as the ability to develop a vision, and to have a change-oriented mind-set and other leadership and ­collaborative skills (Kane et al. 2016). Behavioral competencies also have been recently found as among the most important abilities for big data professional roles, with specific regard to business analysts and data s­cientists

14 

S. BONESSO ET AL.

who are primarily involved in data modeling, analysis, and presentation (Costa and Santos 2017; Kim and Lee 2016; Vidgen et al. 2017). Why do behavioral competencies make a difference for big data roles? The ultimate aim of these roles is to help the business make better decisions, and this implies continually putting themselves in the shoes of different stakeholders (such as senior executives, coworkers, and clients) to enhance the understanding of their needs and deliver valuable insights through persuasive and effective communication. But among others, they also require organizational awareness for identifying the appropriate sources of data, teamwork for promoting collaboration within the data science team and coordination across different business units, and questioning and experimentation skills for identifying alternative methods, analytics techniques, and algorithms. Even if in recent years many efforts have been made to define the skills set of big data professions on the technical side, by identifying the knowledge and capabilities that distinguish the most in-demand professional profiles, there is still not a clear understanding of which emotional and social competencies companies should search and develop for promoting high-performing data science teams.

1.5   The Aim of This Book and Its Structure This book represents the first comprehensive and up-to-date description of the big data professional roles that are emerging as the most in-demand jobs in the labor market and are becoming central in every organization, from for-profit to not-for-profit companies. The book offers an in-depth investigation of the behavioral competencies that these professionals require to achieve higher performance in companies that aim to turn insights from data into competitive advantages. Shedding new light on the human side of big data through the lenses of emotional and social intelligence competencies, this book aims to i) advance the understanding of the requirements of the different professions that deal with big data; ii) provide a comprehensive review of the competency profile required by these professional roles, with a specific focus on the behavioral competencies needed to achieve superior performance at work; and iii) provide an application of the competency modeling process in the area of big data ­professions and offer new empirical insights specifically on the job profiles and related behavioral competencies of data scientists and business analysts. The structure of the book is outlined as follows.

1  THE ORGANIZATIONAL CHALLENGES OF BIG DATA 

15

Chapter 2 – How Big Data Creates New Job Opportunities: Skill Profiles of Emerging Professional Roles. Starting from the shortage of analytical and data science skills necessary to make the most of big data, the chapter offers a comprehensive review and classification of the job profiles of different big data roles. Recent surveys have identified more than 100 job titles adopted in the labor market only referring to data science and analytics professionals. Studies also agree that there is an improper use of the label of “data scientist” to define a plethora of different professions. Main differences among several profiles (chief data officer, data architect, database architect, database administrator, data engineer, data scientist, data analyst, and business analyst) have been defined according to the volume of data analyzed, the tools they use, and the educational background possessed. The chapter outlines that technical skills alone seem to be not enough to succeed in the big data and data science field, whereas possessing behavioral competencies or soft skills has become a mandatory requirement for big data profiles. Chapter 3  – Emotional and Social Intelligence Competencies in the Digital Era. The chapter is meant to provide a clear understanding of why it is important to analyze behavioral competencies of big data professional roles since they are called to understand data, interpret the data, and transmit its meaning to the upper levels of the organizations. The chapter explains the evolution of the emotional and social intelligence competency framework and provides insights on the large body of research that has investigated the impact of behavioral competencies on individual performance. It also introduces the competency framework that will be adopted the empirical analysis described in Chap. 4 providing the classification and the definition of thirty-three behavioral competencies. Finally, it opens the discussion on the role of behavioral competencies in the big data era, identifying the state of the art and the major gaps that need to be addressed. Chapter 4 – When Hard Skills Are Not Enough: Behavioral Competencies of Data Scientists and Data Analysts. This chapter concentrates attention on an analysis of the two most in-demand big data professional roles, data scientists and data analysts. These two profiles have a direct impact on business functions and decision-making processes, and therefore are at the core of organizational changes. The behavioral competencies of both professionals are investigated in order to emphasize the peculiarities of each profession. The chapter illustrates the empirical evidence collected through an in-depth qualitative exploratory study on a sample

16 

S. BONESSO ET AL.

of data scientists and data analysts operating in the Italian context. The study adopts the competency-­based methodology, and specifically, data has been collected through behavioral event interview, a consolidated technique that does not rely on perceptions of the main important competencies for the professional roles under investigation but allows for the detection of the behaviors that are actually enacted in the work environment. The chapter provides an in-depth description of the tasks and responsibilities of the two roles under investigation, as well as of the behavioral competencies manifested in critical events/incidents in which individuals felt effective in performing their job in the organizational context. In particular, the behavioral competency portfolio of data scientists and data analysts is described through a competency framework which encompasses thirty-­three behavioral competencies grouped into six areas: awareness, action, social, cognitive, exploratory, and organizational action competencies. The fifth and last chapter – Managing Big Data Professionals through a Competency-Based Approach – contributes to the current debate on how to overcome the skill shortage that characterizes the demand for big data professional roles. First, it offers managerial insights in describing how organizations and specifically HR practitioners can benefit from the competency-­based approach to increase the effectiveness of the selection and recruiting processes of candidates, achieving a better match between job offers and demand. Second, it provides recommendations for the higher education system to offer better designed curricula for entry-level big data professions. There is increasing attention within different institutions on developing and sponsoring educational programs on data science and business analytics. However, there is still a need to design such programs carefully to provide adequate preparation, both in terms of technical and behavioral competencies.

References BARC. (2015). Big data use cases. Getting real on data monetization. Retrieved from https://www.sas.com/content/dam/SAS/bp_de/doc/studie/ba-st-­ barc-bigdata-use-cases-de-2359583.pdf BARC. (2019). BI Trend Monitor 2019. The world’s largest survey of BI trends. Retrieved from https://www.sitsi.com › download  Beck, M., & Libert, B. (2017, February 15). The rise of AI makes emotional intelligence more important. Harvard Business Review.

1  THE ORGANIZATIONAL CHALLENGES OF BIG DATA 

17

Berners-Lee, T., & Shadbolt, N. (2011, December 31). There’s gold to be mined from all our data. The Times (pp. 1–2). Bowers, M. R., Camm, J. D., & Chakraborty, G. (2018). The evolution of analytics and implications for industry and academic programs. INFORMS Journal on Applied Analytics, 48(6), 487–499. Boyatzis, R. E. (2009). Competencies as a behavioral approach to emotional intelligence. Journal of Management Development, 28(9), 749–770. Brynjolfsson E., Hitt L. M., & Kim H. H. (2011). Strength in numbers: How does data-driven decision making affect firm performance? Working paper, 2011. SSRN working paper. Available at SSRN: http://ssrn.com/abstract=1819486 Chen, C. L. P., & Zhang, C. Y. (2014). Data-intensive applications, challenges, techniques and technologies: A survey on big data. Information Sciences, 275, 314–347. Chen, H., Chiang, R. H. L., & Storey, V. C. (2012). Business intelligence and analytics: From big data to big impact. MIS Quarterly, 36(4), 1165–1188. Cleveland, W. S. (2001). Data science: An action plan for expanding the technical areas of the field of statistics. International Statistical Review, 69(1), 21–26. Costa, C., & Santos, M. Y. (2017). The data scientist profile and its representativeness in the European e-competency framework and the skills framework for the information age. International Journal of Information Management, 37(6), 726–734. Davenport, T. H. (2014). Big data at work: Dispelling the myths, uncovering the opportunities. Boston: Harvard Business Review Press. Davenport, T. H., & Kim, J. (2013). Keeping up with the quants. Your guide to understanding and using analytics. Boston: Harvard Business Review Press. Davenport, T. H., & Patil, D. J. (2012). Data scientist: The sexiest job of the 21st century. Harvard Business Review, 90(October), 70–76. Davies, R. (2016). Big data and data analytics. The potential for innovation and growth. European Parliamentary Research Service. De Mauro, A., Greco, M., & Grimaldi, M. (2016). A formal definition of big data based on its essential features. Library Review, 65(3), 122–135. Dresner Advisory Services Publishers. (2017, December 20). Big data Analytics market study. Retrieved from https://www.microstrategy.com/getmedia/ cd052225-be60-49fd-ab1c-4984ebc3cde9/Dresner-Report-­B ig_Data_ Analytic_Market_Study-­WisdomofCrowdsSeries-2017.pdf Gandomi, A., & Haider, T. M. (2015). Beyond the hype: Big data concepts, methods, and analytics. International Journal of Information Management, 35, 137–144. Gartner. (2018, February 5). Survey shows organizations are slow to advance in data and analytics. Retrieved from https://www.gartner.com/en/newsroom/ press-releases/2018-02-05-gartner-survey-shows-organizations-are-slow-toadvance-­in-data-and-analytics

18 

S. BONESSO ET AL.

George, G., Haas, M.  R., & Pentland, A. (2014). Big data and management. Academy of Management Journal, 57(2), 321–325. Goleman, D. (1998). Working with emotional intelligence. New York: Bantam Dell Pub Group. Google Trends. (2019). Searches for the term Data Science. Retrieved 19 July 2019 from https://www.google.com/trends/explore#q=data%20science Granville, V. (2014). Developing analytics talent. Becoming a data scientist. Indianapolis: Wiley. Günther, W.  A., Mehrizi, M.  H. R., Huysman, M., & Feldberg, F. (2017). Debating big data: A literature review on realizing value from big data. Journal of Strategic Innovation Information Systems, 26(3), 191–209. Jain, V. K. (2017). Big Data and Hadoop. New Delhi: Khanna Publishing. Kane, G. C., Palmer, D., Phillips, A. N., Kiron, D., & Buckley, N. (2016). Aligning the organization for its digital future. Cambridge, Massachusetts, United States: MIT Sloan management review and Deloitte University Press. Kim, J. Y., & Lee, C. K. (2016). An empirical analysis of requirements for data scientists using online job postings. International Journal of Software Engineering and Its Applications, 10(4), 61–172. Kwon, O., Lee, N., & Shin, B. (2014). Data quality management, data usage experience and acquisition intention of big data analytics. International Journal of Information Management, 34(3), 387–394. Lee, I. (2017). Big data: Dimensions, evolution, impacts, and challenges. Business Horizons, 60(3), 293–303. Li, K. C., Hai, J., & Zomaya, A. (2017). Big data management and processing. New York: Imprint Chapman and Hall/CRC. LinkedIn. (2018). LinkedIn Workforce Report. Retrieved from https://economicgraph.linkedin.com/resources/linkedin-workforce-report-december-2018 Marr, B. (2017, January 23). Really big data at Walmart: Real-time insights from their 40+ petabyte data cloud. Forbes. McKinsey Analytics. (2018, January). Analytics comes of age. Retrieved from https://www.mckinsey.com/~/media/McKinsey/Business%20Functions/ McKinsey%20Analytics/Our%20Insights/Analytics%20comes%20of%20age/ Analytics-comes-of-age.ashx McKinsey and Company. (2017, December). The 5 trademarks of agile organizations. Retrieved from https://www.mckinsey.com/business-functions/organization/our-insights/the-five-trademarks-of-agile-organizations McKinsey Global Institute. (2016, December). The age of analytics: Competing in a data-driven world. Retrieved from https://www.mckinsey.com/business-­ functions/mckinsey-analytics/our-insights/the-age-of-analytics-competingin-a-data-driven-world

1  THE ORGANIZATIONAL CHALLENGES OF BIG DATA 

19

Mishra, D., Luo, Z., Jiang, S., Papadopoulos, T., & Dubey, R. (2017). A bibliographic study on big data: Concepts, trends and challenges. Business Process Management Journal, 23(3), 555–573. NewVantage Partners. (2019). Big data and AI executive survey 2019. Executive Summary of Findings. Retrieved from https://newvantage.com/wp-content/ uploads/2018/12/Big-Data-Executive-Sur vey-2019-FindingsUpdated-010219-1.pdf Pigni, F., Piccoli, G., & Watson, R. (2016). Digital data streams: Creating value from the real-time flow of big data. California Management Review, 58(3), 5–25. Rialti, R., Marzi, G., Ciappei, C., & Caputo, A. (2018). Reframing agile organization. Do big data analytics capabilities matter? Academy of Management Global Proceedings. Sanders, N. R. (2016). How to use big data to drive your supply chain. California Management Review, 58(3), 26–48. Sivarajah, U., Kamal, M. M., Irani, Z., & Weerakkody, V. (2016). Critical analysis of big data challenges and analytical methods. Journal of Business Research, 70, 263–286. Vidgen, R., Shaw, S., & Grant, D. B. (2017). Management challenges in creating value from business analytics. European Journal of Operational Research, 261(2), 626–639. World Economic Forum. (2018). The Future of Jobs Report 2018. Retrieved from http://www3.weforum.org/docs/WEF_Future_of_Jobs_2018.pdf

CHAPTER 2

How Big Data Creates New Job Opportunities: Skill Profiles of Emerging Professional Roles

Abstract  Big data jobs will increase in importance over the next years. However, at the international level, the labor market for these professionals is characterized by a critical skill shortage. What are the big data specialist profiles that are most sought in the market? What are their main differences in terms of tasks and skill requirements? This chapter provides a snapshot of the most in-demand big data jobs, contributing to clarify their boundaries. It also delves into the main characteristics of the specific professional profiles that have received increasing attention in recent years, namely data scientists and data/business analysts. The review of the contributions provided by experts and scholars operating in the data science and analytics domain clarifies the main differences between these roles on the technical side. However, despite the increasing importance of soft skills, the behavioral competency profile of big data jobs is still ill defined. Keywords  Big data professional profiles • Skill shortage • Soft skills • Technical skillset When people decide to pursue a career as an aircraft pilot, they are embarking on a professional career in which there is a checklist for literally every conceivable problem scenario. There are many professions like that in today’s world. That’s not the world of a data professional. Data professionals are much more like the creative classes of writers,

© The Author(s) 2020 S. Bonesso et al., Behavioral Competencies of Digital Professionals, https://doi.org/10.1007/978-3-030-33578-6_2

21

22 

S. BONESSO ET AL.

artists, composers, and architects. There are many shades of gray in this profession. (Mahadevan 2018: X)

2.1   The Market for Big Data Jobs: Trends and Skills Shortage At the end of 2018, Gartner’s analysts identified the ten major technology trends that will affect all industries in 2019 (Gartner 2019) and that can be ascribed to the following three main areas: • intelligent (advancement in artificial intelligence): autonomous things, augmented analytics, artificial intelligence-driven development; • digital (virtual and real worlds are blended to create an immersive, digitally enhanced and connected environment): digital twins, empowered edge, immersive experience, digital ethics and privacy, quantum computing; • mesh (connections between an expanding set of people, business, devices, contents, and services to deliver digital outcomes): blockchain and smart spaces. The diffusion of these technological breakthroughs is shifting the boundaries between the work tasks performed by people and those performed by machines and algorithms, with consequent changes in the labor market structure. Some new jobs are emerging, while others are expected to become redundant. However, a study conducted by the World Economic Forum (2018) revealed that the increased demand for new roles will offset the decreasing demand for others. Specifically, since 85 percent of the global companies surveyed indicated their intention to increase their use of big data analytics, big data jobs are gaining even more importance over the next years. Recent studies and projections have been carried out in the US labor market. Research conducted by Burning Glass Technologies in collaboration with IBM and the Business-Higher Education Forum (Burning Glass Technologies 2017) indicated that in 2015 the number of data science and analytics jobs were over 2,350,000, and by 2020 this number was expected to rise by 15 percent, with nearly 364,000 new job postings. A

2  HOW BIG DATA CREATES NEW JOB OPPORTUNITIES: SKILL PROFILES… 

23

similar trend characterizes the European Union: in 2018 the number of big data professionals in the EU28 reached 7.2 million, corresponding to 3.4 percent of the total workforce, with an increase of 8.4 percent over the previous year (The Lisbon Council and International Data Corporation 2019). These professional profiles show a significant shortage (LinkedIn 2018) and are among the most difficult to recruit: in the US on average, these jobs remain open for 45 days, five days longer than the market average (Burning Glass Technologies 2017). This skill gap in Europe grew in 2018 by 18 percent, reaching approximately 571,000 unfilled positions in the EU28 (The Lisbon Council and International Data Corporation 2019). Moreover, a study by APEC showed that India, South Korea, Japan, China, and Australia are the key countries that are investing in big data in the Asia Pacific, but at the same time underlined the skills shortage for data science and analytics jobs in this geographic area (APEC 2017). The demand varies significantly across industries. The sectors that present the highest demand for big data jobs are finance and insurance, professional services, IT, wholesale and retail, and manufacturing. The industries with the lowest presence of big data professionals are construction, transport, and health care (Burning Glass Technologies 2017; Datalandscape 2017; PwC 2017). But what are the big data specialist profiles most sought in the market? What are their main differences in terms of tasks and skills required? This chapter provides a snapshot of the most in-demand big data jobs, contributing to clarify their boundaries. It also delves into the main characteristics of the specific professional profiles that have received increasing attention in recent years, namely data scientists and data/business analysts, due to their high contribution in extracting value from data and consequently due to the effectiveness of the decision-making process.

2.2   Professional Roles in the Data Science Field In the last decade, both the academic and professional communities have provided guidance to distinguish emerging data science and analytics jobs. Even though in this fast-evolving field these professional profiles change constantly with new roles and technical skills requirements, proving a description of their evolution and of their main tasks helps to: (i) improve the understanding of the differences among these jobs that should be part of the data science team of the company, and consequently to better orient

24 

S. BONESSO ET AL.

recruiters’ search in the labor market; and (ii) promote their skills and career development following their evolution over time. Considering different contributions from experts and scholars operating in the data science and analytics domain (such as Burning Glass Technologies 2017; Costa and Santos 2017; Davenport 2014; Marr 2018; McKinsey Global Institute 2016; NewVantage Partners 2019), we have identified the most critical jobs based on their frequency of citation in the studies mentioned above and on their impact in supporting companies to deal with the complexity of big data and to extract value from it. One of the new executive roles created by the digital transformation is that of chief data officer (CDO) – also defined as data analytics officer – who has the responsibility to help the business understand the value of big data; defining and implementing a big data strategy at the company level; and ensuring that the data is correct, secure, and governed properly, especially with regard to privacy and ethics issues. Gartner (2018) defined CDOs as “accountable and impactful change agents,” since they are asked to lead their organizations toward data-driven transformation initiatives. As the company progressively becomes aware of the opportunity offered by big data for its business, the appointment of the CDO as a member of the executive board, reporting directly to the chief executive office, successfully contributes to the diffusion of a data-driven culture (see Sect. 1.4). In the last Big Data and AI Executive Survey (2019), it emerged that 67.9 percent of the companies involved have appointed a CDO, up from just 12.0 percent in 2012, and that 48.1 percent have ascribed to this role primary accountability for data, even though it is still ill defined. In this regard, as shown in the last Gartner Chief Data Officer survey (Gartner 2018), the CDOs’ responsibilities include data management, analytics, data science, ethics, and digital transformation. In summary, their priorities are to: (i) identify, communicate, and pursue business opportunities using available data; (ii) promote a data-driven culture, and a common language and practices about data and analytics across the organization, especially where data is in silos; (iii) increase the transparency about the types of data collected and their use where data and information is in silos; (iv) deal with the security and ethical implications of big data; and (v) guarantee data quality standards. Experts in the big data labor market maintain that CDOs are required to combine a technical background, in terms of expertise or familiarity with the major big data technologies and solutions as well as modeling techniques, with strong behavioral competencies necessary to lead the team and communicate effectively.

2  HOW BIG DATA CREATES NEW JOB OPPORTUNITIES: SKILL PROFILES… 

25

The other highly sought big data profiles can be classified according to the two main processes described in Chap. 1, namely data management and data analysis. The remainder of this section will illustrate the roles dealing with data management that have responsibility for capturing, storing, integrating, transforming, and retrieving data, whereas the following sections will provide a detailed description of the profiles more deeply involved in data analysis, namely data scientist and data analyst/business analyst. Four key main roles are crucial for the development and maintenance of infrastructure for a company’s data ecosystem: data architect, database architect, database developer, and data engineer. The data architect – also conceived as the contemporary data modeler – creates blueprints for data management systems. After assessing the company’s potential data sources, both internal and external, architects design a plan to integrate, centralize, protect, and maintain these sources. They are requested to have expertise with requirement analysis, platform selection, technical architecture design, application design and development, testing, and finally deployment. Among the most required skills are solutions architecture, relational database management systems or foundational database skills, cloud computing, software development, SQL, NoSQL, software development life cycle, data governance, data visualization, data mining, data analysis, and data migration experience. Concerning behavioral competencies, this role should demonstrate primarily analytical problem solving, communication, and leadership in directing and advising the team of data modelers, data engineers, database administrators, and junior architects. The role of data architect cannot be confused with another professional role  – database architect  – who is in charge of the design of the database architecture, meaning that he/she develops modeling strategies to ensure that the database is secure, scalable, and capable of reliable performance. Among the main tasks of this role are developing database solutions to store and retrieve company information; installing and configuring information systems to ensure functionality; and analyzing structural requirements for new software and applications. After the database architecture is designed, a database architect works with other information technology professionals such as programmers, system administrators, analysts, software engineers, and database administrators to implement the database. If the database architect is more responsible for the design of the database’s architecture to meet an employer’s needs, the database developer,

26 

S. BONESSO ET AL.

also known as database administrator, is more responsible for day-to-day operations and necessary infrastructure (Bayern 2019), namely creating and implementing computer databases, developing new applications for databases or modifying legacy applications to work with a database setup, and maintaining and securing the system. Often in many organizations the two roles of database architect and administrator are actually performed by the same person. The main tasks of the database administrator can be summarized as follows: (i) modify and update existing databases, expanding their capacity; (ii) design and develop new databases meeting the company’s needs; (iii) troubleshoot database issues, running performance-testing procedures to ensure the proper operations of a database; (iv) develop database documentation, ensuring that operational manuals and supporting documentation includes information on changes added to the database; and (v) assign accessibility for users and monitor usage. The required technical skill set encompasses: • applying technical design and development skills to the creation of database programs; • analyzing existing databases and data needs of the company to develop effective systems; • using knowledge of specific programming languages and codes to perform specific tasks; • following implementation processes for new databases; and • troubleshooting and providing solutions for any bugs in new database applications. Structured Query Language is the primary language that database developers use. In addition, they can be asked to adopt language programming skills in C, C++, C#, or Java. Besides these technical skills, online job posts highlight the importance of the soft skills “critical thinking,” to translate business needs into database solutions, and social competencies, since a database developer works with different organizational roles to ensure that all databases are functioning as intended and to develop technical projects. The primary activity of data engineers is data conversion and treatment in order to keep data available and ready for data scientists and analysts, so they can spend more of their time running actual analysis rather than implementing infrastructure. Data engineers’ tasks include: i) gathering and processing raw data at scale (including writing scripts, web scraping,

2  HOW BIG DATA CREATES NEW JOB OPPORTUNITIES: SKILL PROFILES… 

27

calling APIs, writing SQL queries, etc.); and ii) processing unstructured data into usable data formats, performing data modeling and design. As indicated by Granville (2014), their activities can be summarized into three actions: extracting, loading, and transforming data. Companies look for data engineers who have extensive experience in building and optimizing data pipelines, therefore in manipulating data with SQL, T-SQL, R, Python, Spark Hadoop, Hive, Oozie, Apache Flume, and Pig (Burning Glass Technologies 2017; Burns 2016). Table 2.1 reports the key responsibility of the discussed professional roles, highlighting some areas of overlapping especially between data architect and data engineer. From the description above, derived primarily by the community of practitioners in this field, it emerges that these roles are clearly defined concerning their main activities and technical requirements. However, there is a lack of academic work in understanding the contribution provided by the data architect, database developer, and data engineer to the business performance and limited attention devoted to the analysis of the behavioral competencies that distinguish these professionals.

Table 2.1  Summary of the main responsibilities of data architect, database architect, database administrator, and data engineer Responsibilities/role

Data architect

Data warehousing solutions Extraction, transformation, and loading Data architecture development Data modeling System development Database architecture testing Installing data warehouse solutions Designing of database to meet scalability, security, performance, and reliability requirements Managing of data structure Maintaining the database’s security Organizing database recovery and backup procedures Monitoring database performance

✔ ✔

Database architect

Database administrator

Data engineer ✔

✔ ✔ ✔

✔ ✔ ✔

✔ ✔

✔ ✔ ✔

28 

S. BONESSO ET AL.

2.3   Data Scientist: Is It Still the Twenty-First century’s Sexiest Job? When in 2008 the influential data scientists D.J. Patil and Jeff Hammerbacher shared their experience respectively in LinkedIn and Facebook and discussed what to call the members of their teams, coining the term “data scientist,” they probably did not imagine that ten years later that role would become the most sought-after profession in the world (Costa and Santos 2017; Davenport and Patil 2012). According to Glassdoor’s 50 Best Jobs in America for 2019, data scientist is the best job in America for the fourth year in a row based on the number of open positions, median base salary, and job satisfaction. LinkedIn’s list of most promising jobs of 2019 confirmed this trend, indicating data scientist as the most in-demand role in the US, and the job site Indeed reported in 2019 that job postings have increased by 256 percent since 2013. Similarly, in Europe and the Asia Pacific region, the demand for data scientists is characterized by continuous growth and, consequently, by a high skills shortage (APEC 2017; Big Cloud 2019). This evidence confirms the prediction made by Davenport and Patil in 2012 when they defined data scientist as “the sexiest job of the 21st century.” Due to the rapid technological changes introduced in the data science field in the last decade, the role of data scientist has evolved dramatically in recent years, generating a lack of consensus about what exactly this individual does and what skills are needed. Data scientists are conceived as a hybrid of the following five traits (Davenport and Patil 2012; Davenport 2014): • Hacker: capability to code (the 2019 Big Cloud’s report on data scientist salaries and jobs in Europe shows that Python is the most popular modeling coding language, used by almost 60 percent of the respondents, followed by R, SQL, JAVA, and Matlab) and familiarity with big data technology architectures (e.g., Hadoop/MapReduce). • Scientist: capability to design experiments and to collect, analyze, and describe findings from data. • Trusted adviser: capability to communicate and manage relationships in order to support executives in the decision-making process. • Quantitative analyst: capability to conduct statistical and visual analysis, also of unstructured data, and to master the machine learning approach.

2  HOW BIG DATA CREATES NEW JOB OPPORTUNITIES: SKILL PROFILES… 

29

• Business expert: knowledge about the specific business enables the data scientist to generate hypotheses and test them quickly and to provide valuable solutions to key functional problems. Deriving a definition of data scientists through a Delphi study that involved several practitioners, Vidgen et al. (2017: 634) highlighted other attributes of data scientists. Specifically they “must be curious, problem-­ focused, able to work independently, and capable of co-creating and communicating stories to the business that form the basis for actionable insight into data.” Thus, a first distinguishing trait of data scientists that explains the use of the term “scientists” is to have an inquisitive nature or an intense curiosity: they ask new open-ended questions and formulate hypotheses to be tested to understand the meaning of data and generate innovative and practical solutions to business problems. Research conducted by Harris and Mehrotra (2014) highlighted the creative and entrepreneurial mind-­ set that characterizes data scientists: they often address original problems and require a high level of autonomy in testing their hypotheses and implementing improvements derived from their discoveries. When in 2006 Jonathan Goldman, former data scientist at LinkedIn, created People You May Know, he demonstrated his inquisitive nature starting from a question: How do users build their network of contacts in LinkedIn? Studying data on social networks already available by the company, he formulated theories and tested some hypotheses, trying to discover models that could predict which type of network a specific user profile would have built based on his/her features. He was in charge of the implementation of a recommendation engine that enabled LinkedIn to support users to get in contact with profiles that the model suggested as of potential interest for the user. The case illustrates how data scientists are motivated in their job by the opportunity to be engaged in meaningful projects that enable them to work with interesting data and original problems (Davenport and Patil 2012; Vidgen et al. 2017). However, recent surveys reveal that data scientists spend most of their time cleaning and organizing data rather than mining or modeling it, and that this activity of data preparation is viewed as the least enjoyable part of their work (CrowdFlower 2017). This explains the low company tenure of data scientists who on average stay one year or less in the same organization (Big Cloud 2019). From the “data” side, Granville (2014) defined their work based on three main activities: discovering good data sources, accessing data, and

30 

S. BONESSO ET AL.

distilling information from data to provide valuable insights for the decision-­ making process. This latter activity implies data exploration, cleaning, summarization and analysis, the development of algorithms that process data automatically, and data presentation. This role is also labelled “advanced analyst” (Burning Glass Technologies 2017) in order to differentiate it from that of the business or data analyst. Data scientists not only clean, analyze, and visualize data, just like data analysts, but they have a deeper expertise in these activities and are able to train and optimize machine learning models. Harris and Mehrotra (2014) illustrated the main differences between these two profiles considering different criteria, such as: • the type of data they work with: analysts use more structured and semi-structured data, whereas scientists also deal with unstructured data; • the tools deployed: analysts use statistical and modeling techniques, whereas scientists adopt mathematical languages, machine learning, natural language processing, and open-source tools that access and manipulate data on multiple servers like Hadoop; and • the nature of the work: report, predict, prescribe, and optimize versus explore, discover, and investigate. Thus, while an analyst may be able to describe trends and translate those results into business terms, the scientist will raise new questions and will be able to build models to make predictions based on new data. In order to do so, the knowledge base expected from a data scientist exceeds that for a computer scientist or a statistician. Their role is different from that of computer scientists since they have a much stronger background in computational statistics, experimental design, sampling, and Monte Carlo simulations. It also differs from statisticians since data scientists’ activity is not limited to data analysis but also encompasses the implementation of predictive algorithms that process data automatically and requires them to master sophisticated statistical techniques that will be applied to manipulate and extract value from large-volume, fast-flowing, and unstructured data (Granville 2014). Thus, data scientists in comparison to data analysts are more technical in nature, dealing with programming, computational complexity, algorithm design, data mining, distributed architecture, and artificial intelligence (Verma et al. 2019).

2  HOW BIG DATA CREATES NEW JOB OPPORTUNITIES: SKILL PROFILES… 

31

This skill set sought in data scientists is confirmed by LinkedIn (2019), which indicates data mining, data analysis, Python, R, and machine learning as the most requested skills. LinkedIn profiles have been used by academic researchers to explore how data scientists define their knowledge base and skill profile. The top ten self-reported featured skills of data scientists confirm those illustrated above: data analysis, Python, R, SQL, machine learning, data mining, research/statistics, SPSS, SAS, and statistical modeling (Ecleo and Galido 2017). Since big data requires the analysis of a huge amount of data in text, video, or image formats, data scientists are requested to be familiar with at least some of the analytical approaches for unstructured data, such as natural language processing for extracting meaning from text. Its application is particularly valuable in investigating customers’ opinion on products or brands (Davenport 2014). Table 2.2 reports a summary of the technical skills of data scientists most frequently mentioned in academic studies. Besides the technical knowledge and skills, the different studies on data scientists’ profiles are devoting attention to those soft or behavioral competencies that they should manifest to work collaboratively within the different parts of the organization and to deliver the results of their work effectively. Table 2.3 reports the main studies that have described the soft skills of data scientists. Table 2.2  Technical skills of data scientists identified in scholarly articles Data scientists’ technical skill set

Sources

Query databases (e.g., SQL)

De Mauro et al. (2016); Harris et al. (2013); Kandel et al. (2012) Costa and Santos (2017); De Mauro et al. (2016); Harris et al. (2013); Kim and Lee (2016) Costa and Santos (2017); Kandel et al. (2012) Dhar (2013)

Work with and manage big data (e.g., data mining process, extract information, and identify patterns) Data cleaning, transformation, and processing (e.g., apache Hadoop) Identify correlation and causation among data Identify patterns, trends, and opportunities Apply statistical methods to analyze data (e.g., Bayesian/Monte Carlo statistics, classical statistics) Validate conclusions

Costa and Santos (2017); De Mauro et al. (2016); Harris et al. (2013) Costa and Santos (2017); De Mauro et al. (2016); Harris et al. (2013); Kim and Lee (2016); Lee and Han (2016) Costa and Santos (2017)

32 

S. BONESSO ET AL.

Table 2.3  Soft skills of data scientists identified in scholarly articles Data scientist’s soft skill set

Sources

Curiosity

Costa and Santos (2017); Davenport and Patil (2012); Vidgen et al. (2017) Davenport and Patil (2012); Kim and Lee (2016) Harris and Mehrotra (2014) Kim and Lee (2016); Shirani (2016) Shirani (2016) Costa and Santos (2017); Harris and Mehrotra (2014) Vidgen et al. (2017) Kim and Lee (2016) Kim and Lee (2016) Costa and Santos (2017); Davenport and Patil (2012) De Mauro et al. (2016)

Creative thinking Exploring new ideas Problem solving Critical thinking Entrepreneurial attitude Working independently Self-motivation Adaptability Business acumen Understanding business context Strategic thinking Communication Teamwork Customer orientation Leadership

Kim and Lee (2016) Costa and Santos (2017); Davenport and Patil (2012); Kim and Lee (2016); Shirani (2016); Verma et al. (2019) Shirani (2016); Verma et al. (2019) Kim and Lee (2016) Kim and Lee (2016); Verma et al. (2019)

As previously highlighted, personal capabilities frequently associated with this role are those related to the generation of innovative solutions, such as curiosity, creative thinking, problem solving, and critical thinking. Moreover, as discussed above, data scientists are characterized by a growth mind-set; thus they are motivated by interesting and original problems, which enable them to set challenging goals and undertake a learning path. They are also required to be adaptable. This means being able to embrace change and find alternative ways to address the problem, being comfortable in dealing with uncertainty and unstructured activities, and learning from failure. Another set of capabilities that seems to distinguish data scientists is that related to business acumen: understanding the business environment and strategic thinking. Data scientists should wear the hat of business experts and advisors, reporting the metrics, such as the return on investment, of the identified solutions in order to quantify their impact on the business. In doing so, they cannot only sit in front of their computers immersed in data in the hope of finding something interesting. They should be fully engaged in the business and work closely with executives to acquire a deep domain knowledge and to provide insight on a strategic

2  HOW BIG DATA CREATES NEW JOB OPPORTUNITIES: SKILL PROFILES… 

33

level (Kim and Lee 2016). They also require autonomy and an entrepreneurial mind-set to explore and take action for business opportunities, to ask the right questions for the specific business, and to be responsible for the implementation of the analytical solutions (Stadelmann et al. 2019). Moreover, data scientists should not carry out their activity being isolated in silos; instead they should be connected with important stakeholders and the other big data roles, specifically data engineers and analysts (Redman 2019). Consequently, among the skills aimed at favoring relationship management, the different studies have indicated teamwork, leadership, and customer orientation. Among the relational competencies identified, experts and scholars acknowledge that data scientists should communicate their findings in a way that can be understood by people outside their field. This competency is usually associated with the capability of using data to tell a story supported by visualization tools. Big data analyses often require reporting of data in visual formats. More sophisticated technologies for displaying data in dashboards and visual analytics have been developed in recent years, with a consequent need to support decision makers in their interpretation. Insufficient or ineffective communication has been identified as one of the main factors that explains the failure of a big data project (Davenport 2014).

2.4   Data Analyst Versus Business Analyst Demand for knowledgeable data analytics professionals will show a trend of dramatic growth (World Economic Forum 2018). The main differences between data scientist and business/data analyst have been delineated in the previous section. However, there is still confusion on what distinguishes a data analyst from a business analyst. They both work with data analytics to help others make better data-driven decisions, and often these terms are used interchangeably; however, their job requirements present some differences. Business analysts are more concerned with the business implications of the data and the actions that should be implemented based on their analysis. They usually leverage the work of data scientists to orient the decision-­ making processes toward solutions. Thus, their main responsibility falls in the area of business advisory (driving decision making, influencing business units’ strategies, reporting strategic insights to partners) and project management (analyzing business needs, communicating the results

34 

S. BONESSO ET AL.

achieved) (De Mauro et al. 2016). Daily tasks usually performed by business analysts can be summarized as follows (Kunis 2019): • analyze and elicit business needs about products, services, and project requirements, often through conversations with stakeholders; • define a business case; • analyze large amounts of complex data to provide the business with fact-based insights; • model and specify through documentation the requirements to validate solutions, obtaining the approval of all relevant stakeholders and ensuring that they meet essential quality standards. A recent study that explored the skill sets required for analytics positions underlined that business analysts require domain-specific knowledge, and the statistical package most frequently adopted consists primarily of Excel (Verma et al. 2019). Data analysts are instead more focused on data, and they create reports and visualizations to explain what insights the data is hiding (De Mauro et al. 2016). Thus, they turn numbers into stories, but spend more time in a silo in comparison to the business analyst. Their activity includes: • writing SQL queries to extract data from the data warehouse, cleaning and organizing raw data; • performing recurring and ad hoc quantitative analysis to find trends in the data and to support day-to-day decision making. For instance, they work with customer-centric algorithm models and tailor them to each customer; • translating data into visualizations and metrics, generating reports, and creating and improving dashboards to help the company interpret and make decisions with the data; • presenting the results of a technical analysis to external clients or internal teams. A restricted group of studies, both from academia and practitioner fields, has attempted to detect behavioral competencies of data/business analysts. A study conducted on job ads from Fortune 500 corporate websites suggested that data/business analysts are required to possess interpersonal skills such as communication and self-motivation, as well as flexibility, customer orientation, planning, and leadership (Lee and Han

2  HOW BIG DATA CREATES NEW JOB OPPORTUNITIES: SKILL PROFILES… 

35

2016). Another study conducted on a sample of business analysts identified as relevant soft skills the ability to challenge, communication, confidence, negotiation, and problem solving (Paul and Tan 2015). Communication skills are often emphasized as important abilities (De Mauro et al. 2016; Verma et al. 2019) since these individuals are required to explain results to a non-expert audience, and they play an intermediate role between organizations and clients. Similar competencies are emphasized by practitioners, who suggest that data/business analysts need to score high levels of communication skills, such as negotiation, since they constantly interact with different organizational areas and clients and raise any potential risks that could affect the organization (Egeland 2015).

2.5   Future Challenges for the Big Data Profession This chapter has provided a description of the big data professional profiles most currently sought by employers, delineating their jobs and their skill profile not only in terms of technical requirements but also illustrating the behavioral competencies relevant to successfully perform in their role. Besides the misalignment between the demand and the availability of these roles in the labor market, companies will face several challenges in hiring and retaining these professionals. The first issue concerns the proliferation of new job titles in the big data labor market that can be explained by a deeper specialization of the knowledge and skills necessary to use the new technologies, especially in the artificial intelligence domain and the more sophisticated analytics techniques. Tasks that were usually included in the job description of data scientists and analysts are currently performed by a specialized role. This is the case for instance with data visualization skills, whose importance in supporting the decision-making process has led to the definition of a dedicated role, the data visualization specialist, or the case of machine learning, where specific professionals, like the machine learning engineer and scientist, are requested to perform sophisticated programming and work with complex data sets and algorithms to train intelligent systems. To build effective big data teams, companies should become aware of the complexity of big data professions and understand the main differences in order to orient their search, identifying the key roles they need to put together in the big data team, rather than looking for a single role that is expected to embrace several knowledge domains.

36 

S. BONESSO ET AL.

A second challenge related to the increasing variety of specialized big data jobs is to identify metrics aimed at measuring their individual performance and the contribution they generate for the business. Thus, how does the professional role improve the decision making in the organization? For each profile specific key performance indicators should be developed in order to monitor the value it is generating for the company, and in so doing supporting a data-driven culture. For instance, some metrics can be related to the assessment of the cost savings, the increased revenue obtained thanks to the algorithm generated, the extent, time, and access to data sources, the number of solutions proposed, or amount of fraud eliminated. Another relevant issue concerns the continuous updating of the competency profiles of these roles that are subject to continuous change and the sources used to identify the skill set. The analysis conducted in this chapter shows that – especially in the case of behavioral competencies – the profile is still ill defined, with a focus primarily on relationship management skills such as communication and teamwork. The complexity of these roles may require a broader set of emotional, social, and cognitive competencies to be performed effectively, as anticipated in Chap. 1. Moreover, academic experts have mainly focused their attention on the analysis of behavioral competencies of data scientists, devoting less attention to the other professional profiles. As a consequence, soft skills of the other job families are derived from practitioners mainly considering the employers’ job requirements that are limited to a few soft skills, primarily the social ones. Finally, the research conducted by professional and academic experts has primarily relied on opinions, which are subject to potential bias. Indeed, data on the skill set of professionals in the big data field is collected through practitioners, for instance adopting Delphi methodology (Vidgen et al. 2017) or analyzing web-based job postings (De Mauro et  al. 2016; Verma et  al. 2019), or consulting the profiles published in LinkedIn (Ecleo and Galido 2017). A more in-depth investigation of the actual competencies activated by these role holders while they are performing their jobs is required to provide a more complete and concrete representation of the set of behaviors that are necessary to achieve superior performance. The following chapters will provide insights on this specific issue.

2  HOW BIG DATA CREATES NEW JOB OPPORTUNITIES: SKILL PROFILES… 

37

References APEC Human Resource Development Working Group. (2017, July). Data science and analytics skills shortage: Equipping the APEC workforce with the competencies demanded by employers. Retrieved from https://www.apec.org/Publications/ 2017/11/Data-Science-and-Analytics-Skills-Shortage Bayern, M. (2019, July 12). Database administrator vs. database architect: What’s the difference?. Retrieved from https://www.techrepublic.com/article/ database-administrator-vs-database-architect-whats-the-difference/ Big Cloud. (2019, January). Data science salary report 2019 Europe. Retrieved from https://bigcloud.io/salary-survey-apac/ Burning Glass Technologies. (2017). The quant crunch: How the demand for data science skills is disrupting the job market. Retrieved from https://www.bhef. com/publications/quant-crunch-how-demand-data-science-skills-disruptingjob-market Burns, E. (2016, August 30). The data engineer is one of the most important jobs in big data. Search Business Analytics. Costa, C., & Santos, M. Y. (2017). The data scientist profile and its representativeness in the European e-competency framework and the skills framework for the information age. International Journal of Information Management, 37(6), 726–734. CrowdFlower. (2017). 2017 Data scientist report. Retrieved from https://www. figure-eight.com/download-2017-data-scientist-report/ Datalandscape. (2017). The European data market study: Final report. Retrieved from http://datalandscape.eu/study-reports/european-data-market-studyfinal-report Davenport, T. H. (2014). Big data at work: Dispelling the myths, uncovering the opportunities. Boston: Harvard Business Review Press. Davenport, T. H., & Patil, D. J. (2012). Data scientist: The sexiest job of the 21st century. Harvard Business Review, 90(October), 70–76. De Mauro, A., Greco, M., Grimaldi, M., & Nobili, G. (2016, June 15–17). Beyond data scientists: A review of big data skills and job families. International Forum on Knowledge Asset Dynamics 11th, Proceedings of IFKAD, 2016, Towards a New Architecture of Knowledge: Big Data Culture and Creativity, Dresden (Germany), 1844–1857. Dhar, V. (2013). Data science and prediction. Communications of the ACM, 56(12), 64–73. Ecleo, J. J., & Galido, A. (2017). Surveying LinkedIn profiles of data scientists: The case of the Philippines. Procedia Computer Science, 124, 53–60. Egeland, B. (2015, August 27). 5 key soft skills for IT business analysts. BAtimes. Gartner. (2018). 3 top take-aways from the Gartner chief data officer survey. Gartner, January 2019.

38 

S. BONESSO ET AL.

Gartner. (2019, February 18). Gartner identifies top 10 data and analytics technology trends for 2019. Gartner press releases. Glassdoor. (2019). 50 best jobs in America for 2019. Retrieved from https://www. glassdoor.com/about-us/bestjobs2019/ Granville, V. (2014). Developing analytics talent. Becoming a data scientist. Indianapolis: Wiley. Harris, J. G., & Mehrotra, V. (2014). Getting value from your data scientists. MIT Sloan Management Review, 56(1), 15–18. Harris, H. D., Murphy, S. P., & Vaisman, M. (2013). Analyzing the Analyzers. An introspective survey of data scientists and their work. Sebastopol: O’Reilly Media. Indeed. (2019, March 14). Top 5 tech skills for data scientists. Retrieved from https://www.beseen.com/blog/talent/data-scientist-skills/ Kandel, S., Paepcke, A., Hellerstein, J.  M., & Heer, J. (2012). Enterprise data analysis and visualization: An interview study. IEEE Transactions on Visualization and Computer Graphics, A18(12 December), 2917–2926. Kim, J. Y., & Lee, C. K. (2016). An empirical analysis of requirements for data scientists using online job postings. International Journal of Software Engineering and Its Applications, 10(4), 161–172. Kunis, L. (2019, January). Business analyst vs. data analyst. Springboard. Lee, C.  K., & Han, H.  J. (2016). Analysis of skills requirement for entry-level programmer/analysts in fortune 500 corporations. Journal of Information Systems Education, 19(1), 17–27. LinkedIn. (2018). LinkedIn Workforce Report. Retrieved from https://economicgraph.linkedin.com/resources/linkedin-workforce-report-december-2018 LinkedIn. (2019, January 10). The most promising jobs of 2019. Retrieved from https://blog.linkedin.com/2019/january/10/linkedins-most-promising-jobsof-2019 Mahadevan, M. (2018). Data professional at work. New York: Apress. Marr, B. (2018, May 9). The 6 top data jobs in 2018. Forbes. McKinsey Global Institute. (2016, December). The age of analytics: Competing in a data-driven world. Retrieved from https://www.mckinsey.com/business-­ functions/mckinsey-analytics/our-insights/the-age-of-analytics-competing-in-a-datadriven-world NewVantage Partners. (2019). Big Data and AI Executive Survey 2019. Executive Summary of Findings. Paul, D., & Tan, Y.L. (2015). An investigation of the role of business analysts in IS development. ECIS 2015 Completed Research Papers. Paper 142. PwC. (2017). Investing in America’s data science and analytics talent. Retrieved from https://www.pwc.com/us/en/library/data-science-and-analyticsskills.html Redman, T.C. (2019, May 16). Do your data scientists know the “why” behind their work? Harvard Business Review.

2  HOW BIG DATA CREATES NEW JOB OPPORTUNITIES: SKILL PROFILES… 

39

Shirani, A. (2016). Identifying data science and analytics competencies based on industry demand. Issues in Information Systems, 17(4), 137–144. Stadelmann, T., Stockinger, K., Bürki, G.  H., & Braschler, M. (2019). Data Scientists. In M.  Braschler, T.  Stadelmann, & K.  Stockinger (Eds.), Applied data science: Lessons learned for the data-driven business (pp.  31–45). Cham: Springer. The Lisbon Council and International Data Corporation (IDC). (2019). Data as the Engine of Europe’s Digital Future: The European Data Market Monitoring Tool Report Update 2018 – Second Report on Policy Conclusions. Verma, A., Yurov, K. M., Lane, P. L., & Yurova, Y. V. (2019). An investigation of skill requirements for business and data analytics positions: A content analysis of job advertisements. Journal of Education for Business, 94(4), 243–250. Vidgen, R., Shaw, S., & Grant, D. B. (2017). Management challenges in creating value from business analytics. European Journal of Operational Research, 261(2), 626–639. World Economic Forum. (2018). The Future of Jobs Report 2018. Retrieved from https://www.weforum.org/reports/the-future-of-jobs-report-2018

CHAPTER 3

Emotional and Social Intelligence Competencies in the Digital Era

Abstract  It is widely acknowledged that emotional intelligence is a crucial component in organizations. It has been proved that leaders and employees who are emotionally intelligent are more efficient, creative, and make better decisions. Although decades of studies in different settings have analyzed emotional intelligence in a variety of roles, there is still a limited application concerning new job profiles such as data professionals. In a world which is rapidly changing because of technological and digital innovation, it is timely to analyze not only the technical skills of these new emerging job profiles but also their behavioral competencies. This chapter aims to first delineate the main characteristics of emotional intelligence. Second, it provides an indepth clarification of the developed competency framework used to detect the behavioral competencies of big data professionals. Finally, it offers insights about how soft skills are considered to be as crucial as technical skills by the labor market despite the difference between machine and human skills that seem to draw attention to technological and data driven competencies. Keywords  Emotional, Social, and Cognitive competencies (ESCs) • Competency-based human resource management • Competency model • Digitalization The study of competencies opens the door to insights about humans and human talent, and potential applications for their development. (Boyatzis 2009: 764) © The Author(s) 2020 S. Bonesso et al., Behavioral Competencies of Digital Professionals, https://doi.org/10.1007/978-3-030-33578-6_3

41

42 

S. BONESSO ET AL.

3.1   Emotional Intelligence and Data Driven Organizations Every day we experience nearly 500 emotional experiences. We usually perceive only a fraction of them, but they influence how we interact with other people, how we perceive events, and how we make decisions, and they eventually impact our job performance. Intense discussions with colleagues, personal loss, expectations or frustrations, and even favorable events could alter our own feelings and those of others. Since the beginning of the 2000s, Google has been concerned about the well-being of its employees. Why was a data-mining giant like Google interested in whether its employees were happy at work or unhappy? Chade-Meng Tan was an engineer hired by Google in 2000, employee number 107. He was one of the first to improve the quality of the site’s search results and he played a key role in the launch of mobile search. In 2007, with a team of leading experts in mindfulness, neuroscience and emotional intelligence, he developed an internal course for fellow Google employees called “Search Inside Yourself” (SIY). In those days, Google allowed its employees to spend 20% of their working time on whatever side project they wanted. It is because of this 20% that Gmail and Maps were developed. Chade-Meng Tan had the idea to offer a curriculum for emotional intelligence because “emotional intelligence can help people succeed” (Meng Tan 2012). Rapidly, the course became so popular that was attended by more than 1000 employees and was featured on the front page of the Sunday Business section of The New  York Times. The main purpose of the course was to help individuals find a balanced awareness of what’s happening around them in a way that diminishes stress and frustration both at work and in their personal lives. SIY evolved from a meditation program into a course on emotional intelligence and that was the turning point: “emotional intelligence even affects the work effectiveness of engineers. Among the top six characteristics that distinguish top engineers from average engineers, only two are cognitive; four have to do with emotional competencies” (Meng Tan 2012). To see whether Meng Tan’s words matched with reality, Google collected feedback and psychometric data. They found that some of the outcomes experienced by the participants were more retention, more promotions, a better work environment, more empathy, less self-rumination and self-perceived stress, and more constructive self-criticism.

3  EMOTIONAL AND SOCIAL INTELLIGENCE COMPETENCIES… 

43

In this chapter, we will briefly describe the model of Emotional Intelligence based on the competencies that enable individuals to demonstrate an intelligent use of their emotions in managing themselves and working with others to be effective at work. A brief history of the concept of Emotional Intelligence will be presented and the development of the competency framework adopted in the following chapter (Chap. 4) will be explained. Finally, implications for a theory of behavioral competencies in a changing digitalized work environment will be discussed.

3.2   Competency-Based Approach and Emotional Intelligence: The Importance in the Workplace It was David McClelland who in 1973 advanced the concept of competence as a basis for identifying what differentiates outstanding performers from average performers at work. By collecting data from more than thirty organizations and interviewing executive roles in different sectors, he showed that a wide range of emotional competencies, rather than a few sets of cognitive competencies, distinguished top performers from average ones (McClelland 1998). McClelland paved the way for scholars and practitioners to use a variety of forms of competency-based human resource management (Boyatzis 2009; Campion et al. 2011; Vos et al. 2015). This was drawn from the main assumption that competencies are essential differentiators of performance. According to McClelland (1973), competence was more important for success in work and in life than was intelligence, as traditionally defined and measured by IQ tests. In particular, McClelland (1994) maintained that “there are alternative combinations of characteristics that lead to success in a particular job” (McClelland 1994, cited by Jacobs 2001: 161). Richard Boyatzis (1982) and Daniel Goleman (1995) were strongly influenced by the work of McClelland, and they developed a theory of performance in the work setting. They suggested that “emotional intelligence is observed when a person demonstrates the competencies that constitute self-awareness, self-management, social awareness, and social skills at appropriate times and ways in sufficient frequency to be effective in the situation” (Boyatzis et al. 2000). This stream of research, based on the theory of performance, pays attention to explaining and predicting the outcome of effectiveness in ­different workplace contexts, usually by observing managers and leaders

44 

S. BONESSO ET AL.

(Boyatzis 1982; Boyatzis et  al. 1995; McClelland 1973; Spencer and Spencer 1993). This is the so-called “competency” approach. According to this approach, a behavioral competency is an underlying characteristic of a person that leads to or causes effective or superior performance. Competencies are hence learned capabilities that lead to effective or superior performance and are reflected by a set of behaviors that share a common underlying intent (Boyatzis 2006, 2009). Indeed, the concept of competency comprehends both action (how an individual behaves according to a specific situation) and intent (how much effort an individual has towards something) (Boyatzis 2009). Specific competencies are identified in four domains: self-awareness, self-management, social awareness, and relationship management (Boyatzis 2008; Boyatzis et  al. 2000; Goleman 1995, 1998). Self-­ awareness is the foundational component, because it is the ability to identify our own emotions and the effect they have on us and others (emotional self-awareness). It implies a profound knowledge about our own strengths and weaknesses that is needed in order to constantly motivate ourselves. Self-awareness is achieved through an accurate self-assessment (Boyatzis 1982) which allows individuals to see their own personal abilities and limitations through a constant search for feedback, and the capacity to learn from mistakes (Boyatzis 1982; Goleman 2001). Self-management refers to “managing one’s internal states, impulses, and resources” (Boyatzis 2016: 288) and it allows individuals to regulate their own emotions and to identify and prevent emotional triggers. In particular, individuals with high self-management competencies are able to avoid distress and disruptive feelings such as rumination. Social awareness collects competencies that refer to knowing and managing emotions in others. Namely, these are competencies that enable individuals to accurately read situations and empathize with the emotions of others. It allows individuals to handle relationships and other’s feelings. On the other hand, relationship management includes competencies that involve the relationships with others and the capacity to induce desirable responses in others. The effectiveness of the relationship skills also depends on the ability to attune ourselves to the emotions of another person. Decades of studies conducted by Boyatzis (1982), Goleman (1998, 2001) and colleagues have shown how emotional self-awareness is a prerequisite for effective self-management, which conversely could predict greater social competencies. Figure 3.1 presents the emotional and social intelligence competency framework.

3  EMOTIONAL AND SOCIAL INTELLIGENCE COMPETENCIES… 

45

Fig. 3.1  The emotional and social intelligence competency framework

The contingency model advanced by Richard Boyatzis (1982) maintained that performance effectiveness derives from the best fit between the individual, his or her job demands, and the organizational environment. Moreover, a bundle of competencies rather than a single one in itself would have a more significant impact on performance, echoing the insight that “In life—and particularly on the job—people exhibit these competencies in groupings, often across clusters, that allow competencies to support one another” (Goleman 2001: 39). This is not the only approach. Another significant line of research, opened by Mayer and Salovey, put into question the way to conceive and derive traditional standards of intelligence (Mayer and Salovey 1997; Mayer et al. 2000a, b; Salovey and Mayer 1990). Individuals with a high Emotional Intelligence score were considered to be able to perceive emotions in themselves and others, but also to regulate them in order to reach a positive status (Salovey and Mayer 1990). Salovey and Mayer’s original model (1990: 189) described emotional intelligence as the “ability to monitor one’s own and others’ feelings and emotions, to discriminate among them, and to use this information to guide one’s thinking and action”. Hence, their framework (Mayer and Salovey 1997; Salovey and Mayer 1990) was cognitive oriented, because it was mainly focused on the mental attitude needed to recognize and interpret emotions (see Goleman 2001 for an in-depth description of the model and its evolution).

46 

S. BONESSO ET AL.

So far, studies have tried to delineate the set of competencies that lead to higher performance in different settings and in a variety of jobs. The following section will describe the empirical evidence found by extant literature.

3.3   Emotional Intelligence and Work Environment The data documenting the importance of emotional intelligence competencies for outstanding performance have been collected for more than two decades. Individuals with a higher emotional intelligence are more effective at work; they are more creative and better leaders (Fernández-­ Berrocal and Extremera 2006; Goleman et al. 2002; Newman et al. 2010; Joseph et al. 2015; O’Boyle et al. 2011). Moreover, a longitudinal study on a large sample of graduates showed that emotional intelligence competencies such as adaptability and teamwork predict career satisfaction and success (Amdurer et al. 2014). Furthermore, a study conducted by food service employees from nine different locations of the same restaurant franchise showed that job satisfaction and performance were highly correlated with employees’ emotional intelligence (Sy et al. 2006). Research has provided empirical insights on those competencies needed to achieve superior performance at individual and organizational levels (Dreyfus 2008; Koman and Wolff 2008; Ryan et al. 2009; Zhang and Fan 2013). Indeed, over the last two decades, emotional intelligence competencies have been found to significantly impact different aspects of the workplace, such as job performance, job satisfaction and job outcomes (Sy et al. 2006), low turnover (Wong and Law 2002), company rank (Lopes et al. 2006), and group performance as well (Koman and Wolff 2008). Depending on the organizational context and situational factors that characterize specific jobs, certain combinations of competencies are more important than others in order to pursue positive performance. For instance, managing groups and interpersonal sensitivity were two competencies found among the most effective scientists and engineers of a major US government research center (Dreyfus 2008). The distinguishing ­competencies demonstrated by the physician leaders who were part of a top-­rated US academic healthcare institution were empathy, initiative, emotional selfawareness and organizational awareness (Hopkins et al. 2015). A study on different sized Italian companies showed that outstanding executives

3  EMOTIONAL AND SOCIAL INTELLIGENCE COMPETENCIES… 

47

showed more efficiency orientation, initiative, self-­confidence, networking, oral communications, persuasiveness, systems thinking, and pattern recognition (Boyatzis and Ratti 2009). Measures of the observed behaviors have been tested and validated by a rich set of research in the last decades and several approaches have been found: such as informants’ ratings (360-degree assessments), behavioral event interviews (Boyatzis 1982; Spencer and Spencer 1993), direct coding of behaviors from audio tapes of critical incidents or videotapes of simulations (Boyatzis 2009; Cherniss 2010), and assessment centers (Thornton and Byham 1982). These methods share the core hypothesis that behaviors are observed and rated by knowledgeable informants rather than by oneself (Boyatzis 2016). Therefore, the literature on behavioral competencies and emotional intelligence has provided empirical insights into the distinctive competencies that explain outstanding performance in specific professional roles (see Table 3.1 for some examples). Despite the large body of research over the last decades that have investigated the distinguishing behavioral competencies of the best performers in a variety of roles, so far, limited attention has been devoted to the job profiles that are emerging in the big data field. As highlighted in Chap. 2, the competency profile of these roles is still ill defined. Studies have relied primarily on methodological approaches subject to potential bias such as asking practitioners’ opinions on the most relevant competencies they think big data professionals should demonstrate or analyzing employer’s web-based job postings. The following section will introduce a competency-­ based framework that can be adopted to the exploration of the behavioural competencies that characterize professionals in the data science field.

3.4   The Behavioral Competencies Necessary in Today’s Workplace: The Development of a Competency Framework The models and studies presented in the previous section have deeply contributed to an understanding of how emotional intelligence competencies represent a crucial determinant of better performance in the workplace. Likewise, the identification of the behavioral competencies needed for a job allows companies to better orient their search in the labor market. However, new job profiles have started to emerge within a business

48 

S. BONESSO ET AL.

Table 3.1  Examples of professional roles analyzed in terms of performance outcomes and related behavioral competencies Authors

Professional roles

Boyatzis (2006)

Leaders in a large US consulting company

Dreyfus (2008)

Williams (2008)

Boyatzis and Ratti (2009) Boyatzis and Ratti (2009) Ryan et al. (2009) Boyatzis et al. (2012)

Performance indicators

Distinctive behavioral competencies

Higher revenue from the Planning, achievement clients and gross margin orientation, self-confidence, taking a risky stand, self-­ control, adaptability, conscientiousness, values learning, networking, leadership, coaching, empathy, facilitating learning, Systems thinking. Scientists and Higher performance Managing groups and engineers working collected through interpersonal sensitivity at a major US nominations from peers, government supervisor, and research center subordinates School principals Higher performance Self-confidence, achievement in a large collected through peers, orientation, initiative, Midwestern supervisor and teachers’ organizational awareness, United States nominations. Use of a leadership, teamwork urban school broader repertoire of district environmental spanning strategies Executives in an Higher performance Efficiency orientation, Italian division of collected through self-confidence, networking, a large nomination from peers pattern recognition, systems multinational and supervisors thinking Managers in Higher performance Empathy, group management, Italian collected through developing others, oral cooperatives nomination from peers communications, and use of and supervisors concepts. European Higher performance Achievement orientation, managers collected through initiative, teamwork and nomination from clients cooperation, and team leadership Divisional Number of financial Adaptability and influence executives in a advisor recruiters whose financial services total compensation company package is entirely based on commissions for new cash invested by clients (continued)

3  EMOTIONAL AND SOCIAL INTELLIGENCE COMPETENCIES… 

49

Table 3.1 (continued) Authors

Professional roles

Performance indicators

Distinctive behavioral competencies

Zhang and Fan (2013)

Chinese project managers working on large and complex construction projects

Meeting project’s overall performance, meeting owner’s requirements, meeting project’s multiple goals, stakeholders’ satisfaction

Emotional self-awareness, emotional self-control, empathy, organizational awareness, cultural understanding and communication

e­ nvironment that have posed different challenges. New technologies, different innovation processes, and more importantly, Industrial Revolution 4.0 have required organizations to change their structures and hire new employees. Consequently, existing competency models must be integrated and take into consideration those individual behaviors that are even more necessary for operating in todays’ more complex organizational environment. Drawing on the extensive contributions provided by the competency-­ based approach (Boyatzis 1982; Boyatzis et  al. 2000; Goleman 1998; Goleman et al. 2002; Spencer and Spencer 1993) but also considering the following: • the evidence that emerged from several interviews we administrated (adopting the Behavioural Events Interview technique) (Boyatzis 1998; McClelland 1998) in the period 2015–2017 to several effective managerial and entrepreneurial roles operating in different industries; • the soft skills most needed by employers in today’s workplace (e.g. Azevedo et  al. 2012; LinkedIn 2019; McKinsey Global Institute 2018; QS 2019; Robles 2012; The World Economic Forum 2018); • the most advanced studies that investigate the effective behaviors that big data professionals should manifest to perform their job effectively and to assume a more innovative and entrepreneurial mindset (see Chap. 2 for a review); we defined a framework that encompasses thirty-three competencies grouped into six thematic areas that are defined as follows:

50 

S. BONESSO ET AL.

• Awareness. Competencies that allow individuals to understand themselves, other people, and the organizational relationships; • Action. Competencies that allow individuals to realize ideas, plans and solutions, and to work methodically and with initiative; • Social. Competencies that allow individuals to interact positively with other people and that help them to work with others effectively; • Cognitive. Competencies that allow individuals to analyze and use information effectively to interpret phenomena or situations; • Exploratory. Competencies related to activating the processes of innovation generation; • Organizational Action. Competencies related to the interpretation of the competitive environment, the identification of business ­opportunities, and the alignment of the individual behaviors to the organizational goals and priorities. The Competency Hexagon – as we labeled the framework – is graphically represented in Fig. 3.2., while Table 3.2 reports in detail the behavioral competencies that contribute to each of the six areas of the Hexagon with the related definition.

Fig. 3.2  The competency Hexagon

3  EMOTIONAL AND SOCIAL INTELLIGENCE COMPETENCIES… 

51

Table 3.2  Competency Hexagon: The thirty-three competencies and related definition Area

Competency

Awareness

Self-awareness

Action

Definition

Capacity to be in tune with your inner self and being able to evaluate the impact of emotions on your actions and work performance, always keeping in mind the guiding values. It is also the capacity to evaluate your inner abilities and limits. It is based on the desire to receive feedback and new perspectives about yourself and to be motivated by continuous learning and self-development Empathy Capacity to sense and accurately understand others’ feelings and perspectives and take an active interest in their concerns Organizational Capacity to locate and decipher social networks and awareness power relations and the ability to understand the “political” balance in any organization and the guiding values and unspoken rules that govern the behavior of its members Efficiency Capacity to perceive input and output relationships and orientation include the concern for increasing the efficiency of actions Achievement Capacity to require high quality standards to try to orientation constantly improve your results, setting challenging and measurable goals, and measuring the progress made Resilience Capacity to recover from adversity and respond to it positively by using personal resources Initiative Capacity to act to accomplish something and to take this action prior to being asked or forced or provoked into it Change agent Capacity to recognize the need for change, to promote and manage it Flexibility Capacity to adapt oneself by modifying one’s behavior in the face of changes, unexpected circumstances or different situations Self-control Capacity to dominate emotions and impulses even in situations of stress or difficulty Accuracy Capacity to develop the activities with precision and to check several times Risk taking Capacity to take a risk or to carry out an activity with an uncertain outcome Risk The capacity to identify in advance possible negative management impacts of uncertain activities and contain losses Collection of Capacity to look for the correct information information (continued)

52 

S. BONESSO ET AL.

Table 3.2 (continued) Area

Competency

Social

Persuasion

Cognitive

Exploratory

Definition

The capacity to convince other people of the value of your point of view and to get their support Conflict The capacity to induce the parties in conflict to have a management dialogue and identify solutions in which everyone can recognize themselves Teamwork Capacity to be collaborative and available to the group, to induce others to engage actively and enthusiastically in the common cause, to reinforce the team spirit and encourage the participation of all members Developing Capacity to stimulate, support and provide resources for others the improvement and growth of other people Networking Capacity to create, maintain, and use personal relationships to achieve goals Leadership Capacity to lead others and trigger phenomena involving emotional resonance, to instill a sense of pride and inspire people through a compelling vision, and to bring out their best aspects Customer Capacity to understand other people’s needs and pay focus attention to their satisfaction Systems The capacity to break down complex problems and thinking understand cause-and-effect relationships between the parties Diagnostic Capacity to conduct an accurate examination of the thinking situation and describe the nature of the problem Pattern Capacity to recognize similarities among issues and recognition make logical connections between concepts of different domains Lateral Capacity to try new ways of looking at problems and thinking adopt unconventional perspectives Questioning Capacity to formulate questions in order to gather information and challenge the current situation Observing Capacity to observe the environment around you in different contexts with the aim of finding new ideas Experimenting Capacity to explore new ideas through experiments and trials (continued)

3  EMOTIONAL AND SOCIAL INTELLIGENCE COMPETENCIES… 

53

Table 3.2 (continued) Area

Competency

Organizational Visionary action thinking Strategic thinking Opportunity recognition Commitment toward the group Integrity

Definition Capacity to create and articulate a vivid future image of your group and/or organization and to define the actions and objectives necessary to achieve it Capacity to understand the strategic and competitive environment of the company Capacity to perceive the opportunities emerging from the environment Capacity to be responsible and to act for the good of the group Capacity to be consistent with yourself

The first four areas of competencies (Awareness, Action, Social, and Cognitive) included in the Hexagon have been defined by the competency-­ based literature. However, within each of these four areas of competency, we have added further skills that are considered to be relevant in the current organizational contexts. Awareness encompasses all of the competencies that in the emotional and social competency framework illustrated in Fig.  3.1 are grouped within the Self-awareness cluster, namely Emotional Self-awareness, and within the Social-awareness cluster are Empathy and Organizational Awareness. These competencies also seem to be relevant for big data professionals. Self-awareness is the fundamental competency of the emotional and social framework because it represents the premise for managing one’s emotions and understanding and managing others (Goleman et al. 2002). With specific regard to empathy and organizational awareness, it has been shown that big data professionals need to have a deep understanding of their business context and of the stakeholders’ needs in order to effectively perform their job (De Mauro et al. 2016). The Action group includes the competencies present in the Self-­ Management cluster (Fig.  3.1), namely Achievement Orientation, Self-­ control, and Flexibility. Moreover, we included Efficiency Orientation, Initiative, and Accuracy as originally described in the codebook of Boyatzis (1982), Change Agent as defined in the Emotional Competency Inventory (ECI) (Wolff 2005), and Collecting Information derived from the work of

54 

S. BONESSO ET AL.

Spencer and Spencer (1993). In this group, we also added Risk Taking and Risk Management, which are competencies that characterize the entrepreneurial mindset (Kyndt and Baert 2015; Morris et al. 2011) and that are expected in the skillset profile of big data professionals (Costa and Santos 2017; Harris and Mehrotra 2014). Indeed, these roles require awareness of the potential returns and losses that a specific proposed solution can bring to the company and accordingly support decision making to maximize the business opportunities. A final competency added to the Action group is Resilience. This competency enables individuals to bounce back from adversity and to retain a sense of hopefulness about the future even in the face of adversity and stress (Dulewicz and Higgs 2005). This is assuming an increasing importance in the labor market. Indeed, in the last QS’s Global Skills Gap (QS 2019), resilience was reported to have the highest deficiency (rated as very important by employers but limitedly present in the profile of the graduate hired) among the skills analyzed in the survey. The Social group encompasses all of the competencies of the relationship management cluster (Fig.  3.1), namely, Conflict Management, Teamwork, Developing Others, Leadership, and Persuasion. In this area of competencies, we added Networking, which was derived from Boyatzis (1982) and from more recent contributions that analyze its role in managerial and entrepreneurial roles (Kyndt and Baert 2015; Snell et al. 2014). Finally, customer focus was included by adopting Spencer and Spencer’s framework (1993). This group of competencies is expected to be relevant in determining the performance outcomes of big data professionals, as discussed in Chap. 2. For instance, data scientists leverage on Developing Others in mentoring junior roles within organizations or need Leadership and Teamwork to work in synergy with others (Kim and Lee 2016; Shirani 2016; Verma et al. 2018). Several studies have emphasized the importance of customer focus for data scientists, since this competency enables them to effectively interpret and satisfy stakeholders’ needs (Kim and Lee 2016). Also persuasion is required to capture the attention of the stakeholders with a compelling data storytelling in order to obtain their support for the proposed solution. The Cognitive area of the Hexagon was derived from the original work of Boyatzis’s (1982) and the subsequent definition of the cognitive intelligence competencies (Boyatzis 2009). Specifically, two competencies, namely Systems Thinking and Pattern Recognition, have been demonstrated to predict outstanding performance across professional

3  EMOTIONAL AND SOCIAL INTELLIGENCE COMPETENCIES… 

55

roles and organizational contexts. Systems thinking is the ability to recognize the factors that have an impact on a complex situation by identifying cause-­ effect relationships among different elements or events. Pattern recognition allows individuals to connect the dots, namely, to identify similarities among different and often distant conceptual domains and to derive new business opportunities. Indeed, pattern recognition – also defined as analogical thinking or associational thinking – is a way to combine previous experiences with new ones and derive potential solutions (Baron 2006; Baron and Ensley 2006). This competency represents the cognitive process of “connecting concepts (ideas, problems, fields of study, events, and trends) that appear, at first glance, to be unconnected” (Dyer et al. 2008: 320). It appears to be central for data professionals as well, because they need to capture meanings or trends in data which are usually unclear at a first glance. We added two further skills to this group of competencies, Diagnostic Thinking and Lateral Thinking. The former is used when individuals make a careful examination of the nature and the causes of a problem and consider why the problem exists, why it is necessary to solve it, who is involved, and how much time is needed (Puccio et al. 2011). Lateral thinking expresses the ability to explore different ways of examining a challenging task instead of accepting what appears to be the solution (De Bono 1992). Whereas vertical thinking builds on existing patterns, lateral thinking seeks to restructure existing patterns or move across patterns by identifying different or unconventional ways and directions to solve a problem (Hernandez and Varkey 2008). Lateral thinking is of great interest in this study, because an unstructured and large volume of data require digging into it and looking at it from different perspectives to find trends and patterns in the data and turn the observations into business solutions (Costa and Santos 2017; De Mauro et al. 2016; Harris et al. 2013). In the Competency Hexagon we added two more areas of competency: Exploratory and Organizational Action. The exploratory competencies describe the behaviors that individuals adopt to scan the world around them and to explore novel ideas. Dyer, Gregersen and Christensen (2008) identified three main exploratory competencies used by entrepreneurs to make sense of different situations: Questioning, Observing, and Experimenting. These three competencies refer to the ability to explore the world and find new ideas by challenging the status quo, by observing and paying attention to what happens, or by experimenting and using a hypothesis testing mindset. These three competencies are consistent with

56 

S. BONESSO ET AL.

previous research that describes big data professionals as individuals who are curious to explore new paths (Costa and Santos 2017; Davenport and Patil 2012; Harris and Mehrotra 2014; Vidgen et al. 2017), and they critically think about a certain situation (Shirani 2016). Finally, the Organizational Action competencies allow individuals to interpret the competitive environment, to sense business opportunities, to think about the future of the organization, and to promote the alignment between individual behaviors and organizational goals. This group includes Visionary Thinking through which the individual is able to create a vivid image of the future of an organization that he or she wants to build in the long term and to share it with other organizational members. Also, it includes Strategic Thinking, which is the ability to understand the strategic and competitive environment of the company (Moon 2013; Puccio et al. 2011). As suggested by previous studies, data scientists need to be driven by business curiosity (Costa and Santos 2017; Davenport and Patil 2012) and oriented by strategic thinking (Kim and Lee 2016). Another competency included in this group is Opportunity Recognition. This is the capacity to perceive opportunities emerging from the environment and to perceive changed conditions or potential resources (Morris et al. 2011). Finally, the framework included Integrity, which is a competency usually linked to authentic leadership and is expected to be found among decision makers (Palanski and Yammarino 2007). Studies have also proved that integrity has a positive relationship with the intentions of the followers and their performance (Dineen et al. 2006; Peterson 2004). This competency has also been found by Robles (2012), who maintained that business executives perceive integrity and work ethic to be among the ten most important soft skills. Moreover, this competency is crucial for big data analytics professionals, because they have daily access to personal and sensitive data about clients, users, and employees.

3.5   The Application of a Behavioral Competency Framework to a Changing Digitalized World A recent article in The Financial Times (Nilsson 2018) reported the story of a young graduate student from a top ranked university. She was just hired when she was asked to use Python, a programming language that she was not familiar with. Rather than giving up, she leveraged on what she learned during her graduate studies that is, to be unafraid. She found

3  EMOTIONAL AND SOCIAL INTELLIGENCE COMPETENCIES… 

57

a way to learn how to master the software and she turned a threat into an opportunity. The interviews of five hundred famous innovators revealed that a “disproportionate number of them” went to Montessori schools, where “they learned to follow their curiosity” (Dyer and Gregersen cited by Brynjolfsson and McAfee 2014: 313). Larry Page, the founder of Google, was one of the Montessori school children and he recalled how “part of that training [was] not following rules and orders [but] being self-motivated, questioning what’s going on in the world, doing things a little bit differently” (Brynjolfsson and McAfee 2014: 314). Both the young students and the outstanding innovators were highly specialized and prepared. Their technical skills were strong and they spent years in refining their knowledge. But the elements that distinguished both the students and the innovators was the capacity to leverage on emotional, social, and cognitive competencies over the years. Therefore, what reports, academia, and policy makers are claiming loudly is that it is imperative to nurture an emotionally intelligent workforce, especially in a fast-changing environment where digitalization is pushing the frontiers between humans and machines. The new workforce needs to be equipped with both hard skills and soft skills. Determining the most important soft skills is the other requirement. As shown in the previous section, different competencies, usually a bundle of competencies, are needed for specific tasks and roles. It is still an open question about the competencies needed for the new emerging job profiles such as data analysts and scientists (World Economic Forum 2018). Likewise, what the new workforce will look like is unclear to policy-makers, business leaders, individual workers, and academics. The following chapter (Chap. 4) will contribute to shed lights on behavioral competencies most frequently manifested by data analysts and data scientists while they actually perform their job.

References Amdurer, E., Boyatzis, R. E., Saatcioglu, A., Smith, M. L., & Taylor, S. N. (2014). Long term impact of emotional, social and cognitive intelligence competencies and GMAT on career and life satisfaction and career success. Frontiers in Psychology, 5, 1447. Azevedo, A., Apfelthaler, G., & Hurst, D. (2012). Competency development in business graduates: An industry-driven approach for examining the alignment

58 

S. BONESSO ET AL.

of undergraduate business education with industry requirements. The International Journal of Management Education, 10, 12–28. Baron, R. A. (2006). Opportunity recognition as pattern recognition: How entrepreneurs “connect the dots” to identify new business opportunities. Academy of Management Perspectives, 20(1), 104–119. Baron, R. A., & Ensley, M. D. (2006). Opportunity recognition as the detection of meaningful patterns: Evidence from comparisons of novice and experienced entrepreneurs. Management Science, 52(9), 1331–1344. Boyatzis, R. E. (1982). The competent manager: A model for effective performance. New York: Wiley. Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis and code development. Thousand Oaks: Sage Publications, Incorporated. Boyatzis, R. E. (2006). Using tipping points of emotional intelligence and cognitive competencies to predict financial performance of leaders. Psicothema, 18, 124–131. Boyatzis, R.  E. (2009). Competencies as a behavioural approach to emotional intelligence. Journal of Management Development, 28(9), 749–770. Boyatzis, R. E. (2016). Commentary on Ackley (2016): Updates on the ESCI as the behavioral level of emotional intelligence. Consulting Psychology Journal: Practice and Research, 68(4), 287–293. Boyatzis, R. E., & Ratti, F. (2009). Emotional, social and cognitive intelligence competencies distinguishing effective Italian managers and leaders in a private company and cooperatives. Journal of Management Development, 28(9), 821–838. Boyatzis, R. E., Cowen, S. S., & Kolb, D. A. (1995). Innovation in professional education: Steps on a journey from teaching to learning. San Francisco: Jossey-Bass. Boyatzis, R. E., Goleman, D., & Rhee, K. (2000). Clustering competence in emotional intelligence: Insights from the emotional competence inventory (ECI)s. In R.  Bar-On & J.  D. A.  Parker (Eds.), Handbook of emotional intelligence (pp. 343–362). San Francisco: Jossey-Bass. Boyatzis, R. E., Good, D., & Massa, R. (2012). Emotional, social, and cognitive intelligence and personality as predictors of sales leadership performance. Journal of Leadership & Organizational Studies, 19(2), 191–201. Brynjolfsson, E., & McAfee, A. (2014). The second machine age: Work, Progress, and prosperity in a time of brilliant technologies. New  York/London: W Norton & Co. Campion, M.  A., Fink, A.  A., Ruggeberg, B.  J., Carr, L., Phillips, G.  M., & Odman, R. B. (2011). Doing competencies well: Best practices in competency modelling. Personnel Psychology, 64, 225–262. Cherniss, C. (2010). Emotional intelligence: Toward clarification of a concept. Industrial and Organizational Psychology, 3, 110–126. https://doi.org/ 10.1111/j.1754-9434.2010.01231.x.

3  EMOTIONAL AND SOCIAL INTELLIGENCE COMPETENCIES… 

59

Cherniss, C., & Boyatzis, R. E. (2013). Leader interpersonal and influence skills: The soft skills of leadership. In R. E. Riggio & S. J. Tan (Eds.), Building interpersonal skills in management programs (pp. 53–72). New York: Routledge. Costa, C., & Santos, M. Y. (2017). The data scientist profile and its representativeness in the European e-Competency framework and the skills framework for the information age. International Journal of Information Management, 37(6), 726–734. Davenport, T. H., & Patil, D. J. (2012). Data scientist: The sexiest job of the 21st century. Harvard Business Review, 90(10), 70–76. 128. De Bono, E. (1992). Serious creativity: Using the power of lateral thinking to create new ideas. New York: Harper Business. De Mauro, A., Greco, M. Grimaldi, M., & Nobili, G. (2016). Beyond data scientists: A review of big data skills and job families. International forum on knowledge asset dynamics 11th, Proceedings of IFKAD, 2016, Towards a new architecture of knowledge: Big data culture and creativity, June 15–17, 2016, Dresden, Germany, 1844–1857. De Vos, A., De Hauw, S., & Willemse, I. (2015). An integrative model for competency development in organizations: The Flemish case. The International Journal of Human Resource Management, 26(20), 2543–2568. https://doi. org/10.1080/09585192.2014.1003078. Dineen, B. R., Lewicki, R., & Tomlinson, E. C. (2006). Supervisory guidance and behavioral integrity: Relationships with employee citizenship and deviant behavior. Journal of Applied Psychology, 91(3), 622–635. Dreyfus, C.  R. (2008). Identifying competencies that predict effectiveness of R&D managers. Journal of Management Development, 27(1), 76–91. Dulewicz, V., & Higgs, M. (2005). Assessing leadership styles and organisational context. Journal of Managerial Psychology, 20(2), 105–123. Dyer, J. H., Gregersen, H. B., & Christensen, C. (2008). Entrepreneur behaviours, opportunity recognition, and the origins of innovative ventures. Strategic Entrepreneurship Journal, 2, 317–338. Fernández-Berrocal, P., & Extremera, N. (2006). Emotional intelligence: A theoretical and empirical review of its first 15 years of history. Psicothema, 18, 7–12. Global Skills Gap Report. (2019). Retrieved in https://www.qs.com/portfolioitems/the-global-skills-gap-report-2019/ Goleman, D. (1995). Emotional intelligence. New York/England: Bantam Books. Goleman, D. (1998). Working with emotional Intelligence. New  York: Bantam Books. Goleman, D. (2001). An EI-based theory of performance. In C. Cherniss, R. E. Boyatzis, & M. Elias (Eds.), Emotionally intelligent workplace (pp. 27–44). The Jossey-Bass Wiley Company: San Francisco. Goleman, D., Boyatzis, R. E., & McKee, A. (2002). Primal leadership: Realizing the power of emotional Intelligence. Cambridge, MA: Harvard Business School Press.

60 

S. BONESSO ET AL.

Harris, J. G., & Mehrotra, V. (2014). Getting value from your data scientists. MIT Sloan Management Review, 56(1), 15–18. Harris, L. R., & Brown, G. T. L. (2013). Opportunities and obstacles to consider when using peer- and self-assessment to improve student learning: Case studies into teachers’ implementation. Teaching and Teacher Education, 36, 101–111. https://doi.org/10.1016/j.tate.2013.07.008. Hernandez, J. S., & Varkey, P. (2008). Vertical versus lateral thinking. Physician Executive, 34(3), 26–28. Hopkins, M. M., O’Neil, D. A., & Stoller, J. K. (2015). Distinguishing competencies of effective physician leaders. Journal of Management Development, 34(5), 566–584. Jacobs, R. (2001). Managing employee competence in global organizations. In J. Kidd, X. Li, & F. Richter (Eds.), Maximizing human intelligence deployment in Asian business (pp. 44–56). New York: Palgrave. Joseph, D. L., Jin, J., & Newman, D. A. (2010). Emotional Intelligence: An integrative meta-analysis and cascading model. Journal of Applied Psychology, 95(1), 54–78. Joseph, D. L., Jin, J., Newman, D. A., & O’Boyle, E. H. (2015). Why does self-­ reported emotional Intelligence predict job performance? A meta-analytic investigation of mixed EI. Journal of Applied Psychology, 100(2), 298–342. Kim, J. Y., & Lee, C. K. (2016). An empirical analysis of requirements for data scientists using online job postings. International Journal of Software Engineering and Its Applications, 10(4), 161–172. Koman, L., & Wolff, S. (2008). Emotional intelligence competencies in the team and team leader. Journal of Management Development, 27(1), 55–75. Kyndt, E., & Baert, H. (2015). Entrepreneurial competences: Assessment and predictive value for entrepreneurship. Journal of Vocational Behavior, 90, 13–25. LinkedIn. (2019) Global talent trends. The 3 trends transforming your workplace. Retrieved from https://business.linkedin.com/content/dam/me/business/ en-us/talent-solutions/resources/pdfs/global_talent_trends_2019_ emea.pdf Lopes, P., Grewal, D., Pepper, J., Gall, M., & Salovey, P. (2006). Evidence that emotional intelligence is related to job performance and affect and attitudes at work. Psicothema, 18(Suppl), 132–138. Mayer, J. D., & Salovey, P. (1997). What is emotional intelligence? In P. Salovey & D.  Sluyter (Eds.), Emotional development and emotional intelligence: Implications for educators (pp. 3–31). New York: Basic Books. Mayer, J. D., Salovey, P., & Caruso, D. R. (2000a). Emotional intelligence as zeitgeist, as personality, and as a standard intelligence. In R.  Bar-On & J.  D. A.  Parker (Eds.), Handbook of emotional intelligence (pp.  92–117). San Francisco: Jossey-Bass.

3  EMOTIONAL AND SOCIAL INTELLIGENCE COMPETENCIES… 

61

Mayer, J. D., Salovey, P., & Caruso, D. R. (2000b). Models of emotional intelligence. In R.  J. Sternberg (Ed.), Handbook of intelligence (pp.  396–420). Cambridge: Cambridge University Press. McClelland, D. C. (1973). Testing for competence rather than for intelligence. American Psychologist, 28(1), 1–14. McClelland, D.  C. (1994). The knowledge-testing-educational complex strikes back. American Psychologist, 49(1), 66–69. https://doi. org/10.1037/0003-066X.49.1.66. McClelland, D. C. (1998). Identifying competencies with behavioral event interviews. Psychological Science, 9, 331–339. McKinsey & Company. (2018). Skill shift: Automation and the future of the workforce. Discussion Paper, McKinsey Global Institute (MGI). Retrieved from https://www.mckinsey.com/~/media/McKinsey/Featured%20Insights/ Future%20of%20Organizations/Skill%20shift%20Automation%20and%20 the%20future%20of%20the%20workforce/MGI-Skill-Shift-Automation-andfuture-of-the-workforce-May-2018.ashx Meng Tan. (2012). Retrieved in https://knowledge.wharton.upenn.edu/article/ googles-chade-meng-tan-wants-you-to-search-inside-yourself-for-inner-andworld-peace/ Moon, B. J. (2013). Antecedents and outcomes of strategic thinking. Journal of Business Research, 66(10), 1698–1708. Morris, M. H., Webb, J. W., & Franklin, R. (2011). Understanding the manifestation of entrepreneurial orientation in the nonprofit context. Entrepreneurship Theory and Practice, 35(5), 947–971. Newman, D. A., Joseph, D. L., & MacCann, C. (2010). Emotional intelligence and job performance: The importance of emotion regulation and emotional labor context. Industrial and Organizational Psychology, 3, 159–164. https:// doi.org/10.1111/j.1754-9434.2010.01218.x. Nilsson, P. (2018, Sept 3). What top employers want from MBA graduates. The Financial Times. Retrieved 15 Feb 2019. https://www.ft.com/ content/64b19e8e-aaa5-11e8-89a1-e5de165fa6191 O’Boyle, E. H., Humphrey, R. H., Pollack, J. M., Hawver, T. H., & Story, P. A. (2011). The relation between emotional intelligence and job performance: A meta-analysis. Journal of Organizational Behavior, 32, 788–818. Palanski, M. E., & Yammarino, F. J. (2007). Integrity and leadership: Clearing the conceptual confusion. European Management Journal, 25(3), 171–184. Peterson, D. (2004). Perceived leader integrity and ethical intentions of subordinates. Leadership & Organization Development Journal, 25(1), 7–23. Puccio, G. J., Mance, M., & Murdock, M. C. (2011). Creative leadership: Skills that drive change. New York: Sage. Robles, M. M. (2012). Executive perceptions of the top 10 soft skills needed in today’s workplace. Business Communication Quarterly, 75(4), 453–465.

62 

S. BONESSO ET AL.

Ryan, G., Emmerling, R. J., & Spencer, L. M. (2009). Distinguishing high‐performing European executives. Journal of Management Development, 28(9), 859–875. Salovey, P., & Mayer, J. D. (1990). Emotional intelligence. Imagination, Cognition and Personality, 9, 185–211. Shirani, A. (2016). Identifying data science and analytics competencies based on industry demand. Issues in Information Systems, 17(4), 137–144. Snell, S. J., Tonidandel, S., Braddy, P. W., & Fleenor, J. W. (2014). The relative importance of political skill dimensions for predicting managerial effectiveness. European Journal of Work and Organizational Psychology, 23(6), 915–929. Spencer, L. M., & Spencer, S. M. (1993). Competence at work: Models for superior performance. New York: Wiley. Sy, T., Tram, S., & O’Hara, L.  A. (2006). Relation of employee and manager emotional intelligence to job satisfaction and performance. Journal of Vocational Behavior, 68, 461–473. Thornton, G.  C., & Byham, W.  C. (1982). Assessment centers and managerial performance. University of California: Academic. Verma, B., Zhang, L., & Stockwell, D. (2018). Roadside video data analysis: Deep learning (Studies in computational intelligence). Singapore: Springer. Vidgen, R., Shaw, S., & Grant, D. B. (2017). Management challenges in creating value from business analytics. European Journal of Operational Research, 261(2), 626–639. Williams, H. (2008). Characteristics that distinguish outstanding urban principals: Emotional intelligence, social intelligence and environmental adaptation. Journal of Management Development, 27, 36–54. https://doi. org/10.1108/02621710810840758. Wolff, S. B. (2005). Emotional competence inventory (ECI) technical manual. Hay GroupMcClelland Center for Research and Innovation. Wong, C. S., & Law, K. S. (2002). The effects of leader and follower emotional intelligence on performance and attitude: An exploratory study. The Leadership Quarterly, 13, 243–274. World Economic Forum. (2018). The Future of Jobs Report 2018. Retrieved from http://www3.weforum.org/docs/WEF_Future_of_Jobs_2018.pdf Zhang, L., & Fan, W. (2013). Improving performance of construction projects. A project manager’s emotional intelligence approach. Engineering, Construction and Architectural Management, 20(2), 195–207.

CHAPTER 4

When Hard Skills Are Not Enough: Behavioral Competencies of Data Scientists and Data Analysts

Abstract  This chapter contributes to the current debate on how to overcome the skill shortage that characterizes the demand of big data professions in the labor market. If the technical competencies expected for these responsibilities have been defined, their behavioral skills are still under-­ explored. The chapter addresses this void through an exploratory study providing empirical evidence collected through the Behavioral Event Interview (BEI) technique. Drawing on an Italian sample of data scientists and data analysts, the study provides a description of the competency portfolio manifested by the two professional roles. The results show that both data scientists and data analysts manifest a wide repertoire of behavioral competencies that are needed to attain successful performance and to face the challenges of the digital transformation. Keywords  Behavioral competencies • Competency-model • Data analyst • Data scientist What makes analytical organizations so interesting, in our view, is the needed combination of human and computational perspectives. Analytical decision-making is at the intersection of individual and organizational capabilities. (Davenport et al. 2010: 17)

© The Author(s) 2020 S. Bonesso et al., Behavioral Competencies of Digital Professionals, https://doi.org/10.1007/978-3-030-33578-6_4

63

64 

S. BONESSO ET AL.

4.1   Emotional Intelligence and Behavioral Competencies of Big Data Professionals With a master’s degree in computer engineering and expertise in real time analysis of sensor data, when Leonardo started to work in the Research & Development Department of his current company he was involved in a project that already had been started by two other colleagues who left the firm in the same period that he had arrived. He was in charge of finalizing a project that required the generation of algorithms and data analysis of a telemetry system. After a very fast inception in which he collected all the information necessary to take charge of the project, he started an in-depth analysis of the work previously done. I found that the system was fallacious, the way in which it was designed prevented the implementation of crucial features of the company’s product. I realized that system was too complex in relation to the actual firm’s needs. I asked myself, what had happened? Why didn’t the others do something else? So, on one hand you have a bit of a feeling, I do not mean resentment, which makes you say what did the other people do before me? I decided to apply another solution to a signal at the analysis and algorithm level, and I found that it generated better outcomes. I brought this solution to mind after having read a paper on dynamics and physical models. I thought to keep the model much simpler from the physics side and to add some changes in terms of signals that had been not considered in the model yet. A thing that is very important in data analysis is the capacity to visually communicate the results, trying to include all of the relevant information in a visual outcome that helps the audience to understand what is happening. For this reason, during an internal weekly meeting I decided to present to the technical team what I was designing and the expected outcomes. I highlighted the coherence of the data and its reliability, pointing out that the solution I proposed was easier to test and to develop. The team was positively impressed by my presentation and the colleagues asked me to explain to the other non-­ technical departments (marketing and design) what this new solution will generate.

What can we learn from this story? Leonardo is a technically prepared data scientist, but what enabled him to achieve a higher performance for his company was his capacity to combine his technical expertise with a repertoire of emotional, social, and cognitive competencies. Specifically, he manifested the capacity to conduct an accurate examination of the ­situation and describe the nature of the problem (diagnostic thinking).

4  WHEN HARD SKILLS ARE NOT ENOUGH: BEHAVIORAL COMPETENCIES… 

65

Then he asked himself questions to challenge the actual model (questioning). He recognized technical similarities between the paper that he read and the problem that he was addressing (pattern recognition). He started to elaborate and test alternatives (experimenting) and demonstrated a concern for increasing the efficiency of action (efficiency orientation). Also, Leonardo relied on a set of social competencies to better understand the actual needs of the company (customer focus) and to communicate effectively his solution to the organization in order to gain the necessary support to implement the solution proposed (persuasion). As discussed in the previous chapter, behavioral competencies have been found to be a critical component in predicting workplace performance across a variety of settings and for different job profiles (Boyatzis 1982, 2006, 2009; Boyatzis and Ratti 2009; Boyatzis and Sala 2004; Hopkins et  al. 2015; McClelland 1973, 1998; Spencer 2001; Spencer et al. 2008; Spencer and Spencer 1993; Ryan et al. 2009). Contemporary research acknowledges that behavioral competencies are as essential as technical skills (Boyatzis 2008; Goleman 1998), because they account for predicting superior individual and team performance (Boyatzis 2009; Druskat et  al. 2013; O’Boyle Jr. et  al. 2011; Stubbs Koman and Wolff 2008). If emotional intelligence is taken for granted, what about its application in a digitalized organizational environment? In particular, what are the behavioral competencies needed by big data professionals who contribute to the generation and manipulation of a large volume of data in our organizations on a daily basis? Despite the increasing effort, mainly from scholars, practitioners and policy makers, to define the characteristics of data workers, a standard and precise definition of the different roles has not yet been reached, as discussed in Chap. 2. If the activities and responsibilities of data analytics’ roles are still in a grey area, their behavioral competencies look even more unclear. The empirical evidence is limited and confusing. As an example, Joseph et al. (2010) suggested that data analytics’ roles need to possess a practical intelligence, which is made up of a set of skills (managerial, intrapersonal, and interpersonal). Costa and Santos (2017) suggested a conceptual model that includes competencies such as business acumen, communication, entrepreneurship, curiosity, and interdisciplinary orientation. Davenport and Patil (2012) added some cognitive competencies like associative thinking to the discussion. This is considered to be an essential ability because it helps data scientists to find a pattern from diverse formats

66 

S. BONESSO ET AL.

of data. In addition, creativity has been found to be a key characteristic because it helps to organize data in visual representations (Harris and Mehrotra 2014; Davenport and Patil 2012). Despite these fragmented insights that attempt to provide a more complete description of data professionals by including these soft skills, a clear comprehension of these roles is still missing. This chapter offers an in-depth description of behavioral competencies of two big data professionals – data scientists and data analysts – in order to provide more clarity about the soft skills needed by these two relevant roles. This study was conducted in the Italian context, where in 2018 the analytics market reached a value of 1393 billion euros with a growth rate of 26 percent (Vercellis 2018). According to the observations of Big Data Analytics & Business Intelligence based in Politecnico of Milan, 56 percent of companies employed data analysts, 46 percent data scientists, and 42 percent data engineers (Vercellis 2018). Organizations are increasingly embracing a data driven culture in order to be successful. However, the majority of the Italian companies are still anchored to a traditional model with underdeveloped analytical skills that, if present, are mostly represented by data analysts or business analysts and by a restricted group of data scientists (Di Deo 2019). In this study, data scientists and data analysts operate in Northern Italy and in different industries. This geographical area is particularly suitable for this research because an impressive digital transformation has involved the majority of its companies, which has required them to hire people in analytical roles (Istat 2018). Different from the studies considered in Chap. 2, which derived competencies from content analysis of job advertisements or from focus groups, this study identifies the competencies by using a consolidated technique, the Behavioral Event Interview (BEI), which allows one to detect when a competency has actually been enacted by an individual who obtains a result (Boyatzis 1982; Gorman et al. 2017; Spencer and Spencer 1993). The competency framework shown in Chap. 3 is adopted to analyze the behavioral competency profile for both of these two roles.

4.2   Data Scientists and Data Analysts in the Italian Context: An Empirical Study This section provides a description of the different phases of the empirical research conducted in the Italian context that involved a sample of data scientists and data analysts.

4  WHEN HARD SKILLS ARE NOT ENOUGH: BEHAVIORAL COMPETENCIES… 

67

The first step was the definition of the sample. Participants were selected by searching their job title on professional networks like LinkedIn and through snowball sampling, that is by asking each selected participant to identify other potential subjects to involve. Therefore, the following conditions were adopted to include a participant in the sample: (i) documented experience in the data analytics area, (ii) active in gathering and analyzing data on a daily basis; (iii) experience in delivering reports for strategic analysis and business choices; (iv) development of statistical analyses on large data sets to define trends and derive business insights used by organizations to set future goals. Drawing on the characteristics of both two profiles described in Chap. 2, we developed a list of criteria to set the boundaries of data scientists and data analysts and to distinguish them from other big data professionals. A check list was then used to select the participants and exclude the ones who did not fit with these criteria. A database of potential interviewees was created and used to contact them and ask their voluntary participation. Twenty-four professionals accepted the invitation to participate in the study, specifically 11 data scientists and 13 data analysts. In terms of sample characteristics, we obtained an heterogenous sample. First, participants had different seniority. On average, they worked for their current organization for eight years with a minimum of two months and a maximum of twenty-five years. Their role seniority was on average 6  years with a minimum of 5  months to a maximum of 20  years. They were employed in 20 different organizations that belong to a variety of sectors: IT and consulting (five companies); software production (five companies); business intelligence and training (three companies); web and marketing (two companies); retail (one company); human resources and ICT (one company); R&D (one company); consulting (one company); and insurance (one company). The average revenue of these companies in 2017 was about 3 million euros. Concerning the number of employees, eleven firms had fewer than 50 employees, two had from 50 to 149 employees, one had from 150 to 249; and six had more than 250 employees. This variety was also reflected at the individual level. The average age of respondents was about 38 years old (with a minimum of 24 and a maximum of 56 years) and 16 percent of the sample was represented by women. Regarding their education, 10 percent obtained a PhD, 50 percent received a master’s degree and 35 percent a bachelor’s degree; only 5 percent did not hold any academic degree. The educational background of the respondents was heterogeneous, encompassing statistics, ­mathematics,

68 

S. BONESSO ET AL.

management and economics, computer science, physics, biology, cognitive science, psychology, and communication. It is common to have heterogeneous educational background: “some of the best and brightest data scientists are PhDs in esoteric fields like ecology and systems biology” (Davenport and Patil 2012: 6). In addition, data scientists and analysts of our sample worked in different organizational departments like R&D, business intelligence, administration and control, or IT. In contrast to extant studies that mapped the soft skills of data analytics’ roles and that mainly used secondary sources such as survey questionnaires (Aasheim et al. 2012) or content analysis of job announcements (Shirani 2016; De Mauro et al. 2016), we adopted a consolidated methodology, namely the Behavioral Event Interview (BEI), that does not rely on perceptions of the main important competencies for the professional roles under investigation, but it detects the actual competencies enacted by the role holders in their work environment (Boyatzis 2009; Emmerling and Boyatzis 2012; Scapolan et  al. 2017). Data was collected between May and June 2018 through semi-structured interviews. The interview protocol comprised an introductory section in which respondents were asked to provide a detailed description of their daily activities and main responsibilities in order to obtain a more in depth understanding of their job characteristics. The core of the interview was concentrated on the BEI protocol (Boyatzis 1998; McClelland 1973) which was developed from the critical incident interview technique (Flanagan 1954). BEI was used to collect actual critical events or incidents in which each respondent was effective in performing his or her tasks in the company. In particular, each respondent was asked to recall five critical situations that had occurred during the last 12 months and to describe the context, the people involved, what he or she thought, felt, or said, and how he or she behaved. In order to avoid potential bias in detecting successful episodes, a description of the final outcome of a certain event was constantly asked and checked by the interviewer. For instance, positive episodes were about winning a new client, solving an issue in data analytics, or the development of new software. On the opposite side, minor issues or revisions in their daily activities were excluded from the analysis. The interviewers detected the intent of the specific behaviors guiding the interviewee though a set of probing questions (Boyatzis 2009). By gathering actions, reactions, and decisions, the interviews on the behavioral episodes made it possible to discern the main behaviors and conse-

4  WHEN HARD SKILLS ARE NOT ENOUGH: BEHAVIORAL COMPETENCIES… 

69

quently the related competencies expressed by the interviewees while performing their job (Spencer and Spencer 1993). In total, we collected 120 episodes. Each interview, which lasted on average 1.5  hours, was recorded and transcribed for subsequent coding. The first step of the analysis was aimed at creating a comprehensive description of the respondents’ duties and responsibilities. We integrated our own understanding of the respondents’ activities within the framework already established in the literature (De Mauro et al. 2016; Harris and Mehrotra 2014). The second step of analysis concentrated on detecting the main behavioral competencies expressed by respondents and classifying them into a portfolio of competencies. To identify the behavioral competencies manifested by the interviewees, we adopted a competency codebook that included 33 competencies clustered into six areas: awareness, action, social, cognitive, exploratory, and organizational action competencies which are illustrated in Chap. 3 (Fig. 4.1). Each competency was measured by analyzing its frequency (Ryan et al. 2009), which was computed by the recurrence with which one specific competency was activated.

4.3   The Behavioral Competency Profiles of Data Scientists and Data Analysts This section aims to discuss the main similarities and differences that emerged from the analysis of the behavioral competencies that were most frequently activated by the two professional roles in their work environment. Considering the six areas of competencies (Fig. 4.1), the profile of the overall sample is illustrated as follows: action (33 percent), social (21 percent), awareness (20 percent), and cognitive (15 percent), followed by exploratory (7 percent) and organizational action competencies (4 percent). A more detailed comparison between the two roles for each of the thirty-three competencies is provided in Fig. 4.2. As reported in Figs. 4.3 and 4.4 and considering the first ten most frequently manifested competencies by the two distinct roles, data scientists and data analysts show the same competencies even though there are some differences in terms of the frequency of activation. These competencies are the following: • self-awareness, empathy, and organizational awareness for the awareness area;

70 

S. BONESSO ET AL.

Fig. 4.1  Behavioral competencies framework: the competency hexagon

• achievement orientation, initiative, and efficiency orientation for the action area; • customer focus and persuasion for the social area; • diagnostic thinking and pattern recognition for the cognitive area. The following sections will provide more evidence and concrete examples on how data scientists and data analysts activate behavioral competencies in performing their job.

4.4   Action Competencies A young data scientist who recently joined one of the largest public Italian companies recalled an episode in which his manager asked him to develop new software: “I decided to anticipate the timeline of the project, doing a

4  WHEN HARD SKILLS ARE NOT ENOUGH: BEHAVIORAL COMPETENCIES… 

71

Fig. 4.2  Competencies of data scientists and data analysts according to the frequency of manifestation

72 

S. BONESSO ET AL.

Fig. 4.3  Competencies of data scientists according to their frequency of manifestation

migration of the analysis from R to Python. My colleagues were thinking about that, but they did not consider it as a priority. I instead thought that, since we had to do the analysis, it was meaningful to start immediately with something that was as close as possible to industrialization”. The most frequent competency area manifested by the sample was Action, which collects competencies that express how individuals realize ideas, plans, and solutions, then work methodically and with initiative. In particular, respondents show high achievement orientation, initiative, and efficiency orientation.

4  WHEN HARD SKILLS ARE NOT ENOUGH: BEHAVIORAL COMPETENCIES… 

73

Fig. 4.4  Competencies of data analysts according to their frequency of manifestation

74 

S. BONESSO ET AL.

An expert data scientist in a large software production company revealed that during the long and complex implementation of a software program that aggregated information about the financial exposure of bank accountants, he first set a tight meeting schedule, sharing the calendar with his staff and all the people involved in the project [Achievement Orientation]. Also, by monitoring each step and driving the situation towards a specific direction: “I tried to convey the situation towards an intermediate solution for different reasons. The first reason was that I know my company and I know that certain things must be dealt with, let’s say, gradually. The second reason was that, in my opinion, as a cost-benefit ratio it was much more advantageous [Efficiency Orientation] and allowed colleagues to continue to consolidate their experiences”. A young business analyst in a large company explained how the team used a data quality architecture developed before his arrival in the organization. After a few months, he realized the inaccuracy of the system and the inefficiency that it was causing to the entire organization. Despite his junior position, he describes “I went to the manager and I offered him the solution: I could work on that project for a month and repeat everything over and over again, by doing at the same time the project I was hired for and eventually assessing whether my solution was better than the old one [Initiative].” His intuition was correct, and the manager decided to change the system in order to avoid a further waste of time and resources. He demonstrated the capacity to take action first, not reacting to events or being forced by them. Initiative and willingness to achieve challenging goals are usually mentioned in the skillset of data scientists (as discussed in Chap. 2). The narratives collected demonstrated that these competences also enabled data scientists to attain effective outcomes. With specific regards to achievement orientation, the two profiles show “to be confident in their ability to learn and to adopt a growth mindset, setting them up in a path to greater performance” (Truninger et  al. 2018: 4). Moreover, different from the literature that defined the skillset of big data professional profiles, efficiency orientation emerged in both profiles as the tenth most frequently activated competency. This can be associated to a pragmatic approach that is typical of project-based jobs like those of data scientists and data analysts who are spurred to provide quick answers to external clients and internal business teams, with a continuous concern for assessing the costs and benefits of the proposed solutions.

4  WHEN HARD SKILLS ARE NOT ENOUGH: BEHAVIORAL COMPETENCIES… 

75

A competency that differentiates the skill profiles of data scientists is flexibility. A young data scientist with a doctoral degree in physics described a difficult project in which he was asked to provide several months’ forecasts, whereas he could return only one-week predictions. After several failing trails, he attempted to consider a different perspective. “At that point, I decided to start over with something else in the sense that I changed my model to manage data. Instead of inserting data for the last N days, I thought to insert data adding information about each day, year, month, and date (whether people were on vacation or if they were unexpected people) and I assigned to each day the variables I had or the variables I could create. A different world opened to me and other different potential models.” This narrative highlights the ability of this data scientist to adapt himself by modifying his behavior in case of required changes or unexpected circumstances.

4.5   Social Competencies A young data scientist working for one of the biggest retail companies described how he explained the entire process to implement a software program to a difficult client and how he put attention on the customer’s needs. According to the data scientist, it was crucial to use visual representations and to rely on “different techniques to present the information you are analyzing and what you are looking for in the data and packaging the results aesthetically by supporting the data with visual messages for the client. In this way, it prevents the client from getting bored while he is waiting for the results [Customer Focus].” Social competencies include capabilities that allow a positive interaction with other people and help to work with others effectively. In particular, data scientists and analysts manifest high customer focus and persuasion. A senior data scientist shared the challenges encountered while negotiating with an important client: “when you go to meet a client and the client thought to spend 1,000 and you ask for 10,000, you need to explain why. All of the ability is there if you can make it clear about the technological complexity and all of the study needed to develop a certain product and the added value the technology will give them. So, if you can explain all these aspects, then you can convince the client and find a compromise, and maybe go from 10 to 8; but for sure it is not going to be 2 [Persuasion]”. Within the social competencies, leadership is the third most manifested competency. A data analyst who was also a project manager of a

76 

S. BONESSO ET AL.

small start-up company, described an episode in which some competing stakeholders were involved: “During the implementation of an application that tracked the movements of the animals, I was responsible to coordinate the team development, which is part of the company. I took care to give to a junior profile the right instructions about what to install inside the platform and how it worked in terms of usability. The fundamental problem was that the thing was managed but no one was making decisions. Therefore, I announced: “Ok, someone needs to make decisions here, we need to make decisions, we need to be clear, we need to be categoric, and I will take full responsibility for making these decisions.” I asked for a mandate to make these decisions and I systematically began to arrange all of the different aspects that were in front of me [Leadership]”. What differentiates data scientists from data analysts for this specific area of competencies are teamwork and networking. A data scientist who works for a company that develops video games based in London recalled an episode in which, working in synergy with the other members of her team and combining their individual expertise, they could solve a computational issue: “Everyone in the group has his own technique, but we were able to understand an issue by working together that we could not understand otherwise. One member of the group recalled an article and she was able to tell us how things were done, but I was more interested in understanding why this was done and in understanding the mathematical reasons. Another member of the group was much more interested in “Eventually, why does it work in this way?” We have three very different personalities, but we started talking and everyone brought his own perspective and eventually we knew something new [Teamwork]”. A senior data scientist with a leading role in a big Italian company described how she implemented a new instrument for an important client whose collaboration was difficult. She recalled how she managed to bind a good relationship with the client and, at the same time, to implement the new complex instrument and sell it successfully to the client. In particular, the implementation stage required more effort than expected, and she used external resources to be acquainted with the instrument: “I had a colleague from another company who already knew this tool. We ­organized a couple of meetings with him and he gave us some information. I asked him how he would do some things” [Networking].

4  WHEN HARD SKILLS ARE NOT ENOUGH: BEHAVIORAL COMPETENCIES… 

77

4.6   Awareness Competencies A data analyst working for a web marketing company recalled one of her most difficult events of managing a client who “lives in a sort of technological illiteracy. After several discussions with the client, I mediated with the client by getting into the client’s mindset so that I could adapt my analysis and my research and guide the other person to acquire a new awareness. I needed patience, empathy, and control of impulsiveness”. The awareness competencies seem to play a crucial role for both profiles. Specifically, emotional self-awareness, namely the capacity to understand one’s own emotions and their effects but also to know one’s inner values, strengths and limits, emerged respectively as the first and second most manifested competencies for data scientist and data analysts (see Figs. 4.3 and 4.4). There are several reasons why this competency is recognized as extremely important to manifest. First, it allows us to understand the impact of our emotions on our decisions and actions. This makes it possible to analyze and correct our decision making processes. Second, being aware of our strengths and weaknesses is the first and necessary step of every effective individual change process that also needs to consider the real starting point in terms of personal characteristics. In addition, the awareness of our individual limits and points of mastery allows us to avoid risky situations and to face issues with an appropriate self-confidence. Finally, understanding ourselves also helps us to manage ourselves and to understand others. Consequently, it is a fundamental component of the manifestation and of the development of other groups of competencies. A young data scientist who was implementing a digital agenda for one of the largest computer companies in the Italian Chamber of Commerce described the first stages of the project and the relationship with different stakeholders involved. In particular, he observed the following: “I started writing the first points and then I tried to leave more room for my colleagues in the subsequent steps, because I knew they were better or more experienced than me” [Self-Awareness]. Another data scientist working for an incubator explained: “In this circumstance I said to my manager that this customer was crazy, that I didn’t want to work with him anymore and that if you came to me it’s because than you are sure that you will have the project finalized within the ­deadline. And he told me that’s true. We have this compromise by which I can tell him all I want, and he knows that I will reach the result” [Self-Awareness].

78 

S. BONESSO ET AL.

Besides self-awareness, the profile of both roles encompasses the capacity to sense others’ emotions (empathy) and the capacity to recognize the values and the culture of an organization but also to understand its informal processes and structure, the unspoken rules and the key power relationships (organizational awareness). During an organizational change process which also involved the transition to a new software system, one of the interviewees described the following: “That company decided to adopt a new software, SAS. The majority of the employees were not familiar with this software and they looked scared; they were resistant to the change. How did I understand that they were skeptical? Well, it was from their attitudes, their behaviors, how they talked to give you some information, how they said uhm” [Empathy]. If data scientists and analysts are good at reading others’ signals, they also manifest the ability to interpret the organizational ecosystem. This was described by a senior data scientist who described a complex negotiation process composed of several “meetings with various subordinates, in order to find the functional and technical needs that were impacted the most. I was looking for the people in the company who could support me and had the power and the skills to do so. I noticed that during the meetings that all of the participants looked at one woman when some uncertain issues emerged and waited for her opinion. I understood that she was informally recognized as the leader and hence that all her doubts had to be dispelled” [Organizational Awareness].

4.7   Cognitive Competencies “I opened the program and I checked it. Controlling programs also means doing a mental process and noticing where there could be a duplication. And the duplication was there. I had the aggregate data and I went backwards, so I looked at the step before the one that generated that data. Then, if the data was duplicated, I had to intervene. If the data was not yet duplicated but the numbers did not convince me, I would have to go back until I got to the input and then to the data that the user gave me in order to see if the data were actually correct. For example, if a file has 10 records three months before and 10 the month after, it has 20 records. This means that something was wrong at that point, which means that I have to check the historical series too [Diagnostic Thinking].” This is how a data analyst described the step by step cognitive process through which he was able to

4  WHEN HARD SKILLS ARE NOT ENOUGH: BEHAVIORAL COMPETENCIES… 

79

detect an error. Even though diagnostic thinking emerges among the ten most activated competencies of both professional roles, it mainly seems to characterize the profile of data analysts. Cognitive competencies are the ability to use information and analyze it effectively to interpret phenomena or situations and to analyze problems with a scientific mindset. Another data analyst recalled a critical event in which she realized that a consistent amount of money was missing from the database: “After several rounds of checks, I realized that money was spread through three different projects. I found the value that did not sound right, and it was quite simple to find it. First, I checked everything because it is easy to make mistakes (I checked twice). Then when I found that the results were correct, I checked every project that I had recorded as critic with the team leaders. So, I did the same for all the projects that were considered as priority and critical. At the end when I found it, I analyzed it again; I analyzed all the indicators of the project. I realized that the project could go out of control, and then I reported it to my boss” [Diagnostic Thinking]. Likewise, for diagnostic thinking, another cognitive competency, pattern recognition is manifested more by data analysts than by data scientists. A data analyst was required to collect data about the behavior of a large sample covering a long period of time. To do so, he adopted a system by which he was able to find an underlying ratio: “I analyzed several keywords that could be linked to the industry, reference keywords. I associated the keywords to various markets linked to the users and therefore to potential markets from which users could arrive. I then matched the information of the various markets and the various keywords, I extrapolated graphs, Excel data, and a document that I eventually presented during an event” [Pattern Recognition]. In this area of competencies, there are two competencies most frequently activated by data scientists in comparison to data analysts: systems thinking and lateral thinking. Systems thinking is the ability to break down complex problems and understand the cause–effect relationships. A data analyst in charge of analyzing several consumers’ data, was asked by the company’s director to understand how a specific parameter changed based on specific characteristics of the clients. He explained “obviously I cannot look at just one customer’s variable and see the impact on the parameter. I have to consider different combinations of these variables to see how they contribute to determine the parameter” [Systems Thinking].

80 

S. BONESSO ET AL.

Lateral thinking is the ability to think outside the box when looking at problems, and it is particularly crucial for generating non-conventional solutions. A senior data scientist explained: “In marketing there is the concept of a funnel, a funnel of people who are contacted at the beginning and gradually there is a skimming up until arriving at the bottom of the funnel, the purchase moment. A client asked: Every week I contact people who are at different stages of this funnel and every week I would like to understand how many of them are going down through the funnel. I then turned around the problem and approached what was the final solution. Let’s see the problem in this way. Every week we understand how many of those who bought were people who were in the funnel, and you had contacted them for the first time 7 days before, or if they were very slow people, you had contacted them more than two months before. So, we will do a histogram every week with a slice that are the hot ones and a slice that are the blue ones, which are the coldest ones. In this way we can see how the sales change over the weeks” [Lateral Thinking].

4.8   Exploratory Competencies A data scientist narrated an episode in which he had to select the data source that would allow him to get access to a high volume of new external data. He critically started to ask to himself what he could have done with such data. “I evaluated the proposed data sources and I created a list of technical questions to address to the providers in order to have more detailed information about the features of this data, such as how frequently the data is updated or which characteristics it includes. In this way, I had a new approach to the problem that highlighted the need to get further information” [Questioning]. “Questioning” is the capability to challenge the current situation, to attempt to understand the nature of the problem and use a different approach to it. As highlighted in Chap. 2 (Davenport and Patil 2012: 5), the ability to scan the environment with a curios attitude allows data scientists to see new opportunities, “a desire to go beneath the surface of a problem and to find the questions at its heart.” In so doing, they create value for organizations, and they are a crucial asset to generate innovation. Despite the fact that exploratory competencies (observing, questioning, and experimenting) are more frequently demonstrated by data scientists than by data analysts, confirming the job requirements described in the academic and

4  WHEN HARD SKILLS ARE NOT ENOUGH: BEHAVIORAL COMPETENCIES… 

81

practitioners’ communities, these competencies have been manifested in our sample less than expected. New ideas for the generation of new solutions are collected by data scientists keeping themselves updated on the evolution of the big data field, especially by reading articles and books or consulting experts on specific technologies and techniques. As a data scientist narrated: “many books have been published on this issue, you have to read and assimilate them and then generate hypotheses. While I read a book on big data, I think about my clients, and I start to imagine for whom I can adapt the technique that I am learning” [Observing]. To explore new solutions and try new things by adopting a trial and error approach, all behaviors are related to the “experimenting” competency. “I had a series of numbers distributed over time, and I was wondering whether these numbers followed a seasonal trajectory as they were involved in a parking lot project. Maybe people preferred to park on Wednesdays and leave on Sundays. It was a classic case, a typical case from a textbook. Then, I tried to apply this idea to the data, and I noticed that it was a great idea [Experimenting], but I realized I could not go further to make forecasts because historical series usually have a problem; unless you have a very long history, the inaccuracy of this data tends to explode in the long run.” This data analyst, in addition to showing pattern recognition by discovering a recurring users’ behavior, also stretched data iteratively and adopted an experimental approach.

4.9   Organizational Action Competencies A data analyst describes how he improved an e-commerce platform and achieved a remarkable growth of turnover. He first monitored the users’ behavior of the platform and consequently adapted its usability accordingly. “Using the database of Italian companies and the ATECO code [a classification of economic activities adopted by the Italian National Statistical Institute], therefore using the sector of the companies belonging to the publishing sector, I extrapolated a list of companies according to turnover, geographical location, and number of employees. Why geographical location and number of employees? Because, actually, the number of employees outlines the size of the publishing house and I was interested in discovering the smaller publishing houses, because the smaller ones are those that print the most interesting books. And then the geographical location, because obviously if you are a publisher based in Milan

82 

S. BONESSO ET AL.

with the headquarters in Rome, which are commercially strategic points, it means that you are hardly a small company that does small activities. In fact, I found several interesting publishing houses in other cities such as M. and C., smaller places on the map but which would have allowed me to obtain interesting results for this type of activity [Strategic Thinking]”. The organizational action area of competencies, which encompasses strategic and visionary thinking, opportunity recognition, commitment toward the group and integrity, includes capabilities that has been expressed by respondents with a frequency of manifestation of about 6 percent. Data scientists and data analysts manifested only one of these competencies at the same frequency level, that is “Commitment Towards the Group”. This is the ability to sacrifice personal targets and needs in favor of the team or of the organization. A young data scientist working for a small company based in London narrated “I worked very hard, and during two weekends I was at work from morning to night because I wanted to finish the project and because that was the turning point for my group as well. If I had missed it, it also would have been bad for the data group” [Commitment Toward the Group]. Within this area, two competencies were demonstrated at a higher frequency of manifestation by data analysts, namely strategic thinking, which is the ability to understand the competitive environment of the company, and opportunity recognition, which is the ability to perceive new business possibilities emerging from the environment. A data analyst operating in a web marketing agency explained how she had to address the need of a high demanding client operating in the food & beverage industry. “The client demonstrated a very negative approach toward e-commerce; he needed to regain confidence on it. The sector is characterized by a high competitiveness; the food & beverage industry is blowing up in the web. The solution I proposed was to work in a market niche such as PDO-­ Protected Designation of Origin products, focusing on the added value offered by the client’s offerings [Opportunity Recognition]. I started an analysis of the food products sold on line considering their geolocation. The analysis was conducted in order to position the products that were more appreciated by the final customers in different geographical markets. In other words, I tried to understand the best positioning of the products for each foreign geographical market [Strategic Thinking]”. The presence of these competencies in the repertoire of data analysts can be explained by the fact that this professional role in some organizational contexts works more closely with the sales department. They

4  WHEN HARD SKILLS ARE NOT ENOUGH: BEHAVIORAL COMPETENCIES… 

83

f­requently meet with clients in order to address their requests that rely on analytics techniques. Thus, strategic thinking and opportunity recognition are often manifested in relation to the businesses of the clients and indirectly to the company’s business. Despite the very low frequency, data scientists were shown to activate slightly more of the remaining two competencies of this area, visionary thinking and integrity. Visionary thinking is the capacity to create a vivid image of a desired future. Managers and leaders who share a vision with employees and make it explicit affect organizational engagement and productivity (Goleman et  al. 2002). The structure of Italian companies is highly centralized and the power is mainly in the hands of the top management; this partially explains the shortage of this competency. The recent drive to embrace digitalization among Italian companies is a further explanation. Indeed, despite the common attempt to adopt a data-driven culture, the top management is still struggling to abandon their traditional role and delegate the decision-making process to other roles such as data scientists or data analysts. Integrity is considered a key trait of professionals who manage a large volume of data on a daily basis; for this reason, it is often taken for granted. This is why it was not manifested explicitly as the specific determinant of role effectiveness, but it was considered as an underlying behavior strictly integrated in the daily activities. If analytics and big data poses a number of questions for policy makers about data protection, privacy, and surveillance, the same issues are expected to receive similar attention from the individuals who work in the field (big data professionals have manipulated and used data pertaining to information about clients, internet users, and retailers, excluding data about internal movement of employees, surveillance, and human resources. However, the recent EU General Data Protection Regulation (GDPR) introduced in the past May of 2019 has impacted both public and private organizations. As the ethical issue in data analytics is becoming even more critical, behavior related to the integrity competency are expected to further increase in terms of frequency of activation.

4.10   The Competency Profile of Data Scientists and Data Analysts: Concluding Remarks The interviews offered a multifaceted picture of the two analytics’ roles. They further enriched the soft skills profile defined by the academic and practitioner communities discussed in Chap. 2. To obtain effective results

84 

S. BONESSO ET AL.

in their job, data scientists and data analysts rely on a broader repertoire of behavioral competencies. They are not confined exclusively to cognitive and social behaviors as job postings in the labor market continue to highlight. The competency hexagon with its thirty-three behavioral competencies has provided a fine-grained framework to understand the complexity of these two roles. Considering the specific area of competencies, the narratives collected directly by the real-life experiences of the role holders pointed out the importance of the action, social and awareness competencies. They not only identified the main commonalties but also the distinctive features of the two profiles. Even though the narratives reported in this chapter exemplified the manifestation of single competencies, data scientists and data analysts simultaneously activated more than one competency in each specific situation in order to achieve a positive outcome. This means that different areas of competencies interact with each other in combinations or bundles. From the interviews, it also emerged that some competencies considered crucial for the big data analytics’ roles, especially in the area of cognitive, exploratory and organizational action competencies, were activated less frequently than expected. Even though these competencies present a lower frequency of manifestation in comparison to others, it does not mean that their impact on the final result is less relevant. In order to attain positive outcomes, data scientists and data analysts are required to activate the aforementioned competencies coherently with the characteristics of the situations that they face in the workplace. In its Data Science Salary Survey, O’Reilly Media (King and Magoulas 2016; Suda 2018) asked data scientists about how much time they spent on specific analytics tasks during their workday. The survey revealed that these roles do not work in silos but are continually exposed to social interaction. Roughly 42 percent of the respondents indicated that they spend between 4 and 8 hours a week in meetings and 6 percent of them spend more than half of their work week in them. Moreover, nearly half of them spent 1–4 hours per week presenting analysis, with 6 percent spending four hours or more per day sharing findings with management. This explains the higher frequency for manifestation of the awareness competencies like empathy and organizational awareness but also for social competencies such as persuasion and teamwork. However, most of their time is devoted to basic exploratory data analysis and data cleaning. These activities are also labeled with terms like data munging or wrangling rather than modelling data. As data

4  WHEN HARD SKILLS ARE NOT ENOUGH: BEHAVIORAL COMPETENCIES… 

85

­ reparation accounts for a high percentage of the work of data scientists, p less time remains for other relevant tasks such as the following: • identifying the exact business problem and then converting it into an analytic problem that can be solved with data; • suggesting machine learning and predictive modelling that explains discrepancies and helps in understanding what went wrong and where, if the previous models do not deliver the outcomes in correlation with the business requirements; • framing open-ended questions on the business; • examining data from a variety of angles to determine hidden weaknesses, trends, and opportunities. From this evidence, it has emerged that the boundaries of the job of these professional roles seem to need further specification in order to redistribute in a more effective way their time toward more highly valued activities that can lead to the actions of specific competencies such as lateral thinking, questioning, observing, experimenting, strategic thinking, and opportunity recognition. Even though the work situations in which data scientists and data analysts can deploy the aforementioned competencies are less frequent than situations in which other areas (such as action or social) can be manifested, when exploratory and organizational action competencies are activated, they may generate even a higher impact in comparison to other behaviors. Consequently, not all competencies are required to be activated at the same level, but they need to be present in the behavioral repertoire of data scientists and data analysts, so that they can manifest them when they are required. As the companies adopt a more data-driven culture and enlarge and empower their data science team, they will be able to exploit the full potential of their data scientists and data analysts.

References Aasheim, C., Shropshire, J., Li, L., & Kadlec, C. (2012). Knowledge and skill requirements for entry-level IT workers: A longitudinal study. Journal of Information Systems Education, 23, 193–204. Boyatzis, R. (1982). The competent manager: A model for effective performance. New York: Wiley Interscience. Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis and code development. Thousand Oaks: Sage.

86 

S. BONESSO ET AL.

Boyatzis, R. E. (2006). An overview of intentional change from a complexity perspective. Journal of Management Development, 25(7), 607–623. Boyatzis, R. E. (2008). Competencies in the 21st century. Journal of Management Development, 27(1), 5–12. Boyatzis, R. E. (2009). Competencies as a behavioral approach to emotional intelligence. Journal of Management Development, 28(9), 749–770. Boyatzis, R. E., & Ratti, F. (2009). Emotional, social and cognitive intelligence competencies distinguishing effective Italian managers and leaders in a private company and cooperatives. Journal of Management Development, 28(9), 821–838. Boyatzis, R. E., & Sala, F. (2004). Assessing emotional intelligence competencies. In G. Geher (Ed.), The Measurement of Emotional Intelligence (pp. 147–180). Novas: Hauppauge, Science Publishers. Costa, C., & Santos, M. Y. (2017). The data scientist profile and its representativeness in the European e-competency framework and the skills framework for the information age. International Journal of Information Management, 37(6), 726–734. Davenport, T. H., & Patil, D. J. (2012). Data scientist: The sexiest job of the 21st century. Harvard Business Review, 90(October 2012), 70–76. Davenport, T. H., Harris, J. G., & Morison, R. (2010). Analytics at work: Smarter decisions, better results. Boston, MA: Harvard Business Review Press. De Mauro, A., Greco, M.  Grimaldi, M., and Nobili, G. (2016, June 15–17). Beyond data scientists: A review of big data skills and job families. International Forum on Knowledge Asset Dynamics 11th, Proceedings of IFKAD, 2016, Towards a New Architecture of Knowledge: Big Data Culture and Creativity, Dresden (1844–1857). Di Deo, I. (2019). Data Analyst: chi è, cosa fa e quanto guadagna un analista dei dati. Retrieved July 1, 2019, https://blog.osservatori.net/it_it/data-analystcosa-fa-stipendio Druskat, V. U., Mount, G., & Sala, F. (2013). Linking emotional intelligence and performance at work: Current research evidence with individuals and groups. New York: Psychology Press. Emmerling, R.  J., & Boyatzis, R.  E. (2012). Emotional and social intelligence competencies: Cross cultural implications. Cross Cultural Management: An International Journal, 19(1), 4–18. Flanagan, J.  C. (1954). The critical incident technique. Psychological Bulletin, 51(4), 327–358. Goleman, D. (1998). Working with emotional intelligence. New  York: Bantam Books. Goleman, D., Boyatzis, R., & McKee, A. (2002). Primal leadership: Realizing the power of emotional intelligence. Boston, MA: Harvard Business School Press. Gorman, J. C., Dunbar, T. A., Grimm, D., & Gipson, C. L. (2017). Understanding and modeling teams as dynamical systems. Frontiers of Psychology, 8, 1053. Harris, J. G., & Mehrotra, V. (2014). Getting value from your data scientists. MIT Sloan Management Review, 56(1), 15–18.

4  WHEN HARD SKILLS ARE NOT ENOUGH: BEHAVIORAL COMPETENCIES… 

87

Hopkins, M. M., O’Neil, D. A., & Stoller, J. K. (2015). Distinguishing competencies of effective physician leaders. Journal of Management Development, 34(5), 566–584. Istat. (2018). Indicatori Industria 4.0-Fattori di digitalizzazione, Istituto Nazionale di Statistica, http://dati.istat.it. Accessed 1 May 2017. Joseph, D., Ang, S., Chang, R. H. L., & Slaughter, S. (2010). Practical intelligence in IT: Assessing soft skills of IT professionals. Communications of the ACM, 53(2), 149–154. King, J., & Magoulas, R. (2016). 2016 Data science salary survey tools, trends, what pays (and what doesn’t) for data professionals. Sebastopol: O’Reilly Media. McClelland, D. C. (1973). Testing for competency rather than for ‘intelligence’. American Psychologist, 28, 1–14. McClelland, D.  C. (1998). Identifying competencies with behavioural-event interviews. Psychological Science, 9, 331–339. O’Boyle, E. H., Jr., Humphrey, R. H., Pollack, J. M., Hawver, T. H., & Story, P.  A. (2011). The relation between emotional intelligence and job performance: A meta-analysis. Journal of Organizational Behavior, 32, 788–818. Ryan, G., Emmerling, R. J., & Spencer, L. M. (2009). Distinguishing high-performing European executives. Journal of Management Development, 28(9), 859–875. Scapolan, A., Montanari, F., Bonesso, S., Gerli, F., & Mizzau, L. (2017). Behavioural competencies and organizational performance in Italian performing arts. Academia Revista Latinoamericana de Administración, 30(2), 192–214. Shirani, A. (2016). Identifying data science and analytics competencies based on industry demand. Issues in Information Systems, 17(4), 137–144. Spencer, L. M. (2001). The economic value of emotional intelligence competencies and EIC-based HR programs. In C. Cherniss, R. E. Boyatzis, & M. Elias (Eds.), Emotionally intelligent workplace (pp.  45–82). San Francisco: The Jossey-Bass Wiley Company. Spencer, L. M., & Spencer, S. M. (1993). Competency at work. Models for superior performance. New York: John Wiley & Sons. Spencer, L. M., Ryan, G., & Bernhard, U. (2008). Cross-cultural competencies in a major multinational industrial firm. In R.  J. Emmerling, V.  Shanwal, & M. Mandal (Eds.), Emotional intelligence: Theoretical and cultural perspectives (pp. 191–208). New York: Nova Science. Stubbs Koman, E., & Wolff, S. B. (2008). Emotional intelligence competencies in the team and team leader. Journal of Management Development, 27(1), 55–75. Suda, B. (2018). 2017 data science salary survey. Sebastopol: O’Reilly Media. Truninger, M., Fernández-i-Marín, X., Batista-Foguet, J. M., Boyatzis, R. E., & Serlavós, R. (2018). The power of EI competencies over intelligence and individual performance: A task-dependent model. Frontiers of Psychology, 9, 1532. Vercellis, C. (2018). Big Data Analytics in Italia: un mercato in evoluzione. Retrieved July 1, 2019, https://blog.osservatori.net/it_it/mercato-big-dataanalytics-in-italia

CHAPTER 5

Managing Big Data Professionals through a Competency-Based Approach

Abstract  Behavioral competencies are more complex and more difficult to assess and to develop than technical skills. On one hand, companies still struggle to accurately assess behavioral competencies of big data professionals during the recruiting process. On the other hand, higher education institutions are still not adequately equipping students enrolled in data science and analytics degree programs with the soft skills requested by employers. The chapter provides methodological recommendations to design courses for competency development based on the Intentional Change Theory. Also, it offers insights for human resource management specialists to revise the way they search and assess behavioral competencies of big data profiles to improve the effectiveness of the hiring process. Keywords  Competency development • Intentional change theory • Higher education • Human resource management • Data science While hard skills may get a candidate’s foot in the door, it’s soft skills that ultimately open it. (Lydia Liu, Head of HR, Home Credit Consumer Finance Co. Ltd., quoted in LinkedIn, 2019 Global Talent Trends)

© The Author(s) 2020 S. Bonesso et al., Behavioral Competencies of Digital Professionals, https://doi.org/10.1007/978-3-030-33578-6_5

89

90 

S. BONESSO ET AL.

5.1   Behavioral Competencies as a Decisive Factor in Hiring Outstanding Big Data Professionals When Lorenzo became involved in his first data project in his new company, he had just successfully completed his Master’s degree in data science. He also had a Bachelor’s degree in computer science, had served prior internships where he strengthened his coding and programming skills and improved his knowledge of Python and R, and had developed a project in the field of machine learning in collaboration with the university from which he graduated. He was asked to work on raw data from an external source, documenting the data cleaning and the subsequent implementation of an algorithm. However, the outcomes he presented did not meet the company’s needs. How did not Lorenzo live up to the hype? What we have learnt in the previous chapters is that big data profiles require multiple skills, from technical to behavioral competencies, to attain outstanding performance. Lorenzo failed in his job for three main reasons. Firstly, he was not aware that his ultimate mission was to solve a business problem, not simply to analyze data or build a great model. He was too focused on the technology part of his job: he acted as a data geek instead of asking himself questions about how the company would benefit from the data, interacting with the executives to better frame the problem, highlighting the output of the model that stakeholders cared about most, removing technical jargon when explaining the work done to non-­ technical people, simplifying the analytical solution and tailoring it to the company’s needs, changing his approach after the preliminary feedback received, and paying more attention to the replicability of the solution in order to pursue efficiency. In other words, he did not manifest the behavioral competencies considered critical for big data professions like strategic thinking, customer focus, empathy, adaptability, and efficiency orientation, as discussed in Chap. 3. The second reason for the failure is that the recruiter probably did not adequately consider and assess the behavioral competencies of the applicants for this position. This can be attributed to the bias that affects big data professionals, who are often seen only as numerically-minded individuals. Big data roles have not long existed: consequently, companies do not yet have a clear understanding of the responsibilities, skillset, and ultimate contribution to the firm’s performance, especially in those organizational contexts that are only now embarking upon a data-driven culture.

5  MANAGING BIG DATA PROFESSIONALS… 

91

Key findings from the last LinkedIn Global Talent Report (LinkedIn 2019) revealed that 89 percent of the firms interviewed said bad hires typically lack soft skills. Companies seem still to struggle to accurately assess behavioral competencies, and if coding or analytics skills are easier to evaluate during the recruiting process, identifying soft skills seems much harder. This explains why this gap emerged too late. The last reason can be ascribed to the misalignment between graduates’ skills and employer’s expectations in today’s labor market. The skills shortage refers not only to data science and analytics competencies, as discussed in Chap. 2, but also to behavioral competencies. Soft skills have become even more difficult to hire, especially since the rise of automation and artificial intelligence that makes emotional intelligence even more valuable (Beck and Libert 2017; Gustein and Sviokla 2018; McKinsey Global Institute 2018; World Economic Forum 2019). For instance, creativity (QS Intelligence Unit 2019; LinkedIn 2019) – that is one of the distinctive characteristics of data scientists – represents the most in-demand skill at the global level and encompasses the cognitive and the exploration competencies included in the Competency Hexagon (Chaps. 3 and 4), like lateral thinking, pattern recognition, questioning, observing, and experimenting, all competencies that machines cannot easily replicate. A recent study from McKinsey predicts that as automation is progressively introduced, the demand for such competencies will sharply increase by 2030 (McKinsey Global Institute 2018). According to the QS report ‘The Global Skills Gap in the 21st Century’ (QS Intelligence Unit 2019), employers also show a low level of satisfaction with graduates’ profiles in other behavioral competencies, such as resilience, adaptability, leadership, and persuasion. Although there is a general agreement that higher education institutions (HEIs) should equip students with these competencies before they enter the labor market, contributing to fill the gap and meet companies’ expectations, effective initiatives in this direction are still limited (Ritter et al. 2018). The next sections of this chapter will address these issues. Firstly, some methodological recommendations for HEIs will be introduced on how to implement competency-based courses within their degree programs in data science and big data. Subsequently, the chapter will provide insights for human resource management specialists on how to revise the way they search and assess big data profiles by adopting a competency-­ based approach.

92 

S. BONESSO ET AL.

5.2   Developing Behavioral Competencies in Data Science and Big Data Academic Programs Behavioral competencies are more complex and difficult to develop than technical skills; thus pursuing their development can face obstacles, from the different roles instructors have to assume, to the active learning strategies and the different training tools they need to adopt (Bedwell et  al. 2014). These elements may discourage the introduction of dedicated academic courses in Bachelor and Master’s programs and in the field of data science and big data, as highlighted in a recent study that investigated how academic curricula are meeting the industry’s analytics needs (Bowers et al. 2018). As demand for big data professionals has seen exponential growth in recent years and this is expected to increase in the future, as discussed in Chap. 2, higher education institutions are still trying to provide an adequate answer to the skills shortage, including in their offerings dedicated programs in data science, analytics, and big data. The introduction of specific courses aimed at developing behavioral competencies within these educational programs seems to be hampered by several factors. One is the lack of awareness among the faculty members in charge of designing the degree programs of the skills necessary for big data jobs to achieve positive performance at work. Another factor is the rapid technological changes that continuously affect the skillset of such professionals. The need to equip students with the most recent advancements in tools and techniques has led universities to adopt a more technologically-oriented approach in designing educational programs. Moreover, even where faculty members acknowledge that aspiring big data professionals should learn emotional intelligence competencies as part of their academic experience, they face credit hours limits, with the consequent difficulty of motivating at the institutional level the decision to leave a technical course out of the program in favor of a non-technical one (Bowers et al. 2018). Since the primary objectives of HEIs include to prepare people to become outstanding professionals (Boyatzis and Saatcioglu 2008), to fully meet the labor market requirements, and to increase students’ employability, more emphasis needs to be placed on emotional intelligence in the degree programs. Dedicated learning experiences should be introduced to allow students who want to pursue a career in the big data field: (i) to become aware of the behavioral competency profile that their future professional role is expected to manifest in the workplace; and (ii) to acquire

5  MANAGING BIG DATA PROFESSIONALS… 

93

the methodology and related techniques to develop the necessary behavioral competencies. An effective methodological approach to competency development has been proposed by those academic programs that have espoused the process of intentional change to skills development (Boyatzis 2006; Boyatzis and Saatcioglu 2008; Boyatzis et al. 2002). The intentional change theory (ICT) is based on the ‘whole person’ pedagogical approach, wherein individuals are engaged in self-directed development, since “learning does not occur until the learner makes it happen” (Hoover et al. 2010: 194), especially when the subject of learning is new effective behaviors that aim to substitute established ineffective habits. The two fundamental aspects of effective application of the intentional change process are desired and sustainable change. Change – and consequently the development of behavioral competencies – is desired by the person, since learning goals are set intentionally by individuals in line with their desired personal and professional life. Differently from the majority of training experiences, the change is also sustainable, in that it lasts for a relatively long time, in contrast to the majority of soft skills training programs in which individuals experience the so called ‘honeymoon effect’ – that is, a short-term improvement followed by a decline within a few months (Boyatzis 2006). The process of intentional change has been successfully applied for competency development in international higher education contexts (Boyatzis et al. 2002, 2010; Boyatzis and Saatcioglu 2008). It has shown its positive learning outcomes in the improvement of students’ competency portfolio in longitudinal studies, and in terms of career satisfaction and success (Amdurer et al. 2014). The learning process involves five discoveries or discontinuities  – also defined as epiphanies, which are effectively wake-up calls, or moments that awaken the person to the need to consider a change – that lead to successive improvements in behavior and subsequent competency development (Boyatzis 2006; Kolb and Boyatzis 1970; Leonard 2008). Table 5.1 offers a representation of the process of intentional change. The first discovery – understanding the ideal self and creating a personal vision – leads individuals toward a mindful reflection on what matters most to them and on who they want to be (their ideal self) in terms of their passions, purpose, desired future, and core values. These components are integrated and expressed in a personal vision statement. In so doing, the change process becomes grounded in intrinsic motivations

94 

S. BONESSO ET AL.

Table 5.1  A representation of the five discoveries in Intentional Change Theory Discontinuity 1. My Ideal Self Who do I want to be?

Discontinuity 2. My Real Self Who am I? What my Ideal and Real Self are similar / different?

Discontinuity 3. My personal learning agenda Which capabilities should I develop to attain the desired future?

Discontinuity 4. Experimenting and practicing the new behavior until mastery How and where I can experience the new behavior in actual safe settings?

Discontinuity 5. Trusting relationships that encourage each stage in the process Who can help me in my learning path?

and beliefs in possibility, spurring individuals to be more resilient and increasing in them a feeling of hope. The second discovery – assessing the real self and comparing it to the ideal self – supports individuals in increasing their awareness of their current self. Usually during this phase individuals are involved in a behavioral competency assessment that allows them to understand the actual level of mastery of the behavioral competencies. The interpretation of feedback is crucial for identifying individual strengths and weaknesses, respectively those areas in which the ideal and real self overlap, and those areas in which the desired future is not consistent with the current self. These first two discoveries, stimulating individuals to ask themselves “Who do I want to be?” (ideal self) and “Who am I?’ (real self), help to nurture individual self-awareness which, as discussed in the previous chapters, is a behavioral competency considered a cornerstone of the emotional intelligence framework and the premise for subsequent development of the other behavioral competencies (Goleman 1998). The third discovery – creating a learning agenda or plan to close the gap between the real and ideal selves  – leads individuals to set learning goals and actions that they enthusiastically look forward to testing. This phase concretely moves a person closer to his or her vision by identifying ways to foster strengths and reduce the identified gap. The fourth discontinuity  – practicing new competencies and behaviors  – is dedicated to actual experimentation with the new behaviors in different personal and professional contexts. The last discovery is focused on identifying and building trusted relationships that support and encourage each step in the change process.

5  MANAGING BIG DATA PROFESSIONALS… 

95

The following section will discuss in more detail how the different discoveries can be tailored in dedicated academic courses to nurture the behavioral competencies of big data professionals.

5.3   Introducing Wake-Up Calls in Data Science and Big Data Educational Programs The story narrated at the beginning of this chapter is an exemplar case of a graduate in the early stages of his career who demonstrates a strong vocational orientation toward the field of big data but who has not yet developed the necessary self-awareness to become attuned to himself, others, and the organizational environment, a state of mind also known as mindfulness. Recent research shows that mindful engineers are more able to generate new ideas, think outside the box, and find better solutions to problems (Rieken et al. 2019). Moreover, mindful executives make more effective choices about how to respond to people and situations, have stronger working relationships, achieve better project outcomes, increase budgets and team headcount, and attain better career outcomes (Goleman and Lippincott 2017). The Intentional Change Theory offers insight into how to practice mindfulness, helping individuals, on the one hand, to keep in mind why they are doing what they do, to discover the calling in their career, and to develop an image of their desired future (the ideal self discontinuity), and, on the other hand, to reflect on their inner capabilities and areas for improvement to effectively direct personal change (the real self discontinuity). Within a course dedicated to the development of behavioral competencies for big data roles, self-awareness can be stimulated through several experiential learning activities, summarized in Table 5.2. During the first wake-up call, ‘Who do I want to be?’, students should be introduced to deep self-reflection on their future professional dreams. The process of visioning helps the individual to imagine tasks in which he/she will be involved in the job, the main responsibilities and interactions within the work environment, and the ultimate purpose of his/her activity. For instance, an aspiring data scientist or data engineer can ask him/herself: What kind of business problems do I want to address? What kind of data do I prefer to work with? In which industry do I wish to operate? In what way does this job help me put into practice my inner values and my passions? What will be my contribution to the performance of the company?

96 

S. BONESSO ET AL.

Table 5.2  Experiential activities for developing self-awareness of the inner identity and future work self, and the necessary behavioral competency profile Discovery and key question

Experiential learning activities

Ideal self who do I want to be in the future?

Self-reflection on values, passions, and dreams, and behavioral competencies necessary to achieve the future Writing ‘my personal vision’ essay Feedback from the facilitator Coaching and peer coaching Vicarious learning through narratives of work experiences and hiring process from professionals, employers, and recruiters in the data science field Assessment of behavioral competencies (multisource assessment or behavioral event interviews) Feedback from the facilitator Coaching and peer coaching

Real self Who am I?

The more detailed the description of the desired professional life, the easier it will be for the students to think about the behavioral competencies necessary to perform the job outstandingly independently of their actual level of mastery. An important role in this phase of discovery of the ideal self – namely, linking the vision of the desired profession with the behavioral skillset necessary to achieve it – is played by the instructor, who assumes the role of a learning facilitator and can activate the five discoveries of Intentional Change Theory. Indeed, trusting relationships have an important role in helping and supporting the individual during each discovery of the intentional change process. Specifically within ideal self discovery, this discontinuity consists of leveraging the person’s key relationships in order better to envision the desired future. By talking with other people, sharing their views of the future, and receiving feedback from them, the individual gains help and encouragement to really look inside themselves and find the motivation to pursue the change process. Facilitators may stimulate participants toward self- and group reflection, helping them to become more aware of the job opportunities in the big data field, delineating the boundaries among different roles, and favoring dialogue with role models who can inspire a more concrete visualization of the future. Indeed, role modeling is often mentioned as one of the techniques to improve mindfulness (Goleman and Lippincott 2017) and, in the specific case of ideal self discontinuity,

5  MANAGING BIG DATA PROFESSIONALS… 

97

interaction with individuals who already perform big data roles may provide further opportunities to gather insights into the job’s main characteristics. Through these professionals’ narratives, students may be involved in a form of vicarious learning, getting acquainted with the situations in which they will probably be most frequently involved in their future career and with the behavioral competencies they will be required to manifest. During the narration of role models’ experiences, the facilitator may help the students to read the behaviors activated by the professionals through the lens of the behavioral competency framework. To increase students’ awareness of the competency profile expected in the desired professional roles, facilitators should engage participants in a conversation with labor market operators, such as participating in recruiting events dedicated to big data professions and consulting online job advertisements in the area of data science. Besides activating trusting relationships with the instructor or with big data professionals, potential employers, and recruiters interested in data science profiles, the participants can be invited to share their image of their professional future with the other participants on the course through a peer coaching experience. Peer coaching is a helpful, mutually beneficial relationship with the goal of personal or professional development. It is based on qualities such as unconditional positive regard, authenticity, mutual trust, and reciprocity of the process (Parker et  al. 2008, 2015). Past evidence has shown positive outcomes of coaching and peer coaching in educational programs in terms of increased self-confidence, empowerment, self-understanding, success in dealing with change, and development of soft skills (Boyatzis et al. 2006; Gerli et al. 2019; Goldman et al. 2011; Parker et al. 2008). In a peer coaching session, participants support each other’s development by listening, asking what may sometimes be provocative questions, and giving critiques, in order to promote clarity from the narrator (Kotlyar et  al. 2015). The attention is on the whole person and, through a process of reflection, peers reciprocally build awareness of the cognitive and affective aspects of their desired future. The ultimate aim of the coaching session is to help each peer to achieve a deeper level of analysis and reflection on personal values, passions, and future dreams, to make hidden assumptions explicit. As suggested by Parker et  al. (2008: 491) “the peers engage in shared sense-making of each other’s worldview.” After this deep exploration of one’s inner identity and of the desired future, the reflections developed during the first discovery can be

98 

S. BONESSO ET AL.

i­ncorporated in a written essay, ‘My Personal Vision’, in which the participants describe their compelling path to their desired career five or ten years in the future (McKee et al. 2008). A prominent part of the personal vision is devoted to narration of the future work self, or to the individual’s representation of their hopes and aspirations for their future working life (Strauss et al. 2012). As prior studies suggest, the clearer and easier it is for a person to imagine their future work self, the more this image represents a motivational driver for proactive career behaviors, such as career planning, competency development, career consultation, and network building (Strauss et al. 2012). To orient the learning path toward competency development, individuals should become aware of their strengths and areas for improvement. For this reason, in the second discovery, students are stimulated to compare their current level of mastery of behavioral competencies with the competency profile expected in their desired professional profile. Analysis of the actual behavioral profile can be conducted with reference to two main methodological approaches to competency assessment. The first consists of multisource feedback, in which their own and external raters’ evaluations are combined to provide a comprehensive view of the most and least frequently manifested behaviors of the student. For instance, the Emotional and Social Competency Inventory  – University Version (ESCI-U) is one of the most effective 360-degree competency assessment tools widely adopted in both academic and organizational settings (Boyatzis et al. 2015; Boyatzis and Goleman 2007). One of the primary benefits of using 360-degree assessment is to collect different perspectives on the person from both the personal and professional contexts. Identifying “if the person uses the competency behavior at home or in leisure settings provides a more thorough review of the person’s range of action. It also suggests different tactics in helping the person change their behavior in either work or personal settings” (Boyatzis 2018: 9). The second approach refers to a third-party evaluation by administering behavioral event (Camuffo and Gerli 2004; Spencer and Spencer 1993) through which the interviewee reconstructs how the person activated specific behavioral competencies in past concrete situations (see Chap. 4 for a more detailed description of this methodology). The behavioral competencies included in the competency hexagon, illustrated in Chap. 3 and used to explore the behavioral profiles of data scientists and analysts, may represent another tool that can be adopted in the assessment process.

5  MANAGING BIG DATA PROFESSIONALS… 

99

Through in-class and individual discussion of this assessment, guided by the facilitator and integrated with peer coaching and coaching sessions, students become aware of their current behavioral portfolio and compare their ideal and real selves, reflecting on their strengths (competencies that are indicated as necessary if they are to obtain their personal vision and that they demonstrate most) and areas of improvement (competencies that are indicated as necessary for their personal vision but that they demonstrate least). The core activity in which students are involved in the third discovery is drafting a learning plan. Therefore, for each competency chosen as a learning goal, the students define timeframes and concrete sets of actions they aim to practice in everyday contexts. In order to be effective and realizable, the learning plan has to be customized based on personal characteristics, needs, and learning styles (Kolb and Kolb 2005; Goleman et al. 2002). The definition of the plan represents a form of active learning, since it “helps participants move rapidly between theoretical ideas and efforts to apply those ideas in the context of their work organizations and within their projects” (Waddock and Lozano 2013: 276). In this stage of the process, the fifth discovery can be activated by means of social learning through interaction with peers. The instructor can support students in the writing of their plan, offering the opportunity to discuss in small teams the learning actions to be implemented. Specifically, facilitators can invite students to form small groups in which participants who aim to learn the same competencies but may prefer different learning styles can discuss possible concrete actions that they can implement to experience the new behaviors. Trusting relationships with those whom the new behaviors are practiced can also be specified by the student in the plan. The learning plan can be revised by the instructor, who can provide valuable insights into how to make it more concrete and actionable. Once the learning plan has been defined, it can be put into practice in the fourth discovery. This discontinuity represents the step in which the person, in line with his/her learning plan, experiments with and practices new behaviors. Changing requires long-lasting practice, reflection, and repetition of new behaviors, allowing new habits to be mastered and applied in all contexts (Boyatzis 2006). Every personal context can be conceived as a learning laboratory where, in safe environments, students can become familiar with the new behaviors without being afraid of negative feedback. However, in order to practice behavioral competencies in situations similar to those that students will face in their data science

100 

S. BONESSO ET AL.

­ rofession, they can use a capstone project or an internship experience p during their academic path as a learning context. A capstone project, as a form of project-based learning, is frequently adopted in data analytics degree programs (Bowers et al. 2018) and has been shown to be effective for facilitating practice of behavioral competencies (English and Kitsantas 2013) through the involvement of students in real data science problems, defined by companies, that should be solved cooperatively. Finally, in order to monitor students’ progression along their learning path, a follow-up session can be organized some months after the end of the course. A few days before the follow-up, the facilitator can ask students to share with the class a presentation in which they describe how they have put into practice the learning plan created during the course. Specifically, the content of the presentation may include: an indication of the competencies that the students aimed to learn and the relevance of these for attaining their personal vision; a brief presentation of their learning plan; the contexts in which they experimented with new behaviors; the way in which they evaluated their learning progress and their level of mastery of the competencies; the difficulties they encountered in practicing the new behaviors; and how their trusted relationships supported them in attaining their results. The follow-up session may provide an opportunity for vicarious learning, since students who aimed to develop the same emotional or social competency can: i) compare their experience with that of their peers, understanding how to remove the obstacles they encountered during their experimentation with new behaviors; ii) learn new techniques to practice the competency; and iii) identify new actions that are related to the same competency but adopt a different learning style. At the end of the presentation of each experience, the instructor can provide further advice that helps to direct students’ efforts toward effective practice of the competencies. Moreover, students can be involved in further experiential learning initiatives (courses, seminars, laboratories, etc.) dedicated to the development of specific behavioral competencies in order to acquire techniques that allow them to put the new behaviors into practice in their daily activities. This section has provided some methodological and didactic insights into how to design a learning experience within academic data science programs through which students may increase their awareness of the importance of behavioral competencies for big data professionals and undertake a path toward their development.

5  MANAGING BIG DATA PROFESSIONALS… 

101

As highlighted by A.  Charles Thomas, PhD in sociology and Chief Data & Analytics Officer at General Motors, during his keynote speech at the 2019 Wharton Customer Analytics conference, “academic programs that train data scientists should include a focus on the whole person, not just the science. Otherwise the result is data scientists who can get too wrapped up in the breadth of data they’re handling and lose focus on actionable insights” (Knowledge@Wharton 2019). The recommendations offered in this section go in this direction. They can also be taken into consideration by corporate academies in their training courses, and by training managers when identifying the training needs and the training programs to offer employees.

5.4   How to Make Hiring more Emotional and Successful When every candidate has the same expertise in Hadoop, or demonstrates the same familiarity with NoSQL databases, how can you distinguish between them? A growing number of international companies, like Google, L’Oréal, and AT&T, have introduced emotional intelligence as a key element of assessment in their hiring processes, attaining positive outcomes in terms of higher productivity, employee engagement, and profit. If, at the educational level, universities play a central role in equipping students with both the technical and the behavioral skills necessary to achieve outstanding performance in big data jobs, at the employer level, companies can reduce failures in the hiring process, recruiting for soft skills and designing human resource management practices according to a competency-based approach. LinkedIn’s Global Talent Trends (LinkedIn 2019) reveals that 57 percent of companies still struggle accurately to assess soft skills in their candidates, and that they continue to adopt unstructured approaches during the selection process based on recruiters’ personal feelings. But how can companies improve their ability to recruit for behavioral competencies? Firstly, they should take into consideration the compatibility or fit between the competencies valued most by the company itself and the competency profile of the applicants. Executives, in collaboration with the HRM department, can benefit from defining the behavioral competencies that the data science team needs to manifest in order to promote a data-­ driven culture inside the organization and to succeed in its digital

102 

S. BONESSO ET AL.

t­ ransformation. Moreover, these competencies have to be indicated explicitly in the job description and in job advertisements to orient candidates’ applications. As illustrated in Chaps. 2 and 4, the different big data profiles (data architect, data engineer, data scientist, etc.) are characterized by different technical and soft skillsets. Secondly, recruiters can be trained in administering and codifying behavioral event interviews in order to assess candidates’ level of mastery of the key competencies for that specific position, addressing questions such as: “Tell me about a time when you had to deal with a highly demanding customer. How did you manage the relationship?” “Have you ever been in a situation in which you needed to adjust your behavior? How did you know and what did you do?” “Tell me the last time you introduced an innovation in your company.” For instance, the Chief Data & Analytics Officer at General Motors spurred the introduction of questions into GM’s interview process that would help determine whether candidates had an aptitude for insights. As he said, “Every single person should have an aptitude for insights. This is no longer a back-office activity. It’s at the table and it should be part of how we make decisions” (Knowledge@Wharton 2019). Moreover, gamification can be integrated in the recruitment process to attract prospective big data profiles and assess their soft skills while candidates are engaged in a simulated work environment. Gamification has been applied during the hiring process to make assessment methods more game-like, improving candidates’ reactions and consequently increasing the accuracy of the prediction of future work behaviors. Recent evidence shows that game elements, such as storylines, feedback, avatars, visuals, and voiceovers, effectively assess candidates’ soft skills such as resilience, adaptability, and decision-making (Georgiou et al. 2019). Assessment of behavioral competencies during the hiring process represents an opportunity for the newly hired to understand their level of mastery of the key behavioral competencies for their job and the organizational culture. A relevant role in promoting individual behaviors coherent with the organization’s goals is played by the induction program, which should be structured in a way that allows the new employee to gain familiarity with the organizational environment and the job requirements. During that period, the skillset of the professional profile should be made explicit, and periodic feedbacks should be provided to support the individual to redirect his/her behaviors to better meet the job and company expectations. Specifically, feedback should be used: i) to help the newly hired to reflect on the behaviors activated and the outcomes attained; ii) to provide exam-

5  MANAGING BIG DATA PROFESSIONALS… 

103

ples of how to activate the appropriate combination of behavioral competencies relevant to face specific work situations effectively; and iii) to provide guidance on how to improve areas of strength further and at the same time work on areas for improvement as assessed during the hiring process. All the aforementioned practices aim to introduce into the organizational context a more fine-grained approach of the analysis of behavioral competencies in order to improve the recruitment and assessment of suitable big data profiles. Despite their relevance, behavioral competencies are still ill-defined for these professions, and further work is required by both the academic and the practitioner communities to understand their effective combination for the different big data job families. The practical insights provided here may help companies not to lose time searching for ‘unicorns’, but to hire the skills that can help address actual business problems and meet specific needs.

References Amdurer, E., Boyatzis, R. E., Saatcioglu, A., Smith, M. L., & Taylor, S. N. (2014). Long term impact of emotional, social and cognitive intelligence competencies and GMAT on career and life satisfaction and career success. Frontiers in Psychology, 5, 1447. Beck, M., & Libert, B. (2017, February 15). The rise of AI makes emotional intelligence more important. Harvard Business Review. Bedwell, W. L., Fiore, S. M., & Salas, E. (2014). Developing the future workforce: An approach for integrating interpersonal skills into the MBA classroom. Academy of Management Learning & Education, 13, 171–186. Bowers, M. R., Camm, J. D., & Chakraborty, G. (2018). The evolution of analytics and implications for industry and academic programs. INFORMS Journal on Applied Analytics, 48(6), 487–499. Boyatzis, R. E. (2006). Intentional change theory from a complexity perspective. Journal of Management Development, 25(7), 607–623. Boyatzis, R. E. (2018). The behavioral level of emotional intelligence and its measurement. Frontiers in Psychology, 9, 1–12. Boyatzis, R. E., & Goleman, D. (2007). Emotional and social competency inventory. Boston, MA: The Hay Group. Boyatzis, R.  E., & Saatcioglu, A. (2008). A 20-year view of trying to develop emotional, social and cognitive intelligence competencies in graduate management education. Journal of Management Development, 27, 92–108.

104 

S. BONESSO ET AL.

Boyatzis, R. E., Stubbs, E. C., & Taylor, S. N. (2002). Learning cognitive and emotional intelligence competencies through graduate management education. Academy of Management Learning and Education, 1, 150–162. Boyatzis, R. E., Smith, M. L., & Blaize, N. (2006). Developing sustainable leaders through coaching and compassion. Academy of Management Learning & Education, 5, 8–24. Boyatzis, R. E., Lingham, T., & Passarelli, A. (2010). Inspiring the development of emotional, social and cognitive intelligence competencies in managers. In M. G. Rothstein & R. J. Burke (Eds.), Self-management and leadership development (pp. 62–90). Cheltenham: Edward Elgar Publishing. Boyatzis, R. E., Gaskin, J., & Wei, H. (2015). Emotional and social intelligence and behavior. In S. Goldstein, D. Princiotta, & J. A. Naglieri (Eds.), Handbook of intelligence. Evolutionary theory, historical perspective, and current concepts (pp. 243–262). New York: Springer. Camuffo, A., & Gerli, F. (2004). An integrated competency-based approach to management education: An Italian MBA case study. International Journal of Training and Development, 8(4), 240–257. English, M. C., & Kitsantas, A. (2013). Supporting student self-regulated learning in problem- and project-based learning. Interdisciplinary Journal of Problem-­ Based Learning, 7(2), 128–150. Georgiou, K., Gouras, A., & Nikolaou, I. (2019). Gamification in employee selection: The development of a gamified assessment. International Journal of Selection and Assessment, 27(2), 91–103. Gerli, F., Bonesso, S., & Cortellazzo, L. (2019). Nurturing students’ awareness of their behavioral competencies: The Competency Lab experience. 5th International Conference on Higher Education Advances (HEAd’19). Valencia, Spain: Universitat Politecnica de Valencia. Goldman, E., Wesner, M., & Karnchanomai, O. (2011). Reciprocal peer coaching: A critical contributor to implementing individual leadership plans. Human Resource Development Quarterly, 24(1), 63–87. Goleman, D. (1998). Working with emotional intelligence. New  York: Bantam Books. Goleman, D., & Lippincott, M. (2017, September 08). Without emotional intelligence, mindfulness doesn’t work. Harvard Business Review. Goleman, D., Boyatzis, R. E., & McKee, A. (2002). Primal leadership: Realizing the power of emotional intelligence. Cambridge, MA: Harvard Business School Press. Gustein, A. J., & Sviokla, J. (2018, July 17). 7 skills that aren’t about to be automated. Harvard Business Review. Hoover, D. J., Giambatista, R. C., Sorenson, R. L., & Bommer, W. H. (2010). Assessing the effectiveness of whole person learning pedagogy in skill acquisition. Academy of Management Learning and Education, 9, 192–203.

5  MANAGING BIG DATA PROFESSIONALS… 

105

Knowledge@Wharton. (2019, July 24). Why future jobs will require data analytics skills. Retrieved from https://knowledge.wharton.upenn.edu/article/ evolution-of-chief-data-officer/ Kolb, D.  A., & Boyatzis, R.  E. (1970). Goal setting and self-directed behavior change. Human Relations, 23(5), 439–457. Kolb, A. Y., & Kolb, D. A. (2005). Learning styles and learning spaces: Enhancing experiential learning in higher education. Academy of Management Learning & Education, 4(2), 193–212. Kotlyar, I., Richardson, J., & Karakowsky, L. (2015). Learning to lead from outsiders. The value of career communities as a source of external peer coaching. Journal of Management Development, 34(10), 1262–1127. Leonard, D. C. (2008). The impact of learning goals on emotional, social, and cognitive intelligence competency development. Journal of Management Development, 27(1), 109–128. Linkedin. (2019). Global talent trends. The 3 trends transforming your workplace. Retrieved from https://business.linkedin.com/content/dam/me/business/ en-us/talent-solutions/resources/pdfs/global_talent_trends_2019_emea.pdf McKee, A., Boyatzis, R., & Johnston, F. (2008). Becoming a resonant leader: Develop your emotional intelligence, renew your relationships, sustain your effectiveness. Boston, MA: Harvard Business School Press. McKinsey Global Institute. (2018). Skill shift: Automation and the future of the workforce. Retrieved from https://www.mckinsey.com/~/media/mckinsey/ featured%20insights/future%20of%20organizations/skill%20shift%20automation%20and%20the%20future%20of%20the%20workforce/mgi-skill-shiftautomation-and-future-of-the-workforce-may-2018.ashx Parker, P., Hall, D. T., & Kram, K. E. (2008). Peer coaching: A relational process for accelerating career learning. Academy of Management Learning and Education, 7(4), 487–503. Parker, P., Wasserman, I., Kram, K. E., & Hall, D. (2015). Relational communication approach to peer coaching. Journal of Applied Behavioral Science, 51(2), 231–252. QS Intelligence Unit. (2019). The global skills gap in the 21st century. Retrieved from https://www.qs.com/portfolio-items/the-global-skills-gap-in-the-21stcentury Rieken, B., Shapiro, S., Gilmartin, S., & Sheppard, S. D. (2019, January 04). How mindfulness can help engineers solve problems. Harvard Business Review. Ritter, B. A., Small, E. E., Mortimer, J. W., & Doll, J. L. (2018). Designing management curriculum for workplace readiness: Developing students’ soft skills. Journal of Management Education, 42(1), 80–103. Spencer, L. M., & Spencer, S. M. (1993). Competence at work: Models for superior performance. New York: Wiley.

106 

S. BONESSO ET AL.

Strauss, K., Griffin, M. A., & Parker, S. K. (2012). Future work selves: How salient hoped-for identities motivate proactive career behaviors. Journal of Applied Psychology, 97(3), 580–598. Waddock, S., & Lozano, J.  M. (2013). Developing more holistic management education: Lessons learned from two programs. Academy of Management Learning & Education, 12, 265–284. World Economic Forum. (2019). The future of jobs report 2018. Retrieved from https://www.weforum.org/reports/the-future-of-jobs-report-2018

Index

A Algorithm design, 30 Analytical tools Apache Flume, 27 Apache Pig, 27 Hadoop, 7, 27, 28, 30, 101 Hive, 27 MapReduce, 7, 28 Oozie, 27 Python, 9, 27, 28, 31, 56, 72, 90 R, 9, 27, 28, 31, 72, 90 Spark Hadoop, 27 SQL, 5, 25, 27, 28, 31, 34 T-SQL, 27 Analytics descriptive, 8, 12 predictive, 8, 12 prescriptive, 8, 12 what they are, 8 APEC, 23, 28 Artificial intelligence, 3, 4, 8, 13, 22, 30, 35, 91 Awareness data analysts of, 77, 78 data scientists of, 77, 78 definition of, 50

B BARC’s BI Trend Monitor 2019, 8 Behavioral competencies data analysts and, 14, 57, 64–85 data scientists and, 14, 57 definition of, 101 methods of, 16, 93, 98 Behavioral Event Interview (BEI), 16, 47, 66, 68, 102 Big data, 11, 24, 36, 66, 83, 85, 90, 101 challenges, 2–16 data driven culture, 66 definition of, 4 Big Data Analytics & Business Intelligence (University of Politecnico di Milano), 66 Big Data Analytics Market, 7 Big Data and AI Executive Survey, 24 Big data professionals, 13, 16 business analysts, 13, 14 current workforce of, 23 data analysts, 15, 66 data scientists, 13–15, 66 future scenario of, 35–36

© The Author(s) 2020 S. Bonesso et al., Behavioral Competencies of Digital Professionals, https://doi.org/10.1007/978-3-030-33578-6

107

108 

INDEX

Big data technology architectures, see Analytical tools Boyatzis, Richard E., 13, 43–45, 47, 49, 53, 54, 65, 66, 68, 92, 93, 97–99 Brynjolfsson, Eric, 2, 57 Burning Glass Technologies, 22–24, 27, 30 Business analytics, see Big data professionals C Chade-Meng Tan (Meng), 42 Cherniss, Cary, 47 Chief data officer (CDO), 10, 15, 24 Cluster analysis/clustering, 69 Cognitive competencies, see Competency hexagon Competency-based approach Boyatzis, R.E., 49 development, 93 phases, 66, 94 Competency codebook, see Competency hexagon Competency hexagon, ix, 13, 36, 43, 57, 64, 65, 69, 78–80 action competencies, 16, 50, 53, 69–75, 84 awareness competencies, 16, 50, 53, 69, 77–78, 84, 97 cognitive competencies, ix, 13, 16, 36, 43, 50, 53, 57, 64, 65, 69, 78–80 exploration competencies, 47, 91 organizational action competencies, 16, 50, 55, 56, 69, 81–85 social competencies, ix, 13, 14, 16, 26, 36, 44, 50, 53, 57, 64, 65, 69, 75–76, 84, 100

Contingency theory approach, see Boyatzis, Richard E. Costa, Carlos, 13, 14, 24, 28, 54–56, 65 D Data analysts behavioral competencies possessed, 15 education and background, 15 job requirements, 33, 80 main tools used, 9, 11, 15, 92 Data analytics data mining, 6, 8, 12, 25, 30, 31, 42 data visualization, 25, 35 data warehousing, 7 Data analytics officer, 24 Data cleaning, 84, 90 Data-driven culture, see Big data Data professionals database administrator, 15, 25–27 database architect, 15, 25–27 data engineer, 12, 15, 25–27, 33, 66, 95, 102 Data Science Salary Survey, 84 Data scientists behavioral competencies, 14, 16, 36, 57, 64–85 education, 68 job requirements, 80 main tools used, 15 profile, 14, 15, 57, 75, 83–85, 98 Data scientists: the sexiest job of the 21st Century (Davenport and Patil), 28 Data workers, see Data professionals Davenport, Thomas H., 2, 4, 8, 13, 24, 28, 29, 31, 33, 56, 65, 66, 68, 80 Deloitte, 13 Delphy methodology, 36 De Mauro, Andrea, 4, 34–36, 53, 55, 68, 69

 INDEX 

E Emmerling, Robert J., 68 Emotional and social intelligence competencies relationship management competencies, 33, 36, 44, 54 self-awareness competencies, 43, 44, 46, 53, 69, 77, 78, 94–96 self-management competencies, 43, 44, 53 social-awareness competencies, 43, 44, 53 Emotional intelligence application, 65 definition of, 45 development, 43 Entry-level big data professions, 16 EU General Data Protection Regulation (GDPR), The, 83 F Facebook, 28 Flanagan, John C., 68

109

Human resource management, 43, 91, 101 key performance indicators, 36 I Innovation, 49, 50, 80, 102 Istat (Istituto nazionale di statistica), 66 J Java, 26, 28 L LinkedIn, 12, 23, 28, 29, 31, 36, 49, 67, 91, 101 Lisbon Council and International Data Corporation (IDC), The, 23

G Gartner’s maturity model, 10 Glassdoor’s 50 Best Jobs in America, 28 Goleman, Daniel, 13, 43–46, 49, 53, 65, 83, 94–96, 98, 99 Google, 42 Page, Larry, 57 Search Inside Yourself (SIY), 42 trends, 6, 7 Granville, Vincent, 8, 27, 29, 30

M Machine learning, 4, 6, 8, 9, 13, 28, 30, 31, 35, 85, 90 Manipulation of data, 5, 65 McAfee, Andrew, 57 McClelland, David C., 43, 49, 65 McKinsey Global Institute, 2, 8, 9, 11, 12, 24, 49, 91 Mehrotra, Vijay, 29, 30, 54, 56, 66, 69 Modelling coding language JAVA, 26, 28 Matlab, 28 R, 9, 27, 28, 31, 90 SQL, 5, 25, 27, 28, 31, 34 Monte Carlo simulations, 30

H Hadoop, see Analytical tools Hard skills, see Soft skills Harris, Jeanne G., 29, 30, 54–56, 66, 69

N Natural language processing, 30, 31 NewVantage Partners Executive Survey, 11

110 

INDEX

O Open-source tools, 30 Organizational structure big data technology, 4 data driven culture, 11, 24, 36, 66, 85, 90, 101 manager roles in, 16, 49 P Patil, Dhanurjay DJ, 4, 13, 28, 29, 56, 65, 66, 68, 80 Program language and tools, 8, 9, 11, 12, 15, 24, 26, 28, 30, 31, 33, 56, 76, 92, 98 See also Analytical tools PwC, 23 Python, see Analytical tools Q Quantitative analysis, 34 R Relationship management, see Emotional and social intelligence competencies S Sala, fabio, 65 Santos, Maribel Yasmina, 13, 14, 24, 28, 54–56, 65 Search Inside Yourself (SIY), see Google Self-awareness, see Emotional and social intelligence competencies Self-management, see Emotional and social intelligence competencies Shirani, Ashraf, 54, 56, 68 Social-awareness, see Emotional and social intelligence competencies

Soft skills, 13, 15, 26, 31, 32, 35, 36, 49, 56, 57, 66, 68, 83, 91, 93, 97, 101, 102 Spencer, Lyle M., 43, 47, 49, 53, 54, 65, 66, 69, 98 SPSS predictive analytics software, see Statistical software Statistical analysis, 8, 28, 67 Statistical software R, 9, 27, 28, 31, 72, 90 SAS, 5, 31, 78 SPSS, 31 Strategy data scientists and, 24, 32, 82 strategic thinking, 32, 56, 82, 83, 85, 90 Structure data, 6, 30 Structured Query Language (SQL), 5, 25–28, 31, 34 T Training program for data scientists, 30, 101 for managers, 101 U Unstructured data analysis of, 26, 28, 30 processing, 27 report, 26 V Visualization of data, 5, 25, 35 W World Economic Forum, 3, 4, 22, 33, 49, 57, 91