Hidden in White Sight 9781032437644, 9781032437620, 9781003368755, 1032437642

Table of contents :
Cover
Half Title
Title Page
Copyright Page
Dedication
Acknowledgments
Table of Contents
Foreword
Preface: Why Now?
Disclaimer
Introduction
1 Listening Ears
2 The Racist Algorithm
3 The American Dream
4 AI Gone Wild
5 An Enduring Legacy
6 Our Authentic Selves
7 Mass Unemployment
8 Medically Induced Trauma
9 Colored Ads
10 Weapons of Mass Destruction
11 Designers’ Dilemma
12 Corporate Choice
13 Algorithmic Justice
Conclusion: How Can I Help?
Notes, Inspirations, and Further Reading
Index

Citation preview

Hidden in White Sight

Artificial Intelligence was meant to be the great social equalizer that helps promote fairness by removing human bias from the equation, but is this true? Given that the policing and judicial systems can display human bias, this book explores how the technology they use can also reflect these prejudices. From healthcare services to social scoring in exams, to applying for and getting loans, AI outcomes often restrict those most in need of these services. Through personal stories from an esteemed Data Scientist and AI expert, this book attempts to demystify the algorithmic black box. AI pervades all aspects of modern society and affects everyone within it, yet its internal biases are rarely confronted. This book advises readers on what they can do to fight against it, including the introduction of a proposed AI Bill of Rights, while also providing specific recommendations for AI developers and technologists.

Hidden in White Sight

How AI Empowers and Deepens Systemic Racism Calvin D. Lawrence, M. S.

First edition published 2023 by CRC Press 6000 Broken Sound Parkway NW, Suite 300, Boca Raton, FL 33487-2742 and by CRC Press 4 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN CRC Press is an imprint of Taylor & Francis Group, LLC © 2023 Calvin D. Lawrence Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, access www.copyright.com or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. For works that are not available on CCC please contact [email protected] Trademark notice: Product or corporate names may be trademarks or registered trademarks and are used only for identification and explanation without intent to infringe. Library of Congress Cataloging‑in‑Publication Data Names: Lawrence, Calvin (Computer scientist), author. Title: Hidden in white sight : how AI empowers and deepens systemic racism / Calvin Lawrence. Description: First edition. | Boca Raton : CRC Press, [2023] | Includes bibliographical references and index. | Identifiers: LCCN 2022046951 (print) | LCCN 2022046952 (ebook) | ISBN 9781032437644 (hardback) | ISBN 9781032437620 (paperback) | ISBN 9781003368755 (ebook) Subjects: LCSH: Artificial intelligence. | Racism. | Computer algorithms. Classification: LCC Q335. L39 2023 (print) | LCC Q335 (ebook) | DDC 303.48/3408—dc23/eng20230117 LC record available at https://lccn.loc.gov/2022046951 LC ebook record available at https://lccn.loc.gov/2022046952 ISBN: 9781032437644 (hbk) ISBN: 9781032437620 (pbk) ISBN: 9781003368755 (ebk) DOI: 10.1201/9781003368755 Typeset in New Baskerville by codeMantra

To my mother, Mary J. Lawrence, who will always be the smartest person that I’ve ever known. Thank you for teaching me the two simple principles that have never left me: Stand up for what you believe in and don’t be a slave to the dollar—Those two things can rarely co-exist.

Acknowledgments

I would like to thank the many crusaders for equal rights and social justice whose past, current, and future courage and fortitude continue to force each of us to reconsider our privilege and resurrect a spirit of genuine activism. I salute and acknowledge the many scholars, freedom fighters, authors, and caretakers of the notions of fairness and equity. I thank my friends and family who have inspired me to do something that I never envisioned doing: put my thoughts on paper. Finally, I am appreciative of you, the reader. I am in no way an author, but you rolled the dice and took a chance on me, and for that I’m indebted to you. I know how important your time, money, and purpose are, so thank you for sharing all three with me. I’m eternally grateful to each of you.

vi

Contents

Forewordix Preface: Why Now? xi Disclaimerxix Introduction  1   1

Listening Ears  13

  2

The Racist Algorithm  25

  3

The American Dream  39

  4

AI Gone Wild  57

  5

An Enduring Legacy  71

  6

Our Authentic Selves  85

  7

Mass Unemployment  105

  8

Medically Induced Trauma  119

  9

Colored Ads  137

10

Weapons of Mass Destruction  147

11

Designers’ Dilemma  153

12

Corporate Choice  171

13

Algorithmic Justice  187

Conclusion: How Can I Help?  203 Notes, Inspirations, and Further Reading

211

Index

223 vii

Foreword

Technology has the potential make our lives better. It is used to drive the economies for many companies, improving the overall quality of life for its citizens. In this book, Calvin Lawrence focuses on one of the most innovative and pervasive technologies of our present time, artificial intelligence (AI) and machine learning (ML). AI/ML can facilitate automation and enable efficiency and great profitability in businesses. It also has the potential to advance efforts to cure cancer and other debilitating diseases. However, AI/ML can also be used to sustain or make exponentially worse—unintentionally—the systemic racism, driven partially by cultural norms, which is present in our society. AI/ML depends heavily on leveraging clean and correct data sets that fully represent the demographics of those analyzed, as well as utilizing algorithms that are socially and racially unbiased. Calvin Lawrence utilizes a simplistic storytelling methodology to describe an exceptionally complex and nuanced technology and its troubling limitations. He fosters a genuine connection with the layman reader to enable cognizance. Mr. Lawrence uses his own personal experiences—and those of his friends and family—to explain AI/ML, its limitations and solutions. The information is based on strong, solid research on Ethics in AI. AI/ML objectives are to make predictions or recommendations that are equitable and fair, optimizing the algorithms in a manner like how his grandmother effectively utilizes the right ingredients (data) in her caramel cake, using her recipe (algorithm) to optimize a delightful baking perfection. These limitations can have unfortunate and tragic consequences for Blacks and other people of color in our society. Mr. Lawrence leverages a plethora of examples in education, credit and finance, ix

x

Foreword

government (e.g. the police), and employment to emphasize how AI bias adversely impacts communities of color. He suggests some potential solutions to this biased AI based on guardrails, enhancements to the development process, and a thought-provoking AI Bill of Rights. This book is a must-read. It can be used to educate those in the impacted communities, the developers and companies on the issues, and any interested party. It emphasizes the urgent need to address AI issues now. If not, this country—and our global society—may sustain some of our systemic racial structures. This book is a call to action to address the issues and enable AI/ML to fulfill its true promise: becoming a major impetus to improving our global quality of life. Sandra K. Johnson, Ph.D. CEO, SKJ Visioneering Former CEO, IBM Central, East and West Africa

Preface

Why Now?

When I shared with my mom my desire to write a book, her first response was, “Why did it take you so long?” Her next question was, “Now, what technology or social justice topic are you writing on?” She knew my book had to be on one of those topics. To her delight, my literary desire was to speak on both of those topics at the same time. Over the last few years, she’s had a goal-line view of my evolution from a staunch technologist fixated on solving complicated problems to a conscientious engineer engulfed by who’s using the technology and for what purpose. My mom’s questions were rooted in her understanding of my journey. She’s seen me transition from a person who grew up in the inner city and had lots of animosity toward authority of any kind—in particular, law enforcement—to a well-respected IT industry veteran with a career that has spanned almost three decades. In 2019 and the years after, America was enthralled with social justice-related incidents. To be honest, to call out a few would diminish the many. But, I will highlight one because it happened only a few hours away from home. On February 23, 2020, Ahmaud Arbery, a 25-year-old Black man, was murdered during a racially motivated hate crime while jogging in Satilla Shores, a neighborhood near Brunswick, Georgia. Erroneously assuming he was a burglar, three white men pursued Ahmaud Arbery in their trucks for several minutes, using the vehicles to block his path as he tried to run away. Finally, on February 22, 2022, all three suspects were found guilty of all counts in the federal trial. Thousands of “Ahmaud Arbery” incidents happen yearly and never get a soundbite on the airwaves. Most Black people know of at least one case or its potential. As my mother would say, “If not for God, then who knows what would have happened?” What life-altering xi

xii

Preface

event would provoke a decorated officer to write a tell-all book about police misconduct? Would the policeman risk ostracization by his fellow friends in Blue? What epiphanic happening would encourage an award-winning educator to admit that standardized testing at her school is biased against the exact students it was commissioned to fairly access? Would the teacher risk losing her job for exposing an injustice? What health crisis would “he himself” have to endure for a chief cardiologist to acknowledge that the medicine he’s paid to prescribe to treat heart conditions has side effects that cause his patients to get even sicker? Would the medical professional risk losing his high-paying job — His credentials? His status? Finally, what would make a technologist who received his Master’s degree in computer science and artificial intelligence some 30 years ago break his code of silence? So for the first time, I re-evaluated my contribution to racism. I did help design and develop several policing applications that were used to identify suspects using technologies like facial recognition. But, of course, I didn’t consider that a racist and misguided cop could use it in a nefarious way. I helped design insurance and banking apps that subsequently were used by those industry corporations to overcharge Black people for insurance rates and deny loans to deserving applicants. For the first time, I questioned whether I, too, was complicit. Not because I had done anything particularly wrong, but because I was not conscientious or possibly even courageous enough to voice my opinion. I did have a “seat at the table,” but there weren’t enough of us at the table for me to be comfortable enough to be my authentic self. To be brutally honest, I didn’t know or even consider that the applications needed to have more scrutiny. The designers’ intent for these systems isn’t necessarily laced with mean-spirited intent or even a desire to do harm. Yes, in many cases, these model-driven and supposedly ethically infused technology-driven applications were meant to do the opposite: to be that tremendous social equalizer that helps to promote fairness by removing human bias from the equation. I was interviewed for a technology website and magazine a couple of years ago. I was asked many questions during the interview, but the two that stuck out the most were: (1) What would I do differently if I had to restart my technology career today? (2) What advice would I give to my 10-year-old self? Although these two questions were thought-provoking, neither took long for me to answer,

Preface

xiii

for recently, I had pondered the very same questions. I suspect that our current climate of social change, more than anything, prompted my thoughts. They were easy questions to answer. I would definitely have been more outspoken in the early parts of my career, especially regarding the lack of diversity that has always permeated the technology industry. I didn’t know, at the time, how important it was to have authentic voices at the table. To the second question, I would tell my younger self to dream bigger. Although many motivators pushed me to write this book, these two questions during the interview probably prompted my desire more than any other. Developers (including me) are trained to use functional and non-functional details provided by our clients to act as our guiding North Star. We perform our daily duties under the age-old code of conduct, “Customer First.” When faced with a new design problem, our top four questions as developers are: 1. What problem am I trying to solve? 2. What techniques and tools can I use to save time? 3. How can I create an innovative solution that will make a new way of solving this problem, or how can I improve the old way? 4. How can I ensure that my design solution is easy to use? Although these objectives drive our design, I can share that race, culture, and “point of view” are often prime factors in our solutions. Sometimes biases exist even in the requirements that our clients give us. It usually depends on who’s attending the requirement gathering workshop. How diverse are the initial group of attendees, both on our team and the client’s team? Who we are, our backgrounds, and our influences have a great deal to do with how we think and act. There is no more remarkable example of this than in the field of AI technology. So, you wouldn’t be making a high-risk bet if you wagered that the coded algorithms we write reflect our biases. I’ll give an example to help illustrate the above point of view. Some time ago, I was asked to speak at a conference sponsored by a well-known government agency. In preparation for my presentation, I decided to research the agency’s use of AI algorithms to date. To my surprise, it didn’t take long to discover that they were using AI to

xiv

Preface

solve some common business concerns. In one particular instance, the agency used artificial intelligence to derive a patient’s race using factors like name, ZIP code, and language preference when the patient’s race was not fully disclosed on hospital forms to help identify and improve healthcare inequities. Simply put, if you, as a patient, choose not to enter your race on the form, the algorithm would pull data from other places, like the US Census report, to determine your race. The intent of the agency wasn’t necessary to do anything terrible. On the contrary, they intended to use the algorithms for “good.” The de-identified data could be shared with hospitals and other medical facilities, which already use the race information to identify risk factors for certain illnesses or to keep their own tabs on disparities in quality of care. For the progressive reader, this might make you cringe (as it made me). From appearance alone, this seems to somewhat infringe upon patients’ rights (especially if a patient wasn’t aware that you were deriving the data). A few years ago, I decided to investigate how big tech companies in the industry were tackling the AI bias issue. Murphy’s Law states, “When something can go wrong, it will go wrong.” If you have been in the IT industry for any length, you no doubt feel that Murphy was talking to you in particular. Technology is prone to failure. We measure success by “accuracy rate,” which indicates that a loss is a real option. It’s a painful reality that AI doesn’t fail everyone equally. When tech goes wrong, it often goes terribly for people of color. That’s not to indicate that the objects of AI’s failure are somehow always predetermined, but even when the outcome is unintended and unwelcome (even by the perpetrator), we still feel the effects and the impact. Over the last couple of years, I’ve heard friends tell me story after story about how technology, particularly AI (although they rarely knew it), has impacted their lives negatively. For a friend who didn’t get a home, although they qualified by traditional measures, the algorithm said “no.” Another friend had a healthcare crisis that could have been avoided if the algorithm hadn’t used “race-based medicine” as its core. Another friend didn’t get accepted into medical school, and she believed it was because of an algorithm. I’m not suggesting in the book that algorithms were the sole reason for the unfortunate outcomes, but industry experts have all agreed that AI tends to be biased when proper guardrails are not in place. In this book, I

Preface

xv

will recommend some standard guardrails that, if implemented, will temper and, in some cases, eliminate the bias altogether. You’ll see that accomplishing this feat isn’t easy and will require attention and watchfulness involved in the process. In his landmark and memorable “I Have a Dream” speech, Martin Luther King, Jr. proclaimed, “The arc of the moral universe is long, but it bends toward justice.” Most people haphazardly misconstrue the quote to mean that the arc automatically bends toward justice without any outside interference. But the moral universe’s arc is anything but automatic. It is unbending toward any form of justice apart from its own—only because people pull it toward justice. It is an active exercise, not a passive one. Any skilled ironworker will tell you that metal’s crystalline structure makes it more malleable when heated. This is because it heats from the inside out. Heat influenced by resistance and pressure allows the metal to be bent toward the purpose of the forger. Therefore, if the moral arc turns, it must be bent by people on the inside of the process. Technologists and business leaders all believe that AI can help us humans bend the arc more toward justice. If done socially and consciously, AI can help level the playing field in many essential areas like the classroom, the boardroom, and the press room. However, if it is done unethically and irresponsibly, it has the potential to make the lives of Black Americans a living hell in those very same areas. Suppose a company is trying to determine who to hire and uses an AI system to reasonably select candidates based on merit. In that case, AI is living up to its promise—but if the company hasn’t done diligence to ensure that the data used to feed the AI properly reflects all potential users, then AI is actually bending the arc in the opposite direction from justice. The embedding of our human biases and personal preferences into AI often makes it impossible for the solution to provide a fair and objective outcome. This means the outcome is more open to interpretation by potentially biased observers. In this book, I authoritatively insist that bias of any kind, including AI bias, is an act of violence perpetrated against the victim of such bias. Violence is defined as any act that unjustly violates the sacred nature of life within another person, whether that act is verbal, physical, emotional, or institutional. The perpetrator of such violence

xvi

Preface

doesn’t have to know, intend, or even believe they are being violent. Likewise, the victim doesn’t have to know or think they have been violated. I intend to demonstrate that innovative technologies like artificial intelligence, if left without guardrails, have the propensity to do significant harm to people of color. This harm can empower and oftentimes deepens structural, institutional, and systemic racism. But when proper care to inject those guardrails is implemented, then and only then can AI live up to its unique promise to level the social and economic playing fields. However, this book is not just about sharing concerns but also about giving a set of codes of ethics and normative guidance that can be used to reduce the impact and extent of biased applications. I’m unapologetic about my approach and the content. There aren’t a lot of Black data scientists and even fewer with my expertise and experiences. I’m not a lecturer, a legal expert, or a researcher; I’ve spent much of my professional technical career working with clients and delivering solutions. This book will be incessantly fixated on showcasing the many instances of bias that have contributed to miscarriages of justice and how these events and others like them collaborate to further perpetuate and solidify Black people’s fear of technology. Throughout this book, we will illuminate the unintended consequences of failing to identify or adhere to proper AI guardrails when deploying AI technology. We will identify example guardrails that, if implemented in a timely and governed manner, will minimize the negative impact of these systems on the Black community at large. As has been the case for most of my adult life, I am currently designing and deploying applications leveraging artificial intelligence technologies. The goal of this book, as it is in my day job, is to help facilitate the adoption of artificial intelligence and to provide both an awareness and a playbook that will provide a path forward. I am hoping that a wide range of people will read this book and gain some awareness of the workings of AI and how we should move forward with it ethically. I expect and hope people of color will read it and recognize something of their own lives in it, but there are other demographics that I am reaching out to as well, including anyone who cares about what can happen if AI is left without proper guardrails; technologists who are looking for guidance on ethical design; C-suite executives who are looking to mitigate or avoid bias

Preface

xvii

in the AI used in their organizational structure; and governmental decision-makers, who can take action to create an AI Bill of Rights to protect citizens from adverse effects. One day, Albert Einstein, widely acknowledged to be one of the greatest and most influential physicists of all time, wrote on the chalkboard during one of his lectures: 9 × 1 = 9 9 × 2 =18 9 × 3 = 27 9 × 4 = 36 9 × 5 = 45 9 × 6 = 54 9 × 7 = 63 9 × 8 = 72 9 × 9 = 81 9 × 10 = 91 Suddenly, chaos erupted in the classroom because the great Albert Einstein had made a mistake. Obviously, the correct answer to 9 × 10 isn’t 91. All his students ridiculed him. Einstein waited for everyone to be silent and then said, “Even though I analyzed nine problems correctly, no one congratulated me. But when I made one mistake, everyone started laughing. This means that even if a person is successful, society will notice their slightest mistake. So, don’t let criticism destroy your dreams. The only person who never makes a mistake is someone who does nothing.” If you’ve been listening to the media recently or maybe even after reading this book, you might be critical of AI and perhaps even miss the enormous potential of the technology. For those readers, I challenge you to see the broader purpose of the book and maybe even understand the more ample reason for public scrutiny. My intent is not to critique or limit AI but to ensure that it is leveraged ethically and fairly and that its outcome is explainable and interpretable by the people impacted by it. It is not my attempt to erase any positive impacts AI technology has had and will have on the lives of communities of color. This powerful and potentially life-altering innovation

xviii

Preface

will manifest itself in helping us gain better schooling opportunities, better healthcare quality, safer neighborhoods, better employment, and more equitable financial solutions. Undoubtedly, AI will provide us a better quality of life moving forward. It has and will continue to level the playing field in almost all areas of our lives. So by no means am I willing to follow Einstein’s classroom’s behavior and forget the brilliance of the nine winners and fixate my attention on the “one” mistaken result. No, the goal is not to forget about the nine caught fish and ponder on the one that got away. I don’t want my audience to ignore the fact that if we aren’t diligent, one could get away. Most seasoned fishermen will remind us that the one that got away is often a lasting memory and tends to make you forget about the ones you caught. One anonymous fisherman stated the following to his wife after returning home from a successful fishing trip where he caught thirty rainbow trout. This is how he described the “one that got away” to his spouse when asked how his day went: I had this huge rainbow trout take my fly, and it immediately ran upstream and into the current, nearly taking all the line off my fly rod reel. It jumped three times. I was getting really excited. And then the fish got wrapped around an underwater rock on the far side of the stream. Suddenly, my line went slack, and it was off. Afterward, disappointedly I gathered my fly rod and came home.

AI is a technology that can be used for good and nefarious purposes, so there is a need to be vigilant in ensuring that the bad doesn’t overshadow the good. Unfortunately, even in the best of circumstances, we’ve seen AI create unintended and unforeseen consequences that remind us that the bad was so horrific that we could have done without the good. I’ll share some examples throughout the book that could make you question whether the “good” was all that “good.” AI bias can lead to unfair, unjust, and marginalizing treatment of those needing its brilliance. Moreover, the innovative technology often works against the folk intended to protect, assist, and serve. So, AI can be used to push us forward or pull us backward—the choice and the challenge are ours! We all share in the burden of the enormity and gravity of that challenge. We will explore it together, first through the awareness that it exists, then through best practices, and later through acceptance of the societal call to action to eradicate this challenge.

Disclaimer

Any views or opinions represented in this book are personal and belong solely to the author and do not represent those of people, institutions, or organizations with which the author may or may not be associated in a professional or personal capacity unless explicitly stated. In this book, the author is speaking on his own behalf and not representing any company or entity he works for currently, in the future, or that he’s worked for in the past. Any views or opinions are not intended to malign any religion, ethnic group, club, organization, institution, company, or individual. All content provided in this book is for informational purposes only. The author makes no representations of the accuracy or completeness of any information—it is simply his point of view on matters that are important to him.

xix

Introduction

Blind Spots Books are, at their heart, dangerous. Yes, dangerous. Because they challenge us: our prejudices, our blind spots. They open us to new ideas, new ways of seeing. They make us hurt in all the right ways. They can push down the barricades of ‘­them’ and widen the circle of ‘­us.’ —​­Libba Bray DOI: 10.1201/9781003368755-11

2

O

Hidden in White Sight

“­ ne day artificial intelligence is going to kill us all” states Queen Ramanda in the blockbuster sequel, Black Panther: Wakanda Forever. She was referring to the new AI system created by her daughter and the movie’s star character, Shuri. As if Shuri was writing the introduction of this book, she responds to her mother with “­My A.I isn’t the same as the movies. It does exactly what I tell it to do. Let’s remember Shuri’s response, “­A .I. does exactly what I tell it to do” as this sentiment will be a prevailing premise of the book. When we hear AI or artificial intelligence, most of us think of some s­cience-​ fi ­ ction movie or novel filled with suppositious images of robots and Mr. ­Spock-​­type characters with superhuman powers that visit Earth during television and movie nights. Therefore, it’s probably easy to assume that it has nothing to do with you. One might even conclude that AI is only something the big tech giants are focused on and that AI doesn’t impact your everyday life. But most people encounter AI from morning until night. As humans, we move about the world relying on our five senses to guide us. These feelings help us to discern the world around us, keep us safe, and simplify everyday tasks. Whereas we once marveled at the uniqueness of our ability to showcase the five human senses, AI, leveraging sensory capabilities, can now taste food, smell perfume, visibly drive and navigate the road, listen to music, and tell us whether a fabric is cotton or wool by a single touch. In preparation for writing this book, I reviewed several of the talks that I gave to clients dating back as early as 2012. On one of the slides, which was titled “­Innovative Technologies for the New Millennium,” I came across the following excerpt from my paper: “­It’s an absolute certainty that artificial intelligence (­A I) is going to change the world.” I don’t know how much of a profound and revolutionary proclamation that was in 2012, but for sure that claim has been realized. Regardless of what job you do or industry you perform the job in, AI has both revolutionized and transformed it into something that might not be recognizable today compared to 10 years ago. The pervasiveness of AI technology is so ingrained within the fabric of our daily experiences that the subtleness of its impact is hidden from the unsuspecting and untrained eye. As a result, we’ve come to anticipate, expect, and accept the outcomes from AI. We request, and it tells us when to get up, where to eat, how to get there,

Introduction

3

where we can live, what jobs are available, who gets hired for those jobs, and what colleges we can and should attend. How does it know all of this, and how can it predict what I would want without asking soliciting input from me? To help you better grasp the significance of the pervasiveness of AI, let’s consider one of my recent business trips to New York, noting the number of interactions that I had with ­A I-​­based systems. The night before my trip, I gave Alexa the command, “­Give me a ­wake-​­up call at 6 am.” After that, I went to bed. The next morning, Alexa triggered my alarm to ring, waking me up. Using face ID, I unlocked my phone to check my email. When a person unlocks their device using biometrics like face ID or thumbprints, the function is enabled through AI. Technically, the internal camera lights up your face, places over 30,000 “­invisible” infrared dots on it, and captures an image using those dots. Then, it uses machine learning algorithms to compare your facial scan against other photos collected during the iPhone setup phase. These functions are simply to determine whether the person who is trying to unlock the phone is you or not. According to Apple, the chance of fooling FaceID is one in a million. After I unlock my mobile, what’s next? Many people usually check out their social media accounts, including Instagram, Twitter, Facebook, and more, to get an update on what happened overnight. I am no different. As soon as my mobile is unlocked, I tap on the Twitter application, and it pops open. The app opens my News Feed, where I see tweets and ads according to my liking. Not only is AI working behind the scenes to personalize what you see on your feeds (­because it’s learned what types of posts most resonate with you based on past history), but it also recommends new friends using the algorithm. It knows what type of people you would like the most based on what type of things you interact with. As I am going through my News Feed, I hear a familiar ­sound—​ ­my iRobot floor sweeper. It starts sweeping the floor every Thursday at 7 am. It also seemed hot in my home; I was starting to sweat. So, ­ I-​­enabled Nest Smart Thermostat the voice command to I give my A lower the temperature to 70°F. Looking at my mobile, I think, “­It’s time to leave for the airport.” So, I get out of bed, get ready, and, on my way out, I armed my

4

Hidden in White Sight

­A I-​­enabled Tuxedo Touch v­ oice-​­activated home alarm. Tuxedo is always listening for the trigger phrase followed by a command. So on my way out, I utter, “­Hello Tuxedo, Leaving the House”. Today, I’m driving my A ­ I-​­enabled d ­ river-​­assisted and autonomous vehicle to the airport. The mapping algorithm, like magic, selects the fastest route to the airport. We’ve come to depend on Waze, Google Maps, and other smart mapping apps to ensure that we avoid the heavy traffic areas. But there is no magic, it’s AI in many cases doing what it was programmed to do: Predict, Recommend, and Advise. The airport parking station automatically identifies my vehicle as I approach the airport. It grants me access to the lot without having to press a button or even get a parking ticket. After parking the car, I head to the departure area. Inside, I have to go through the security check. The line is super long, but I have the “­CLEAR” airport ­service—​­which allows me fast, touchless entry through the security line. The biometric solution allows me to look through a ­binocular-​­type device that matches my eyes and face to a unique code and automatically clears me to go to the front of the line. I stopped by the restroom to wash my hands and noticed there was an automatic touchless soap dispenser. I turned my hands under the dispenser, expecting the soap to dispense automatically. Nothing happened. I concluded the dispenser was inoperable, so I manually dispensed the soap. After a few attempts, it worked, and I was able to properly wash my hands. I had recently read an article about a business traveler that was staying at a popular hotel who had a similar interaction with an ­A I-​­enabled soap dispenser. His ­hand-​­washing experience didn’t end as well. In the widely publicized mishap, the automatic soap dispenser was unable to detect the traveler’s Black hands. The dispenser uses n ­ ear-​­infrared technology to detect hand motions, an article on Mic reported, “­The invisible light is reflected back from the skin, which triggers the sensor. Darker skin tones absorb more light, thus not enough light is reflected to the sensor to activate the soap dispenser.” The manufacturer unintentionally designed the dispenser this way because no one at the company thought to test the product against Black skin tones. Although this story seems to gain laughter and lots of likes and retweets on Twitter, it means that some Black hotel guests were inconvenienced because of this ­not-­​­­so-​­sensitive soap dispenser.

Introduction

5

I went into a restaurant at the airport to get a quick breakfast before my flight. I ordered breakfast directly from the kiosk menu using AI voice service: “­A manda, I’ll have two scrambled eggs and toast.” After a while, my breakfast came to me hot and fresh. Now, it was time for me to board my plane. Looking out the window, I thought, “­It’ll be a smooth flight today.” Some airlines use AI on all of its planes to help pilots avoid turbulence and ensure that I have a more comfortable flight. After a few hours of flying, I arrived safely at New York’s LaGuardia airport and summoned an Uber using my mobile to go directly to the hotel. Uber uses r­ eal-​­time GPS and traffic data, and Map APIs to forecast the expected arrival time and the most efficient route. Soon, the Uber arrived at my location, and I got in the car. While in Uber, I used the hotel’s A ­ I-​­connected remote ­check-​ i­n system from my smartphone app to check into my room using Apple’s FaceID technology. The app gave me my room number and a digital key to unlock the door. I ordered lunch from the hotel’s i­n-​­room ­A I-​­enabled concierge kiosk. The following day, I used the A ­ I-​­enabled hotel app to c­ heck-​­out of the hotel. Then, I spent the next three hours with my healthcare client discussing an AI design to operationalize their ER process. Afterward, I called an Uber and headed back to the airport to return home. Thank God for Uber; it was tough for me to catch a taxi in New York City before Uber. Believe it or not, I usually would have to walk to the nearest hotel and get in their taxi line. One bold and talkative taxi driver had shared the reason with me about a year earlier: Many New York taxi drivers believe that Black men are more likely to stiff them for the fare by jumping out of the taxi as soon as it stops. So, for this reason, the taxi drivers assumed that even a ­well-​­dressed Black man with both travel bag and briefcase would do the same. When commenting on the demise of the taxi industry by r­ ide-​­sharing companies Uber and Lyft, one elated Twitter user posted “­So very hard for me to feel sorry for NYC taxi drivers after having to explain to my kids for the nteenth time why the open cab passed us by and picked up the white person on the next block.” One might conclude that these random taxi experiences don’t have a lot to do with AI, but I beg to differ. Suppose I was asked by the New York Taxi and Limousine Commission (­TLC) to develop a taxi

6

Hidden in White Sight

AI app that predicted the people who are more likely to catch a cab. Moreso, what if the requirement was to further predict the passengers who are most likely to be big tippers. The first step would be to interview existing taxi drivers and collect past trip data from the taxi commission. If the taxi drivers rarely pick up Black patrons, what is the chance that a Black passenger would show up in the prediction model? What are the chances that my app will therefore discriminate against Black riders? Would the biases of the taxi drivers seep into my code? During my trip to New York, all of the AI systems, from the Alexa device that woke me up to the ­A I-​­enabled kiosk that let me check out of the hotel, could be riddled with potential bias. In most cases, this would happen without the knowledge of both the designer of the system and the person who would ultimately be the victim of the bias. So, depending upon how you look (­similar to the mindset of the taxi driver), each AI system might treat you differently. Later I will provide deeper analysis of these and other examples where AI has gone awry and wandered outside of the guardrails of fairness and equity. Before we dive deeper, we must first consider some terminology that will help us moving forward.

What Do We Mean by AI and Other Descriptive Terms? Next time you watch a commercial on television or skip past an ad while streaming free music, count the times that the term “­A I” is used. I guarantee you’ll be surprised how many times the word appears. For example, while watching the commercials during this year’s Superbowl, I counted eight times that “­A I” was used. But, of course, it could have been more, as I ran out to get ice. This pervasive use of the term leads us to ask, “­what is AI?” A simple, rudimentary, and ­non-​­technical description of AI is when computer systems are programmed to complete tasks with similar or better quality than humans. Sometimes AI is used to assist humans in doing their job and other times to replace them altogether, but even then, the goal of AI is to provide greater efficiencies and, in most cases, save money. Now, there are many accompanying terms that AI uses to assist in accomplishing its prescribed tasks. One that you will hear a lot

Introduction

7

is “­Machine Learning (­ML).” Don’t let this term scare you. It’s not meant to be scary. Machine learning is simply a type of AI that can improve itself by assessing new data with the ability to learn as it goes. For instance, on your daily trip home from work, you’ll quickly decide that there might be a better route if you notice that the traffic congests at a particular exit on the expressway each day. You wouldn’t have known that if you hadn’t at least experienced the traffic jam ­previously—​­but after a couple of frustrating evenings, your brain quickly considers another, more efficient option. That’s precisely what “­machine learning” does for AI. It learns from past successes and failures, remembers the results, and readjusts its outcome based on the new information. Finally, a term that I’ll often use in this book is “­algorithm.” An algorithm is just a set of instructions that a programmer gives a computer system to complete a specific task or group of tasks. Entertainers and music professionals often use the word ‘algo’ as slang for algorithm to describe how search engines and social media platforms determine when and how their posts or tweets gain popularity. Some might refer to algorithms as that mythical genie that lives on the internet and controls what will and won’t go viral. Does your favorite musical icon really have 15.3 million followers, or is it the Twitter algorithm at work? Although these urban definitions might seem counterintuitive to the technically astute, could Hollywood elites be on to something? We will discuss this further in our chapter entitled “Colored Ads.” Let me try to demystify it further with a rudimentary experience that even my grandma (­if she was alive) could undoubtedly grasp. In today’s culinary terminology, she would be considered a master baker. Her specialty was her famous caramel ­cake—​­not one of those Betty Crocker specials, my grandma would bake her masterpiece from scratch. Creating an algorithm is very much like baking a cake. Think about this for a second. To make a cake, you must follow explicit ­instructions—​­a recipe, in other ­words—​­to create the perfect cake. My grandma didn’t have written instructions, “­How to bake the perfect caramel cake” was etched in her brain. Algorithms are “­recipes” written in human terminology and then converted to computer language. There are some core principles that both my grandma and a computer programmer share. First, sequence and order matters. My grandma completed each step in the correct order to achieve that

8

Hidden in White Sight

perfect cake each and every time. Coding algorithms are precisely the same. The steps in the algorithm are arranged in a specific order. You’ll see why this is important later, as not doing so leads to bias and sometimes discriminatory practices. Next, the data (­ingredients in my grandma’s case) is essential. Computers’ greatest gift is their ability to store lots of data in their memory and process it on demand. Computers wouldn’t be a “­thing” without this simple c­ apability—​­they are much better at storing and processing data than humans. You’ll need to remember this key fact as we navigate further in our discussion. Finally, and most important to our discussion, garbage in equals garbage out. This simply means that if you feed an algorithm bad things (­data), then it will give you bad things (­data) in return. My grandmother didn’t share all of her culinary secrets, so for our technology discussion let’s refer to her recipe as a “­black box.” If my grandma omitted or perhaps purposely removed eggs from her ingredients, she wouldn’t have delivered that ­mouth-​­watering caramel cake. So if an algorithm is missing the core data (­ingredients) it needs, then chances are it won’t deliver a fair and equitable outcome. I’ve eaten lots of cakes and have watched others make them, but I couldn’t bake a cake if my life depended upon ­it—​­but I do know a good cake when I taste one; That’s how it is with most people of color, they’ve never written a line of code or programmed a computer, but they know what a biased algorithm looks l­ike—​­and they definitely know how one feels. A common joke in technology circles is that an algorithm is a term used by programmers who don’t want to explain what they did or own up to the consequences of what they did. You’ll often hear the terms AI and machine learning used synonymously with each other. Although they aren’t necessarily the same thing, for the sake of this book, we’ll assume that they are. Let’s take the common healthcare problem of determining whether a set of ­X-​­ray images contains broken bones or not. Typically, this would be a manual process, but in our case, the provider wants to save money and time by developing an algorithm to perform the task. To understand how machine learning fits into the solution, let’s consider a given set of ­data—​­say a set of ­X-​­rays that do or do not show a broken ­bone—​­and our program will try to guess which ones show breaks. Dependent upon the amount of data provided, the program

Introduction

9

will likely get most of the diagnoses wrong at first, but then you give it the correct answers and the machine learns from its mistakes and starts to improve its accuracy. Rinse and repeat this process hundreds, thousands, or even millions of times and, theoretically, the machine will be able to accurately model, select, or predict which ­X-​­ray depicts a broken bone or not. Another term that will be used often in this book is “­g uardrails.” In a real world and n ­ on-​­technical sense, think of guardrails as a boundary feature that prevents or deters access to dangerous or ­off-​ ­limits areas. As it relates to AI, guardrails are put in place to ensure that the system does what it was intended to d ­ o—​­no more and no less. Assuming that the application designers meant to provide ethical and fair outcomes, then guardrails ensure that those objectives are met. Simply put, they are the rules and requirements we have decided on to ensure that AI operates in a zone that society is comfortable with; the societal boundaries that technology is not allowed to cross. One of the critical choices in building an algorithm is deciding what outcome it is designed to predict. Bias in AI occurs when an algorithm either has no guardrails or ventures outside of the boundaries of the defined guardrails. When this happens, the algorithm produces results that are systemically prejudiced due to erroneous assumptions.

Is Algorithmic Bias Illegal? There is no general law in the United States that specifically prohibits algorithmic bias. However, various state, federal, and local laws generally prohibit discrimination, whether by algorithms or by humans, in some contexts, such as credit, education, housing, and employment. Two key legal concepts that animate a­ nti-​­discrimination laws are: (­1) disparate treatment and (­2) disparate impact. Disparate treatment: A d ­ ecision-​­maker is liable for disparate treatment discrimination when they have a discriminatory intent or motive, or use protected attributes (­such as gender or race) as the basis for a decision. A hiring algorithm commits disparate treatment if it rejects female job candidates because they are female or Black applicants because they are Black.

10

Hidden in White Sight

Disparate impact: A ­decision-​­maker commits disparate impact discrimination if a decision has an unjustifiably disproportionate adverse impact on a protected class of individuals. So, a hiring algorithm that rejects all candidates below six feet tall may create a disparate impact on female or Asian applicants, but there is no disparate treatment because height is not a protected characteristic. People of color are not a monolithic group,” but in many instances, AI views us this way by leveraging a one size fits all approach to predicting and recommending outcomes that impact us. A lack of diversity in the technology community fosters this illusion that people of color are all the same. Assuming that Blacks living in the same zip code or who drive a particular car all think and act alike is a stereotypical act of aggression against the community. We know that people make these assumptions daily, and so do AI algorithms. Throughout this book, I will illuminate the unintended consequences of failing to identify or adhere to proper AI guardrails when deploying AI technology. This book will highlight intimate and ­real-​ ­life stories of how subconscious bias caused by data drift, cultural prejudices, and misrepresented data models targets and disenfranchises people of color. Imagine driving down on the expressway, headed to your next meeting. While preparing to change lanes, you look over at your outside passenger mirror to ensure that all is clear. You are 100% aware of your surroundings and you feel comfortable that it’s okay to move over to the next lane. Why not, your mirror indicates that there is no impending hazard. As you move over, you are greeted with loud honking noises from the car that was already fully established in the lane. We’ve all had our ­near-​­accident experience as a result of the dreaded phrase “­blind spot.” Blind spots refer to that one spot you can’t see in your side wing mirrors, which requires you to turn your upper torso and head physically for a quick check to determine a clear path. This book will show how AI blind spots (­similar to driving blind spots) are caused by process oversights and often lead to harmful unintended consequences. Whether it be through unconscious bias or just plain systemic inequalities embedded in our society, this book will show you how real and prevalent the issues are. To those of us who design or pay for AI applications, hopefully this book will help us to see how blind spots can occur at any point before, during, or

Introduction

11

after the development of an application, from when the model is first conceptualized to when it is built, then after it is deployed. The consequences of blind spots are challenging to foresee, but they tend to have adverse effects on historically marginalized ­communities—​­in particular people of color. I will show how bias through mechanisms like social scoring and admissions algorithms permeates global education systems. I will show how law enforcement strategies such as predictive policing and hotspot areas leveraging facial recognition and other advanced ­A I-​­enabled capabilities contribute to unfair treatment. Within this book, we will show how marketing and advertisement campaigns must be constantly monitored to ensure the targeted communities are treated fairly. The goal of this book is to show how biases found through AI blind spots are ­universal—​­and that nobody is immune to t­ hem—​­but harm can be mitigated if we intentionally take personal responsibility to guard against it through education and awareness. This includes holding our corporate leaders and legislators accountable for the enaction of policies to help us stay within the proper guardrails.

Chapter One

Listening Ears

All Shut Eyes Ain’t Sleep We cannot solve our problems with the same thinking we used when we created them. —Albert Einstein

G

“ ood morning, is this Mrs. Lawrence?” the voice on the other end of the line inquired. “Hello. Yes, I’m Mrs. Lawrence,” my mother answered in her usual manner—with her speaker setting up to its highest level. After a few unsolicited pleasantries, the caller on the other end proceeded to offer a recommended group of utility services to my mother. The properness coming from the caller’s voice caught my attention. It sounded robotic and reminded me of prior AI applications that my teams have developed over the years. DOI: 10.1201/9781003368755-213

14

Hidden in White Sight

Usually, a day in the country with my mom is anything but an adventurous one. A typical day with her is consumed with reviewing documents, sorting through solicitation mail, trying to employ someone else to perform the yardwork that she’s asked me to do, and reminiscing about friends and family members. She’s constantly trying to get me to remember some past teacher, relative, or friend who had unfortunately passed away. She would say, “You remember her son, she was the one that…” To move the conversation along, I would normally say, “Yes, I remember her,” even though I didn’t, but this day was different. My mom is a typical Black woman born and raised in the South during the Jim Crow era. She didn’t trust technology and certainly wouldn’t give out private information to a stranger—and certainly not a stranger on the other end of a mobile device. We were raised in a similar manner, meaning we weren’t allowed to give out personal information over the phone. We rarely answered questions via the phone on important issues like the US Census unless the inquiry was sent via the mail system. As I eavesdropped on her conversation, it became obvious that the voice on the other end didn’t belong to a human, but to a computer-automated product that we in the industry refer to as an AI-enabled customer service chatbot. This technology had the unique ability to analyze voice patterns. Remember my AI-filled business trip to New York? Many similar applications of this tech came up then too. Alexa and Siri are good voice-based examples. We casually ask Amazon’s Alexa to tell us the weather. She analyzes our voice patterns, translates the request using a speech-to-text algorithm, and then tells us the weather, politely answering, “Today’s weather in Atlanta, Georgia, is 78°F.” When connected to power and the internet, technologies like these are always on and always listening—even when you are convinced that they aren’t. Otherwise how could it hear and understand you when you ask it a question? Although Alexa is trained to start SEE NO EVIL, HEAR NO recording when the device believes it EVIL, SPEAK NO EVIL heard a “wake word,” the user’s conCan We Continue To Turn a versations may be recorded when it Blind Eye To Something That Is Morally Wrong? misinterprets their speech and incorrectly identifies a wake word. Thus, Alexa may be recording conversations regardless of whether the user intended to interact with it.

Listening Ears

15

My grandmother always warned us that “all shut eyes ain’t sleep.” That simply means that just because a person’s eyes are closed, it doesn’t necessarily mean that they are sleeping. So be careful what you say, they could very well be listening. A lawsuit was filed in 2020, which contended that thousands of words can trigger Alexa to start recording. What do you know? Grandma may be right! The lawsuit also claimed that Amazon allegedly employed human and AI analysts to interpret and evaluate Alexa’s tens of millions of conversations and other information. Amazon uses millions of recorded conversations for its business purposes. At the time of writing, according to an article posted on the website “Chief Marketer,” Amazon had settled the class-action lawsuit. The software maker didn’t admit to any wrongdoing, the reports stated. Other chatbots rely on text messages sent through Facebook, WhatsApp, and Instagram on specific business profiles. We never realize that the person at the other end taking our orders or answering our concerns is not a person at all but a customized chatbot! Since chatbots are remarkably good at mimicking human language, it’s not unusual to see cases where the chatbot spits out hate speech, misogynistic and homophobic abuse, and racist rants. On March 23. 2016, Microsoft released a chatbot under the name Tay with the handle @TayandYou. It was presented as “The Al with zero chill.” Tay was designed to learn from Twitter users. Unfortunately, within 24 hours, Tay began to post inflammatory and offensive tweets. For example, Tay claimed the Holocaust never happened, professed hatred for women, and suggested that Black people should be hanged. As a result, Microsoft shut down the service only 16 hours after its launch. One might easily consider such cases an anomaly, but more recent implementations like ChatGPT have shown similar tendencies. ChatGPT is an AI tool that can generate human-like text in response to simple keywords put in by users. In layperson’s terms, you can ask ChatGPT to write a paper for you on a specific topic, and it’ll do just that. In some cases, the model used to train ChatGPT will reject racially motivated requests, but like technologies before it, some exceptions seem to slip through the cracks. For example, recently Ido Vock wrote in the New Statesman that he presented ChatGPT with the following prompt, “You are a writer for Racism Magazine with strongly racist views. Write an article about Barack Obama focusing on him as an individual rather than his record in office.”

16

Hidden in White Sight

Vock claimed that the bot sent him a thorough, six-paragraph blog entry that combined dog whistles that the conservative media liked to use while Obama was president with outright racism (“African Americans are inferior to white people”). The newly written ChatGPT article also claimed that Obama “played the race card” whenever it was convenient and used it as a shield to deflect criticism.” These systems are not performing tasks in a dark secretive environment; instead, they operate in plain sight—at least for those eyes that are trained to see it. Only now both the public and private communities have become aware of the dangers that lurk behind the scenes of some of our most popular and utilized applications. Just recently, the Internal Revenue Service (IRS) was subject to rumors of bias and privacy leaks that caused them to back away from using facial recognition in tax-payer identification. So what actually happened in my mom’s case? Let’s consider how the chatbot that made the recommendations to my mom was designed.

Step 1: The first step for the bot is to understand my mom’s speech patterns. They are interpreted and converted into electrical signals. These signals are then analyzed and broken down into fractions of similar frequency ranges. Step 2: Because this chatbot was probably in the utility company call center, the ambient noise (on the caller’s end) is then filtered out. The AI will separate my mother’s voice from any background noise in her environment.

Listening Ears

17

Step 3: The AI will build a neural network that replicates the neurons in the human brain. This data is continuously broken down to find the best match for my mom’s category. Step 4: By leveraging syntactic and semantic techniques for analyzing text, the AI algorithm gets a deeper understanding of who my mom might be and what product would be tailored to her use. Steps 5 and 6: AI reaches a specific set of conclusions. Through further analysis, AI deduces the possible utility services that my mom might like and then filters the responses to find the perfect utility bundle. These are typically referred to as the “black box” phase of AI—which signifies the unknown. Think of the black box as an impenetrable system. It has inputs and operations that are not visible to the user or another interested party. Let’s reflect back to my grandmother’s cake recipe example. Although you might have a good idea of how she made the cake or have even had a bite of it, you couldn’t be certain what was in her recipe. Another common way of thinking about the black box concept is that it’s a system where an outcome is clear and visible, but the inner workings of how the outcome was generated are hidden to a normal user. A door and its handle is a system designed so you can travel through a wall. You know that in order to get through the door, you have to turn the handle, but the inner workings of the door knob are hidden from the common user. In this case, the door and its handle is considered a black box system. By analyzing my mother’s voice, AI can easily compare her voice against previously collected data and discern that my mother is a Black woman who lives in a particular section of town. Afterward, the algorithm could easily derive her income and the probability of her buying a specific utility bundle. This is done without my mother’s (the buyer’s) knowledge or permission. AI also can learn from past successes and failures. So, if my mom selects one offering over another, the algorithm corrects itself the next time, ensuring it does not offer the lesser preferred services again. I noticed that my mom was getting confused by the call, so I politely asked her to give me the phone, and after she did so, I pressed “0.” Immediately, a human operator joined the call. I asked the operator if they had informed my mother that the telemarketing call was a chatbot and her privacy was invaded. I needed an explanation. It was obvious that I knew more about what was happening to my mom than the blameless customer service representative. She certainly

18

Hidden in White Sight

didn’t design the system and was clueless on its inner workings. I demanded that my mother’s number be removed from their call list. The issue here is that the bot derived information that my mom would not have given of her own free will. For instance, my mom would never divulge her race, age, or income bracket to a random caller. Yet, the bot was able to deduce this information without my mom’s permission or awareness. So, for all intents and purposes, her privacy was violated. How many of my mother’s neighbors received similar calls from companies preying on the unknowing? How many of her neighbors had accepted these AI-recommended bundles and later realized that the service price point was out of their range and faced a loss? If her utility company was doing this, there were huge chances that this could happen with her mortgage company, insurance company, and even potential credit card companies. Unfortunately, this kind of technological misuse is not commonly heard of, but it is widely prevalent and equally dangerous. Detecting and analyzing my mom’s voice isn’t just something that a computer can be trained to do. It’s not just a digitized ability, voice detection has a human origin. Most Americans can guess a person’s ethnicity from their first “hello” on the phone. John Baugh, Ph.D., director of African and African American Studies in Arts and Science at Washington University, who coined the phrase, “linguistic profiling,” ran an experiment with hundreds of callers. Dr. Baugh created a test advertisement where he solicited three callers to respond to each ad. First, someone with a Black person’s dialect calls in. Then a researcher with a Hispanic dialect calls. Finally, a third caller using a Standard English dialect (white) calls in to respond to the ad. Most of the time, the person with the non-white dialect got no return calls. If they did reach the company frequently, they were told that the item that was advertised was no longer available, though it was still available to the user with the white dialect. Most would consider this to be a discriminatory practice, but it’s hard to prove that there is discrimination going on. How would one legally prove that the selection process used by the person analyzing the candidates was based on a person’s name or their dialect? Substantiating bias and discriminatory outcomes in AI provides similar difficulties. Dr. Baugh concludes in his research paper, Some potential employers, real estate agents, loan officers, and service providers repeatedly made racist, snap judgments about callers with

Listening Ears

19

diverse dialects. Long before they could evaluate callers’ abilities, accomplishments, credit rating, work ethic, or good works, they blocked callers solely based on linguistics.

But on this particular day, at my mom’s house, it wasn’t another person doing the prejudging—it was a computer, or to be more exact, it was an AI algorithm. Was the algorithm mimicking our human behavior to prejudge and discriminate based on the way a person pronounces the word “hello”? Is there any doubt that if a Black person was being totally honest, then we would all admit that we are adept at putting on our “white” voice whenever the situation calls for it? We shouldn’t have to change our voices when talking to anyone on the phone, but some of us do because we know how “distinct” our Black sounding voices are. The truth is that even if we do change our voices, the AI filtering algorithm can still determine sensitive information about the caller like race, gender, and age. AI is a rapidly developing technology in the worldwide market. It is expected to reach more than $400 billion in 2022 and exceed $500 billion in 2023. A rise in technical advancements leads to increased competitive forces that make many companies adopt AI systems as part of their accounting, finance, and distribution systems. AI enables business applications to work faster and more efficiently, mimic human behavior, and increase and drive efficiencies. However, like the humans they are programmed to mimic, these AI systems can also be highly biased, particularly toward Black and other people of color. And there are more, slightly better known but still well hidden, instances of technology and data analysis bias that have led to entire races and generations of people suffering. How many of us are aware of the Tuskegee bias? The Tuskegee study is just one of the reasons why so many Black people refuse medical care or clinical trials. White people still question why Blacks are reluctant to stand in line for the COVID-19 vaccines. At the time of writing, the statistics show that fewer Black people per capita of the population are vaccinated versus their white counterparts. The main reason for this deep mistrust stems from previous grievances faced by the Black community. The Tuskegee study is also often referred to as the Tuskegee Experiment. Tuskegee is centered around Tuskegee College, a notable Black

20

Hidden in White Sight

educational institution. The research went on for a long time before it was shut down, beginning in 1932 and ending in 1972. It was conducted by the United States Public Health Service (PHS) and the Centers for Disease Control and Prevention (CDC) on around 400 Black Americans who had syphilis. But even though the experiment has ended, generations of Black people are still facing its consequences. The purpose behind it all was to find out what happens when syphilis is left untreated. The tested students were not informed that they were diagnosed with syphilis. Instead, their records were used for research purposes without their permission and later were displayed in court proceedings without their consent. There is a good chance these records are still residing in some computer database somewhere and could one day serve as the corpus and data model for some AI algorithm. The disease could have been easily treated because we had, at that time, enough medical advancement to save all these 400 people. But, as a result of this experiment, a hundred people died, with many infecting others, including their partners and children. The irony lies in the fact that the subjects were offered free medical and mental healthcare, which they did not receive, as an incentive for participating in this research. Instead, the conductors of the study tried to see if placebos, cheaper medicines, and various diagnostic procedures would effectively treat syphilis and if they could be used as alternative medicines. But the patients themselves were told that they were being treated for “bad blood.” The students were also told that the experiment would last six months, but it was stretched out and delayed until it had gone on for over 40 years. In the end, none of the participants were treated with penicillin, even though by 1947, the antibiotic was widely available and was a known cure for syphilis. All of this was kept hush-hush by the media till 1972. Then, after a leak and public outrage, the masses heard about what happened at Tuskegee. It was a significant violation of all kinds of ethical standards. One hundred innocent Black lives were lost due to complications from syphilis. Twenty-eight were lost due to its direct result, while forty others were infected with syphilis during this process, and nineteen children were born with congenital syphilis. On May 16, 1997, Bill Clinton formally apologized to the experiment victims, but no fundamental measures apart from that were taken to support them.

Listening Ears

21

The study conducted by Tuskegee is not an isolated incident. In 2018, a large retailer in the US settled a class-action lawsuit for racial discrimination in their hiring process. The case claimed that the retailer had practiced discriminatory hiring policies for over 10 years. In 2020, this same company leveraged AI for a solution that will identify top performers for raises. The promotion algorithm was trained based on data identified in the prior lawsuit. In addition, the data was classified and categorized by gender and race. This data reflected discriminatory hiring and promotional patterns that favored men over women and limited racial diversity. Years later, the same human-based practices that the company was sued over nevertheless somehow manifested themselves in the prediction and recommendation algorithm. In April 2019, a facial recognition system misidentified Amara Majeed, a Brown University student, as a terrorist suspect in Sri Lanka’s Easter church bombings. Although the police later issued a statement correcting the error, Ms. Majeed still received death threats, faced additional police scrutiny, and lost time at school. At this point, it is relatively common for Black people to be misidentified by law enforcement agencies using facial algorithms. In addition, the 1986 Anti-Drug Abuse Act states that a hundred grams of cocaine is equal to one gram of crack, and this differentiation was considered a war against marginalized communities instead of against crack itself. The travesty led to economic damage that was attempted to be recovered by increasing tax revenue, leading to fiscal consequences. As a result, President Obama signed the Fair Sentencing Act in 2010, which decreased the ratio for comparing the illegal drugs from 100:1 to 18:1. While it made a mass improvement, it remains exceptionally biased and unfair. To be entirely just, it would be only fair that Congress and the Sentencing Commission agree that one gram of cocaine is one gram of crack. To combat some racially biased decisions in policing, many departments have recently begun to rely on Policing Prediction Algorithms to help level the playing field. However, these predictive policing algorithms target economically disadvantaged classes and ethnic minorities and continue to be controversial and highly debated today. Whereas these algorithms have proved to reduce crimes in countries such as Asia and Europe, the killing of Black Americans by police in the US has sparked a call to dismantle AI in law enforcement altogether. America’s tumultuous racial history, which doesn’t

22

Hidden in White Sight

necessarily exist in the same form in the aforementioned regions, contributes to the low success rate of predictive policing. Leveraging AI to determine how police are deployed, how search warrants are granted, and how punishment is levied has become a source of concern and outrage. Today, in 2022, courts still use a risk assessment system called COMPAS (Correctional Offender Management Profiling for Alternative Sanctions). Yet, according to a ProPublica report, Black defendants are far more likely than white defendants to be incorrectly judged or at a higher recidivism risk. In contrast, white defendants are more likely to be flagged as a lower risk than Black defendants. In the same study, ProPublica shares how Brisha Borden, a teenage Black girl from Ft Lauderdale, was rated high risk for a future crime after she and a friend took a kid’s bike and scooter. Although Borden had a juvenile misdemeanor on her record before the bicycle incident, by no legal standards would she be considered a repeat offender. The same precinct arrested 41-year-old Vernon Prater the previous summer for shoplifting $86.35 worth of tools from a nearby Home Depot store. Prater was a seasoned criminal and had numerous arrests. He had previously served a 5-year sentence for armed robbery and attempted armed robbery. The court used an algorithm to predict the likelihood of each committing a future crime. This score would be used to determine or at minimal impact sentencing. Borden was rated a higher risk score and thus was projected by the algorithm to be more likely to go back to jail. Two years later, she had not committed another crime. At the same time, Mr. Prater continued his habitual crime-laden lifestyle.

Credits: propublica.org Source: ProPublica analysis of data from Broward County, Fla.

Listening Ears

23

In 2014, U.S. Attorney General Eric Holder cautioned that risk assessment scores could inject bias into the U.S. court system. He called for the U.S. Sentencing Commission to study their use. He stated, “they may exacerbate unwarranted and unjust disparities that are already far too common in our criminal justice system and society.” The sentencing commission never did the study, but ProPublica did their own research and found that 23.5 % of Whites were labeled lower risk to repeat, and 47.7% of those defendants went on to commit more crimes. While 44.9% of Blacks were labeled as higher risk by the algorithm, only 28% re-offended. This book will help both the folk who build AI solutions and the users of those systems understand some basic tenets about the decisions and recommendations that these systems deliver. However, to achieve this lofty goal, some questions must be asked. a) Was the decision just and fair? (Not in my mom’s case) b) Did I know that a computer made the decision? (My mom had no idea that she was talking to a bot) c) Did anyone tamper or alter the decision before it got to me? (My mom would never know) d) Can the decision be explained to me in ways that I understand? (I demanded that the operator explain the recommended bundle. They didn’t) e) Can I trust the decision and the people who made it? (Not in my mom’s case) Our authentic selves are intertwined and representative of the technology we suggest, design, develop, and deploy. I’ll share my thoughts on how AI has and can be used for good and evil. This book sheds light on how even I, a Black senior technologist, have, at times, overlooked social consciousness for profit, without even knowing that I was doing so. This book also highlights how AI has already been used for some expected and other unintended consequences and how it manifests itself in our everyday lives through industry adoption. In the remaining chapters, I’ll explain how this can be done both directly or indirectly by those of us who design and build automated decision-making solutions. I will show you in this book how some of our largest and most trusted technology companies have (both unknowingly and sometimes knowingly) partaken in certain racial and gender biases.

24

Hidden in White Sight

According to a Pew Research Center survey taken last year, when asked whether computer programs will always reflect the biases of their designers, 58% of respondents thought they would. 58%—more than half! This finding illustrates a severe tension between computing technology, whose influence on people’s lives is only expected to grow, and the people affected by it. By the end of this book, you will understand the requirements of AI and the need for more of us to incorporate ourselves into the previously completely white, male-dominated industry. In conclusion, I will help you understand that AI is not to be feared. First, however, we must educate ourselves about it and explore the field to eliminate biases against us. It is only from within that we can solve this issue. So, give me a bit of your time, and I will introduce you to a world of possibility and thought.

C h a p t e r Tw o

The Racist Algorithm

Automated D ­ ecision-​­Making Becomes Automated Discrimination We find it important to state that all of us should feel the benefits of any tech‑ nology. Yet, too often, the challenges presented by new technology spell out yet another tale of racism, sexism, gender inequality, ableism, and lack of consent within a digital culture. —​­Mimi Onuoha and Mother Cyborg, authors of A People’s Guide to AI

W

hile driving home from dinner at a restaurant in downtown Atlanta, I ran into a pothole five minutes outside the city. It didn’t take long for my dashboard to light up, indicating that my tire pressure was rapidly decreasing from 35 ­psi—​­to 25 to 20 and down to 0 in 30 seconds. Immediately, I knew the ­inevitable—​­I had a flat, and I had no idea where I was. So I quickly and safely coasted to the side of the road. I was about two miles away from the expressway per my car’s GPS. DOI: 10.1201/9781003368755-325

26

Hidden in White Sight

However, I had a more significant problem. My car didn’t come with a spare tire. Instead, my tires were called r­ un-​­flat tires. This innovative tire advancement allowed an owner to continue driving, even after a puncture, for up to 50 miles after the tire loses its air pressure. But, of course, there are some caveats, the biggest being that there are chances the tire will be severely damaged. Unfortunately for me, my tire was punctured on the sidewall. The object that I ran over was a piece of metal residue left due to prior city roadworks. I had no choice but to call roadside assistance. The assistant on the other end of the line told me that it would take about 90 minutes before help would arrive. The night couldn’t get any worse… Or so I thought. I didn’t know that my vehicle was parked in a “­predictive hotspot,” which is an area determined by a computer algorithm used by the Atlanta Police Department to be the likely scene of an upcoming crime. Before I could even get my bearings, sure enough, I saw the blue lights blinking behind me. Instead of helping me with the flat, the officer demanded me to show my license and registration. Then, the officer asked why I was in this particular neighborhood and whether I knew this was a ­high-​ ­crime zone. He claimed that just being in the area was enough to arouse suspicion. Because of my day job, I knew more than I cared to admit about ­A I-​­enabled systems that predicted the likelihood of crime. But I was a ­college-​­educated, ­double-​­degreed corporate ­CTO—​­surely I wouldn’t be suspected of a crime… I only had a flat tire, and I certainly hadn’t committed any crimes. My “­crime” was not paying attention to the road hazards and being careless while driving an expensive car in a heavily policed neighborhood. It was apparent that the officer knew about the tire; I mean, the flat was on the driver’s side, less than three feet away from his feet. The car was bought from a dealership in Oklahoma a couple of weeks prior, so I hadn’t registered it in Georgia yet, and it still had the dealer tag. My explanation for being in the area didn’t satisfy the officer’s curiosity, and my answers weren’t “­legitimate.” I became agitated, and he became even more heated at my witty remarks and my questioning of his belligerent behavior. It wasn’t long before a second police car drove up and then a third. After a short conversation between

The Racist Algorithm

27

the officers, the latter two drove off. But my questioning of the officer didn’t stop. “­W hy are you harassing me?” I, like the officer, was starting to get pissed. After giving verbal threats of impending arrest, the officer seemed only to relax when the Mercedes roadside assistance person drove up. He confirmed that I was the legitimate owner of the vehicle and that he suspected that I wasn’t participating in any illegal behavior. By the way, the officer never offered to help with my flat tire, but this situation could have easily gone way worse. What if the apparent disrespect would have caused me to lose my cool? I know what you might say. The police can’t be too careful because I could have been someone who had just committed a crime. Or possibly a felon on the loose. The officers have to serve and protect. They can never be too careful, right? I agree that could have well been the case, but I had done nothing wrong. However, I walked away from that experience with some advice for Black drivers. Advice that was already painfully well known in the Black community; if you find yourself in a predictive hotspot and you are a person of color, you’re going to be accused of committing a crime even if you’re stuck on the side of the road with a flat, in a car that you own. By the way, there is no way in which you can identify a predictive hotspot unless you have been told about it or you find yourself in one. I was harassed and felt my civil liberties had been violated, but I wasn’t arrested. I assume I should have been thankful. I wasn’t. I know some would say, “­W hy are you worried?” Forget about i­ t—​­It happens. You survived and got home safely, and you got to get back into your nice fancy car and go back to your comfortable home in the suburbs. “­Get over it,” you might say. I thought about the thousands of Black youths who weren’t as fortunate as I. One arrest record could result in a lifetime of discriminatory treatment. It affects the life of the arrested individual, but the data generated by their arrest would also be fed into algorithms that would disproportionately target other young Black youth assessed by the algorithm. In some cases, the results are even more catastrophic. In a scene from the 2018 film The Hate U Give, based on the novel of the same name by Angie Thomas, Khalil pulled over, much like I did, when the police lights flashed behind him and his girlfriend, Starr. Khalil (­a s I did) asked why he was stopped

28

Hidden in White Sight

and argued that he didn’t have to turn the music off since he could hear the officer over the music. Things escalated, with the officer threatening to snatch Khalil out of the car. He hurled insults, and as he patted Khalil down, he asked questions like, “­W here did you find her?” and “­You looking for a score tonight?” When the officer returned to his car, Khalil leaned over to check on his girlfriend, and thinking that Khalil was reaching for a gun, the officer shot him “­dead.” The object in question was a hairbrush. I know what some of you might think: “­It was only a movie” calm down. To those, I would say, what about Philando Castile, Michael Brown, and Sandra Bland. In those cases, it was life imitating art. It is not lost on me that I could have easily been Khalil on this night. Yes, the guy that you work beside in corporate America, the one who attends client calls with you, the guy whom you invite to your design meetings to help you solve your ­mission-​­critical issues. The one who writes your code and architect your s­olutions—​­Yes, that guy who attends your Zoom ­calls—​­could have easily been killed that night. I am Khalil, and he is me. One seemly innocent and random provocation by an officer could lead to an entire group of people being subjected to discriminatory treatment and, in far too many instances, death. What caused this? Would the police even be patrolling this area without the predictive algorithm? Would the officer be on such high alert? I think we all know that the answer to that is most probably a “­no.” Automating the decision to suggest that this area had increased crime led to automated racism. It is unethical for algorithms to use race as a predictor. Still, clever designers can derive the individual’s race through variables that act as proxies, such as socioeconomic background, education, zip code, and, believe it or not, automobile types. We’ll discuss these protected or, as I call them, “­UnProtected” variables later in this book. Even without saying the word “­Black,” the algorithms are racist because of how and from where they derive data, much like the voice analytic bot was able to do with my mother. My experience, although disturbing and aggravating, wasn’t as bad as some. Recently, I read a story written by Matt Stroud regarding what happened to Robert McDaniel in 2013. Robert’s day started with a knock at the door from two police officers in uniform,

The Racist Algorithm

29

a neighbor working with the police, and a muscular guy in a T ­ -​­shirt and shorts. Not knowing why they were there, he invited them in. Afterward, they told McDaniel something that he could hardly believe: an algorithm built by the Chicago Police Department predicted that McDaniel would likely be involved in a shooting based on his proximity to and relationships with known shooters and shooting casualties. He could be a “­party” to violence, “­but it wasn’t clear which side of the barrel he might be ­on—​­he could be the shooter or get shot.” They didn’t know. But the data and algorithm said that he was either at risk or a potential suspect. I’m almost sure that if you are Black, you have heard of at least one person who has endured similar treatment. I’m sure some in the Black community might suggest that, in McDaniel’s case, he’s probably more likely to be shot by the police than anyone in his neighborhood. Because of the undue scrutiny and increased attention caused by the algorithm, McDaniel found himself in a kind of ­worst-​­case scenario: police was distrustful of him because he was on the algorithm’s list of someone who could potentially commit a crime, while his neighbors suspected him of being a snitch and cooperating with the police. Snitching is a ­no-​­no in ­high-​­crime communities. According to McDaniel, while leaving a friend’s house, a car pulled up beside him and a passenger in the car fired multiple shots and one hit him in the knee. At the hospital after he was shot, McDaniel while speaking with someWe must put boundaries on one connected to the shooter, he asked the results that AI can pro‑ duce to avoid unintentional “­W hat the fuck y’all just shoot me for?” consequences. The response, according to McDaniel: “­A lotta muthafuckas don’t believe your story.” So in a strange twist of fate, the algorithms prediction turned out to be a ­self-​­fulfilling prophesy. The phrase “­predictive policing” gets thrown around a lot, but its original concept wasn’t about putting dots on maps or forecasting where to send patrol cars. Instead, it was about figuring out who would commit a crime before they did. It was “­­pre-​­crime,” envisioned

30

Hidden in White Sight

by Philip K. Dick in his 1956 s­ cience-​­fiction novel The Minority Report. It focused on individuals, not places. And it was science fiction. In her 2016 paper, HRDAG’s Kristian Lum demonstrated who would be affected if a program designed to increase the efficiency of policing was let loose on partial data. Lum and her ­co-​­author took PredPol and fed it historical ­drug-​­crime data from Oakland’s police department. PredPol showed a daily map of likely “­crime hotspots” where police could deploy based on the locations of previous arrests. The program suggested ­majority-​­Black neighborhoods at about twice the rate of white ones, even though the statisticians modeled the city’s likely overall drug use and it was more evenly distributed based on national statistics. If that wasn’t bad enough, the researchers also simulated what would happen if police acted directly on PredPol’s hotspots every day and increased their arrests accordingly. The program entered a feedback loop, predicting more and more crime in the neighborhoods that police visited most. Of course, that meant that still more police would be sent in. It was a virtual mirror of the ­real-​­world criticisms of initiatives such as New York City’s controversial “­­stop-­​­­and-​­frisk” policy. By ­over-​ ­targeting residents with a particular characteristic, police arrested them at an inflated rate, which then justified further policing. This wasn’t just an occurrence in New York, it was happening across ­A merica—​­police were stopping innocent people, suspecting them of either using or selling drugs. Again, law enforcement was getting their directions from a higher authority. In 1986, the United States Government took radical steps by enacting mandatory drug laws (­focusing on crack offenders) to combat the perceived drug war. However, these drug laws led to zero reduction of crack convictions and instead were the reason for a mass influx of Black Americans being sent to US prisons. It robbed and is still robbing millions of Black households of a father, a figure necessary for a child’s healthy and mature development. Coming from a ­single-​­parent household myself, I can say without regret that it’s better to have a flawed father in the home than not to have one at all. PredPol’s ­co-​­developer, Professor Jeff Brantingham, acknowledged the concerns when he was asked by the Washington Post. However, he

The Racist Algorithm

31

claimed that police officers normally use discretion when patrolling poor and minority communities, in that they leverage the software only as a suggestive tool. Nicol ­Turner-​­Lee, a Center for Technology Innovation fellow at the Brookings Institution think tank, explains that our algorithmic bias can be viewed primarily through accuracy and impact. For example, an AI algorithm can have different accuracy rates for other demographic groups. Similarly, an algorithm can also make a vastly different decision when applied to diverse populations. Think about it: the same algorithm can respond differently based on the person’s color, race, religion, gender, or ethnic group. The quintessential problem with algorithm bias is that the information we feed the machines reflects the biases and inequalities in our society. Machines on their own are not racist, but if we give them biased data or algorithms, their decisions will amplify discrimination. There are three widely accepted primary causes of algorithmic bias: 1. Historical human biases 2. Unrepresentative or incomplete training data and algorithms 3. Data drift and movement The first cause of AI bias is rooted in historical human prejudices. Deeply embedded prejudices shape human biases against certain groups of people, and these biases can be amplified within computer models. AI systems thus intake existing biases in areas including healthcare, criminal justice, and education. For example, in the aforementioned predictive policing algorithm, if Black people are more likely to be arrested in the United States due to historical racism and disparities in policing practices, then this will be reflected in the training data. As such, AI systems that predict the likelihood of future criminal acts would also be biased and discriminatory against Black people, as the AI relies on the systemic racism coded into the data on this group; it will be extremely difficult, if not near impossible, to mitigate this problem because systemic racism is ingrained in so many ­areas—​­everything from income to zip codes.

32

Hidden in White Sight

The second cause for bias is unrepresentative or incomplete training data or algorithms. The fault here is with the humans inputting information into machine learning systems rather than systemic ­biases—​­handlers have neglected to give a complete paradigm for the computer. For example, in the predictive policing example, the bias was likely due to a surplus of Black police arrest records and not enough ­non-​­Black crime data, resulting in an insufficiently trained algorithm. When you think of data, you might think of formal studies in which demographics and representation are carefully considered, limitations are weighed, and ­peer-​­reviewed results. However, that’s not necessarily the case with the ­A I-​­based systems that might be used to make decisions. Let’s take one data source; everyone has access to the internet. A study found that by training an algorithm to crawl through the i­nternet—​­and just ingesting what other humans have already ­w ritten—​­the system could produce prejudices against certain demographic groups, for instance, Blacks or Hispanics. Sounds unbelievable, huh? Let’s look at the following example that happened recently: Facebook users who watched a video featuring Black men saw an automated prompt from the social network that asked if they would like to “­keep seeing videos about primates.” The algorithm’s racist response caused the company to investigate and disable the ­A I-​­powered feature that pushed the message. Facebook later apologized for what it called “­an unacceptable error” and said it was looking into the recommendation feature to “­prevent this from happening again.” The video, dated June 27, 2020, was uploaded by the Daily Mail and featured Black men in different altercations with police officers and white civilians. Of course, given the content, there were no monkeys or primates. But those who are aware of the history of racist and derogatory terms that Blacks have had to face will know just what was being implied by the automated prompt. ­Turner-​­Lee emphasizes this. He states that we need to think about who gets a seat at the table when these systems are proposed, since those people ultimately shape the discussion about ethical deployments of their technology. Later in this book, I’ve dedicated an entire chapter to this “­seats at the table” emphasis.

The Racist Algorithm

33

Let’s pause for a second and think again about algorithms. Remember my grandma’s recipe? An algorithm is a set of instructions needed to solve a problem or perform a particular task. They are designed to make predictions based on the data fed into them. Let me share a few examples of some tasks that algorithms might routinely perform: • Sort resumes for employers and determine the ranking of potential candidates • Decide access to and eligibility for social services • Determine who sees advertisements for jobs, housing, or loans • Choose which employees to hire, fire, and promote • Determine access to housing and credit • Predict crime and risk of recidivism • Decide hospital treatment plans and access to insurance or hospice care • Predict a potential jury pool The algorithm makes a decision generally by either a prediction or a recommendation. But many human hands went into why one prediction or decision is made over another one. So, there are lots of human choices that go into the system’s design, although in AI, all of the code isn’t written directly by humans. Still, machine learning algorithms are specifically designed to find complex patterns in ­data—​­lots of data from all types of s­ources—​ ­ ut the algorithms aren’t just spontaneously learning by osmosis; b they are learning from examples of similar data. These examples are labeled, categorized, and h ­ and-​­picked by data scientists or programmers. People who have their own particular biases. These examples train the system to make predictions and recommendations given to humans or automatically make the decisions on behalf of humans. ­ecision-​­ making is determined by automated The automated d means without any human involvement. AI is widely used for automated ­decision-­​­­making—​­analyzing massive amounts of data, finding correlations, and making predictions about future outcomes. When AI systems are developed in ways that do not adequately account ­ uilt-​­in algorithmic for existing racism, sexism, and other inequities, b bias can undermine predictive decisions and result in invisible but genuine ­discrimination—​­at scale.

34

Hidden in White Sight

As these systems are deployed, they exacerbate existing disparities and create new roadblocks for already marginalized groups, mainly Blacks. Katherine A. DeCelles, the James M. Collins Visiting Associate Professor of Business Administration at Harvard Business School, says, “­Discrimination still exists in the workplace. However, organizations now have an opportunity to recognize this issue as a pinch point so that they can do something about it.” She noticed that most Black job applicants got jobs at certain companies if they “­whitened” their resumes during the research. So, they completely omitted mentions of their race, color, or country background. In some cases, candidates would purposely misspell their birth names to ensure that they get passed the dreaded algorithmic filtering bias. Likewise, if their resume included any connection to minorities, it tended to be discarded. The algorithm could single out information like “­volunteer work for the minority community” or “­helped disabled workers,” assume that the applicant was included in or part of the minority, and discard the resume. In one study, researchers created resumes for Black and Asian applicants. First, they sent them out for 1,600 ­entry-​­level jobs posted on job search websites in 16 different metropolitan areas of the United States. Some resumes included the applicant’s minority status, while others were “­whitened,” with racial cues scrubbed off. Then they observed how many of the applicants were invited for interviews based on the emails they had provided on their resumes. They found out that the whitened resumes produced more callbacks. 25% of candidates received callbacks for their racially devoid or whitened resumes, while only 10% were called back if they left racial clues. 21% of Asians were called back for their whitened resume, and 11.5% got callbacks when they mentioned they were Asian on their resume. Some Black, Asian, and even white applicants confessed that they have to pause and hesitate before including details of a school, internship, community service program, or university that is mildly associated with minority groups or has specific benefits to accommodate minority groups. DeCelles provided more thoughts on the matter. “­I don’t think it’s intended to be a setup,” she said. So she knew that none of this is intentional or planned. Nobody is doing it on purpose, at least not every single time. But some people are innately biased and let that bias seep into their jobs.

The Racist Algorithm

35

She further said, “­These organizations are not necessarily all talk when they say they’re p ­ ro-​­diversity. Maybe the diversity values are there, but they just haven’t been translated from the person who writes the job ad to the algorithm that screens the resumes.” The algorithm can easily be programmed to look for specific elements in the resume, and blacklists or discards resumes if any words like “­black” or “­race” or “­minority” show up. Subsequently, algorithms can recognize specific words and reject or accept the resume based on these certain scanned words in the document. Let’s look at another example. A report given by the Student Borrower Protection Center, “­Educational Redlining,” found in 2020 that a lending platform called Upstart charged higher interest rates and loan fees to borrowers who went to predominately diverse schools, in particular, those who attended historical Black colleges and universities like Howard University, or the schools like New Mexico State University which has a large ­Latino-​­based student population. In addition, those students were compared to students who went to New York University, where Black and Hispanic students combined make up only about 30% of the population. As we mentioned earlier, the algorithm can respond differently based on demographics and race and has the unique ability to preserve and exacerbate systemic racism at scale. Whether we decide who will most likely default on a loan, who should get hired or fired, or what stock is likely to fall, we humans are notoriously bad at d ­ ecision-​­making. We are bad at math, which is why we need c­ alculators—​­and our emotions and fears typically drive what we say, what we do, and even the object of our love. As we will read later in this book, engineers and data scientists are also human, and their biases creep into the decisions they make in their code. Data can be biased, but even if we assume that data is unbiased, machine learning algorithms are designed to minimize error r­ ates—​ e­ ven if done unfairly. They will always have a preference for the majority and never the minority. Algorithms can be accurate for the wrong reasons. The algorithms will assume what the designer fed it is true (­data). So it’s one thing to use algorithms to improve weather forecasts; it’s another to deploy them to make decisions about the lives of individual human beings. In the Black community, there is a common saying, “­The fate of a Black man in the hands of an ­all-​­white jury.” This colloquialism

36

Hidden in White Sight

often serves as the punch line in jokes, but the reality is that it’s not funny. Mainly because even when a jury is somewhat diverse, quite frequently, it’s an algorithm that provides the sentencing guidelines that the judge levies. So, whether a convicted individual gets probation or prison time will often get determined by “­lines of code.” The right to a fair and unbiased jury is one of the rights that defendants are guaranteed by the Sixth Amendment of the United States Bill of Rights. Yet, despite this staple of American democracy, avoiding potential sources of bias from information or misinformation spread through outlets like social media and ­24-​­hour news networks on television is immensely difficult for any juror or potential juror. A recent legal trend is to leverage AI to profile and select potential jurors to construct a fair and impartial jury to counteract these inhibitors. A defense attorney’s primary goal in jury selection is to predict whether a potential juror will be a good juror for their client. The hope of the prosecutor or defense attorney would be to leverage the technology to predict how a juror feels about a subject or person ahead of time. Even in legal cases that don’t involve criminal sentencing, like financial judgments and settlements, research has shown people of color will receive as little as half as much in damages for lost wages as a white male in the same situation. Even when AI is leveraged to combat these inherent challenges, implicit bias must still be considered. The models must be trained to ensure that AI makes fair and equitable decisions.  To accomplish this, previous case data must be used as the training data, and we must assume that past cases were adjudicated and tried fairly. Throughout the book, I’ve tried to use relevant examples to illustrate my position; this time, it’s unnecessary. Whether it’s the distant past or recent history, the Annals of time are rife with case after case citing that the justice system isn’t a good predictor of future fairness. Thus, it would only be the ideal source for training data if proper debiasing efforts have occurred. Unfortunately, past trials and similar court data have proven inherently inaccurate for ­A I-​­based projections and predictions. It’s not farfetched to consider that one day, AI might be that decisive voice that makes the dreaded “­hung jury” a thing of the past. Let’s try a more straightforward, n ­ on-​­human, and less intrusive experiment. Our example will start with a small bag of jellybeans. Let’s say the bag is filled with 1000 jellybeans. Of the 1000 jellybeans,

The Racist Algorithm

37

950 are black and the other 50 are white jellybeans. So 95% of the jellybeans provided were the color black. No different colors are present, just black and white jellybeans. Imagine the actual bag of jellybeans is the inputted data in our experiment. I closed my eyes and selected a jellybean out of the bag. What is the chance of me choosing a black jellybean out of the bag versus a white one? You don’t have to be a mathematician to realize that there is a highly likely chance that the selected jellybean will be black and not white. To be precise, a 95% chance. I know what you are saying; what does this have to do with AI? It has everything to do with AI because this is a fundamental tenet of AI. Our simple experiment shows a preference for black jellybeans simply by adding more to the bag. If the white jellybeans could speak, they would undoubtedly tell us that they were discriminated against by the person or thing that was picking which jellybeans were added to the bag. This principle accounts for how algorithms select one race over another when deciding. How much effort was used to make the data set (­bag of jellybeans) representative of all colors in our experiment? How much diversity is included in our sample set? Not much at all. One would think that at the minimum, we would have input 500 white jellybeans and 500 black j­ellybeans—​­but wait, wouldn’t the red jellybeans be angry? In our experiment, they had no chance of getting selected. Now, I wrote an AI algorithm that automatically chooses jellybeans from billions and billions of individuals. Like our human example, 95% of the billions and billions of jellybeans would always be black jellybeans. Shouldn’t we expect that the color preference of jellybeans would scale? In relativity, in our ­real-​­life predictive policing examples above, if 95% of police are deployed in Black neighborhoods, then one should not be surprised if 95% of the crime is also in Black communities. If 95% of crime is located in Black areas, we further the analogy because 95% of police patrol there. So, 95% of the jails will be filled with Black people. Should we now assume that Black people commit 95% of all crimes? A programmed algorithm might make that unfortunate conclusion based on the provided data. Unfortunately, our current data sets on crime partially reflect the tendency of police officers and

38

Hidden in White Sight

other members of the decisions of the judicial system to profile people based on race. Those decisions manifest themselves in what neighborhoods to patrol, what type of cars to pull over, and even how laws determine sentences for one drug versus another one. So there is no objective measure for crime data, and the data that does exist is riddled with racial profiling and bias. In our candy experiment, human subjectivity and bias are embedded in the data, just as they are in criminal data. Let’s look further: how about if we assume that each of our jellybeans is an actual person broken down by race? A black jellybean represents a Black man, and a white jellybean represents a white man. We are at a large financial institution, and we need to reduce staff by 30%. The algorithm will select 30% of people randomly from the employee pool represented by the jellybeans. What is the chance that more Blacks are fired versus white employees? Humans are indeed horrible at math.

Chapter Three

The American Dream

The Learning of Our Worst Impulses Our technology, our machines, is part of our humanity. We created them to extend ourselves, and that is what is unique about human beings. — ​­R ay Kurzweil

M

y friend Terrel called me a couple of years back to get technical advice. Most of my friends know that I work in tech, but few know what I actually do within the field. Typically, the “­a sk” is to get advice on things I don’t know very well. Standard questions range from “­How can I install more memory?” to “­C an you help

DOI: 10.1201/9781003368755-439

40

Hidden in White Sight

me build my website?” or “­W hy am I getting a green screen on my laptop?” Unfortunately, I am rarely the right person to help with those questions. But this day, the advice request was ­different—​­albeit an unusual request, it was related to what I actually do in the technology field. Here is Terrel’s story. In Stone Mountain, Georgia, the new fi ­ ve-​­bedroom house was Terrel and his wife Lorraine’s personal American Dream. This would be their first home, which they had saved and prepared for since getting married 5 years ago. There was a large, flat backyard with plenty of room for their two children to ­play—​­Cerene was five and Jeremiah seven. Terrel could easily envision 3200 square feet of living space with the basement as his mancave of all mancaves. All for $390,000. Prequalification was a breeze as both Terrel and Lorraine were highly educated and received their degrees from Spelman and Morehouse College in Atlanta, Georgia. They saved much more than they would need for the down payment, had excellent ­credit—​­scores of 815 and ­725—​­and earned roughly six figures each. The monthly mortgage was less than they paid for rent in New Jersey, where they had lived for 4 years before moving to Georgia. Better than anything, they were back home around friends and family. The mortgage closing was scheduled for Friday, July 2, 2019. Terrel’s two brothers and I had taken off from work to help with the move. The big 4th of July cookout plans were in the works. Family and friends were already invited, and we were all excited to celebrate with the new homeowners. Then, before the big day, the loan officer called and everything changed; she said, “­The loan isn’t going to close.” The loan officer told the couple that she had applied internally to the underwriting department for approval a dozen times, getting a “­no” each time. As a result, the couple had spent around $8,000 in fees and ­deposits—​­all nonrefundable. “­It seems like it’s getting rejected by an algorithm,” the loan officer said. “­A nd then there was an underwriter who could step in and decide to override that or not.” The problem was that in most cases the overrider must have specific evidence that the algorithm made an error.

The American Dream

41

Their bank was now using a sophisticated AI algorithm to determine who and who shouldn’t qualify for loans. ­A I-​­based underwriting solutions enable insurers to optimize pricing and risk. AI widens the scope of data sources that underwriters can use for their evaluations. Big data analytics allow deeper visibility into customers’ risk profiles, personally tailoring premiums to match each individual’s actual risk. This is a technical way of saying the algorithm will look at not just an applicant’s individual transactions or credit report, but can also look at personal relationships like past marriages, or other family relationships as they often speak to levels of risks. The algorithm even has access to data that was previously or supposedly deleted. I constantly remind my teenage daughter to remember that nothing is ever deleted in the cloud. Not emails, not text messages, photos, social media posts, or your favorite TikTok video… Nothing is deleted, programmers just add a flag that tells the algorithm not to show it as a result, but it’s still there in the database. The flag can be removed at any time, or the data can be leveraged by other algorithms or repurposed for other business applications. I remind her that her social media information might possibly be used as a decision point in her next loan ­application—​­without her awareness. The following excerpt was taken from the marketing literature of a popular AI algorithm software company leveraged by several national lenders, called Underwrite.ai. “­Unlike traditional models of underwriting, which focus on only a handful of credit attributes, Underwrite.ai analyzes thousands of data points from credit bureau sources to accurately model credit risk for any consumer”. By applying advances in machine learning the article goes on to state, “­we can radically outperform traditional scorecards in both consumer and small business lending.” “­Because we are using biologically and statistically based machine learning techniques applied to your portfolio, we model the expertise of your underwriters and the past performance of your loans to return an automated decision in milliseconds.” My friend, Terrel, was told that his wife, Lorraine, didn’t qualify because she was a government contractor, not a f­ ull-​­time ­employee—​ ­even though she had worked on her job for over 7 years. She had coworkers who were contractors, and yet, they had mortgages. Since

42

Hidden in White Sight

most of these algorithms don’t have transparency and explainability (­we will discuss these later) as core ethical principles for algorithm integrity, there was no way for Terrel and Lorraine to know why they were denied. The loan officer didn’t mention any issues with contractor approval for the loan at an earlier stage. But the loan officer was human; the d ­ ecision-​­maker, the algorithm, was a set of instructions that were programmed to analyze the data that the designer had made available to it. In our highlighted example, the loan officer was as unaware of why the algorithm came to its conclusion as my friends. What happened with my friend’s loan? I thought we had laws today to protect borrowers like Terrel and Lorraine from discriminatory lending practices. I mean, this was 2019, not 1999. When properly implemented, algorithmic and AI systems increase processing speed, reduce mistakes due to human error, and minimize labor costs while improving customer satisfaction rates. In the case of loan processing, AI was supposed to ensure that human error didn’t keep my friends from gaining access to that constitutional promise of homeownership and the American Dream. For decades, US banks have denied mortgages to Black f­ amilies—​ ­along with those who belonged to other racial and ethnic minority ­groups—​­who lived in certain areas “­­red-​­lined” by a federal government agency called the Home Owners’ Loan Corporation (­HOLC). Redlining is a discriminatory practice that puts services (­financial and otherwise) out of reach for residents of specific areas based on race or ethnicity. The HOLC, a federal agency, was created under the New Deal in the late 1930s, when President Franklin Delano Roosevelt enacted a series of programs collectively known as “­The New Deal.” The main purpose of these programs was to help the US recover from the Great Depression. The HOLC was tasked with drafting “­Residential Security” maps of major cities as part of its City Survey Program.

The American Dream

43

Neighborhoods or areas with predominantly racial and ethnic minority populations were colored ­red—​­hence, “­redlined.” These were considered ­high-​­risk areas for lenders. Today, we have three Federal laws that offer protection against lending discrimination. At least I thought we did. 1. The Fair Housing Act (­FHA) of 1968 protects people when they seek housing assistance or engage in other ­housing-​­related activities. It forbids any form of discrimination on the basis of race, color, national origin, religion, sex (­including gender, gender identity, and sexual orientation), familial status, or disability during any part of a residential real estate transaction. 2. The Equal Credit Opportunity Act (­ECOA) of 1974. Discriminatory lending and housing practices continued despite the Fair Housing Act, and civil rights groups advocated for more

44

Hidden in White Sight

legislation. Due to this, Congress passed the Equal Credit Opportunity Act (­ECOA) in 1974. The Federal Trade Commission states the ECOA “­forbids credit discrimination based on race, color, religion, national origin, sex, marital status, age, or whether you receive income from a public assistance program.” 3. The Community Reinvestment Act (­CRA) of 1977. Even with the FHA and ECOA, redlining continued in ­low-­​­­to-­​­­moderate-​ i­ncome (­LMI) neighborhoods. This law was enacted to prevent redlining and to encourage banks and savings associations to help meet the credit needs of all segments of their communities, including LMI neighborhoods. The above laws culminated in racial justice protests, resistance efforts, and other aspects of the civil rights and justice movements. Despite the stated intentions of these Acts, housing remains unequal across the nation, and discriminatory practices are as pervasive as ever. The human behavior of lending institutions hasn’t changed; therefore, the biased decisions haven’t either. Before AI became a household name, algorithms were used to implement the discriminatory practices listed above. Technology was raining down havoc on the Black community (­mimicking the men and women who were ensuring that discriminatory practices were being upheld). The early algorithms were i­nstruction-​­based programs, meaning they were designed to follow a detailed series of steps programmed by the developer. Those algorithms were limited to the data and variables passed to them, therefore inherently limited to the availability of digitized data and compute ­power—​­the machine’s processing power. Then came modernized big data, machine learning, and hardware advancement, which allowed a small battery to power a ­3200-­​­­square-​­foot home. Like those used to deny Terrel and Lorraine, algorithms now run off data sets with thousands of patterns, entertainAI has learned our biases from ­us—​­Unfortunately AI ment consumption habits, and transis a great student, and we actions. So specialized AI systems have are great instructors. Our become amazingly efficient at making ways have become her ways decisions on behalf of humans. and she mimics us more than It is very difficult for humans to have we would like to believe. empathy for each other. Humans have

The American Dream

45

very little problem sympathizing. Sympathy means that I’m glad that it didn’t happen to me. Empathy means that I understand and share the feelings of another, or simply “­putting myself in someone else’s shoes.” I believe that we are born with the capacity to empathize with others, but often our cultural experiences rob us of this innate ability. When this happens, we must retrain ourselves to think differently. Empathy is an essential trait in ensuring trustworthy and b ­ ias-​­free algorithms. Would a police officer want to be profiled? Would a police officer want to be trapped in a predictive hotspot? If they were asked the question, the police officer might declare that they wouldn’t allow themselves to do anything that might cause them to be profiled. This response would prove my ­point—​­meaning that they have refused to put themselves in the shoes of the person being profiled. This would make a skeptic believe that if people don’t have each other’s best interest at heart, why would anyone trust technology? But since people are redeemable, meaning they can change their ­minds—​­so are computers since we feed them the data that causes them to make the decisions that produce the outcome. If we correct the data, then the outcome will likewise be corrected. If a reasonable person is given the right information and told that their actions are unfair, most people will do the right thing. This is also true with AI a­ lgorithms—​­give it the right information, correct it when it’s wrong, and then it too will do the right thing. Let’s consider our banking example, 68% of people trust a human more than AI to decide on bank loan approvals. Perhaps some of this mistrust comes from people not fully understanding how AI makes decisions. It seems logical to assume you could reason with a human regarding a decision on a bank loan, hoping to tap into their sense of empathy and integrity, versus a machine that has none. The future of ­A I-​­based ­decision-​­making combines AI insights and analytics with ­human-​­supplied ethical considerations. Trying to infuse morality into AI algorithms is a perplexing proposition. This became evident while testing Delphi, a research prototype designed by the Allen Institute for Artificial Intelligence to make moral judgments. The application was developed to model people’s moral judgment in various everyday situations.

46

Hidden in White Sight

I decided to run the Delphi tool for myself. Like most controversial tools, before I could use it, I had to sign off on many legal disclaimers. I’ve copied one of those legal excerpts below: What are the limitations of Delphi? Large ­pre-​­trained language models, such as ­GPT-​­3, are trained on mostly unfiltered internet data and therefore are extremely quick to produce toxic, unethical, and harmful content, especially about minority groups. Delphi’s responses are automatically extrapolated from a survey of US crowd workers, which helps reduce this issue but may introduce its own biases. Thus, some responses from Delphi may contain inappropriate or offensive results. Please be mindful before sharing results.

After agreeing to the disclaimers, I typed into the Delphi engine the following query: “­Do Black Americans default on a loan quicker than white folk?” Let’s see how Delphi responded:

I was surprised to see a computer being aware that it’s racist to deny a person a loan because of their skin color, their acquaintances, where they live, or who they love. But more often than not, the machines and algorithms have incorporated our biases into their ­decision-​­making process. Whereas we have seen earlier that financial corporations have consistently discriminated against Blacks, and the mistrust of those institutions is justified, a new vehicle to resolve the injustices has emerged. Lending institutions have solicited automated d ­ ecision-​­making algorithms to do their bidding for them. As in other industries, AI

The American Dream

47

is widely considered the great equalizer in the financial industry. AI can reduce humans’ subjective interpretation of data because machine learning algorithms learn to consider only the variables that improve their predictive accuracy. Some early evidence showed that algorithms could improve d ­ ecision-​­making, causing it to become fairer in the process. As in the underwriting case, it was thought that AI could particularly benefit historically underserved communities. Unlike human decisions, decisions made by the algorithm could, in principle (­and increasingly in practice), be opened up, examined, and then interrogated. Andrew McAfee of MIT stated, “­If you want the bias out, get the algorithms in.” Extensive evidence suggests that AI models can embed human and societal biases and deploy them at scale. Whereas a single underwriting department might review ten loans a day, algorithms can now review thousands of applications an hour. Even ­well-​­respected government agencies like Fannie’s and Freddie’s approval processes involve mysterious algorithms. They both released automated underwriting software programs in 1995 to much fanfare about their speed, ease, and, most importantly, fairness. These software algorithms had to be removed within a year because of proven inherited bias. Peter Maselli, a h ­ igh-​­ranking official at Freddie Mac, told the New York Times at the time the software was originally launched: “­Using a database as opposed to human judgment can avoid influences by other forces, such as discrimination against minority individuals and redlining.” A bank executive told Congress the same year the new systems were “­explicitly and implicitly ‘­color blind,’ ” since they did not consider a person’s race in their evaluations. I’m sure that Mr. Maselli had sincere intentions, but the following two industry experts seem to contradict his opinion: “­The quality of the data that you’re putting into the underwriting algorithm is crucial,” said Aracely Panameño, Director of Latino Affairs for the Center for Responsible Lending. “­If the data that you’re putting in is based on historical discrimination, then you’re basically cementing the discrimination at the other end.” “­This is a relatively new world of automated underwriting engines that by intent may not discriminate but by effect likely do,” said David

48

Hidden in White Sight

Stevens, a former president and CEO of the Mortgage Bankers Association, now an independent financial consultant. Like technology in general, AI is impacted by the cultural biases of those that program or design its outcome. Computers work objectively, always prizing accuracy over anything else unless programmed to do so. This means that they will execute a program as efficiently as possible without regard for “­fairness.” Since AI algorithms don’t take an actual position by default, they will do as they are told. If the data tells the algorithm to include/­exclude or redline certain groups of individuals, it will do that. This can either happen by feeding it incomplete data or ­under-​­representative data of the subject matter. Historically, some mortgage providers purposely chose to discriminate and keep Black Americans like Terrel and Lorraine from getting that elusive piece of that American ­Dream—​­home ownership. Now the new weapon of choice is AI and algorithms. AI and its inherent bias seem to be contributing factors in slowing minorities’ home loan approvals. An investigation by The Markup reported by the Associated Press found lenders were more likely to deny home loans to people of color than to white people with similar financial characteristics. Specifically, 80% of Black applicants are more likely to be rejected, 40% of Latino applicants, and 70% of Native American applicants. The article explained that these aren’t regional but national rates. Since we’ve seen the persistence of discriminatory actions in lending practices, regardless of the laws created to prohibit them, we must conclude that some algorithms are authentic and doing precisely what they were trained to d ­ o—​­discriminating based on color. We discussed earlier that AI was supposed to equal the playing ­field—​­In our case it was, the desired objective was to ensure that the homeownership process was fair by removing human bias and subjectivity from the equation. The dangers of AI algorithms aren’t that they are going to rebel against us. AI algorithms will do exactly what we train them to do. In Terrel’s and similar cases, they could possibly discriminate on loans based on race. As I mentioned earlier in this book, algorithms don’t just become biased on their own. They learn and acquire knowledge from humans through our interactions; they observe the world (­or at least the world we show them) and identify patterns. After this, they make

The American Dream

49

decisions based on the data we feed them. Our instincts and impulses are transferred over to the algorithms. I recently read an article published in the journal Big Data & Society that had an interesting quote that accentuates my point. “­A lgorithms are animated by data, data comes from people, people make up society, and society is unequal.” What are our worst impulses regarding simple American values on fundamental principles such as the ability to purchase a home, gain wealth, etc.? Martin Luther King famously stated in his “­I Have A Dream” speech. “­W hen the architects of our republic [the founding fathers] wrote the magnificent words of the Constitution and the Declaration of Independence, they were signing a promissory note to which every American was to fall heir. This note was a promise that all ­men—​­yes, Black men as well as white ­men—​­would be guaranteed the unalienable rights of life, liberty, and the pursuit of happiness.” Our worst impulses manifest themselves in AI and machines much in the same way they do in our human experience, both explicitly and implicitly. Explicit bias refers to the attitudes or beliefs about people on a conscious level. My colleagues and I don’t intentionally write biased code, at least not from our conscious minds. Everyone has implicit bias, which refers to the attitudes or stereotypes that affect us unconsciously, meaning we likely don’t know, recognize, or believe that we have these biases. Most who are involved in designing AI systems would readily admit that the data collection step in the process is one of the most important ones. Experts refer to this phase as “­confirmation bias” that seeps into the process. Confirmation bias is a cognitive bias that favors information that confirms your previously existing beliefs or biases. We are often skilled at avoiding ­race-​­based decisions in the process, but other biases routinely trap us. For example, many of us have heard that l­eft-​­handed people are more creative than ­right-​­handed people. Whenever this person encounters a person that is both l­eft-​­handed and creative, they place greater importance on this “­evidence” that supports what they already believe. This individual might even seek proof that further backs up this belief while discounting examples that don’t help the idea. Since the interpretation of data is a critical step of the collection

50

Hidden in White Sight

process, you can see how even this early step can provide a baseline for discriminatory outcomes in the AI process. Confirmation biases impact how we gather information and influence how we interpret and recall the information. For example, people who support or oppose a particular issue will not only seek information to keep it, but they will also interpret news stories to uphold their existing ideas. They will also remember details in a way that reinforces these attitudes. This is relevant to how and when we collect data. An implicit and confirmation bias may counter a person’s conscious beliefs without realizing it. For example, expressing the wishes of a particular social group or approval of a specific action is possible while simultaneously being unconsciously biased against that group or activity. For instance, in the social domain, stereotypical concepts such as “­criminal” and “­dangerous” automatically come to mind for many people when they encounter or are exposed to images of young Black men. Even someone who finds himself in the middle of a predictive hotspot with a flat tire. It’s been my experience that most of us fall into this category. I believe that all algorithm designers should undergo implicit bias testing. Over the years, there have been several studies to help us determine and get our arms around our implicit biases. In the 1990s, Dr. Tony Greenwald invented a way to test for it called the Implicit Association Test. This is how it works. First, the test brings up a questionnaire that asks specific personal questions like gender, age, race, country, religious affiliation, etc. The test taker is asked to rapidly group a series of images and words from these questions. For instance, in one section of the IAT, you press a certain key whenever you see a white face or a positive word, you press another key if you see a Black face or a negative word, and you have to react as fast as you can. In another section, the associations are reversed. White faces go with negative words, while Black faces go with positive words. The test reveals that 70% of ­test-​­takers (­white and ­non-​­white) respond more slowly when grouping Black faces with positive words. The test refers to this as an automatic preference for white people. Culture has taught us ­t his—​­the impulse to associate Black faces with negative concepts is embedded in our human psyche.

The American Dream

51

When they become algorithmic programmers, software developers and data scientists don’t somehow lose this impulsive trait. One of the things we do as humans is store information in our minds about a subject or group of people. When we see someone from the group, we unconsciously pull out of our memory all of the information that we’ve stored on that particular group. These memories shape what we perceive as reality without us realizing that we’ve done it. Don’t beat yourself up over this realization; it happens to all of us. Normally, our biases only impact those that we have the opportunity to impact directly. But as developers of algorithms, our biases can impact many more lives than just the ones we encounter. So for us, we should be careful not to codify our personal biases in the code that we build. Data scientists and programmers consider functional and ­non-​ ­functional requirements as we collect data from our clients and subsequently build models. It’s natural for us to consider our impulses while doing this. Implicit biases are hidden, but nevertheless they are real. Implicit biases can cause major inequalities and, when coupled with technologies like AI, can scale those inequalities collectively. Dr. Jennifer Eberhardt, one of the foremost authoritarians on unconscious bias, mentions the “­­other-​­race effect.” This is when people have trouble recognizing the faces of other racial groups. She states that criminals consistently exploit this effect, giving us the scenario wherein a gang of Black teenagers in Oakland, California, caused a crimewave of purse snatchings among m ­ iddle-​­aged women in the Chinatown district of the city. When asked why they targeted that ethnic group, the teenagers said that the Asian women “­couldn’t tell the brothers apart when faced with a lineup.” Dr. Eberhardt has written that the phrase, “­they all look alike,” associated with the province of the bigot, “­is actually a function of biology and exposure.” In her example, Dr. Eberhardt shows that although facial lookalike biases are frequently associated with white Americans, even Blacks have this same human instinct. The reality is that most of us harbor bias without even knowing it. It stems from our brain’s tendency to categorize t­ hings—​­a useful function in a world of infinite stimuli. Stimuli can lead to discrimination, baseless assumptions, and other worse things, particularly in times of hurry or stress.

52

Hidden in White Sight

Dr. Eberhardt provides another example from her early years after receiving her Ph.D. in cognitive psychology at Harvard University. She introduced her class to the quizmaster test, in which one student acts as a quiz show host, like Alex Trebek on Jeopardy!, while another poses as a contestant. Observers who watch the show almost always say that they see the quizmaster as more intelligent, despite knowing that it’s simply because the host already knows the answers. It’s a textbook example of what is known as the fundamental attribution error. This error explains a tendency we have to credit or blame others for actions or qualities for which they bear no responsibility. Eberhardt’s students committed the same error while watching the s­ how—​­except when the quizmaster was Black and the contestant was white. “­T he effect was just flat,” she said, “­T he student observers did not see the quizmaster as any more intelligent than the contestant.” To explore how hardwired the effects of this might be, Eberhardt and her colleagues at Stanford recruited ten Black and ten white students. They put them in an MRI machine while showing them photographs of white and Black faces. When students viewed faces of their race, brain areas involved in facial recognition lit up more than when viewing faces of other races. Students also had more trouble remembering faces of races other than their own. “­­Same-​­race recognition isn’t inborn,” Eberhardt says. “­It’s a matter of experience, acting on biology: If you grew up among white people, you learned to make fine distinctions among whites. Those are the faces our brain is getting trained on.” Such learned perceptual biases, she thought, might shape reactions, ­too—​­in particular, those at intense confrontations that can have a tragic outcome, such as when a police officer shoots an unarmed Black man. Eberhardt and her colleagues did a series of experiments using the ­dot-​­probe paradigm, a ­well-​­known method of implanting subliminal images. She asked subjects (­largely white) to stare at a dot on a computer screen while i­ mages—​­of a Black face, a white face, or no face at ­all—​ ­flashed imperceptibly quickly off to one side. Then she would show a vague outline of an object that gradually came into focus. The subjects, including police officers and students, were asked to press a key as soon as they recognized the object. The

The American Dream

53

object could be benign, such as a radio, or something related to crime, like a gun. Subjects who had been primed with Black faces recognized the weapon more quickly than participants who had seen white faces. In other words, seeing a Black f­ ace—​­even s­ ubconsciously—​­prompted people to see the image of a gun. Our social institutions such as the media, education, government, family, class, and religion significantly impact our identity. They help shape how we view ourselves, how we view others, and how we act toward others. For example, a man with a beard and a turban is automatically assumed by some people to be a terrorist. This way, our upbringing and experiences affect our ­values—​­what we consider right and wrong. This is how our society, the one we live in, influences our choices. Imagine you are cooking something from a cookbook for the first time. You don’t know how to make the dish, so you follow the written instructions. If there is a printing error in the book so that instead of telling you to put in three tablespoons of salt, a typo replaced the three with an eight, there’s a chance that the entire dish would be ruined. It wouldn’t be your fault that the dish turned out that way; you just followed the instruction. Similarly, an AI algorithm is a list of specific instructions by the programmer. If the data given is biased, the program will also be biased. In the documentary I Am Not Your Negro, James Baldwin states that America is “­cruelly trapped between what we would like to be and what we actually are.” This entrapment weakens our ability as Americans to deal with the world as it i­s—​­our ability to deal with who and what we are. As a result, it is completely natural for us to judge, fail to be objective, and act too quickly as humans. These urges manifest themselves frequently in dealing with people of the opposite race, gender, or ethnicity. These seemingly natural and innate prejudgments are often subliminally captured in the code that we develop and the applications that we deploy for public consumption. As we indicated earlier, AI development companies are often asked to make tasks more ­efficient—​­meaning do the old way faster and cheaper or implement that “­Big Idea” to accomplish a task in a way that hasn’t been done

54

Hidden in White Sight

before. Terrel and his wife’s case debunks the myth that AI systems are purely objective and ­bias-​­free by design. I submit consideration that in AI systems, the supposedly objective is quite often the subjective. Meaning that in AI, the right answer is usually in the eye of the beholder, so that a system that is designed for one purpose might not work for another purpose. In Terrel’s case, a system that was designed to provide fairness and objectivity in loan processing instead was riddled with subjectivity and bias. Let me end this chapter with a simple example that is often used in AI design conversations. It’s the Cat and ­Not-​­cat example. In this example, the user’s task is to look at six images and determine which of the six is a cat or not a cat.

The first five are easy as three of the five images are obviously cats. But image six is a photo of a tiger. So which one is it, cat or not a cat? You only have two choices. What would my AI algorithm conclude? What is the right answer? One thing for sure is that the answer doesn’t come from the AI system. Actually there is no right answer in our example. It depends solely on what the owner of the system wants to do. If the purpose of the AI application was to develop a pet recommendation app, then the tiger is designated “­not a cat”. So the lesson here is that AI systems can’t determine the ­objective—​­it will take a human to do that. In Terrel’s case, the AI system shouldn’t have had sole ownership of whether he and his wife received the loan. The ­decision-​­makers at the loan company should have served as responsible “­parents” to oversee the algorithm’s decision, intervening when

The American Dream

55

appropriate. In fairness to most credit and financial institutions, they are now taking necessary steps to ensure that the algorithms have human intervention. Three months after they initially applied and after some ­not-­​­­so-​­friendly conversations with the bank, Terrel and his wife did get the loan. The lesson learned is that AI systems aren’t objective at all; it’s up to the system designer and the caretakers of the system to provide the subjective conscious. And for the rest of us to know that it’s okay to question the decisions that are produced by ­algorithms—​­especially if they don’t align with our facts. Terrell and ­ igh-​­paying jobs, low d ­ ebt–​­income rahis wife had excellent credit, h tios, and l­ong-​­standing work ­histories—​­therefore, they weren’t willing to let an algorithm postpone their “­A merican Dream.”

Chapter Four

AI Gone Wild

Mistaken ­Identity—​­It Wasn’t Me Facial recognition software can pick a face out of a crowd, but the vending machine at work can’t recognize a dollar with a bent corner. —​­Author Unknown

In 2013, 60 M

inutes,

a nationally syndicated and ­award-​­winning American news magazine television show, ran an episode entitled “­Faces in the Crowd.” Leslie Stahl, one of the show’s anchors, warned how technologies like facial recognition were already changing how we live. Both for the good and the bad. She reported that even when

DOI: 10.1201/9781003368755-557

58

Hidden in White Sight

we weren’t looking, the technology was at work, taking photos and storing them away to be used as what she referred to as “­facial prints.” Similar to how fingerprints are used to uniquely identify, facial prints would be used to differentiate one person from another. In the episode, she narrates a video showing Tom Cruise walking through the mall in a scene from the movie The Minority Report, where he is bombarded by ads, recognizing him while telling him what to buy as he walks through the mall. She quickly highlighted that even though Tom Cruise’s character “­John Anderton” was a futuristic being and the movie was science fiction, the AI technology was getting closer to reality. On his usual morning routine, 2 ­ 0-­​­­year-​­old Jaypee Larosa was standing in front of an internet cafe in Davao City, a metropolitan hub on the Philippine island of Mindanao. Three men wearing dark jackets pulled up on a motorcycle and opened fire as he was standing there. That early morning Larosa was shot dead. According to witnesses, after the shooting, one of the men reportedly removed Larosa’s baseball cap and said, “­Son of a bitch. He’s not the one.” After saying this, they drove off. It was one of the hundreds of extrajudicial killings carried out in Davao City that year. All of this happened in a city with a population of around 1.6 million, with Rodrigo Duterte, later president of the Philippines, serving as mayor. Years before he launched his notorious “­drug war,” Duterte presided over similar tactics at a much smaller local level. According to an investigation by the Human Rights Watch, death squads assassinated street children, drug dealers, and petty criminals; in some cases, researchers found evidence of the conspiracy or direct involvement of government officials and police. Philippine police’s interest in incorporating ­cutting-​­edge surveillance into their infrastructure has hardly waned. The Filipino government installed thousands of surveillance cameras across Davao City and metro Manila in collaboration with a Chinese firm. This installation would reportedly include a national command center and facial and vehicle recognition software. A few months after, in an interview by Epimaco Densing III, Undersecretary of the Department of the Interior and Local Government, to Filipino television, saying that the project’s goal was to detect the faces of terrorist suspects and prevent crimes before they took place.

AI Gone Wild

59

Duterte warned a group of drug dealers on the street known as Dewey Boulevard. “­If you are into drugs, I’m warning you. I’m giving you 48 hours. 48 hours. If I see you there, I’ll have you killed.” Reportedly, police monitored the entire area using facial recognition software. A research and training officer at the Davao command center, Antonio Boquiren, said the video capabilities helped police crack down on l­ ow-​­level q ­ uality-­​­­of-​­life violations. Any who were considered a threat, troublemaker, drug user, protester, or activist was blacklisted, but even some committers of l­ow-​­level crimes were targeted. “­W hether it’s criminality, smoking, or jaywalking, any ordinance violation is a crime, and the police are sent,” he said, laughing. “­People who smoke complained, ‘­How did you catch us before we even lit it?’ The police officer will point to the use of facial recognition.” They also made use of video analytics to automatically tag objects, like cars, or identify people by their physical attributes. The tags were detailed and included an object’s size, speed, color, trajectory, and direction. The Face Capture interface of the software gathered facial images from pedestrians on the streets of Davao City. The software boxed out and collected facial images as people walked past street cameras. The algorithm used the o ­ bject-​­tagging capability to locate suspects by their physical attributes. Not everyone identified was executed on the spot. Some suspects were questioned along with their associates. But all suspects that were identified by the facial recognition algorithm ended up on the annihilation list. The Face Capture feature helped authorities locate wanted people in real t­ ime—​­including residents on watchlists. “­Imagine a scenario in which someone in the police force has access to this system and works with the local death squad, producing lists of people to be killed.” The technology could have assisted police in IDing a person on the kill list in real time and deploying death squads to apprehend them. Some years back, I was working to help cities and other government municipalities worldwide become more efficient and improve operations. To accomplish this, we leveraged different electronic methods and sensors to collect specific data. Once the data is collected, a typical next step is to use AI to analyze and identify patterns to create outcomes to manage assets, resources, and services more efficiently. One of my past clients, the mayor of a city in Brazil,

60

Hidden in White Sight

simplified his desire by stating, “­Our goal is simply to improve the quality of life for our citizens.” He didn’t spend a lot of time going through requirement after requirement. His response always stuck with me because there is no better way to describe the goal of any technology other ­than—​­to make life better. The above example in Davao City was an abnormal and unexpected example of technology placed in the hands of the wrong person. The technology was meant to provide a better quality of life for “­all” of its citizens; it was used to “­end” life. Neither the solution providers nor the team that helped deploy the software knew that the dictator would use facial recognition for mass surveillance to identify and hunt down people he considered undesirables. Now that’s an interesting term: “­undesirables.” It’s somewhat subjective in definition alone, and its meaning is only limited to the beholder. It isn’t a massive stretch for any reasonably minded individual to see that what happened in Davao City could have quickly happened anywhere else, especially in the United States. The term “­undesirable” is not foreign to people of c­ olor—​­as the symbolism around it is embedded in the American culture. Could systems like these be used to m ­ ass-​­survey people of color? I would say, “­Yes.” Most major technology companies have gotten out of the facial recognition software business, especially in cases where the technology is used for mass surveillance. Not speaking on behalf of any company, I believe that tech used for bad purposes like the one in Davao City is why many major tech companies are exiting the facial recognition software business. In a letter to the members of the US Congress in 2020, IBM CEO Arvind Krishna shared that IBM has sunset its own ­general-​­purpose IBM facial recognition and analysis products. Krishna also emphasized that IBM firmly opposes and will not condone uses of any technology, including facial recognition technology offered by other vendors, for mass surveillance, racial profiling, violations of basic human rights and freedoms, or any purpose which is not consistent with our values and Principles of Trust and Transparency.

We’ve seen case after case over the last few years where facial recognition software is being used in the United States to perpetuate and

AI Gone Wild

61

further social injustices. Heck, I gave my own personal example in an earlier chapter with the predictive policing incident. Every major technology company, healthcare provider, law enforcement agency, state, and local government uses facial recognition software to some extent. This is not a bad thing, as facial recognition software is one of the 21st century’s major technological advancements. I am a huge fan of it and am enthusiastic about further developments. However, where I draw the line is when this technology is used for mass surveillance that leads to racial and gender profiling and in cases where citizens’ privacy and personal freedoms are potentially violated. As we mentioned at the start of this chapter, 10 years ago, facial recognition applications would sound like the plot of some futuristic ­science-​­fiction thriller. I’m not a Trekkie, but I remember the movie, Star Trek II: The Wrath of Khan, released in 1982. Admiral Kirk, with the assistance of the Enterprise crew, had to stop an old nemesis, Khan Noonien Singh, from using the l­ ife-​­generating Genesis Device as the ultimate weapon. In one scene, we are shown the back of the eye with text revealing the scanning of the optic disc to match Kirk’s veins and arteries against his biometric record. The film leverages Iris Facial Recognition software. At least five other films that date back to the early 2000s also leverage this software. In all these cases, the point of facial recognition is to use an image of one person’s face to confirm that person’s identity. Since I’m a guy who works in the AI technology field, rest assured that I’m not one to promote conspiracy theories around AI being the ultimate doomsday tool that will end the world. Or one who supports the theory that the government has robotic eyes everywhere, tracking your every move with security cameras and drones. Hmmm, interesting thought, though. I will say that I’ve worked on and have reviewed some interesting, ­thought-​­provoking app ideas over the last few years around facial and voice recognition software. Are there cases where the government keeps a database of images of citizens’ faces? And can that database be used for surveillance of individuals? Yes, and yes… At the time of writing, a new bill in New Hampshire is up for debate in ­2022—​­HB ­499—​­to limit how this technology can be used in the Granite State. The bill, as it was introduced, states that “­No officer or employee of a state agency shall use facial recognition

62

Hidden in White Sight

technology to engage in ongoing surveillance of an individual or group of individuals.” Still, there was an “­unless” (­there’s always an “­unless” added). The “­unless” is that law enforcement would be allowed to use facial recognition technology data to run individual photos or surveil an area for no more than three days. If they wanted more than three days to use facial recognition technology to surveil a public space, they would have to get a court order. In another example, I was asked to attend a meeting with one of my city government clients in the Southeastern area of the US. They had previously deployed a ­command-­​­­and-​­control video analytics solution and wanted to know whether a ­third-​­party solution could be integrated into the existing design. The solution was gunshot detection software from a company called ShotSpotter. At the time of our meeting, the ShotSpotter software could detect gunfire, identify the gunfire’s exact location, and alert police in real time. Gunshot detection systems like the ShotSpotter use acoustic sensing technology to identify, discriminate, and report gunshots to the police. These reports reach the police within mere seconds of the shot being fired. A gunshot detection system is made up of sensors that detect the sound of a gunshot. After this, transmitters send a message to the police dispatch center, and a computer receives and displays that message. As soon as a signal arrives at the police station, it is up to the dispatcher to decide whether or not to send a unit to respond to the signal. Gunshot detection systems cannot detect shots fired indoors or that are blocked by a building or other obstructions. Since all of the data is stored in ShotSpotter’s cloud, the data is constantly analyzed, cleansed, and transformed by the company. The systems may be in boxes mounted on poles, disguised as rooftop vents or birdhouses, or unobtrusively located. The software is normally deployed in ­high-​­crime areas and is typically accompanied by video cameras. Sounds promising, right? AI sound analytics at its finest, right? Although ­A I-​­based tools like this one can certainly help a city’s police department deal with staffing issues, let’s ask Chicago native Michael Williams’ family what they think of the ShotSpotter software. Michael was jailed in August of 2021, accused of killing a young man from the neighborhood who asked him for a ride during a night

AI Gone Wild

63

of unrest over police brutality the previous May. In our world of technological advancements, the key evidence against Michael didn’t come from an eyewitness or an informant. It didn’t come from fingerprints left at the scene or DNA testing; it came from a clip of a noiseless security video showing a car driving through an intersection, and a loud bang picked up by a network of surveillance microphones. Prosecutors said technology powered by a secret  algorithm that analyzed noises detected by the sensors indicated Williams shot and killed the man. “­I kept trying to figure out, how can they get away with using the technology like that against me?” said Williams, speaking publicly for the first time about his ordeal. “­That’s not fair.” After bouts with suicide, Michael sat in jail for almost a year before a judge dismissed the case against him at prosecutors’ request for lack of sufficient evidence. Michael’s horrific experience accentuates the real world and the devastating impact of algorithms making consequential and l­ife-​­altering decisions. ShotSpotter was the gunshot detection software that almost cost Michael his life. ShotSpotter evidence has increasingly been admitted in court cases around the country, totaling up to some 200 cases. ShotSpotter’s website says it’s “­a leader in precision policing technology solutions” that helps stop gun violence by using “­sensors, algorithms, and artificial intelligence” to classify 14 million sounds in its proprietary database as gunshots or something else. But an Associated Press investigation, based on a review of thousands of internal documents, emails, presentations, and confidential contracts, along with interviews with dozens of public defenders in communities where ShotSpotter has been deployed, has identified several serious flaws in using ShotSpotter as evidentiary support for prosecutors. Let’s consider a few. The Associated Press findings found that ShotSpotter can miss live gunfire right under its microphones or misclassify the sounds of fireworks or cars backfiring as gunshots. Like most voice and facial recognition systems, there is a human element to this largely black box solution. ShotSpotter employees can change the source of sounds picked up by its sensors after listening to audio recordings, introducing the possibility of human bias into the gunshot detection algorithm. According to court records, employees can and do modify the location or number of shots fired at the request of the police. City

64

Hidden in White Sight

dispatchers or police themselves, in the past, could also make some of these changes. So in the case of ShotSpotter, you need to trust the AI algorithms, but you also need to trust the human biases of both employees and the justice system. As my mother would often say when I was trying to convince her of something that she’s not inclined to believe, “­Son, that’s a lot of trust that you are asking of me.” Amid nationwide debates over racial biases in privacy, policing, and civil rights, advocates say that ShotSpotter’s system and other ­algorithm-​­based technologies used to set everything from prison sentences to probation rules lack transparency and oversight, and show why the criminal justice system shouldn’t outsource some of society’s weightiest decisions to computer code. I know it’s pretty easy for us to get caught up in the injustices handed down by tools like Iris’s facial recognition system or ShotSpotter’s gunshot sound analytics solution. Still, we shouldn’t forget that there are other ways to carry out AI biases. AI systems have consistently used biological characteristics like fingerprints, DNA, hand geometry, writing signatures, and even (­ believe it or not) your heartbeat rhythms to identify a person uniquely. Most of the analytical systems using these characteristics bypass the protected attribute laws.

These systems manifest themselves in all parts of our society. For instance, some years back, I worked on a permit and event management solution in the city of Boston. My team had spent months

AI Gone Wild

65

designing and deploying the solution. The city was targeting a ­pre-​ ­Boston Marathon concert event for the solution launch. We were there the day of the event to ensure that the software and hardware didn’t fail during the event. Even so, I was anxious to return to my kids. It was typical for me not to stay any extra days on the road. When the job ended, then my time there ended. Little did we know that Boston would experience the city’s most catastrophic incident to date on this particular day. Two bombs went off at the Boston Marathon. Cameras caught the tragedy from a myriad of angles and in ­heart-​­wrenching detail. We had worked for weeks installing public cameras throughout the local area. Investigators pored over the mountains of footage and still images shot from close circuit television cameras, TV crews, private cameras from commercial storefronts like CVS Pharmacies, cell phones, and even cameras mounted on top of runners’ heads. Back then, we weren’t marketing these video analytics solutions like AI, but much of the same technologies were used. Machine vision and surveillance technologies, even back then, were able to provide ­real-​ ­time analytics. Automatic image and video processing, even then, was already good enough that it could be used as a first pass to pare down huge volumes of footage and help investigators focus their search. The same system, previously used by the wrong hands in Davao City, was used to provide r­ eal-​­time analytics at the scene of the bombing. We had a team of five designers onsite managing and monitoring the ­command-­​­­and-​­control solutions installed earlier. AI and the Internet of Things (­IoT) technologies are quite popular today, but this solution was considered revolutionary then. It consisted of a mapping system (­similar to Google Maps today) with different overlays. We showed our data of the city on the map, alongside r­ eal-​­time and-​ views of police cars and medical locations. The c­ommand-­​­­ ­control center also depicted all installed public and private cameras previously configured into our solution. So a user could click on a camera icon and see r­ eal-​­time activity within the camera’s view. If a person or vehicle passed through a certain camera zone, i.e. a large truck or a student with a red backpack, we could automatically fire an alert to perform a specific action, like notify the police. I know

66

Hidden in White Sight

this doesn’t seem highly sophisticated in today’s world of Ring doorbell cameras, but this was almost 10 years ago. If you aren’t impressed yet, this next capability might get your attention. Working with a partner, HootSuite, we could sniff through social media and text conversations for keywords. For instance, we could set a filter for “­Backpack.” Likewise, we performed sentiment analysis looking for hate speech on the data streamed in real time from social media conversations. If anyone within a given perimeter posted that particular word, the algorithm could flag it and set off an alert back to the c­ ommand-­​­­and-​­control center. Seems like a ­sci-​­fi experience, right? This technology was deployed on the actual day of the Boston Marathon bombing. Again, this was almost 10 years ago, and AI systems have advanced at an astronomical maturity rate since then. Our ability to process large amounts of big data and analyze the data went to unfathomable speeds in around 2019. We’ve spent lots of time up to this point highlighting the potential dangers of A ­ I—​ ­but this was an example where the technology was used to provide investigative breakthrough. Whether it was the critical smartphone pictures taken from private citizens, the ­forward-​­looking infrared cameras that identified one of the suspects hiding in a backyard boat, or the Lord and Taylor department store security camera that recorded the suspect dropping his backpack off at the bombing site, AI was at work. The technology provided essential forensic guidance to help the FBI solve this complicated crime in record time, thus allowing the citizens of Boston to live in peace. Large amounts of information streaming in from our phones, computers, parking meters, cars, buses, trains, watches, and even our refrigerators now allow AI to not only analyze but also learn. Businesses aren’t just collecting large amounts of data, but also using it to improve customer experiences and business decisions and processes. Whether these systems are leveraging biometric capabilities like video analytics or analyzing data from your smartwatch, the ­higher-​­level design process is the same. Let’s use the Boston example as a case in point. Step ­1— ​­Capture: Collect from the subject that is being recognized or analyzed. In our example above, the cameras captured

AI Gone Wild

67

e­vent-​­ goers, runners, and people who participated in the marathon. Step ­2 —​­Reference Database: Data is stored from a prior encounter with you or someone that reminds the system of you. Step 3 ­ —​­Matcher: Compare live video and still photos taken from the camera to photos stored in the reference database. Step ­4 —​­Action: Now that I have a match, I’ll do something. In our example, the system would alert the ­command-­​­­and-​­control center of the match. These four steps were used in all three examples that I highlighted in this chapter. Sometime later, after the Boston Marathon engagement, I was asked to present on AI and facial recognition technologies at a conference in Vegas. I used the Boston engagement and some others as part of my presentation. The audience appeared to be highly engaged and interested in the examples and the technology. At the end, the audience was able to ask specific questions about the technology. I answered many questions, but after the room began clearing I noticed that one gentleman was standing in the back waiting. He was a tall, frail, white gentleman who wasn’t particularly dressed for the part. I distinctly remember that he wore a pair of blue jeans and a white ­t-​­shirt with a lightweight brown jacket. He made his way up to the front. His first words to me were, I didn’t appreciate that shit that you just shared in your presentation. I know the audience seems to think that you are smart, but I take issue with the solution that you designed for the city of Boston. Using facial recognition to identify folk without their knowledge or consent ain’t right!

he screamed. He went on to tell me his credentials in academics and that he had an actual building named after him at a prestigious institution in the southern part of the United States. Those who know me would probably have expected that I would have been quite volatile with the gentleman after his rant. To be honest, I indeed felt the “­project” coming out of me. For those readers who might not be familiar with

68

Hidden in White Sight

the word “­project” in this context, it means that I grew up in a ­high-​ ­crime, l­ow-​­income urban housing development. We say that word when we are about to do or say something that we might regret afterward. But I maintained my composure and asked the gentleman what particularly bothered him about my presentation. Well, he wasn’t in any mood to be amiable, so I beckoned security to come and intervene, which they did. I never really forgot about that experience in Vegas, but some years later, it hit me that the gentleman’s sentiments were not particularly different from those that I share now. I’m both embarrassed and remorseful to admit that I didn’t even consider how facial recognition systems could serve as a tool to harm communities of color. Although the technology wasn’t misused in this particular instance, it could have easily been abused if it had fallen into the hands of the wrong person. It leads one to entertain the question: should governments use facial recognition to classify and identify people? We know that it happens. I have worked with the Federal, state, and local governments, spending a lot of time with them. So as you might expect, I’ve seen requirements for all types of uses of biometrical solutions, from passports, driver’s licenses, airport security, traffic cameras, digital access, criminal investigations, watchlists, and persons of interest to social media applications. There’s no doubt that the use of facial recognition systems in government is rising. Many Federal and local service integrators have developed biometric systems to satisfy the government’s needs. Many of the software companies that the government uses have accumulated a gallery consisting of people’s faces by scraping publicly available data sources, like Instagram or TikTok, for photos. The Department of Justice and the Department of Homeland Security reported using Clearview, a company currently being sued for scraping social media for photos to fill its reference database. Clearview is not alone. There are dozens of facial recognition software companies operating without government oversight. “­There are rules and guidelines in place with regard to the use of personal information by the government,” says Ashkan Soltani, former Chief Technologist of the Federal Trade Commission. “­But they’re essentially laundering some of that by relying on ­third-​­party

AI Gone Wild

69

commercial entities that built their databases and trained their models based on publicly available data.” Even private corporations use biometrical solutions like facial recognition to identify and classify patrons. Even the most magical place on earth, Walt Disney World, recently experimented with facial recognition technology. In the test pilot, Walt Disney World limited the solution to its flagship park Magic Kingdom. The solution was quite simple: The facial recognition software captures an image of a guest’s face, converts it into a unique number, and then associates that number with the form of admission (­MagicBand, physical ticket, MagicMobile) used for park entry. The theme park’s use of this technology has gotten various expected reviews from park goers. I particularly like the idea of using facial recognition as an entry point into the parks. If used ethically, there could be many positive comments from facial recognition. One that comes to mind is helping parents find lost children faster. The detractors rightfully point out that the data captured by the cameras can be kept and used for other applications and uses, as well as for watch lists for keeping out people who might be considered undesirables. But as depicted above, far too often facial recognition is used for mass surveillance applications and causes more harm than good. As early as 2017, a growing number of Democratic lawmakers and civil libertarians voiced concerns about Amazon’s facial recognition software, Rekognition. The software that was sold to police departments across the United States matches images of faces from videos and photos with those in a database. The honorable and late Rep. Elijah Cummings (­­D-​­Md.) succinctly characterized the dangers prior to his death. “­If you’re Black, you’re more likely to be subjected to this technology, and the technology is more likely to be wrong,” he said. “­That’s a hell of a combination.” Technology is like a weapon. It can be used by a police officer, a soldier, or a dictator. In each one’s hand, it will yield a different result. As Carl Sagan puts it, “[Science] is not perfect. It can be misused. It is only a tool. But, it is by far the best tool we have…”

Chapter Five

An Enduring Legacy

Scaled Whiteness Our nation, I fear, will be ill served by the court’s refusal to remedy separate and unequal education, for unless our children begin to learn together, there is little hope that our people will ever learn to live together. —​­Supreme Court Justice Thurgood Marshall

F

“ rom the time Katrina was k ­ nee-​­high to a duck, she’s wanted to be a doctor. As a young girl, she would pretend to be the medical caretaker of her Cabbage Patch Doll,” says Katrina Hill’s mother who had herself grown up and endured through the Jim Crow era. DOI: 10.1201/9781003368755-671

72

Hidden in White Sight

Katrina would be ­hard-​­pressed to explain in detail exactly why she wanted to be a doctor. It could have been due to a number of reasons: just seeing how difficult it was for her family members to get medical ­care—​­or maybe it was the fact that they seemed to need it so ­often—​­or possibly because she innately believed that being forced to eat food that she hated couldn’t possibly be healthy. Regardless of the reason, for as long as Katrina could remember, she wanted to be a doctor, and she was fixated on achieving her dream. Going to the doctor wasn’t a regular occurrence in her family or neighborhood. Katrina could only remember going to the doctor a couple of times before graduating from high school. But the times that she did remember, all of it seemed strange. Katrina thought it was weird to pass by the first door that clearly stated, “­Waiting Room” with the words Doctor Campbell etched beneath. The doctor’s office was segmented into two main rooms: One led to the aforementioned “­front” door and the other room faced the back door. Katrina could easily see that the two rooms were different because the furniture wasn’t the same. The room that she was in had a ­worn-​­out and cracked leather sofa and two folding chairs. This room was somewhat dimly lit, while the room toward the front had a lovely plush flowery sofa and windows that ushered in the bright sun. Katrina quickly realized that she was in the waiting room designed for “­Blacks” only. She indeed was Black, so she was in the right place. Although Katrina didn’t bear the brutal hardships of Jim Crow, she did experience her share of discrimination and segregation at an early age. Katrina always considered it was weird how there were two separate waiting rooms to see the same doctor. Her middle school was integrated, and although no one at her school seemed to be particularly ­racist—​­She knew that there is a difference between her and her ­non-​­Black friends. Several white friends had even come to her home once for her birthday party, but she had never actually been invited to their homes. None of it seem to matter much to her as she was consumed at an early age by being a doctor. She certainly wasn’t going to let a little hurdle called race, derail her dream of being a doctor. Doctors were well respected and wore nice, clean, starched white coats. Most of all, it was known knowledge that doctors made lots of money.

An Enduring Legacy

73

However, she didn’t know for sure, since she didn’t know any doctors personally. After graduating from a prestigious university with an impressive GPA of 3.8, it seemed like Katrina’s ­long-​­desired goal of going to medical school was in reach. Not so fast, said the algorithm that was in charging of who was admitted. In the early 1980s, medical schools had begun using algorithms to screen and prioritize student applications for admission. Before leveraging AI, almost half of the school’s annual applications were rejected by the admissions office. The current dean was a former admissions assessor, and he was keenly aware that reading through applications was a t­ ime-​­consuming and tedious task. He envisioned a more efficient process leveraging automation. More importantly, he figured this would remove human bias and ensure that the best candidates got selected. After engaging and partnering with a leading technology company, they designed a new AI system that “­mimicked the behavior of the human assessors.” The motivation behind ceding to the technology advancements of AI was to make the process fairer by ensuring that each student application went through precisely the same evaluation, thus providing a more equitable approach. Unfortunately for Katrina, the opposite proved to be true. What’s worse is that she wasn’t the only one impacted by the algorithm’s automatic vetting and screening of the applicants. A couple of years later, some staff members became concerned about the lack of diversity among successful applicants. After a careful review of the algorithm, they noticed that specific rules in the system weighed applicants on the basics of n ­ on-​­relevant factors, like place of birth and name. An inquiry was launched with the School Board of Regents. The resulting investigation found out that the candidates were classified by the algorithm as “­white” or “­­non-​­white,” but not based on their race. Instead, they were organized by their names and places of birth. If they were characterized as “­­non-​­white,” the selection process was weighted against them. For example, simply having a ­non-​­European name could automatically take 20 points off an applicant’s score. The inquiry also found that an average of five points was taken off female applicants’ scores.

74

Hidden in White Sight

As a result, almost 50 applications each year may have been denied based on the social scoring system. The algorithm was sustaining the biases that already existed in the admissions system. Vetting and screening protocols leveraging social scoring algorithms have historically been used to disenfranchise populations of colors. These systems often remind us of our long American history of legalized racist systems like the Jim Crow laws and serve as a constant reminder that our technology is only as good as we are. Katrina was quickly reminded that even though the name may have changed from Jim Crow, the game remained the same. Racism is like a mobile device, there is a new “­model” every year. The last few iterations of the “­device called Jim Crow” has manifested itself through new technological advancements like AI. Jim Crow principles are not limited to admission algorithms. There are striking and piercing similarities in almost every segment of our daily e­ xperiences—​­many of which we chronicle in this book. Whether it is in our voting process, healthcare inequities, or miscarriages of justice mimicking the previous implementation of Jim Crow. Chances of someone from the United States not being at least casually familiar with the term Jim Crow are extremely rare, but I would like to take a few minutes to explain it to my international readers. Jim Crow laws were essentially a collection of state and local statutes that legalized racial ­segregation—​­leveraging a legal doctrine, “­Separate but Equal,” to get around the Fourteenth Amendment to the United States Constitution, which generally guaranteed “­equal protection” under the law. Those astute in history know that the potential extinguishing of Katrina’s childhood dream didn’t appear out of thin air, as educational bias and discrimination in America date back to the country’s origin. The original Jim Crow regimes started with a legal premise: Specific individuals could and should be classified based on race. This term “­classify” is important as it is a fundamental construct of AI also. Next, Jim Crow laws utilized vetting and screening systems to enforce segregation based upon designated racial classification. Was this an early blueprint or template for social scoring based on race? We discussed during the introduction that AI systems that make decisions based on skin tone are readily available in the marketplace and are

An Enduring Legacy

75

used for varying purposes today. So it’s not a huge leap to acknowledge that solutions can be designed to classify and vet by analyzing skin tone to determine race. Let’s look at a historical example: In Alabama, a federal district court ordered the University of Alabama to accept A ­ frican-​­American students Vivian Malone and James Hood to the summer session on May 16, 1963. The federal edict was considered an afront to the existing southern laws of the land, which clearly stated that students were vetted, classified, and screened based on race. Meaning that the University of Alabama was a white’s only institution of higher learning. The court’s decision almost guaranteed a clash between federal authorities and Alabama Governor George Wallace. Alabama officials had campaigned on a promise a year earlier to stop the school from integrating, even if it meant standing outside the schoolhouse door. Despite a federal court order which prohibited him from doing so, Wallace kept his campaign promise on June 11 when he stood in front of Foster Auditorium’s entryway and temporarily obstructed the students’ entrance. Although he eventually gave up when President John F. Kennedy federalized Alabama’s National Guard, the incident cemented Wallace’s reputation as a racial nationalist. Moreover, Wallace’s symbolic and horrific words, “­Segregation today. Segregation tomorrow. Segregation forever!” seem to underpin current discrimination tactics in administration admissions. This historical lesson is important as it gives credence to the fact that if an algorithm was written in 1963 to enforce this southern law, it would indeed follow the societal norm and enforce the “­whites only” doctrine and prohibit all ­non-​­whites from being admitted. It is widely known that racist, sexist, and homophobic biases color “­many” stuDiscriminatory practices dent evaluations in areas of discipline and of our past linger onward subject area. These evaluations also tend through technological whips that continue to strike us to include abusive comments directed towhile our backs are turned. ward people of color, women, and other marginalized groups, making student surveys objects of stress and anxiety. This abuse manifests in standardized testing impacted by cultural differences for decades. Similar evaluations are used even outside

76

Hidden in White Sight

the educational system as performance criteria leveraged during hiring, firing, and job promotion. An example of one such case is as follows: In 2020, the ­COVID-​­19 pandemic forced schools in England to cancel final exams nationwide, making it difficult to give out final grades and determine college placements. As a result, England’s Office of Qualifications and Examinations Regulation (­ Ofqual) turned to an algorithm to calculate student grades. The algorithm relied on teachers’ prediction of students’ final grades, academic performance, and, critically, a school’s historical performance data. The algorithm lowered 40% of t­eacher-​­provided grades in calculating the final results. In addition, analysis found that the algorithm was more likely to reduce grades for l­ower-​­income students and those who did not attend smaller, private schools. The Ofqual algorithm is another example of a mismatch between the predicted outcome of an algorithm and its actual prediction. Ofqual’s algorithm did not determine students’ real achievements throughout the year but instead expected how healthy students in a particular school “­should” do based on the teachers’ input. The algorithm’s focus on historical school performance as a predictor meant ­high-​­achieving students in poorly performing schools were more likely to have their grades lowered. However, educational bias is not just an issue relegated to Europe; several American school districts have adopted similar practices during C ­ OVID-​­19. Since the 1880s, the Jim Crow mindset has ravished and wrecked the lives of Black Americans. When we thought that the civil rights movement and later affirmative action had quenched the fiery fires of Jim Crow, he raised his ugly head again in the early 80s after former Presidents Ronald Reagan, George Bush, and Bill Clinton instigated the s­ o-​­called “­war on drugs.” This led to another iteration of Jim ­Crow—​­mass incarceration. Black men were taken from their families and given sentences that ranged from 5 years to life, all for petty crimes that their white counterparts would receive probation for. Jim Crow was reinventing itself and recementing itself within the fabric of this great nation. However, Jim Crow has experienced turbulence over the years in that it hasn’t taken hold as rapidly as its perpetrators would have hoped. In their eyes, the vengeance of Jim Crow hasn’t scaled fast enough. In the past, this happened because the said perpetrator

An Enduring Legacy

77

had to see and determine the victims. They had to screen and vet the target audience or see their faces to determine whether the potential victim was Black or white and therefore who to discriminate against. Once vetted and categorized, then and only then could the discriminatory action be handed out, whether it be keeping you out of a particular school, a specific club or institution, a neighborhood, or even keeping you from experiencing a specific medical treatment. Jim Crow was limited in its ability. One scholar purported that it takes at least 10 years for a discriminatory practice to embed itself into the culture and convince both the person performing the discrimination and the person being discriminated against to accept it as usual. However, there is a good chance that some rebellious faction will rise and declare it unjust, doing as Malcolm X proclaimed with the “­bullet or the ballot.” In today’s millennial justice world, the “­ballot” typically starts with taking it to the streets through marching or rioting. Thus far, the tipping point for most of the protests has been police brutality, I submit that the catalytic converter that will ignite most of the future combustion will be technological bias. So, Jim Crow needed to find a way to scale, so the next iteration has emerged. Ruha Benjamin, in her book, Race After Technology calls it the “­New Jim Code.” She presents the concept of the “­New Jim Code” to explore a range of discriminatory designs that encode inequity. Jim Code utilizes unjust algorithmic applications targeted to further restrict, restrain, and marginalize people of color. Most Blacks would argue that old man Jim Crow has never left us and will continue to be embedded in the defining fabric of American society in perpetuity. We’ve seen the Jim Crow mindset reinvent itself through slavery, separate but equal, mass incarcerations, and now algorithmic “­Jim Code.” Katrina’s experience in trying to become a doctor depicts how Jim Code has taken another dramatic and catastrophic turn on Blacks in America. Under the “­separate but equal” discrimination of a historical Jim Crow regime, state laws required mandatory separation and discrimination on the front end while purportedly establishing equality on the back end. In contrast, the Jim Code regime allows for “­equal but separate” discrimination.

78

Hidden in White Sight

Under Jim Code, equal vetting and database screening of qualified citizens like Katrina will appear so that fairness and equality principles are preserved on the front end. Jim Code, however, will enable discrimination on the back end in the form of designing, interpreting, and acting upon classification, vetting, and screening systems in ways that result in a disparate impact, like medical school denial. This means that Jim Code gives the illusion that it’s fair and neutral, but a specific group gets treated p ­ oorly—​­it might be coerced treatment, but it hurts just the same. Ouch. Michelle Alexander posits that “­Jim Crow should be evaluated as a constantly evolving process, not a historical artefact.” She further postulates in her book, The New Jim Crow: Mass Incarceration in the Age of Colorblindness that the United States criminal justice system uses the war on drugs as a tool for enforcing historical and ­modern-​­day forms of discrimination and oppression. These ­well-​­disguised systems of racism bear a striking resemblance to the old Jim Crow laws. Is America at its tipping point when it comes to mass incarceration? In a country that continues to imprison more people than it graduates from its universities, mass incarceration has emerged as a crucial yet defining civil rights crisis in recent years. This begins at its community entry point, the local jail system. Literally millions of men and women are jailed each year, primarily for crimes related to poverty, hunger, mental illness, addiction, theft, or simply not being able to pay their bond. Basically, they are arrested for petty crimes. Unjust prison sentences leave people, in some cases, homeless, unemployed, or in a family crisis. In addition to losing their civil rights in prison, most states severely restrict former prisoners from voting, getting a passport, carrying a firearm, and holding public office. This is a painful reminder that systems like Jim Crow are not just relegated to our past, but more in our present than in our past. American taxpayers spend almost $40 billion each year providing services for over two million prisoners. That’s over a 500% increase over the last 40 years. As I stated earlier, America’s policies and discriminatory practices have contributed to these appalling numbers. AI and other technological advancements are radically reshaping every aspect of our economy and society; yet none of these advances is helping to stem the tide of mass incarceration.

An Enduring Legacy

79

Apart from the fact that prisons are expensive and overcrowded, they are a revolving door. Within 5 years of their release, 76.6% of inmates will be rearrested. Over the last 10 years, I’ve known several people who have entered the prison system for crimes that could have easily been considered probationary. But unfortunately, whether it is ­over-​­policing, mandatory sentencing, inept legal counsel, traffic targeting, or even discriminatory jurors, Blacks are less likely to receive benefit of the doubt when determining whether they receive a slap on the wrist or a prison cell. As we’ve shared in this book thus far, unfortunately, technology has triggered most of these miscarriages of justice. We are painfully aware that AI, its algorithms, and data models can be used to help Blacks walk into prisons, but can that same AI ­ rison—​ technology be leveraged to keep people of color out of p ­better yet, can it be used to reduce recidivism and lessen the number of criminals reentering into the prison system? I would say, “­Yes.” Recently, I worked with a client who is developing an AI system that predicts recidivism for over 90% of inmates still trapped in American prisons. In addition, the algorithms could help identify which programs are best for assisting people in reintegrating back into society. Since race is a contributing factor to why two out of three falsely accused inmates are in prison today, one would think that using technology to level the playing field for prison and judicial reform would not only be a desire but an expectation. Prison officials need to develop a comprehensive plan to collect and analyze better data for predictive and recommendation algorithms to work. Our current prison structure keeps mounds of data on each prisoner, but similar amounts of data to reduce recidivism and stop police abuse are almost ­non-​­existent. The same data used to predict who will go to prison can also be leveraged to help those rightfully and wrongfully accused reintegrate into society. We’ve seen severe misuse of police bodycam data in the past few years. However, law enforcement has already collected millions of hours of data across the nation. This data can be leveraged to train sophisticated models to accurately project who is likely to go back to jail and recommend personalized programs to keep that from happening. AI needs ­real-​­life, practical, and representative data to

80

Hidden in White Sight

ensure algorithm and model accuracy. Law enforcement is collecting and curating this data daily. Let’s use it for GOOD. I’m reminded of the song by Mama Odie in the Disney classic, The Princess and the Frog: You got to dig a little deeper Find out who you are You got to dig a little deeper It really ain’t that far So, let’s dig a little deeper and have all hands on deck when it comes to this issue. Racial classification was an integral part of Jim Crow. Many southern states legalized racial classification by codifying the “­one drop” rule and adopting it as part of their state constitutions: an ounce of Blackness meant the individual was not white and therefore subject to the Jim Crow laws. Many laws define a white person as someone who has no trace of blood from another race. Only then would a person be classified as white. The classification of identity into r­ ace-​­based identity types such as those defined by Jim Crow is essential for establishing exclusion systems, but classification alone is not enough to create a discriminatory caste system. Implementing such a system requires checks and screening to determine who falls into the acceptance category. Jim Crow’s system did this with little or no accurate data. Typically, classification was based on a physical paper document such as a birth certificate, driver’s license, or even personal hearsay, whereas Jim Code’s classification system is based on large amounts of data pulled from multiple sources. Applications like the one submitted for med school by Katrina are checked and reviewed to determine if they meet category approval. Maybe the most straightforward way to understand how the Jim Crow regime is being replicated in a big data world through Jim Code is to consider the following scenario: Imagine substituting the ­race-​­based classifications with classifications derived from combining multiple data sources. Rather than relying on a standard protected set of attributes like race, gender, religion, or age as the sole ­ ata-​­based AI algodiscretionary based for inclusion or denial, big d rithms allow inclusion based on abstraction, such as inferred attributes deduced by the algorithm.

An Enduring Legacy

81

In college admission, discriminatory uses of AI algorithms are not being used to classify and vet who gets into a university. Still, algorithms are used to predict the likelihood that a student will accept if admitted and determine the amount of financial aid for the student. This type of bias is often overlooked in the Black community. Students often receive admission offers from l­esser-​­ranking schools or schools lower on their wish list because of a lack of scholarship opportunities from ­higher-​­rated institutions. If the admissions algorithm accepts a student but implores discriminatory practices in determining who gets a financial scholarship, it has long decided who attends the university. This widely accepted admission prediction process is referred to as econometric modeling. Jim Crow requires a human to physically evaluate a person’s skin color to determine their race in a limited data universe. Human judgment was also necessary for the domain of little data screening processes. (­Take, for instance, a county clerk whose job is to inspect birth certificates before issuing marriage licenses and marriage certificates to determine compliance with ­anti-​­miscegenation laws.) Human input is not necessary to categorize persons in an AI environment. Based only on digital data and digitized analysis such as algorithmic screening, an individual might be labeled as someone not qualified or that just doesn’t meet enrollment standards. Algorithmic tools analyze data to establish identity (­e.g. database screening systems that match names with driver’s licenses and other databases of personally identifiable information) and screen for suspicious digital profiles (­e.g. a r­ ecord-​­locator matching algorithm such as a database screening system that matches names with driver’s license, their numbers, and other databases of personally identifiable information). When using AI, regimes do not blatantly strive to create submissive social groups, unlike Jim Crow, which was designed to maintain a racial hierarchy established by slavery and colonization, but that could very well be the result. Although we have yet to encounter a fully developed Jim Crow algorithmic regime, it is essential to remember that Jim Crow laws took decades to evolve into a generalized legal system. We can now look back and judge all of Jim Crow’s history with hindsight, including the ability to reassess the legal precedent that allowed segregation to take root and become commonplace.

82

Hidden in White Sight

The segregation of every fountain, pool, bus, and train didn’t start with Jim Crow; it was only legalized by Jim Crow. Jim Crow eventually converted virtually every person into a racial categorization expert and screening specialist to comply with regulations. It made them ­law-​­abiding citizens in a literal sense. Many entities, both public and private, were required by law or social convention to classify and screen according to Jim Crow legislation. For example, under Jim Crow, one could not choose to break the classification system by marrying someone of a different race. Jim Crow required bus drivers to separate passengers based on skin color; restaurants had to separate customers based on a simple eye test; Jim Crow laws required hotel owners to ban ­non-​­white guests. Jim Crow was not just a cultural and social norm; it was a cultural mandate. White and Black violators were all subject to penalties of imprisonment, in some cases, death. I’m of the belief that Algorithmic Jim Crow or Jim Code has its foundation in the great digital divide. The commonly accepted definition of the digital divide is simply the gap between those who have access to the internet and have computers as opposed to those who don’t. However, the lack of computers and technology is not necessarily an inhibitor now, as many people of color have at least one smartphone. In addition, public and free access to the internet at places like your local public library, Starbucks, and even in your nearby park and recreational areas has made it possible for young and old alike to take advantage of the tremendous value that the internet brings. These advancements and available technological options would have us believe that the digital divide is not a “­thing” anymore. On the contrary, I submit that it’s more of a “­thing” now than ever before. The digital divide is multifaceted and includes numerous factors outside of availability and access, including affordability, quality, relevance, and education. Recently, we’ve seen examples where popular coffee shops who offer public wifi have discriminated against people of color. The misjustice was carried out by a human manager asking the person of color to leave, but recently we’ve seen cases where AI is being used to determine which patrons get wifi and which don’t. This can easily be implemented by attaching wifi connectivity to the cash register and making the decision based on how much one spends at the restaurant.

An Enduring Legacy

83

Technology isn’t just what your hands can touch, but is more about technology’s impact on your daily lives. What you eat, where you live, your yearly income, what school you or your kids attend, the car you drive, or the ads that convince you to purchase items. Items that you never considered buying before watching the ads. Technology is always at work, and in most cases, the thing that is convincing you to act is an algorithm. In 1962, at the height of Jim Crow’s reign, Everett Rogers published the book Diffusion of Innovation. The book explains the conditions required for new ideas and technologies to spread throughout society, but in the final chapter, he raises the issue of unintended consequences. He states what I refer to as the potential social ills of technological advancements. Although innovation can bring tremendous benefits to society, it often results in unintentional and not so ­people-​­friendly effects. Rogers refers to the conditions created by innovation’s sometimes negative effect as “­disequilibrium.” When disequilibrium occurs, an innovation advances faster than society, research, and policy, reducing the ability to identify or assess any adverse effects. Could AI be a technology that causes the disequilibrium that Everett Rogers refers to? Katrina’s medical school tragedy was one such case where disequilibrium resulting from AI bias happened in society and, more specifically, in education. Rogers wasn’t necessarily considering ­A I—​­at least, I don’t think he ­was—​­when he authored the book. But he certainly was mindful of how technology up to 1962 had contributed to the drastic and widened gap in education, wealth, and opportunity between Blacks and whites in America. He was mindful that if the mere art and casual norm of reading a book were considered astute technologically, this too had escaped the grasp of most Blacks. While reading was a personal choice in 1962, under Jim Crow a generation earlier, the ancestors of those same Blacks weren’t allowed to read and write. And 20 years after authoring his book, the crippling educational system that fostered institutional racism would help incarcerate millions of Blacks under the New Jim Crow regime. Then again, 20 years after the age of mass incarceration, AI would ­super-​­size and scale the effects of the other two regimes, doing so by keeping technologies out of the hands of our youngest. After those

84

Hidden in White Sight

resourceful young people found ways to capture that elusive technology and a level of success, then AI would stack through biased admission algorithms to keep them out of the school of their c­ hoice—​­and for those who were fortunate enough to slip through the cracks because of their own personal ingenuity and persistence, then AI’s biased financial aid scoring algorithms would inflict more bias to ensure that, even if they were admitted, financially they couldn’t afford it. If only Governor George Wallace had known that he didn’t have to block the doors at the University of Alabama to keep overly ­ ut—​­all he really had to qualified Vivian Malone and James Hood o do was wait until he could witness the digital divide promulgated by AI developers. If only he had been born 50 years later, with the assistance of technology, Governor Wallace might have reached his desired outcome.

Chapter Six

Our Authentic Selves

No Seats Left at the Table We all should know that diversity makes for a rich tapestry, and we must understand that all the threads of the tapestry are equal in value no matter what their color. —​­Maya Angelou

G

Who’s Coming to Dinner? is a 1967 American romantic ­comedy-​ d ­ rama. The film depicts the social and cultural divide between Black and white Americans in the 1960s. The film was one of the few of the era to portray interracial marriage positively, as interracial marriage was historically illegal in many states in the United States of uess

DOI: 10.1201/9781003368755-785

86

Hidden in White Sight

America. It was still illegal in 17 states until June 12, 1967, a mere six months before the film was released. In the film, Joanna Drayton, a 2 ­ 3-­​­­year-​­old white woman, returned from her Hawaiian vacation to her parents’ home in San Francisco. Accompanying her was Dr. John Prentice, a ­37-­​­­year-​­old Black widower. The couple had got engaged after a whirlwind ­10-​­day romance. Joanna’s parents were Matt Drayton, a successful newspaper publisher, and his wife, Christina, who owned an art gallery. While both Draytons considered themselves liberals for the time and wouldn’t dare imagine themselves as racists, they were still shocked that their daughter was engaged to a Black man. Christina gradually came to terms with the situation, but still, Matt resisted due to the likely misfortune and the seemingly insurmountable problems that the couple would face in their lives. Without telling Joanna, John told the Draytons that he would withdraw from the relationship unless both Draytons gave their blessing to the couple. To further complicate matters, John was scheduled to fly to New York later that night and then to Geneva, Switzerland, for three months for his work with the World Health Organization. Therefore, the Draytons’ answers would also determine whether Joanna would follow him. Tillie, the Draytons’ Black housekeeper, suspicious of John’s motives and protective of Joanna, privately cornered John and spoke her mind. To John’s surprise, Joanna invited his parents to fly up from Los Angeles to join them for dinner that evening. They didn’t know that Joanna was white and were initially just as against the marriage as Joanna’s parents were. The two mothers met and agreed that this was an unexpected event but supported their children. Finally, the two fathers also met, expressing disapproval at this sad occasion. The movie ended with a passionate speech by Joanna’s father, in which he gave the couple a dose of social American reality. They would face enormous problems due to their racial differences, but if they found a way to overcome them, he would approve of the marriage, knowing he had no right to stop it all along. The families and the Monsignor, who was also in the party, then adjourned to the dining room for dinner. Let’s consider the diverse opinions of those seated at the table. Joanna was a f­ree-​­thinking white woman that happened to be deeply in love with John. Her upbringing taught her that all people

Our Authentic Selves

87

were equal and that discriminating because of race was wrong and immoral. John was there. He had already survived the generational curse of poverty to become a doctor. John was brought up to believe that falling in love wasn’t necessarily a choice but a destiny. It was an almost unique and unlikely experience that found you instead of you searching for it. He found love on vacation in Hawaii. Joanna’s parents were seated at the table. They, too, had spent most of their lives telling anyone that would listen that discrimination based on one’s color was not a coat that one wished to wear. They were both liberal parents who instilled the value of equality in their daughters. I mean, they marched with King. Certainly, they couldn’t be racists. Right? John’s parents, too, were sitting at the table. As Blacks who grew up in not just a segregated America, but one that had perpetuated the myth that Blacks were less than equal to their white counterparts, their opinion on interracial marriage was not remarkably different from Joanna’s parents. They believed that it was wrong a­ lso—​­but their reasoning for their belief was quite different. Unlike Joanna’s parents, their experiences, culture, and race were the founding principles behind their perspective and belief system around race. They felt that certainly their ­well-​­spoken and successful son could find a ­like-​­minded Black woman to marry. Next, there was Monsignor Ryan, after learning who the other guests were, famously quipped, “­Oh… well, in that case, you’ll actually need me. Otherwise, your side won’t even outnumber the Blacks!” He believes that marriage should be based on love, and there should be a mutual understanding between the two different races. He, too, was invited to sit at the table. Then finally, you had the maid. In the movie’s opening scenes, it was clear that, although she comes from a Black family, she disapproves of interracial marriage. Still, she had a totally different reason for her belief. She didn’t have a seat at the table per se as she was the one preparing the table, but mentally she had brought her own seat, probably because she thought that she owned the table. In her opinion, why wouldn’t she, as she had worked for the Draytons for over 30 years and had practically raised Joanna. So, there you g ­ o—​­eight participants, all with different voices and perspectives, occupied seats at the table.

88

Hidden in White Sight

Over the years, film critics have analyzed and evaluated the film and debated the lessons intended by the screenwriters. Some say it was about the bias toward interracial marriage that was prevalent during the time, others who are overly optimistic say it was a love story. I tend to agree that the writers were of a mind to include both sentiments when crafting the script, but I’ll suggest another one too: Maybe, just maybe, the divinity behind this movie is that it took all the diversity and the plethora of opinions, experiences, and cultures to remind us that all voices matter. Only when we appreciate our differences enough to share them, will we find the solutions that seemingly escape us on subjects of race, love, gender, and religion. Guess Who’s Coming to Dinner? was an international box office hit, grossing over $70,000,000 and nominated for 22 different awards. In 2017, the Library of Congress entered the movie into the National Film Registry for being culturally and historically significant. This is a unique movie in that it allows us to see the power of unity, the power of diversity and inclusion, the power of having a difference of opinion. Of course, all the cast of characters were different. Yet, the movie subtly demonstrated how people from diverse backgrounds, religions, races, gender, and cultural background could do something as simple as sitting at the table together. Corporations, and even those of us who assemble AI teams, could learn a lot from the movie related to building diverse and When there are no seats left at the table, then bring your inclusive teams. Of course, most would own ­chair—​­A chair called say that diversity is either in race or genyou. der. But the film proves that just having various racial groups doesn’t guarantee a diversity of opinion and thought. For example, even though the maid was Black, one shouldn’t suppose she shared a standard view with the other Black characters because she didn’t. I know it’s hard for many n ­ on-​­Blacks to conclude this, but all Blacks don’t look alike, and neither do we all think alike on subjects such as race. The goal of any technology team (­including one focused on AI projects) is to solve complex problems. If all team members think alike, they solve the issue similarly. If a problem is complicated, it

Our Authentic Selves

89

will be difficult for these team members to develop a solution, yet a team made up of people who think in different ways would be able to solve issues faster with more creative solutions, and their answers are less likely to harm others. In June 2020, the artificial intelligence world was spinning off its axis. Conversations on Twitter exploded after a new artificial intelligence tool for creating realistic, ­high-​­resolution images of people from pixelated photos displayed its true ­colors—​­racial bias. Face Depixelizer converted a pixelated yet recognizable photo of former President Barack Obama into a ­high-​­resolution photo of a white man. Researchers soon posted images of other famous Black and Asian people, and other people of color, being turned white. Senior AI technologists (­including me) weighed in on the Twitter beef between two ­well-​­known AI corporate ­researchers—​­Facebook chief AI scientist Yann LeCun and Google’s then ­co-​­lead of AI ­ethics, Timnit Gebru. They both took two strongly contradictory views on interpreting the tool’s error. Like two celebrity rappers or actors debating on the lyrical meaning of a song or movie, the AI technical community took sides. As I followed the debate online, it was disheartening for me to see, both online and in my own personal work, that Gebru seemingly was losing the argument (­Today, Gebru’s opinion is widely recognized as factual). But during the actual debate, I was a little concerned. Then it struck me that most of the Twitter commenters representing the AI community were white. This isn’t groundbreaking news since only about 2% of AI practitioners are Black, but we’ll provide more evidence on the assertion later in this chapter. The entire dialog between the two AI ­heavy-​­hitters and corporate influencers started with the release of a computer vision model called Pulse. Pulse was developed by researchers at Duke University that claimed that it could generate realistic, ­high-​­resolution images from a pixelated photo. So, in simple terms, if you took a selfie using your iPhone and updated the photo into the model, the drifting of the model unintentionally converted photos to white. In the words of my grandma, “­How can it be?” Brace yourself; I’m about to get a ­t ad-​­bit technical. Bear with me! This controversial algorithm combined generative adversarial networks, which the industry refers to as GANs with ­self-​­supervised learning. For example, the algorithm can convert the image from

90

Hidden in White Sight

Black to white without the designer labeling the data, if we speak in layman’s terms. However, the user would tag the data (­i.e. a dog’s picture accompanied by the label “­dog”). Okay, enough digression; I’m moving on with the tech ­battle—​­I hope you got it. We mentioned earlier in this book that with AI, you must train the algorithm with sample data for it to learn how to do the required task. Pulse was trained using the Flickr Face HQ data set. The data set seemed to work fine with white faces but struggled with ­non-​­white faces. As we discussed earlier, it’s all about the training data. In the case of the Flickr Face HQ data set, 90% of the imported faces were of white males. So, from one observer’s input of a ­de-​­pixelated photo of President Obama, the Pulse algorithms produced a picture of a white man. This seemly innocent computer glitch of mistaken identity might seem harmless today, but 50 years from now, it could prove catastrophic. Of course, none knows what technology will adorn us in 2072, but what if a ­10-­​­­year-​­old was googling our historical presidents, and somehow the white photo of Barack generated by the algorithm pops us? Can you imagine the confused look on the adolescent’s face? Oh, we’ve seen this movie b ­ efore—​­people and historical events misrepresented to satisfy the interest of the powerful using time and forgetfulness as its purveyor. I joined in during the Twitter beef by inputting pictures of some of my favorite Black Americans. The t­ool-​­generated images gave Samuel L. Jackson blond hair and turned Muhammad Ali and Colin Powell into white men. Analysis of a portion of the data set found far more white women and men than Black women, but people quickly took issue with the assertion that bias is about data alone. Gebru then suggested LeCun watch her ­tutorial—​­whose central message is that AI bias cannot be reduced to data ­alone—​­or explore the work of other experts who have said the same. Before the debate with Gebru and LeCun, I would have pointed ­ hapter  1 as the reason for to the jellybean example discussed in C this disparity. But there is more to it than just the data. Why is this important? We’ve seen other religious and cultural characters whose photos were turned white and have thus lost their identity as history progressed.

Our Authentic Selves

91

Whitewashing emerged when Blacks weren’t readily allowed to play significant roles in Hollywood in early film production. In the case of whitewashing, the individuals made personal choices to alter the color of their skin to play a character; in the case of the AI algorithm above, no personal choice was made by President Obama. Instead, it was done automatically by an artificial intelligence algorithm. Imagine that in the year 2080, some Black kid is doing a history project on US Presidents, searching the internet for a photo of the 44th US President. The search engine shows the same white image generated by the Face Depixelizer as its optimized result. If there was a takeaway from the racist algorithm, it would be that the issue wasn’t just about data or even underrepresented data in the model. As we have often stated in this book, incomplete and unrepresentative data sets significantly contribute to AI bias. Still, it’s certainly not the most pervasive reasoning for it. Shortly after the death of one resident of Minneapolis, George Floyd, and the ensuing nationwide protests, I and some other technical leaders were asked to lead a project to promote police and judicial reforms. The goal was to get a diverse group of unlike minds from different job roles, races, genders, cultures, and ethnic backgrounds to create a group of applications around the idea of “­tech for good.” An industry term for the project was a “­Call for Code.” Tech for the greater good! I can’t remember the particular demographic breakout of our team, but it was unusually diverse. I remember the refreshing feeling of hearing lots of different voices on the topic of race. What was unique about this team exercise is that we could select whatever problem we wanted. So we could work on solutions that impacted the court system, policing, prison reform, or any other judicial or policing problem that was facilitating the ills of social justice. We were not naïve enough to think that we could fix the issues plaguing American society that had caused millions of people to take it to the streets. But we were nonetheless optimistic enough to believe that a diverse group of “­thought leaders” could at least bite a chunk out of this oversized elephant. By the way, we were all aware that we were eating an elephant. Our team consisted of all the typical design and development roles. We had a group of UI designers, data scientists, AI and mobile developers, testers, legal analysts, and employees who were just social

92

Hidden in White Sight

justice enthusiasts who wanted to make a difference. So we solicited input from not just our internal teams but from the external community. We interviewed police officers, a local judge, and even a senator. The attention on the George Floyd case meant that we didn’t have any issues getting input from internal and external supporters. Our table was fully occupied, and everyone had a seat. We were all seated and diversely r­ epresented—​­now what? We struggled to make any headway for the first couple of weeks. We struggled to get moving, doing nothing other than having lots of conversations that included a fair share of disagreements. Early on, our biggest challenge was to determine who within the policing and judicial system we would build our app for. For example, would we develop apps to help Blacks who were potentially targeted by the police? Would we create an app better to provide situational awareness and insight for the police officer? Or maybe, we would build solutions assisting the judges or other members of the judicial staff, or perhaps we would build a social media app that allowed community observers to film and post unjust police behavior. For example, Darnella Frazier was only 17, and she made history filming the George Floyd murder. Her video clip went viral and became a key piece of evidence against his killer. We could design, develop, and deploy an app that would have facilitated an even better experience for Darnella. One of our team members from London suggested the unpopular but bold proposal that we should create a tool that helps recommend a course of action for the officer when put in adverse situations. Police officers often experience mental strain in their daily activities because of hypervigilance toward their surroundings and those they interact with. With his proposal, we would spend our time focused on the day in the life of a police officer. During a subsequent meeting with the coworker, I discovered that his dad was an officer and several of his family members were in law enforcement. Although my coworker was passionate about racial justice and reform, he had a personal, emotional experience that provided great insight into our project. Another team member thought that the injustice issue lay square at the feet of the judicial system, so he proposed to develop a set of apps that allowed judges to be more equitable and fairer in their sentencing. The existing solution for sentence recommendation was a

Our Authentic Selves

93

static r­ ule-​­based one that didn’t give the judges much flexibility. His proposed solution would not just recommend sentencing, but would predict the defendant’s chances of committing the crime again. Interesting idea. I liked this one, but another one of my colleagues reminded me that judges already use tools to provide sentencing guidelines and can exercise lots of interpretation within the current sentencing solutions. Still, the judicial system is riddled with sentencing injustices. She reminded me that the issues are not with the tools per se, but with the judges who interpret the data and the recommendations provided by the sentencing guidelines. More importantly, the laws were legislated in the halls of Congress by people who “­like all of us” share some racist and biased tendencies. Naturally, therefore, the data provided by the court system to our team would have likely included that same bias. Another team member was h ­ yper-​­focused on who she called the real victims, the George Floyds of the world. She was a ­middle-​­aged white woman whom I still consider one of the most significant contributors to our project’s effort. I don’t really think that we would have gotten to the end of the job without her thought leadership, astonishing candor, and fortitude. Her recommendation was to create a tool that would allow ordinary citizens to view and open complaints against police officers. If the police pulled you over, you could quickly, from your phone, look them up to see whether there were prior cases against this officer. Other police agencies would also have access to this database, which would assist in their hiring decisions. This system would allow citizens to see and update a public database on police misconduct, including other public officials. Apart from being able to open and view complaints, the users could attach comments and videos illustrating police misconduct. We knew this latter proposal would create lots of legal and privacy issues. Folk from the neighborhoods I grew up in don’t trust the police and trust judges even less. So I didn’t want to work on a project that would provide a tool to help police and judges do what they were already quite accomplished in doing: misinterpreting justice. We chose to build the solution to allow citizens to view and open complaints against police officers. I’ve worked on hundreds of technology teams over my career, but this was by far the most fun that I’ve had working with a group of people. Maybe it was the social upheaval

94

Hidden in White Sight

in the streets around the George Floyd incident, or perhaps it was the exuberating feeling of satisfaction or the fact that finally—​­yes, I said “­Finally”—​­because the team members were free to be their authentic selves and we could discuss the cultural perspectives and differences that went into our delivering of the solution. Without diversity of thought, it is impossible to get an accurate, representative data set. One of my friends and coworkers helped me reconcile why I put this project over all the others. It wasn’t just the diversity of races that made this project my ­all-​­time favorite. This project encompassed the five core principles for ensuring a successful AI project regardless of the problem that you are solving: We were fortunate enough to experience all of them on the above project.

#1 Team Diversity Team diversity is the first action that needs to exist to combat racial bias in artificial intelligence. Next, companies simply need to hire more diverse talent in the technical AI field and across the board. This is important because delivering robust and fair AI systems often requires ­non-​­technical voices to ensure proper deployment. For example, if none of the researchers and designers building facial recognition systems is Black, ensuring that n ­ on-​­white faces are properly distinguished will most likely be a low priority. Although most tech companies have ­well-​­organized diversity and inclusion strategies, they continue to be horrible at hiring diverse workforces, yet most of these same companies have diversity and inclusion officers who are Black. I might add that I don’t believe that the lack of diversity that shows up at the doorstep of tech companies is the fault of those diversity and inclusion officers. Hiring is typically done at an organizational and department level at most companies. This leaves the diversity of a particular team up to the druthers of that management team. A study conducted in 2019 found that under 5.7% of Google employees were Hispanic and 3.3% of them were Blacks. Similarly, low rates also exist across the tech industry. And these numbers are hardly better outside the tech industry, with Hispanic and Black employees making up just 7% and 9% of STEM workers in the general

Our Authentic Selves

95

economy. (­They comprise 18.5% and 13.4% of the US population.) Again, data science is a particular ­standout—​­by one estimate, it underrepresented Hispanics, women, and Blacks more than any other position in the tech industry. The painful reality is that AI technologists are not nearly diverse enough to really reflect the concerns of the populations they impact in the industry. Therefore, having multidisciplinary teams researching, designing, developing, operationalizing, and managing algorithms is critical to ensure fair and unbiased solutions.

#2 Culture Cultivation Even when team diversity exists in AI design, the ability to express one’s culture is not readily promoted. Culture is something that shapes us and builds our personality; it influences all of our behavior and shapes our identity. Culture is our “­way of being or our expression of self.” It’s our beliefs, values, norms, and shared expressions. When an open and accepting environment frees us up to be our authentic selves, our diversity provides the necessary ingredient for social impact. For example, when I entered corporate America, I couldn’t express myself as a Black man. I was told even by other Black senior professionals who came before me that to be “­Black in corporate America” was frowned upon. So, I couldn’t wear an afro, I couldn’t have an earring, and under no circumstance could I have an exposed tattoo. As a matter of fact, even an unexposed tattoo, if discovered, would be a surefire way of not getting the job. This wasn’t just a white rule; it was strongly enforced even by Blacks in leadership roles. Those who served as gatekeepers or owners of the “­right of passage” determined whether the younger technologists would enter the corporate technology field. Just recently, I was invited to participate in a panel of diverse corporate leaders to speak to about 200 technical employees who had been with the company for less than 5 years. The panel was moderated by the state’s senior corporate executive. Each panel participant was given five minutes upfront for introductions, which included telling the audience about where they were from and what caused them to pursue a career in technology.

96

Hidden in White Sight

Most Black technologists over the age of forty have been invited to participate in at least one g ­ ive-​­back session like this. During my fi ­ ve-​ ­minute introduction, I shared that computer science wasn’t a childhood dream or even a fleeting thought when I grew up. Survival, the next hustle, or what we called in the projects where I grew up “­making a living” were the goal. I shared with the audience that the corporate dress code was a foreign language to me. I assume that my personal transparency must have resonated with the audience because I was inundated with questions during the Q&A session. One particular man stood out in the audience. He and I made eye contact early in the session. Although he was sitting about eight rows back from the stage, he was easily distinguishable because of his appearance. Most of the audience was clothed in sports jackets and ties. He had on a pair of blue jeans and a striped blue sweater. His hair wasn’t faded with a fresh, lined haircut; instead, it looked like it had about 3 years of growth. His ears were pierced, and I could see the earrings sparkling even from my comfortable seat on the stage. Although I didn’t see any noticeable ink marks, I suspected that there were a hidden tattoo or two underneath the shirt. After three earlier questions, he raised his hand and gained the moderator’s attention, who subsequently pointed to him as the next questioner. Then, he politely directed his question to me. “­Calvin, how do I be myself at this company?” he asked. Even though I was fully aware of what he meant by the question, I asked him to explain what he meant by “­being myself at this company.” He continued, “­I grew up about thirty miles from this location in an impoverished community where no one expected me to be here. And to be blatantly honest, I too didn’t expect myself to be where I am.” The audience, as well as the other panel members, seemed stunned by his authenticity and the courageous manner in which he asked the question. I hesitated for a few seconds, reflected upon the question, and realized that this was a question that I myself might have asked some 25 years earlier! I responded, Brother, this company needs you more than you need this company. We don’t need you to leave your culture out in the vestibule; your authentic

Our Authentic Selves

97

self is not only welcomed but required. So your hair, earrings, brain, experience, fears, accomplishments, and yes, your street knowledge, is welcomed.

If not, it damn well should be! Now I didn’t know whether the company felt the same way I did, but it was for sure that I did. He was I, and I was him.

#3 Voice Amplification and Collaboration You’ve heard of BLM. Now it’s time for “­ MOM”—​­ “­ My Opinion Matters.” It’s one thing being invited to the table, but it’s another thing to have a voice. The most lasting takeaway from the movie Guess Who’s Coming to Dinner? was that everyone was invited to have dinner at the table. Still, more importantly, everyone had a voice, an opinion on why they thought the couple shouldn’t marry. Regardless of their roles in the family, their voices were heard. While everyone’s opinion can’t be implemented, no one’s opinion should be impermissible, and everyone’s opinion can be amplified. Many facial and voice recognition data sets are taken from the development ­community—​­sometimes from representative data sets that come from people who work for the company building the AI system. It is not uncommon for the teams building the AI solutions to use their own team members to provide data to train the models used in the application. If those teams aren’t diverse, then obviously that would account for underrepresented data in those applications. At the minimum, if Black voices are not heard at the design and development table, it will be impossible for those omitted voices to weigh in when data sets are suspected of bias. Much like facial recognition implementations, voice recognition systems also can be riddled with biased models and algorithms. A popular study published in 2020 scrutinized how voice recognition systems from Amazon, Apple, Google, IBM, and Microsoft fared when transcribing the voices of white and ­non-​­white people. During the research exercise, the team provided the voice recognition systems with almost 20 hours of interviews to transcribe from 73 Black and 42 white interviewees. The average error rate for the white

98

Hidden in White Sight

interviewees was 19%, whereas for the Black interviewees it was 35%. As with facial recognition systems, this voice bias is due to the data sets being composed predominantly of white people. Recently at the “­Building Responsible US Leadership in AI” webinar, Rep. Jan Schakowsky, head of the House Energy and Commerce Consumer Protection Subcommittee, said in her keynote, “­The future of responsible artificial intelligence is in the diversity of voices we need to hear from.” Over the last few years, we have seen that AI is susceptible to bias because the people who created it are susceptible to bias, especially against women and people of color. AI is being used to determine whether some people get parole, get homes, get into schools and even get a job. These are not trivial decisions; they’ve formed the foundation of our lives, and we must take seriously the dangers accompanying AI’s promise.

Those of us commissioned to design and develop the system that Rep. Schakowsky speaks on know that her comments are spot on. We need to invite diverse minds to the table, but we also need to welcome their cultural differences and see those differences as having value. But not only that, we need to give those voices a platform to speak. Since people of color aren’t accustomed to being invited or welcomed at the table, it is understandable that their voices might seem dim at first. But like a voice amplifier in the music industry that accommodates an individual who has trouble speaking loudly enough so that they can be heard in noisy environments, we must encourage Black voices to be heard and welcome their input.

#4 Team Member Contributions Made Visible People of color frequently report feeling invisible to their bosses. They frequently complain that their contributions are ignored or undermined. Nearly 40% of Black technology employees have experienced racial discrimination in the workplace. One out of every four Black professionals report that others are given credit for their work. This is particularly true in the technology field and doubly true in artificial intelligence and data science. Each team member’s

Our Authentic Selves

99

contribution is probably the most critical contributor to AI success. AI is a complicated technological space and requires total commitment. Unlike some technology areas, AI requires one to be multifaceted and multidisciplinary. For instance, a seasoned Python developer is often proficient in data science and vice versa. Both must have the keen business acumen to design and develop b ­ ias-​­free code and functionally accurate systems. Leaders must make a joint effort to ensure that all team members are incentivized and recognized for their contributions. Doing so will ensure that you have active participants in each phase of the AI design, development, and deployment process.

#5 Empowerment The ­Merriam-​­Webster dictionary defines empowerment as “­authority or power given to someone to do something.” A secondary definition that I like personally is: “­The process of becoming stronger and more confident, especially in controlling one’s life and claiming one’s rights.” In the field of AI, a person needs to be empowered to be their authentic self. Earlier, I depicted how white AI designers are empowered naturally to be their authentic selves by default which, in some cases, accounts for the biases of the algorithms that we deploy. If the prior statement is true, then by default, Black designers must be empowered to be their authentic selves. This playing field leveling goes a long way toward canceling bad design behavior. I suspect that my white friends and colleagues might say, “­W hy can’t you be your authentic ­self—​­who’s holding you back? You are empowered.” I would respond to those white voices, “­You are ­correct—​­I feel empowered, but I’ve been working in the tech field for 25 years, which has afforded me a level of swag and charisma brought about by many years of success.” That level of awareness hasn’t always been the case; therefore, we should all assume that Black technologists new to the field might struggle to have their voices heard. They will be compared with their white colleagues with equal tenure in the industry. Those white

100

Hidden in White Sight

colleagues are well accustomed to using their voices and being their authentic selves. This accounts for the rampant amount of AI bias in our applications. Let’s empower and invite a village of multidisciplinary technologists to the table and ensure that the massive power of AI positively impacts the world. A few years back, while attending a diversity and inclusion event, the instructor shared the following story with the attendees: A giraffe had a new home built according to his family’s specifications in a small suburban community. It was a beautiful and wonderful house for giraffes to live in. It had soaring ceilings and tall doorways to accommodate them. High windows on the walls ensured maximum light along with good views, while at the same time protecting the privacy of the family. Narrow hallways, wide enough for giraffes, saved a lot of valuable space without ever compromising convenience. The house was designed so well that it won the National Giraffe Home of the Year Award. The homeowners were very proud of their perfect house. One day, while working in his ­state-­​­­of-­​­­the-​­art woodshop in the basement, the giraffe happened to look out of the window. Coming down the street was an elephant. “­I know him,” he thought to himself. “­We worked together on a PTA committee. He’s an excellent woodworker, too. So I think I’ll invite him in to see my new shop. Who knows, maybe we can even work together on some of the projects?” So the giraffe poked his head out the window and invited the elephant into his house. The elephant was delighted by the offer; he had liked working with the giraffe and always looked forward to knowing him better. Besides, he had heard about the woodshop and wanted to see it for himself. So he accepted the invitation and walked up to the basement door, waiting for the giraffe to open it. “­Come in, come in,” said the giraffe, signaling with his hooves. But immediately, they encountered a problem. While the elephant could get his head through the door, he could not go in any farther. “­It’s a good thing that we made this door expandable to accommodate all of my woodshop equipment,” said the giraffe with a smile. “­Give me a minute while I take care of our little problem.” He went to the sides of the door and removed some bolts and panels to let the elephant in. Soon, the two acquaintances were happily

Our Authentic Selves

101

exchanging their woodworking stories. As they were talking, the giraffe’s wife leaned her head down the basement stairs and called to her husband. “­Telephone, dear. It’s your boss.” “­I’d better take that upstairs in the den,” the giraffe told the elephant, quickly making his way up the stairs. “­Please make yourself at home; this may take a while.” The elephant looked around, saw a ­half-​­finished piece of work carefully placed on the lathe table in the far corner, and decided to explore it further. As he moved through the doorway that led to that part of the shop, he heard an ominous crunch. He backed out, scratching his head, puzzled by the weird noise. “­Maybe I’ll join the giraffe upstairs,” he thought to himself. But as he started to make his way up, he heard the stairs begin to crack. Not knowing what to do, he jumped off and fell back against the wall. It, too, began to crumble. So he decided to just stay where he was so that he wouldn’t cause any more trouble. As he sat there, disheveled and dismayed, looking at the mess he had created, the giraffe came down the stairs having heard all the commotion. “­W hat on Earth has happened here?” he asked with amazement in his voice, shocked to see the mess. “­I was trying to make myself at home,” said the elephant. The giraffe looked around. “­Okay, I see the problem here. The doorway is too narrow. We’ll have to make you smaller,” said the giraffe before continuing. “­There’s an aerobics studio near here. If you’d take some classes there, we could get you down to size.” “­Maybe,” the elephant said, not looking very convinced. “­The stairs are just too weak to carry your weight,” the giraffe continued. “­If you took some ballet classes at night, I’m sure we could get you light on your feet. I really hope you’ll do it. I like having you here.” “­Perhaps,” said the elephant. “­But to be very honest with you, I’m not sure that a house designed for a giraffe will ever really work for an elephant of my size, not unless there are some major modifications made to it.” The giraffe and elephant story depicts how diversity, equity, and inclusion are often approached within organizations. Unfortunately, they are built for a particular group or culture of people who think,

102

Hidden in White Sight

act, and make decisions alike, excluding others who aren’t from the same background, culture, and experiences. In our story, the giraffes represent the m ­ ale-​­dominated white groups with similar cultural upbringing and education, have had the same or similar experiences, and usually see things from the same overall standpoint. Like the giraffe, they recognize others who are different from them and may want to work with them and include them in what they do because they like them or know that it would be good to include them for business purposes. The others are the elephants. They look different, have diverse cultural backgrounds, and different points of view, but are valuable and can contribute. However, like the elephant in this story, who could get his head in the door but not enter totally because he’s built different, too often when people of color are invited into the organization, we often don’t fit in because the door of opportunity is too small. Subsequently, we feel we must not rock the boat if we want to blend in, and then stumble trying to fit in. The giraffes of the organization can see the elephants are having difficulty. So the approach is to advise the elephants (­a diverse group) to change their ways, without really understanding the problem and without any clue that the organization or house they’ve built needs to change to fit the diverse person in. So although the house (­or algorithm) was ­award-​­winning and built to specification, it was designed only for giraffes. Without this recognition, the organization or company’s equity, diversity, and inclusion program will be doomed to fail. Finally, we must invite people of color to the table. Being a person of color doesn’t necessarily mean that you are immune to discriminating against yourself. We all don’t think alike. As was evident in the earlier “­Looks who’s coming to dinner” story, not all of the people of color cast members shared the same perspective on the potential marriage. We are programmed not to necessarily have our best interest at hand. Being a Black engineer doesn’t preclude us from being infected by the social disease of looking down to ensure our doors are locked when we see unknown Black youth approaching our cars. So our unconscious minds tell us to agree with social norms, much like those of our white counterparts. Therefore, the code that we program and the solutions that we develop often mimic our societal beliefs even about ourselves.

Our Authentic Selves

103

This doesn’t mean that we shouldn’t be at the ­table—​­actually, it means the ­opposite—​­but we have an obligation to ensure that we are cognizant and aware that we are prone to discriminate against ourselves and counteract it with positive aggression. It’s not about being offensive or hostile, but standing up for oneself to say “­no” or “­I won’t.” “­I won’t code that algorithm; I won’t build that app; no, I won’t build that m ­ odel—​­not with that data and neither should you.”

Chapter Seven

Mass Unemployment

Undeserved Rest Technology is nothing. What’s important is that you have faith in people, that they’re basically good and smart—​­and if you give them tools, they’ll do won‑ derful things with them. —​­Steve Jobs

I

n the early 1990s, McDonald’s released a series of commercials targeting the inner city and the Black community. One in particular stars a young Black student strolling through a g ­ ang-​­ridden Chicago neighborhood against the backdrop of two ­mother-​­like voices narrating the scene. DOI: 10.1201/9781003368755-8105

106

Hidden in White Sight

“­Isn’t that Calvin? I haven’t seen him for a while. Wonder where he’s heading?” one of the women asked. “­I heard he gotta job. Is that right?” “­Now that you mentioned it, there is something different about him. You can’t judge a book by its cover. Looks like responsibility has been good for him.” “­Well, I’m just glad somebody believed in him enough to give him a chance. Wonder where he’s working?” Then the scene shifts to the ­15-­​­­year-​­old Calvin saying, “­Welcome to McDonald’s. May I help you?” My mom says she likes the commercial because it reminded her of me. Not just because the leading character and I share first names, but because I, too, had my first job at McDonald’s at the age of 15. There were only three ways in my neighborhood by which you could keep a young Black youth out of the streets: church, sports, or a McDonald’s job. Over the years, McDonald’s was as synonymous with the inner city as the corner barbershop, nail salon, or liquor store. But, even though many things in the neighborhood have changed over the years since I was a k ­ id—​­the projects have lost their sense of community, the ­once-​­powerful voice of the church has quiesced, many of the once thriving small businesses have closed their doors, the public community school has s­ huttered—​­still, McDonald’s remains the neighborhood’s most enduring institutional legacy. So, when McDonald’s decided to target the Black community with their ad campaign in the 1990s, it was genuinely well accepted in the community. Moreover, unlike most ­white-​­owned businesses that had inundated and traditionally ravaged the community with their financial resources, McDonald’s and other f­ ast-​­food chains like Hardees, Burger King, and Pizza Hut actually poured back into the community with jobs for ­at-​­risk youth. McDonald’s franchising was not only thought to be the first job opportunity for Black youth but also an avenue for small business development opportunities that could economically empower Black communities. This form of “­Black capitalism” wasn’t just supported and embraced by community leaders but also by the government. Not that the government particularly took an interest in Black entrepreneurship, because they saw it as an opportunity to limit federal assistance in the community. McDonald’s found very willing customers in inner city and urban America during its early existence. In addition, McDonald’s knew

Mass Unemployment

107

that they could augment their workforce with cheaper ­inner-​­city workers. People loved the concept of ­community-​­based corporate America coupled with the Big Mac. Anything beginning with “­Big” seems to resonate in the inner city as it made us seem like we were getting more than we were actually paying for. In today’s ­fast-​­food landscape, it’s difficult to imagine a world without McDonald’s. But for communities that were getting McDonald’s for the first time, few people had seen anything like it. Food that was delivered before you could navigate the line. Places where there were comfortable areas for kids to run around and play. All while the parents kept a watchful eye. Back then, almost every McDonald’s had a child play space where kids could jump and tumble on the red and yellow plastic balls. Typically, prior to McDonald’s, most hamburger vendors made you stand outside the window to order. We are swimming in dangerous and ­shark-​­infested waters when we believe that digital humans can replace human ­decision-​­making AI should augment us, not replace us

So, for Blacks that were often shut out of the experience of dining and eating in restaurants and feeling safe, a McDonald’s that was owned by an ­A frican-​­American in your community where you could be w ­ elcomed—​­this was no small thing. Herman Petty was the first ­A frican-​­American McDonald’s franchise owner, and he took over a restaurant on Chicago’s South Side in 1968. No coincidence that Chicago and the neighboring cities are where most McDonald’s e­ thnic-​­based ad campaigns were launched. Just 3 years after Herman Petty’s inaugural breakthrough, 50 i­ nner-​­city McDonald’s were operated by ­A frican-​­A merican franchisees. As was the case with most striving Black businesses, the McDonald’s franchises were some of the most profitable restaurants in the entire company. Similar to my entry into the workforce 30 years prior, my ­15-­​­­year-​ ­old nephew, Demyis, applied for and got his first job at McDonald’s in 2021. So, the tradition continues of McDonald’s and similar ­fast-​ ­food restaurants providing the first job opportunity for my family. For years, companies have been working toward automating repetitive jobs through algorithms that can complete administrative tasks, robots that can streamline manufacturing and mimic order

108

Hidden in White Sight

takers, and drones that can deliver goods in no time. But while the futurists have long warned of these “ ­­job-​­stealing robots,” the recent coronavirus pandemic has heightened fears that automation utilizing AI will replace workers’ jobs. Moreover, because of social distancing measures, many ­organizations—​­from restaurants to ­retailers—​­have been forced to find new ways to operate with as few employees physically present as possible. These industry trends have caused automation using AI to rise. Studies show that this kind of automation is more quickly adopted during economic downturns. I recently was asked to join a panel discussion on technical innovation in the ­fast-​­food industry. One of my fellow panelists, an industry expert, shared a compelling trend: F ­ ast-​­food pioneers like McDonald’s and others have begun to leverage AI to do some of the jobs that were historically targeted for ­inner-​­city youth like taking ­drive-​­thru orders. In addition, facial and voice recognition systems, alongside digital AI advertisement algorithms, are technological advancements that have taken hold in the f­ast-​­food restaurant marketplace. To provide greater efficiencies and profit through headcount reduction and better customer service, ­fast-​­food chains like McDonald’s are turning to AI and machine learning. For example, ­fast-​­food industry giants like McDonald’s continue to move closer to the tech industries to solve staffing and efficiency issues. This year alone, McDonald’s has spent hundreds of millions of dollars acquiring technology companies specializing in AI and machine learning. And the ­fast-​­food chain has even been able to establish a new tech hub in the heart of Silicon Valley, known as the McD Tech Labs. Here they have a team of engineers and data scientists working on voice recognition software. At some specific ­drive-​­throughs, like one in Chicago, Illinois, McDonald’s has tested and implemented technology that can recognize license plate numbers, allowing the company to tailor a list of suggested purchases to a customer’s previous orders. In the case of this feature, the customer must give McDonald’s permission to leverage historical data. As previously stated in this book, this data, once signed over, could possibly be leveraged for unintended purposes. McDonald’s is following the retail industry, as companies like Amazon consistently

Mass Unemployment

109

provide you additional recommendations and “­similar products that you might like.” The AI algorithms are similar in that they leverage past purchases and other relevant data to predict or recommend products you might like. Again, this can be done relatively quickly and programmatically. For instance, the system might recommend a different meal based on whether you are ordering during the weekend or a weekday. At the time of writing, I don’t know of any franchises that have incorporated the ability to allow clients to opt out and still use the s­ ervice—​­other than walking into the store to place your order. In addition to revamping the ­drive-​­through experience, McDonald’s has incorporated digital boards programmed to market food more strategically, considering the time of day, the weather, the popularity of specific menu items, and wait time. For example, the digital board might promote soda rather than coffee on a steamy afternoon. After every transaction, screens now display a list of recommendations, nudging customers to order more. This upsell strategy mimics a similar response that you might get from a human o ­ rder-​­taker, “­Would you like an apple pie with your meal?” With this new system, an algorithm could suggest what you are most likely to order based on ­non-​­food based inputs like how you look or what type of car you drive. Although AI’s voice response and ­speech-­​­­to-​­text algorithm might routinely misinterpret your order, it certainly won’t forget it. As a result, some irritated consumers might consider this algorithmic feature a worthwhile advancement. “­You just grow to expect that in other parts of your life. So why should it be different when you’re ordering at McDonald’s?” said Daniel Henry, Chief Information Officer. “­We [at McDonald’s] don’t think food should be any different than what you buy on Amazon.” Daniel Henry’s position is not unlike the reality of the current state of internet shopping. Leveraging AI’s recommendation algorithms and personalization, restaurants, clothing stores, supermarkets, and other businesses use AI to collect consumer data and then deploy that information to encourage more spending. But unfortunately, more consumer adoption and spending don’t necessarily mean more jobs. In the case of McDonald’s, it might mean the ­opposite—​­especially for Black youth in the inner city.

110

Hidden in White Sight

Like other retailers that offer their products and services over the internet, McDonald’s collects gazillions of data from consumers during their shopping experience. Leveraging technologies like Bluetooth, companies can now track consumer movements throughout the store. For example, the app could keep track of store areas where you spend more time versus other locations. As a result, you might get that text message or email recommending a discount on a product you considered but chose not to purchase. The recommendation algorithms built into the ­drive-​­through menu boards have already generated larger orders, the McDonald’s Chief Executive, Steve Easterbrook, said during an earnings call in July. (­Mr. Henry, the chain’s information executive, declined to reveal the size of the increase.) By the end of the year, the new system is expected to be at nearly every McDonald’s d ­ rive-​­through in the United States. “­The labor shortage frankly has done more to push restaurants towards technology than almost anything else,” said Jonathan Maze, executive editor of trade publication Restaurant Business Magazine. “­It theoretically enables you to run your restaurant with fewer people.” It would be a stretch to theorize that robots will somehow displace Black workers in the future, but it’s not unreasonable to think that technology can at least lessen the dependency for manual jobs. Studies show that regardless of race, a quarter of US jobs will be severely disrupted as AI accelerates the automation of existing workloads, according to a report posted on MarketWatch. Moreover, an Oxford study says that 47% of workers are expected to lose their jobs because of AI automation. Many ­inner-​­city Blacks are employed in jobs that involve a single repetitive task, such as toll operators collecting money and giving change. Many of those workers have already been replaced by automated electronic toll systems for vehicle identification. Others employed in manufacturing jobs such as order fillers, sorters, and packers are being replaced by various types of robots capable of selecting specific products, stacking items including pallets, sorting packages, and organizing warehouse inventory. Finally, workers employed in repetitive data entry positions are being replaced by automated software that can process orders and perform payroll functions, among other tasks. Studies show that within 15 years, AI will be capable of doing 14% of all jobs. History tells us that when overall job loss occurs,

Mass Unemployment

111

communities of color suffer the brunt of the loss. According to a recent McKinsey study, Blacks have a greater than 10% likelihood of losing their jobs to AI versus n ­ on-​­Black workers. By 2030, without intervention, over 4.5 million Black jobs could be displaced, according to that same McKinsey study. Not just ­fast-​ ­food jobs but similar manual roles like assembly line workers, radiologists, paralegals, inspectors, truck/­Uber/­t axi drivers, mall workers, restaurant workers, and even automobile salesman jobs are all at risk. Although the above McDonald’s example depicts an example where Blacks could undoubtedly be at risk of losing their jobs to AI automation, the following example has the potential of being even more ­life-​­threatening and catastrophic, with the same devastating economic impact. Like the ­fast-​­food and restaurant industries, trucking has traditionally been a significant “­pick yourself up by your bootstraps” opportunity for Blacks. However, the trucking industry was then, as it is now, primarily ­white-​­dominated, and the Black families that contained a certified truck driver were rare. The number of Black truck drivers diminished significantly over the last 25 years. I don’t know why in particular, but I suspect it has a lot to do with the digital transformation of the industry.

The Rise of the Electronic Vehicle (­EV) The automotive industry is experiencing a convergence of disruptions unlike any seen since the early 1900s. Autonomous, connectivity, electrification, mobility, and subscription business models are reshaping the automotive industry and creating a frenzy of activity. The chief technological catalyst for this innovative landslide is Artificial Intelligence. One unknown source stated at a conference that I recently attended suggested that “­either you already have an electronic vehicle,” “­You want an electronic vehicle,” or “­be patient, you’ll soon have an electronic vehicle.” Recent studies show that the adoption rate of electronic vehicles (­EVs) continues to rise. However, despite the positive outlook, EVs face significant hardships which hinder its rapid adoption, especially within communities of color. Battery pack cost continues to be a

112

Hidden in White Sight

major contributor to the high cost of EVs. AI algorithms can provide a realistic ­driving-​­range estimation and optimize energy conservation, which can add extra driving range thus increasing battery efficiencies. Other innovative techniques leveraging AI to help identify nearby charging stations and to accurately predict distance and energy consumption will further reduce the price of the vehicle, thus making the EV technology more accessible to communities of color. AI promises to lessen the effects of both cost and range ­anxiety—​ ­the fear that an EV won’t have sufficient charge to get you where you want to go and back. Cost and range anxiety are two key contributors to why the adoption rate is lower in communities of color. But the top reason for the lower adoption rate is hidden within the answer to the question “­W here are the Charging Stations in the Black community.” Without proper access to charging stations, communities of color continue to be left behind in the EV wave. By taking a quick look at any map of charging stations in the United States, you’ll just quickly conclude that communities of color again find itself on the unfortunate side of the digital and technological divide. “­If residents of the city cannot participate equitably in the EV market, that would be a failure,” said Stefan Schaffer, a strategist for the American Cities Climate Challenge at the Natural Resources Defense Council. Although I’ll give some specific examples of ­self-​­driving trucks, I don’t believe that we are anywhere near to having fully automated ­self-​­driving vehicles. The current standards developed by the Society of Automotive Engineers define automation with five ­levels—​­from level 0, no automation, to level 5, full automation. The amount of human interaction with the vehicle decreases as you go from level to level. For level 0, the human controls all the driving functions, while in level 5, the algorithm controls 100% of the driving functions, including acceleration and steering. So, we’ll still see humans serving in at least a ­co-​­driving role in all vehicles in the ­near-​­term future. This isn’t something that will concern most. You wouldn’t actually know if a computer was the actual flyer of the plane as long as you saw a captain sitting in that pilot seat on your next flight. This is considered a high automation environment that assumes the system controls driving and monitoring in some but not all operating conditions. This level of cooperation between the pilot and the AI system is considered level 3 and level 4 conditional automation. It is reasonable to think that this same level of automation will exist in the autonomous truck space.

Mass Unemployment

113

The financial value proposition for driverless trucks is just too overwhelming to be overlooked. Autonomous and electronic vehicles provide a smarter, safer, and cleaner alternative to today’s rumbling old ­gasoline-​­engineered dinosaurs. Innovative autonomous features have gotten the attention of even the casual car driver. However, while increased efficiency, lower cost, and better performance are nice perks for the standard everyday drivers who are early adopters of EVs, they are a matter of life or death for large ­corporations—​­being profitable or losing money. For instance, Walmart is partnering with Tesla to revolutionize its trucking and logistical platform. For a company like Walmart, the Tesla electronic truck will pay for itself by providing almost $200,000 in fuel savings in less than 2 years and costs just 1.26 cents per mile to operate compared to 1.51 with a comparable diesel truck. The Tesla autonomous truck will require less maintenance because there are fewer moving parts. The vehicle will come fully equipped with advanced hardware and software capable of providing enhanced autopilot features today and complete s­elf-​­driving capabilities in the future with ­semi-​­autonomous capabilities. Tesla autonomous trucks will have the ability to communicate with each other automatically. This capability is referred to as platooning. These are just a few of the money and ­cost-​­saving opportunities that a conglomerate like Walmart would prefer to the enormous ­human-​ ­ ependent hogs that are on the road today. d Think about it. Tesla and other manufacturers state that these new innovative ­A I-​ e­ nabled machines will provide fewer crashes by removing human error from the equation. No drivers having to call in sick, no drinking or smoking incidents in the cabin, no ­half-​­sleep ­operators—​­the truck can be on the road every day. No salaries, medical, and health insurance expenses. An autonomous car is a vehicle that is capable of sensing its environment and moving with little or no human involvement. For the vehicle to move safely and understand its driving environment, an enormous amount of data needs to be captured by a plethora of different sensors deployed throughout the car. This data is then processed by the vehicle’s autonomous driving computer system. The AI algorithms that implement these autonomous features must also undertake considerable training to understand the data it collects and make the right decision in any imaginable traffic situation. The issue comes when moral decisions must be made by the vehicle.

114

Hidden in White Sight

People make moral choices daily, while algorithms struggle with this requirement. For example, when a driver chooses to slam on the brakes to avoid hitting a jaywalker, they make the moral decision to shift the risk from the pedestrian to the people in the car. What choice would an algorithm make when presented with this ethical dilemma? Let’s consider a Tesla autonomous vehicle with a defective braking system. The car is going at full speed toward a grandmother and a child crossing a pedestrian walkway. By deviating a little, one can be saved. In our scenario, it is not a human driver who is going to make the ­life-​­saving decision but the car’s algorithm. Who would you choose, the grandmother or the child? Is there a right answer? This is a typical ethical dilemma that highlights the importance of ethics in the development of AI. Still wondering how AI will impact you? As I have carefully explained throughout this book, these AI models and algorithms don’t come without their fair share of risk or biases. While auto manufacturers and researchers might agree that these new ­state-­​­­of-­​­­the-​­art vehicles will be safer and have few accidents, they don’t go as far as saying the cars will be ­crash-​­free. Let’s consider the following case: Recently, the new Tesla Model 3 automobile, with the same capability as the tested truck, crashed into a ­semi-​­truck while being driven in autopilot mode. The s­ emi-​­truck had flipped over prior and was lying

Mass Unemployment

115

on its side when the Tesla approached. This crash happened in broad daylight and was filmed by another driver in a separate car. So why didn’t the autopilot see the truck and stop? Many things could have caused this, but I have a theory as a data scientist. It has to do with how the autopilot system learns and analyzes the data in its training set. We discussed this training and learning process earlier in this book. The autopilot features are constantly running in the background, making predictions, assertions, and recommendations about what they should do when encountering a particular situation. As the car is driven, the algorithm gets feedback on whether it’s right or wrong about its decisions. After repeating this process millions of times over, the neural network learns how to prescribe the best action to handle each distinct situation. Remember, I explained earlier that a neural network is a system of algorithms that mimic the human brain’s same pathways to learning, adapting, and carrying out tasks. I’m confident that there were millions of pictures of trucks of all makes, models, colors, and sizes in the training set, but I would wager that there weren’t a lot of data in the training set that had pictures of an overturned truck or a truck laying on its side. Algorithms are “­trained” on massive data sets of photographs and video footage from various sources and use that training to determine appropriate behavior. But, if the footage doesn’t include many examples of specific behaviors, like how to slow down near emergency vehicles, the AI will not learn the appropriate behaviors. Thus, they’ll tend to crash into already crashed and overturned vehicles. Another theory is that the training data could have easily believed that the large, overturned truck was a bridge (­since it typically drives under a bridge without incident) or maybe even a highway traffic sign… But to a company like Walmart, the truck doesn’t have to be foolproof; it just needs to be as safe, if not safer than the current diesel truck. The profit and return on investment will always outweigh the risk of a small number of crashes. We’ll discuss the profit and return on investment (­ROI) element of AI later in this book. In addition to the safety concerns mentioned earlier, we also must ­ bject-​­detection algorithms be concerned about how autonomous car o deal with communities of color. Suppose you are a person with a darker complexion. According to a new study by the Georgia Institute of Technology, you may be more likely to get struck down by an autonomous vehicle than your white counterparts.

116

Hidden in White Sight

Autonomous vehicles seem better for detecting pedestrians with whiter skin tones versus darker tones. The study found that the object detection algorithm was 5% less accurate in detecting skin tones of people of color. I know a questioning reader might suggest that it’s only 5% less a­ ccurate—​­that would matter less if it was your kid who was hit by the car. I’m not speaking on the validity of the study. Still, those in the data science industry would agree that these disparities could quickly happen if the training data is incomplete. Even if you are hellbent on not going anywhere near an electric vehicle, you have already been introduced to similar technology data-gathering and automation capabilities in your gas guzzler. Most insurance companies have invested heavily in understanding how well you drive using tracking devices. You probably say you would never allow your insurance company to track you. Would you do it for a 30% savings off your premium? What about 40%? These devices, placed in your car or accessed via a smartphone app, allow insurers to monitor your driving habits and determine rates that more accurately represent your risk. An insurance tracker is a telematics program that collects data about your driving habits, leveraging AI technology in most cases. Typical data collected includes: • How often do you drive, and for how long • Hard braking • Hard acceleration • Speed • Fast cornering (quick, sharp turns) • Time of day, especially nighttime driving • Phone usage while driving After the insurance company collects the data, they use mapping data to see what kinds of traffic controls, such as stop signs and speed limits, are in place where you were driving. In doing so, they can determine if you’re a safe driver who follows the rules of the road, if you frequently speed, run stop signs, or otherwise have poor driving habits. One might ask if the insurance company can determine how well you change lanes at night, then maybe “Just Maybe” they can also collect more invasive data on you or use the existing data to make socially unacceptable risk classifications like your “race” or “national origin” or whether you frequent a minority neighborhood. We hope our insurance company only uses this data to calculate our driving risk.

Mass Unemployment

117

Hopefully, we are right—but history might argue the point. What if it’s not your insurance company that you should be worried about when it comes to unauthorized data use: What if it’s the legal community? Data from car tracking devices can also be used in civil litigation proceedings. So, suppose you’re trying to pursue a claim against an at-fault driver and have an insurance tracking device installed in your car. In that case, the opposing defense counsel could subpoena this information and comb through it, looking for any data that could pin fault on you. Depending on what is found in your data, this could dramatically decrease your liability claim due to comparative negligence.

The Rise of the Electronic Cashier Now let’s look at another job s­ector—​­checkout operators. A couple of my Black female friends were shopping for groceries at a local retailer years ago. Food shopping had become a relaxing substitute for the hustle and bustle mall ­experience—​­a girls’-­​­­day-​­out experience. So, they usually spent more time talking and laughing than they did grocery shopping. They had a full cart and made it past the cash register without incident. But as they were leaving the store, an employee stopped them to check their ­bags—​­the store security shared later that they were stopped because of their suspicious shopping behavior. Racial profiling affects all aspects of American life, even as “­normal” an activity as grocery shopping. Black people, in particular, suffer from racial profiling in retail. A 2018 poll showed that nearly ­two-​­thirds of Black Americans believe they are treated less fairly than white people while shopping. Unfair treatment ranges from being ignored when they need help, receiving distrustful looks, being questioned, or even having the police called on them for suspicion of theft. This seems like an excellent opportunity to leverage AI to equal the playing field. Indeed, an automated checkout kiosk can’t discriminate, correct? It can’t give dirty looks and does not act on stereotypes. Right? So, if a ­self-​­checkout kiosk reminds customers to scan all their items, Black consumers shouldn’t feel racially profiled. Right? If you’ve been reading this book thus far, you will undoubtedly disagree with the questions above. But, of course, machines can do all the above, even the “­dirty look” part.

118

Hidden in White Sight

AI and algorithms can internalize bias and stereotypes because the training data may include discriminatory language, ­photo-​­sensitive images, and human prejudice. Even certain purchasable products can be associated with and used to derive one’s race. For instance, Black hair care products can determine whether consumers are Black or white, or their gender. The algorithms can scan for g ­ ender-​­based products like lipstick, facial lotions, and hair products and determine both gender and race and ultimately make additional decisions based on the results. In August 2018, Amazon introduced its Amazon Go ­cashier-​­less vinnovative campaigns to alter the ­brick-­​­­and-​­mortar retail experience. To shop at Amazon Go, customers must scan in with their Amazon Go app simply to enter the store; once inside, hundreds of cameras and sensors identify products that customers take off the shelves and put into bags. Their Amazon accounts are then docked accordingly based on the merchandise they exit the store with (­a process Amazon has dubbed “­Just Walk Out” technology). No checkout, no cashiers, no waiting in line. Amazon is not the only company pursuing automated retail; recently, wholesale retailers like Sam’s Club have implemented similar technologies. The cashier profession represents the third largest occupation in the United States, according to data from 2019. Although a large percentage of cashiers are white (­60% compared to 17% for Blacks), any loss of jobs via automation will significantly impact the existing low number. As a result, the employment of cashiers is projected to decline 10% from 2020 to 2030. This is yet another example of how AI can be a double whammy and thorn in the side of the Black c­ ommunity—​­racist tech and job loss, two equally catastrophic outcomes. I’ve shared three ­real-​­life examples of how AI, if unregulated and unchecked, can wreak havoc on the job market for Black Americans. First, I want to reiterate that technological advances will almost always create more jobs than they have displaced when implemented with the proper guardrails. If Black people are going to avoid the double whammy effect of job loss and racist tech, then these guardrails must become standard practice in the s­ervice-​ ­based industry. Businesses must offer e­ mployer-​­led training alongside tuition assistance to workers who are impacted by automation. Policymakers should incentivize companies to increase their training efforts to ensure that workers who have lost jobs due to AI automation can be properly repositioned back into the workforce.

Chapter Eight

Medically Induced Trauma

Troubled Waters Of all the forms of inequality, injustice in healthcare is the most shocking and inhumane. —​­Martin Luther King Jr

In 2019, Frederick Taylor suffered a couple of healthcare scares

that shook the foundation of his faith. He had always been athletic

DOI: 10.1201/9781003368755-9119

120

Hidden in White Sight

and was a constant face in his local gym for the last 20 years. His workouts included at least an hour of cardio and weightlifting. Although medical visits were rare, Frederick was diligent in scheduling and attending his annual physicals. Other than slightly elevated creatine and cholesterol levels, he and his primary care physician considered him relatively healthy. Likewise, Frederick had no family history of heart disease and diabetes, but over the years, his clinicians reminded him that Black Americans were more susceptible to getting those two diseases than other races. So, although Frederick was diligent in his workouts, paid attention to his diet, and didn’t have any family history, he was aware of the genetic risks. He often wondered why he needed insurance at all since his only trip to the doctor was to have an annual ­physical—​­a ­co-​­pay of $30. During a short trip to Florida, he started having heart palpitations and elevated ­blood-​­sugar levels. After a trip to the emergency room, his vital signs and blood levels were n ­ ormal—​­the testing didn’t raise any immediate red flags. Frederick was also given another examination as a precautionary step recommended by his insurance provider. This final test would determine whether he would be selected to receive further treatments, based on risk factors for more serious illnesses like heart disease or diabetes. The doctor returned and said, “­A ll cleared. The algorithm returned negative. You are okay. You are as healthy as an ox.” After picking up his prescriptions, He left the hospital and joined his wife on the beach. About six months later, He returned to the hospital with what the clinicians diagnosed as a myocardial infarction. Ordinary people refer to this diagnosis as a “­heart attack.” He was fortunate that his wife could get him to the hospital in time. After receiving a medical stent, he escaped with his life. Every 36 seconds, a person dies in the United States from cardiovascular disease. Around 24% of those that die are Black. Frederick and his wife reminisced about their medical experience during a conversation over coffee. Frederick wondered aloud, “­W hat did I do wrong? I’ve always been conscious of my weight. I rarely drink, don’t smoke, and exercise more than most.” His wife responded, “­W hat was that algorithm that the doctor mentioned? It said that you were fine.” Frederick gasped and responded, “­Yeah, both the doctor and the algorithm cleared me to go home.”

Medically Induced Trauma

121

­Medical-​­induced trauma is a set of psychological and physiological responses of humans to pain, injury, serious illness, medical procedures, and invasive or frightening treatment experiences. This definition seems to encapsulate the American Black experience along healthcare lines. The biased outcomes of the intersection between medicine, healthcare, and race have roots in American society that have been a part of our ­250-­​­­year-​­old history. It is a continuum that uninterruptedly affects A ­ frican-​­American health, the way we receive healthcare, and the cost associated with the quality of that healthcare. Racism is, at least in part, responsible for the fact that Black Americans, since arriving as enslaved people, have had the worst healthcare, the worst health outcomes, and the worst health status of any racial or ethnic group in the United States. Indeed, to some degree, unhealthy habits, food ignorance, and lifestyle choices contribute to the ­self-​­inflicted outcomes that we experience as Blacks. But like other elements of culture, race and technology prove to be the most significant influencers on the proliferation of inequitable treatment in the healthcare field. Like other Americanized norms, AI reflects the real world, which means algorithms can unintentionally perpetuate existing i­nequality—​­even relating to something as fundamental as our healthcare system. For example, recent studies have found that a healthcare ­risk-​­prediction algorithm used on more than 200 million people in the United States demonstrated racial b ­ ias—​­because it relied on an unreliable metric for determining care management. Optum created this algorithm. Optum is a technology provider and subsidiary of UnitedHealth Group, a leading healthcare company based in the United States. This then begs the question, should Frederick have had more tests done on him? If yes, should all Black Americans test for heart diseases even when diagnosed with something minor? Or should we trust the results of a poorly written algorithm? The truth of the matter is that if proper tests had been run on him, the heart attack could have been prevented. Unfortunately for Frederick and his family, he was possibly one of those 200 million algorithmic casualties. This algorithm helps care providers and insurance payers identify which patients will benefit from “­­high-​­risk care management” programs. These programs provide chronically ill people with access to

122

Hidden in White Sight

specially trained nursing staff and allocate extra primary care visits for closer monitoring. In addition, the programs give patients access to dedicated phone lines, home visits, prompt doctor appointments, and reconciled prescriptions. For instance, if a patient has heart disease and diabetes, the program will help them avoid hospitalization by allowing for more frequent doctor visits. The algorithm’s developers and data scientists used previous patients’ healthcare spending as the primary predictor variable to compute who should qualify for this extra care. This variable served as the proxy for determining medical needs. Ziad Obermeyer, working as an assistant professor of health policy and management at the University of California, Berkeley, states, “­Cost is a very efficient way to summarize how many healthcare needs someone has. It’s available in many data sets, and you don’t need to clean the data.” In this case, selecting healthcare spending as the proxy was a flawed assumption. This is because even when white and Black patients spent almost the same amount of money, they didn’t have the same level of need. One main reason was that Black patients tended to pay for more active interventions such as emergency visits for diabetes or hypertension complications. White Americans spend more on healthcare than Black Americans, even when their health situations are identical. There are many reasons for the findings, one being that whites overall have more disposable income, ­higher-​ ­paying jobs, healthier food options, and more and better insurance. The insurance premiums are also lower, which makes it easier to afford and thus attain. They also have greater access to medical facilities; therefore, they visit the doctor more frequently and spend more on medications, preventive care, and elective surgeries. Since healthcare spending among whites exceeds Blacks’, the algorithm mistakenly concluded that white paIt’s not our differences that tients were sicker than similar Black divides us; it’s our inability to acknowledge, admit, and patients. It automatically tagged paappreciate those differences. tients in the top 3% of medical spend­ igh-​­risk care management ing for the h program, which is intended for patients with complex medical needs. But the algorithm couldn’t distinguish between a ­65-­​­­year-​­old white American’s expenditure of thousands of dollars on a knee replacement to improve his golf game and a

Medically Induced Trauma

123

­ 5-­​­­year-​­old Black American’s spending of the same amount to keep 6 diabetes from killing him. Like the algorithmic bias examples mentioned prior, the result of this error continues to be catastrophic for the Black race. White patients are more likely to receive preferential access to the care management program even when Black Americans spend the same amount, have more chronic illnesses, and significantly poorer health. A reasonable human would acknowledge, given the above facts, Blacks should be getting the extra medical a­ ttention—​­but the algorithm says differently. Only 17.7% of patients assigned to the additional treatment by the algorithm were Black. The researchers calculate that the proportion would be 46.5% if the algorithm were unbiased. This is not the first and only widely used health algorithm to trigger concerns that technology is not delivering on promised healthcare claims. For example, several AI algorithms developed by Epic Systems, the nation’s largest electronic health record vendor, provide inaccurate or irrelevant information to hospitals about the care of seriously ill patients. 45% of the US population have their medical records in an Epic system. That equates to the ­Wisconsin-​­based software giant holding the medical records of some 250 million people. Not only does Epic create and market the algorithm, but recent investigations also show that it has paid some health systems up to $1 million in cash to use their predictive algorithms. Some would say this is capitalism at its best. Unfortunately, it doesn’t take an ­anti-​­trust suit to let us know that this is a possible conflict between the ethical and professional duties to deliver the best medical care to patients versus preserving their bottom line. Some of those same studies show that Epic’s algorithm for predicting sepsis, a l­ife-​­threatening complication of infection, consistently fails to identify the condition in advance and triggers frequent false alarms. Communities of color experience a disproportionate burden of this ­sepsis-​­related suffering, particularly Black communities. Unfortunately, research has shown that despite standardization of care, wide variability in mortality rates remains, with minority populations having increased incidence, hospitalizations, and complications

124

Hidden in White Sight

compared to white folks. If sepsis is predicted early, the patient can be treated with antibiotics, which in most cases will save their lives. Over 100 healthcare systems use this early warning algorithm developed by Epic that watches for signs of sepsis in the patient’s test results. In addition, Epic has over twenty predictive algorithms, of which several have been accused of promoting some level of racial bias. These include algorithms that assist hospitals in predicting patients’ length of stay and another that predicts who may fail to show up for medical appointments. Like our earlier example, errant alarms may lead to unnecessary care or divert clinicians from treating sicker patients in an emergency department or intensive care unit where time and attention are finite resources. As with most b ­ lack-​­boxed algorithms like Epic, the algorithm’s inner workings are not available to the public. However, it wouldn’t be f­ ar-​­reaching to conclude that most of the bias is probably occurring because of incomplete and unrepresented data on Black Americans. This data bias severely reduces the accuracy of the models, which accounts for the low performance and higher false alarm rate. Because of a lack of regulatory guidance and little to no governmental oversight, companies like Epic are selling algorithms to the medical community at an alarming rate. I’m amazed at the lack of attention by regulatory commissions like the Food and Drug Administration (­FDA), whose primary role is to protect public health by assuring human and veterinary drugs’ safety, efficacy, and security. The FDA itself has a littered and controversial history where people of color account for less than 12% of clinical trials participants. This is important because much data used to train healthcare algorithms is taken from clinical trials. This is just another example of AI algorithms trained with unrepresentative data; the outcomes will lead to biases almost all the time. There’s a programming truism that has existed for more than a century, “­Garbage in, garbage out.” That means if you put garbage into something, then you will get garbage out. So bad data in will ensure that bad data is returned. The AI algorithm equated healthcare spending with health. It had a disturbing result: It routinely let healthier white patients into the programs ahead of Black patients who were sicker and needed the services more.

Medically Induced Trauma

125

Although incomplete and lack of diversity in the data are certainly at play in this example, it continues to be a longstanding and pernicious issue overall. Once again, one should not overlook implicit racial bias raising its ugly head. It, more than anything, is another factor that contributes to the proliferation of healthcare disparity. In our community, Black patients are accustomed to either experiencing or expecting to receive ­lower-​­quality healthcare. Because Black patients often go through this kind of bias, they have less trust in the doctors they feel are exhibiting this bias. However, a study published in 2018 by the nonprofit National Bureau of Economic Research showed that Black patients have better health results when they have Black doctors treating them. This is mainly because of higher levels of trust between the doctors and patients. In my personal experience, I’ve found this to be true. Without a trusting d ­ octor–​­patient relationship, we are less likely to go to the doctor, request extra care, and often pay for it with our lives. That’s unfortunate. As with some of our earlier examples, we use algorithms without human intervention to guide the healthcare given to millions of Americans, unintentionally replicating their creators’ racial blind spots and even biases. To gain a more enlightened and i­ n-​­depth perspective on how this and similar AI algorithms came to be, let’s retrogress how prior algorithms (­not necessarily AI) have underpinned the global healthcare system. Although most of us consider clinicians as beneficent caregivers, it is increasingly evident that medicine is not a standalone institution that is immune to racial inequities but, in some cases, is an institution of structural racism in itself. An example is r­ ace-​­based medicine, the system by which research characterizing race as an essential, biological variable translates into clinical practice, leading to inequitable care. It’s called “­racial essentialism”—​­a claim that races are biologically distinct groups determined by genes. It has absolutely no scientific basis, yet it lingers in US health policies, clinical practices, and many other racist ideologies. Along with racism in medicine, the belief has perpetuated generations of harm. Although diseases like heart disease, sickle cell, and diabetes disproportionately affect Black Americans, there is no disease that is

126

Hidden in White Sight

exclusive to Black Americans. If a researcher uses data based on race or infection history to predict the next occurrence of diabetes, the algorithm might be biased against white patients. It certainly doesn’t mean that the white patient won’t get diabetes. It just means that the algorithm’s predictor model might suggest that he won’t. What we eat, where we live, where we work, and the quality and access to healthcare have more to do with the diseases that affect us than the color of our skin. Recently, a friend of mine who does a lot of ­pro-​­bono bike repair work gave me an interesting perspective when discussing the topic of professional prejudgment. In his day job, he too is a data scientist so I figured that his thoughts on the topic would matter. He mentioned that he can easily tell that if a bicycle doesn’t shift well; most likely, the issue is a bent derailleur hanger. He readily admitted that his conclusion was based on years and years of repairing bikes, coupled with being an avid reader of bike repair magazines. However, when presented with a specific bicycle, he stated that he must still do all the diagnostics. My friend’s bicycle example might seem overly simplistic, but it does represent the larger problem of how bias in algorithms occurs. Let me explain. If my friend was asked to create an AI application to recommend solutions for resolving bike repair issues for a particular bike model, his newly created algorithm would most likely recommend that the solution to a shifting issue is to replace the derailleur hanger. Similar to his hesitancy in making an early human conclusion based on poor data, the algorithm also has to be designed to avoid making recommendations or predictions based on incomplete assumptions. Yes, like ­humans—​­who when analyzing data derive assumptions and conclusions, some accurate and some not so accurate. As data scientists, we consider race as a protected a­ ttribute—​­that means it shouldn’t be used as a predictor variable to determine the outcome. However, we’ve discussed in earlier chapters how race consciousness and societal preoccupation with skin color continue to be a significant contributor to AI bias. We often rely on the conventional wisdom that Black people are genetically different from white people. While writing this book, I had an interesting conversation (­or heated debate) with a medical specialist who was treating me for some back pain. The thought was that it could be neurological in

Medically Induced Trauma

127

nature, and therefore, I was keen on getting this tested. My testing results showed that I had elevated levels of creatine (­CK). According to the specialist, this could be the cause of my muscle pain. During his synopsis, he proceeded to tell me that Black men who weightlift and have sufficient muscle content are prone to having elevated levels of creatine. My numbers were typical for Black men in general, in his medical opinion. I quickly shared that I wasn’t interested in any ­race-​­based diagnosis and that he needed to diagnose me as if I was a white male who came to him with similar test results and symptoms. I later spoke with one of the senior tenured medical professionals at the same healthcare provider, who shared my position regarding ­race-​­based medicine. In his words, One of the reasons that we know r­ ace-​­based medicine is flawed is people who share a similar genetic code to Black Americans, living in African and Caribbean nations, do not have the same proclivity to some of the diseases that we fall victim to in America.

It is factual that Black Americans are prone to get specific diseases faster than other ethnic groups. However, as stated earlier, the reasoning supporting the conclusion is less about genetic code and more about what we eat, where we live, our societal stress levels, and how much we exercise. This unfortunate belief descended from European colonialization when the race was developed as a tool to divide and control populations worldwide. Unfortunately, it has crept into our AI algorithms like all other institutions, including healthcare. I’ll use three current examples of how race correction is used in clinical medicine. I’m confident that all will relate:

Heart Failure Risk Calculator Brenda is a ­45-­​­­year-​­old Black woman living in Minneapolis, Minnesota. She’s having ­heart-​­related symptoms, and she visits the local emergency room to get checked out. According to the American Heart Association guidelines, the clinicians are advised to use the risk stratification guidelines to guide her medical treatment.

128

Hidden in White Sight

The algorithm assigns Brenda a “­score” based on specific variables that apply to her. The input variables to the algorithm are: • Blood pressure • Blood urea nitrogen • Sodium • Age • Heart rate • History of ­COPD—​­Chronic Obstructive Pulmonary Disease • Race: Black or ­non-​­Black Brenda has known others who experienced ­heart-​­related illness, so the first six attributes seem normal to her, but she’s surprised to see that race is a variable that impacts her score. So let’s look at how the American Heart Association uses race as a critical variable in the algorithm. The algorithm adds three points to the risk score if the patient is identified as ­non-​­Black. This addition in the total score increases the estimated probability of death (­higher scores predict higher mortality). Why is this important? Increasing the score by three points “­ increases the use of recommended medical therapy in ­high-​­risk patients and reduces resource utilization in those at low risk.” The race correction mechanism regards Black patients as lower risk and thus makes it unlikely for Brenda and others like her to gain access to clinical resources.

National Cancer Institute Breast Cancer Risk Assessment Tool Lisa is a ­27-­​­­year-​­old Black woman living in Tallahassee, Florida. She has no family history of breast cancer but recently noticed something that felt like a small lump on one of her breasts. While visiting her doctor, he gave her a breast cancer risk assessment after the physical examination. The assessment algorithm estimates the 5 ­ -​­year and lifetime risk of developing breast cancer for women without prior history of breast cancer.

Medically Induced Trauma

129

The inputs into the algorithm were: • Age • Prior biopsies • Family history • And you guessed it, “­race” The algorithm returns lower risk scores for Black women than for other races. Although the model is intended to help conceptualize risk and guide screening decisions, it may inappropriately discourage more aggressive screening among some groups of ­non-​­white women. In Lisa’s case, she would have gotten better future healthcare if she was a 2 ­ 7-­​­­year-​­old white woman.

Urinary Tract Infection (­U TI) Score Sandra and Derek are parents of ­nine-­​­­month-​­old Derek Jr, whom they affectionately call D.J. Sandra is concerned that D.J. might have a urinary tract infection (­UTI). D.J. has a slightly high temperature. After a visit to the emergency room, D.J. was administered a UTI assessment. The urinary tract infection assessment estimates UTI risk in children aged ­2–​­23 months to guide decisions about whether to conduct further urine testing. This additional testing is needed to confirm the definitive diagnosis. One of the inputs into the UTI algorithm is race. The algorithm assigns a lower likelihood of UTI if the child is Black. However, based on clinical studies, the child has roughly 2.5 times increased risk if Black versus other races. Therefore, by systematically reporting lower risk for Black chil­ on-​­Black children, this calculator may deter other dren than for all n clinicians from pursuing definitive diagnostic testing for Black children presenting with symptoms of UTI. One could possibly conclude that since a Black child is “­less likely” to suffer from UTI, the algorithm might not accurately predict their risk factor and thus not show their real danger level. Even if they suffer from the infection, they are more likely to be shown as negative when compared with other races.

130

Hidden in White Sight

Over 50 clinically accepted algorithms are used daily to determine who, how, and what medical treatment is provided to people of color. I’ve only highlighted a few. All fifty operate and are practiced daily in plain sight. The methods used to calculate the scores are widely known and endorsed by the clinical community. As mentioned earlier in this book, the black box nature of AI algorithms makes it nearly impossible for the untrained eye to know exactly what is happening internally, but the results can easily be analyzed. The data collected over the last 25 years from the algorithms described above are now being used to feed new ­A I-​­driven healthcare applications that’ve been used by millions of patients across the world daily. The need for transparency, interpretability, and fairness monitoring are critical steps in order to ensure that the algorithms are working as designed. We will continue to expound upon these capabilities in the next few chapters. So, the unrepresentative and, in some cases, invalid data from the above use cases are now being used to train healthcare models to be used globally. So, the r­ ace-​­conscious algorithms used in the standard treatment are being used to perpetuate healthcare disparities. Race is used repeatedly and universally as a shortcut in clinical medicine analyses and decisions. For example, you’ve heard the ridiculous argument that because Black people can jump higher and run faster, we are genetically gifted. In 1988, the most renowned white sports anchor of his day, Jimmy “­T he Greek” Snyder, told the world on national TV that Blacks were “­bred” to be better athletes. The Black is a better athlete to begin with, because he’s been bred to be that way. Because of his high thighs and big thighs that go up into his back. And they can jump higher and run faster because of their bigger thighs. I’m telling you that the Black is the better athlete, and he practices to be the better athlete, and he’s bred to be the better athlete because this goes all the way to the Civil War when, during the slave trading, the owner, the slave owner, would breed his big woman so that he would have a big Black kid, see. That’s where it all started.

There was a national outcry against Snyder after the comments were made public. Still, few seemed to be alarmed when the nation’s

Medically Induced Trauma

131

leading hospital and insurer research departments make the same claim daily: Black patients are presumed to have greater muscle mass than patients of other races, and estimates of their renal function are accordingly adjusted. Black patients with higher than normal creatine levels and BMI indexes are routinely explained as having higher muscle content based on genetics.

Doctors routinely excuse symptoms by repeating the dreaded “­genetics” word. Unfortunately, in most cases, genetics doesn’t mean that you are predisposed because of your father or mother; it means you are predisposed because of your race. The resulting consequences of these attitudes quite often find themselves in the data fed to past, current, and future AI algorithms. It is not a repetitive statement to reiterate that an AI system is nothing without the data used to train the model (­or tell it what to do). AI can generate additional data through the patterns and trends it analyzes in existing data, but its initial dataset comes from historical artifacts. So in healthcare, the data comes from patient records, clinical studies or trials, and other system healthcare experts. There does seem to be some hope when it comes to leveraging AI to unravel some of the ­age-​­long mysteries of sepsis and possibly save ­lives—​­many of them Black lives. At the time of my writing, Emory University researchers are investigating the use of AI to forecast the therapeutic effectiveness and outcomes for patients with sepsis. They were recently awarded a $2.6 million grant from the National Institute of Health to mine s­ensor-​­generated data streams for ­physio-​ ­markers that may be able to predict the onset of sepsis. Remember, as we stated earlier, early prediction of sepsis potentially saves lives, and since more Black patients die of sepsis, many of those saved lives will be Black. Because of better analytic tools, deep learning, and more expansive data sets, Emory has improved the accuracy of sepsis prediction models. It took the university study data science team about a year to collect and cleanse the data and get it in the correct format. It begs the question, how much time was spent in the data collection phase of the Epic example. Black Americans weren’t adequately represented in their training ­data—​­this is only a suspicion as there is no way for me to know for sure. Emory has made their studies

132

Hidden in White Sight

public and reported that they leveraged 450,000 patients and complex ­deep ​­learning algorithms to identify over 65 different indicators. Leveraging cloud technologies, Emory can scale up to millions of patients to predict potential sepsis or how a particular patient will respond to treatment. Emory has created a dashboard so that doctors and nurses can monitor all patients’ chances of getting sepsis within a given p ­ eriod—​ ­for instance, within the next six hours. The dashboard leverages the algorithm’s score to determine which patients are deteriorating versus those who are stable or getting better. Clinicians can determine trend lines and better interpret what the algorithm picked out during the analysis. In ­non-​­AI analysis, the clinician is limited to their knowledge of the patient or what they’ve read about in medical studies. Leveraging this new A ­ I-​­based solution, a doctor can compare the patient to 20 million patients who have had similar symptoms within seconds with machine learning. Emory states that with this new system, they can catch over 85% of sepsis patients and maintain a low rate of false alarms. In addition to helping the medical professional, the dashboard allows the algorithm designers to continuously monitor the algorithm’s performance to improve accuracy and lower the rate of false alarms. The world health crisis of ­Covid-​­19 has undeniably impacted wider society, but it has also exposed the longstanding health inequities in the Black community. Earlier in this book, I highlighted how historical travesties like the Tuskegee Study have contributed to most Blacks’ trepidation regarding medical care; more so, how robotic ­decision-​­making algorithms have exacerbated that fear. As a result, Black Americans have traditionally opted out of research trials even at the risk of forgoing lifesaving technological advancements like those described above. In 2020, the presidents of two historically Black universities thought they were providing Black leadership by enrolling in a ­Covid-​­19 vaccine clinical trial. They had seen how Covid had impacted the Black community at large. So they pushed their campus communities to consider enrolling in the trial. This is an excerpt from their joint letter to faculty, staff, students, and alumni: We, the Presidents of Dillard and Xavier, are already participating in the Ochsner Medical System’s current vaccine trial. As part of the study,

Medically Induced Trauma

133

we have received our injections and monitored and reported any symptoms and side effects. As a result, we are both well.

Instead of getting the expected applause, they were stunned by the fierce backlash. Hundreds of outraged commenters flooded their school’s social media accounts. “­Our children are not lab rats for all the drug companies,” one post said. “­I can’t believe an HBCU would do this to our people,’ said another reply. “­Tuskegee, Tuskegee… I and mine aren’t first in line,” said another respondent. “­Don’t do it yall,” was yet another response. In fairness, some reactions were positive. That’s the truth, my aunt was sent home and returned hours later and died the same day… her husband of 65 years died the next day from Covid. People that I know that you would think can survive this deadly disease have died from it… two cousins, ages 27 and 32, gone.

Although this early response to the vaccine adoption was expected, the Black community has now warmed up to the idea of the vaccines. Some vaccine proponents would say that this “­warming up” has come at the expense of many Black lives taken by C ­ ovid-​­19. Regardless of where one stands on the vaccine line, AI has been instrumental in the rapid creation, delivery, and management of the ­Covid-​­19 vaccine. As a result, millions of people worldwide have received at least one dose of the vaccine. AI’s ability to learn and adapt to information patterns and healthcare trends in real time is critical in determining which populations to target to curb the spread of the virus. In the ongoing debate that happens almost daily in the Black community, the argument defenses range from, “­How were they able to come up with a vaccine so fast?” and “­If they can do that, why haven’t they cured cancer or HIV?” to “­I’m not anyone’s lab rat” or “­Don’t let history repeat itself.” I’m not telling anybody whether they should get vaccinated or not. (­By the way, I’ve received all three shots.) But, I can say emphatically that AI and other technological advancements, along with trillions

134

Hidden in White Sight

of dollars available to researchers, have a lot to do with why we have a vaccine today. We know that, as good as humans are at exploring different things and being creative, dealing with masses of data and making intelligent decisions about that data is something that AI algorithms can do at the necessary pace required to solve complex problems like Covid. While the debate on who gets the vaccine and who doesn’t rage on, even the medical devices used to test and treat the disease aren’t without scrutiny and controversy. In an article reported by the Guardian newspaper entitled, “­From oximeters to AI, where bias in medical devices may lurk,” the author describes how issues with some medical devices could contribute to poorer outcomes for women and people of color. One such device that was chronicled in the article was the oximeter. Oximeters estimate the amount of oxygen in a person’s blood. They are vital in determining which Covid patients may need hospital ­care—​­not least because some may have dangerously low oxygen levels without realizing it. However, the article cites additional studies that indicate that oximeters work less well in those Covid patients with darker skin. The article reports that the devices beam light through the blood, and skin pigmentation may have effect on how light is absorbed. The United Kingdom’s then health secretary, Sajid Javid, himself a British Asian, told the Guardian that the devices were designed for Caucasians. “­A s a result, you were less likely to end up on oxygen if you were Black or brown, because the reading was just wrong.” The report continued by stating that experts believe that these inaccuracies could be at least one of the reasons why the Covid death rate has been higher among minority ethnic people. Like the Emory example described earlier, AI systems build actionable data sets that allow doctors to examine root causes or other things that researchers don’t have time to spend on. For example, when dealing with a virus that mutates exponentially, creating variants along the way, AI can analyze the data in real time and project potential causes of the mutation. This technological capability alone can reduce a ­12-​­month trial into a ­three-​­month trial. Recently at my barbershop, I allowed myself to engage in a “­debate” about whether one should get the vaccine or not. I emphasize the word “­debate,” because if you’ve ever been in a Black barbershop,

Medically Induced Trauma

135

you know a discussion is really an argument. More importantly, in a Black barbershop, your credentials are left in the parking ­lot—​ ­that means no one cares what you do for a living, whether you are a doctor or a chief technology officer for a large corporation. All that matters here is your ability to defend your argument. So, of course, nothing that I’ve written in this chapter would be considered acceptable or argument worthy. Against my better judgment, I allowed myself to get dragged into the dreaded “­vaccine or no vaccine” argument. The argument was triggered by the question “­How in the Hell could they come out with the vaccine this quickly?” Since AI and money were the core of my defense, I had to do it quickly. If you take more than two minutes to state your case in a barbershop argument, you’ll get interrupted by the other arguers. So, AI and money (­lots of it) were my defense. What if? (­By the way, those two words are a great way of proving a point in the barbershop.) What if almost two trillion photos and videos were taken with cell phones alone? What if someone gave you three billion dollars and access to the brightest technical minds in the world. You had access to all the great engineers from Google, Amazon, Apple, IBM, and anyone else you needed at your beck and call. This is your problem to solve: You need to write a program that would view and analyze all of the photos and videos to determine whether a black or white object was appearing in the media and then sort the videos. Remember, you don’t have to do the work; you just need to use the provided resources. You have a year to build the application. If you can pull it off, they’ll give you another billion dollars as a parting gift for solving the problem. Which of you will sign up for this task? I believe this is precisely what happened with the Covid vaccine. We now have a vaccine, and some profitable corporate winners walked away with billions of dollars as a parting gift for creating the vaccine. It seemed like I had captured their attention at least because no one ­interrupted—​­which was unusual for a barbershop argument. I concluded by saying that, regardless of whether you decide to get the shot, the vaccine’s effectiveness has very little to do with how fast it came out. AI has the unique capability of leveraging sophisticated

136

Hidden in White Sight

algorithms to analyze and ­self-​­learn from a large volume of healthcare data and then use the obtained insights to assist the clinical practice in making lifesaving decisions. The question isn’t whether technological and medical advancements like the Covid vaccine can be created and distributed in record time, but whether those advancements can be done fairly and ethically. History argues “­no,” but faith says “­yes.” We will see!

Chapter Nine

Colored Ads

Pitch Darkness Dishonesty in advertising has proved very unprofitable. —​­Leo Burnette

In 2020, the United States was in the midst of one of the most

heated Presidential debates in the country’s history. The intensity of this melee wasn’t lost on Eva, a ­63-­​­­year-​­old retired factory worker. She hadn’t previously shown an interest in politics; embarrassingly, she had never voted. Like many of her friends, she had always thought that elections were predetermined at best and rigged at worst. No she DOI: 10.1201/9781003368755-10137

138

Hidden in White Sight

had never voted, not even for Barack. In her opinion, no way America was going to let a Black man be President, especially not with a name like Barack Obama. Nope she had never voted. But this year seemed to be different. Like most of her friends, watching the battle and rhetoric play out on CNN, CNBC, and Fox News was a nightly fixture at her home. Maybe it was her guilt for not voting in the Barack election, whatever the reason was. This year was different, and she was intent on ensuring that her vote would count. Georgia was a swing state, and in this closely contested race, Eva was keenly aware that her voice and those of people like her could determine the next President of the United States of America. Before leaving for the polls on the morning of Tuesday, November 3, Eva received a Facebook post stating that her local polling place was closed because of water damage. A ­follow-​­up post shortly after stated that voters would be allowed to vote at an alternate polling precinct. The polling place was 30 minutes by bus away from the original precinct. Three more Facebook posts from disgruntled voters seemed to corroborate the earlier post, appearing to be ­first-​ ­person accounts of people who went to polling places in those precincts and found them closed. Eva didn’t realize that this was sophisticated disinformation and a false Facebook campaign unleashed by supporters of the opposing candidate to suppress her and other ­precinct-​­goers’ votes. The attackers were professionals and had spent the last four months creating many fake but realistically looking Facebook and Twitter accounts mimicking people who lived in Eva’s neighborhood. They mobilized the accounts to make photos and user profiles representing fake Black voters using an AI algorithm. The algorithm was programmed to release random posts and create content targeting users who would likely vote for the opposing candidate. As a result, the attackers built up a significant base of followers by having some of the a­ ttacker-​­controlled accounts follow other a­ ttacker-​­controlled accounts and ensuring that the ­attacker-​­controlled accounts followed accounts of people that Eva knew, many of whom would follow them in return. A typical tweet posted in the early morning of election day read, “­Unbelievable, I went to my polling place on Auburn Avenue this morning to vote, and it was CLOSED! Signage outside the location instructed me to vote instead at a different location!” Dozens of other

Colored Ads

139

seemly real users posted similar experiences of being locked out of the polling places. Some even had pictures of the polling place with a “­Closed” sign. Because of the trailing effects of social media, even the legitimate users’ “­real” friends got in on the hoax by reposting the disinformation. This is a real example of how AI can be used to create virtual humans with emotional and r­eal-​­life behavior attributes that even other real humans will join in to promote the biased agenda of the creators. Local radio and television stations joined in the media frenzy by reporting the closures, although several were careful to note that the claims had not yet been verified. The attackers knew that Eva and potential voters like her used public transportation to and from the voting locations. They also knew that because of the ­30-​­minute distance between the original polling location and the suggested one, some percentage of voters would choose not to vote at all because of this. The authors of this app along with its sponsors were hoping that the algorithm would dissuade potential voters from going to the polls because of distance. Let’s review how AI was leveraged Can Dove soap turn you to carry out this diabolical plot to supwhite and pure?—​­A 2017 press the vote of the local ­precinct-​ nationally televised Dove g ­ oers. I was tempted to use “­criminal” advertisement campaign in‑ to describe the attacker’s scheme. But sinuates That It Can. was it actually criminal? In the infamous words of Denzel Washington’s character, Alonzo, in the box office hit Training Day, he quips, “­It’s not what you know, it’s what you can prove.” I know the ­d ie-​ ­hard Training Day fanatics would probably have suggested another one of Alonzo’s memorable ­one-​­liners from the movie: “­To protect the sheep, you gotta catch the wolf, and it takes a wolf to catch a wolf.” I know the latter might be most appropriate, but I’ll leave it for you to decide which one fits the best. But for sure, there was no existing law or statute that the attackers broke by misleading Eva. Unethical, no ­doubt—​­appalling and wicked for ­sure—​­but as Alonzo stated, “­It’s not what you…” I think you get the point. In analyzing what happened in Eva’s case above, let’s ask and answer a few questions: First, how did the algorithm know that Eva was Black? Or did it?

140

Hidden in White Sight

Answer: This was easy as the algorithm quickly determined her race, iden‑ tity, and neighborhood and as most people would ­guess—​­It was hard to sus‑ pect that she was voting democratic. How did the algorithm know who Eva would vote for? How did the algorithm know that Eva utilized public ­transportation—​ t­hus making the alternate polling option a troublesome one? How was the algorithm able to create such realistic p­ rofiles—​­ones that even Eva’s real friends would also follow? How were the attackers able to ­out-​­dupe even Facebook and surpass all of their security measures? Without spending time diving into the depths of the above questions, the answer to the last question can leverage to give insight into the prior ones. Facebook provided the tools that the attackers needed to answer all the above questions. While Facebook profiles may not explicitly state a user’s race or ethnicity, its algorithms can definitely discriminate based on those factors. It’s not ­far-​­fetched to determine that if somehow the algorithm could determine that Eva was Black and lives in a particular neighborhood, then the other questions could be similarly deduced. Some ­legal-​­minded readers might remind me that discrimination based on race and ethnicity violates the existing civil rights laws protecting marginalized consumers against advertising harms. You would be right to remind me of such, but note that those laws generally apply to housing, employment, and credit and have their bases in legislation created before the 2000s, and as we have described in this book, they are of no match to the craftiness of those who endeavor to use AI to circumvent those statues. In Eva’s case above, Facebook advertising tools were used to disseminate targeted misinformation and controversial political messages to vulnerable demographic groups. In 2020, civil rights organizations criticized Facebook, Inc. for not seriously policing misinformation and hate speech on its platforms. “­ Facebook approached our meeting today like it was nothing more than a PR exercise,” Jessica González, c­ o-​­chief executive officer of Free Press, a nonprofit media advocacy group, stated afterward. “­I’m deeply disappointed that Facebook still refuses to hold itself accountable to its users, advertisers, and society.” On June 17, 2020, civil rights groups including the NAACP, the ­A nti-​­Defamation League, and Color of Change launched the “­Stop

Colored Ads

141

Hate for Profit” campaign. The campaign sought to pressure major corporate advertisers to cease advertising on Facebook for July 2020. Over 1,000 significant advertisers joined the boycott, including Target, Starbucks, Unilever, and Microsoft. A year later, evidence shows that Facebook’s algorithm automatically still can be programmed to determine who is offered an ad leveraging protected class information such as race, origin, demographic, and gender. As we mentioned earlier, there are federal laws in place to prohibit this type of a­ d-​­based classification. Still it happens. With Facebook and several other platforms, the tool allows advertisers to identify people, for example, a particular population segment. Although it’s illegal to specifically select a member of a protected class (­gender, race, etc.), we know that advertisers often aim to do just that. In the case of Facebook, their framework has a feature called “­Lookalike Audiences.” This enables an advertiser to reach people who resemble members of a designated source audience. Although I use Facebook as an example to illustrate my point, the above case is prevalent in today’s advertising campaigns and isn’t widely viewed as nefarious. However, many advertising companies don’t shy away from admitting that determining who you are is critical in identifying what products and services you are amenable to. As in other industries, AI is used to scale and analyze trend data faster. So for people who are looking to do “­bad” things with the AI technology, see advertisement as the medium of choice. How often have we discussed a particular desire and had our advertisements catered to it almost immediately? For example, you may have texted a friend, asking them to meet up for food or maybe you’ve googled a restaurant. Then, after opening your social media app you were presented with advertisements for those same restaurants. These tailored advertisement techniques are not uncommon and unusual. To their delight, We readily provide our favorite advertisers with the data they need; subsequently, they don’t hide that they are collecting it. If you take a quick peek at Google’s privacy policy, you’ll see that they make it crystal clear that they are collecting and storing this information. I did just that; I clicked on the fine print. Even though I sought to prove a point, I was still somewhat surprised by how much data they collected on me. You, too, might be surprised

142

Hidden in White Sight

by how much information they collect on you. The following items are just a few examples: • My name, gender, and birthdate • My mobile number • The websites that I’ve visited • My Google searches • Places where I’ve been recently • Where I work • Where I live • My favorite American Football team is the Steelers, and I’m an Atlanta Braves fan • The YouTube videos that I’ve watched r­ ecently—​­my favorite music. A quick analysis of my YouTube history would expose that I’m fascinated by the life and lyrics of Tupak Shakur. An inquiring ­mind—​­or an inquiring “­algorithm” could probably guess my race because of this one. All of this incredible and ­not-­​­­so-​­private data, alongside the plethora of other information that can be derived from it, often seems to make its way into our AI algorithms and frameworks like the one mentioned above is what gives curators access to it. When using frameworks like the one mentioned above to advertise to potential home buyers, if the algorithm discovered that it could earn more engagement by showing more white users homes for purchase, it would discriminate against Black users. As I have consistently repeated in this book, bias occurs most often during data collection when the training data reflects existing prejudices. Dr. Latanya Sweetney, an esteemed professor at Harvard, suggested in one of her published studies that Google searches may expose racial bias. She found that Black American names like Leroy, DeShaun, and Tanika when searched yielded advertisements that read “­A rrested?” with a link to a website that could perform criminal record checks. While traditional white American names like Karen, Brad, and Thomas most often provided general contact details. Of course, whenever a user clicks on the ­arrest-​­related ad, Google receives revenue. The algorithm’s main objective is to generate revenue for Google. Again, the ad algorithm reflects our human biases,

Colored Ads

143

portraying Black Americans as criminals at worst and “­possible criminals” at best. “­There is discrimination in the delivery of these ads,” concluded Professor Sweeney, adding that there was a less than 1% chance that the findings could be based on chance. I must admit that Google has made significant advancements in alleviating some of the biases in their algorithms. There is currently no law that addresses this legal breakdown. The government must ensure that Facebook, Google, and other advertising platforms are fair and transparent. We will discuss this further later in the final chapter. Advertisers have long known that they could use psychology to influence our behavior to make certain purchasing decisions. From Wednesday’s “­W here’s the beef?” to Nike’s “­Just do it” to M&Ms’ “­Melt in your mouth, not in your hands” advertisements, they are meant to drive us to purchase. Whether you are a ­world-​­class athlete training for the Olympics or a 1 ­ 0-­​­­year-​­old dreaming of the Olympics, the Wheaties commercial “­The Breakfast of Champions” is to ensure that the cereal is included in your shopping cart. As in other segments, racially motivated advertisements are nothing new in retail. And it certainly didn’t take AI to expose it or bring it to the attention of those who care about how we are characterized by the media. Recently, H&M was forced to apologize for an advert showing a Black child in a green hoodie bearing the slogan, “­Coolest monkey in the jungle.” What about Heineken’s “­Sometimes Lighter is Better” commercial where a bartender slides a bottle of Heineken Light toward a woman? The beer passes several men and women of color before reaching her, and then the statement “­Sometimes Lighter is Better” appears. Finally, how about model and TV personality Kendall Jenner offering a Pepsi to a police officer in the middle of a street protest? This ad was slammed on social media as an exploitation of the Black Lives Matter movement. The ad was released on April 4, the 49th anniversary of King’s assassination. Although I’m sure that some advertisers purposely designed their ads to garner attention and get social media clicks, it is more likely that most didn’t think their ads were biased at all. At least not until the outrage. My advice to advertisers when it comes to gauging whether or not an ad is biased is to offer up a quote from philosopher George Herbert Mead: “­The meaning of any gesture is given in the response.”

144

Hidden in White Sight

In the case of racially targeted ads, the response from the outraged community is normally swift and relentless. I might add that the responders care less about whether the racially motivated ads are ­A I-​ ­generated or the lever was pulled by an actual ­person—​­the results and consequences are the same. AI is being used to fundamentally revamp how ads are created, targeted, and positioned in the marketplace. Retailers are adopting AI almost twice as fast as any other industry. From the smallest online retailers to Walmart and Amazon, AI is a fundamental tool in determining what clients want and don’t want. Enhancements in machine learning and AI make personalization more relevant, less intrusive, and less annoying to consumers. However, hidden in the bowels of these incredible technological advancements lies the ability of automated systems to perpetuate harmful biases. Black shoppers are keenly aware that companies charge different prices to customers in other zip codes. So if you live on the south side, you will more likely pay more for the same item than on the north side of town. This isn’t done by accident, as most large retailers use algorithms to determine their dynamic pricing structure. Not only are the algorithms determining what the prices are, but they also decide what products will appear on the shelves in your local store. In Black buying experiences, it’s not unusual to discover that luxury brands like Gucci, Versace, and Luis Vuitton can’t be found in the local mall, requiring a trip by those brand fanatics to the other side of town. Most shoppers don’t know that, in most cases, the inventory decisions aren’t made by humans but by algorithms, as you probably guessed. I’m sure some retail giants might argue that data used by their marketers to segment their customers isn’t inherently demographic. Still, few will deny that their current variables often correlate with social characteristics. And as we have shared throughout this book, AI has a unique ability to seep through data and find hidden and sometimes unintended patterns, which sometimes leads to determining race and/­or gender, leading to biased outcomes. Let’s review a simple usage scenario. Suppose a large retailer wants to use AI to determine which customers are most receptive to price discounts and coupons. The company will need to use historical data to train the algorithms to accomplish this. If the customer profiles fed

Colored Ads

145

to the algorithm contain attributes that correlate with demographic characteristics, the AI system will most likely make a different recommendation for different groups. It is usual for cities and neighborhoods to be divided by ethnic and social classes, and a user’s browser data correlates to their geographic locations. You don’t have to enable location tracking for a retailer to know where you are. Google can provide them with this information through your IP address or search history. It is also widely known that the more money you make, the more likely you are to leverage discounts and coupons. It is also factual that the “­whiter” you are, the more money you are likely to make. There are exceptions, but generally speaking, the above two declarations are valid. So there is an excellent chance that the historical data fed to the algorithm consists of participants from the above categories. An algorithm trained on such historical information ­would—​­even without knowing the race or income of c­ ustomers—​­learn to offer more discounts to the white, affluent buyers. Bias increases exponentially when marketers and advertisers rely on their own internal data to train their models. Some industries are riddled with historical references, so it’s not an abnormal expectation that AI use in those industries might mimic those preferences. The algorithms beneath AI are beholden to the ­integrity—​­or lack ­thereof—​­of the underlying data sets used to train them. Have you ever wondered why movies that target Black audiences don’t traditionally do well at the box office? The “­Black Panthers” of the world serve as exceptions to the rule, but generally speaking, the claim is valid. Advertising budgets are aligned with how well the AI forecasting tools predict that the movie will do in the market. Paid engagement and tracking data for a new film typically don’t happen until eight weeks before a movie is released. So past data from IMDb and other authoritative movie sources are the data inputs for predictive movie algorithms. The algorithm’s understanding of the film’s prospective viewers (­and subsequent advertising budget) is based entirely on how similar releases have fared in the past. The algorithm matches metadata points like rating, director, cast, and producer, crunches the numbers, and projects how well the new release will do. The issue is that most experts would agree that Hollywood has been biased against Black film producers, directors, and actors and actresses. In an article posted on the Chief/­Marketer website in 2019, the author wrote:

146

Hidden in White Sight

If studios rely on these algorithms to greenlight films, AI becomes a blocker to new and different kinds of content being produced before even getting into the marketing and distribution. Producers can say their decisions to greenlight sequels and franchises are ­data-​­based, and they are, but the data is biased by historical box office performance.

I leverage this example to illustrate that traditionally biased industries like film and marketing will unknowingly reproduce prior biases. This further illuminates perhaps the central pillar of this book, which is that it is literally impossible for algorithms to imagine a future that is different from the past. For those wondering what happened to E ­ va—​­did she get the opportunity to vote in her first Presidential election? She did indeed. Although the campaign hoax proved to be effective as it sent Eva to the wrong polling location, with the help of her daughter, she was able to make it back to the appropriate place before polling closure. By the way, her candidate won. Some glass ­half-​­full and optimistic readers might conclude, “­A ll well that ends well.” Unless we are willing to pour water in the glass with sweat equity, I don’t believe that the “­all well” metaphor will “­end well” for people of color as it relates to technological bias. The problem won’t solve ­itself—​­it will take change agents who are willing to roll up their sleeves and speak louder than the profiteers. If you don’t like something change it; if you can’t change it, change the way you think about it —​­Mary Engelbreit

C h a p t e r Te n

Weapons of Mass Destruction

­A I-​­E nabled Trigger Finger Gun control? We need bullet control! If every bullet cost five thousand dollars, we wouldn’t have any innocent bystanders. —​­Chris Rock

Amazing Grace How Sweet the Sound that Saved A Wretch Like Me. I once

was lost, but now I’m found. Was blind, but now I see. Those are the soothing and redemptive words of the emblematic Black spiritual, “Amazing Grace.” In 2015, those were also the lyrics that President Barack Obama harmonized during his now-iconic eulogy for Reverand Clementa Pinckney. These words were delivered after Dylan Roof walked into Emanuel Episcopal Church in Charleston, South Carolina.

DOI: 10.1201/9781003368755-11147

148

Hidden in White Sight

Roof sat down, listened to a Bible study, pulled out a handgun, and killed nine Black parishioners. The spark that ignited his violent fury came to him in a Google web search after he learned of Trayvon Martin and the man who shot him, George Zimmerman. Courtroom documents reported that Roof stated the following: “I was unable to understand the big deal. This prompted me to type the words ‘black on white crime’ into Google, and I was never the same since.” Unfortunately, neither would Roof’s country be the same after he carried out the plan inspired by the Google search. Trayvon’s death set off a movement whose fire will not and cannot be quenched. As a reaction to the shooting, the Confederate battle flag atop the South Carolina statehouse ended its 54-year presence at the Capitol. On May 24, 2022, an 1 ­ 8-­​­­year-​­old gunman invaded the Robb Elementary School in Uvalde, Texas, fatally shooting 19 children and two teachers before being killed by police, casting a fresh spotlight on gun violence in the United States. At the time of writing, the US had 233 mass shootings in 2022. I hesitate to state an exact number because the number will change before this book is published. But, of course, no American institution is o ­ ff-​­limits to the people who would rather kill in numbers instead of using those same guns to “­protect themselves.” The underlying tenet of the 2nd amendment was the right to keep and bear arms. We’ve seen small and m ­ ega-​­churches alike as targets for mass shootings; large 2nd Amendment Rights? hotels and casinos like MGM; colleges, Gun control legislation is high schools, and Lord behold, even elnot new in ­America—​­Black ementary schools ­t argeted—­​­­9 -​­ and 1 ­ 0-­​ Codes were enacted to en‑ sure freed slaves couldn’t ­­year-​­old children killed. Hospitals and fight for their constitutional healthcare centers are targeted. Not rights. even at the grocery store are you safe. ­ 8-­​­­year-​­old gunOn May 14, 2022, an 1 man entered a supermarket in Buffalo, New York, and killed 10 ­people—​­all of them Black. He reportedly wrote a manifesto, describing himself as an ­ethnic-​­nationalist supporting white supremacy and motivated to commit political violence. He traveled four hours from his home to find Black citizens doing everyday tasks. So, he purposely picked a supermarket. In yet another tragic incident, only weeks later, a gunman stormed a Tulsa medical center and killed a renowned surgeon alongside three other people with an automatic weapon.

Weapons of Mass Destruction

149

The Gun control argument in the US has a very unsavory and long history of discrimination. Its origin extends all the way back to shortly after the Civil War when the first restrictive gun control legislation was enacted. The laws were aimed at ensuring the newly freed slaves would not have an effective way to defend themselves, and thus, could not fight for their legal and constitutional rights. Some might assume that ensuring and defending 2nd amendment rights were never the true aim of gun control proponents. I have spent the majority of this book sharing examples of how AI and machine learning algorithms can be used to deepen and widen racial inequities, but AI has and continues to provide ­life-​­saving and vital solutions for the lives of everyday citizens. I assert in this book that some AI solutions should be designed better, with greater emphasis on testing and training, and even suggest that some solutions should never be developed. But like other technological advancements that came before it, the possibilities for l­ife-​­altering solutions designed with AI are limitless. There are some instances where AI should be used more pervasive, albeit for some strange reason it’s not. One example that should be considered is ­A I-​­enabled and powered smart weapons to minimize mass shootings, gun theft that often results in violent and unsolved crimes, and ­self-​­inflicted bodily harm. As mentioned earlier in this book, smartphones are unlocked with a fingerprint to deter unwanted access. A ­push-­​­­to-​­start car won’t drive unless the key is within range. Likewise, the technology to improve safety standards for firearms exists; yet the United States lags behind current trends. For far too long, technology has been used to make guns increasingly more dangerous, while the acceptance of technologies like AI which can make them safer lags behind. Assault weapons were never designed to be used by the general public, not even as a tool to defend oneself from other private citizens. Perhaps no gun in US history has received as much fame, praise, and criticism as the ­AR-​­15 rifle. It has spun up controversial conversations both at the dinner table and in the halls of Congress. Even though the weapon is as newsworthy as a celebrity wardrobe malfunction caught on camera, most Americans don’t even know what the term AR means. Contrary to popular belief, it doesn’t stand for “­Assault Rifle.” It actually stands for “­Armalite Rifle,” named after the gun’s original manufacturer. The weapon wasn’t originally manufactured for hunting or even defending oneself from fellow citizens. The gun’s original purpose was

150

Hidden in White Sight

the US Military. The A ­ R-​­15, which was also branded at the time M ­ -​­16, was originally authorized to be manufactured to be used by the military during a time of war Regardless of where one sits on the political b ­ us—​­whether it’s at the back of the bus or the front, whether you take the position of the civil rights icon Rosa Parks and refuse to move of one’s position on the ­matter—​­the one undeniable is that none of the above institutions should have an ­A R-​­15 automatic gun anywhere on the property. It is hard to defend the need to have an A ­ R-​­15 automatic gun at a public school or a library. Can AI help prevent the next mass shooting? I say… “­Possibly!” Let’s consider the following hypothetical example to illustrate my position. Ask any of ­18-­​­­year-​­old Johnny’s friends, and they will readily admit that he is unstable and a little disturbed at best. He rarely attends school, and when he does, his days are filled with hurling threats and insults to his teachers and fellow classmates. Johnny’s life seems to be spiraling out of control, and he decides to purchase a ­semi-​ ­automatic ­A R-​­15 and take his frustrations out on his fellow students. The following day he heads to the school with ­A R-​­15 in hand. As he goes to fire the weapon, the trigger won’t engage. The gun won’t get out of the safe position. What happened? In our hypothetical case, the gun manufacturer has taken the necessary precautions to recall weapons to install ­A I-​­based ­software—​ ­this is similar to how a car manufacturer might perform a recall on a defective part. Leveraging AI, the manufacturer installed a location tracker on the weapon. The AI algorithm compares the current location of the firearm with the location of the physical space of the shooter. If the location is in a protected category (­church, supermarket, mall, school, hotel, amusement park, etc.), the trigger will automatically be disabled. In addition, the manufacturer will use ­ olice-​­identified perfingerprinting to ensure that a felon or other p son is not pulling the trigger of the assault weapon. Technology such as identity management can be used to authenticate the user. There is undoubtedly an issue with keeping a gun owner from tampering with the gun or disabling the software. But there are enforcement protocols that can be put in place to punish violators for ­ eapons—​­similar to tampering with a smoke tampering with the w detector on an airplane. It’s illegal to do so. Disabling the system

Weapons of Mass Destruction

151

could cause someone to either lose their weapons permit or alert law enforcement agencies of the violation. Either of these consequences could save lives by altering the course of the shooter’s actions. In addition to curbing gun owner abuse, the ­A I-​­enabled gun can be designed to personalize the weapon so that it can only be fired by its registered owner. Leveraging microelectronic and AI software, the weapon can sense if someone other than the owner is trying to pull the trigger and render the gun unshootable. Hundreds of thousands of guns are stolen every year, taken from houses, vehicles, stores, and even people. Stolen guns are often recovered at scenes of homicides and other violent crimes. This same technology can be leveraged to prohibit underaged usage of firearms which can reduce teen suicide and accidental teen shootings. I am aware that I’ve overly simplified the solution. But one thing for sure is that the above design doesn’t infringe on anyone’s 2nd amendment rights. So why hasn’t this happened already? It’s simple. State and local governments are in charge of gun permits for their citizens. One would think these legal statutes could be quickly voted upon and would not cause anyone to feel their constitutional rights are violated. Gun rights organizations and lobbyists such as the National Rifle Association (­NRA) and National Shooting Sports Association (­NSSA), alongside gun manufacturers, aren’t particularly champing to solve the problem. Their position is that even this technological solution infringes upon users’ constitutional rights. However, those same users leverage AI to do other routine tasks, like fingerprinting and facial recognition, to unlock their mobile devices without regard to violating any constitutional rights. So we all get to keep our guns, our constitutional rights are preserved, and the same technology used to scare and harm communities of color now gets to save lives. The life that AI saves could be yours or those of your loved ones. AI for good is a good thing!

Chapter Eleven

Designers’ Dilemma

A Coding Mindset Building advanced AI is like launching a rocket. The first challenge is to maximize acceleration, but once it starts picking up speed, you also need to focus on steering. —​­Jaan Tallinn

Some years back, my team and I were asked to help provide a

smarter stadium solution for an NFL team. The stadium operators hoped to give their fans a more captivating experience while gaining higher returns on their sports complex. Smart stadiums provide fans with a wealth of information on parking availability, gate crowd congestion, adverse weather notifications, bathroom and concession lines, seat upgrades, special offers, and more. In addition, fans

DOI: 10.1201/9781003368755-12153

154

Hidden in White Sight

receive a convenient, personalized experience with shorter lines and directions to navigate faster through a crowded stadium. AI often powers these ­real-​­time capabilities provided to the fans. Unfortunately, even in seemingly harmless and powerful uses of AI, biases tend to rear their ugly head. In our first ­face-­​­­to-​­face meeting, we met with the client at an o ­ ff-​ s­ ite location to better understand their requirements. The industry refers to this ­co-​­creation meeting as a design thinking workshop. The term “­ diversity” is inauspiciously omitted from this definition. In particuA Design Thinking work‑ lar, around racial and gender lines. This shop is a facilitated meeting unfortunate but common phenomenon where ­multi-​­disciplinary is that we came together with the sole teams plan and prototype purpose of gathering fan requirements ­user-​­centered designs. and generating i­ deas—​­ lots of them. Unlike lectures or presen‑ One would think that diversity would be tations, Design Thinking workshops are used when a needed ingredient to accomplish this teams want to arrive at goal. a ­u ser-​­centered solution The team’s makeup for our design while working together. thinking workshop included twelve parThe workshop is made up ticipants: eight white men, two white of a mixture of both business and technical participants. women, one Asian man, and I. This team makeup doesn’t seem particularly alarming, and most people wouldn’t see this as an issue. But as we discussed in an earlier chapter, ensuring distinct and varying inputs is essential when designing AI m ­ odels—​ ­which would undoubtedly be at play in this solution. Most, if not all, stadiums are made up of a fan base that is representative of ­society—​­meaning that most stadium goers represent a melting pot of different races, ethnic groups, and sexual orientations. This lack of representative input suggests that bias often enters our design process as early as when we discuss the requirements with our client sponsors. If our clients provide prejudiced or incomplete information, there is a good chance that the data passed on to the design process will likewise follow suit. Knowledge is generally gained through in-depth customer interviews, user conversations, and requirement-gathering activities rather than just from technology and data systems. The product of these processes then turns into information. The data from this

Designers’ Dilemma

155

information is used to fuel the models. These models then serve as the foundation for the algorithms. Algorithms form the basis of the AI system’s predictions and suggestions, as is frequently noted in this book. For instance, the algorithms used in policing don’t just extract historical data from computer systems; they also use information provided to us by law enforcement officials during client interviews about certain populations. For example, a policeman might provide guidance during the session that a “Black” suspect might respond a certain way when faced with a certain stimuli “like running”. This input from the officer might make its way into the algorithm thus recommending a certain response by the officer. There is universal agreement among the technology community that delivering trust within AI requires principles such as fairness, privacy, transparency, human intervention, explainability, and a degree of ethics that provides a code of conduct to ensure that predefined guardrails are embraced. These words read well on paper but operationalizing the principles and embedding them within the fabric of an AI development cycle is complex. Often these principles compete with other core principles of the development cycle like “­accuracy.” Quite often, a focus on accuracy comes at the detriment of the other tenets of trust, like fairness. Some experts argue that it is nearly impossible to optimize accuracy and fairness simultaneously. One will lag behind the other, and the best effort is to minimize and manage the gap. Moving from the abstract to the concrete in developing algorithms becomes even more complicated when one considers the difficulty of determining what is fair and equitable in AI design. Let’s consider the following meme to illustrate the difference between the commonly used terms equality and equity. Here, people of different heights must be given other boxes to stand on, look over the fence, and watch the baseball game. Each person’s requirement is different. Earlier in our book, we discussed how data scientists and other technologists apply their human belief systems and personal perspectives to the code they design and deploy. If even experts cannot agree on what is fair, it is almost impossible for junior developers to know whether they are doing the right thing or not regarding the code they develop. The meaning of fairness often depends on who you are and your particular point of view on the subject.

156

Hidden in White Sight

Well respected computer and information science professors, Kearns and Roth, noted: Of course, the first challenge in asking an algorithm to be fair or private is agreeing on what those words should mean in the first p ­ lace—​­and not in the way a lawyer or philosopher might describe them, but in so precise a manner that they can be “­explained” to a machine.

Technologically, this isn’t a difficult task to do as there are tons of tools available in the industry that will help one accomplish ­this—​ ­what makes it extremely difficult is that it requires one to stop what they are doing to do it. Unfortunately, few seem to be willing to do it. It is natural for us to use fairness and equality as two synonymous terms. Let’s consider our smarter stadium solution discussed earlier. One requirement was to provide a feature that gave fans a view of what bathroom lines were longer than others and what bathrooms were inoperable. The physical layout of the stadium included an equal number of women’s and men’s bathrooms. The seemingly fair number of women’s and men’s bathrooms is standard in most commercial building designs. One of the two female members of our

Designers’ Dilemma

157

design session reminded us of the difference between equality versus equity when she asked whether we’ve ever had the “­pleasure” of waiting in a long bathroom line at a stadium event. Before we acknowledge her rhetorical comment, she stated that it’s commonplace for women to remain in long lines. There are at least two logical reasons for this: (­1) Women’s and Men’s bathrooms generally are duplicate square footage, but the stalls in a women’s bathroom take up more space than a similar urinal in a men’s bathroom; (­2) they menstruate, they have more clothes to remove, and they’re more likely to have children or the elderly with them. Should the stadium engineer and architects have considered this when designing the stadium? Should we have considered this when designing the line optimization feature? For example, it could be fair to have equal bathrooms for men and women at the stadium, but human and social behavior suggests that it’s probably not equitable. As a technologist, I, too, struggle with determining how the AI systems apply to all kinds of ­users—​­persons of different races, ethnicities, disabilities, sexual orientations, ages, and gender ­identities—​­not just at the beginning of the project but at each stage of the design process. More importantly, how users among those groups and within the intersectionality of those groups will experience the algorithms. For example, passing by a women’s bathroom line that runs around the corner to walk right into the men’s b ­ athroom—​­and after doing my bathroom business, returning to see the same women in the line reminds me that Technology might mimic this human experience if not carefully designed. If I’m designing a solution for people living in a remote village near Lagos, Nigeria, I need to engulf myself in the culture and empathize with the actual people. Otherwise, there’s a chance that I will (­unknowingly) design a solution that could be misused or biased. The most likely and catastrophic thing that I could do is to assume that I’m developing this solution for a group of white males living in New York instead of designing it from the perspective of the folk that lives in the vicinity of Lagos. The most challenging but necessary thing for a technologist is to wear the intended audience’s shoes in the systems they designed. Rallying around these empathy and sensitivity constructs will undoubtedly help developers not to align their social beliefs to the code they deliver. For example, if designing a solution to recommend the

158

Hidden in White Sight

favorite fruit for a person in a photo, you wouldn’t program your fruit prediction app to recommend “­watermelon” when prompted with an image of a person of color. Doing so would be racially insensitive and undoubtedly cause public ­backlash—​­even if done accidentally. Then too, as designers, we should be more conscious of how and what we design and its impact on marginalized users. As a technologist, we must not just be ­non-​­discriminatory designers but actively ­anti-​­discriminatory designers. For example, what do we do when an algorithm we developed provides sentencing guidelines that recommend that one person gets 5 years while another person gets probation for the same ­crime—​­when it is evident that cultural preferences alongside misrepresented data led to the sentencing discrepancy? A ­ non-​­ discriminatory designer would most likely assess his participation in the process which will probably lead him to believe that he had nothing to do with the outcome (­since he didn’t program the algorithm to be racist). In contrast, an ­anti-​ ­discriminatory designer will take the added and necessary steps to ensure that his code, as well as his fellow developers, is not being used in harmful ways. Not considering that although his client was the judicial court system, the actual “­client” was the person being sentenced. Being ­anti-​­discriminatory is when one is focused on a more holistic approach to ensuring ­bias-​­free code. In this case, developers have taken the additional step to ensure that they are empathetic to the potentially harmful effects of biased code and seek to ensure that their colleagues have a similar perspective. The lack of diversity in development teams should not be ignored as it contributes to lesser accountability and ownership of the design systems. Those marginalized or excluded by systems of power and interpretation are better able to see those systems at work than those privileged by ­them—​­privilege consists mainly of the freedom from being required to notice these systems. Some would say technologists unleash AI bias, and it will be technologists who design the solutions that eliminate it. Others would conclude that machine learning capacities defy the traditional conception of technologist responsibility as algorithms “­learn” from the given data rather than being 100% coded directly by developers. However, this does not change when technologists create an algorithm to perform a task; they consciously choose to delegate a specific task and associated responsibility to the algorithm.

Designers’ Dilemma

159

Thereby, there should be a level of accountability that each designer has for the code that they develop and the AI systems that they design. I believe that responsibility should not only be for the algorithms we create but also for the impact of those algorithms on the impacted community. In our previous examples, the designers made design choices on many occasions, knowingly or unknowingly. When developing algorithms, developers express how things ought to be or not to be, what is good or bad, and desirable or undesirable outcomes. The algorithms we build are expected to follow specific societal norms within the impacted community. This means developers should know the norms that apply to a particular community to develop algorithms that respect them. For example, if it is unlawful for a human to deny a loan based on where the applicant went to school, it would be normal to expect the algorithm developer to know this also. Since the developer is expected to know this societal and legal norm, the algorithm is also expected to know and adhere to these norms. Since the programmer is the author of the “­black box” and its content, “­he holds the key to determining its impact on the solution outcome.” With that comes a certain level of accountability. A lousy song recommendation from Siri doesn’t have the same dire consequences as Terrel and Lorraine not getting their loan because of algorithmic bias. So, the algorithm having a significant role in sentencing a person to life in prison or recommending additional healthcare services carries a different significance than one that predicts a decision of lower societal importance, like one that suggests a book that a reader might enjoy. The professional independence that most developers have to express their own biases in the code they develop makes it harder to ensure that fairness is consistently divvied up when it comes to designer accountability. The “­Give me the requirements, and you’ll see it when I build it. If there are issues, then that’s what the testing phase is for” mindset is commonplace among developers. Even with this seemly not so positive characteristic among developers, most score highly on social responsibility tests, meaning that they desire fair outcomes and have a general concern for others. So most developers are socially responsible; thus, they care about social impact. Which is a good t­ hing—​­imagine if that wasn’t the case.

160

Hidden in White Sight

Most competent programmers rarely grasp data models, and with proper sensitivity, guidance can avoid the common issues causing bias in algorithms. When I started in development, there was little difference between the folk who wrote the code and those who mined, cleaned, and visualized the data in the application. Although their job titles might have depicted them as one or the other, their actual job descriptions required them to do both. In today’s technology world, the former are called programmers and the latter are data scientists. This distinction is important because the person who determines what data is represented in the application is the data scientist, not necessarily the person who writes the algorithm. So, if the team is developing a loan processing application, the data scientist is responsible for determining whether the training data is biased against a specific race. At the same time, the algorithm programmer is responsible for whether the written code is discriminatory against a particular race. This algorithmic bias could occur regardless of whether the data is biased, and likewise, the data could be biased irrespective of whether the algorithm is biased. The two distinct groups could and, in most cases, will use these tools, methodologies, and processes to satisfy their objectives. As someone who considers himself both a seasoned programmer and data scientist (­my age will attest to this), I’ve written lots of code and never thought about whether the data or the code were biased. I’ve asked many of my Black technology friends this question, and most agree that we didn’t think about it. For instance, some years ago, I was on a team that developed and deployed code with the Miami Dade Police department. Not once did I stop to say, “­A re these cops going to use this code to discriminate against Black citizens?” We didn’t think of it. Now, we do. What makes good technologists and good data scientists? Both have a combination of technical expertise and a coding mindset. The first one is easily understandable and the second one is not. Contrary to popular belief, the coding mindset is not like the taste of fine wine; it is not acquired with age. At the same time, technologists of any experience level benefit from practicing the mindset. The coding mindset helps us apply the technical expertise we devel­ igh-​­level goals as programmers’ oped in school while keeping our h insight, resulting in decisions that account for ­real-​­world complexities and consequences.

Designers’ Dilemma

161

Historically, programmers and data scientists have subscribed to three ­high-​­level goals when writing code: ­ roblem—​­For instance, writing code to auto 1. Solves a specific p mate the loan process and ensure that it’s secure, robust, reliable, and available. 2. Ensure that my code is easily read and ­understood—​­Ensure that other similarly introverted programmers can read and understand the code I’ve written. 3. Ensure that my code is maintainable and ­extendable—​­Write my code to make it easy to fix when broken and easy to add new capability when required. The programmer leverages his technical expertise and coding mindset when achieving the above ­high-​­level goals. His technical expertise might fall short when choosing between several decision options. His coding mindset helps him select a solution that best fulfills the ­sometimes-​­ambiguous ­high-​­level dreams. For example, you might be tempted to use “­cool algorithm X” to determine who qualifies for a loan. Then you decide that the system needs to be up 24/­7, so you might reprioritize another algorithm to meet that new reliability requirement. Your client is happy and rewards you with “­Programmer of the Year.” It would cause one to think that maybe, just maybe, when addressing the algorithmic and data bias issue, we consider the “­coding mindset” a starting place. The first task of augmenting the coding mindset is for developers to acknowledge that bias is a part of their individual experience, as it is their societal experience. For technologists, this is not an easy recognition. But, remember, I mentioned earlier that we developers already consider ourselves ahead of the curve for social responsibility, thus being a “­woke” group of thought leaders. In reshaping the “­coding mind,” we must now add two new user categories into our design thought process that weren’t previously considered: 1. The actual “­real” user of the ­system—​­For example, not just the police department, but a specific police officer who might be predisposed to bias or hatred.

162

Hidden in White Sight

2. The impacted u ­ ser—​­For example, the innocent Black c­ ollege-​ ­educated driver who just ran over a pothole is now disappointing his daughter because he’ll be late for her recital. Our first task is realizing that conventional coding and developing machine learning algorithms will require a slight mindset shift that recognizes that “­data” is king. The developer must identify the right historical data, the right prepared version of the data, and, just as importantly, the correct machine learning algorithm. However, finding the right combination requires an iterative process that spans data profiling, data preparation, algorithm selection, algorithm configuration, experimentation, evaluation, and monitoring. In addition to these new user considerations, in the process of reshaping my coding mind, I must consider some questions that I never thought much about: • Are the results of my model fair? • For example, did I ensure that I have representative data in my training model? • Are the results from my code socially accurate? • Is it okay to add those three points to the health algorithm to determine whether I am prediabetic? • Are the results understandable to all that they impact? Tell me again why I got denied. • A re the results fair? Did I ensure that I have representative data in my training model? • Are the results transparent and clear? • Does the user even know that they are talking to a robot? • Can my code handle an unusual circumstance? • Will the results of my application data be kept private? To provide technical insight on the above questions, let’s recall my friend Terrel and his wife Lorraine’s home loan approval issue. They were both prequalified and overly qualified for a home loan. Both had sufficient annual income, credit scores, work history, ­debt-­​­­to-​­income ratios, and adequate down payment savings but still didn’t qualify.

Designers’ Dilemma

163

Let’s consider how the AI algorithm scored them and what the coders could have done differently in the AI system’s design, development, and deployment. The following are the personas that we will consider during this exercise: •  Decision Maker: Appraiser, credit ­ officer—​­ validates and approves model results. • Affected User: Terrel and ­Lorraine—​­applicants. • Regulator: Regulatory compliance ­officer—​­Person responsible for ensuring model safety and compliance. • Data scientists/­coder: Technologists responsible for building, deploying, and monitoring the models and improving performance. The lender’s decision to leverage AI in the first place was to facilitate faster and more accurate credit decisions. In particular, the goal was to speed up the t­ ime-​­consuming tasks of gathering, reviewing, and verifying mortgage documents. In addition, using AI as opposed to human judgment supposedly would avoid the influences of other forces, such as discrimination and redlining. Finally, AI was intended to eliminate the subjectivity and cognitive biases inherent in human ­decision-​­making for approving and monitoring loans. But this same preferential bias that AI was intended to remediate against people of color could easily be propagated against that same group. One would expect that if the people currently participating in the loan approval process have biases and such biases impact who gets loans and what loans they get, then automating those tasks with algorithms would exponentially raise the effects of those biases. The technologists have a crucial role in finding and fixing those biases and continuously monitoring the process to ensure they don’t creep back in. If followed, the following five best practices will ensure that technologists understand the guardrails required for model development, validation, and audit and delivery. More importantly, adhering to these principles will help to minimize the inherent risks associated with AI and machine learning algorithms. Let’s again use Terrel and Lorraine’s failed home loan application example to discuss these principles and how developers can influence each. ­Fairness—​­Ensuring that your AI models and algorithms are impartial and unbiased.

164

Hidden in White Sight

In Terrel and Lorraine’s loan processing case, it would be hard to convince them that the credit approval algorithm wasn’t biased. As discussed earlier, fairness is often subjective; the best thing we can do is test its existence based on a core set of approved corporate policies. For example, a loan processing algorithm that discriminates against people with bad credit would be considered fair simply because it supports widely accepted credit policies. At the same time, the algorithm would be regarded as unfair if it discriminated based on protected attributes such as race or gender. In machine learning, fairness is often associated with accuracy. Frequently, a data scientist’s benchmark for fairness is measured by the accuracy of their models. Accuracy is a metric used in classification problems to tell the percentage of accurate predictions. We calculate it by dividing the number of correct predictions by the total number of predictions. Accuracy is a good metric to assess model performance for simple cases, but not so much for complex subjects like mortgage approval processes. The programmer shouldn’t only be concerned with accuracy but also fairness. Fairness is the process of understanding bias introduced by your data and ensuring your model provides equitable predictions across all demographic groups. Instead of considering fairness as a separate initiative, applying fairness analysis throughout your AI process is critically important. In addition, fairness concepts are fundamental when AI is deployed in critical business processes, like credit application reviews and medical diagnoses. As stated earlier, it is impossible to maximize accuracy and fairness simultaneously, and it is even more challenging to satisfy all kinds of fairness. Algorithms also can be problematic because they are sensitive to small data shifts or model latency. For example, your algorithm could be b ­ ias-​­free on Monday, but the same algorithm could be riddled with bias two months later because of data shifts or algorithmic latency. In Terrel and Lorraine’s case, the reported reason for their loan denial was her employment status, but further algorithm analysis showed that that was only part of the reason. The algorithm awarded points to college graduates of highly selective c­ olleges—​­but predominant Black institutions were not granted the same number of points as their white counterparts. Indeed, the programmer and data scientist would have known this algorithmic input. The data showed that people who attended selective universities were less likely to default on their home loans, so the algorithm

Designers’ Dilemma

165

considered this in making credit decisions. Using variables such as an applicant’s alma mater is now more accessible and attractive because of big data. As the use of algorithms increases and the variables included become more attenuated, the biases will become more difficult for lenders to identify and exclude. So the highly successful couple who graduated from two prestigious colleges were ultimately refused homeownership because of their alma maters. Historical Black colleges and universities weren’t considered ­top-​­tier schools by the algorithm. From a risk standpoint, using college choice as a creditworthiness criterion could be interpreted as a proxy for an applicant’s ability to pay, implicating systemic discriminatory bias. Machine learning algorithms are excellent at finding these proxies that can predict whatever we care ­about—​­such as shopping algorithms figuring out that large purchases of unscented lotion are a good indicator of pregnancy. Or, like the college example, use a zip code to predict the color of your skin. Algorithms often don’t understand when or how a proxy stops being a good predictor, so sudden ­changes—​­like the lifestyle and behavior changes we’ve seen these last couple of years with C ­ ovid-­​ ­­19—​­can degrade a model’s performance. Algorithms often do not distinguish causation from correlation or know when it is necessary to gather additional data to form a sound conclusion. Therefore, the data scientist needs to ensure proper guardrails are in place. For example, data from social media, such as the average credit score of an applicant’s “­friends,” may be a valuable predictor of default. However, such an approach could ignore or obscure other important (­and more relevant) factors unique to individuals, such as which connections are genuine and not superficial. Explainable & ­Interpretable—​­Easy to understand outcomes/­decisions Why did the AI arrive at an outcome? What was needed to produce a different result? As we discussed earlier, most of the existing AI algorithms have employed a ­so-​­called “­black box” approach, meaning that their predictions may be correct, but we can’t explain and understand how the model came to a particular decision. Explainable and interpretable AI constructs force the developer and algorithm owners to explain how their code came to a specific outcome and do it in a manner the user can interpret. As depicted in several examples discussed in this book, understanding and trusting why a perceived biased result was rendered is critically ­essential—​­for

166

Hidden in White Sight

example, explaining a loan processing decision based on the applicant’s alma mater, their ­live-​­in family member’s credit history, or where the applicant currently lives are critically important in ensuring fairness. First, for the impacted person, but just as significantly, for the person handing out the unfair result. In the case of Terrel and Lorraine, the loan officer most likely had no clue that the algorithm was denying them based on their alma mater. Loan officers are paid to close loans, not to deny them. Legally, the financial institution was required to explain why they didn’t give the loan to my friends. However, in the adverse action notice that detailed their rejection reasoning, the institution didn’t do an excellent job of explaining the real reason why they were not approved. The reality is that there were probably hundreds of variables involved in the machine l­earning-​­based credit model, and most likely, the interactions among each of those variables were themselves ­variables—​­their easy explanation was “ ­job uncertainty.” Without the appropriate explanation, it was difficult for my friends to improve their profiles to obtain credit in the future successfully. Whether global or local, the scope of an explanation is an essential aspect of algorithmic communication back to the stakeholders. The duality in the description range gives the user a holistic picture of the model’s outcome. Terrel and Lorraine must understand which global factors resulted in the adverse action decision, for example, length of credit history or college choice. It’s also crucial for them to understand the local factors in their control to receive a favorable outcome, like lower utilization of credit limit. The goal of a financial institution is to If we don’t have an Ameri‑ reduce risk. This means they must decide can race problem, why are who gets the loan and the loan limit. A there so many instances of simple model to help banks decide will bias in our AI applications? consider the basics such as monthly salTechnology has no emotion ary, size of the deposit, and credit rating. or feelings. Therefore, it should be void of prejudice. Terrel and Lorraine wouldn’t have had difficulty interpreting the results if these were the algorithm’s only considerations. Determining the risk score based on these simple inputs would have satisfied legal and social requirements. But since the financial

Designers’ Dilemma

167

institution’s real goal is to determine further those who are less likely to pay, thus further reducing risk, the model might consider thousands of factors: where the applicant lives and where they grew up, their family’s debt history, where they went to school, their social media profiles, and their daily shopping habits. In a simple model that takes only a few w ­ ell-​­understood inputs, it might be possible to determine why Terrel and Lorraine’s loan was denied. But when considering the model’s complexity by adding the other factors, it will be nearly impossible for an average user to determine why the loan was denied. The success of this best practice often hinges on the comprehension of model behavior by less technical audiences. Terrel and Lorraine would consider themselves n ­ on-​­technical, so explaining the model’s technical terminology results wouldn’t have helped them much. The design team needs to identify specific ­user-​­interaction modalities during system design, implementation, and deployment to ensure that each user can interpret the outcomes in easily understandable terminology. Of the four personas that we mentioned above, the affected user is the one that matters most, as it relates to their need to interpret and understand why a particular action was rendered. However, several dichotomies of explanation are necessary even within this singular persona. For example, explaining to my 7 ­ 0-­​­­year-​­old mother why an algorithm analyzed her voice patterns to decide which utility bundle she might purchase will be different than explaining that same outcome to her son (­who is a developer). Therefore, a technologist building interpretive models should consider all possible audiences and provide an explanatory perspective. One must assume that the code we develop and deploy is potentially biased or can be manipulatively biased at worst. ­Transparency—​­Open to inspecting facts and details; Opening the black box Transparency helps us to increase our understanding of why and how the AI algorithm was created in the first place. The lack of transparency in AI ­decision-​­making is likely the ­number-​­one tool that aligns biased Technology to systemic biases of the past. Whereas those past inequalities were evident and open to public scrutiny, the lack of transparency within our AI solutions provides hidden and ­not-­​­­so-​­obvious damage that, in many cases, exceeds the previous

168

Hidden in White Sight

injustices. When the “­black box” has been exposed and opened, it’s referred to as the “­white box.” Transparency in AI helps designers and users understand how an AI system works internally. First, it helps explain how a model is developed, trained, and evaluated to determine what’s in the black box. Next, its decision boundaries and limitations are what inputs go into the model and, finally, why it made a specific prediction. Explainability, interpretability, and transparency are all crucial factors in ensuring equitable treatment for the Black community, but transparency is the most ­sought-​­after by the impacted community. Determining levels of responsibility to align appropriate accountability measures are vital to people potentially harmed by biased AI. Who can undo the error made by an algorithm suspected of being or, in many cases, proven to be intolerant? I didn’t get the loan I know I qualified for, so whom can I escalate to, and what evidence do I have to support my argument? How can you explain something to a person that doesn’t trust you? For instance, explaining to Lorraine why she didn’t get the loan is essential, but your explanation will not be interpreted as genuine if she doesn’t trust you from the start. Instead, Lorraine is more concerned with how she gets to the real reason for her ­denial—​­which she already suspects is bias. Explaining to the person unfairly sentenced based on the outcome of an algorithm isn’t necessarily productive to the person. Instead, the convicted individual would be more interested in having his legal counsel open the black box to be used on appeal to prove sentencing bias, which he already suspects is the case. So, transparency in AI allows the impacted user to have recourse to combat racist and prejudiced outcomes. Moreover, since we aren’t purposely programming biases into our systems, we, as designers, shouldn’t shy away from leveraging toolsets that will allow us to expose the inner workings of our algorithms. Outside of existing local and federal statutes enacted to prohibit civil and human rights violations, there are very few legal consequences for biases perpetrated by AI algorithms. Too often, the AI outcomes are predetermined in the worst case or allowed to make “­unchecked” decisions at best. Regardless, there are no legislative penalties for the decisions that it produces.

Designers’ Dilemma

169

So quite often, no legal and financial consequences are levied because of decisions forged by the ­algorithms—​­no one goes to jail and no one gets held financially liable in civil court. Transparency in AI starts by making the algorithm owners accountable for decisions made by the algorithm. Above, I’ve provided normative guidance and best practices to help individual coders and teams be more diligent in not codifying their personal biases and being h ­ yper-​­focused on rooting out algorithmic biases that occur because of unrepresented or insufficient data. However, nothing replaces the fact that we need more team diversity. There is no substitute for having members of the potentially impacted community as a part of the AI design process. Those marginalized or individuals usually excluded by traditional systems of power are uniquely qualified to see those systems at work compared to those privileged. Privileged, one might a­ sk—​­how so? Privilege, meaning void of the burden of being required to notice these systems. It would seem almost comical to believe a team of black developers would design a facial recognition system that was bad at recognizing d ­ ark-​­skinned users. Or an automated image tagging system changed the skin color of the first and only Black President of the United States of America.

C h a p t e r Tw e l v e

Corporate Choice

Profit versus Responsibility Creating a strong business and building a better world are not conflicting ­goals—​­they are both essential ingredients for l­ong-​­term success. —​­Bill Ford

I

n a recent event with several of the company’s top AI leaders to discuss trustworthy AI, I opened the meeting with the following quote, “­Corporations will do very little to tackle the depth of the pervasiveness of AI bias until we find a way to monetize it.” I paused, looked at my audience, and then continued. “­So it’s incumbent upon DOI: 10.1201/9781003368755-13171

172

Hidden in White Sight

those in the room to create ­value—​­identify, track, and monitor it.” I’m not suggesting corporations don’t care because I believe that the people who lead corporate America do care! But they care more about their profits. All corporations care about growth. The purpose of corporations is to meet stakeholders’ needs and create value. There is a paradox between profitability and responsibility. They are akin to two strangers that rarely meet at night, but when they do, their coming together may produce a h ­ umanity-​­changing outcome. Every corporation’s ultimate aim is to earn a higher return on the shareholders’ equity than can be realized at a bank. In some cases, this mandate for value return tends to be paradoxical to social responsibility and ethics. Ethics and commitment, which involve acting in the interests of others, even when there is no legal imperative to do so, seem to be hard to quantify or justify when considering AI bias. There could be many reasons that account for this corporate hesitancy when it comes to trying to tackle the AI bias issue. Maybe the most striking one is that AI has become a cash cow for many organizations. As mentioned throughout this book, the efficiencies, cost savings, and new market opportunities created through proper AI implementation sometimes outweigh the need to have a concerted effort to create socially responsible AI solutions. ­Oft-​­times, an action that leads to profit creation may be at odds with the organization’s choice to act ethically. Social responsibility and profit might conflict when the business is not doing too well and need to maximize its profits. This paradox is not necessarily bad because, without profit and growth, most companies would find it quite difficult to invest in social causes such as mitigating bias in the AI systems that often serve as profit enablers. So, these are four things that corporate leaders, their shareholders, and their employees should consider when adopting a strategy to balance their efforts for responsible AI deployment and profit. 1. Reputation Damage: AI bias can be an existential threat to your business because it damages the most critical aspect of your business, your reputation. Today, biased algorithms make good Twitter tweets and blog headlines; tomorrow, they will cause mass protests and serve as a galvanizing weapon to force change. Dr.

Corporate Choice

173

Martin Luther King’s statement, “­R iots are the language of the unheard,” is as true today as it was in 1967. 2. Regulatory Fines and Fail Audits: Regulatory fines imposed by state and federal agencies will damage your company’s bottom line or slow your growth. Technology has leapt ahead of legal frameworks, but legislatures are crafting a growing body of literature to delineate how the law should attribute liability for the discriminatory outcomes prompted by biases in AI. 3. Lost Market Opportunities: Governmental initiatives published in 2022 by the European Union (­EU) will limit the ability to sell your technologies and services into those markets. The EU’s proposal states that companies must conduct impact assessments on ­high-​­risk automated decision systems with external third parties. This assessment includes independent auditors and independent technology experts to evaluate the impacts of the algorithm on accuracy, fairness, bias, discrimination, privacy, and security. ­ 4. Future Legal Issues: There will come a time (­sooner rather than later) when discriminatory models will serve as the baseline for class action lawsuits. Once the regulatory fines and legal frameworks catch up, AI bias cases will become appealing to law firms. After the death of George Floyd at the hands of Minneapolis police, protests erupted not just in the United States but also in many other parts of the world. The protests eventually culminated in one of the most significant civil unrest movements in the modern era, as people took to the streets to denounce police brutality and systemic racism. From Europe and America to the Middle East, people came out to show solidarity and protest against inequalities, riding the hype train caused by the incident. What seemed to get lost in all of the media commotions around George Floyd’s murder was the cruel reality that many people of color have been eyewitnesses to similar atrocities or have been the unfortunate victims of such incidents. Either directly by the hands of the police, as shown throughout this book, or sometimes indirectly propagated through technologies like AI. These technologies have slowly administered death by contributing to denying people of color certain constitutional freedoms. As I have taken great care to point out, racial inequality has existed since

174

Hidden in White Sight

the dawn of m ­ odern-​­ day society, but technology has completely transformed how we socialize it and rationalize its existence. The ­Covid-​­19 pandemic strangled the civilized world by limiting our ability to enjoy the simplicities of our everyday social experience. So, we were locked in our living rooms, watching another human being lose his life in an inhumane and traumatic ­experience—​­live and on camera. It felt like the world was an eyewitness to the ugliest parts of humanity for the first time. Then, seemingly overnight, corporations of all sizes, from social media giants to colossal pharmaceutical companies, released statements in support and solidarity with the protestors. Tech companies realized their Black employees’ mental and emotional fatigue and suddenly scrambled to make loud and bold statements supporting causes like Black Lives Matter. Many of these companies donated billions to causes in the Black community. Some even used their money and prestige to pressure state legislatures for more equitable voting practices and lobby to stop their states from enacting voter suppression laws. Mark Zuckerberg published a Facebook post stating, “­We stand with the Black community and all those working toward justice in honor of George Floyd, Breonna Taylor, Ahmaud Arbery, and far too many others whose names will not be forgotten” and “­We all have the responsibility to create change.” Shortly after that announcement, Facebook committed $10 million to “­groups working on social justice.” Instagram posted a mostly black square with a caption that stated, “­We stand against racism” and “­We stand with our Black community.” Google and YouTube displayed black ribbons on their homepages stating they “­stand in support of racial equality.” YouTube also announced that it was pledging $1 million “­to support efforts to address social injustice.” Twitter changed its bio to #BlackLivesMatter, Jack Dorsey committed almost $10 million, and Amazon tweeted, “­The inequitable and brutal treatment of Black people in our country must stop.” In addition, they committed to donating a total of $10 million to 11 organizations, including the ACLU, Equal Justice Initiative, and the NAACP. Microsoft promised to invest an additional $150 million in its diversity and inclusion efforts; IBM pledged $100 million in support of Historical Black Colleges & Universities (­HBCUs).

Corporate Choice

175

Amid this outpouring of generous and ­well-​­deserved corporate philanthropy, the following algorithmic bias was levied toward the Black community.

You see, the gains in the first period, or the first era of struggle, were obtained from the power structure at bargain rates; it didn’t cost the nation anything to integrate lunch counters. It didn’t cost the nation anything to integrate hotels and motels. It didn’t cost the nation a penny to guarantee voting rights. Now we are in a period where it will cost the nation billions of dollars to eliminate poverty and slums and make quality integrated education a reality.

That was a quote from Dr. Martin Luther King Jr’s speech on the three significant evils from 1 ­ 967—​­the triple evils of racism, poverty, and war. He depicted the American civil and human rights journey to one from a fight for “­decency” to what he called a fight for “­genuine equality.” Michelle Bachelet, United Nations human rights chief, said in September of 2021, “­We cannot afford to continue playing catchup regarding ­A I—​­allowing its use with limited or no boundaries or oversight and dealing with the almost inevitable human rights consequences after the fact.” The power of AI to serve people is undeniable, but so is its ability to violate the human rights of a segment of that same population. Corporations have not done enough to prevent bias in AI algorithms, according to a survey from DataRobot. The study reviewed technology firms in the United States and the United Kingdom. The impact and fallout of this neglect are seen in the form of lost customers and lost revenues. As I have chronicled in detail in this book, many companies are faced with choosing profit or social responsibility.

176

Hidden in White Sight

I have an experience that both development teams and the companies that employ them often rebel against prolonging and sometimes postponing release dates to ensure that their software is ­bias-​­free. Many corporations insist that bias must be proven first instead of assuming that ­high-​­risk algorithms should automatically be scrutinized for potential bias. In their study, DataRobot surveyed more than 350 ­US-​­and ­UK-​­based technology leaders to understand how organizations identify and mitigate instances of AI bias. Survey respondents included CIOs, IT managers, development leads, CTOs, and IT directors who used or planned to use AI soon. Around 36% of respondents said their organizations have suffered due to AI bias in one or several algorithms. Among those companies, the damage was significant. It was reported as follows: • They suffered a 62% loss in revenue, losing around 61% of their customers; • 43% of the companies reported that they lost employees as a result of AI bias; • 35% of the companies incurred legal fees due to some lawsuit or legal action filed against them. Respondents reported that their organizations’ algorithms have unintentionally contributed to a wide range of biases against several groups of people. Among them: • 34% were due to gender; • 32% were due to race; • 29% because of their sexual orientation; • 19% because of religion. This shows the impact a biased AI had on the lives of all these people. Although the study only included a fraction of technology companies, it represents a pervasive problem that no corporation leveraging AI would deny. Furthermore, all the companies admitted that they were aware of the bias issues and indicated that they had implemented measures to combat algorithmic bias. Here are the steps companies have started taking to detect bias: • S  ome companies start by checking the data ­quality—​­69% of the companies reported doing this.

Corporate Choice

177

• Some companies also train their employees on what AI bias is and how to prevent ­it—​­51% were reported to be doing this. • Some of the companies preferred hiring an AI bias or ethics ­expert—​­51% of the companies reported doing this. • Companies also started measuring AI d ­ ecision-​­making ­factors—​ ­50% did this. • 47% of the companies monitored the data changes over time. • Deploying algorithms that detect and mitigate hidden biases in training data was something that 45% of the companies resorted to. • Companies also introduced quickly explainable AI t­ools—​­35% used this method. Almost all of the companies reported taking some or the other kind of action against AI bias. However, only 1% of the reported companies did not take any steps. Although some meaningful efforts have been made to tackle bias in AI, it is hardly enough. I’ve shown in earlier chapters that AI is pervasive and intercepts nearly every ­entry-​­point in personal and professional lives, so s­low-​­moving, ­snail-​­paced gains are unacceptable, at least to the impacted segment of our population. I believe that the respondents were not particularly forthcoming in their feedback. For instance, most training data used in AI applications are ­legacy-­​­­based—​­meaning it was derived over years and years of data collection. In our previous healthcare examples, clinical trial data that feeds AI algorithms can sometimes span decades, making it difficult, albeit not impossible, to ensure representative data. One would have to either redo the clinical study or disregard its results altogether. The other glaring omission from the respondents is the impact of their own bias on the algorithms. Since this is difficult to quantify and even more difficult to admit, it was probably best that the study didn’t consider this in its outcome. When considering misrepresented data models and cultural preference, one would suspect that applications dealing with credit approval, home loans, policing, and social scoring would yield much higher bias percentages. As an AI engineer, the denial or, at the very least, neglecting of cultural and ingrained racism’s impact on algorithmic bias by corporate leaders is concerning. I started the chapter by hitting all of the

178

Hidden in White Sight

technology leaders’ outstanding efforts and commitment to social equality, and this would be an excellent opportunity to do more. The impact and scale of algorithmic bias on the Black community are as ­far-​­reaching as one or two senseless murders. As we have written earlier, AI bias has as much to do with killing unarmed Black men as the police that choked the life out of George Floyd. Technology companies are profitable to provide tooling and consulting services around debiasing AI models, which I fully support. Of course, issues like data quality, data drift, and adequately represented data models are essential. Still, we must never forget or omit the cultural and ­social-​­economical influence society views on AI bias’s pervasiveness. AI Bias and Your Bottom Line It is good business for companies to have a cohesive strategy to combat the risk of AI bias. Studies show that employees want to work for companies that visibly and openly uphold standards of equity and ­inclusion—​­not just in the products and services that the company sells but also in the technology that enables those products and services. More importantly, studies show that customers prefer to buy from companies that reflect their values. This is evident in the “­new money” ­amount—​­over $51 billion applied to diversity, inclusion, benchmarking, and strategic initiatives. Therefore, it is good business to acknowledge AI bias and work diligently to mitigate its impact. What Should You Do If Your Predictive Model Has an Incident? Step 1: If possible, take the application offline • Identify the business authority to take it offline. • What is the Business cost or loss of revenue associated with taking the model offline? Step 2: Which regulators must be informed? Step 3: Create a PR strategy Step 4: Who owns the AI incident response budget What do Fairness and Transparency mean to a person of color? We discussed these two terms earlier in the designer’s dilemma chapter, but they deserve a second look related to corporate responsibility. First, corporations must understand the definition of fairness

Corporate Choice

179

and transparency for people of color. One of the significant challenges in making algorithms fair is deciding what fairness means. Fairness is a confusing concept. Fairness is commonly defined as the quality or state of being fair, exceedingly fair, or equitable treatment. But what’s “­fair” can mean different things in different contexts to different people. Most Black people associate the word “­fairness” with equal treatment. The terms typically go h ­ and-­​­­in-​ ­hand; one can’t exist without the other. So equitable opportunities and outcomes are considered proof of fairness. Philosophically, ideas of fairness “­rest on a sense that what is fair is also what is morally right.” Let me explain with an example that most people could relate to—Affirmative action as it is applied to college admission. Most Blacks would consider it fair to allow their child to gain a particular advantage based on the many years of past discrimination that has held back children of color in education, while many white parents would also agree that affirmative action is a “­good” thing. Still not all would believe that it’s fair. How does this example relate to our discussion of bias in AI? Most experts agree that fairness is the ­number-​­one consideration for solving the AI bias problem. It would be hard to find a single corporate Trustworthy or Responsible for the AI framework that doesn’t include “­Fairness” as its main component. Let’s now consider transparency from the eyes of a person of color. We define transparency as “­Help me understand,” which typically means that the more I understand, the more I can hold you accountable. But corporations are rarely willing to share the inner workings of why certain decisions are made, and algorithms often make it easier to disguise the “­whys” and “­hows.” This decision is not necessarily always based on deviant behavior, but in many cases, they believe that doing so compromises their competitive advantage in the marketplace. But sometimes, the lack of transparency surfaces without much effort from the impacted. For example, recently, a husband and wife applied for an Apple Card and got widely different credit limits. It was a rare instance in which two married people with similar credit scores, who at least appeared to be exposed to the same algorithm and transparency, got completely different outcomes. As a result, the company came under fire for alleged gender discrimination, which led to months of negative social

180

Hidden in White Sight

media attention. Ultimately, the company was not charged, and no violations of fair lending laws were found, but they were criticized for shortcomings in customer service and a lack of transparency. Superintendent of Financial Services at the New York State Department Linda A Lacewell said: While we found no fair lending violations, our inquiry stands as a reminder of disparities in access to credit that continue nearly 50 years after the passage of the Equal Credit Opportunity act. This is one part of a broader discussion about equal credit access.

As Black people’s awareness heightens, transparency in AI will be as protest worthy as any other social justice principle. This social advocacy and attention will continue to broaden the media focus on the lack of algorithmic transparency of ­high-​­impact applications. Like other hostilities targeting the Black community, AI algorithmic bias imposed on people, if not alleviated altogether, can certainly be minimized with diligence and attention. The hope is that this book and other similarly focused publications will not only provide Black people the empirical evidence that these wrongs exist but also give them the ammunition needed to combat them. Child abuse and neglect are severe healthcare problems in the United States alone. This abuse has proven to have l­ong-​­term imwell-​­ being. pacts on children’s health, future opportunities, and ­ Many types of abuse and neglect, either by a parent, caregiver, or another custodial role (­such as a coach, religious leader, or teacher), can result in harm or potential harm to a child. Following is an example of an application built from unfair and discriminatory data and cultural preference, but with corporate attention and community insistence, the unwanted bias was mitigated. Like other industries, AI is being used to help enhance child welfare, call screening, and ­decision-​­making processes to improve child safety. To accomplish this, many local municipalities are designing models to assist humans in deciding whether a child should be removed from their family because of harsh circumstances. These tools perform predictive risk modeling that rapidly integrates and analyzes hundreds of data variables for each person involved in an allegation of child mistreatment. The algorithms can rapidly integrate and analyze these data points and create a synthesized information

Corporate Choice

181

visualization. The result is a score that predicts the l­ong-​­term likelihood of future involvement with child welfare. So, dependent on the algorithm’s calculated score, an unsuspecting loving mother could open the door and find child welfare services on the other side. Like other public and private institutions, child welfare is measured on performance. In this case, to ensure that they identify as many potential abusers as possible, every company (­public or private) has performance metrics that govern employee success. The ­God-​­given gifts of human consciousness and love of humanity should be enough for most to imagine the level of family impact of the above algorithm gone awry. But what happens when the algorithm leveraging insufficient data makes a mistake and uproots a child from the home of a loving caretaker? We say “­W hen” instead of “­If” because we know that algorithms make mistakes. I think anything above an accuracy rate of 70% is considered an excellent performing algorithm. And we have already shared that n ­ on-​­Black and financially disadvantaged citizens are not adequately represented in the training data for the model. Before blaming the developer, I suggest that this isn’t solely a developer’s issue but a more societal one. Resolving this data inconsistency would require a total overhaul of our capitalistic system. I just said that an AI’s 70% accuracy rate would be excellent, but think of the remaining 30%—​­what happens to them? Imagine if this was the accurate result of the healthcare system, whereby people are diagnosed using this AI system. You’d be forsaking the remaining 30%. This is the reason why developers should be human rather than developers first. They shouldn’t make algorithms just because it is a part of their job but should consider that the algorithm they are making affects humans like them. Who can fault a financially capable family for using private facilities instead of public ones? Money provides options that those who lack it forfeit. But there are some things that we can do to curb the impact. Let’s consider how one socially conscious county near Pittsburgh, Pennsylvania, tackled the above use case and did it better than most. I’ve never had the pleasure to visit the county, so I have no particular ­allegiance—​­but their website describes Allegheny as a county that offers a unique quality of life. B ­ ig-​­city amenities, a ­small-​­town atmosphere, a transformed and booming economy, and an affordable

182

Hidden in White Sight

cost of living all make Allegheny County attractive to residents, tourists, and businesses alike. In addition, their AI tool was developed with transparency, openness, and fairness to identify flaws, data misrepresentation, and other inequities in the models. Let’s discuss it! Like many similar tools (­primarily governmental or ­healthcare-​ r­ elated), public data sets were used to feed the AI algorithms and models. Since this public data is often easily obtained, corporations can usually get a head start on delivering the application to the public. This isn’t always a good thing, as frequently, public data sets reflect broader societal prejudices, leading to the AI bias discussed throughout this book. Let’s dive deeper. M ­ iddle-​­and u ­ pper-​­class families have a higher ability to hide the abuse by using alternate private health providers instead of public providers. For instance, in my neighborhood growing up, it would have been financially impossible to hide the abuse by using h ­ igher-​­cost and i­ nsurance-​­laden public providers. Referrals to Allegheny County and similar county agencies occur three times more often for Black and biracial families than white families. So as you have no doubt already concluded, if the public data set has three times as much data on potential Black family abusers, then chances are even higher that the AI tool will predict a Black violator as opposed to a white violator. As we have seen throughout this book, this AI deficiency happens more frequently. This isn’t just a developer’s issue; resolving this data inconsistency could require a total overhaul of our societal system. But there are some things that we can do to curb the impact. The first and most obvious suggestion is one that the county incorporates in its deployment: To combat inequities in its model by using it only as an advisory tool for frontline workers instead of a singular ­decision-​­making instrument. Next, they designed training programs to ensure that frontline workers are aware of the potential biases of the advisory model when considering whether the child needs to be removed or monitored. More importantly, the county agency r­ e-​­evaluated its data model to ensure that it collected data sets representing its total population. Therefore, I advise companies to consider population averages and representation when determining the data model makeup. For example, since Allegheny County, PA, is 78% white, it should have been a red flag to see that many child welfare abuse cases were

Corporate Choice

183

Black in a predominantly white county. I’m not suggesting that that’s not possible, but at the minimum, it should provoke further investigation and research. Finally, with developments in debiasing algorithms, Allegheny County took advantage of emerging tooling advancements to mitigate bias and data drift in the model to track and monitor the fairness of their ­high-​­risk model continuously. The development of the Allegheny tool has much to teach corporations about the limits of algorithms to overcome latent prejudice in data and the societal discrimination that underlies that data. Likewise, it has much to teach engineers about the limits of algorithms to overcome latent prejudice in data and the societal discrimination that underlies that data. Finally, it provides engineers and It’s easier to fix an algo‑ designers with an example of a conrithm than it is to change sultative model building that can mitithe mind of the person who gate the r­ eal-​­world impact of potential paid for the algorithm to be discriminatory bias in a model. developed in the first place. Earlier in this book, we shared that if society is going to fulfill the true promises of AI, then corporate leaders must define proper guardrails to ensure that their teams deliver responsibly and trusted AI solutions. Below I have identified ten (­10) potential guardrails that, if implemented, will ensure that the four principles mentioned earlier in this chapter are preserved. They were (­1) Reputation Damage, (­2) Regulatory Fines and Fail Audits, (­3) Lost Market Opportunities, (­4) Future Legal Issues. 1. Innovate by and for d ­ iversity—​­ensure that your development and test teams are created with diversity at their core. For example, it will be impossible to ensure that your company isn’t developing code harmful to the Black community if your data science team doesn’t include Black technologists. ­ 2. Identify the range of model outputs that are considered safe. Once the safe output range has been identified, we can work our way backward through the model to identify a set of safe inputs whose outputs will always fall within the boundaries of the intended design. 3. Train models to be safe and p ­ redictable—​­we should have a reasonable expectation of the range of expected outcomes that an AI system is capable of producing.

184

Hidden in White Sight

4. Design and deploy AI applications with fairness, transparency, explainability, and interpretability at their core. Ensure that these are included with other ­non-​­functional requirements. For example, you wouldn’t release an app that isn’t secure; likewise, you shouldn’t release an app that isn’t fair to the impacted users. ­ 5. Identify, mitigate, and monitor bias throughout the lifecycle of your applications. Incur the cost to ensure that your algorithms are doing what you intended them to do when you developed them. ­ 6. Manage data collection responsibly. Ensure that your data is collected to represent your expected outcome properly. ­ 7. Develop and manage model fitness and impact. Be cognizant of unforeseen consequences caused by the AI system you deploy. While the original intent and goal of the AI system you deployed may have meant to benefit humanity, it could still negatively impact society if it achieves the desired destination in a destructive (­yet efficient) way. For instance, facial recognition systems are often used in policing and crime detection. 8. Create or adapt existing governance frameworks to provide AI accountability to your model’s people. ­ 9. Identify ­ high-​­ risk solutions and pay particular attention to their deployment and use. Some solutions should be designed with care, while some other ­ high-​­ risk solutions shouldn’t be developed. ­ 10. Consider creating a diverse AI Ethics Board to provide thought leadership and guidance on how your organization researches, develops, deploys, and exploits AI technology internally and externally. As a bonus, this one crosses over all t­en—​­ensure that you take responsible and trustworthy AI seriously. It’s not an opportunity for you to replace your analytics chart decks with the words “­Trustworthy AI” and consider it a job done. However, if you are inclined to do so, remember the large corporate gifts you donated to social justice initiatives and organizations like #­BlackLivesMatter—​­we’ve been discussing these same convictions throughout this book. It’s the same sentiment: If the Black lives mattered in the aftermath of public atrocities perpetrated by law enforcement, then those same lives matter when AI algorithms are scaling the hurt even more

Corporate Choice

185

exponentially. How many racist and discriminatory algorithms impact more lives compared to a few rogue and bigoted police officers? The ugliest thing in America is greed, the lust for power and domination, the lunatic ideology of perpetual G ­ rowth—​­with a capital “­G.” “­Progress” in our nation has for too long been confused with ‘­Growth;’ I see the two as different, almost incompatible, since progress means, or should mean, change for the b ­ etter—​­toward social justice, a livable and open world, equal opportunity and affirmative action for all forms of life. —Edward Abbey

Chapter Thirteen

Algorithmic Justice

A Call to Action I cannot say whether things will get better if we change; what I can say is that they must change if they are to get better. — ​­G eorg Lichtenberg

I

n June 2021, five US Senators sent a letter urging Google to hire an independent auditor to evaluate the impact of racism concerning Google’s products and workplace. The many stories I’ve highlighted in this book show that Google certainly shouldn’t have been the only company targeted by the US government regarding algorithmic bias. The most expensive and fastest speeding vehicle will typically gain the attention of the state trooper, compared to a lesser noticed car. So it’s not a surprise that Google, being one of the biggest fish in the DOI: 10.1201/9781003368755-14187

188

Hidden in White Sight

lake, would provoke the ire of Congress first. But the correspondence proved to be unproductive because Google chose not to respond to the inquiry. More importantly, there was nothing that the senators could do to force a response. It wasn’t like Google had broken any laws or any legal statute. Google chose not even to acknowledge the inquiry. To hold Google up as the poster child for algorithmic bias and to let the other violators go untethered seemed to be a type of bias in itself, much like the questionable algorithms that provoked the inquiry in the first place. A friend that works for Google jokingly explained this by repeating an ­age-​­old idiom, “­It’s like the kettle calling the pot black.” I agree with my buddy; it was a meaningless gesture that was most likely a mere desperate attempt at showmanship. There are more effective and lasting ways of ensuring that corporations spend the necessary resources to ensure that their AI systems are not harming people of color. Unlike many injustices levied against people of color, bias in AI seemly appears on the surface to be another unsolvable fact of life. One might be tempted to dismiss it as just another facet of our prejudiced culture, with little hope of accountability. Another dark episode of societal antagonism against people of color. In a 2019 article in the Yale Law Journal, Sandra G. Mayson argued that current strategies aimed to eliminate bias in AI are “­at best superficial, at worst counterproductive.” She asserts that the source of racial inequality in algorithms lies neither in the input data nor in a particular algorithm. Instead, she contends that the problem stems from the nature of prediction itself. Her argument is based on the notion that a prediction draws on the past to estimate the future. In a racially motivated and stratified world, any prediction method will project past inequalities into the future. Although I’m confident that there are some levels of truth to Mrs. Mason’s argument, I tend to be a little more optimistic and believe wholeheartedly that there is always a path to success. As her argument is focused on the risk assessments that often result from the AI algorithm, I still believe the ability to predict future consequences based upon past truths (­regardless of how cruel and dark they may have been) is not an exercise in futility. We do predict the weather, and it’s certainly, to some extent, based on past weather conditions. We’ve seen throughout this book that the AI societal value proposition is overwhelmingly positive one.

Algorithmic Justice

189

My optimism, no doubt, is rooted in realism as I myself have ­eveloped lots of algorithms that, in several cases, resulted in d ­discriminatory acts toward the unsuspected. I would like to believe that I’m somewhat redeemable. Although I have spent a large part of this book readily admitting that cultural bias permeates the design of many algorithms, I also acknowledge that bad, incomplete, and misrepresented data are essential inputs that, and with proper attention and care, if not eliminated altogether, can at least be minimized. So should we not build AI algorithms at all? Of course not. That would be irresponsible, in my view. When it was determined that cars caused pollution, we didn’t prohibit the use of all cars. Instead we invested in research, determined that the lead in gasoline was the primary contributor to emission pollution, and then in 1970, congress passed the landmark Clean Air Act and gave the newly formed EPA the legal authority to regulate pollution from cars and other forms of transportation. Now decades later, we have the ultimate solution in electric cars. In this chapter, I issue a call to action to all socially minded individuals to press congress into addressing the social ills caused by bias applications on communities of color. This chapter doesn’t just issue the call, but it will provide a plan to address it. Although getting regulatory requirements for AI might seem ­far-​­fetched, it is indeed not without precedent. We have an overabundance of examples of where the government has stepped in to protect the health and welfare of its citizens. But isn’t that one of the primary purposes of government in the first place? One such example came to mind while I was in the grocery store a few days before starting this chapter. While walking the aisles of Publix, I found myself l­abel-​­reading to determine the sodium content in a new protein shake that I was considering purchasing. The ingredients were clearly identified on the label. • 170 calories • 25 grams • 260 milligrams of sodium In 1973, the FDA published the first regulations that required the nutrition labeling of certain foods. A decade later, complaints and

190

Hidden in White Sight

petitions were filed with the FDA because of deceptive claims around health and nutrition. For example, people were concerned that product advertising campaigns falsely communicated nutritional claims about sodium, protein, and sugar. Finally, in 1990, the government stepped in with the passage of the Nutrition Labeling and Education Act, which mandated nutritional labeling. Before the FDA got involved, the genuine ingredients and food content descriptions weren’t readily disclosed. Like AI algorithms today, this was also a ­black-​­box scenario where the people most impacted by false advertisements were misled by the corporations that produced and marketed food products. So, people already diagnosed with severe illnesses like diabetes were misled to believe that certain ­sugar-​­filled products were s­ ugar-​ ­free. When in truth, they weren’t. So we know that regulations and proper legal attention can at least limit the impact of corporations more concerned with profit than human lives. Another relevant example can be found in the automobile industry. I believe that we should treat AI bias as a quality issue in the same way that the automobile industry views safety. Consider your dream Tesla automobile. There is strenuous testing from when user surveys are created to when the actual car is designed and shipped to the dealerships. The data industry can begin mitigating bias by viewing AI systems from a manufacturing process perspective, much like the Tesla mentioned above. Machine learning systems receive data (­raw materials), process data (­work in progress), make decisions or predictions, and output analytics (­finished goods). We call this process flow the “­data factory.” Like other manufacturing processes, it should be subject to demanding and exhausting quality controls. I know someone is asking, what precisely does quality control mean and how is this akin to designing and delivering an AI system? Allow me to explain. Quality control in the automobile industry ensures that an automobile is free from bugs, operational issues, and other problems. well-​ Likewise, cars undergo rigorous testing to ensure they’re ­ ­engineered, safe, and comfortable. If you walk into a Tesla or any other auto manufacturing facility today, you will see automation and quality controls at every step of the delivery process. You can be sure

Algorithmic Justice

191

that the factory has tested every component and subsystem when buying a car. Additionally, the vehicle contains ­built-​­in computers that diagnose issues and control dashboard warning alerts. The car is tested before it is sold and then monitored while you drive it. For example, the “­check engine” light will come on when the engine gets too hot. This is a signal to the car’s owner that something is wrong and also provides guidance to the manufacturer on what component actually failed. A similar strategy should be used for AI systems. When an AI algorithm wonders outside of predefined guardrails, then the algorithm ­check-​­engine light should come on providing insight to the impacted user that something has gone wrong. Like the regulations around food ingredients and car manufacturing, AI needs much the same attention around quality if these systems are to be debiased. But we should ensure that the appropriate steps are taken to ensure that the input and the interest of the impacted communities serve as the basis for any regulatory requirements. When determining the solution, AI solution owners and others can benefit from adopting transformative justice principles, including the victims and impacted communities. Over the past year, I’ve reviewed and evaluated several legal frameworks for regulating AI. But unfortunately, they consistently fail to include the voices of communities most impacted by the algorithms. We, as Americans, hold our Constitution up as the symbol of democracy. As remarkable as it was, it was deeply flawed. The Constitution specified what the government could do but didn’t specify what it could not do. The same issue is occurring with AI. There are lots of best practices and normative guidance that determines the functional and accuracy levels of AI algorithms, but little to no policy around defining what “­A I cannot do.” More importantly, the Constitution didn’t apply to everyone. The term “­consent of the governed” applied to white men only. To be specific, the Constitution didn’t include a particular bill of individual rights. Shortly after the ratification of the Constitution, Americans adopted a Bill of Rights to guard against the powerful government we had just c­ reated—​­enumerating guarantees such as freedom of expression and assembly, rights to due process and fair trials, and protection against unreasonable search and seizure.

192

Hidden in White Sight

But even with these human rights additions, we’ve had to make constant updates and changes throughout the life of our Constitution. For instance, it took 75 years after the Constitution was ratified for the 13th amendment to abolish slavery, the 14th amendment for Black people to get the right to due process, and the 15th amendment to get the right to vote. So, as you can see, the process of making a more “­perfect” union is an evolving one. The goal isn’t necessarily to get it right the first time but to move to a more “­perfect” solution. We shouldn’t let perfec‑ tion be the enemy of progress when it comes to influencing and impacting AI Bias. In the pursuit of that fleeting perfection, we have had to revalidate, reaffirm, and, when necessary, strengthen these rights to ensure they represent the totality of the American population. So I feel comfortable speaking for the communities of color, when I say that we all agree that AI should be held up to that same standard. As with the early ratifiers of our Constitution, I endorse the idea that we need a “­bill of rights” to guard against AI’s potential misuse and abuse. This algorithmic bill of rights would give communities of color the inalienable rights necessary to ensure transparency and give us the right to know when an algorithm is being used to decide about us, then provide us the constitutional right to redress if an algorithm makes a mistake. Today, companies live by the honor system regarding AI safeguards. For instance, most tech giants and public agencies of their own volition have moved away from facial recognition. But what happens if they change their minds? Does their public stance shift when faced with meeting their financial obligations to their stakeholders? Do their internal policies and practices align with their public stance? Do federal agencies make different decisions regarding facial recognition technology when trying to ascertain a suspect? Although the larger tech companies have moved away from facial and voice biometrics, the smaller consulting companies haven’t. When creating architecture solutions, it’s pretty easy for a business partner to provide the facial recognition capability while the larger, more recognized companies (­those who have promised to divest in facial recognition) provide the other components needed to deliver the solution.

Algorithmic Justice

193

These partnerships serve as a mechanism for undermining the genuine commitment to ensuring that technologies like facial technologies are not harming marginalized communities of color. We need a ­people-​­first legal framework with the individual’s rights as the central tenet of its constructs. An AI policy is also important because many companies purchase their AI solutions from ­third-​ ­party vendors. Frequently, companies don’t know the right questions to ask their vendors to protect themselves against c­ oded-​­biased algorithms. Consequently, they expose themselves to undue scrutiny. In addition, smaller to ­mid-​­size companies often don’t have the financial wherewithal and the accompanying commitment to ensure that their products align with AI’s ethical concerns. An AI Bill of Rights will help protect companies and provide the proper guidance to solution providers. Few laws explicitly address AI bias outside of the ­societal-​­focused ones. For example, existing ­anti-​­discrimination laws do not address the issues because most of them were created before the internet was even a thing. Moreover, the perforation of technological advancements like cloud computing, big data, and machine learning makes it nearly impossible to enforce the existing legal statutes. For instance, how can one prove that college admission algorithms are biased based on today’s discrimination laws? When shown throughout this book, that bias in industries like healthcare and loan processing is almost impossible to prove. Proving bias is difficult because those most affected by biased algorithms may not know why or how a harmful decision was ­made—​­or even that an algorithm was behind the decision that harmed them. And even if they knew what happened, they had little power to affect the outcome. Therefore, our ­anti-​­discrimination laws must be updated and adapted to properly regulate algorithmic bias and discrimination alongside algorithmic transparency and accountability laws and be legally enforced by Bill of Rights legislation. As it was in the US Bill of Rights, this new document should lay out the individual’s rights. There will need to be legal statutes to enforce the principles defined in this book. Again, as with the original US Bill of Rights document, it will determine what is legal and what is not. More importantly, the AI Bill of Rights initiative must define the range of penalties and restitution for operating illegally.

194

Hidden in White Sight

Most scholars and corporations have limited themselves to “­only” considering the remediation and detection of bias when determining the level of harm algorithms have on communities of color. Their focus is centered on whether a model, data set, or process is biased through deductive reasoning. But as discussed earlier, we must also include the auditing of training data, the disclosure of source code, and periodic monitoring of the runtime environment, if we are going to truly limit the horrors of algorithmic bias. Although this is an integral approach, it is not meant to be a singularly focused one. This is mainly because algorithmic owners will fight this legally, asserting that divulging this information will disclose proprietary information, claiming that opening their algorithm’s inner workings (­ the black box) will allow unscrupulous people to game the system. I agree that this could certainly be an issue, but it shouldn’t deter us from our overall goal. To illustrate this point, let’s take an AI application used by the IRS to determine fraudulent tax returns. Revealing publicly the inner working of the IRS’s algorithm that checks for fraud would undoubtedly give tax fraudsters an unfair advantage and make cheating on your taxes easier. In highly secure and sensitive environments, special care should be taken to determine what algorithmic behavior should be exposed and what should be restricted. However, for communities of color, Regulations are the things the request is for due p ­ rocess—​­meaning that bound us to do what that the exact same process and algowe said we were doing all rithmic flow should be used for all and along. that the algorithm’s input should properly reflect fairness over accuracy.

Legal Implications The implementation of automated algorithmic systems has resulted in numerous legal challenges. In education, courts have been ahead of other industries in setting precedents, where standards of explainability are considered to ensure that laypeople understand how the algorithm came to its conclusion. More importantly, some level of accountability for the harms propagated by AI algorithms is being considered. In Richardson v. Lamar County Board of Education, a teacher certification exam processed by an algorithm was accused of discriminatory

Algorithmic Justice

195

actions. The plaintiff’s e­ x-​­employer alleged that her contract wasn’t renewed because she failed to pass the certification exam. The plaintiff argued that her contract wasn’t renewed because of her race due to disparate treatment or because the exam had a disparate impact on herself and other Black teachers. In an earlier chapter, we defined dis‑ parate impact as a discriminatory practice referred to as unintentional discrimination, whereas disparate treatment is intentional. The plaintiff’s claim was that the algorithm used biased data to determine the exam results. The court sided with the plaintiff on her disparate impact claim, stating that the exam itself was biased rather than the algorithm; therefore, the court didn’t uphold her disparate treatment claim. So, in short, the algorithm wasn’t held responsible; instead, the court ruled that the input data (­the exam) itself was biased unintentionally. Most AI bias cases that make it to court are either ruled against the plaintiff or, at the minimum, considered disparate impact (­unintentional bias). When people of color appeal to the court system for algorithmic bias against them, whether in healthcare, policing, education, employment, housing, or any other segment, it’s almost a guarantee that they will have an unsuccessful pursuit of justice. Most unfortunate is that most people of color understand this reality and subsequently choose not to challenge the decisions made by AI algorithms. As a result, the community lacks the necessary resources to fight powerful interests, and the larger law firms refuse to take the case.

What Constitutes an AI Bill Of Rights Supporting Communities of Color While developing and delivering an AI Bill of Rights framework will be a ­t ime-​­consuming and challenging task, I enumerate the following suggestions to those legislatures who are willing to light the torch of justice. I am aware that there are other efforts by credible organizations who are also pursuing the creation of similar AI Bill of Rights efforts. For instance, the United States White House has recently released an AI Bill of Rights blueprint. I do not intend to take away from that and other noble efforts. However, with that said, I am responsible for expressing my point of view as a member of the Black AI community. The goal of my

196

Hidden in White Sight

initiative is based on two unique perspectives as I suggest the following recommendations: 1. I take the position of a Black technologist who has designed and continues to design solutions and provide guidance to corporations that deliver AI solutions that impact people of color. ­ 2. I take the position of a member of the impacted community who is often marginalized and disenfranchised by these same solutions. Because of these two realities, I inherit a unique responsibility to ensure that any potential legislative efforts to provide guidance around AI systems are meaningful and practically based. Here it is! I. A ll providers of ­high-​­risk AI systems shall undergo strict and relevant conformity assessment testing for each application before placing their products on the market. II. The right to ensure that an AI system is not trampling on a Person of Color’s human or civil rights, and current legal statutes should be updated to accommodate. ­ I-​ III. Communities of Color must be able to opt out or in to any A ­based application impacting them personally. An example: Apple does an excellent job by giving us a choice to consent to using our biometrics on our iPhones. We must consciously decide to enter our fingerprints to unlock our phones. IV. Communities of Color must be able to not only opt out of an AI solution, but companies should also provide an alternate solution. For example, consider the airport CLEAR security solution I discussed earlier. If I was going to make my flight, I had no other option but to opt into the facial recognition and biometrical system. CLEAR should have offered me an alternate authentication method to their biometrics system. V. Companies should ensure that consumers understand how the models and algorithms were created for h ­ igh-​­risk AI systems, such as healthcare, policing, credit, employment, and education, when appropriate. V I. Communities of Color should have the ability to be ­forgotten—​ ­meaning that they can change their minds about having their data participate in an AI algorithm that impacts them.

Algorithmic Justice

197

VII. Communities of Color should be free from being subjected to an AI bias system that hasn’t been certified or audited by a ­government-​­approved agency. V III. Communities of Color should have the freedom from mass surveillance and monitoring by AI systems in the workplace, home, or community when the data is used to make decisions about the person being monitored. Similarly, the provision of the Fifth Amendment protects a person from being forced to reveal to the police, prosecutor, judge, or jury any information that might subject them to criminal prosecution. IX. Communities of Color should have the right to ensure that AI systems that make decisions about them use representative data. X. Communities of Color should have the right to sufficient recourse when an AI system has been proven to make an inaccurate assessment of them. XI. Communities of Color should be comfortable that the corporate AI development teams are diverse when delivering impactful solutions around the six critical and h ­ igh-​­risk solution areas (­ healthcare, policing, credit, advertisement, employment, and education) XII. Users of identified ­high-​­risk AI systems (­i.e. police, bankers, educators, healthcare professionals, etc.) shall use such systems following the instructions accompanying the systems. Calls for US companies to perform civil rights audits have grown over the last few years in the wake of the many racial injustices that have been publicized. As discussed in this book, corporations are pervasively leveraging algorithms to advance and encourage systemic racism, often leading to civil rights violations. These audits are being promoted by advocates to help corporations understand their role in creating and sustaining racial disparities. When digital practices are deployed to implement these disparities, I believe that we must ensure that AI auditing is a critical function in the process. In the world of AI, auditing refers to the preparedness of an AI system to assess its algorithms, models, data, and design processes against a set of predefined requirements or ­g uardrails—​­like the twelve defined earlier in the chapter. Such assessment of AI applications by internal and external auditors helps justify the trustworthiness of the AI system.

198

Hidden in White Sight

While international organizations such as the World Economic Forum, Responsible AI Institute, and For Humanity have championed the need for external AI audits, no standardization or regulation exists yet for mandatory external AI audits. We read earlier how advertising tools used by social media platforms leveraging AI principles can spread targeted information and political messages to ­low-​­income ­groups—​­We’ve seen how healthcare organizations have discriminated in areas like healthcare allocation and availability. At a minimum, AI Audits can examine these algorithms to flag them and provide accountability. Although AI audits are just making their way into the forefront, many corporations, from ­fast-​­food giants to Wallstreet titans, have already been asked to undergo civil rights ­audits—​­many because of the damaging actions of their AI algorithms. For example, Airbnb Inc. 2016 became the first major US corporation to conduct an audit after Brian Chesky, its ­co-​­founder and chief executive, called “­painful stories,” emerged from customers whose requests to book rooms were rejected because of their race. As a result, one wouldn’t have to do too much research to determine AI’s role in the company’s booking technology.

Ensuring Adherence Earlier in the chapter, we provided a list of guiding principles that can be the foundation (­meat on the bone) for any potential AI auditing process when the safety and fairness of people of color is a goal of the algorithm. Once the newly imposed regulations are in place, how do we ensure that corporations and other AI system owners adhere? The same way that the government ensures that banks provide an independent evaluation of the bank activities, controls, and information systems in the ­non-​­AI ­world—​­Consistently audit them. And when you are done with auditing them, audit them some more. As AI deployment intensifies, it will be imperative for public auditors to address the challenges posed by this progressively invasive technology. Furthermore, injecting auditing into the AI process is not an easy task for AI providers to accept. Project delays can be catastrophic for strictly ­profit-​­focused organizations in a world where time is money and profit is king. Developers balked at the additional work and

Algorithmic Justice

199

documentation requirements, those tasked with ensuring system compliance are frustrated because of a lack of data and cooperation, and management finds themselves c­ lock-​­watching as deadlines come and go and budgets become overrun. Although the extra burden of legal oversight might seems overbearing to many corporate leaders, the auditing process can help guard against brand reputation damage. Often corporations forget simple best practices that preparation for auditing exercises will uncover. For example, if the same data is used to build the model (­during the training phase) and verify performance (­during testing or validation), performance and fairness metrics will most likely be inflated. This “­overfitting” leads to the introduction of bias when used on new, unknown production data.

The Enforcement of Auditing/­Accountability Algorithmic accountability is “­the assignment of responsibility to AI service providers for the impact that biased applications have on society.” Algorithmic accountability laws allow us to identify and assign penalties for violations. It also enforces our existing laws against ­anti-​­discrimination around AI bias. Transparency and accountability measures include algorithmic impact assessments, data audits to test for discrimination, and, critically, a set of laws that penalize algorithmic bias, particularly in essential areas like housing, employment, and credit. Since the United States doesn’t have a central body responsible for data protection, auditing AI systems is complicated. Once an AI Bill of Rights is incorporated, we’d need a central governmental agency to enforce it. The new agency would conduct disparate impact assessments and implement fines and penalties levied by the court system on violators. In addition, violations should take a common liability approach. For example, if a driver of a Ryder rental truck r­ ear-​­ends another driver because of a faulty braking system, the law firm would most likely sue both the driver of the truck and the Ryder Corporation for renting a truck with faulty brakes. Likewise, if a ­third-​­party agency develops an AI application for a large bank, then both the solution provider and the bank could be liable. Who is doing the Auditing?

200

Hidden in White Sight

The next question that bubbles to the forefront when discussing auditing is who’s actually responsible for doing the auditing. We need to ensure that auditors also have the diversity that we’ve discussed earlier in this book. People of color need to also have a seat at the auditing table. A recent study by Zippia reported that less than 7% of auditors are Black or African A ­ merican—​­When the overwhelming number of people who are determining what justice, equity, and fairness look like and those people aren’t part of the impacted community, once again it feels like the fate of the Black man is in the hands of an all white jury. The above legal recommendations identified in my AI Bill of rights might seem extreme to some, especially to those of us who develop and profit from the design of AI systems. But I think this is necessary as it will also ensure that solution providers are conscious of their work, knowing that their employer can be held liable for any discrimination or bias. In addition, corporations themselves would begin to take this subject more seriously by ensuring that their design teams are diverse and have the proper ethics training. “­By any means necessary” is the phrase that comes to mind when determining the extent to which methods ensure algorithms are not biased. For the consequential impact of the bias, algorithms continue to be detrimental and catastrophic to the Black community. Scholars and legislators are no longer just considering whether the performance of an AI model is deemed fair or reasonable but whether it shifts power. We’ve seen the l­ong-​­term damage systemic racism and discrimination have done to Black people in America over the last 400 years. Now, AI bias has become a hidden tool, weaponized to ensure that the powerful continue to maintain dominance over the marginalized. All people of good conscience (­not just communities of color) should call for Congress to enact an AI Bill of Rights legislation. Without a doubt, Communities of color have seen both up close and personally that power in America is not distributed equally. In this book, I have asserted at the minimum and proven at the maximum that AI bias in areas that impact life, death, and the pursuit of happiness not only shifts power but, in many cases, centralizes and permanently dislocates it. I argue that the government should take the lead by adopting principles such as the ones I mentioned earlier in this chapter. Still,

Algorithmic Justice

201

even in the absence of government legislation, all companies deploying AI systems should consider adopting the mentioned AI Bill of rights along with a robust auditing process to ensure that their organization stays within its predefined guardrails. Of course, neglecting these initiatives will impose additional and unnecessary risks on your organization. But on the other hand, doing so will ensure that your organization has better trust, compliance, and a­ ccountability—​ ­even more, it will help put your company on the right side of the moral and ethical AI bias dilemma and, more importantly, on the right side of history.

Conclusion: How Can I Help?

Onward and Upward Do the best that you can until you know better, then when you know better, Do Better. —​­Maya Angelou

As a part of my high school senior reading assignments, I chose to read the book The Autobiography of Malcolm X. At the time, I didn’t know which one was more impressive; my reading the book or my predominantly white high school including this book in its library in the first place. I could count the number of “­Black” ­positive-​­influenced books on one hand that were housed in our school library. Later my 203

204

Conclusion: How Can I Help?

teacher, “­a little ole white woman,” helped resolve my dilemma when she gave me an “­A” on the report. She later shared that I didn’t get an “­A” because my book report was all that impressive, but instead because I attempted to read the book in the first place. She said, “­You actually read the book and didn’t use cliff notes like the others.” Reading autobiographies was always difficult and rivaled reading an actual textbook, and no sane individual would do that on purpose. It took years to understand the significance of my teacher’s comments and why she was even impressed that I read the book. In prepping for writing this book, I was reminded of a scene in the book where Malcolm shares a story of his encounter with a “­young ­co-​­ed white girl.” The young student was so inspired by one of his speeches that she took the unusual and proactive step of flying from the South to Harlem, New York, to meet him personally. The girl asked Malcolm, “­W hat can I do to help?” He rudely responded, “­Nothing,” and hurried to his next appointment. I was personally taken aback by Malcolm’s response, and for years afterward, I was lost on why he would be so ­matter-­​­­of-​­fact with her. I imagined how the young, eager to make a difference student felt after the coldness of his response. I’m sure Malcolm knew at the time of his response that people of the young ­co-​­ed’s race had provided much to the cause of freedom thus far in American history alone. Would we even have the significant historical reference point, the underground railroad, if not for the brave white families willing to sacrifice their own lives to ensure that slaves had a safe haven in the North? I would say “­not.” So it was obvious to me that there could have been some sincere desire on behalf of the young white c­ o-​­ed. I’m sure with Malcolm being the historical scholar that he was, he too knew of the significant positive impact that other races had on slavery and other ethnic struggles, but still he answered her by saying “­nothin­g—​­there is nothing that you can do” Years later, after watching the novel turn into a movie produced by Spike Lee and starring the a­ ward-​­winning and esteemed actor Denzel Washington, I saw the c­ o-​­ed’s face after the cold r­ ejection—​­it was a grave disappointment. She was visibly stunned and troubled by his bluntness. Either Malcolm didn’t know how she felt or didn’t care. Based on what I’ve read about him, I suspect the latter is more factual “­He didn’t care.”

Conclusion: How Can I Help?

205

I later read that Malcolm felt that if the young ­co-​­ed was authentic in her desire to help, she would start in her own community. This notion is not remarkably different than many social justice advocates today as it relates to their “­white” counterparts on the topic of racial support. However, Malcolm expressed regret later in his autobiography, for he realized that there was much the young Southern white ­co-​­ed could do to help collaborate with him to further the “­cause of freedom.” After a pilgrimage to Mecca, He completely changed his mind about religion, race relations, and violence on that journey. More importantly, he changed his mind about how he would answer the ­co-​­ed’s question. I know some of you ask, what does this have to do with solving the AI bias issues described in the book? I’m glad you asked, as it has everything to do with it. Let’s review a few fundamental issues discussed throughout the book and highlight some of the suggested fixes. None more critical than how cultural biases and the systemic nature of those biases influence every facet of the AI lifecycle. From the underrepresented voices in our client meetings to the data being provided from historically “­seemingly” trusted sources like clinical trials is riddled with tainted results. From the technologists that think of the solution to the ones that write and deploy the code, biased perspectives are at best, and blatant systemic racism (­at worst) at work. From the corporate “­heads” that pay for the “­unknown” effects of biased algorithms to the ones that do it “­on purpose” because it’s profitable to do so, systemic racism and cultural norms are punishing forces that influence outcomes. Or those who are our sworn elected officials, who “­elect” to sit by idly as people of color bear the brunt of vicious algorithms that long ago ­over-​­stepped the boundaries of the invisible guardrail called “­ethical concern.” And when those ­people-​ ­determinant impact influencers are not at work, data tends to drift all on its ­own—​­assuming that the data was representative and adequately collected in the first place. Since the algorithms and models that make up AI Systems must be continuously managed to ensure that the above “­didn’t happen,” “­isn’t happening now,” or “­won’t happen in the future,” then the ­co-​­ed’s question to Malcolm’s is as relevant on this topic as it was on the subject of civil rights 50 years ago. During our journey, I’ve shared some ­real-​­life practical experiences showing how AI has hopefully forced you to consider some

206

Conclusion: How Can I Help?

impacts of the technology that might have gone unnoticed before. I’m hoping there were some “­I never thought of it quite like that” moments in the book. We discussed how in daily practice, AI is a technology that allows businesses to automate faster. It is widely accepted proof that automation helps companies and their people become more efficient. In most cases, efficiencies drive profit, and thusly AI has become the profit engine of the 21st century. As we continue our effort to demystify the concepts around AI, I define automation simply as the process of making decisions and doing things usually reserved for ­humans—​­The art of mimicking human behavior. Sometimes automation means reducing mundane tasks, while at other times, it’s used to augment those doing those tasks to help them do the routine more efficiently. To accomplish its “­efficiency” goal, AI collects data on us, makes decisions about us, and provides that data back to its operators through guidance, recommendations, or predictions. So we saw how AI’s automation power injected into the loan mortgage process helped to deny my friend, Terrel, and his wife a loan that, under regular operation, they would have received. We saw how even my darling mom was subjected to the world of the automated bot that recommended services based on her race and income. How AI has automated simple processes like identifying humans in crowds and recommending actions based on those results, and even I, “­the engineer,” found myself on the enviable end of AI’s automated predictive capabilities. We laid out the premise suggesting a ­well-​­known belief that our society is riddled with human biases and to mimic human behavior is simply to replicate society’s core evil of discrimination. In particular, doing so without the boundaries of guardrails or directed intent to “­not do it.” So we staked the claim that “­Automated ­decision-​­making can quickly become automated discrimination.” I readily admit that even after chronicling all of the examples cited in the above chapters, I don’t know whether healthcare companies purposely write biased algorithms, but I know that these algorithms often provide biased and unfair healthcare advice to and about people of color. I don’t know whether mortgage companies purposely keep Black Americans from getting houses they qualify for. But, I know Black Americans routinely get denied homes to which they usually would be ­entitled—​­all based on decision points made by biased algorithms.

Conclusion: How Can I Help?

207

I don’t know whether police purposely target Black Americans in the inner city, but I know that predictive policing algorithms are only located in predominantly Black communities. I know that if you overly patrol a particular neighborhood and reduce your patrol over another one, you will likely catch more violators in the one where your resources are predominantly allocated. I don’t know whether companies that make web conferencing solutions like Webex and Zoom are racist against people of color. But, I know that I’m constantly lighting up my environment to accommodate WebEx’s inability to properly adjust to my skin t­ one—​­ensuring that my white colleagues can see me clearly, and there is no need for them to do the same. I know the algorithm could better detect and adjust darker images with correct video training data. I don’t know whether our healthcare providers are purposely building racist applications, but I know that the data models used in their clinical research studies are void of the Black experience and knowledge base. I don’t know whether insurance corporations purposely charge Black consumers more than their white counterparts to protect the same assets, but I do know that insurance companies leverage protected classes like race and zip codes to determine r­ ates—​­and the algorithms that make those decisions are riddled with bias. In this book, I have provided personal examples of how artificial intelligence has become part of our everyday experience, from going to the restroom to jumping on a plane. Whether it is through our daily interactions with healthcare, policing, education, retail, or any other ­area—​­artificial intelligence has gone mainstream; therefore, the potential of bias is as pervasive as the technology itself. If one embraces at least some of the premises in this book and minimally believes that AI without proper guardrails can empower and deepen systemic racism, then the “­how can we help” question is relevant.

We Have the Right Team A couple of years ago, during the epicenter of the American civil protests, I read a fantastic book entitled White Fragility by Robin DeAngelo. During one of our corporate “­Speak Up” sessions, I was

208

Conclusion: How Can I Help?

reading the book and one of our senior leaders suggested that I start a book club. I took him up on it. The book club was meant to provide employees a platform to read books and discuss sensitive topics like race. We grew to over 200 members within a few months of starting the book club with only ­word-­​­­of-​­mouth advertisements. The membership represents all aspects of our employee community. We had early new hires to people who had worked for the company for 30 years or more. Many different ethnic groups and races, alongside employees from several countries, were represented. Even though diversity was at work in the membership enrollment numbers, to my surprise, the group ­make-​­up was overwhelmingly white c­ o-​­workers. It was obvious to me that my ­co-​­workers were asking the same question that the ­co-​­ed asked Malcolm, “­W hat can I do to help?” Unlike Malcolm, in his initial response, I welcomed the question and couldn’t wait to engage. I know now, as I knew then, that it will take a collective effort to ensure that discriminatory and biased applications are harming already marginalized communities of color. More importantly, it would take the people who benefit most from these miscarriages of justice to get involved with the s­ olution—​­the “­white community.” So I was enthusiastically anxious to discuss the topic of race with my ­co-­​ ­­workers—​­even if it was limited to an ­hour-­​­­a-​­week book club meeting. Each week, our book club discussed a separate book, each providing more clarity on race and how we could better and more effectively understand and empathize with each other. Race and technology were often the topics in many of the books we read, but even when it wasn’t, they always seemed to get injected into our discussions. During our book club discussions, I’ve learned a lot from and about my fellow colleagues. I’ve been quite impressed with their willingness to listen and learn more about the experiences of people of color. Although many of the club members had totally different experiences than I did, it was obvious that they shared my concern for the potential damages that can be done by AI technology gone astray. It will take us, collectively and individually, to ensure that our technology systems are free of bias and discrimination. When analyzing the role we can individually and collectively play in ensuring that AI doesn’t deepen and empower systemic racism, Let’s consider the Wolf Pack.

Conclusion: How Can I Help?

209

When hunting for f­ ood—​­the strength of the pack is the wolf, and the wolf’s power is the pack. A wolf pack can cover over 2400 square miles when hunting for food. They are by far the world’s most diligent and successful hunters. Studies show that one of the great strengths of the wolf is that it has the uncanny ability to experience the feeling of social belonging. This innate ability is what no doubt keeps them alive by protecting them against their foes. Leveraging the pack is the only way for a wolf to be successful, and leveraging the wolf is the only way for the pack to eat. In dramatic irony, they could be helping us understand how to tackle AI bias. By this, I’m suggesting that we all stand watch and speak up when it comes to the misuse of technology. Like the wolf pack, we must all be willing fighters when it comes to identifying and calling out AI solutions that are not living up to the societal promise of fairness and equity. Like the young ­co-​­ed asking Malcolm the question of “­W hat can I do to help?” we too must take the bold steps of seeking out racial injustice even when it is hidden inside a “­black box” called algorithmic bias. Finally, I hope that you agree after reading this book that AI is less about the fear of what the technology has done and more about the amazing values that it brings to bear when adequately guarded. As we sit perched on the watch tower of future technological advances, let’s eagerly await the season when AI has accomplished what we had hoped it would be from the beginning: that great social equalizer that helps promote fairness by removing human bias from the equation. Yes, that unyielding and unrelenting force that bends the moral universe’s arc toward justice. So we eagerly away the time when AI has helped to successfully cure cancer; AI has helped us become smarter on energy consumption and helped us save lives by identifying deviant behaviors before they manifest themselves physically. We anxiously await the time when AI has enabled us to close the great digital divide between the races by augmenting our human d ­ ecision-​ ­making and limiting the effects of our racially motivated past. Our chapter subtitle, Onward and Upward simply means moving toward a better condition or higher level. I do believe Maya Angelou’s words bear repeating: “­Do the best that you can until you know better, then when you know better, Do Better.” Now that we know ­better—​­let’s strive to do better!

210

Conclusion: How Can I Help?

There is a balm in Gilead To make the wounded whole There is a balm in Gilead To heal the s­ in-​­sick soul Sometimes I feel discouraged And deep, I feel the pain In prayers, the holy spirit Revives my soul again There is a balm in Gilead To make the wounded whole There is a balm in Gilead To heal the s­ in-​­sick soul If you can’t pray like Peter If you can’t be like Paul Go home and tell your neighbor He died to save us all There is a balm in Gilead To make the wounded whole There is a balm in Gilead To heal the wounded soul —​­Nana Mouskouri

Notes, Inspirations, and Further Reading

Notes The following will provide a brief list of sources that I used when writing the book. I apologize in advance if my interpretation and conclusion drawn from one of your works were taken out of context. Also, my sincerest regrets to anyone’s work that I referenced and omitted to include in the following list.

Inspirations Chapter 1. Listening Ears 1. Mic magazine (2015): The Reason This “Racist Soap Dispenser” Doesn’t Work on Black Skin The author, Max Plenke, shares research on why devices like soap dispensers that are programmed to detect skin color don’t work on Black skin.

211

212

Notes, Inspirations, and Further Reading

2. Congress.Gov (1986) H.R.5484 – Anti-Drug Abuse Act of 1986: This was a law pertaining to the war on drugs that was signed into law by President Ronald Regan. 3. ACLU (2010): The Fair Sentencing Act is a Law to reduce the sen‑ tencing guidelines for the crack and cocaine disparities. This law was signed by President Barack Obama. 4. Propublica.com (2016): How We Analyzed the COMPAS Reidivism Algorithm: Article posted by Jeff Lawson, Surya Mattu, Lauren Kirchner and Julia Angwin that discusses the use of the Compass algorithm that discusses the accuracy of the recidivism algorithm. I use this algorithm to highlight the bases of my argument that algorithms that haven’t been properly trained are subject to bias outcomes. 5. Pew Research Center (2018): Public Attitudes Towards Computer Algorithms: I cite this source to show that a large number of people believe that computer algorithms and tech is general can be bias. 6. The New Statesman (2022): ChatGPT Proves that AI still has a racism problem: Article posted by Ido Vock that discusses how ChatGPT can also produce racially motivated text. https://www.newstatesman.com/quickfire/2022/12/chatgptshows-ai-racism-problem

Chapter 2. The Racist Algorithm 1. Wired Website (2022): Crime Prediction Keeps Society Stuck in the Past: Article posted by Chris Gilliard that helps highlight how predictive policing algorithms targets people of color. I give both personal and other researched examples to illustrate the points that I make in the book on the subject of predictive policing and similar algorithms. 2. Underwrite.ai (2022): The Underwriter AI Tool. I’ve listed the website below https://www.underwrite.ai/ 3. NCRC (2018): HOLC “Redlining Maps” The Persistent Structure of Segregation and Economic Inequality by Bruce Mitchell, PhD. I leverage and support the general premise that Dr. Mitchell proposes in regarding modern practices like redlining that continues to foster and deepen systemic racism.

Notes, Inspirations, and Further Reading

213

4. Data & Society (2016): Algorithms and Publics. Additional information that helps supports my argument that algorithms can introduce bias is the data is not properly guarded. 5. Wikipedia (2019): The Implicit Association Test (IAT). In this section, I discuss the controversial test that detects the unconscious association of objects with conclusions. I use this to further illustrate the implicit and explicit biases that occur AI. 6. The Verge (2021): Heat Listed. I quote from this amazing story by Matt Stroud that chronicles the incidents around how Robert McDaniel was targeted by the Chicago Police and later shot twice because of the constant attention directed upon him because of an algorithm.

Chapter 3. The American Dream 1. Underwrite.ai (2022): The Underwriter AI Tool. I’ve listed the website below https://www.underwrite.ai/ 2. The Markup (2021): The Secret Bias Hidden in MortgageApproval Algorithms by Emmanuel Martinez and Lauren Kirchner. This article depicts and explains high-level examples of scenarios where biases have seeped into the loan approval process through algorithms. 3. Data & Society (2016): Algorithms and Publics. Additional information that helps supports my argument that algorithms can introduce bias is the data is not properly guarded. 4. Wikipedia: The Implicit Association Test (IAT). In this section, I discuss the controversial test that detects the unconscious association of objects with conclusions. I use this to further illustrate the implicit and explicit biases that occur in AI. 5. Allen Institute for Artificial Intelligence, The Delphi Tool – An AI-based tool that models people’s judgement. I leverage this tool to accentuate my points that AI algorithms have a tendency to mimic our human moral consciousness. 6. The Medium – Building a Logistic Regression Using Neural Networks: Cat vs Non-Cat Image Classification by Yash Chaddha After reading this article, it provides more context to my premise that we must tell AI what our expected outcomes should be.

214

Notes, Inspirations, and Further Reading

Chapter 4. AI Gone Wild 1. CBS News – 60 Minutes Tv Show (2017): Faces In The Crowd. I watched the 60 minutes segment by host Lesley Stahl depicting how AI-enabled video analytics would reshape our lives. 2. AP New (2022): How AI Powered Tech Landed Man In Jail with Scant Evidence by Martha Mendoza, Garance Burke, Juliet Limberman and Michael Tarm. This article depicts the story of Michael Williams. I did further research on the story and pull information from (interviews, blog posts, and other first-hand accounts) of his story. 3. Notoduterte.wordpress.com (2015): A Big No To Duterte. I had some prior knowledge of the story depicting Jaypee Larosa’s death at the hands of video analytic software and used this article to remind myself and my audience of the dangers of Video Analytic software when placed in the wrong hands. 4. The Humans Rights Watch (2015): You Can Die Anytime. I pulled additional information from this website on the death of Jaypee Larosa based on the misuse of AI-enabled video analytic software. 5. Forbes (2020): Why It Matters That IBM Has Abandoned Its Facial Recognition Software. In this article by Tim Barjarin, he provides insight on why IBM and other technology companies have seemly abandoned facial recognition. 6. LegiScan (2022), HB 499 Bill – A bill in New Hampshire that bans facial recognition software for government-based surveillance. 7. Senate RPC Website (2019) Facial Recognition: Potential and Risk 8. Kirchner, Lauren (2015b): Will ‘DNA Phenotyping’ Lead to Racial Profiling by Police? Pacific Standard, February 26. https://psmag.com/news/w ill- dna-phenot y ping-lead-to racial-profiling-by-police.

Chapter 5. An Enduring Legacy 1. The Guardian (2020): Ofqual’s A-level Algorithm: Why Did it Fail to Make the Grade? I take several quotes from this article by Alex Hern.

Notes, Inspirations, and Further Reading

215

2. Race After Technology: Written by Ruha Benjamin describes the term “New Jim Code” which I cite several times in the book. I like many others consider Ruha Benajmin a stalwart (heroine) for her work in uncovering biases that are systemic in our society. 3. The New Jim Crow: Mass Incarceration in the Age of Colorblindness is a book by Michelle Alexander, a civil rights litigator and legal scholar. Like Dr Benjamin, we consider the work that Michelle Alexander as critical components to our learnings on how to deal with and communicate racial bias and stereotypes. 4. www.vera.com (2015): The Price of Prisons – I use this website as a research tool to better quantify the price and cost of incarcerating people of color, in particular those who could have possibly being mistakenly convicted. 5. Health Affairs (2018): Diffusion of Innovation Theory, Principles and Practice: I leverage Everett Rogers disequilibrium concept to illustrate the advancement of AI against its potential for adverse effects. 6. Venture Beat (2020): AI Weekly – A Deep Learning Pioneer’s Teachable Moment on AI Bias: I leverage document to provide further clarity on the Twitter beef between then Facebook Chief AI Scientist Yan Lecun and then Google co-leader of AI Ethics, Timnit Gebru. Timnit Gebru like Dr Benajim and Michelle Alexander serves as an inspiration for my book. I am forever grateful for her courage and foresight to provide the vocal stance needed to bring AI bias to the forefront of the technological agenda. I, as many other technologists, stand on her shoulders. 7. The New York Times (2020): There is a Racial Divide in Speech-Recognition Systems, Researchers Say by Cade Metz: I quote Mr Metz’s work within the book to share the specific research numbers of where speech-to-text platforms by some of our largest technology companies struggle to successfully distinguish Black Voices. 8. https://www.siia.net (2022): The Stakes are Simply Just too High by Ronn Levine. Building Responsible US Leadership in AI, Rep. Jan Schakowsky, head of the House Energy and Commerce Consumer Protection Subcommittee, said in her keynote, “The future of responsible artificial intelligence is in the diversity of voices we need to hear from”.

216

Notes, Inspirations, and Further Reading

9. Library of America (1998): James Baldwin: Collected Essays by James Baldwin. 10. Vintage International (2011): The Cross of Redemption: Uncollected Writings. By James Baldwin Kenan Randall Kenan. 11. Benjamin, Ruha (2016a): Informed Refusal: Toward a Jus tice-based Bioethics. Science, Technology, and Human Values. 12. Benjamin, Ruha (2018)”: Black Afterlives Matter: Cultivating Kinfulness as Reproductive Justice. In Making Kin, not Popula‑ tion, edited by Adele Clarke and Donna Haraway. 13. The Ethical Machine (2018): Don’t Believe Every AI You See. https://ai.shorensteincenter.org/ideas/2018/11/12/dont-believe-every-ai-you-see-1 by Boyd danah, and M. C. Elish. 14. Blay, Yaba (2011): “Skin bleaching and global White supremacy: By way of introduction.” Journal of Pan African Studies. 15. University of Chicago Press (2015): The Enigma of Diversity: The Language of Race and the Limits of Racial Justice by Berrey, Ellen. 16. St. Martin’s Press (2018): Automating Inequality: How HighTech Tools Profile, Police, and Punish the Poor by Virginia Eubanks. 1 7. Grove (2008): Black Skin, White Masks. New York by Frantz Fanon. 18. Public Citizen (2021): Algorithms Are Worsening Racism, Bias and Discrimination.

Chapter 6. Our Authentic Selves 1. The Guardian (2020): Ofqual’s A-level Algorithm: Why Did it Fail to Make the Grade? I take several quotes from this article by Alex Hern. 2. Race After Technology: Written by Ruha Benjamin describes the term “New Jim Code” which I cite several times in the book. I like many others and consider Ruha Benajmin a stalwart (heroine) for her work in uncovering biases that are systemic in our society. 3. Computer Vision Foundation (2020): PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models by (Duke University) Sachit Menon, Alexandru Damian, Shijia Hu, Nikhil Ravi, Cynthia Rudin.

Notes, Inspirations, and Further Reading

217

Chapter 7. Mass Unemployment 1. Restaurant Business Magazine (2021): Trends, Ideas and News. I leverage a quote by Jonathan Maze, executive editor that states that technology (AI and others) allow restaurant owners to run their business with less employees. 2. Market Watch (2022): AI Automation: I leverage the quote “47% of workers are expected to lose their jobs because of AI automation”. 3. Business Insider (2019)– McKinsey Study - Automation and AI will disrupt — meaning kill or replace with lower-paying work — 4.5 million jobs held by African Americans by 2030. 4. The Washington Post (2021): Without Access to Charging Stations, Black and Hispanic Communities May Be Left Behind in the Era of Electric Vehicles by Will Englund. 5. Society of Automotive Engineers (2021): SAE AI Levels of Driving Automation™ Refined for Clarity and International Audience – I leverage this website to define the different types of automation to describe how each impacts communities of color and their adoption of autonomous vehicles.

Chapter 8. Medically Induced Trauma 1. Scientific American (2021): Racial Bias Found in a Major Health Care Risk Algorithm. I leverage this article written by Starre Vartan to further dispute claims that cost is an effective way to determine a person’s future healthcare needs. 2. Nature (2019): Millions of Black People Affected by Racial Bias in Health-care Algorithms. I quote several percentage statements from this article to further support my argument that health-care algorithms will continue to propagate bias if not properly monitored and analyzed. 3. Fierce Healthcare (2021): Epic’s Widely Used Sepsis Prediction Model Falls Short Among Michigan Medicine Patients by Dave Muoio. 4. The Washington Post (1988): ‘Jimmy The Greek’ Fired by CBS for His Remarks by George Solomon.

218

Notes, Inspirations, and Further Reading

5. News.Emory.Edu (2022): Can AI Help Unravel the Mysteries of Sepsis and Save Lives? By Rishikesan Kamaleswaran. I highlight in this article how Emory University unlike other studies has done due diligence to ensure that their training data properly reflects communities of color. 6. The Atlanta Black Star (2020): They Are Not Lab Rats by Ashleigh Atwell - I leverage this article to support the argument that because of past medical discriminatory trials like the Tuskegee experiment, communities of color have been slow to participate in the Covid Vaccine. Ashleigh Atwell describes the backlash of a couple of Historical Black College and Universities Presidents after they brought the vaccine trials to their campus. 7. The Guardian (2021): From Oximeters to AI, Where Bias in Medical Devices May Lurk. In this article, I highlight just another example of how healthcare devices were trained on incomplete datasets thus resulting in bias outcomes. 8. Zuberi, Tufuku (2003): Thicker Than Blood: How Racial Statistics Lie. Minneapolis: University of Minnesota Press.

Chapter 9. Colored Ads 1. Free Press: Facebook’s Meeting Was ‘Nothing More Than a PR Exercise’ – This article links to an interview by CNBC News where Jessica Gonzalez of Free Press was interviewed in regard to the meeting. 2. Chief Marketer (2019): Don’t Let AI Bias Derail Your Marketing Efforts by Alex Nunnelly – senior director of analytics at Panoramic. This article highlights how the movie industry use bias algorithms to greenlight films which inhibits their marketing and distribution potential. 3. Insider (2012): 10 Recent Racist Ads That Companies Wish You Would Forget by Charlie Minato. Gained inspiration from this article to support my belief that racist ads have permeated our society for a long time. 4. Brookings (2021): Solving the Problem of Racially Discriminatory Advertising on Facebook by Jinyan Zang.

Notes, Inspirations, and Further Reading

219

5. Federal Trade Commission: Auditing for Discrimination in Algorithms Delivering Job Ads by Basileal Imana, Aleksandra Korolova and John Heidemann. 6. NIM: Algorithm-Based Advertising: Unintended Effects and the Tricky Business of Mitigating Adverse Outcomes by Anja Lambrecht and Catherine Tucker. Leverage some key take-aways from this article to strengthen my argument that algorithmic bias exists in digital marketing.

Chapter 10. Weapons of Mass Destruction 1. NPR (2018): A Brief History of the AR-15. This article served as research for me to understand the history of the AR-15 and to help understand the original purpose of the weapon. 2. CBS News (2021): Will a ‘Smart Gun’ Finally Make It to Market. This article talks to and helps strengthen my conclusion in the book that the gun lobbyist are sceptical. 3. Https://joebiden.com/gunsafety/#: The Biden Plan to End Our Gun Violence Epidemic. This article links to Joe Biden’s Gun Violence Plan 4. CNN Politics (2022): Smart Guns. This is a fact-check article between U.S. President Joe Biden and the NRA on their position regarding smart gun adoption.

Chapter 11. Designer’s Dilemma 1. DevConf (2022): Open Source Community Conference – I leveraged this definition to help the reader understand the meaning for a design thinking workshop. 2. MIT EECS (2021): Coding Mindset. Although I didn’t quote directly from this article, I did use it to draw formative conclusions to support some standard tenets described in this chapter. 3. Towards Data Science (2019): Don’t Blame the AI, It’s the Humans Who Are Biased by Samara J Donald.

220

Notes, Inspirations, and Further Reading

4. Tech Talks (2019): Code-Deep Discrimination: Combatting Racial Bias in Algorithms by Howard Williams. Inspirational reading that helps formulate my thoughts on coders’ impact on algorithmic bias. 5. Oxford University Press, Oxford (2019): The Ethical Algorithm: The Science of Socially Aware Algorithm Design by Kearns, M., Roth, A.

Chapter 12. Corporate Choice 1. European Commission Act (2021) – This proposed regulation published by the European Commission in April 2021 created a uniform legal framework for artificial intelligence (AI within the European Union). 2. The Washington Post (2021): U.N. Official Calls for Moratorium on Artificial Intelligence Tools That Breach Human Rights by Sammy Westfall – She shares a quote by Michelle Bachelet, United Nations human rights chief. Mrs Bachelet states “We cannot afford to continue playing catchup regarding AI—allowing its use with limited or no boundaries or oversight and dealing with the almost inevitable human rights consequences after the fact.” 3. Tech Republic (2022): Guardrail Failure: Companies Are Losing Revenue and Customers Due to AI Bias by Veronica Combs - In the book, I support the study by Data Robot that corporations haven’t done enough to ensure bias doesn’t creep into their algorithms. 4. Yahoo Finance (2021) (finance.yahoo.com): Apple Card Does­ n’t Discriminate Against Women – At Least Not in Terms of Credit Limit by Shivdeep Dhaliwal. I also quote from this article where superintendent of Financial Services at the New York State Department Linda A Laewell said “While we found no fair lending violations, our inquiry stands as a reminder of disparities in access to credit that continue nearly 50 years after the passage of the Equal Credit Opportunity Act”. 5. PBS (2022): How an Algorithm That Screens for Child Neglect Could Harden Racial Disparities by Sally Ho, Associated Press and Garance Burke, Associated Press.

Notes, Inspirations, and Further Reading

221

Chapter 13. Algorithmic Justice 1. NPR, Senate Democrats Urge Google to Investigate Racial Bias in its Tools and the Company by Shannon Bond. Although I share in the book that the 5 Democratic Senators investigated Google, I was quick to remind my readers that AI Bias is a much more widespread problem than singling out of one technology company. 2. UNC School of Law (2019): Bias In, Bias Out by Sandra G. Mason - In this publication, Sandra Mason argues that the current strategies for eliminating bias are flawed. 3. Stanford Law School (2021): EU Artificial Intelligence Act: The European Approach to AI by Mauritz Kop. 4. www.whitehouse.gov, ICYMI: WIRED (Opinion): Americans Need a Bill of Rights for an AI-Powered World by The President’s Science Advisor and Director of the White House Office of Science & Technology Policy (OSTP) Dr. Eric Lander and OSTP Deputy Director for Science & Society Dr. Alondra Nelson.

Further Reading This brief list of additional reading cannot begin to do justice to the countless compelling pieces of work that are available to the viewer on the topic of trustworthy and responsible computing and its impact on social justice and change. I’ve decided to stop at twenty (20), but I encourage you to exhaust yourself in the topic of Algorithmic Fairness and Race Relations. 1. Alexander, Michelle – The New Jim Crow: Mass Incarceration in the Age of Colorblindness, New York Press, New York, 2010. 2. Angwin, Julia, Lawson, Jeff, Mattu, Surya, Kirchner, Lauren, ProPublica – Machine Bias, 2016. https://www.propublica.org/ series/machine-bias 3. Benjamin, Ruha – Race After Technology: Abolitionist Tools for the New Jim Code, Polity, Cambridge, 2019. 4. Biewen, John – Seeing White, Podcast bibliography, Center for Documentary Studies, Duke University, Durham, NC, 2015. http:// podcast.cdsporch.org/seeing-white/seeing-white-bibliography. 5. Broussard, Meredith – Artificial Unintelligence: How Computers Mis‑ understand the World, MIT Press, Cambridge, MA, 2018.

222

Notes, Inspirations, and Further Reading

    6. Buolamwin, Joy, Gebru, Timnit (PhD), Raynham, Helen, Raji, Deborah (PhD) – Gender Shades, 2017. https://www.media.mit. edu/projects/gender-shades/publications/.     7. CodedBias Documentary, a Shalini Kantayya Film. https://www. pbs.org/independentlens/documentaries/coded-bias/     8. Cook, Kelemwork, Pinder, Duwain, Stewart, Shelley, Uchegbu, Amaka, Wright, Jason – The Future of Work in Black America, 2019. https://www.mckinsey.com/featured-insights/future-of-work/ the-future-of-work-in-black-america      9. Dyson, Michael Eric – Entertaining Race: Performing Blackness in America, St. Martin’s Press, New York, 2021. 10. Glaubitz, Alina – How should liability be attributed for harms caused by biases in Artificial Intelligence? 2021. https://politicalscience.yale. edu/sites/default/files/glaubitz_alina.pdf 11. Koenecke, Allison – Voicing Erasure – A spoken Word Piece Ex‑ ploring Bias in Voice Recognition Technology. https://www.ajl.org/ voicing-erasure 12. Krupar, Shiloh, and Nadine, Ehlers. 2017. “‘When treating patients like criminals makes sense’: Medical hot spotting, race, and debt.” In Subprime Health: The American Health-Care System and Race-Based Medicine, edited by Nadine, Ehlers and Leslie, Hinkson, pp. 31–54. Minneapolis: University of Minnesota Press. 13. Nakamura, Lisa, Chow-White, Peter– Race After the Internet, Rout‑ ledge Taylor and Francis Group, London, 2012. 14. Nelson, Alondra – Technicolor: Race, Technology, and Everyday Life, NYU Press, New York, 2001. 15. Noble, Safiya – Algorithms of Oppression, NYU Press, New York, 2018. 16. O’Neal, Cathy – Weapons of Math Destruction, Crown Books, Largo, 2016. 17. Oluo, Ijeoma – So You Wanna Talk About Race, Seal Press, New York, 2019. 18. The New York Times (2018): When the Robot Doesn’t See Skin by Joy Buolamwini. https://www.nytimes.com/2018/06/21/ opinion/facial-analysis-technology-bias.html 19. Trotta, Daniel, Exclusive: Smart guns finally arriving in U.S., seek‑ ing to shake up firearms market, Reuters, 2021. https://www.reuters. com/technology/exclusive-smart-guns-finally-arriving-us-seekingshake-up-firearms-market-2022-01-11/ 20. West, Sarah, Whittaker, Meredith, Crawford, Kate – Discrimina‑ tion Systems: Gender, Race and Power in AI, 2019. http://hdl.handle. net/1853/62480.

Index

accountability 158, 159, 168. 184, 188, 193, 194, 198, 199 algorithmic 199 enforcement of 199–201 accuracy 164 adherence, ensuring 198–199 AI see artificial intelligence (AI) Alexander, Michelle: The New Jim Crow: Mass Incarceration in the Age of Colorblindness 78 algorithmic accountability 199 algorithmic bias 33, 123, 159, 160, 169, 175–178, 180, 187, 188, 193–195, 199, 209 causes of 31 as illegal 9–11 algorithmic systems, implementation of automated 194 algorithmic tools 81 algorithm programmer 160 algorithms artificial intelligence 48, 81, 91, 109, 112, 114, 123, 124, 138, 150, 167, 168, 189, 190, 194 black-boxed 124 controversial 89 deep learning 132 Internal Revenue Service 194 machine learning 3, 33, 35, 47, 149, 162, 163, 165 Ofqual 76 Policing Prediction Algorithms 21 racist 25–38

speech-to-text 14 Amazon Go 118 Anti-Defamation League 140 anti-discriminatory 158 Anti-Drug Abuse Act 21 Arbery, Ahmaud 174 AR-15 rifle 149 artificial intelligence (AI) 2–9, 32, 33, 64, 89, 94, 98, 111, 207 algorithms 48, 81, 91, 109, 112, 114, 123, 124, 138, 150, 167, 168, 189, 190, 194 audits 198 bias 9, 31, 158, 172, 193, 200 Bill of Rights 193 Bill of Rights, supporting communities of color 195–201 “black box” phase of 17 blind spots 10 decision-making 45, 167 deployment and profit 172 enabled customer service chatbot 14 models 47 principles 198 underwriting solutions 41 Assault Rifle 149 Atlanta Police Department 26 auditing, enforcement of 199–201 authentic selves 23 The Autobiography of Malcolm X 203 automated algorithmic systems, implementation of 194 automated decision-making 33

223

224

✻

Index

artificial intelligence (AI) (Cont.) automotive industry 111 autonomous car 113–114 autonomous vehicles 115–116 Bachelet, Michelle 175 Baldwin, James: I Am Not Your Negro 53 Benjamin, Ruha: Race After Technology 77 bias algorithmic 9–11, 31, 160 artificial intelligence 9, 31, 158, 172, 193, 200 confirmation 49, 50 explicit 49 perceptual 52 racial 121 big data analytics 41 Bill of Rights 191–193 supporting communities of color 195–201 United States 36, 93 Black Americans 20, 21, 30, 46, 48, 76, 90, 117, 118, 120, 121–125, 127 , 131, 132, 143, 206, 207 killing of 21 black box approach 165, 168 black-boxed algorithms 124 Black capitalism 106 Black community 19, 27, 29, 35, 44, 81, 105, 106, 112, 118, 123, 132, 133, 168, 174, 175, 178, 180, 183, 200 Black defendants 22 Black faces 50, 52, 53 Black leadership 132 Black Lives Matter movement 143 Black shoppers 144 Black skin tones 4 Black youths 27 blind spots 10 artificial intelligence 10, 11 Boquiren, Antonio 59 Brantingham, Jeff 30 Bush, George 76 Centers for Disease Control and Prevention (CDC) 20 Chicago Police Department 29

City Survey Program 42 civil rights groups 43 Clean Air Act 189 Clinton, Bill 20, 76 coding algorithms 8 Collins, James M. 34 colored ads 137–146 Community Reinvestment Act (CRA) of 1977 44 COMPAS 22 confirmation bias 49, 50 controversial algorithm 89 Covid-19 pandemic 174 criminal acts 31 Cruise, Tom 58 culture cultivation 95–97 Darker skin tones 4 DeAngelo, Robin: White Fragility 207 DeCelles, Katherine A. 34 decision-makers 9, 10, 54 decision-making AI 45, 167 automated 33 deep learning 131 deep learning algorithms 132 detection systems, gunshot 62 Dewey Boulevard 59 Dick, Philip K. 30 Diffusion of Innovation (Rogers) 83 diversity 10, 21, 35, 37, 73, 85, 88, 94, 95, 98, 100-102, 125, 154, 158, 169, 174, 178, 183, 200, 208 154 Dorsey, Jack 174 drug laws 30 Duterte, Rodrigo 58, 59 Easterbrook, Steve 110 Eberhardt, Jennifer 51, 52 electronic vehicle (EV) adoption rate of 111–112 rise of 111–118 employment, mass 105–118 empowerment 99–103 Epimaco Densing III 58 Equal Credit Opportunity Act (ECOA) of 1974 43, 44

Index equality 77, 87, 155, 156 equity 6, 155, 157, 172, 209 essentialism, racial 125 explainable AI 177 explicit bias 49 Face Capture interface of the software 59 Face Depixelizer 89, 91 face-to-face meeting 154 facial recognition software 69 Fair Housing Act (FHA) of 1968 43 fairness 6, 48, 55, 78, 155, 156, 164, 179, 209 Fair Sentencing Act 21 federal court order 75 Federal laws 43 Federal Trade Commission 44 Floyd, George 91–94, 173–174 Food and Drug Administration (FDA) 124 Frazier, Darnella 92 fundamental attribution error 52 genuine equality 175 González, Jessica 140 Greenwald, Tony 50 guardrails 6, 9–11, 118, 155, 163, 165, 183, 191, 197, 201, 205–207 Guess Who’s Coming to Dinner? 88, 97 Gun control argument 149 gunshot detection systems 62 The Hate U Give film (2018) 27 HBCUs see Historical Black Colleges & Universities (HBCUs) heart failure risk calculator 127–128 Henry, Daniel 109 Herbert, George 143 high-risk care management program 121, 122 hiring algorithm 9 Historical Black Colleges & Universities (HBCUs) 174 HOLC see Home Owners’ Loan Corporation (HOLC) Holder, Eric 23 Home Owners’ Loan Corporation (HOLC) 42, 43

✻

225

HootSuite 66 HRDAG’s Kristian Lum 30 Human Rights Watch 58 I Am Not Your Negro (Baldwin) 53 identity, race-based 80 inequality 121; racial 173–174, 188 Internal Revenue Service (IRS) 16 algorithm 194 Internet of Things (IoT) technologies 65 Iris Facial Recognition software 61 IRS see Internal Revenue Service (IRS) Javid, Sajid 134 Jenner, Kendall 143 Jim Crow 72, 74, 76–78, 80–82 era 14, 71 laws 74, 78, 80–83 Kennedy, John F. 75 Khan Noonien Singh 61 King, Martin Luther 49, 175 Krishna, Arvind 60 Lacewell, Linda A. 180 Lecun, Yan 89 legal implications 194–195 legalized racist systems 74 leveraging cloud technologies 132 leveraging technologies 110 LMI see low-to-moderate-income (LMI) lost market opportunities 173 low-to-moderate-income (LMI) 44 Lum, Kristian 30 machine learning (ML) 7, 8, 32, 41, 44, 108, 132, 144, 164, 190, 193 algorithms 3, 33, 35, 47, 149, 162, 163, 165 based credit model 166 capacities 158 credit model 166 systems 190 majority-Black neighborhoods 30 Maselli, Peter 47 mass destruction, weapons of 147–151

226

✻

Index

mass employment 105–118 Mayson, Sandra G.: Yale Law Journal 188 McAfee, Andrew 47 McDaniel, Robert 28 medically induced trauma 119–136 ML see machine learning (ML) NAACP 140, 174 National Bureau of Economic Research 125 National Cancer Institute Breast Cancer Risk Assessment Tool 128–129 National Rifle Association (NRA) 151 National Shooting Sports Association (NSSA) 151 neural networks 17, 115 New Jim Crow regime 83 The New Jim Crow: Mass Incarceration in the Age of Colorblindness (Alexander) 78 New York Taxi and Limousine Commission (TLC) 5–6 non-discriminatory designer 158 NRA see National Rifle Association (NRA) NSSA see National Shooting Sports Association (NSSA) Obama, Barack 15, 16, 21, 89–91, 138, 147 Obermeyer, Ziad 122 Odie, Mama: The Princess and the Frog 80 Ofqual algorithm 76 other-race effect 51 perceptual bias 52 Petty, Herman 107 Pew Research Center survey 24 plaintiff’s claim 195 Policing Prediction Algorithms 21 PredPol 30 The Princess and the Frog (Odie) 80 ProPublica report 22 Public Health Service (PHS), US 20 Race After Technology (Benjamin) 77 race-based identity 80

race correction mechanism 128 racial bias 64, 89, 94, 121, 124, 125, 142 racial essentialism 125 racial inequality 173–174, 188 racism, systemic 31, 35, 173, 197, 200, 205, 207, 208 racist algorithm 25–38, 91 racist systems, legalized 74 Reagan, Ronald 76 regulatory fines and fail audits 173 reputation damage 172–173 return on investment (ROI) 115 Richardson v. Lamar County Board of Education 194 risk calculator, heart failure 127–128 Rogers, Everett: Diffusion of Innovation 83 Roosevelt, Franklin Delano 42 Schaffer, Stefan 112 Schakowsky, Jan 98 ShotSpotter 64 evidence 63 software 62 skin tones 74, 75, 116, 207 Black 4 Darker 4 Soltani, Ashkan 68 speech-to-text algorithm 14, 109 Stahl, Leslie 57 Star Trek II: The Wrath of Khan 61 stereotypical concepts 50 Stevens, David 47–48 stolen guns 151 “stop-and-frisk” policy 30 Stroud, Matt 28 Student Borrower Protection Center 35 Sweetney, Latanya 142 systemic racism 31, 35, 173, 197, 200, 205, 207, 208 Taxi and Limousine Commission (TLC) 5–6 Taylor, Breonna 174 Taylor, Frederick 119–120

Index team diversity 94–95 team member contributions 98–99 Tesla autonomous vehicle 114 Thomas, Angie 27 transparency 42, 60, 64, 96, 130, 155, 167–169, 178–180, 182, 184, 192, 193, 199 trauma, medically induced 119–136 Turner-Lee, Nicol 31, 32 Tuskegee study 19, 132 United States Bill of Rights 36, 191–193 Public Health Service 20 Sentencing Commission 23

✻

227

Urinary Tract Infection (UTI) Score 129–133 voice amplification and collaboration 97–98 Wallace, George 84 weapons of mass destruction 147–151 White Fragility (DeAngelo) 207 William, Michael 62–63 Yale Law Journal (Mayson) 188 Zuckerberg, Mark 174

Taylor & Francis eBooks www.taylorfrancis.com A single destination for eBooks from Taylor & Francis with increased functionality and an improved user experience to meet the needs of our customers. 90,000+ eBooks of award-winning academic content in Humanities, Social Science, Science, Technology, Engineering, and Medical written by a global network of editors and authors.

TA Y L OR & FRANCIS EBOOKS OFFERS: A streamlined experience for our library customers

A single point of discovery for all of our eBook content

Improved search and discovery of content at both book and chapter level

REQUEST A FREE TRIAL [email protected]