Book Review: Weapons of Math Destruction - How Big Data Increases Inequality and Threatens Democracy
Premise
How faulty and self-fulfilling prediction models can have profound, long-lasting and devastating effects on society. Cathy O’Neil, a data scientist with experience in a wide range of industries, from banking to advertising startups, takes us on a deep dive into industries that use poor machine learning models that, willfully or not, have an incredibly negative impact on people’s lives.
My thoughts
Having worked in tech for over a decade I always assumed that “algorithms rule our lives”.. I just never quite appreciated to what extent this was true. The education system, justice system, banking system, employment, healthcare, elections.. the list goes on, and it's overwhelming. The book is thoroughly researched, each chapter encompassing a different industry. I really enjoyed this book, and it certainly makes me think twice about a lot of things I took for granted.
Favorite highlights
- In WMDs (weapons of math destruction), many poisonous assumptions are camouflaged by math and go largely untested and unquestioned.
- The human victims of WMDs, we’ll see time and again, are held to a far higher standard of evidence than the algorithms themselves.
- Big Data processes codify the past. They do not invent the future. Doing that requires moral imagination, and that’s something only humans can provide. We have to explicitly embed better values into our algorithms, creating Big Data models that follow our ethical lead. Sometimes that will mean putting fairness ahead of profit.
- Our national motto, E Pluribus Unum, means “Out of Many, One.” But WMDs reverse the equation. Working in darkness, they carve one into many, while hiding us from the harms they inflict upon our neighbors near and far.
- So the first step is to get a grip on our techno-utopia, that unbounded and unwarranted hope in what algorithms and technology can accomplish. Before asking them to do better, we have to admit they can’t do everything.