Sara Wachter-Boettcher, Technically Wrong: Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech (Book Review)
Well . . . This is another one of those funny books that is sort of a “5” and sort of a “3.” The book broadly claims that the tech industry builds interfaces and products that are (not necessarily intentionally) biased. The book says that the main driver is the homogeneity of tech company investors and employees.
There is no doubt in my mind that this is true, and on that basis, I’d recommend this to anyone in or outside of tech. We product builders and designers are doing a crap job of acknowledging the incredibly broad types of people and styles of interaction out there. Because of tech’s homogeneity, there’s so much stuff that just isn’t thought about critically (e.g., image analysis software not being able to analyze non-white faces). But as I’ll get into it in a moment, I would very strongly recommend this to historians of technology as a little guide to problems that deserve significantly more research. The author’s a web consultant, but I think we need to bring out the scholars. There’s good stuff here about geography, the 2010s -- more reports of personal experiences would make the story even more valuable. (I keep thinking about to another book I reviewed: Turco’s The Conversational Firm, which shows how far we can get with ethnographical strategies.)
There are some arguments here that are very dear to my heart. For example, on p. 137 and chapter 3, the author notes how engineers and product designers will focus on the main experimental flow, and minimize the importance of “edge cases.” For example, say 80% of the users are young, and only 20% are old (perhaps needing bigger fonts). Well, the company is going to focus on where the money is: So font-changing features may be downplayed. The author rightly stresses harm and consequences: Even though the 20% might not be where the money is, the negative consequences of not helping them out with a useful UI can cause a lot of damage. One area I have been concerned about is privacy and security in healthcare. Say a login code is sent to an email: But that email might go to a shared account. For the most part, this is probably not troubling: The user “opted in,” supplying that email. But should we work harder to ensure that only the individual can access that account? What if it’s a shared account and medical details about domestic violence make it to that address. Again, say the patient has signed a consent to allow that message to go via email to a particular address. Should that minority example make us very concerned to protect the “minority” user pattern? I think so. The book does a good job walking the reader through this.
But I have some concerns:
Geography: Time and time again, the examples lean towards west coast companies: Uber, Facebook, Twitter, etc. There are some exceptions. But I’d like to know: If the California tech culture is so bad, are there other places that are better?
Timespan: Is this a particularly bad moment? Wachter-Boeettcher provides the appalling facts around the decline of women computer science majors (37% in 1984, 18% in 2014). “I can’t pretend to know the precise reason for this shift” (p. 182). Me neither. But this book is so anchored in the present, it begs the question of how we would assess, say, the tech culture of the 80s. I bet it was better. But was it? Just as an example, back in the day, Ann Wollrath was the lead author on the original RMI article. Big stuff. What was the culture? It would mean a lot if Wollrath told us that it was the same back then. Then we might understand the core problem as a more broader ill.
Intentions: There are some good anecdotes here about how female voices are used for Siri, Alexa, and Google Maps (etc.) (pp. 36-38). Right. But what conclusion should we draw? “Women are expected to be more helpful than men . . . The more we rely on digital tools in everyday life, the more we bolster the message that women are society’s ‘helpers’” (p. 38). I get this. But then the author says: “Did the designers intend this? Probably not.” I protest! Go out and interview the designers! What were their reasons? Apple, in particular, thinks hard about this stuff. What were the factors going into a female Siri, and how did they outweigh providing other Siris (male; accented; whatever)? I want to know. The book makes an insinuation, but I think there’s a real research task to be performed. Bring out the ethnographers.
In large part, the book is driven by articles in the tech media. The next step is to get out there and start quoting people on their individual experiences, in order to test some claims (e.g., is the problem peculiarly tech in California in the 2010s? Or is it men in tech (more geography would help)? Or even a side-effect of the investment structure and capitalism (seems implicit in chapter 9) -- and, in particular, figure out where people are doing it right, and why. [The one positive example given in the book is Slack, but I’m not going to give much quarter there: Slack was produced by advertising and exploring the corporate customer’s desire to control discourse in the company, not by inclusiveness.]
Originally reviewed on Goodreads on 7 January 2018.
Comments