We are a crowd of imperfect thinkers with a tendency to pick sides and choose favorites. Unfortunately, imperfect thinking often leads to an imperfect reality. The recently invigorated  Black Lives Matter movement has exposed many institutions and social structures pervaded by bias. As calls for reform swell, we must consider how our technology is influenced by bias too. In the context of artificial intelligence (AI), the important discussion becomes: how do human biases manifest in the AI we create and what can we do to fix it?

To begin answering this question, we turn to Detroit’s facial recognition program. On a January afternoon earlier this year, Detroit police were investigating the theft of five watches from a Shinola retail store. Detectives pulled grainy security footage of their suspect, ran it through the city’s facial recognition software, and the AI returned a hit: 42-year-old Robert Julian-Borchak Williams. Police promptly arrived at Williams’ home and handcuffed him in front of his distraught wife and two daughters. They were given no more explanation than the words “felony warrant” and “larceny.” At the station, Williams was led to an interrogation room and shown three photos: two from the surveillance camera and one of Williams’ driver’s license. Williams looked the photos over and shook his head incredulously. Holding the surveillance photos up to his own face, Williams scoffed, “I hope you don’t think all Black people look alike.” Williams was detained for 30 hours and then released on bail before charges against him were dropped as a result of insufficient evidence.

How do we know for certain that Williams was not the thief? The alleged suspect in the security camera footage was wearing a St. Louis Cardinals hat. Williams, a Detroit native, said he would “under no circumstances” rep Cardinals merchandise.

‍
There must be someone to blame for Williams’ false accusation. Initially, the Detroit Police Department (DPD) seems a likely target. But the DPD was only complicit in racism; the root of the bias stems from the facial recognition system itself. Somehow, the program did not produce correct output and misidentified Williams’ face as that of another Black man. If these errors were consistent across demographics, we could speculate that the AI was simply not sufficiently developed for use in law enforcement. Unfortunately, that’s not the case. A 2018 joint study between Microsoft and MIT’s Media Lab reported that across three commercial facial recognition systems, error rates were 10 to 20 percent greater for darker subjects than lighter subjects (positive predictive values for lighter male subjects were all above 99 percent). The numbers were worse for dark-skinned women, who were misidentified as men 31 percent of the time. Bias is characterized by this deliberate discrimination towards a population. It poses an imminent threat to AI’s integrity because bias often exists beyond physical code. As a result, you can know that an AI system is producing biased output without any semblance of why it’s doing so.

AI biases leave us at a crossroads: if we decide to trust AI and its outputs, we may end up reinforcing biases and unconsciously discriminating against marginalized populations; if we decide to not trust AI, we may be abandoning a technology with revolutionary potential. Both of these options are less than ideal. The better choice is to attack bias itself. To do so, it’s important to understand the mechanics of how biases can creep into AI systems in the first place.

AI systems learn to make decisions based on training data. Inputs are fed into the system, the AI returns an output, and then internally adjusts how information is connected and weighted according to the margin of error between input and output. Training data often comes from the material world — be it puppy photos, applicant resumes, or historical crime data — and, unfortunately, our world is rampant with bias. When training data contaminated with bias is given to an AI as input, the AI will reflect and intensify that bias in its output as it learns to minimize error. In the case of facial recognition software, there is disparity in accuracy among race and gender because light-skinned men comprise the largest fraction of image datasets, while dark-skinned women are the least photographed demographic. Robert Julian-Borchak Williams was falsely accused because there are not enough pictures of Black people among the largest commercial datasets.

AI can exploit biases based on what information it’s designed to consider and prioritize. Say a ride-share service wants to develop an AI model to predict how willing a passenger would be to pay a certain premium. To define this goal on a computational level, the company needs to specify whether they want to maximize profit margins or maximize the number of rides given. The AI, however, is designed to make decisions for business rather than discrimination or fairness. If the AI learned that giving out rides to only wealthy passengers willing to pay steep premiums was an effective way to maximize profit, it would ostracize lower-income riders, even though that may not have been the company’s intention. Biases can inadvertently arise when an AI latches onto sensitive or meaningless information to make decisions; it’s the AI programmer’s responsibility to prioritize the information that will produce equitable software.

No matter how biases surface in AI, the problem doesn’t lie with the technology itself. A microprocessor cannot be intrinsically biased; instead, the problem is human. AI is built and trained on human intuition, and if our conventions and institutions are biased, then the outputs of our AI will be too. Until we eliminate or fully recognize our biases, they will be perpetuated and magnified by the technology that we create. It may not be possible to have an unbiased human, so it may not be possible to build an unbiased AI, but we can certainly do better. As Rep. Alexandria Ocasio-Cortez put it: “If you don’t fix the bias, then you’re just automating the bias.”

The challenge of bias isn’t a reason to stop investing in AI or burden developers with regulations. It just requires attention and effort. I see three essential steps to diminishing AI biases:

First, there should be a standardized definition of what a bias-less AI looks like. One promising approach is “counterfactual fairness,” which holds that “a decision is fair towards an individual if it is the same in (a) the actual world and (b) a counterfactual world where the individual belonged to a different demographic group.” If a credit card company’s lending algorithm gives a subprime loan to a Black woman, a White man with the same credit score should get an identical loan; if not, we know that the algorithm is making decisions under the influence of bias.

Second, AI should be deployed using responsible business practices to mitigate bias. AI systems often fall into a “portability trap:” the model is designed to be used for different tasks under different circumstances, but in doing so ignores a lot of social context. Leaders must understand the social foundations that they are working with before thinking about software. Should a task exist in the first place? Does it matter who creates it? Who will deploy it and on which population? Helpful frameworks for recommended processes and technical tools include Google AI’s Responsible Practice Portfolio and IBM’s Fairness 360. The Alan Turing Institute’s Fairness, Transparency, and Privacy Group is another great resource to stay up to date on AI biases.

Third, and most importantly, we need to tackle human biases (easier said than done). In line with the ideals of the Black Lives Matter movement, we must engage in discussions about human biases, neutralize their effects, and counteract their prejudice. When we identify a bias buried within an AI, it’s not enough to just fix an algorithm; we must also fix the human biases underlying it. Diversifying the field of AI itself is a means to this end. AI researchers are quite homogeneous: primarily males, of particular racial demographics, without disabilities, who grew up in high socioeconomic areas. If AI creators come from diverse backgrounds and are acutely aware of discriminatory practices, AI systems will likely become less biased.

With great power comes great responsibility, and AI is no exception. AI biases are highly problematic. However, we are currently experiencing a time of global recognition and reform. Now, more than ever, we must not forget that our most powerful technologies are susceptible to our basic human weaknesses.