IS ARTIFICIAL INTELLIGENCE BIASED?

While machines are technically without bias, time and time again, we see cases of racism and sexism, to name a few. Veronica Stivala speaks to AI expert Dr Charlie Abela to find out whether AI can be prejudiced.

---

Entrepreneur Tony Arcieri recently tried out a “horrible experiment”. When he posted a picture that included US Senator Mitch McConnell and Barack Obama on Twitter, he discovered that the image would be consistently cropped to hide the former president. This led to the unearthing of an algorithm that was automatically focusing on white faces over black ones.

While Twitter has since apologised for this “racist” image cropping algorithm, this is just an example of a series of biases Artificial Intelligence (AI) has been exhibiting, often to the surprise of the programmers themselves. How so?

Let’s go back a bit. Machine learning is a subfield of Artificial Intelligence and is the science of getting applications to learn and improve without being explicitly programmed to do so. A machine learning algorithm is a trained program by feeding it massive amounts of data for it to find or recognise patterns in unseen data. These algorithms are suitable when dealing with uncertainty in different contexts. For instance, they can be used to classify images according to human subjects, or translate historical manuscripts or identify whether two drugs can result in an adverse reaction or not if taken together. Such problems do not necessarily have a binary solution of the type, yes or no, but involve an element of uncertainty that cannot be addressed using traditional deterministic algorithms. 

Dr Charlie Abela, a lecturer in AI within the Department of Artificial Intelligence at the University of Malta, explains this to me as we discuss how and why machines can be biased. This latter phrase sounds contradictory. How can a machine be biased? Indeed, Dr Abela states that “a machine learning algorithm is theoretically neutral and without prejudice”. Nevertheless, he goes on. There have been situations where an algorithm unintendedly produced results that were consistently in favour or against an individual, a group, or specific characteristics that are considered unfair, such as age, gender, race, disability or ethnicity. 

For instance, in a study published in Science in 2019, researchers found that the algorithm used in a US health facility to identify and support patients with complex needs was racially biased. The researchers discovered that the data used to train the algorithm was incomplete and that the algorithm was predicting healthcare costs rather than illness. This penalised black patients since, in general, less money is spent on care for these patients due to unequal access to care facilities. 

Bias has also been found in algorithms that drive Amazon’s recruiting system that consistently favoured men over women and Facebook’s advertisement algorithm, which was found to discriminate on which users see which ads. In all these cases, it boils down to the fact that, in Dr Abela’s words, “algorithms are as smart as the data that they are trained on and the values held by the individuals that create and use them”.

Now, while we talk about AI in general, there are different classifications of AI – so are some more prone to being biased than others? The short answer is yes. One of the most common classifications is the technology-oriented approach that classifies AI under narrow (or weak), general and superintelligence. The narrow AI is the type we presently have. It is goal-oriented and designed to undertake one specialised task, for instance, speech and language recognition (used in Apple’s Siri), diagnosing cancer or driving a car. 

Artificial General Intelligence (AGI), also referred to as strong AI, and Artificial Super Intelligence (ASI) are still far from being realised. Still, these types will be able to understand human intelligence and have cognitive abilities that exceed those of humans. “That narrow AI is liable to prejudice is known,” asserts Dr Abela, who goes on to explain how “this predominantly depends on the criteria humans apply in the development of the algorithms and their training process”. Yet, while the realisation of AGI may remain in the domain of speculation for years to come, Dr Abela points out that “the extensive and important work being done to address the challenges underlying the fundamental behaviours and limits of today’s AI, including prejudice, can provide insights that can eventually shape the AI of the future”.

Let’s look further into how AI learns. One of the issues driving this prejudice is that AI can only work with what it has. So, if, for instance, it is learning from more images of women than men in the kitchen, then it will go on to assume that it is mainly women who are to be found in the kitchen. Dr Abela explains further: studies have shown that banks of research image collections, including some supported by important companies, included gender bias in their representation of activities associated with cooking and sports. Women featured more in images related to shopping and washing, while men were depicted more in images related to coaching and shooting. Such popular image collections are used by many researchers testing or validating their research approach, and thus potentially bias is introduced in their results by proxy. The data collected and pre-processed must reflect as much as possible the particular domain while at the same time is weeded from discriminative features. 

How to solve it? Scientists are investigating ways to cure automated systems of hidden biases and prejudice. The approaches are varied, yet what the majority do is to try to minimise the unethical effect of attributes such as race, colour, sex, disability and age, to name a few. Dr Abela cites a recent research study related to autonomous cars where the focus was on minimising identification errors related to pedestrians with darker skin. In this case, the goal was to maximise the importance of some of these protected attributes. “It is therefore important, that when developing or training a machine learning algorithm, one is aware of the possible bias and how this can influence the model in a particular context, as this can result in a serious flaw or a potential benefit,” he says.

Machine learning algorithms require data to learn. And it is data, rather than the algorithm itself, that is the leading cause of bias. So, in Dr Abela’s words, “training algorithms using biased data will, of course, affect the model”. While large corporations have made available large-scale datasets that are annotated to facilitate the training of deep learning-based models, the annotated datasets have still been found to be biased. This is an inescapable part of human nature that is influenced by the individual’s environment and experiences. “The training data may contain human subjectivity, or it may reflect secondary effects emanating from historical or societal inequities”. 

He goes on to add how the language used in news articles may, for instance, include word embeddings that exhibit gender stereotypes present in society. In 2018 Reuters reported that Amazon’s recruiting tool penalised applicants’ resumes that included the word ‘women’. The company realised that the machine learning algorithm was trained using a dataset of resumes collected over a 10-year period predominately coming from male applicants. Thus, female applicants were underrepresented in the dataset, and this intrinsically resulted in the algorithm preferring male candidates. 

The million-dollar question remains: can AI be taught to overcome prejudice? If, at some point, researchers manage to achieve AGI (or ASI), will this AI be fair, just and removed from prejudice? “I don’t have an answer for this, and I would be speculating if I said that this would be the case,” confides Dr Abela. He notes how AI today is already playing a role in shaping our lives: our health, safety, education, and opportunities. “It is crucial that we can monitor and assess whether or not these systems treat people differently. 

The advent of deep learning brought a growing need to understand what is happening under the hood and how the algorithm arrived at the result. This, in turn, increased the need to build models worthy of human trust using techniques through which the reasoning behind these models is explained.” To improve and evolve AI into one that is critical and fair, then “explainability complimented with an ethical framework is key to enhance transparency and to provide insights”.

---

FEATURED IN ISSUE #2

NOV ‘20