Gender Data Bias

Data plays an increasingly important role in our lives. Personal, business and even policy decisions are increasingly made by algorithms. And surely an algorithm that runs with almost no human involvement cannot be ‘biased’? Unfortunately, it can. Every algorithm is built on certain assumptions and depends heavily on them. The insights and ideas of the model’s designer ultimately have a major influence on the outcomes. This blog discusses one specific consequence: gender bias. What exactly is it? Gender bias in data is a bias (a distortion in the result caused by external factors) related specifically to gender. Many algorithms are based on data that does not represent the population adequately. For example, input from women is often lacking. This can lead to one-sided or biased results. But what if important decisions are made based on the output? Gender bias in data is visible in practice, but it can also be ‘hidden’ in areas such as machine learning.

Examples of gender bias in data in practice

• When women are involved in a car accident, they are 47% more likely than men to suffer a serious injury and 71% more likely to suffer a moderate injury. They are also 17% more likely to die in a car accident than men[2]. Why? On average, women sit further forward, are shorter and therefore need to sit more upright to see better. This happens because cars have been designed using crash-test dummies based on the average man. In 2011, female crash-test dummies were included in production for the first time in the United States.
• In 2014, Amazon introduced an algorithm to automatically select top talent from a large number of CVs. It seemed promising, but CVs containing the word “women’s” received lower ratings. Identical CVs that differed only in whether the applicant was a man or a woman were also rated differently. This was because the model had been trained on CVs from the technical sector dating back 10 years. As women were underrepresented in that sector, the training data consisted mainly of men’s CVs.
• In medicine, heart attacks are also recognised less quickly in women than in men, and medicines more often fail to work properly, simply because they have been tested on groups in which the majority are men[6].

The examples above have the same underlying cause. The algorithms’ training data is not representative of the population as a whole, with serious and sometimes even dangerous consequences in these cases.

Gender data bias in AI and data science

Gender bias can also occur in machine learning models, including self-learning algorithms that you might expect to be neutral. If the data fed into an algorithm already contains bias, that bias may not only be reflected in the model’s output; it may even be amplified. Below, we highlight several data science and/or AI techniques and methods where this occurs.

Image Recognition[4]

Image recognition models can learn from images without requiring specific prior knowledge.

Image recognition software, including software used by major providers, clearly shows stereotypes. Images of shopping and washing are associated with women; sports coaching and shooting with men. The software is also more likely to label men standing in kitchens as women. In addition, tools such as Google Cloud Vision assign markedly different labels to images of men and images of women.
Images of women are much more often labelled according to their appearance, while images of men are labelled according to professional characteristics. This is also shown in the image. Women are also less likely to be recognised in images than men.

Voice Recognition

Voice AI, or speech recognition, is being used increasingly often. Google has its own widely used model for this. Google has stated that its speech recognition is 95% accurate. However, that accuracy is not the same for every group. The tool is 13% more accurate for men than for women. It also works better for some dialects and accents than for others[5].

Assessing how well speech recognition works for each group would make it possible to introduce targeted improvements. This also starts with the data fed into the system: involving a diverse group gives a clearer picture of its accuracy.

Natural Language Processing and Word Embeddings

Gender bias can also occur in Natural Language Processing. One example is the use of word embeddings [7]. Word embeddings are trained on how often words occur together in text corpora. In one such model, trained on Google News data, the distance between the words ‘nurse’ and ‘woman’ is much smaller than the distance between ‘nurse’ and ‘man’. This indicates gender bias.

There are so-called ‘debias’ methods for machine learning models that reduce bias. For example, the word ‘nurse’ from the previous example can be moved closer to the midpoint between ‘man’ and ‘woman’. It is also often possible to reduce the amplification of bias caused by some machine learning models.  Still, prevention is better than cure here: the cause is already present in the data fed into the algorithm. What’s more, these debias methods can only be applied once the person designing the algorithm has noticed the bias. Only when you, as a data scientist, are aware of the bias in the data can you do something about it.

Gender bias is not the only kind of bias found in such algorithms. Every algorithm is strongly influenced by the assumptions behind it, so gender bias is just one illustration of why the data fed into an algorithm matters.

Conclusion

Gender bias in data therefore plays a significant role and clearly has real-world consequences. But the first steps have been taken. Addressing the problem starts with acknowledging it. Awareness appears to be growing, and the issue is becoming more visible. The ‘person’ behind the algorithm is therefore crucial to its outcome. So, more women in data science would certainly be a good start!

References

1. A. Linder & M. Y. Svensson (2019) Road safety: the average male as a norm in vehicle occupant crash safety assessment, Interdisciplinary Science Reviews, 44:2, 140-153, DOI: 10.1080/03080188.2019.1603870
2. https://www.theguardian.com/lifeandstyle/2019/feb/23/truth-world-built-for-men-car-crashes
3. https://www.bbc.com/news/technology-45809919
4. Schwemmer, C., Knight, C., Bello-Pardo, E. D., Oklobdzija, S., Schoonvelde, M., & Lockhart, J. W. (2020). Diagnosing gender bias in image recognition systems. Socius, 6, 2378023120967171
5. Tatman, R. (2017). Gender and Dialect Bias in Youtube;s Automatic Captions
6. Hamberg, K. (2008). Gender bias in medicine. Women’s health, 4(3), 237-243.
7. Bolukbasi, T., Chang, K. W., Zou, J. Y., Saligrama, V., & Kalai, A. T. (2016). Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29, 4349-4357.