Gender Data Bias

Data plays an increasingly important role in our lives. Personal, business and even policy decisions are increasingly made by algorithms. Surely an algorithm that runs with almost no human involvement cannot be ‘biased’? Unfortunately, it can. Every algorithm is built on certain assumptions and depends heavily on them. The insights and ideas of the model’s designer ultimately have a major influence on the outcomes. This blog looks at one specific consequence: gender bias. What exactly is it? Gender bias in data is a bias (a distortion in the result caused by external factors) related specifically to gender. Many algorithms are based on data that does not represent the relevant population. For example, there is often a lack of input from women. This can lead to one-sided or biased results. But what happens when important decisions are made based on those results? Gender bias in data is visible in practice, but it can also be ‘hidden’, for example in machine learning.

Examples of gender bias in data in practice

• When women are involved in a car crash, they are 47% more likely than men to suffer a serious injury and 71% more likely to suffer a moderate injury. A woman is also 17% more likely than a man to die in a car crash[2]. Why? On average, women sit further forward, are shorter, and therefore need to sit more upright to see clearly. This happens because cars have been designed using crash test dummies based on the average man. In 2011, female crash test dummies were used in production for the first time in the United States.
• In 2014, Amazon introduced an algorithm to automatically select top talent from a large number of CVs. It looked promising, but CVs containing the word “women’s” received lower scores. Identical CVs that differed only in whether the candidate was a man or a woman were also scored differently. The reason was that the model had been trained on CVs from the technical sector dating back 10 years. Because women were underrepresented in that sector, the training data consisted mainly of men’s CVs.
• In medicine, heart attacks are also less likely to be recognised in women than in men, and medicines are more likely not to work properly, simply because they have been tested on groups made up mostly of men[6].

The examples above share the same cause. The algorithms’ training data is not representative of the population as a whole, and in these cases that has serious, sometimes even dangerous, consequences.

Gender data bias in AI and Data Science

Gender bias can also occur in machine learning models, including self-learning algorithms that might be expected to be neutral. If bias is already present in the data fed into an algorithm, it may not only be reflected in the model’s output. It may even be amplified. Some of the techniques and methods within data science and/or AI where this occurs are highlighted below.

Image Recognition[4]

Image recognition models make it possible to learn from images without requiring specific knowledge.

Image recognition software, including software used by major providers, shows clear stereotypes. Images of shopping and doing laundry are linked to women; coaching sports and shooting are linked to men. Men in kitchens are also more often labelled as women by this software. In addition, when images of men and women are analysed, Google Cloud Vision, for example, assigns clearly different labels to each group.
Images of women are much more often labelled according to physical appearance, while images of men are labelled according to professional characteristics. This is also shown in the image. Women are also less likely to be recognised in images than men.

Voice Recognition

Voice AI, or speech recognition, is an increasingly common method. Google has its own widely used model for this. Google itself has said that its speech recognition is 95% accurate. However, this accuracy is not the same for every group. For men, the tool is 13% more accurate than it is for women. It also works better for some dialects and accents than for others[5].

Examining how well speech recognition works for each group would make it possible to implement targeted improvements. This starts with the data fed into it: involving a diverse group gives a clearer picture of its accuracy.

Natural Language Processing and Word Embeddings

Gender bias can also occur in Natural Language Processing, for example when using word embeddings [7]. Word embeddings are trained on the co-occurrence of words in text corpora. In one such model, trained on Google News data, the distance between the words ‘nurse’ and ‘woman’ is considerably smaller than the distance between ‘nurse’ and ‘man’. This indicates gender bias.

Machine learning models have so-called ‘debiasing’ methods that reduce bias. The word ‘nurse’ from the previous example, for instance, can be moved closer to the midpoint between ‘man’ and ‘woman’. It is also often possible to reduce the amplification of bias caused by some machine learning models. Yet in this case, prevention is better than cure; the cause is already present in the data fed into the algorithm. Moreover, these debiasing methods can only be applied once the person designing the model has noticed the bias. Only when you, as a data scientist, are aware of the bias in the data can you do something about it.

Gender bias is not the only bias found in such algorithms. Every algorithm is strongly influenced by its underlying assumptions, so gender bias is just one illustration of why the data fed into an algorithm matters.

Conclusion

Gender bias in data therefore plays a significant role and clearly has consequences in practice. But the first steps have been taken. Tackling the problem starts with recognising it. Awareness appears to be growing, and the issue is becoming more visible. The ‘person’ behind the algorithm is therefore crucial to its outcome. So, more women in data science is a good start!

References

1. A. Linder & M. Y. Svensson (2019) Road safety: the average male as a norm in vehicle occupant crash safety assessment, Interdisciplinary Science Reviews, 44:2, 140-153, DOI: 10.1080/03080188.2019.1603870
2. https://www.theguardian.com/lifeandstyle/2019/feb/23/truth-world-built-for-men-car-crashes
3. https://www.bbc.com/news/technology-45809919
4. Schwemmer, C., Knight, C., Bello-Pardo, E. D., Oklobdzija, S., Schoonvelde, M., & Lockhart, J. W. (2020). Diagnosing gender bias in image recognition systems. Socius, 6, 2378023120967171
5. Tatman, R. (2017). Gender and Dialect Bias in Youtube;s Automatic Captions
6. Hamberg, K. (2008). Gender bias in medicine. Women’s health, 4(3), 237-243.
7. Bolukbasi, T., Chang, K. W., Zou, J. Y., Saligrama, V., & Kalai, A. T. (2016). Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29, 4349-4357.