Preprint Article Version 1 Preserved in Portico This version is not peer-reviewed

A Survey on Bias in Deep NLP

Version 1 : Received: 1 March 2021 / Approved: 2 March 2021 / Online: 2 March 2021 (09:17:15 CET)

A peer-reviewed article of this Preprint also exists.

Garrido-Muñoz , I.; Montejo-Ráez , A.; Martínez-Santiago , F.; Ureña-López , L.A. A Survey on Bias in Deep NLP. Appl. Sci. 2021, 11, 3184. Garrido-Muñoz , I.; Montejo-Ráez , A.; Martínez-Santiago , F.; Ureña-López , L.A. A Survey on Bias in Deep NLP. Appl. Sci. 2021, 11, 3184.

Journal reference: Appl. Sci. 2021, 11, 3184
DOI: 10.3390/app11073184

Abstract

Deep neural networks are hegemonic approaches to many machine learning areas, including natural language processing (NLP). Thanks to the availability of large corpora collections and the capability of deep architectures to shape internal language mechanisms in self-supervised learning processes (also known as "pre-training"), versatile and performing models are released continuously for every new network design. But these networks, somehow, learn a probability distribution of words and relations across the training collection used, inheriting the potential flaws, inconsistencies and biases contained in such a collection. As pre-trained models have found to be very useful approaches to transfer learning, dealing with bias has become a relevant issue in this new scenario. We introduce bias in a formal way and explore how it has been treated in several networks, in terms of detection and correction. Also, available resources are identified and a strategy to deal with bias in deep NLP is proposed.

Keywords

natural language processing; deep learning; biased models

Subject

MATHEMATICS & COMPUTER SCIENCE, Artificial Intelligence & Robotics

Comments (0)

We encourage comments and feedback from a broad range of readers. See criteria for comments and our diversity statement.

Leave a public comment
Send a private comment to the author(s)
Views 0
Downloads 0
Comments 0
Metrics 0


×
Alerts
Notify me about updates to this article or when a peer-reviewed version is published.
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.