Version 1
: Received: 5 March 2024 / Approved: 6 March 2024 / Online: 6 March 2024 (11:02:52 CET)
How to cite:
Xia, L.; Cai, J.; Huang, E.; Liu, J. Advancements in Word Sense Disambiguation: A Poly-Encoder Bert Model Perspective. Preprints2024, 2024030316. https://doi.org/10.20944/preprints202403.0316.v1
Xia, L.; Cai, J.; Huang, E.; Liu, J. Advancements in Word Sense Disambiguation: A Poly-Encoder Bert Model Perspective. Preprints 2024, 2024030316. https://doi.org/10.20944/preprints202403.0316.v1
Xia, L.; Cai, J.; Huang, E.; Liu, J. Advancements in Word Sense Disambiguation: A Poly-Encoder Bert Model Perspective. Preprints2024, 2024030316. https://doi.org/10.20944/preprints202403.0316.v1
APA Style
Xia, L., Cai, J., Huang, E., & Liu, J. (2024). Advancements in Word Sense Disambiguation: A Poly-Encoder Bert Model Perspective. Preprints. https://doi.org/10.20944/preprints202403.0316.v1
Chicago/Turabian Style
Xia, L., Enpei Huang and Junbang Liu. 2024 "Advancements in Word Sense Disambiguation: A Poly-Encoder Bert Model Perspective" Preprints. https://doi.org/10.20944/preprints202403.0316.v1
Abstract
In the application domain, accurate word sense identification is crucial for improving the performance of machine translation, information retrieval and end-to-end communication tasks. However, word polysemy is a major obstacle to accurate semantic identification. Therefore, word semantic disambiguation has always been one of the key challenges in natural language processing and has attracted the attention of a large number of researchers. This research proposes an innovative disambiguation algorithm based on the large-scale Bert model and the Polly encoder framework, and introduces WordNet as a benchmark for word semantic. By exploiting the ability of the pre-trained model to extract and learn semantic information, and using a specially designed forward propagation algorithm and loss function to fine-tune the large-scale Bert model, the model has high Accuracy and robustness. In this research, several experiments were conducted on the Semcor 3.0 semantic dataset. The experimental results show that the model proposed in this research shows excellent performance on the Semcor test set, with an Accuracy of 86.1% and an F1 score of 0.847, which is a significant improvement over the traditional model.
Keywords
NLP; Bert model; Semcor dataset; Transformer; Word semantic Disambiguation
Subject
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
