Version 1
: Received: 18 February 2020 / Approved: 19 February 2020 / Online: 19 February 2020 (10:51:41 CET)
Version 2
: Received: 29 February 2020 / Approved: 2 March 2020 / Online: 2 March 2020 (15:28:34 CET)
How to cite:
Pan, Y.; Li, X.; Yang, Y.; Dong, R. Dual-Source Transformer Model for Neural Machine Translation with Linguistic Knowledge. Preprints2020, 2020020273. https://doi.org/10.20944/preprints202002.0273.v1
Pan, Y.; Li, X.; Yang, Y.; Dong, R. Dual-Source Transformer Model for Neural Machine Translation with Linguistic Knowledge. Preprints 2020, 2020020273. https://doi.org/10.20944/preprints202002.0273.v1
Pan, Y.; Li, X.; Yang, Y.; Dong, R. Dual-Source Transformer Model for Neural Machine Translation with Linguistic Knowledge. Preprints2020, 2020020273. https://doi.org/10.20944/preprints202002.0273.v1
APA Style
Pan, Y., Li, X., Yang, Y., & Dong, R. (2020). Dual-Source Transformer Model for Neural Machine Translation with Linguistic Knowledge. Preprints. https://doi.org/10.20944/preprints202002.0273.v1
Chicago/Turabian Style
Pan, Y., Yating Yang and Rui Dong. 2020 "Dual-Source Transformer Model for Neural Machine Translation with Linguistic Knowledge" Preprints. https://doi.org/10.20944/preprints202002.0273.v1
Abstract
Incorporating source-side linguistic knowledge into the neural machine translation (NMT) model has recently achieved impressive performance on machine translation tasks. One popular method is to generalize the word embedding layer of the encoder to encode each word and its linguistic features. The other method is to change the architecture of the encoder to encode syntactic information. However, the former cannot explicitly balance the contribution from the word and its linguistic features. The latter cannot flexibly utilize various types of linguistic information. Focusing on the above issues, this paper proposes a novel NMT approach that models the words in parallel to the linguistic knowledge by using two separate encoders. Compared with the single encoder based NMT model, the proposed approach additionally employs the knowledge-based encoder to specially encode linguistic features. Moreover, it shares parameters across encoders to enhance the model representation ability of the source-side language. Extensive experiments show that the approach achieves significant improvements of up to 2.4 and 1.1 BLEU points on Turkish→English and English→Turkish machine translation tasks, respectively, which indicates that it is capable of better utilizing the external linguistic knowledge and effective improving the machine translation quality.
Keywords
linguistic knowledge; neural machine translation model; machine translation tasks; knowledge-based encoder; model representation ability
Subject
Computer Science and Mathematics, Artificial Intelligence and Machine Learning
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
