Preprint Article Version 1 Preserved in Portico This version is not peer-reviewed

Dual-Source Transformer Model for Neural Machine Translation with Linguistic Knowledge

Version 1 : Received: 18 February 2020 / Approved: 19 February 2020 / Online: 19 February 2020 (10:51:41 CET)
Version 2 : Received: 29 February 2020 / Approved: 2 March 2020 / Online: 2 March 2020 (15:28:34 CET)

How to cite: Pan, Y.; Li, X.; Yang, Y.; Dong, R. Dual-Source Transformer Model for Neural Machine Translation with Linguistic Knowledge. Preprints 2020, 2020020273 (doi: 10.20944/preprints202002.0273.v1). Pan, Y.; Li, X.; Yang, Y.; Dong, R. Dual-Source Transformer Model for Neural Machine Translation with Linguistic Knowledge. Preprints 2020, 2020020273 (doi: 10.20944/preprints202002.0273.v1).

Abstract

Incorporating source-side linguistic knowledge into the neural machine translation (NMT) model has recently achieved impressive performance on machine translation tasks. One popular method is to generalize the word embedding layer of the encoder to encode each word and its linguistic features. The other method is to change the architecture of the encoder to encode syntactic information. However, the former cannot explicitly balance the contribution from the word and its linguistic features. The latter cannot flexibly utilize various types of linguistic information. Focusing on the above issues, this paper proposes a novel NMT approach that models the words in parallel to the linguistic knowledge by using two separate encoders. Compared with the single encoder based NMT model, the proposed approach additionally employs the knowledge-based encoder to specially encode linguistic features. Moreover, it shares parameters across encoders to enhance the model representation ability of the source-side language. Extensive experiments show that the approach achieves significant improvements of up to 2.4 and 1.1 BLEU points on Turkish→English and English→Turkish machine translation tasks, respectively, which indicates that it is capable of better utilizing the external linguistic knowledge and effective improving the machine translation quality.

Subject Areas

linguistic knowledge; neural machine translation model; machine translation tasks; knowledge-based encoder; model representation ability

Comments (0)

We encourage comments and feedback from a broad range of readers. See criteria for comments and our diversity statement.

Leave a public comment
Send a private comment to the author(s)
Views 0
Downloads 0
Comments 0
Metrics 0


×
Alerts
Notify me about updates to this article or when a peer-reviewed version is published.
We use cookies on our website to ensure you get the best experience.
Read more about our cookies here.