Incorporating Source-Side Phrase Structures into Neural Machine Translation

Abstract

Neural Machine Translation (NMT) has shown great success as a fully Statistical Machine Translation (SMT) model in several languages. Early NMT models are based on sequence-to-sequence learning that encodes a sequence of source words into a vector space and generates another sequence of target words from the vector. In those NMT models, sentences are simply treated as sequences of words without any internal structure. In this article, we focus on the role of the syntactic structure of source sentences and propose a novel end-to-end syntactic NMT model, which we call a tree-to-sequence NMT model, extending a sequence-to-sequence model with the source-side phrase structure. Our proposed model has an attention mechanism that enables the decoder to generate a translated word while softly aligning it with phrases as well as words of the source sentence. Experimenting on Chinese-to-Japanese and English-to-Japanese translation tasks, we report different trends between the sequence-to-sequence and tree-to-sequence models. Experimental results on the English-to-Japanese dataset demonstrate that our proposed model considerably outperforms the sequence-to-sequence NMT models and compares favorably with the state-of-the-art tree-to-string SMT system.