[PDF] Top 20 Neural Machine Translation with Supervised Attention

... the attention with GIZA++. With the help from GIZA++, our supervised attention based NMT (SA-NMT) significantly reduces the AER, compared with the unsupervised counterpart ... See full document

... In this paper, we alleviate the above issue by uti- lizing the alignments (human annotated data or ma- chine alignments) of the training set. Given the alignments of all the training sentence pairs, we add an alignment ... See full document

... IBM Models (Brown et al., 1993) depict- ed the word reordering knowledge as position- al relations between source and target word- s. Koehn et al. (2003) proposed a distortion model for phrase-based SMT based on jump ... See full document

... several machine learning ...using machine learning ...a translation memory tool which worked as a sub- system in their transfer-based MT ...their machine learning model which classifies the ... See full document

... of machine translation, pairing a recurrent neural net- work grammar encoder with a novel atten- tional RNNG decoder and applying pol- icy gradient reinforcement learning to in- duce unsupervised ... See full document

... our supervised neural machine translation (NMT) systems that we developed for the news translation task for Kazakh↔English, Gujarati↔English, Chinese↔English, and English→Finnish ... See full document

... Table 1 lists the speeds and performances of the proposed models. Clearly the proposed approach improves the translation quality in all cases, although there are still considerable differences among the proposed ... See full document

... Sparse Attention Sparse attention and its con- strained variants have been used to address the coverage problem in NMT (Malaviya et ...of attention that each source word can ...sparse ... See full document

... global attention approach gives a significant boost of ...local attention model with predictive alignments (row local-p) proves to be even better, giving us a further improve- ment of ... See full document

... based neural ma- chine translation models (Bahdanau et ...chine translation tasks. The soft-attention mech- anism computes the context (encoder-decoder at- tention) vector for each target ... See full document

... Beam search is known to improve quality of NMT translation output. However, it is also known that larger beam size does not always helps but rather hurts the quality (Tu et al., 2016a). There- fore it is important ... See full document

... We have presented a neural attention-based model for abstractive summarization, based on recent de- velopments in neural machine translation. We combine this probabilistic model with a ... See full document

... This is the second time for SRCB to participate in WAT. This paper describes the neural machine translation systems for the shared translation tasks of WAT 2019. We participated in ASPEC tasks ... See full document

... multi-hop attention mechanism, although Haddow et ...multi-hop attention mecha- nism outperformed the baseline RNN model in the evaluation of the language pairs of CS-EN, EN-CS, ET-EN, EN-ET, FI- EN, and ... See full document

... multi-head attention to learn different aspects of feature through different at- tention ...German-English translation tasks demonstrate that our proposed model outperforms the Transformer baseline as well ... See full document

... the machine translation literature, there are two major streams for integrating visual information: approaches that (1) employ separate attention for different (text and vision) modalities, and (2) ... See full document

... of attention mechanism in rationalizing a model’s predictions in NLP (Jain and Wallace, 2019; Serrano and Smith, 2019) and they all target text classification tasks where atten- tion is over the input ...the ... See full document

... Neural machine translation (NMT) has gained a lot of attention recently due to its substantial im- provements in machine translation quality achiev- ing state-of-the-art ... See full document

... the attention history of each source ...the attention history. The attention of the current target word is determined based on the concatenated representa- ...attentive neural mod- els, which ... See full document

... different attention transformations for a toy example with three source ...the attention values on the probability ...cumulative attention each word has ...cumulative attention for the three ... See full document