[PDF] Top 20 Tensor2Tensor for Neural Machine Translation

... convolutional neural machine translation without this bottleneck was first achieved in Kaiser and Bengio (2016) and Kalchbrenner et ...(Extended Neural GPU) used a recurrent stack of gated ... See full document

... Neural machine translation (NMT) has emerged recently as a promising statis- tical machine translation ...NMT, neural networks (NN) are directly used to produce translations, ... See full document

... sampled translation from the teacher model, out from the source side sentences, as the bilin- gual training ...a neural network is less noisy and more ... See full document

... three neural machine translation (NMT)- based models (LSTM, CNN, and transformer) and a statistical machine translation (SMT)- based model is evaluated against six learner corpora ... See full document

... In our experiments, we used up to six encoder-decoder NMT models (three pairs); English→French, French→English, English→Czech, Czech→English, English→Ger- man, German→English. All systems were trained on the available ... See full document

... A translation memory (TM) is proved to be helpful to improve neural machine translation (NMT). Existing ap- proaches either pursue the decoding efficiency by merely ac- cessing local ... See full document

... (Koehn et al., 2003), NMT has trouble with low- frequency words or phrases (Arthur et al., 2016; Kaiser et al., 2017), and also generalizing across domains (Koehn and Knowles, 2017). A num- ber of methods have been ... See full document

... Finally, we perform a full comparison between the various methods for integrating lexicons into the translation process, with results shown in Table 4. In general the bias method improves accuracy for the auto and ... See full document

... The network architectures of NMT models are simple but effective. It produces a sentence represen- tation with the encoder, then the decoder generates the translation from the vector of sentence repre- sentation. ... See full document

... layers was set to 512, and that of the inner feed- forward neural network layer was set to 2048. The heads of all multi-head modules were set to eight in both encoder and decoder layers. In each training batch, a ... See full document

... Datasets We conduct experiments to evaluate the effectiveness of our proposed method on two widely adopted benchmark datasets: the WMT14 1 English → German translation (En → De) and the WMT14 English→French ... See full document

... That the margin between the SDEC and SENT models is not large was to be expected for two reasons. First, Chung et al. (2015) and Fraccaro et al. (2016) have shown that stochastic RNNs lead to enormous improvements in ... See full document

... fixed-length vector and same is decoded to gen- erate the target sentence (Cho et al., 2014). The simple RNN adopted Long Short Term Memory (LSTM), which is a gated RNN used to improve the translation quality of ... See full document

... Mart´ın Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Cor- rado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael ... See full document

... a single source sentence (Tu et al., 2018). Re- cently, there have been few attempts to model the semantic information across sentences. The basic ideas are to store a handful of previous source or target sentences with ... See full document

... Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexan- dra Constantin, and Evan ... See full document

... In this context, it was also claimed that PE was killing translators’ creativity. The word ‘cre- ativity’ seemed to be used here in the sense of being able to produce a new text, a process ac- companied by attention, ... See full document

... single machine regardless of the number of GPUs or amount of available memory; one simply iter- ates over multiple batches and accumulates the re- sulting gradients before committing a weight up- ... See full document

... The method we proposed focuses on training process only. There is no restriction for the neural network structure. It can be used in any data paral- lelism framework and then distributed onto multi- GPUs. Also, ... See full document

... statistical machine transla- tion (SMT) that typically has a huge phrase/rule ta- ...and machine transla- tion community (Kalchbrenner and Blunsom, 2013; Cho et ...a neural decoder gen- erates the ... See full document