[PDF] Top 20 Exploiting Deep Representations for Neural Machine Translation

... Deep representations have proven to be of pro- found value in machine translation (Meng et al., 2016; Zhou et al., 2016). Multiple-layer encoder and decoder are employed to perform the ... See full document

... Deep neural models have recently achieved a great success in a wide range of ...building deep CNNs, some promising improvements have also been achieved on building deep NMT ...their ... See full document

... word units are likely to represent the word mor- phemes. The words are segmented into sub-word units, which are to some extent close to the lin- guistic word stems and suffixes. For example, the word “adventurer” is ... See full document

... in neural machine translation ...hidden representations of a well-trained end-to-end dependency parser, which are re- ferred to as syntax-aware word representations ... See full document

... distributed representations of words, often called word em- beddings, in almost all NLP ...recursive neural networks, ...convolutional neural networks, ... See full document

... in exploiting abundant monolingual cor- pora to improve low-resource neural machine translation (NMT), the synthetic bilingual cor- pora generated by NMT models trained on limited authentic ... See full document

... We apply GCNs to the semantic dependency graphs and experiment on the English–German language pair (WMT16). We observe an im- provement over the semantics-agnostic baseline (a BiRNN encoder; 23.3 vs 24.5 BLEU). As we use ... See full document

... Gulcehre et al. (2015) first investigate the target- side monolingual data in NMT. They propose shal- low and deep fusion methods to enhance the decoder network by training a big language model on target- side ... See full document

... use deep learning models and is recommended by several universities, even in non-English-speaking countries (Bousquet, ...translating Deep Learning by using deep learning ...French translation ... See full document

... In this work, we introduce a new type of lin- ear connections for multi-layer recurrent networks. These connections, which are called fast-forward connections, play an essential role in building a deep topology ... See full document

... fast deep-learning inference is an is- sue that deserves dedicated tools that are not com- promised by competing objectives such as training or support for multiple ... See full document

... model representations have been successful in a wide range of lan- guage understanding ...trained representations into sequence to se- quence models and apply it to neural ma- chine ... See full document

... As for the resource-rich En-XX data, we sep- arately used two parallel corpora with two dif- ferent target languages. One is the IWSLT 2015 English–Chinese corpus (IWSLT En-Zh) (Cettolo et al., 2015), comprising 209,491 ... See full document

... Neural machine translation has significantly pushed forward the quality of the ...ness. Neural models are trained on large text corpora which contain biases and ...in neural ma- chine ... See full document

... We analyze the impact of three design de- cisions in multi-task learning: the tasks used in training, the training schedule, and the degree of parameter sharing across the tasks, which is defined by the network ar- ... See full document

... the deep transition RNN to NMT. As shown in Figure 1, in a deep transition RNN, the next state is computed by the sequential application of mul- tiple transition layers at each time step, effectively using ... See full document

... In this paper, we propose a multi-source NMT approach for the low-resource NMT to explicitly utilize the source-side linguistic knowledge, which models the word sequence in parallel to the linguistic features by using ... See full document

... Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, ... See full document

... Since BT is widely acknowledged and effective to improve the NMT model, there has been several works investigating back translation from differ- ent views. Poncelas et al. (2018) study on how using the back ... See full document

... The two language pairs react differently to k- best hypotheses rescoring (+k-best rescored in the tables). For Czech, this has nearly no impact on translation quality according to the metrics, whereas it provides ... See full document