[PDF] Top 20 Unsupervised Neural Machine Translation with Weight Sharing

... Unsupervised neural machine translation (NMT) is a recently proposed approach for machine translation which aims to train the model without using any labeled da- ... See full document

... Recently, several UBWE methods (Conneau et al., 2018; Artetxe et al., 2018a) have been applied to UNMT (Artetxe et al., 2018c; Lample et al., 2018a). These rely solely on monolingual corpora in each language via UBWE ... See full document

... The unsupervised parses are trained over sub- words; if the induced hierarchies have a linguis- tic basis, we would expect the model to combine subwords into words as a first ... See full document

... WMT19 unsupervised news translation ...the unsupervised translation direction: ...our unsupervised neural and statistical machine translation ...German-to-Czech ... See full document

... Baselines. We compare our methods with two baseline models: 1) The copied monolingual data model (Currey et al., 2017) which copies tar- get in-domain monolingual data to the source side; 2) Back-translation ... See full document

... As discussed in Section 2, phrase-based SMT follows a modular architecture that combines several scoring functions through a log-linear model. Among the scoring functions found in standard SMT systems, the distortion ... See full document

... recurrent neural network decodes the source hidden sequence into the target ...a translation requires sam- pling at each step the most probable target word from the distribution and then feeding it back to ... See full document

... (Zhao et al., 2004) tackle LM adaptation for SMT. Similarly to our work, they use automati- cally generated hypotheses to perform adaptation. We extend their work by using the hypotheses also for TM adaptation. ... See full document

... One main drawback of this approach is that the whole source sentence has to be stored in a fixed- size context vector. To overcome this problem, (Bahdanau et al., 2014) introduced the soft attention mechanism. Instead of ... See full document

... Often monolingual data is used to further im- prove the performance of a Neural Machine Trans- lation model (Sennrich et al., 2016a; Currey et al., 2017). Backtranslation is one such popu- lar method in ... See full document

... the translation qual- ity that we achieved with the phrase-based un- supervised approach ...the unsupervised “wbw” experiment from Table ...word-by-word translation with- out a language model (LM) or ... See full document

... We used 1 million sentence pairs that are con- tained in train-1.txt of ASPEC-JE corpus for training the translation rule tables and NMT models. We also used 3 million Japanese sen- tences that are contained in ... See full document

... We prune the SCFG rules (corresponding to hyperedge) that are already discovered from syn- chronous hypergraphs using two constraints. Firstly, we prune a hypergraph by viterbi pruning with p = 1 (Huang, 2008). Secondly, ... See full document

... of neural networks follows two main strategies: (i) model parallel evalu- ates different model layers on different work- ers (Coates et ...ral machine translation systems have been recently trained ... See full document

... While machine translation has traditionally re- lied on large amounts of parallel corpora, a re- cent research line has managed to train both Neural Machine Translation (NMT) and Sta- ... See full document

... In this paper, we alleviate the local optimality of back-translation by learning a policy (takes the form of an encoder-decoder and is defined by its parameters) with future rewarding under the reinforcement ... See full document

... First, they carefully initialize the MT system with an inferred bilingual dictionary. Second, they leverage strong language models, via train- ing the sequence-to-sequence system (Sutskever et al., 2014; Bahdanau et al., ... See full document

... improve translation quality for low-resource lan- guage pairs, we included the en→de pair for a detailed analysis on the impact of using partial translations in addition to much more training ...ficult ... See full document

... Elastic Weight Consolidation (Kirkpatrick et al., 2017) is a simple, statistically motivated method for selective regularization of neural network pa- rameters. It was proposed to counteract catas- trophic ... See full document

... word translation tables inferred from cross-lingual embeddings according to the approach in ...frequent translation patterns, and then generate denoised pseudo data to guide the subse- quent NMT ... See full document