[PDF] Top 20 Improving Neural Machine Translation Models with Monolingual Data

... Neural Machine Translation (NMT) has obtained state-of-the art performance for several language pairs, while only us- ing parallel data for ...side monolingual data plays an ... See full document

... target-side monolingual data into NMT through multi-task ...lingual data from ...parallel data, we believe there is value in pursuing this line of re- search further to simplify training ... See full document

... We experimented on two corpora: (1) ASPEC (Nakazawa et al., 2016), using the top 100K sen- tence pairs for training En–Ja models and top 1M sentence pairs for training Ja–En models, and (2) LDC, 5 which ... See full document

... To explore the effect of other types of noise, we fine-tuned our baseline model on different external datasets (see the “Unconstrained" rows in Table 3 and 4). We experimented with human transcript and ... See full document

... Both our baseline NMT and fixnorm models suf- fer from the problem of shifted alignments noted by Koehn and Knowles (2017). As seen in Figure 2a and 2b, the alignments for those two systems seem to shift by one ... See full document

... In the second step, for each triplet from the ref- erence set we select n nearest neighbors. Candi- dates that have been chosen for one reference set triplet were excluded for the following triplets. If more than the 100 ... See full document

... only monolingual data to model inconsisten- cies between sentence-level ...document-level data is parallel, and translations are sampled from the source side of the sentences in a group rather than ... See full document

... abundant monolingual cor- pora to improve low-resource neural machine translation (NMT), the synthetic bilingual cor- pora generated by NMT models trained on limited authentic bilingual ... See full document

... IBM Models 1 to 5 and an HMM word alignment model to find statistically motivated alignments between ...all data and use all unique sentences from the Simple Wikipedia part of the PWKP training set to train ... See full document

... Table 3 shows the evaluation results of the baseline and the 3 regularization variants on the En⇒De dataset. Notice that we use the base model while Chen et al. (2018) and Ott et al. (2018) use big models. The FHR ... See full document

... chine Translation (SMT) systems is the dearth of high-quality bitext in the domain of ...adaptation data selection: the idea is to use language models (LMs) trained on in-domain text to select ... See full document

... language models are trained on the English side of the parallel corpus, and on monolingual corpora (around 10 billion words from Gigaword (LDC2011T07)), ... See full document

... of translation in the context of SMT. They found that a translation model based on the S → T portion of the parallel corpus results in much better translation quality than a translation model ... See full document

... Another approach of using monolingual corpora of the target language is to learn models using syn- thetic parallel sentences. The method of Sennrich et al. (2016a) generates synthetic parallel corpora ... See full document

... noise data. In our setup, we trained Zipporah models for both language pairs Sinhala–English and ...(probabilistic translation dic- tionaries and language models) were trained on the provided ... See full document

... training data for training an NMT ...parallel data, which then guides the train- ing of a final student model trained on the com- bination of clean and noisy ...Chinese-English translation, ... See full document

... source-side monolingual data in con- ventional SMT with a self-learning ...more translation rules in SMT and we also adapt a multi-task learning framework to take full advantage of the source-side ... See full document

... To recap our answers to our initial questions: the quality of BT actually matters for NMT (cf. § 3.1) and it seems that, even though artificial source are lexically less diverse and syntactically complex than real ... See full document

... a data-hungry approach, requiring a large amount of parallel data to reach reasonable per- formance (Koehn and Knowles, ...parallel data, ...obtain monolingual data in either the source ... See full document

... pivot-based machine translation, text is first translated from the source language into the pivot language, and then from the pivot language into the target ...strong translation performance (Johnson ... See full document