[PDF] Top 20 The Effect of Translationese in Machine Translation Test Sets

... the test data from WMT16, WMT17, and WMT18 news translation tasks (newstest2016, newstest2017, and newstest2018) exclusively, be- cause they provide results using the direct as- sessment (DA) score (Graham ... See full document

... The differences between O → T and T → O are often large, with up to 1.5 Bleu points difference for English–German. This is quite notable since the actual models in the SMT systems are identi- cal; the only difference is ... See full document

... However, translationese includes features involv- ing both source and target ...opposite effect), simplification – a tendency to simplify the language used in transla- tion, normalisation – a tendency to ... See full document

... 6.1.1 Type–Token Ratio. Translated texts have been shown to have lower type-to-token ratio (TTR) than original ones (Al-Shabab 1996). Figure 1 compares the TTR of the trans- lation outputs of S → T, T → S, UNION, and ... See full document

... capture translationese, ...different translation directions in English-German language ...in machine translation evalua- tion, and include manually-checked frequencies of less easily ... See full document

... WMT test suite calls focus on translation test suites, where the participants are asked to produce translations of the source sentence instead of scoring given hy- ...lation test suites ... See full document

... SAO Test Suite consists of 10 multi-language audit reports issued by the ...by translation agencies and finally corrected by the authorized auditors from the respective ...end effect of this careful ... See full document

... challenge sets. Such sets have the ad- vantage of being interpretable: they test for spe- cific phenomena that are important for humans and are crucial for language ...challenge sets were ... See full document

... and machine translation, there are a large and ever increasing number and we can distinguish two sub-areas of ...the translation process from those which use corpora to study the translation ... See full document

... and translation models domain- adaptation without explicit bilingual in- domain training ...source-language test corpus is combined with the correspond- ing hypotheses generated by the transla- tion system ... See full document

... Our end-to-end translation experiments were tuned and evaluated on sentences up to length 40 from the NIST MT04 and MT05 test sets. For these experiments, we trained on a 22.1 million word parallel ... See full document

... Our test suite focuses on allowing the comparison of end-to-end context- aware NMT systems, and we thus extract a large number of inter-sentential anaphora, with meta-data allowing for a focus on inter-sentential ... See full document

... on counts of numbers and punctuation, combined with contrastive BLEU score. This seems to rather overfit a peculiarity of the particular development set and indeed performs much lower on the final test set of the ... See full document

... The test is per- formed as follows. For each test English sentence y of Set4, the confusion grammar defines a full confusion set N (y) via a hypergraph ... See full document

... of translationese. We also believe that combining various feature sets is a key to improving the accuracy, and especially the robustness, of translationese ...feature sets; more work should be ... See full document

... neural machine translation system for any ...neural translation model when adapt- ing to new source languages, which makes its application to new languages extremely fast and ...translate test ... See full document

... based machine translation technique [1]. Rule Based Machine Translation provides high quality translation and requires in depth knowledge of the language apart from real world knowledge ... See full document

... Machine translation (MT), which is a field of NLP, can be employed for the translation of speech or text in a source language (SL) into the target language ...optimal translation devoid of any ... See full document

... In Machine Translation (MT) research, broadly-defined test suites have not been used apart from several singular at- tempts (King and Falkedal, 1990; Isahara, 1995; Koh et ... See full document

... Jan-Thorsten Peter, Hermann Ney, Ondˇrej Bojar, Ngoc-Quan Pham, Jan Niehues, Alex Waibel, Franck Burlot, François Yvon, M¯arcis Pinnis, Valters Sics, Joost Bastings, Miguel Rios, Wilker Aziz, Philip Williams, Frédéric ... See full document