[PDF] Top 20 Re evaluating the Role of Bleu in Machine Translation Research

... that Bleu is insufficient by showing that Bleu admits a huge amount of variation for identically scored ...hypothesis translation that receive the same Bleu ...same Bleu score but a ... See full document

... semantic machine translation evaluation that we are ...of machine translation output by assessing the accuracy for key semantic ...semantic role labeling into statistical machine ... See full document

... We present the first challenge set and eval- uation protocol for the analysis of gender bias in machine translation (MT). Our ap- proach uses two recent coreference resolution datasets composed of English ... See full document

... evaluation re- veals for the first time which metric vari- ants significantly outperform others, op- timal metric variants distinct from cur- rent recommended best variants, as well as machine ... See full document

... It is nowadays unanimously recognized that Ma- chine Translation (MT) engines based on the neu- ral encoder-decoder architecture with attention (Cho et al., 2014; Bahdanau et al., 2014) constitute the new ... See full document

... We have presented an evaluation of discourse- level NMT models through the use of two dis- course test sets targeted at coreference and lexical coherence/cohesion. We have shown that multi- encoder architectures alone ... See full document

... than translation, has not (yet) been established as a major research ...MT research on translation of newspaper texts, which does not readily lend itself to task- based ...MT research ... See full document

... instant translation has also increased, thus the amount of sites that offer this kind of service have popped up all over the ...on research and also share and distribute more linguistic resources with each ... See full document

... include translation evaluations, crowdsource evaluations, and crowdsource ...now, translation evaluations are performed by ...NIST, BLEU, and METEOR (Doddington, 2002) (Baner- jee and Lavie, 2005) ... See full document

... for Machine Translation for low resource ...gual Machine Translation are two important areas of research that tackles this ...significant BLEU im- ... See full document

... The advent of neural techniques in machine trans- lation (MT) (Kalchbrenner and Blunsom, 2013; Cho et al., 2014; Sutskever et al., 2014) has led to profound improvements in MT quality. For “easy” language pairs ... See full document

... of machine translation (MT) weigh many aspects of translation, including ade- quacy, fidelity , and fluency of the translation (Hovy, 1999; White and O’Connell, ...of machine trans- ... See full document

... Given that many small changes have been made to the original BLEU design, our replica- tion would not produce the same scores from the current version of BLEU. Nevertheless, the in- verted behavior was ... See full document

... Decision trees are merely one form of classifier that could be used for the automated evaluation of an MT system. In preliminary experiments, the accuracy of classifiers using support vector machines (SVMs) (Vapnik, ... See full document

... It is worth noting that there are other auto- matic MT evaluation metrics that depend on the same precision-based score with primary differ- ences in how the Count match (ngram) is measured; Giménez and Màrquez (2007) ... See full document

... Topics”. Research in this period is charac- terized by a more linguistically-oriented focus, in- cluding syntactic topics like unification and catego- rial grammars, formal syntactic theory, and preposi- tional ... See full document

... Based Machine Translation, augmented with a re-ranking heuristic based on dissimilarity, and trained on a monolingual parallel ...when evaluating the output of simplification ... See full document

... the translation produced by a system is a paraphrase of the reference ...a re-examination in the light of the development of these novel MT metrics for which we can ask “How much better, if at all, do these ... See full document

... An automation system would be very beneficial here, in this scenario. The analysis using the automated system can help the radiologists or experts to enhance the diagnosis accuracy. An when an automated system is ... See full document

... We use a phrase-based statistical machine trans- lation system which is similar to (Och and Ney, 2004). The decoder adopts the regular distance distortion model, and also incorporates a maxi- mum entropy based ... See full document