mt evaluation
Extending the BLEU MT Evaluation Method with Frequency Weightings
... In this paper we present the result of an ex- periment on augmenting BLEU N-gram compari- son with statistical weight coefficients which capture a word’s salience within a given docu- ment: the standard tf.idf measure ...
8
Approximating a Deep Syntactic Metric for MT Evaluation and Tuning
... Our primary objective is to create a good metric for automatic MT evaluation and possibly also tun- ing. We are not interested much in how close is our proposed approximation to the (automatic or man- ual) ...
7
CobaltF: A Fluent Metric for MT Evaluation
... the MT output, which have been widely used in the related fields of speech recognition (Uhrik and Ward, 1997) and quality estimation (Specia et ...of MT fluency that takes into account the number of ...
8
SPEDE: Probabilistic Edit Distance Metrics for MT Evaluation
... standard evaluation data sets and metrics as Pado et ...regression-based MT evaluation. We consider four widely used MT metrics (BLEU, NIST, METEOR (Banerjee and Lavie, 2005) ...human ...
8
Semantic Textual Similarity for MT evaluation
... to MT evaluation, we first, pre- process the pairs from Microsoft Research Para- phrase Corpus (Dolan and Brockett, 2005) with dates and time normalization, and then optional modules are applied depending ...
7
On reducing translation shifts in translations intended for MT evaluation
... the evaluation of machine transla- ...for MT evaluation due to the bias of each PE towards its MT system caused by the previ- ously mentioned system’s “shining ...automatic MT ...
8
Fully Automatic Semantic MT Evaluation
... As mentioned above, despite the fact that the semi- automatic HMEANT metric recently proposed by Lo and Wu (2011b,c,d) shows a higher correlation with human adequacy judgments than all commonly used automatic MT ...
10
Regression for Sentence Level MT Evaluation with Pseudo References
... to MT evaluation, both with human references (Corston-Oliver et ...translation evaluation reports a myriad of criteria that people use in their judgments, but it is not clear how these factors should ...
8
A Human Judgement Corpus and a Metric for Arabic MT Evaluation
... automatic evaluation metric with a higher τ value is making predic- tions that are more similar to the human judgments than an automatic evaluation metric with a lower τ ...art MT evaluation ...
7
CDER: Efficient MT Evaluation Using Block Movements
... automatic evaluation and the sum of fluency and adequacy was ...human MT evaluation, we have also calculated Kendall’s correlation coefficient τ ...
8
Alternative Objective Functions for Training MT Evaluation Metrics
... MT evaluation metrics are tested for cor- relation with human judgments either at the sentence- or the corpus-level. Trained metrics ignore corpus-level judgments and are trained for high sentence-level ...
6
On the Robustness of Syntactic and Semantic Features for Automatic MT Evaluation
... Linguistic metrics based on syntactic and semantic information have proven very effective for Automatic MT Evaluation. However, no results have been presented so far on their performance when applied to ...
9
APE at Scale and Its Implications on MT Evaluation Biases
... In this work, we train an Automatic Post- Editing (APE) model and use it to reveal bi- ases in standard Machine Translation (MT) evaluation procedures. The goal of our APE model is to correct typical errors ...
11
Extending MT evaluation tools with translation complexity metrics
... automated MT evaluation scores such as BLEU, which otherwise are variable across texts of different ...the MT systems evaluated on different ...automated MT evaluation packages – BLEU ...
7
A Graphical Interface for MT Evaluation and Error Analysis
... both MT systems and evaluation measures by offering a rich set of metrics and meta-metrics for assessing MT quality (Gim´enez and M`arquez, ...automatic MT evaluation is still far from ...
6
The Parameter Optimized ATEC Metric for MT Evaluation
... In MT evaluation, word order refers to the extent to which an MT output is interpretable following the information flow of its reference ...an MT output has many matched words but does not ...
5
Manual and Automatic Paraphrases for MT Evaluation
... Paraphrasing of reference translations has been shown to improve the correlation with human judgements in automatic evaluation of machine translation (MT) outputs. In this work, we present a new dataset for ...
6
Targeted Paraphrasing on Deep Syntactic Layer for MT Evaluation
... In this paper, we present a method of im- proving quality of machine translation (MT) evaluation of Czech sentences via targeted paraphrasing of reference sen- tences on a deep syntactic layer. For this ...
8
RED: A Reference Dependency Based MT Evaluation Metric
... In this paper, we propose a reference dependency based automatic MT evaluation metric RED. The new metric only uses the dependency trees of the reference, which avoids the parsing of the potentially noisy ...
10
MT Evaluation: Human Like vs Human Acceptable
... (MT) Evaluation are mostly based on met- rics which determine the quality of a given transla- tion according to its similarity to a given set of ref- erence ...an evaluation metric is its level of ...
8