[PDF] Top 20 Exploring Prediction Uncertainty in Machine Translation Quality Estimation

... in uncertainty modelling are not captured by the usual point estimate metrics commonly used for this ...tive uncertainty is useful, either in human decision making or by propagating this information for ... See full document

... Due to space constraints, we only present results for the en-fr test sets, but those for the en-es test sets fol- low the same trend. Multitask models lead to further improvements, particularly visible for post-editor 3 ... See full document

... the quality of texts translated auto- matically by Machine Translation (MT) systems have become a necessity in many ...as quality estimation (QE), or also confi- dence ... See full document

... of quality estimation system predictions and gold labels, in addition to being the official measure applied to scoring vari- ants of tasks in WMT (Bojar et ...of quality estimation systems ... See full document

... As a term of comparison, we consider these baselines in our experiments. A simple to implement but difficult to beat baseline when dealing with regression on tasks with different distributions is to compute the mean of ... See full document

... one translation direc- tion, English-German, language-specific features can be ...noun translation errors from English to German as well as measuring the amount of reordering done by the SMT ... See full document

... a prediction value for all data points in the test set, this results in a system as good as the baseline system when evaluated with ...the prediction distribution would effectively be a con- stant and its ... See full document

... From a practical point of view, providing the best trade off between accuracy and computational time (He and Wang, 2012), PA represents a good solution to meet the demand of efficiency posed by the CAT framework. For ... See full document

... These challenges have been separately ad- dressed from different perspectives in few recent works. Huang et al. (2014) proposed a method to adaptively train a QE model for document- specific MT post-editing. ... See full document

... Confidence Estimation features and a learning mechanism trained on human ...traditional machine translation evaluation and the quality estimation tasks recently ...its quality, ... See full document

... on quality estimation have proved the consistency of the automatic predicted score with the human evaluation ...the quality score for selecting the n-best candidate translations of the MT ...a ... See full document

... word-level quality estimation task and confidence estima- tion for interactive translation prediction is that each human interaction in the interactive trans- lation prediction setting ... See full document

... of Translation with Explicit OR- dering) scores MT outputs by aligning them with given ...a translation sen- tence and the corresponding reference ... See full document

... Quality estimation task in WMT18 (Specia et al., 2018) (QET18) address machine translation per- formance prediction (MTPP), where translation quality is predicted without ... See full document

... on Quality Estima- tion (QE), ...supervised Machine Learning (ML) problem and designed additional features and combined these with the baseline feature set to estimate ... See full document

... Machine translation quality estimation is a challenging task in the WMT evaluation ...matic quality estimation, and in this paper, we propose neural network fea- tures, including ... See full document

... of translation quality and subsequently employ HTER scores as a gold standard representation when evaluating QE systems, ultimately leading to rankings for competing ...of translation quality, ... See full document

... chine translation toolkit (Koehn et ...the translation model and lan- guage ...ferent quality translation systems, details of which are described in Table ... See full document

... Ondrej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, Radu Soricut, Lucia Specia, and Aleˇs Tamchyna. 2014. Findings ... See full document

... using machine translation services. However, though machine translation techniques have been advanced a lot, the machine translation quality is far from satisfactory, and ... See full document