[PDF] Top 20 Refining Word Segmentation Using a Manually Aligned Corpus for Statistical Machine Translation

... the segmentation of an au- tomatic segmenter with reference to a WA cor- pus revealed a number of ...the word “bao fa” in Figure 1. Empirically we observed that this word is seg- mented as a single ... See full document

... Chinese. Word seg- mentation is thus normally adopted as an ini- tial step in most natural language processing tasks for these Asian ...Although word segmentation techniques have improved greatly ... See full document

... bilingual corpus with the relevant language segmented into basic writ- ing units ...of using the output from an existing statistical word aligner to obtain a set of candidate ...candidates ... See full document

... of translation is higher as compared to other ...their translation corresponding to high frequency ...on manually corrected corpus, as is evident from the manual evaluation and improved BLEU ... See full document

... vectors using neural net- works, the network is presented with a phrase cor- ...phrase corpus is similar to a word cor- pus except that some words are joined to make up ...phrases using a ... See full document

... Our work is based on the POS-based reorder- ing model described by Niehues and Kolss (2009), in which POS-based rules are extracted from a word aligned corpus, where the source side is part-of-speech ... See full document

... In statistical machine translation, the smallest unit is usually the word, defined as a token delimited by ...parallel corpus of source and target text, the training procedure first ... See full document

... Unsupervised word segmentation (UWS) can provide domain-adaptive segmenta- tion for statistical machine translation (SMT) without annotated data, and bilin- gual UWS can even optimize ... See full document

... and word alignment ...document-aligned corpus into a parallel topic-aligned cor- pus, then learning word alignments us- ing co-occurrence ...topic- aligned corpus is ... See full document

... novel segmentation approach for Phrase-Based Statistical Machine Translation (PB-SMT) to languages where word boundaries are not obviously marked by using both monolingual and ... See full document

... 6.2 Translation Task: Small Track IWSLT We evaluate our full model, using both monolin- gual and bilingual information, on the IWSLT ...training corpus was segmented using the unigram seg- ... See full document

... of statistical machine translation for new language pairs and domains: a reduction in the cost of cre- ating new training data, and the development of more efficient methods for exploiting existing ... See full document

... Data sparsity is one of the major problems for statis- tical learning methods in natural language process- ing (NLP) today. Even with the huge training data sets available in some tasks, for many phenomena that need to ... See full document

... for word- reordering: Given a sentence outputted by the al- gorithm, we regard it as a permutation of the cor- rect sentence, and count the the number of in- version pairs in it, which can be seen as the dis- ... See full document

... for statistical machine translation is described by Lee ...Arabic-English translation takes as input POS-tagged English and Arabic text, where the Arabic words have been pre-segmented into ... See full document

... described word-to-phrase alignment mod- els capable of good quality bitext word ...Chinese-English translation and alignment they compare well to Model-4, even with large ...improves ... See full document

... trained with SRILM (Stolcke, 2002) on the target side of the training data. The weights of the log- linear model were optimized with MIRA (Watan- abe et al., 2007) on a held-out development set re- served for this ... See full document

... phrase-based statistical MT system and this im- provement is statistically ...into statistical MT via the intro- duction of two new features, we could explore other alternative ways of ... See full document

... do word sense disambiga- tion models help statistical machine trans- lation quality? We present empirical re- sults casting doubt on this common, but unproved, ...assumption. Using a state-of- ... See full document

... apply machine learning or feature engi- neering to the task of reranking the systems, so we provided several ...weights using pairwise ranking optimization (PRO; Hopkins and May, 2011), with a perceptron as ... See full document