Similar to other natural language processing tasks, in statistical machine translation, to empirically verify every new approach or idea to get accepted by the community as an effective approach, it needs to be compared to valid baseline. Therefore, along the liter- ature study we implemented a state of the art phrase-based decoder as a foundation for our future experiments. We re-implemented the existing phrase-based decoder in the group with focusing on modularity and speed. To reach a good speed performance and also taking advantage of object orientation for modularity, we developed the system in Java and C++ programming language.
Tagine4, is a phrase-based multi-stack, multi-beam decoder with ability to add more features very easily. Main features are categorised in three classes: Translation model, language model and distortion model. Additional features such as phrase penalty5 can be integrated without affecting other components of the system. All the features are managed by a component which acts as an interface between the features and the
4TrAnslation enGINE
main decoder. Tagine has a component to manage all the constants and parameters of the decoder through configuration files, so many adjustments can be made without recompiling the system. It reads a generated phrase table in a widely used format as its translation model and has an interface to interact with SRILM [Stolcke, 2002] and IRSTLM [Federico and Cettolo,2007] language model tool-kits.
We have developed a web application which connects to Tagine through a socket and can perform the translation online. This version has an implementation of the binary phrase table of [Zens and Ney,2007], so it can use very large phrase tables or multiple languages simultaneously.
As described in Section2.2the problem of statistical machine translation can be written as:
ˆe(f) =arg max
e
(Σiλifi(e, f)) (2.7.1)
where fi is a feature and λi is its weight in this log-linear model. The performance of
the this translation system is largely dependent on the finding proper weights for the features (λs). To optimise the weights, we use Minimum Error Rate Training algorithm of Franz Och [Och, 2003]. To find the best weights for the features, MERT algorithm needs a large amount of candidate translations, so it iteratively runs the decoder with the best weights from the previous iteration and cumulatively use the translations to find the minimum points.
Summary
In this chapter, we briefly introduced machine translation and the main ideas of statisti- cal MT. The word-based models of IBM and their application in the heuristic phrase ex- traction method of phrase-based models were also discussed. We pointed out that the two main models of statistical MT are translation and language models and described some of their attributes. In addition in Section 2.6, some of the main automatic met- rics and the logic behind MT evaluation were discussed. This chapter was concluded by a description of our implementation of a phrase-based decoder and experimental framework used for the rest of the thesis.
CHAPTER 3
Reordering in Statistical Machine
Translation
There are several factors contributing to the difficulty of machine translation. For ex- ample, different levels of morphology between the source and target languages can make it difficult to generate the right verb, with correct morphology, in the target side. Another problem is the issue of unknown words. If both languages are using the same alphabet and are historically related, there is a chance of success in translating un- known words, if simply the unknown word is reproduced in the target side. However, if the languages are historically distant or more importantly the alphabets are different, this approach has a little chance to succeed.
One of the major problems in MT is different word orders between the source and the target language. In translating a source sentence to a target sentence, reordering is the requirement of the decoding algorithm to skip words and cover them after translating later words in the sentence. In other words, reordering in statistical machine translation is the need for not necessarily sequentially translate all the phrases in a source sentence. The main reason that reordering is needed is the fundamental word order differences between languages. The main word order in some languages is Subject-Object-Verb such as Persian1and in some it is Subject-Verb-Object such as English. There are other word order differences, such as the order of noun modifiers and the noun between
1Although most of the sentences are in the form of SOV, the word order is almost free and many
different languages.
In this chapter, we define different types of reordering and overview the current ap- proaches proposed to deal with the problem. In the rest of this chapter, we discuss the difference between the main SMT approaches in tackling the word order differences and the role of syntactic knowledge in some of these works.
3.1
Different Types of Reordering
We categorise different reorderings based on the width of the distance between the words that are moved to be translated consequently. There are many ways to categorise them into a few classes such as short, medium and long distance reorderings. Here, we discuss short and long distance reorderings and show examples of both to demonstrate the difference between them and the reason different methods are used to deal with them.
3.1.1 Short Distance Reordering
An intuitive difference in word order between languages is the location of noun modi- fiers with respect to the noun. For example, in English adjectives precede the noun, but in French they follow the noun. Figure3.1shows an alignment for a translation from French to English2.
As you can see from the alignment matrix, to translate such a sentence, the decoder should allow a jump over the word group and a jump back to translate pse. This kind of permutation that requires a skip of a few words, less than three, is called short distance or local reordering. Fortunately, phrase-based models are relatively successful in this kind of reordering. Three different features of phrase-based models enable it to handle this situation effectively. Firstly, it is likely to extract the phrase groupe pse during train- ing and translate this phrase in one phrase application, hence with correct word order. Secondly, n-gram language models are very effective to assign a higher probability to the correct permutation of the words in such a short distance. Finally, the distance
CHAPTER3: REORDERING INSTATISTICALMACHINETRANSLATION
<
s
>
( applause from the pse group )
<s> ( applaudissements du groupe pse )
Figure 3.1:A French to English translation with a local reordering.
based reordering model (see Section3.2.1) does not penalise short distance jumps such as 1 and−1 too harsh, consequently, there is chance for other features like language model to compensate the penalty.
3.1.2 Long Distance Reordering
Although, a good language model or a lexicalised distortion model (see Section3.2.3), can deal with local reordering problem, they are not usually sufficient for long dis- tance reorderings. A long distance reordering3 is needed, where one is dealing with languages with very essential different word orders. For example, many German sen- tences are in SOV order which is very different from English word order, SVO. In these cases, the problem is a long distance between the subject and the verb. After translat- ing the subject, the decoder has to skip rather a large number of words to reach the verb and consequently jump back after translating the verb. Figure3.2shows an exam- ple of German to English translation which requires a long distance jump to correctly translating the verb.