Top PDF Machine Translation and Post-editing : Impact of Training and Directionality on Quality and Productivity

Machine Translation and Post-editing : Impact of Training and Directionality on Quality and Productivity

Machine Translation and Post-editing : Impact of Training and Directionality on Quality and Productivity

6. Conclusions and future work After having conducted our pilot study, we are able to answer the research questions raised above. Research questions number 1 and 2 were related to the impact of training on the linguistically correctness and accuracy of post-editing. Specifically, they sought to answer to what extent post-editing performed by group A (question 1) and by group B (question 2) is linguistically correct and accurate. Results showed that group A (Modern Languages students) only obtained more successful edits than group B (Translation students with training on Translation) in the grammar/syntax and punctuation subcategories in Text 2. Group B obtained more successful edits than group A in all subcategories in Text 1 and in the spelling and mistranslation subcategories in Text 2. However, ANOVA results proved that the difference between both groups was only significant in punctuation (p=.0479) and mistranslation (p=.00283) in Text 1, which reveals that the impact of training is not relevant in this pilot study. This result is consistent with Temizöz (2013) according to whom post-editing performance between different educational profiles (engineers and professional translators) were similar in terms of quality.
Show more

11 Read more

Post-editing Productivity with Neural Machine Translation: An Empirical Assessment of Speed and Quality in the Banking and Finance Domain

Post-editing Productivity with Neural Machine Translation: An Empirical Assessment of Speed and Quality in the Banking and Finance Domain

3 Assessment of Translation Productivity We conducted a productivity test of domain- adapted NMT on the premises of Migros Bank. Subjects translated texts under two experimental conditions. In TM-O NLY , they used the trans- lation workbench known from their their daily work, including a domain-specific TM, a domain- specific TB, and any online services (except ma- chine translation) of choice. The same setup was used in P OST -E DIT , except that sentences with no fuzzy match of at least 80 % in the TM were pop- ulated with MT within the translation workbench. We did not show MT where high fuzzy matches were available because editing high fuzzy matches is more efficient (Sánchez-Gijón et al., 2019). Materials We used four German source texts from Migros Bank. The texts had not been trans- lated by any of the translators involved in the ex- periment before, and had been excluded from the MT training material (see below). The TMs con- tained several exact and high fuzzy matches for each text (Table 1).
Show more

7 Read more

Post editing Productivity with Neural Machine Translation: An Empirical Assessment of Speed and Quality in the Banking and Finance Domain

Post editing Productivity with Neural Machine Translation: An Empirical Assessment of Speed and Quality in the Banking and Finance Domain

Subjects A total of four professional translators took part in the productivity test, two each for the target languages FR (FR-1, FR-2) and IT (IT-1, IT-2). All were members of Migros Bank’s inter- nal translation team. They were therefore familiar both with the software used and with the language and terminology of the documents to be trans- lated. FR-1, who joined the organisation shortly before the experiment, was less experienced than the other participants. All subjects had been post- editing outputs of the MT systems used in the ex- periment (see above) for three months, and had re- ceived four hours of post-editing training.
Show more

6 Read more

A fuzzier approach to machine translation evaluation: A pilot study on post editing productivity and automated metrics in commercial settings

A fuzzier approach to machine translation evaluation: A pilot study on post editing productivity and automated metrics in commercial settings

Another possible explanation for Translator 1’s performance would be that the quality of the raw MT output is low. However, Transla- tor 2’s productivity gains and comparison with past projects’ performance contradict this. We therefore concluded that the most probable ex- planation to the difference in terms of produc- tivity might be due to the MTPE experience of both translators. In fact, studies about impact of translator’s experience agree that more ex- perienced translators do MTPE faster (Guer- berof Arenas, 2009), although they do not usu- ally distinguish between experience in transla- tion and experience in MTPE.
Show more

6 Read more

Productivity and quality in MT post-editing

Productivity and quality in MT post-editing

Ana.guerberof@logoscript.com Abstract Machine-translated segments are increasingly included as fuzzy matches within the translation-memory systems in the localisation workflow. This study presents preliminary results on the correlation between these two types of segments in terms of productivity and final quality. In order to test these variables, we set up an experiment with a group of eight professional translators using an on-line post- editing tool and a statistical-base machine translation engine. The translators were asked to translate new, machine-translated and translation-memory segments from the 80-90 percent value using a post-editing tool without actually knowing the origin of each segment, and to complete a questionnaire. The findings suggest that translators have higher productivity and quality when using machine- translated output than when processing fuzzy matches from translation memories. Furthermore, translators’ technical experience seems to have an impact on productivity but not on quality. Finally, we offer an overview of our current research.
Show more

9 Read more

The Impact of Machine Translation Quality on Human Post Editing

The Impact of Machine Translation Quality on Human Post Editing

4.3 Productivity by System and Post-Editor While the large differences between the post- editors are unfortunate when the goal is consis- tency in results, they provide some data on how post-editors of different skill levels are influenced by the quality of the machine translation systems. Table 3 breaks down translation speed by ma- chine translation system and post-editor. Interest- ingly, machine translation quality has hardly any effect on the fast Post-Editor 1, and the lower MT performance of system UU affects only Post- Editors 3 and 4. Post-Editor 2 is noticeably faster with UEDIN - SYNTAX — an effect that cannot be
Show more

9 Read more

Identifying the machine translation error types with the greatest impact on post-editing effort

Identifying the machine translation error types with the greatest impact on post-editing effort

Assessing PE Effort via Process Analysis According to Krings (2001) , there are three main types of process- based post-editing effort. Of these three, the easiest to define and measure is temporal effort: how much time does a post- editor need to turn machine translation output into a high quality translation? The second type of post-editing effort is somewhat harder to measure, namely technical effort. Technical effort includes all physical actions required to post-edit a text, such as deletions, insertions, and reordering. The final type of effort is cognitive effort. It refers to the mental processes and cognitive load in a translator’s mind during post-editing, and can, presumably, be measured via fixation data. While it is important to distinguish between these three types of post-editing effort conceptually, it must be noted that they are, to some extent, related to one another. Temporal effort is determined by a combination of cognitive effort and technical effort, and while an increase in technical effort does not necessarily correspond to an increase in cognitive effort, the technical process is still guided by cognitive processes.
Show more

15 Read more

Post Editing System For Statistical Machine Translation

Post Editing System For Statistical Machine Translation

Hindi is the mother tongue language, official language of India and is 4th most widely spoken in the world. In the 2001 Indian survey, 258 million (258,000,000) people in India reported Hindi to be their native language; whereas Punjabi is the official language of Punjab state of India and the 11th most widely spoken in India. It is also the fourth most vocal language in England and Wales and third most spoken in Canada. Both the language having great impact on Indian officials, journals, articles. The problem starts when the natives of Punjab cannot understand the Hindi language. In order to make it possible very few translation software are available in market but quality of these software are not up to the mark. The idea of this paper is to describe a system that will improve the quality of translation from Hindi to Punjabi. In this paper, describe a post editing module for statistical based Hindi-Punjabi Translation system that will compare the result with previous machine translation system.
Show more

6 Read more

A translation robot for each translator? : a comparative study of manual translation and post-editing of machine translations: process, quality and translator attitude

A translation robot for each translator? : a comparative study of manual translation and post-editing of machine translations: process, quality and translator attitude

misspelling of compounds for technical texts), and some occurred equally frequent across both translation methods (such as terminology issues for technical texts). The abundance of wrong collocations in PE can be explained by the fact that student translators are often not critical enough of literal MT translations (Depraetere, 2010), although wrong collocations were slightly more problematic for human translation during the technical translation task. The abundance of terminology issues in both methods of translation is in contrast with Guerberof (2009), who found that post-edited segments contained more terminology issues than segments that were translated from scratch. The fact that many of the most common error types overlap between human translation and post-editing (four error categories for newspaper articles and six for technical texts) could indicate that these translation methods are not as different from one another as sometimes thought (O'Brien, 2002). Knowledge of error types can be integrated into post-editor training and translation tool development. Perhaps an extra warning could be integrated into a tool whenever certain polysemous words or awkward collocations could occur in the MT output. Such a tool would also benefit from a spell-checker, since this could reduce the large number of misspelled compound nouns, typos and punctuation errors found in the pretests. Regarding post-editor training, the post-editing of different text types could be used to make students aware of text type specific issues. For example, terminology, logical problems, missing or superfluous articles and untranslated text only belonged to the most common problem categories for technical texts, but not for newspaper articles. The misspelling of compound nouns, which was relatively problematic in the post-editing of newspaper articles (accounting for 6% of all PE errors), was a far more common issue for technical texts (accounting for 10% of all HT errors and 16% of all PE errors made). Regarding technical texts specifically, it is striking that, even when given a terminology list to adhere to, one of the most common problems in students' translations and post-edited texts is incorrect terminology. Depraetere (2010), like Guerberof (2009), suggested that "it is also necessary to provide a terminology list" (para. 6), but it would seem from our experiment that students need additional terminology management training on top of the terminology list.
Show more

267 Read more

Productivity and quality when editing machine translation and translation memory outputs: an empirical analysis of English to Welsh translation

Productivity and quality when editing machine translation and translation memory outputs: an empirical analysis of English to Welsh translation

developers themselves but there is now a large body of academic research based on controlled and/or longitudinal studies which does lend support to the claim that editing TM and MT output as opposed to translation ‘from scratch’ (i.e. without these tools) may in fact be more efficient and does not necessarily lead to a decrease in quality (see Section 4). The claim that the translation process can be rendered more efficient by this editing process can be taken to refer to the decrease in cognitive effort as well as referring to decreases in terms of processing time and concomitant increases in words per minute/hour. Relevant studies that have investigated comparative cognitive effort between translation and post- editing MT, using pause metrics or eye-tracking variables, include O’Brien (2006b), Carl, Gutermuth and Hansen-Schirra (2015) and Koglin (2015) and those investigating the same psychological construct between translating and editing TM outputs include O’Brien (2007a), Mellinger (2014) and Screen (2016). All of these studies found that the editing process made translation cognitively easier for the translator as well as easier in terms of the physical processes of text production. Turning to translation speed and productivity, the published literature is also fairly uniform in its conclusions and includes studies by Lange and Bennett (2000), O’Brien (2006, 2007b), Offersgard et al. (2008), Brkić et al. (2009), Groves and Schmidtke (2009), Guerberof (2009, 2012, 2014), Flourney and Duran (2009), Kanavos and Kartsaklis (2010), Plitt and Masselot (2010), Skadiņš et al. (2011), Federico et al. (2012), Green et al. (2013), Aranberri et al. (2014), Elming et al. (2014), Moran et al. (2014), Silva (2014), Zhechev (2014), Uswak (2014) and Carl, Gutermuth and Hansen- Schirra (2015). As these studies informed the deductive hypotheses related to productivity measured in this study, all will now be briefly described below and the number of translators and type of source text will be given (if noted in the original publication). Only studies involving translation professionals have been reviewed below, but further studies that have used bilinguals or translation students when comparing translation speed are Koehn (2009), Daems et al. (2013), De Sousa et al. (2011), Lee and Liao (2011), García (2011), Yamada (2012), Vázquez et al. (2013), Läubli et al. (2013) and Depraetere et al. (2014). These also report greater efficiency when using TM and MT.
Show more

24 Read more

Machine Translation Post-Editing at TransPerfect - the 'Human' Side of the Process

Machine Translation Post-Editing at TransPerfect - the 'Human' Side of the Process

3/ Explain what is expected from the linguists in terms of final translation quality. As many other LSPs, we distinguish two types of MTPE quality, which are Light and Full post-editing. Light PE consists of only correcting major MT errors in order to make the translated text understandable. The quality of the translation after Light PE is lower than the one expected in a regular translation project, it can sound too literal or unnatural and contain minor objective errors. In contrast, Full PE quality requirements are higher and the translation after Full PE cannot contain any errors or stylistic defects. Our PE training includes a detailed description of the two post-editing quality levels with examples, which is one of the most important and difficult aspects of PE work.
Show more

8 Read more

Multi Engine and Multi Alignment Based Automatic Post Editing and its Impact on Translation Productivity

Multi Engine and Multi Alignment Based Automatic Post Editing and its Impact on Translation Productivity

The term Post-Editing (PE) is defined as the correction performed by humans over the translation pro- duced by an MT system (Veale and Way, 1997). It is often understood as the process of improving a translation provided by an MT system with the minimum amount of manual effort (TAUS Report, 2010). While MT is often not perfect, post-editing MT can yield productivity gains as post-editing MT output may require less effort compared to translating the same input manually from scratch. MT outputs are often post-edited by professional translators and the use of MT has become an important part of the translation workflow. A number of studies confirm that post-editing MT output can improve translators’ performance in terms of productivity and it may positively impact on translation quality and consistency (Guerberof, 2009; Plitt and Masselot, 2010; Zampieri and Vela, 2014). The wide use of MT in modern translation workflows in the localization industry, in turn, has resulted in substantial quantities of PE data which can be used to develop APE systems.
Show more

12 Read more

Impact of Automatic Segmentation on the Quality, Productivity and Self reported Post editing Effort of Intralingual Subtitles

Impact of Automatic Segmentation on the Quality, Productivity and Self reported Post editing Effort of Intralingual Subtitles

subtitles in terms of quality, productivity and self-reported post-editing effort. Quality has been evaluated objectively through precision, recall and F1-score metrics; a post- editing task has been carried out to obtain objective mea- sures of productivity; and the self-reported effort has been assessed subjectively through a ranking questionnaire. All evaluations have been performed through comparison with the main technique employed for automatic subtitle seg- mentation nowadays, which is based in counting characters. The quality achieved by the proposed CRF-based classifier has been shown to outperform that of the counting charac- ter technique by far. Post-editing productivity has shown to increase up to three times and the self-reported effort of the post-editing task to decrease three points. In addition, post- editors have found subtitle segmentations generated by the machine learning method to be of better quality, easier and less boring to post-edit respecting the guidelines of good segmentation and their post-edited versions thought to be better segmented. These successful results show the poten- tial of machine learning to model the segmentation rules employed in traditional subtitling from a relatively small corpus of already segmented subtitles.
Show more

5 Read more

PET: a Tool for Post-editing and Assessing Machine Translation

PET: a Tool for Post-editing and Assessing Machine Translation

For both language pairs, post-editing only the best machine translations according to any QE model allows more words to be post-edited in a fixed amount of time than post-editing randomly selected machine translations (“unsorted”). The best rate is obtained with time as response variable in both fr-en and en-es datasets. This shows that the implicit an- notation of time using PET is a promising way of collect- ing training data for quality estimation. In this case, PET was used not only for data collection purposes, but also as a means to evaluate the usefulness of the QE models and compare different variations of such models.
Show more

6 Read more

Real Time Adaptive Machine Translation for Post Editing with cdec and TransCenter

Real Time Adaptive Machine Translation for Post Editing with cdec and TransCenter

This paper describes the end-to-end machine translation post-editing setup provided by cdec Realtime and TransCenter. As the quality of MT systems continues to improve, the idea of using automatic translation as a primary technology in assisting human translators has become increas- ingly attractive. Recent work has explored the possibilities of integrating MT into human transla- tion workflows by providing MT-generated trans- lations as a starting point for translators to cor- rect, as opposed to translating source sentences from scratch. The motivation for this process is to dramatically reduce human translation effort while improving translator productivity and con- sistency. This computer-aided approach is directly applicable to the wealth of scenarios that still re- quire precise human-quality translation that MT is currently unable to deliver, including an ever- increasing number of government, commercial, and community-driven projects.
Show more

6 Read more

Translators’ perceptions of literary post-editing using statistical and neural machine translation

Translators’ perceptions of literary post-editing using statistical and neural machine translation

translations that contain fewer errors (Klubička, Toral, and Sánchez-Cartagena 2017, Way 2018b), NMT has been found to produce less literal translations than the previously dominant paradigms (Castilho et al. 2017a). These findings are based on evaluations to date, which have been carried out using technical and educational texts. The current study has two objectives: firstly, to assess translators' attitude towards PE of NMT output, about which little or no research has yet been published, and secondly, to test the capability of a state-of-the-art NMT system to aid PE of literary texts, a challenging long-lifespan text type, to which MT and PE is not usually applied. We compare levels of PE effort and perceived post-task acceptability when translating from scratch, post-editing phrase-based Statistical MT (SMT), and post-editing NMT. Our hypothesis is that NMT PE will be more productive and acceptable than SMT PE, but that participants will prefer to translate from scratch, as literary PE is not yet a common task. We consider an evaluation of literary MT and PE timely due to the advent of NMT, for which claims have been made regarding high quality and the ability to place translated words in the appropriate context, and the growing availability of e-books. These e-books are an ideal resource for training literary-adapted MT systems, both using monolingual data (novels in a digital format) and bilingual or parallel data (digital novels and their translations). Literary
Show more

22 Read more

Post-editing neural machine translation versus translation memory segments

Post-editing neural machine translation versus translation memory segments

Even though NMT allows a translator to achieve better quality output than other MT systems, the results obtained suggest that NMT output does not seem to boost productivity as much as might be expected. NMT output achieves good performance results in terms of edit distance and number of edits required, meaning that the output requires less editing effort than TM fuzzy matches in general. However, the time invested in post-editing NMT output is, in general, higher. A caveat here is that throughput may increase as post-editors become more accustomed to the types of errors that are produced by an NMT system. Furthermore, those translators who perceived that MT boosts their productivity actually performed better when post-editing MT segments than those translators who perceived MT as a poor resource. To delve deeper into the reasons why the correlation between an increase in quality and a decrease in editing and editing time does not occur, the results presented suggest that it might not be enough to merely collect data on performance and perception. The results point to the fact that translators’ perceptions of MT tends to match their real productivity, even when they carry out a blind task and there are no hints as to the provenance of the segments they are editing. This suggests that this issue should also be addressed by gathering data of a cognitive nature on the process of post-editing MT output.
Show more

74 Read more

Using the TED Talks to Evaluate Spoken Post editing of Machine Translation

Using the TED Talks to Evaluate Spoken Post editing of Machine Translation

Mesa-Lao (2014) surveyed the post-editors’ views and atti- tudes before and after the introduction of speech technology as a front-end to a computer-aided translation workbench. The survey shows that people tend to respond positively to- wards ASR used in post-editing, and they seem willing to adopt it as an input method for future post-editing tasks. In another user-oriented study, Dragsted et al. (2011) inves- tigated the efficiency that can be achieved by using speech recognition software for translation tasks. With sufficient training and practice, the speech recognition’s time con- sumption appears to approach that of sight translation, and speech recognition quality appears to approach that of writ- ten translation. These studies thus indicate the need for, and the potential acceptability of, voice-based post-editing of MT output, although more studies focusing on possible uses in multilingual social networks and/or access from mo- bile devices would be welcome.
Show more

8 Read more

Ensembling Factored Neural Machine Translation Models for Automatic Post Editing and Quality Estimation

Ensembling Factored Neural Machine Translation Models for Automatic Post Editing and Quality Estimation

Martins et al. (2016) introduced a stacked archi- tecture, using a very large feature set within a structured prediction framework to achieve a large jump in the state of the art for Word-Level QE. Some features are actually the outputs of stan- dalone feedforward and recurrent neural network models, which are then stacked into the final sys- tem. Although their approach creates a very good final model, the training and feature extraction steps are quite complicated. An additional disad- vantage of this approach is that it requires "jack- knifing" the training data for the standalone mod- els that provide features to the stacked model, in order to avoid overfitting in the stacked ensemble. This requires training k versions of each model type, where k is the number of jackknife splits.
Show more

8 Read more

POST-EDITING SERVICE FOR MACHINE TRANSLATION USERS AT THE EUROPEAN COMMISSION

POST-EDITING SERVICE FOR MACHINE TRANSLATION USERS AT THE EUROPEAN COMMISSION

Future prospects We are witnessing the beginning of a communications revolution created by the Internet and other on-line facilities, which will inevitably influence customer expectations in relation to rapid response. There can no longer be any doubt of the real demand within the institution for urgent translations for information purposes. While it is essential that documents of publication quality continue to be the full responsibility of professional translators, there are a number of text types which lend themselves very well to machine translation. The setting up of a post-editing service aimed at MT users in the Commission is a new and pioneering venture, and tradition dies hard. Nevertheless, there is a powerful economic motivation to turn machine translation resources to good account. The PER project has developed in the context of the Commission's current management policy, which is to contain expenditure and improve the cost/quality ratio of all its activities. The number of official languages is due to increase from 11 to 22 in the foreseeable future with the arrival of the Eastern European countries, creating an additional strain on finite resources. The post-editing of machine translation seeks to strike the right balance between time, quality, and available capacity. Applied to the right types of text, it offers a pragmatic approach to three main areas of concern: increased productivity, effective use of existing tools and reduced costs.
Show more

6 Read more

Show all 10000 documents...