[PDF] Top 20 Selective Attention for Context aware Neural Machine Translation

... quality translation for a full document. Recent works in context-aware NMT consider only a few previous sentences as context and may not scale to entire docu- ...for ... See full document

... To examine the effectiveness of our proposed methods on achieving sentiment-aware transla- tion, we manually construct an ambiguous test set. Sentences in this test set do not contain sentiment- bearing ... See full document

... informative context from sequential ...an attention based neural network for image caption task and ad- vance the state-of-the-art results; Yin et ...the attention structure between a pair of ... See full document

... The translation of pronouns presents a special challenge to machine translation to this day, since it often requires con- text outside the current ...overall translation quality are ill- ... See full document

... In neural machine translation (NMT), genera- tion of a target word depends on both source and target ...a translation while target contexts affect the flu- ...source context and ... See full document

... our attention on the fact that the classical formulation of the statis- tical machine translation framework, implicitly disregards the role of source-context information within the ... See full document

... training neural networks, al- lowing models to learn alignments between dif- ferent modalities, ...the context of NMT, Bahdanau et ...of attention-based architectures for ... See full document

... model can be reduced dramatically as only the characters need to be modeled explicitly. This enables the character-level NMT model to solve many scalability issues, both in terms of computational speed and memory ... See full document

... larger translation units. Here, the neural network produces a translation of the entire ex- tended ...and attention for the entire ...External context is not marked with specific ... See full document

... that those regional visual attributes would assist LSTM to generate better and more accurate repre- sentations. The illustration of the proposed model is depicted in 3. We will first explain how to deter- mine multiple ... See full document

... (Bahdanau et al., 2015; Luong et al., 2015). The encoder-decoder architecture predicts the target word with a target hidden-state and a context vec- tor. This context vector is calculated as a weighted ... See full document

... words, it is the Euclidean projection of the scores z onto the probability simplex. These projections tend to hit the boundary of the simplex, yielding a sparse probability distribution. This allows the de- coder to ... See full document

... the attention history of each source ...the context vectors to provide a rep- resentation which is able to capture the attention ...The attention of the current target word is determined based ... See full document

... Encoder-Decoder neural net- work (HRED) that accounts for the conversation ...generating context-aware responses by using a hierarchical joint attention ...from machine ... See full document

... Soft attention (Bahdanau et ...artificial neural net- work models for natural language understanding tasks like reading comprehension that take short passages as ...to attention in NLP select words ... See full document

... of context-aware models is more affected by the lack of the training data than sentence-level NMT models, we investigated the impact of large-scale parallel data on the trans- lation quality of ... See full document

... Table 1 shows the results 7 . In this table, we can observe that our proposed models, Cor-m and Cor-g, outperformed the baseline Concat in terms of BLEU scores at every unit length. Interestingly, at the setting of n = ... See full document

... novel context- aware smoothing method to dynamically learn a Context-Aware Representation (CAR) for each word (including OOV words) depending on its local context words in a ...the ... See full document

... sophisticated context- aware approaches propose to modify the NMT ar- ...as context sentence the previous source sentence, showing how NMT systems can also benefit from larger ...sentence ... See full document

... Standard machine translation systems pro- cess sentences in isolation and hence ig- nore extra-sentential information, even though extended context can both prevent mistakes in ambiguous cases and ... See full document