[PDF] Top 20 CKY based Convolutional Attention for Neural Machine Translation

... the attention scores of an in- stance in the test ...the CKY-CNN. Note that an attention score of the first layer of the CKY-CNN corresponds to an atten- tion score of the hidden layer of the ... See full document

... document-level attention computed is sentence-based and static (computed only once for the sentence being ...hierarchical attention network (HAN) (Yang et ... See full document

... We set the dimension size of the hidden states to 512 in both of the LSTMs and the Tree LSTMs. The dimension size of embedding vectors is set to 512 for the words and to 256 for the characters. In our proposed ... See full document

... new attention mecha- nism that explicitly takes the attention history into consideration when generating the attention ...tention based NMT, the next word to attend is highly dependent on the ... See full document

... Neural machine translation (NMT) has gained a lot of attention recently due to its substantial im- provements in machine translation quality achiev- ing state-of-the-art ... See full document

... RNN based source-to-target at- tention mechanism where the number of parame- ters increases by repeating the calculation of multi- head attention for a single-source encoder like multi-hop attention ... See full document

... understanding neural ma- chine translation ...and attention, expanding search tree manually, and changing attention weight ei- ther manually or ... See full document

... into attention-based Neural Machine Trans- lation (NMT) for further improving translation ...enable attention mech- anism to attend to source words re- garding both the semantic ... See full document

... Although the NMT systems have several advantages, their performance is restricted in case of low- resource language pairs for which sufficiently large parallel corpora is not available and the language pairs whose ... See full document

... multi-modal, attention- based Neural Machine Translation (NMT) models which incorporate visual features into different parts of both the encoder and the ...lutional neural ... See full document

... the translation perfor- ...of Attention In order to evaluate whether the proposed model is able to capture phrase patterns or not, we visualize the attention layers in the encoder 2 ...pay ... See full document

... key-value attention achieved statistically significant results for the experiments with NTCIR-10 and ASPEC, though the experi- ments with IWSLT07 showed no such statistically significant ...the ... See full document

... multi-hop attention converges faster than the orig- inal ...hop attention (Fig- ure 1). Recently, many Transformer-based pre- trained language models such as BERT have been proposed and take about a ... See full document

... “ 使用 其 成员国 以外 的 武装力量 ” output by the sequential model, means “use the armed forces other than its member states” where “other than its member states” is incorrectly interpreted as a complement to “armed forces”. This is ... See full document

... We proposed augmenting transformer models with an alignment head to help extract alignments in scenarios such as dictionary-guided transla- tion. We demonstrated that the alignment- assisted systems can achieve ... See full document

... In parallel, the concept of “attention” has gained popularity recently in training neural networks, al- lowing models to learn alignments between dif- ferent modalities, e.g., between image objects and ... See full document

... the attention-based unknown word replacement method (UNK replacement in the table) consistently improves the BLEU scores by about ...the translation scores, and in particular, the BLEU score improves ... See full document

... Since syntactic GCNs produce representations at word level, it is straightforward to use them as encoders within the attention-based encoder- decoder framework. As NMT systems are trained end-to-end, GCNs ... See full document

... network based deep learning architec- tures for the transliteration of named en- ...different neural machine translation (NMT) frameworks: recurrent neural net- work and ... See full document

... an attention based neural network for image caption task and ad- vance the state-of-the-art results; Yin et ...the attention structure between a pair of convolution networks for answer se- ... See full document