Neural network language models
The Fixed Size Ordinally Forgetting Encoding Method for Neural Network Language Models
... neural network language models, where the fixed- size FOFE codes are fed to FNNs as input to predict next word, enabling FNN-LMs to model long-term dependency in ...
6
Decoder Integration and Expected BLEU Training for Recurrent Neural Network Language Models
... Neural network language models are often trained by optimizing likelihood, but we would prefer to optimize for a task specific metric, such as BLEU in machine trans- ...recurrent neural ...
7
Incorporating Side Information into Recurrent Neural Network Language Models
... Recurrent neural network language models (RNNLM) have recently demonstrated vast potential in modelling long-term dependen- cies for NLP problems, ranging from speech recognition to machine ...
6
Future word contexts in neural network language models
... recurrent network language models (bi- RNNLMs) have been shown to outperform standard, unidirectional, recurrent neural network language models (uni-RNNLMs) on a range of ...
8
Deep Neural Network Language Models
... years, neural network language mod- els (NNLMs) have shown success in both peplexity and word error rate (WER) com- pared to conventional n-gram language mod- ...Deep neural networks ...
9
Random Walks and Neural Network Language Models on Knowledge Bases
... Neural Network Language Models have become a useful tool in NLP on the last years, specially in se- ...two models proposed in (Mikolov et ...feedforward Neural Network ...
6
Training Neural Network Language Models on Very Large Corpora
... Normally, the output of a speech recognition sys- tem is the most likely word sequence given the acoustic signal, but it is often advantageous to pre- serve more information for subsequent processing steps. This is ...
8
Pre Computable Multi Layer Neural Network Language Models
... can pre-compute the dot product between the 250- dimensional word embedding and the 250 × 500 section of the hidden layer. This results in four 500-dimensional vectors for each word that can be stored in a lookup table. ...
5
Using Factored Word Representation in Neural Network Language Models
... based language models (Bilmes and Kirchhoff, 2003), most n-gram language models only use one ...in neural network based language models, it is very easy to add ad- ...
9
Measuring the Influence of Long Range Dependencies with Neural Network Language Models
... recurrent network architecture for LMs was proposed in (Mikolov et ...and neural network), in- stead of using a traditional way, through interpola- ...different models is computationally very ...
10
Incremental Adaptation Strategies for Neural Network Language Models
... that neural net- work language models outperform back- off language models in applications like speech recognition or statistical machine ...a neural network ...
9
Convolutional Neural Network Language Models
... feed-forward network after using MLP Convolution, depending on the setup and ...convolutional neural networks is key to having better ...for language could be the effect of the convolution on the ...
10
Neural Network Language Models for Candidate Scoring in Hybrid Multi System Machine Translation
... Since LMs are probability distributions over sequences of words, they are a great tool for estimating the relative likelihood of whether some sequence of words belongs to a certain language. Sentence per- plexity ...
8
Rescoring a Phrase based Machine Transliteration System with Recurrent Neural Network Language Models
... 47 Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, pages 47–51, Jeju, Republic of Korea, 8-14 July 2012.. c 2012 Association for Computational Li[r] ...
5
Efficient Subsampling for Training Complex Language Models
... Neural Network Language Models (NNLM) have gained a lot of interest since their introduction (Ben- gio et ...standard language mod- eling, words are treated as discrete symbols, NNLM ...
9
Sparse Non negative Matrix Language Modeling
... for language modeling that can efficiently incorporate arbitrary ...SNM language models on two cor- pora: the One Billion Word Benchmark and a subset of the LDC English Gigaword cor- ...SNM ...
14
Enhancing recurrent neural network-based language models by word tokenization
... estimate language models from a given cor- ...different neural network architectures to estimate the language models from a given corpus using unsupervised learning neural ...
13
Converting Continuous Space Language Models into N Gram Language Models for Statistical Machine Translation
... Neural network language models, or continuous-space language models (CSLMs), have been shown to improve the performance of statistical machine translation (SMT) when they are ...
6
A Latent Variable Recurrent Neural Network for Discourse Driven Language Models
... based language modeling. Our models in this paper provide a uni- fied framework to model the context and current sen- ...RNN language model that generates the current ...Context Language Model ...
11
Investigations on Phrase based Decoding with Recurrent Neural Network Language and Translation Models
... current neural network (RNN) language and translation models during phrase- based ...feedforward neural mod- ...RNN language and translation models into a phrase-based ...
10