neural language models
Reusing Weights in Subword Aware Neural Language Models
... improves language modeling quality while de- creasing the total number of trainable parameters almost two-fold, since most of the parameters are due to embedding ...subword-aware neural language ...
11
Strategies for Training Large Vocabulary Neural Language Models
... Training neural language models over large cor- pora highlights that training time, not training data, is the main factor limiting ...most models are still making progress after one ...
11
Improving Neural Language Models with Weight Norm Initialization and Regularization
... in neural language models (NLM) as well as in other sequence processing net- works that operate on large ...fine-tuned language models and observe that a NLM learns word vectors whose ...
8
Dependency Recurrent Neural Language Models for Sentence Completion
... Most neural language models consider the to- kens in a sentence in the order they appear, and the hidden state representation of the network is typically reset at the beginning of each sen- ...ral ...
7
Deep Neural Language Models for Machine Translation
... has been an active body of work recently in uti- lizing neural language models (NLMs) to improve translation quality. However, to the best of our knowledge, work in this direction only makes use of ...
5
Dependency Recurrent Neural Language Models for Sentence Completion
... ant of neural language models with 54.7% to 55.5% accuracy (Mnih and Teh, 2012; Mnih and Kavukcuoglu, 2013). We conjecture that their su- perior performance might stem from the fact that LBLs, just ...
7
MoL 2019 02: Neural language models with latent syntax
... considered neural network language models that incorporate syntactic ...a language model p(x) is obtained through approximate marginalization over y using a discriminative proposal ...as ...
114
Generalizing and Hybridizing Count based and Neural Language Models
... with neural network ...into neural language models, which allows for more direct learning of n- gram weights (Mikolov et ...and neural models, finding that neural ...
10
Compressing Neural Language Models by Sparse Word Representations
... for language modeling. Existing neural language models typically map discrete words to distributed, dense vector ...press neural language models by sparse word ...
10
Networks and Neural Language Models
... with neural networks, it is more common to avoid most uses of rich hand- derived features, instead building neural networks that take raw words as inputs and learn to induce features as part of the process ...
21
Learning to Compose Spatial Relations with Grounded Neural Language Models
... deep neural architecture successfully learns grounded spatial descriptions in a way that the learned functions are similar to the ones that generated the ...that language is compositional both at the level ...
13
Transductive Learning of Neural Language Models for Syntactic and Semantic Analysis
... ductive models (using a fine-tuned LM on each test ...transductive models consistently outperformed the baselines, which suggests that transductive LM fine-tuning improves performance of neural mod- ...
7
N gram and Neural Language Models for Discriminating Similar Languages
... deep neural network can outperform the traditional n-gram model for this task, but only once the data set size is dramatically increased and given more time to experiment on the network parameters and ...
8
Quantity doesn’t buy quality syntax with neural language models
... Recurrent neural networks can learn to predict upcoming words remarkably well on average; in syntactically complex contexts, however, they often assign unexpectedly high probabil- ities to ungrammatical ...the ...
7
Improving Neural Language Models by Segmenting, Attending, and Predicting the Future
... a neural network, for example a long short-term memory (LSTM; Hochreiter and Schmidhuber, 1997) network, and is represented with a distributed ...most models do not explicitly consider syntax, they still ...
11
Using Morphological Knowledge in Open Vocabulary Neural Language Models
... character-level language mod- elling, leveraging their ability to handle the long- range dependencies required to model language at the character ...training models on automatically acquired subword ...
11
Hierarchical Representation in Neural Language Models: Suppression and Recovery of Expectations
... sequence models have led to a marked increase in performance for a range of Natural Language Processing tasks, but it re- mains an open question whether they are able to induce proper hierarchical ...
10
Decoding with Large Scale Neural Language Models Improves Translation
... In this work, we extend the NPLM of Bengio et al. (2003) in two ways. First, we use rectified lin- ear units (Nair and Hinton, 2010), whose activa- tions are cheaper to compute than sigmoid or tanh units. There is also ...
6
Online Representation Learning in Recurrent Neural Language Models
... In order to further explore the relationship be- tween D and M, we trained a number of smaller models with different values, under the constraint D + M = 100. To reduce computation time, only half of the training ...
6
Neural language models as psycholinguistic subjects: Representations of syntactic state
... from language models. That is, we examine language model behavior on artificially constructed sen- tences designed to expose behavior that is cru- cially dependent on syntactic state representa- ...a ...
11