neural network-based language models
Enhancing recurrent neural network-based language models by word tokenization
... the network are the previous n-words according to the language models ...final network output is computed using the Softmax activation function [3] to ensure that network output is a ...
13
Clinical Phrase Mining with Language Models
... deep neural network based Language Models (LM), such as BERT [26] and ELMo [27], which are trained on biomedical texts or clinical texts, ...phrase, based on which we build a ...
8
Using Factored Word Representation in Neural Network Language Models
... -gram based language models (Bilmes and Kirchhoff, 2003), most n-gram language models only use one ...in neural network based language models, it is ...
9
Adapting Grammatical Error Correction Based on the Native Language of Writers with Neural Network Joint Models
... and neural network joint mod- els (NNJMs) by adapting an NNJM based on the L1 background of the writers and integrating it into the SMT ...a network trained on general-domain data to be closer ...
11
The Edinburgh/JHU Phrase based Machine Translation Systems for WMT 2015
... 6-gram language models on the target ...bilingual neural network models: one over the source and one over the source and target, and an NPLM language model over the tar- ...
8
Rescoring a Phrase based Machine Transliteration System with Recurrent Neural Network Language Models
... 47 Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, pages 47–51, Jeju, Republic of Korea, 8-14 July 2012.. c 2012 Association for Computational Li[r] ...
5
Factored Language Model based on Recurrent Neural Network
... The standard back-off in an n-gram LM first drops the most distant word (w i − n+1 in the case of Eq. (1)), and then the second most distant word etc. until the unigram is reached. However, the factors in FLM occur ...
16
Incremental Adaptation Strategies for Neural Network Language Models
... of neural network language mod- els, most of them very ...corpora based on similarity between sentences) and (Duh et ...three models: n-gram, RNN and interpolated LM on two SMT systems: ...
9
Future word contexts in neural network language models
... on language models has focused on util- ising history information, the future word context information has not been extensively ...recurrent neural net- work language ...sigmoid based ...
8
Pre Computable Multi Layer Neural Network Language Models
... years, neural network models have significantly improved accu- racy in a number of NLP ...ral network models compared to alternate models, such as maximum entropy classi- ...
5
A Latent Variable Recurrent Neural Network for Discourse Driven Language Models
... The Penn Discourse Treebank (PDTB) provides a low-level discourse annotation on written texts. In the PDTB, each discourse relation is annotated be- tween two argument spans, Arg1 and Arg2. There are two types of ...
11
Off topic Response Detection for Spontaneous Spoken English Assessment
... spoken language assessment systems are becoming increasingly impor- tant to meet the demand for English sec- ond language ...question based on bag-of-words represen- tations. An alternative framework ...
10
Random Walks and Neural Network Language Models on Knowledge Bases
... We have performed some qualitative analysis, which indicates that there is a slight tendency for corpus embeddings (with the window size used in the experiments) to group related words (e.g. physics - proton), and not so ...
6
Generalizing and Hybridizing Count based and Neural Language Models
... with neural network ...them based on the distribution of the current document, albeit in a linear ...into neural language models, which allows for more direct learning of n- gram ...
10
Incorporating Side Information into Recurrent Neural Network Language Models
... Neural network approaches to language modelling (LM) have made remarkable performance gains over traditional count-based ngram LMs (Bengio et ...recurrent models (Mikolov et ...LM ...
6
Training Neural Network Language Models on Very Large Corpora
... computation based on Gaussian short ...acoustic models include 23k position-dependent triphones with 12k tied states, obtained using a divisive decision tree based clustering algorithm with a 35 base ...
8
Investigations on Phrase based Decoding with Recurrent Neural Network Language and Translation Models
... the models a better chance to influence translation in comparison to rescoring, as rescor- ing is limited to scoring and reranking fixed n- best ...Recently, neural networks were used for standalone ...
10
Measuring the Influence of Long Range Dependencies with Neural Network Language Models
... The contribution is this paper is two-fold. We first analyze the results of various NNLMs to assess whether long range dependencies are efficient in lan- guage modeling, considering history sizes ranging from 3 words to ...
10
Networks and Neural Language Models
... While the XOR function cannot be calculated by a single perceptron, it can be cal- culated by a layered network of units. Let’s see an example of how to do this from Goodfellow et al. (2016) that computes XOR ...
21
Artificial Intelligence Applications and Future Research Directions
... AI based factors; DL is specified as ANN with composite ...is based on its ...difficult models, with computing power to train and with automatic feature ...
6