Word-based language model
LMSim : Computing Domain specific Semantic Word Similarities Using a Language Modeling Approach
... finding word similarity using topic models (Blei et ...of word context to capture its ...order word co-occurrence, they do not consider order of other words within the context of a ...considering ...
6
Language Model Based Arabic Word Segmentation
... natural language processing applications because a word often conveys complex meanings decomposable into several morphemes ...natural language systems including machine translation (Brown et ...
8
Gated Word Character Recurrent Language Model
... Moreover, word–character hybrid models have been studied on different NLP ...a word–character hybrid language model on Chinese using a neural network language model (Bengio et ...
6
Syntax Based Word Ordering Incorporating a Large Scale Language Model
... is word ordering. Word ordering is a com- putationally difficult problem, which can be constrained to some extent for particu- lar applications, for example by using syn- chronous grammars for statistical ...
11
RNN language model with word clustering and class-based output layer
... Brown word clustering algorithm is pro- posed to construct a class layer for the ...the word error rate ...full model without a class ...hierarchical word cluster ...
7
CMUQ@QALB 2014: An SMT based System for Automatic Arabic Error Correction
... statistical language model- ing techniques and a phrase-based machine transla- tion ...the word movement as well as punc- tuation problems, which can produce a more accurate ...
6
Incorporating Word and Subword Units in Unsupervised Machine Translation Using Language Model Rescoring
... Cross-lingual word embeddings can provide a good initialization for both the NMT and SMT ...space based on a bilingual dictionary of a negligibly small ...foregone word pairs, but by considering ...
8
Automatic Extraction of New Words from Japanese Texts using Generalized Forward Backward Search
... We present a new word extraction method for Japanese based on expected word frequency, which is computed by using a statistical language model and an N-best word segmentation algorithm..[r] ...
12
Translation Model Based Cross Lingual Language Model Adaptation: from Word Models to Phrase Models
... this model selected large quantities of irrele- vant text, which may degrade the adapted LM (Eck et ...our word TM based cross-lingual data selection model, we operate at the sentence level ...
11
A Generalized Framework for Hierarchical Word Sequence Language Model
... HWS model is essentially an n-gram lan- guage model based on a different assumption that a word depends upon its nearby high-frequency words instead of its preceding ...n-gram language ...
7
UNBNLP at SemEval 2019 Task 5 and 6: Using Language Models to Detect Hate Speech and Offensive Language
... 3.1.1 Word-level LMs: n-gram and LSTM For each language, we grouped the training in- stances based on their gold standard labels — giv- ing us two corpora per language — with one con- sisting ...
5
The TALP–UPC Spanish–English WMT Biomedical Task: Bilingual Embeddings and Char based Neural Language Model Rescoring in a Phrase based System
... 5-gram language model is estimated on the target side of the cor- pus using interpolated Kneser-Ney discounting with SRILM (Stolcke, 2002) (SLM, small lan- guage ...big language model). ...
6
A kind of intelligent question answering system based on sentence similarity calculation model
... method based on keywords feature embodies the surface information of the sentences; the similarity method based on semantic distance embodies the semantic information of every word of a sentence in ...
8
The Karlsruhe Institute of Technology Translation Systems for the WMT 2014
... discriminative word alignment. For word reordering, we use word lat- tices with long range reordering ...two word-based language models, a bilingual language model, ...
6
Wider Context by Using Bilingual Language Models in Machine Translation
... bilingual language model that extends the translation model of the phrase-based SMT ap- proach by providing bilingual word ...bilingual word context, this approach enables us ...
9
Morphological Analysis for Unsegmented Languages using Recurrent Neural Network Language Model
... guage model of raw word sequences, but by using a semantically generalized language model based on word embeddings, RNNLM (Recurrent Neural Network Language Model) ...
6
Accurate Word Segmentation using Transliteration and Language Model Projection
... the model analyzes the input. Our ap- proach is based on semi-Markov discrimina- tive structure prediction, and it incorporates English back-transliteration and English lan- guage models (LMs) into WS in a ...
7
A Dynamic Language Model Based on Individual Word Domains
... a language model for each significant word, which is formed in the same manner as the global language ...The word language-training corpus to be used is the amalgamation of the ...
6
A Ranking based Approach to Word Reordering for Statistical Machine Translation
... the word order in target ...to word reordering. The ranking model is automatically derived from the word aligned parallel data, viewing the source tree nodes to be reordered as list items to ...
9
Feature based Neural Language Model and Chinese Word Segmentation
... tributed word representation of neural language model have been proved very useful in NLP ...shares word representations across the tasks of language modeling, part-of-speech tag- ging, ...
7