statistical n-gram modeling
Sampling Informative Training Data for RNN Language Models
... of statistical language modeling seeks to learn a joint probability distribution over se- quences of natural language ...language modeling, far below those of traditional n-gram models ...
5
Statistical Input Method based on a Phrase Class n gram Model
... A phrase model has less cross-entropy than a word model (Mori et al., 1997). Phrase modeling makes a more accurate language model. However the vocabulary size of phrase models is larger than that of word models, ...
14
Unlimited vocabulary speech recognition for agglutinative languages
... language modeling re- sults by the statistical morphs seem to be at least as good, if not better (Hirsimäki et ...the statistical morphs for three agglutinative languages and describe three different ...
8
Evaluation of Language Models over Croatian Newspaper Texts
... language modeling involves techniques and procedures that assign probabilities to word sequences or, said in other words, estimate the regularity of the ...of statistical language models, reviews their use ...
34
Abbreviated text input using language modeling.
... We address the problem of improving the efficiency of natural language text input under degraded conditions (for instance, on mobile computing devices or by disabled users), by taking advantage of the informational ...
20
N gram based Tense Models for Statistical Machine Translation
... In this paper, tense modeling is done on the target- side language. Since our experiments are done on Chinese to English SMT, our tense models are learned only from the English text. In the literature, the ...
10
Smooth Bilingual N Gram Translation
... We are only aware of one work that performs a systematic comparison of smoothing techniques in phrase-based machine translation systems (Foster et al., 2006). Two types of phrase-table smoothing were compared: black-box ...
9
N Gram Based Statistical Machine Translation versus Syntax Augmented Machine Translation: Comparison and System Combination
... Along with inserting extra words and wrong lexical choice, another prominent source of incorrect translation, generated by the N - gram system, is an erroneous grammatical form selection, i.e., a situation ...
9
Statistical Representation of Grammaticality Judgements: the Limits of N Gram Models
... We distinguish our task from the standard task of error detection in NLP (e.g. Post (2011)), that can be used in various language processing systems, such as machine translation (Pauls and Klein, 2012), language ...
9
Secure Bio-Cryptographic Authentication System for Cardless Automated Teller Machines
... An intelligent TC for Arabic language approach was proposed in [25] using the statistical n-gram stemmer, a hybrid approach of Document Frequency Thresholding and Information Gain fo[r] ...
10
Modeling of term distance and term occurrence information for improving n gram language model performance
... The normalization of distant bigram counts, as indicated in Eq.6, aims at highlighting the infor- mation provided by the relative positions of words in the history-context. This has been shown to be an effective manner ...
5
SB@GU at the Complex Word Identification 2018 Shared Task
... for modeling native and non-native perceptions of complexity, and this in- formation can be exploited at training time, us- ing features that rely on the number of native and non-native annotators could not be ...
7
A HYBRID METHOD OF LINGUISTIC APPROACH AND STATISTICAL METHOD FOR NESTED NOUN COMPOUND EXTRACTION
... and statistical approach for extracting Arabic collocations from collected Arabic newspaper ...bi- gram candidates, and then the POS tagger used for disambiguate the words that have more than one linguistic ...
8
Vector based Natural Language Call Routing
... Based on the statistical discriminating power of the n-gram terms extracted from the caller's request, the caller is 1 routed to the appropriate destination, 2 transferred to a human ope[r] ...
28
Faster and Smaller N Gram Language Models
... To test our LM implementations, we performed experiments with two different language models. Our first language model, W MT 2010, was a 5- gram Kneser-Ney language model which stores probability/back-off pairs as ...
10
N gram based Machine Translation
... Table 9 presents the four best BLEU results for the EPPS translation task in the first TC-STAR’s evaluation campaign, where the results corresponding to our n-gram- based translation system are provided in ...
24
Re evaluating Automatic Metrics for Image Captioning
... BLEU Papineni et al., 2002 Machine translation n-gram precision ROUGE Lin, 2004 Document summarization n-gram recall METEOR Banerjee and Lavie, 2005 Machine translation n-gram with synon[r] ...
11
Statistical modeling of Tamil nadu annual rainfall
... The annual rainfall data for a period of 63 years (1950-2012) of Tamil Nadu have been analyzed for climatologically pattern and trend. Climate change for a region can be analyzed only on a long–term average. Detecting ...
5
HIVEC: A Hierarchical Approach for Vector Representation Learning of Graphs
... work of Mikolov et al. [1], traditionally used in language modeling, where representations of words is learnt using a deep learning method from the context where they occur in a sentence. The notion of context is ...
7
A Large Scale Distributed Syntactic, Semantic and Lexical Language Model for Machine Translation
... 5-gram/2-SLM+2-gram/4-SLM+5- gram/PLSA language model improves both signif- ...for N -best list ...the n-grams received higher scores from BLEU; ours did ...
10