[PDF] Top 20 A Perplexity Based Method for Similar Languages Discrimination

... In our previous participation (Gamallo et al., 2016) in the DSL 2016 shared task we pre- sented two very basic systems: classification with ranked dictionaries and Naive Bayes classifiers. The results showed that ranking ... See full document

... Meaningful subword features could well give a boost to existing systems, even if they are based on simple extraction methods. In fact, an off-the-shelf Naive Bayes classifier using a contrastive word and subword ... See full document

... In Table 8, the comparative results between open and close approaches in the development set are shown. It is noteworthy that both approaches ob- tained lower results with the same groups (es, pt and hr). Regarding ... See full document

... other languages using automatically-induced crosslingual POS taggers, and (2) the difference in accuracy for discriminating English varieties between representations based on a coarse-grained universal ... See full document

... the languages in Group F, and an evaluation for groups A-E was therefore performed separately; we discuss only this latter task and evaluation ...approaches based on character or byte sequences have shown ... See full document

... between similar languages (DSL) shared task ...system based on support vector machines (SVM) with linear kernel and using character ngram features, which obtained the first rank at the closed ... See full document

... (Slavic languages), Hindi-Nepali (Indo-Aryan lan- guages) and Spanish-Portuguese (Romance lan- ...systems based on NMT for Hindi → Nepali ...both languages and also trained an NMT system with copy ... See full document

... The discrimination between soybean and rice plots was expected to be difficult because of the weather condition, with obviously visible interference by ...imaging method aided in the discrimination ... See full document

... proper discrimination between closely related ...certain languages make it very hard to classify them correctly using standard methods that have been proposed in the ...specific discrimination tasks ... See full document

... are based on character ...statistical method for language identification based on Markov models to compute the likelihood of the character ...related languages and ...other languages) ... See full document

... (Discriminating Similar Lan- guages) shared task has been organized in 2014 and the task participants presented their systems at the VarDial workshop at COLING ...of similar languages and language ... See full document

... determines the language group, the second works with empirically selected features that achieved best performance for the specific language group. (Lui et al., 2014) also define a two phase ap- proach involving a ... See full document

... 1999), based on On- tologies and taxanomy (Gruber, 1993; Resnik, 1995), Vector Space Model (VSM) with similarity measures like Dice similarity, Jaccard’s similarity, cosine similarity (Salton, ...dian ... See full document

... well-established languages and reported in Gamallo et ...40 languages was ...the perplexity between two recognized ...the perplexity-based distance between Catalan and Spanish is about ... See full document

... systems based on Support Vector Machines: SVM with a flat combination of features and SVM ...a method of adaptation to the test set, which may indicate potential pitfalls with some of the data ... See full document

... used the randomly generated embeddings in both the CNN and LSTM models in the range [−0.25, 0.25]. We used 80% of the training set for training and 20% of the data for validation set and performed 5-fold cross validation ... See full document

... Uralic languages from the ...Uralic languages (Kone Foundation, ...relevant languages based on the ISO 639- 3 division of the Uralic languages (SIL, ...relevant languages, such ... See full document

... Discriminating similar languages and language variants is useful for several purposes. As the typical linguistic processing pipeline is tailored to a specific language and variant, it is important to have a ... See full document

... In this work we evaluated the Discriminating be- tween similar languages 2015 shared task. We em- ployed the continuous Skip-gram model to gener- ate distributed representations of words and sen- tences ... See full document

... identifying languages from very little data, from multi- languages input or discriminating between extremely sim- ilar languages are bottlenecks of this field (Ljubeˇsic and Kranjcic, 2014; Zampieri ... See full document