[PDF] Top 20 Oracle and Human Baselines for Native Language Identification

... also on the test set from the 2013 shared task, shown in Table 2. For both Tables 2 and 3 we report a random baseline and the best performances on the Shared Task: the first number is the top performer from the shared ... See full document

... the oracle score, we also use an Accuracy@N ...the Oracle combiner that is limited to the top N ranked candidates in order to minimize the influence of a single classifier having chosen the correct label by ... See full document

... Native language identification: Malmasi and Cahill propose a function to measure feature independence for an NLI system, and analyze its effectiveness on a standard NLI corpus; Malmasi et ...an ... See full document

... In this paper we describe the approaches we explored for the 2017 Native Language Identification shared task. We focused on simple word and sub-word units avoiding heavy use of hand-crafted features. ... See full document

... This study is also novel in its use of post- corrected POS tags. As noted in §5.1, while the POS-based results here are different from those of previous studies that have used automated tagging methods, it is unclear if ... See full document

... ond language acquisition for identifying learner dif- ...in language and sound transfer, which are how people apply knowl- edge of their native language, and its phonology and orthography, ... See full document

... straightforward for humans to complete. Machines typically trade-off some performance for ease of ap- plication and reduced cost. Equally fascinating are tasks that seem non-trivial to humans, but on which machines, ... See full document

... and language transfer ...how language is processed in the brain in ways that are not possible by just studying mono- linguals or single L1-L2 pairs, thereby providing us with important insights that ... See full document

... Native Language Identification (NLI) is the task of automatically predicting the native language (L1) of a speaker given an unlabeled artifact such as a writing sample or speech ... See full document

... Foreign Language (TOEFL). The essays are written by 11 native lan- guage speakers ...glish language proficiency level (high, medium, or low) based on the judgments of human assessment ... See full document

... of native language identi- fication in the context of user generated content (UGC) in online ...between native and nonnative authors; (ii) deter- mining to which language family the ... See full document

... As noted by Bestgen et al. (2012), Jarvis and Paquot (2012), and Tetreault et al. (2012), there are certain disadvantages to using the ICLE for NLI research. One problem made especially clear by Bestgen et al. is that ... See full document

... Previous works on identifying the native lan- guage from texts explored a large variety of features, including lexical and part-of-speech (POS) features (Koppel et al., 2005a), charac- ter n-grams (Ionescu et al., ... See full document

... tive Language Identification (Tetreault et al., 2013; Malmasi and Dras, 2017) and many other NLP tasks (Goutte et al., 2014). Among many alter- natives, we focus on voting, a conceptually and practically ... See full document

... than, and that do not require the manual construction of, the above approach. Taking the trees produced by statistical parsers, we use tree cross-sections as features in a machine learning approach to deter- mine which ... See full document

... the native language-specific topic (i.e. characteristic of the native language) and the other is the generic topic ...second language — English in our ...a language- specific ... See full document

... There has been a great deal of work on NLI in recent years. The methods employed have ranged from some combination of lexical, part-of-speech and n-gram features (Koppel et al., 2005), to syntactic features (Wong and ... See full document

... Native language identification (NLI) is the task to determine the native language of the author based on an essay written in a second ... See full document

... downloaded from the NLI Shared Task website. 1 To date, nearly all approaches have treated the task of NLI as a supervised classification problem where statistical models are trained on data from the different L1s. The ... See full document

... Second Language Acquisition (SLA) investigate the multiplex of factors that influence our ability to acquire new languages and chief among these is the role of the learner’s mother ...of Native ... See full document