[PDF] Top 20 A Study on Chinese Spelling Check Using Confusion Sets and?N-gram Statistics

... The second visually-similar-character set collects characters with similar Cangjie codes (倉頡碼, shorten as CJie hereafter). Cangjie is a well-known code map of Chinese characters. Each Chinese character is ... See full document

... most Chinese characters have other characters similar to them in either shape or pronunciation, an intuitive idea for CSC is to construct a confusion set for each ...the confusion sets (Zhang ... See full document

... NTOU Chinese spelling check system participating in CLP- 2014 ...Bakeoff. Confusion sets were expanded by using two language resources, Shuowen and Four-Corner ...find ... See full document

... for Chinese erroneous characters can be traced back to the detection and correction method put forward by Chang ...of confusion between the characters. Using such databases of computer characters ... See full document

... most Chinese character detec- tion systems are built based on confusion sets and a language ...for Chinese char- acter error detection in recent years (Huang et ...and confusion set, ... See full document

... Chinese spelling errors frequently arise from confusion between multiple Chinese characters which are phonologically and visually similar, but semantically distinct (Liu et ...2013 ... See full document

... everyday Chinese writing, there exist a va- riety of problematic usage of language, one of which is the spelling error referred in this ...Such spelling errors are mainly generated due to the ... See full document

... machine using written language to obtain relevant information effi- ciently and effectively ...of spelling correction. This work presents a novel spelling error detection and correction method based ... See full document

... character confusion sets which are edited based on phonologically and visually similarity between ...the confusion sets provided by SIGHAN Bake-off 2013 are used in both CSC systems (Wu et ... See full document

... possible spelling error detection will be added to the system to improve the detection ...the n- gram language models only aim at capturing the local contextual information or the lexical regularity ... See full document

... SIGHAN-8 Chinese Spelling Check task. The proposed joint bi-gram and tri- gram language model is helpful to determine the better character sequence as the results for detection and ... See full document

... for Chinese character error detection can be generated and tested by the chi-square test on the basis of a large ...building confusion sets and automatically generating a ... See full document

... We experiment on a medium-size (2 layers with 650 hidden states) LSTM language model (Zaremba et al., 2014) over two corpora: Wiki- text (Merity et al., 2016) and Google Billion-Word (Chelba et al., 2013) (1B). We adopt ... See full document

... the size of a language model can be reduced drastically by using his pruning algorithm. Kneser’s results improve with longer contexts and a same number of parameters. For example, reducing the size of the standard ... See full document

... 2. The statistical significance of automatic evaluations should be a good predictor of the statistical signifi- cance of human assessments with high reliability. The first criterion ensures whenever a human recognizes a ... See full document

... A non-word spelling error occurs when the written string cannot be found in a dictionary, such as in fly *fron Paris. The typical approach is finding a list of candidates from a large dic- tionary by edit distance ... See full document

... find the various types of spelling errors in the text. And error correction is to replace some inappropri- ate words and characters by some reasonable ones. With the close connection of mainland Chi- na and Hong ... See full document

... predefined confusion word set, such as { affect, effect } or { complement, compliment } , and provides the most appropriate word choice given the ...of spelling cor- rection (Bergsma et al., 2010) is based ... See full document

... As in China, the shared currency of the European Union is managed by one institution, the European Central Bank. The Central Bank was established in 1998 by the Treaty on European Union and its seat is in Frankfurt. Its ... See full document

... 3.1 Frequency Table and Its Alignment In order to resolve ambiguity and also recog- nise OOV terms, statistical information of n- gram string patterns in test files should be col- lected. There are in total ... See full document