Comparative Analysis Of Advanced Classification Techniques For Multilingual Ocr Systems

1036

Comparative Analysis Of Advanced Classification

Techniques For Multilingual Ocr Systems

Rohit Verma, Dr. Jahid Ali

Abstract : Classification engineering is reported to be very critical and tedious task in the field of data, image and pattern recognition. Labelling the images into one of the earlier defined categories, is the responsibility of a typical classifier. Preprocessed image does require for decent results. For fantastic and fabulous achievements, image should be free from any kind of noise and should be normalized to the acceptable parameters. There are myriads of classification techniques but the most challenging aspect is to identify the best technique which could intelligently recognize optical characters on the basis of predefined features of characters. This paper elaborates extremely important classification techniques viz. K-Nearest Neighbors (KNN), Support Vector Machine (SVM) and Artificial Neural Networks (ANN) and Convolutional Neural Network (CNN). Various classification techniques are compare on the basis of literature so that the researchers could take the advantage and select the best possible technique for their objectives.

Index Terms : Image Recognition, Image Classification, neural networks, Optical Character Recognition (OCR), nearest neighbor classifier, Machine Learning, Support Vector Machine classifier.

------

1 INTRODUCTION

Classification is defined as a machine learning application of assigning labels to the unknown data on the basis of available set of labeled data. The extremely important task involves the prediction of unknown data based on predefined features. The set of such features which are helpful for recognition of a stranger is referred to as feature vector. For recognition purpose, firstly, a training set of features is developed and then the character to be recognized is processed to produce the same set of the features. These features are compared with the feature vector. The class processing the nearest neighbor is assigned to that character. For attaining high degree of accuracy, choice of features play extremely important role. Most relevant features could improve the accuracy rate to manifolds. Although countless classification techniques are available for achieving the targets, but different classification techniques are meant for different types of classification. In reality classification phase is the critical decision making phase which uses the features extracted in theprevious phase to classify the objects.

2 PHASES OF OCR

Following steps could be considered extremely important for the recognition of characters:[1]

2.1 Characters pre-processing

This task includes improve the intensity of characters under observation, Skew removal if any, removal of noise, normalization of the text to the desired level and component analysis of the characters.

2.2 Character detection and extraction

In this step the exact position of the characters is calculated. In case of Devanagari script the characters from the word are segmented from the bottom. Whereas in case of roman characters, they are already segmented. So no significant efforts are employed to achieve the objectives. For the segmentation purpose, initially characters are thinned and then header line is found by knowing the maximum intensity of white pixels. After getting the row number, those columns are set to ‗0‘ where only one pixel is found as shown in the Figure.

Fig1. Character Segmentation of Devanagari

2.3 Classifier training

This task is extremely important for the categorization purpose. Most significant features of characters are considered for the training of the classifiers. On the basis of excellent choice of these features a high degree of accuracy could be achieved. The extremely vital features that could be considered for desirable results may be Local Binary Pattern(LBP) Hu‘s moment invariants, Zone densities and identification of horizontal and vertical lines in each zone for instance. Other features that could be considered are elaborated in the paper [2]. R Verma et al. presented a set of features and categorized them as per their characteristics.

2.4 Character classification

This is the finalstep of classification. Important features of the unknown character are generated and compared with pre-generated feature vector of desired domain. On the basis of the match Found, a suitable label is allocated to the character.

3. OVERVIEW OF CLASSIFIERS

Numerous classifiers are available for the recognition of characters for the applications in banking industry, money ————————————————

 Rohit Verma, Research Scholar, IKG Punjab Technical University,Kapurthala, India, [email protected]

 Dr. Jahid Ali, Director, Shri Sai Iqbal College of Management & IT, Pathankot, India,

1037 counting, postal department, automatic data entry etc.

Majority of classifiers that are employing for attaining the objectives are: k-Nearest Neighbor (KNN), Support Vector Machine (SVM), Artificial Neural Network (ANN), Convolutional Neural Network (CNN) and Local Binary Pattern (LBP). The present work is focused on the comparative study of the most commonly employed classifiers in view of the domain where these classifiers are used. S. Arora in the paper [3] performed comparative analysis of SVM and ANNs by applying these classification techniques on Devanagari character set. After applying preprocessing techniques, vital features viz. shadow features, view based features, chain code histogram features and longest run features were extracted. The extracted features are input to ANN classifier and in SVM for classification. Desirable accuracy was reported by applying these classification techniques on a count of 7154 data samples of Devanagari script. The most commonly used classifiers are described here:

3.1 Artificial Neural Network (ANN)

ANN imitates developing an intelligence that emulate the intelligence of human beings. ANN comprises of cycle of layers from input layer to the output layers in between hidden layers. Each layer has neurons as their fundamental element, which passes information layer to layer. Neurons of various layers are connected by weights to the neurons of the precursive and subsequent layers. ANN classification technique employs non-parametric approach[1] and the performance of the classifier depends upon the architecture and count of inputs. Anita pal. [4] Reported an accuracy of 94% by applying two-layer perceptron neural network on handwritten English characters. In the paper [5], Manish Kumar et.al reported maximum accuracy of 83% by applying feed forward back propagation neural network on a sample of 297 hand written characters.

Fig2. Neural Network with six inputs and four outputs

Further it was concluded that recognition approaches depend on nature and count of the sample data under observation. Reetika et.al. in the paper [6] reported an accuracy of 98.77% in the process of recognition of English characters by employing SURF features using Neural Network. The reported SURF Features in addition to neural network has noticeable performance and produces absolutely disjoint set of feature points which enhances the computing speed manifold.

3.2 K-Nearest Neighbors (KNN)

KNN is significantly important classification algorithm which has countless applications in Machine Learning (ML). This algorithm is based on the approach of classifying the objects on the basis of closest training match in the feature domain. It is a type of instance based learning, where computation is deferred till classification. Using the approach of k-NN an object is classified on the basis of maximum votes of its k-neighbors and if k = 1, only class is assigned to that object [7]. It is proved to give fabulous results in the domain of supervised learning and finds great applications in pattern recognition field. Though KNN can be used for both classification and regression predictiveproblems but high degree of accuracy and desirable results are achieved in classification applications.

Fig. 3 K-NN Classification Diagram

S. Roy in the paper [8] reported an accuracy of 97.67% for classifying the MNIST numeral digits images with KNN and ten features. It is observed that in the domain of pattern recognition KNN is a non-parametric approach used for classification and regression. V. Vaidehi [7] proposed that k-NN classifier produce better and significant results than its other competitors due to reduce error rate and lesser execution time. In addition, this classifier works better with images under different lighting conditions and is independent of image resolutions. In the paper [9], Zanchettin et.al. Proposed a hybrid approach of using K-NN with SVM (Support Vector Machine) for the recognition of hand written cursive English characters and it was observed that the recognition rates improved significantly with this hybrid approach of K-NN and SVM.

3.3 Support Vector Machine (SVM)

1038 Fig. 4 Support Vector Machine

K. Gauri et al. in paper [11] reported an accuracy of 97.16% for digits, 95.74% accuracy for capital letters and 92.19% for small letters with SVM classifier. Furthermore, it was reported that SVM employs Structural Risk Minimization principle (SRM) which minimizes an upper bond on the expected risk, In contrast ERM minimizes the error on the training data. Because of this superiority, Support vector machine gave better results. In addition, due to the absence of Local Minima in SVM, it possesses lesser value of execution time in comparison to MLP. Mustafa S. Kadhm et al. in the paper [12], purposed a new architecture based on SVM classifier for recognition of hand written Arabic words. Since the purposed research was based on recognition of word, so character segmentation stage was not required. Further, it was reported that the best accuracy of 96.317% was recorded for the recognition of Arabic Hand written dataset with multiple feature extraction methods and SVM classifier. With the experimental results it was also proved that the polynomial kernel of SVM is convergent and produce excellent results in comparison to other kernels of SVM. In the paper [13], T. W. Umapada Pal, reported an accuracy of 95.13% for the recognition of DevNagari character recognition. To achieve high degree of accuracy two sets of features were used with combined classifiers SVM and MQDF (Modified quadratic function).

3.4 Convolutional Neural Network

Convolutional Neural Network (CNN) is extremely important and widely employed technique for pattern recognition

challenges. Myriads of reasons have been developed for the famousness of CNN. Traditionally, in character recognition models, feature extraction is is done manually. Whereas in CNNs, the weighted convolutional layer employed for feature extraction as well as the fully connected layer employed for classification are determined during the training phase. As a reward, memory requirements and computational complexity reduced significantly and performance becomes noticeable for applications where the input has local correlation. A high recognition degree of accuracy is achieved with CNN only. CNN is reported to give a fantastic accuracy of 99.77% testing with MNIST handwritten digits dataset [14]. Furthermore, CNN are shift Invariants because same weight configuration is employed across space. Another important feature of CNN is its noticeably reduced memory requirement. In addition, CNN takes lesser time to train and provides excellent performance. The only limitation of CNN is that it requires tremendous amount of data for achieving desirable and fabulous results.

In paper [15], A. El-Sawy et al. achieved an accuracy of 94.9% in the recognition process of Arabic handwritten characters by Implementing CNN.

It was observed that CNN reported best results in contrast to other Deep learning architectures in the domain of image and big data.

Fig 5. A typical Convolutional Neural Network [16]

In the paper [17], W. Yang et al. proposed an Enhancement of DCNN approach to online Chinese hand written characters by employing domain specific knowledge viz. non-linear normalization, path signature, for instance.

TABLE 1: Performance comparison of various classification methods with accuracy

S.NO. Classifier DataSet Data Set

Size Accuracy Reference

1 SVM and MQDF

Devnagari handwritten

characters

36172

characters 95.13% [13]

2 SVM Arabic hand

written dataset 1675 words 96.3% [12]

3 SVM Numeric digits -- 97.16% [11]

4 SVM English alphabets 2600 95.23% [10]

5 SVM and KNN Cursive

characters 57293 89.65% [9]

6 SVM Assamese

character set -- 96% [18]

7 Neural Network Hand written

characters 297 83% [5]

8 MLP

Isolated hand written Bengali

Characters

3000 95.10% [19]

1039

10 Neural Network Kannada

characters 2500 96.8% [21]

11 Neural Network Small English

alphabets 520 82.5% [22]

12 Neural Network English alphabets 650 96% [23]

13 Neural Network English alphabets -- 85% [24]

14 KNN MNIST data set of

characters 60000 97.67% [8]

15 KNN English alphabets 200 76.9% [25]

16 CNN Arabic hand

written characters 16800 94.9% [15]

17 CNN English

Characters 62992 81.05% [26]

18 DCNN

Hand written Devnagari characters

-- 98% [27]

19 CNN UNIPEN dataset -- 93.7% [28]

20 KNN English printed

characters 180 91.25% [29]

Note: KNN-K-Nearest Neighbors, CNN-Convolutional Neural Network, DCNN-Deep Convolutional Neural Network, SVM- Support Vector Machine, MLP-Multi Layer Perceptron, MQDF-Modified Quadratic Discriminant Function.

Author‘s compilation

4. CONCLUSION

In this paper, widely employed classification techniques for the characters recognition have been reviewed. It is observed that depending upon the dataset used, various classification techniques provide multi-degree of accuracy. In some cases SVM and KNN are proving their presence where as in others CNN is proved to be excellent. In comparison to other classification techniques, CNN gained much popularity because of less memory requirements and reduced training time. But at the same time CNN requires huge amount of data for fabulous performance. Trends have been noticed where CNN achieved an accuracy of 99.77% in case of English characters using MNIST dataset.

REFERENCES

[1] P. Kamavisdar, S. Saluja, and S. Agrawal, ―A survey on image classification approaches and techniques,‖ Int. J. Adv. Res. Comput. Commun. Eng., vol. 2, no. 1, pp. 1005–1009, 2013.

[2] R. Verma and I. K. G. Punjab, ―A Review of key features instrumental in recognizing the Multi-lingual Optical Characters,‖ Int. J. Res. Eng. Appl. Manag., vol. 05, no. 02, 2019.

[3] S. Arora, D. Bhattacharjee, M. Nasipuri, L. Malik, M. Kundu, and D. K. Basu, ―Performance Comparison of SVM and ANN for Handwritten Devnagari Character Recognition,‖ Int. J. Comput. Sci., vol. 7, no. 3, pp. 1–10, 2010.

[4] A. Pal and D. Singh, ―Handwritten English Character Recognition Using Logistic Regression and Neural Network,‖ Int. J. Sci. Res., vol. 5, no. 6, pp. 750–754, 2016.

[5] M. K. Sahu, N. K. Dewangan, and M. T. Scholar, ―Handwritten Character Recognition using Neural Network,‖ Intern. J. Adv. Res. Comput. Commun. Eng., vol. 6, no. 6, pp. 11–14, 2017.

[6] R. Verma and R. Kaur, ―Enhanced Character

Recognition Using Surf Feature and Neural Network Technique,‖ Int. J. Comput. Sci. Inf. Technol., vol. 5, no. 4, pp. 5565–5570, 2014.

[7] V. Vaidehi et al., ―Person Authentication Using Face Detection,‖ Proc. World Congr. Eng. Comput. Sci., p. 6, 2008.

[8] S. Roy and M. Saravanan, ―Handwritten Character Recognition using K-NN Classification Algorithm,‖ IJARIIE, no. 5, pp. 1245–1250, 2017.

[9] C. Zanchettin, B. Leite, D. Bezerra, and W. W. Azevedo, ―A KNN-SVM Hybrid Model for Cursive Handwriting Recognition,‖ in WCCI 2012 IEEE World Congress on Computational Intelligence, 2012, pp. 10–15.

[10] S. Sharma, ―A SVM based Character Recognition System,‖ in IEEE International Conference on resecnt trends in Elecgtronics information & Communication Technology, 2017, pp. 1703–1707. [11] G. Katiyar, A. Katiyar, and S. Mehfuz, ―Off-Line

Handwritten Character Recognition System Using Support Vector Machine,‖ Am. J. Neural Networks Appl., vol. 3, no. 2, pp. 22–28, 2017.

[12] M. S. Kadhm, ―Handwriting Word Recognition Based on SVM Classifier,‖ Internatioal J. Adv. Comput. Sci. Appl., vol. 6, no. 11, pp. 64–68, 2015. [13] T. W. Umapada Pal, Sukalpa Chanda, ―Accuracy

Improvement of Devnagari Character Recognition Combining SVM and MQDF Indian Statistical Institute,‖ in Proc. 11th Int. Conf. Frontiers …, 2008 - Citeseer, 1977.

[14] D. Ciregan, U. Meier, and J. Schmidhuber, ―Multi-column deep neural networks for image classification,‖ Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., pp. 3642–3649, 2012.

1040 Convolutional Neural Network,‖ WSEAS Trans.

Comput. Res., vol. 5, pp. 11–19, 2017.

[16] K. S. Choi, J.-S. Shin, J.-J. Lee, Y. S. Kim, S.-B. Kim, and C.-W. Kim, ―In vitro trans-differentiation of rat mesenchymal cells into insulin-producing cells by rat pancreatic extract.,‖ Biochem. Biophys. Res. Commun., vol. 330, no. 4, pp. 1299–305, 2005. [17] W. Yang, L. Jin, Z. Xie, and Z. Feng, ―Improved

Deep Convolutional Neural Network For Online Handwritten Chinese Character Recognition using Domain-Specific Knowledge.‖

[18] D. Das, R. Devi, S. R. M. Prasanna, and S. Ghosh, ―P ERFORMANCE C OMPARISON OF O NLINE H ANDWRITING R ECOGNITION S YSTEM F OR A SSAMESE L ANGUAGE B ASED ON HMM A ND SVM M ODELLING,‖ pp. 87–95, 2014.

[19] J. Bakas, ―A Comparative Study of Various Classifiers for Character Recognition on Multi-script Databases,‖ Int. J. Comput. Appl., vol. 155, no. 3, pp. 1–5, 2016.

[20] A. B. R. V. K. H.M, ―Performance Analysis of KNN and SVM Classifiers Using Handwritten Kannada Vowels Recognition,‖ Int. J. Sci. Res., vol. 4, no. 7, pp. 1284–1288, 2015.

[21] R. S. Kunte and R. D. S. Samuel, ―A simple and efficient optical character recognition system for basic symbols in printed Kannada text,‖ Sadhana 32 (5), pp. 521–533., vol. 32, no. October, pp. 521– 533, 2007.

[22] Y. Perwej and A. Chaturvedi, ―Neural Networks for Handwritten English Alphabet Recognition,‖ IEEE, vol. 20, no. 7, pp. 1–5, 2011.

[23] R. D. Vipin, ―Character Recognition using Neural Network,‖ Int. J. Adv. trends Comput. Sci. Eng., vol. 2, no. 2278, 2013.

[24] D. R. C. Ankit Sharma, ―Optical Character Recognition using Neural Network,‖ Int. J. Eng. Trends Technol., vol. 4, no. 8, pp. 711–715, 2014. [25] V. Ong and D. Suhartono, ―Using K-Nearest

Neighbor in Optical Character Recognition,‖ IEEE, vol. 7, no. 1, p. 53, 2016.

[26] H. Zhao, Y. Hu, and J. Zhang, ―Character Recognition via a Compact Convolutional Neural Network,‖ 2017.

[27] M. Jangid and S. Srivastava, ―Handwritten Devanagari Character Recognition Using Layer-Wise Training of Deep Convolutional Neural Networks and Adaptive Gradient Methods,‖ J. Imaging, vol. 4, no. 2, p. 41, 2018.

[28] A. Yuan, G. Bai, L. Jiao, and Y. Liu, ―Offline handwritten English character recognition based on convolutional neural network,‖ Proc. - 10th IAPR Int. Work. Doc. Anal. Syst. DAS 2012, pp. 125–129, 2012.