Analysis of Two Intermediate Layers in Neural Network on Optical Character Recognition: A Result Analysis

(1)

International Journal of Emerging Technology and Advanced Engineering

Website: www.ijetae.com (ISSN 2250-2459,ISO 9001:2008 Certified Journal, Volume 4, Issue 5, May 2014)

374

Analysis of Two Intermediate Layers in Neural Network on

Optical Character Recognition: A Result Analysis

Shital Patil

1

, Prof. G. K. Andurkar

2

1,2

E&TC Department, Government College of Engineering, Jalgaon, India

Abstract— Handwritten Character Recognition, usually abbreviated to HCR, is the mechanical or electronic translation of images of handwritten document into machine-editable text. An emerging technique in this particular application area is the use of Artificial Neural Network implementations with networks employing specific guides (learning rules) to update the links (weights) between their nodes. Such networks can be fed the data from the graphic analysis of the input picture and trained to output Characters in one or another form. Specifically some network models use a set of desired outputs to compare with the output and compute an error to make use of in adjusting their weights.

Keywords— OCR, Handwritten Character Recognition (HCR).

I. INTRODUCTION

Handwritten Character Recognition is the mechanical or electronic translation of images of handwritten Characters (usually captured by a scanner) into machine-editable form. Handwritten Character recognition has variety of applications in various fields like reading postal zip code, passport number, employee code, bank cheque, and form processing. Handwritten Character recognition is an important component of character recognition system. The problem of the handwritten Character recognition is a complex task due to the variations among the writers like style of writing, shape, stroke etc. Compared to the problem of printed Character recognition, the problem of handwritten Character recognition is compounded due to variations in shapes and sizes of handwritten characters.

Handwritten Character recognition can be differentiated into two categories i.e. Online Handwritten Character recognition and Offline Handwritten Character recognition. On-line handwritten Character recognition deals with automatic conversion of Characters, which are written on a special digitizer, tablet PC or PDA where a sensor picks up the tip movements as well as pen-up/pen-down switching [1]. Off-line handwritten Character recognition deals with a data set, which is obtained from a scanned handwritten document. Though academic research in the field continues, the focus on handwritten Character recognition has shifted to implementation of proven techniques.

Handwritten Character recognition (using optical techniques such as mirrors and lenses) and digital character recognition (using scanners and computer algorithms) were originally considered separate fields. Because very few applications survive that use true optical techniques, the handwritten character recognition term has now been broadened to include digital image processing as well. For more complex recognition problems, intelligent character recognition systems are generally used which generally deals with the non cursive handwritings .

A System for Optical Character Recognition (OCR) has been a frontline research area in the field of human-machine interface for the last few decades. Recognition of Indian language characters has been a topic of interest for quite some time. The need for efficient and robust algorithms and systems for recognition is being felt in India, especially in the post and telegraph department where OCR can assist the staff in sorting mail [2].

II. BACKGROUND

Handwritten Character recognition comes under the category of OCR. In this dissertation ,multiscript feature has been included in the recognition system so as to increase the capability.

 In 1929 Gustav Tauschek obtained a patent on OCR in Germany, followed by Handel who obtained a US patent on OCR in USA in 1933 (U.S. Patent 1,915,993). In 1935 Tauschek was also granted a US patent on his method (U.S. Patent 2,026,329).

 Tauschek's machine was a mechanical device that used templates[13]. A photodetector was placed so that when the template and the character to be recognized were lined up for an exact match and a light was directed towards them, no light would reach the photodetector.

 In 1950, David H. Shepard, a cryptanalyst at the Armed Forces Security Agency in the United States, was asked by Frank Rowlett, who had broken the Japanese PURPLE diplomatic code, to work with Dr. Louis Tordella to recommend data automation procedures for the Agency. [1,20].

(2)

International Journal of Emerging Technology and Advanced Engineering

375

The second system was sold to the Standard Oil

Company of California for reading credit card imprints for billing purposes, with many more systems sold to other oil companies.

 Other systems sold by IMR during the late 1950s included a bill stub reader to the Ohio Bell Telephone Company and a page scanner to the United States Air Force for reading and transmitting by teletype typewritten messages. IBM and others were later licensed on Shepard's OCR patents.

III. BASIC THEORY

The concept of Neural Networks is highly inspired by the recognition mechanism of the human brain[1,20]. The human brain is a complex, nonlinear and parallel computer, whereas the digital computer is entirely the opposite, it is a simple, linear and serial computer. The capability to organize neurons to perform computation is many times faster than a modern digital computer in existence today. Human vision is a good example for understanding this difference.

A. Neural Network

[image:2.595.60.263.502.661.2]

They have the task to receive input data from other neurons or external sources and use this to compute new data as output for the neural network or input data to the neurons of the next layer. Communication channels, better known as the weights, carry the received or computed data. The weights, which connect two neurons posses certain, values and will be adjusted upon network training[19].

Fig 1: Structure of a simple Fully-Connected Neural Network with four layers

Network model is shown Fig 1. In this project, four layers were used where the layers are organized as follows; the first layer is the input layer that receives data from a source. The fourth is the output layer that sends computed data out of the neural network.

The second and third layer is called hidden layer, whose input and output signals remain within the neural network, see Fig 1. In the particular example, the network is fully connected, which means that every neuron in one layer is connected with all neurons in the preceding layer and so on.

B. Learning Process

Learning processes are important to the neural networks. These sets of rules formulate how the weights of a network are to be adjusted [11]. From a higher perspective these rules can be considered to be the mechanism that makes the networks learn from their environment and improve their performance accordingly. A key feature of Neural Networks is an iterative learning process in which data cases (rows) are presented to the network one at a time, and the weights associated with the input values are adjusted each time[13].

Advantages of neural networks include their high tolerance to noisy data, as well as their ability to classify patterns on which they have not been trained. If networks are trained carefully, networks can exhibit some capability for generalization beyond the training data.

For example how much is it going to Intrusion detection using neural networks rain the next year? Based on the rain behaviors for the passed 30 years a neural network, if trained with the proper learning algorithm, which is determined by the analysis and preprocessing of the data, can produce a reasonable prediction on how much it is going to rain the next year.

Another field, that is quite more interesting and way more challenging, is the stock market. Undoubtedly, investors will be interested in knowing how the value of a stock is going to develop in the short and long term. Based on the former developments of the stock for the passed years, a neural network model can be trained to predict when a stock will yield the largest profit. But the stock market is a very complex field and it is doubtful that such a model will be invented [1].

The definition of the learning process implies the following sequence of events:

1.The Neural Network is stimulated by an environment.

2.The Neural Network undergoes changes in it parameters because of stimulation.

3.The Network responds in a new way to the environment due to the changes that occurred in its internal structure.

(3)

International Journal of Emerging Technology and Advanced Engineering

376

IV.TECHNICAL OVERVIEW

[image:3.595.55.278.279.522.2]

The HCR software breaks the image into sub-images, each containing a single number. The sub-images are then translated from an image format into a binary format, where each 0 and 1 represents an individual pixel of the sub-image. The binary data is then fed into a neural network that has been trained to make the association between the number image data and a Unicode value that corresponds to the number. The output from the neural network is then translated into editable number and saved as a file.

[image:3.595.53.274.547.666.2]

Fig 2: Technical Overview of Handwritten Character Recognition

Fig 3: Steps involved in the Character recognition process

Fig 2 demonstrates of a neural network used within an handwritten Character recognition (HCR) application. The original document is scanned into the computer and saved as an image. [8]

Fig 3 shows the steps in HCR. Initially the scanned document is divide into individual lines. These individual lines are composed of Character digits and blank spaces.

Now, these Character digits are encapsulated in a minimum possible rectangle called glyphs. When analysed, these glyphs are made of black and white pixels comprising the shape of the Character digits. Here, black pixel denotes 1 and the white background denotes 0.These combination of zero and one is then fetched as a input data in the neural network where the activation function is implemented on every neurons, so as to process input data. Finally, a Unicode is generated by the output neurons which resemble the designated Character digital.

V. ALGORITHM

The MLP learning algorithm for two hidden layer is outlined below.

1.Initialize the network, with all weights set to random numbers between –1 and +1.

2.Present the first training pattern, and obtain the output.

3.Compare the network output with the target output. 4.Propagate the error backwards.

(a) Correct the output layer of weights using the following formula.

Wh2o =Wh2o + (ηδoOh2) Where Wh2o is the weight connecting hidden unit h2 with output unit O, η is the learning rate, Oh2 i s the output at hidden unit h2. δo is error given by the following.

δo = Oo( 1 - Oo) ( To - Oo) (b) Correct the input weights using the following formula.

Wh 1 h 2= Wh 1 h 2 + ( η δh2Oh1) Where Wh 1 h 2 is the weight connecting two hidden layers with , Oh1 is the input at node of the second hidden layer, η is the learning rate. δh2 is calculated as follows.

δh2 = Oh2(1- Oh2)Ʃ(δoWh2o) (c) Correct the input weights using the following formula.

Wi h 1= Wi h 1 + ( η δh1Oi) Where Wi h 1 is the weight connecting node i of the

input layer with node of the first hidden layer, Oi is the input at node of the hidden layer, η is the learning rate.

δh1 is calculated as follows.

δh1 = Oh1(1- Oh1) Ʃ(δoWh1h2)

5.Calculate the error, by taking the average difference between the target and the output vector.

6.Repeat from 2 for each pattern in the training set to complete one epoch.

(4)

International Journal of Emerging Technology and Advanced Engineering

377

VI. RESULT ANALYSIS

Experiments were performed on different samples having mixed scripting languages on characters using single and double hidden layer. Following Table 1 shows this analysis.

This result analysis shows that handwritten character recognition system using double hidden layer is better than single.

TABLE I RESULT ANALYSIS

SAMPLE NO.

Multiscript HNR Single Hidden Layer

Output

Multiscript HNR Double Hidden Layer

Output Correc

tly Recogn ized charact er

Accur acy

Corr ectly Reco gnize d chara cter

Accur acy

Sample 1 (Arial Font Characters )

42/48 87.50 %

47/48 98%

Sample 2 (Comic Font Characters)

23/29 87.33 %

29/29 100%

Sample 3 (Roman Font Characters )

40/48 85% 47/48 98%

Sample 4 (Tahoma Font Characters)

42/48 87.50 %

48/48 100%

Fig.4: Comparision Graph Chart

VII. CONCLUSIONS

The The proposed method for the Multiscript Handwritten Character Recognition using multilayer perceptron algorithm may be shows the remarkable enhancement in the performance when two hidden layers are used. If the accuracy of the results is a critical factor for an Character recognition application, then the network having many hidden layers should be used but if training time is a critical factor then the network having single hidden layer (with sufficient number of hidden units) should be used. The number of hidden layers is proportional to the number of epochs. This means that as the number of hidden layers is increased, the training process of the network slows down because of additional branching of weight adjustment. However, the training of the network will be more accurate if more hidden layers will be used. This accuracy may be achieved at the cost of network training time.

REFERENCES

[1] S. M S.Rajsekaran ,G.A.Vijayalakshmi Pai ”Neural Network Fuzzy Logic,and Genetic Algoritms”, Prentice Hall India,2007 [2] Bremananth R, Won Prakash A “Tamil Numerals Identification”

IEEE conference on Advances in Recent Technologies in Communication and Computing”, November, 2009.

(5)

International Journal of Emerging Technology and Advanced Engineering

378

[4] Umapada Pal, Kaushik Roy,Fumitaka Kimura “Bangla

Handwritten pin code string for Indian postal

automation“ Eleventh international conference on frontiers in handwritten recognition (ICFHR) Canada, 2008 .

[5] S.V. Rajashekararadhya P. Vanaja Ranjan , “Efficient Zone based features extraction algorithm for hand written numeral recognition of four popular south Indian scripts”, Journal of Theoretical and Applied Information Technology,Vol.17,no.1,2008.

[6] K. Roy, S. Vajda ,”A System towards Indian Postal Automation”, 9th International Workshop on Frontiers in Handwriting Recognition (IWFHR-9 ) 2004.

[7] M. I Razzak, S.A.Hussain, A. Belaid, “Multi-font Numerals Recognition for Urdu Script based Languages”, International Journal of Recent Trends in Engineering, Vol 2, No. 3, November 2009.

[8] Dong Xiao Ni,Proceedings of Students/Faculty Research Day,CSIS , Pace University,”Application of Neural Network to character recognition”, May 2007.

[9] Hirokazu Muramatsu, Takashi Kobayashi,” Improvement of Matching and Evaluation in Handwritten Numeral Recognition Using Flexible Standard patterns”, Seventh IEEE International Conference on Document Analysis and Recognition (ICDAR) 2003.

[10] Marco Alfonse, Mohamed Almorsy, Mohamed Samir Barakat, “Eastern Arabic Handwritten Numerals Recognition”, International Journal of Computer and Electrical Engineering, Vol. 2, No. 2, April, 2010.

[11] B.V.Dhandra, R.G.Benne, Mallikarjun Hangarge. “Kannada, Telugu and Devanagari Handwritten Numeral Recognition with Probabilistic Neural Network: A Novel Approach” ,IJCA Special Issue on “Recent Trends in Image Processing and Pattern Recognition RTIPPR, 2010.

[12] A. S. M. Mahbub Morshed , Md. Shahidul Islam ,”Automatic sorting of mails by recognizing handwritten postal codes using neural network architecture” International Conference on Communications and Information Technology 2007.

[13] Reza Gharoie Ahangar ,Farajpur Ahangar, ” Farsi Character Recognition using Artificial Neural Network” IJCSIS International Journal of Computer Science and Information Security,Vol.4, no.1&2,2009

[14] Altun,H.,Curtis,KM,”An improved neural network learning”. Proceeding of Fourth IEEE International Conference on Electronics, Circuits and Systems,VOL.1,Page29-33.

[15] Kurt Alfred Kluever ,”Introduction to Artificial Intelligence OCR using Artificial Neural Networks” Golisano College of Computing and Information Sciences Rochester Institute of Technology February 18, 2008.

[16] Sebastian Seung,9.641 Lecture 4:September 17,2002. Multilayer Perceptron and Back Propagation Learning.

[17] J. L. Mcclelland and D.E. Rumelhart, "Learning Internal Prepresentation by Error Propagation" Parallel Distributed Processing, Vol. 1, 2003.

[18] Anita Pal and Daya Shankar Singh,” Handwritten English Character Recognition using NeuralNetwork”, International Journal of Computer Science and Communication,Vol 1.No.2, July-December, 2010.

[19] Liana M. Larigo, ”Off-line Arab Character Recognition : A Survey “, Member IEEE, and Venu Govinraju, Senior Member, IEEE ,1997.