Automatic Fuzzy License Plate Recognition Based on Deep Learning

2017 2nd International Conference on Computer Engineering, Information Science and Internet Technology (CII 2017) ISBN: 978-1-60595-504-9

Automatic Fuzzy License Plate Recognition

Based on Deep Learning

XUEFENG TANG and PING ZHOU

ABSTRACT

At present, the clear license plate recognition algorithm has been mature, but for the blurred license plate which cannot be recognized by the human eye, the recognition rate of the traditional license plate recognition algorithm is too low. In view of this, a license plate character recognition algorithm based on convolutional neural network structure is proposed. In this study, a large number of training samples of fuzzy characters are constructed, and a convolutional neural network is trained for blind segmentation of fuzzy license plate characters. By calling the trained convolutional neural network, we can recognize the characters after blind segmentation. The experimental results show that the training set accuracy is close to 100%, and the accuracy of the test set is more than 93%. It can recognize the license plate characters which cannot be recognized by the human eye.

KEYWORDS

Convolution neural network; license plate recognition; fuzzy; sample.

INTRODUCTION

In the 90s of the last century, China began to study its own license plate recognition system, but most of these license plate recognition systems can only identify simple restricted scenes, such as parking lots, traffic signs and other [1]. In the actual traffic conditions and the environment is more complex, the picture blurred. Even preprocessing and two valued images cannot be well recognized, not to mention extracting character images. For example, Jia Xiaodan [2] proposed a new method of license plate character segmentation based on prior knowledge, but in his paper, the license plate can be clearly seen by the human eye recognition, but the image quality is much better than that of the plate more.

In the context of criminal investigation and some special applications, we are concerned with the key plates that cannot be identified by the human eye. The existing license plate recognition system is only suitable for the license plate which is easy to be recognized. Although some scholars have studied low quality license plate recognition, their research is based on some simple scenes. Their algorithms are far removed from the characters that other people's eyes cannot recognize. For example, Wan Yan proposed a two stage license plate character recognition method based on poor quality images, but the quality of the license plate is better than that of this paper _________________________________________

[3]. Figure 1 shows the quality differences between the car license plate and other research plates.

EXPERIMENTAL PRINCIPLE

Make A Large Number of Fuzzy License Plate Characters Training Set

At present, there are no fuzzy character sets, and the training of convolutional neural networks requires a large number of fuzzy sample sets. In order to solve the problem of obtaining training samples, this paper studies the degradation model of license plate characters, simulates the degradation process of characters, and generates the simulated license plate characters set.

The blurred image caused by surveillance video results in the fact that the human eye cannot recognize the characters of the license plate, mainly including defocus blur, motion blur and down sampling.

1) Defocus blur

Focusing is an important problem in imaging systems. Defocus is caused by different shooting conditions or different depths of target regions. This blur is called “defocus blur”.

2) Local motion blur

When shooting moving targets, which means that in the process of camera exposure, the target object in the static background in front of mobile, then after imaging, only the target region image motion blur, the background is still clear, this is called "fuzzy partial motion blur".

3) Down sampling

Because the surveillance camera usually monitors the scene in a wide range, the angle of view is generally larger, and it can achieve large area surveillance at the same time. The target object is only a small part of the surveillance scene. For example, the number of pixels in the license plate area is less, the license plate characters are very small, and a lot of information is lost.

To build a training set of degraded license plate characters by simulating the actual scene, we must study the degradation factors of the license plate characters in the actual scene, and establish a model that is close to the actual degradation factors as far as possible.

a) The more clear license plate images in other studies.

(b) Low quality images studied in this paper.

g(x,y) f(x,y)h(x,y) (1)

If there is additive noise, it is expressed as:

) , ( ) , ( ) , ( ) ,

(x y f x y h x y n x y

g    (2)

) ,

(x y

h is point spread function. Fig. 2 is a training set of generated license plate characters.

License Plate Character Recognition Based on CNN

LICENSE PLATE CHARACTER RECOGNITION PROCESS BASED ON CNN

The license plate character recognition method based on CNN is composed of two parts: the first part, the input training image set training CNN classification model; the second part will be to identify the license plate image input the trained model for recognition. The method flow is shown in figure 3.

Number: 3 Letter: A Chinese characters: Zhe FIGURE 2. Generation of license plate characters training set.

After treatment training set

Training learning

Well-trained CNN

Test set

Image preprocessing

Call CNN recognition

Output recognition results

CARRIES ON THE IMAGE PRETREATMENT TO THE INPUT LICENSE PLATE

The original character image cannot be directly used for training and recognition of CNN, so it is necessary to preprocess the original color image so that it can be easily identified. Preprocessing includes blind segmentation, de-drying, gray scale, normalization and other preprocessing operations. Since most of the preprocessing in this article is similar to the existing preprocessing operations, the blind segmentation is only described below.

Traditional character segmentation methods include template matching method [4], horizontal projection method [5], clustering analysis [6], etc. Because the fuzzy license plate texture and feature are not obvious, the traditional license plate segmentation algorithm is difficult to segment characters.

In this paper, the license plate is segmented according to the character position and the proportion of each character in Chinese national standard. In this way, without considering the texture features of the license plate characters, each character can be independently segmented. Figure 4 shows the segmentation effect.

CONVOLUTIONAL NEURAL NETWORKS IN THIS PAPER

Convolutional neural network is a multilayer feedforward network, each layer consists of a number of convolution feature maps, each of which consists of multiple neurons [7]. The input is a two-dimensional matrix, after the transfer to the convolution convolution kernel function layer network layer (C layer) and sampling layer (S layer), convolution layer and sampling layer alternately, output layer feedforward network full link, the dimension of the output layer is determined by the number of classification tasks in [8].

In this paper, N*N pixel input images and convolution kernels are used. The correction linear unit (ReLU) is chosen as the activation function. Unlike other activation functions, ReLU does not require pre-training. Compared to other functions, ReLU runs very fast and has higher accuracy. Therefore, ReLU has gradually become the mainstream activation function.

a) Original license plate image

binary classifier. In this study, the license plate characters were mutually exclusive, and a picture could not belong to one character and another character. So this paper chooses softmax regression.

As shown in Figure 4 network structure as an example, using the ReLU activation function, at the same time, the introduction of the convolution step shown in Figure 5 in the convolution process, it can reduce the network parameters in large-scale network at the same time.

Figure 2.2.3 network input for a 28 * 28 image, Conv1 volume layer by 20 4 * 4 convolution with the input image convolution, convolution step is 2, 20 13 * 13 feature maps; Conv2 layer volume, convolution kernel size is 3 * 3, convolution step 2, after the 50 convolution 6 * 6 feature maps; Conv3 volume layer convolution kernel size is 3 * 3, the convolution step is 1, after convolution generated 60 4 * 4 Pool1 for feature maps; sampling layer, the maximum pool, step 2, obtained after sampling 80 2 * 2 feature maps; Ip1 is the connection layer for the output layer; Output.

Input chart size 28*28

Conv2: convolution kernel 3*3, step size: 1, output: 60, activation function: ReLU

Pool1: convolution kernel 2*2, step size: 1, output: 80, activation function: ReLU

Ip1: full connection layer

Output

Conv1: convolution kernel 4*4, step size: 2, output: 50, activation function: ReLU

FIGURE 5. Network structure.

Step：1 Step：2

EXPERIMENTAL RESULTS AND ANALYSIS

Training of Convolutional Neural Networks

In the experiment, epoch was set to 400, the learning rate was 1, and 90 images were studied at each iteration. Each epoch needs 108 iterations, and the total number of iterations is 43200.

As you can see from figure 7, the abscissa represents the number of iterations

2

10

 _{, and the ordinate indicates the recognition accuracy. With the increase of} iterations, the accuracy of network identification is also increasing. In more than 10000th times, the accuracy rate has been greatly improved. After 20000 times, the accuracy of network identification is over 90%. After that, the rate of recognition increases steadily, and the recognition rate is 99.8%.

Performance of Training Set and Test Set

As shown in Figure 8, the accuracy of the training set is over 99%, and the accuracy of the test set is over 93%.

FIGURE 8. Accuracy of training set and test set.

FIGURE 9. Recognition results.

The Results of License Plate Recognition

Figure 9 is the recognition result. From the recognition results, good results have been achieved. Theoretically, no matter how vague the image is, as long as there are characteristics, it can achieve the purpose of identification by training the network. In fact, the effect is not very good for some images that are too vague and few features.

CONCLUSION

ACKNOWLEDGEMENTS

Corresponding author: Xuefeng Tang, 584870606@qq.com, Information Institute, Zhejiang Science and Technology University, Zhejiang Hangzhou 310018 China.

REFERENCES

1. Feng Yu, "Research on Chinese Character Recognition of Low Quality License Plat", Liaoning Normal University, 2011.

2. Jia Xiaodan, "Research on Segmentation of Degraded License Plate", Liaoning Normal University, 2008.

3. Wan Yan and Liu Wei, "A Two-stage License Plate Character Recognition Method Based on Poor Quality Images", 2012.

4. Cheng Guangtao, Chen Xue, Zhang Wenzhi. License plate character segmentation method based on vertical projection and template matching [J]. Journal of North China Institute of Aerospace Engineering, 2013, 01:19-21.

5. Yongshun Hou, Xunhui Qin, Xiangdong Zhou, Xi Zhou, Ting Zhang, "License plate character segmentation based on stroke width transform", Image and Signal Processing (CISP) 2015 8th International Congress on, pp. 954-958, 2015.

6. Zouaoui Abderaouf, Benblidia Nadjia, Oukid-Khouas Saliha, "License plate character segmentation based on horizontal projection and connected component analysis", Computer Applications & Research (WSCAR) 2014 World Symposium on, pp. 1-5, 2014.

7. Li Shanshan, Li Yimin, Guo Guo. Research on the algorithm of hierarchical license plate character recognition based on neural networks [J]. Industrial instrumentation and automation, 2016, 46 (2): 7 - 10.