Pedestrian Recognition and Tracking of Dancing Robot Based on Laser Scanner

2016 Joint International Conference on Artificial Intelligence and Computer Engineering (AICE 2016) and International Conference on Network and Communication Security (NCS 2016)

ISBN: 978-1-60595-362-5

Pedestrian Recognition and Tracking of Dancing Robot

Based on Laser Scanner

Zhao LIU

_{, Li-Bin SONG}

_,

_{Mei-Xiao GENG}

_,

Tao YU

_{, Zeng-Xi WANG}

1_{Tianjin Research Institute for Advanced Equipment, Tsinghua University, Beijing, China}

2_{Tsingyan Huayu Robot and Automation (Tianjin) Co., Ltd., Tianjin, China}

a_{[email protected],} b_{[email protected],} c_{[email protected],} d_{[email protected],} e_{[email protected]}

*Corresponding author

Keywords: Pedestrian Recognition, Pedestrian Tracking, Laser Scanner; Dancing Robot, Particle Filter.

Abstract. Dancing robot should plan a dynamic path based on the motion of pedestrians to ensure the safety of others, so a pedestrian recognition and tracking method based on laser scanner is proposed. At first, the data collected by the laser is preprocessed. To segment the environment rapidly, the DBSCAN algorithm is optimized to cluster the data. Then a pedestrian recognition method is proposed by using the distance from cluster to laser scanner and pedestrian width range. At last, the laser data is converted into video data frames. So the particle filter algorithm is adopted to track the moving pedestrian. Experimental results show that the proposed method has a good real-time performance and high application value.

Introduction

Robot technology has achieved amazing development in recent decades. There are some intelligent robots which can replace or assist people in some occasions, such as hospital, restaurant, bank. Because the robot needs to coexist with people in some situations, so they should ensure the safety of people when they are working. Hirata [1] has developed dance partner robot that can direct contact with a dancer. It is of great significance to recognize and track pedestrians in surroundings for the movement and trajectory planning of the dancing robot.

There are lots of method of pedestrian recognition and tracking method based on robots. By analyzing video data to achieve the pedestrian tracking is currently the most commonly used method [2]. However, due to the camera scope, different light and the pedestrian posture change, pedestrian tracking by video information becomes extremely tough. Using laser scanner to realize pedestrian tracking is another common method [3]. Laser scanner is a new measuring instrument which has the characteristics of high precision, quick measurement speed and wide measurement range.

A pedestrian recognition and tracking method is proposed in this paper. The method is implemented on the platform of the dancig robot. The laser scanner is placed in the position of one meter off the ground. The laser scanner is URG of HOKUYO. After preprocessing and clustering of the distance data obtained by the laser scanner, pedestrian recognition and tracking are completed, and the trajectory of pedestrians is drawn at the same time.

Overview of the Method

Figure 1. Flow chart of pedestrian tracking based on laser scanner.

Pedestrian Recognition Based on Laser Scanner

Data Preprocessing

The data collected by the laser scanner is the distance between the object and the laser scanner [4]. The first step is median filter processing which is used to do the source data correction. For a particular data point, the distance data of m points in the neighborhood of this point is sorted. Then make the [(1+m)/2]th distance instead the distance of this point, where m equals to 5. Thus, the points belonging to the same object are concentrated which is good for clustering.

The source data belongs to polar coordinate system. For the sake of facilitating the data analysis, the data is converted to rectangular coordinate system [5]. The source data format is as follows:

, 0,1,...,

k k

s d k N. (1)

Where dk is the distance data of the kth point. Convert all points to rectangular coordinate system:

T

( , ) , 0,1,...,

k k k

u  x y k  N . (2)

Where uk is the rectangular coordinate of the kth point. xkand yk are X axis and Y axis coordinates

respectively.

The angular resolution of the laser scanner is β, and the first scanning angular is 45°. The distance data of the kth point P is dk. So the rectangular coordinate of point P is Eq. 3and Eq. 4:

cos( 45 )

k k

x d k   . (3)

sin( 45 )

k dk k

y     . (4)

Data Clustering

Due to the distribution characteristics of the laser data, a method of data clustering [6] based on density is adopted in this paper. DBSCAN (Density-based Spatial Clustering of Applications with Noise) [7] is a density-based clustering method which is used to find the dense regions among the dataset. Meanwhile, DBSCAN is not easily affected by noise and outliers.

point is named as a core point. Then the core point and the whole points of ε- neighborhood compose of a cluster. If a cluster contains multiple core points, merge the clusters which these core points belongs. After this, the final cluster result is gotten. In this paper, ε is set to 100mm, and MinPts is set to 5.

For DBSCAN algorithm, when getting the core points Pk, the distance between Pk and all the rest

points should be calculated. So the time complexity is too high. To solve this problem, this paper simplifies the calculation of the core points. When finding the core points, only the distance from Pk

to the points around Pk are calculated. Specifically, the calculation region consists of the points from

P(k-MinPts) to P(k+MinPts). After the optimization, the clustering efficiency of DBSCAN algorithm is greatly improved, and the clustering effect did not get worse.

Pedestrian Recognition

The distance between the laser scanner and the object directly affects the number of points on the object. In addition, the relative position of the pedestrian in the process of motion and different states of the pedestrian both have an impact on the number of points on the pedestrian which we can see in Fig.2 (a).

The number of the points on the pedestrian's body is estimated by the distance between the pedestrian and the laser scanner. Fig.2 (b) is the estimation of points number in the pedestrian cluster.

Figure 2. Pedestrian recognition.

The average distance of the cluster can sign as r, and the width of the pedestrian’s waist is Wp.

According to the different states of pedestrians, the range of Wp is Wmin<Wp<Wmax. If r is radius and

Wp is as arc length, then there is an Eq.5.

180 p

W r. (5)

Use Eq.5 to get the central angle θ. The angular resolution β and the central angle θ are combined to determine the point number of the pedestrian cluster. The calculation method is as follows:

180 = c

p r

W P

 



. (6)

Pnum is the point number of the pedestrian cluster. The estimation range of Pnum can be calculated by

the range of Wp which we can see in Eq.7. In this paper, the range of Wp ranges from 200mm to

500mm. Then the pedestrian cluster is gotten by the method described above.

180 _min 180 _max

num

W W

P

r r

   

 

Pedestrian Tracking Based on Laser Scanner

It is difficult to track the pedestrian using the source laser data. Given this, the continuous scanning data is displayed graphically. Actually, the source laser data is converted into a video stream of 10 FPS. This way not only simplifies the process of tracking but also enhances the tracking accuracy.

Graphical Display of Data

The scanning interval of the laser scanner is 100ms. To accelerate the speed of tracking, the source data is displayed in a image by reducing 20 times. All of points are made a mapping to the image to facilitate the tracking more intuitively and efficiently.

Pedestrian Tracking by Particle Filter

After pedestrian recognition, the pedestrian position is regarded as the initial position. Initializing the tracker is the first step. Then extract the feature and do particle filter frame by frame. At last, the pedestrian position of each frame is gotten and the trajectory is drawing at the same time.

Particle filter [8] is adopted to track the moving pedestrian. The number of particles is 100. For the feature extraction, color distribution probability density is used to describe the model of the target region. The color histogram of RGB is counted to represent the color distribution. The pixel which is far away from the center is given a small weight. The weight calculation function is:

2

1 1

( )

0 Others

k R _  R R 

 . (8)

Where R is the distance from the pixel to the region center. The coordinate of the region center is set as X_c( , )x y_{c c} . Moreover, the coordinate of the ith pixel is set as X_i ( , ),x y i_{i i}  1, ,n. Hx and Hy

are width and height of the target region. So the target region can be described by 2 2

x y

a H H . The calculated way of the current state of the target region is as follows:

( )

1

( ) n c i [ ( ) ] ( 1, , )

u

t i

i

X X

p X f k h X u u m

a 





  _ _   

  . (9)

1 1

n _{c i}

i

f

X X

k a 





 _ _

 

. (10)

In the above equations, n is the pixel number of the region, and m is the number of histogram features. [ (h X_i)u] is used to determine whether the color of the position Xi belongs to the uth

feature. If it is true, the value is 1; otherwise, the value is 0.

After getting the color histogram distribution p(Xt) of the current particle, the color histogram

distribution is compared with the color histogram distribution q of the target region by Bhattacharyya distance. The distance calculation method is Eq.11.

( ) ( ) 1

[ ( _t), ] m u ( _t) u

u

d p X q p X q



2

1 [ ( , )]

1 _{exp( )}

2 2

t t

d p X q w







  _{. (12)}

Eq.12 is the weights of the current particle, and it is gotten according to the Bhattacharyya distance in Eq.11. σ is Gauss squared error which equals 1 in the paper.

Calculate the position of the pedestrian in the current frame according to the weight of the particle. According to the above method, the pedestrian in each frame image is tracked and the trajectory of the pedestrian is gotten easily.

Experiment and Analysis

For pedestrian recognition, do recognition frame by frame until the method recognized one pedestrian. Fig.3 is the process of pedestrian recognition in this paper. To describe the process much more vividly, the result is shown graphically.

(a)

Cluster1

Cluster2

Cluster3

Cluster4

(b) (c)

Figure 3. The process of pedestrian recognition.

Fig.3 (a) is the source data of the laser scanner. After data preprocessing, measurement error is reduced. And the coordinate system transformation follows. For data clustering, the DBSCAN algorithm is optimized to speed up the recognition. The average time of clustering is shorted from 2750mm to 65mm, and the clustering result is shown in Fig.3 (b). During the pedestrian recognition, the point number of the pedestrian is estimated using the distance and the body width of a pedestrian. By analyzing every cluster, the pedestrian is filtered that is signed by a white rectangular in Fig.3 (c). Once the proposed method recognized a pedestrian in one frame, the pedestrian position in this frame is the initial tracking position. Then particle filter algorithm is used for pedestrian tracking. Fig.4 is the tracking and trajectory result of a pedestrian in linear and curvilinear motions.

(a) (b)

Fig. 4 (a) is the tracking result of a pedestrian in a linear motion. As the various states of the pedestrian in during the motion, there are some small errors in the process. However, it does not affect the tracking result. Fig.4 (b) is the tracking result of a pedestrian in a curvilinear motion. Because the movement direction of the pedestrian changes rapidly, so the picture can be seen that volatility is evident. As we can see, the algorithm timely corrects the tracking route.

The two groups of experiments show that the tracking algorithm can track a pedestrian rapidly and accurately, and it can rectify the tracking deviation promptly.

Summary

A method of indoor pedestrian recognition and tracking based on laser scanner is studied in this paper. In the first stage, the laser data preprocessing and the optimized DBSCAN algorithm are used to prepare the subsequence processing. Then a pedestrian recognition method is proposed. The method is not sensitive to the different distance or shape of pedestrians. Because all of the laser data are shown as an image, so the particle filter based on color is used to track the pedestrian. Experimental results show that the pedestrian tracking algorithm is proved with the advantages of real-time and high accuracy. The proposed method provides a real basis for dynamic path planning of dancing robot. Pedestrian tracking in the complex scene will be the research priorities of the next phase in this paper.

Acknowledgement

This research was financially supported by the National Natural Science Foundation of China (No. 51005126).

References

[1] Y. Hirata, T. Hayashi, T. Takeda, et al. Step estimation method for dance partner robot MS DanceR using neural network, International Conference on Robotics and Biomimetics, IEEE，2005, pp. 523-528.

[2] Z. Jiang, Q.H. Du, W. Mora, et al. Multiple pedestrian tracking using color and motion models, International Conference on Digital Image Computing：Techniques and Applications, IEEE，2010, pp. 328-334.

[3] T. Horiuchi, S. Thompson, S. Kagami, Y. Ehara, Pedestrian tracking from a mobile robot using a laser range finder, International Conference on Systems, Man and Cybernetics, IEEE, 2007, pp. 931-936.

[4] B. Ling, S. Tiwari, Z. Li, D.R. Gibson, A multi-pedestrian detection and counting system using fusion of stereo camera and laser scanner, Proc., SPIE Optics and Photonics, 7798(2010) 1-20. [5] M. Ogaz, R. Sandoval, M. Chacon, Data processing from a Laser Range Finder sensor for the construction of geometric maps of an indoor environment, 52nd IEEE International Midwest Symposium on, IEEE, 2009, pp. 306-313.

[6] C. Mertz, L.E. Navarro-Serment，R. Maclachian，et al.Moving object detection with laser scanners, Journal of Field Robotics, 30 (2013)1083-1094.

[7] Q.S. Rong, J.B. Yan, G.Q. Guo, Research and Implementation of Clustering Algorithm Based on DBSCAN, Computer Applications, 4 (2004)45-46.