Multi Feature Based Multiple Particle Filters for Object Tracking

2016 International Conference on Artificial Intelligence: Techniques and Applications (AITA 2016) ISBN: 978-1-60595-389-2

Multi-Feature Based Multiple Particle Filters for Object Tracking

Ying-dong MA

*

and Yu-chen LIU

College of Computer Science, Inner Mongolia University, Hohhot, China *Corresponding author

Keywords: Video surveillance, Gaussian mixture modeling, Multiple particle filters.

Abstract. Multiple Object tracking is a fundamental step for many computer vision applications. However, detecting and tracking objects in complex background is still a challenging task. This paper proposes an approach, which combines an improved Gaussian mixture modeling with multiple particle filters for automatic multiple targets detecting and tracking. For Gaussian mixture modeling, we make improvement on Gaussian mixture modeling in the phase of model updating by using the expectation maximization algorithm and M recent frames with weight parameters of Gaussian distributions. In the tracking stage, we integrate multiple features of targets, including color, edge and depth, into multiple particle filters to improve the performance of object tracking. By comparing with various particle filter approaches, the experimental results show that our approach can track multiple targets in complex backgrounds automatically and accurately.

Introduction

Video surveillance systems have been widely used in various applications, such as intelligent transportation system and social public security. However, in most of these systems, multiple targets tracking is still a challenging task. The main challenges of these applications include cluttered backgrounds, fast moving targets, abrupt object motion, and partial or full object occlusions.

In this paper, we propose a multiple object detection and tracking method based on the Gaussian mixture modeling (GMM) and multiple particle filters (MPFs) algorithm. In the first step, we improve the Gaussian mixture modeling method to construct the background model for moving objects segmentation. The original Gaussian mixture modeling, which is proposed by Stauffer and Grimson [1], is a popular algorithm that has been widely adopted for moving object detection. However, the original method has high computational cost and it is sensitive to object shadows. Although researchers have proposed variable approaches to deal with these difficulties [2-8], there are still some problems not fully resolved. A typical example is the work proposed in [4], in which moving object shadows can be removed to reduce the interference of background reconstruction. Some methods improve the speed of background modeling, while they pay the price for inaccuracy [6-8]. By analyzing the GMM algorithm, we found that some pixels may take a long time to be confirmed as background pixels due to the update process of GMM. To tackle this problem, we use the Expectation Maximization algorithm (EM) and the M recent frames to accelerate the update progress of the GMM algorithm.

The particle filter approach has been widely applied in applications of moving object tracking. The main reason is that the algorithm can deal with non-Gaussian situation and is able to manage multiple hypotheses [9-11]. However, the original approach cannot identify individual objects. It has other problems such as the coalescence problem and exponential complexity as the number of tracking objects increasing. Some subsequent approaches tackle these drawbacks by using the multiple particle filters [12,13].

and tracking system. In comparing experiments, the proposed method shows better performance than other MPFs based tracking approaches.

[image:2.595.105.494.112.203.2]

Input image Object detection multi-feature based MPFs Object tracking Figure 1. Object detection and tracking method overview.

Moving Object Detection

An adaptive Gaussian mixture modeling [1] can be constructed using a mixture of K Gaussian distributions to model each pixel in the scene. Although the robustness of Gaussian mixture modeling has been demonstrated, there are some drawbacks of this algorithm, such as slow adaptive speed in the case of sharp background changing. If the first value of a certain pixel is a foreground value, the background component will only have one Gaussian distribution. With one color background values,

it will take log_(1)(T) _{frames until the background pixel is considered as part of background}

component. In this case, the adaptive speed of the means and the covariance matrices can be very slow and it makes the result of background subtraction might be worse after initialization.

In order to tackle this problem, we use the expectation maximization algorithm to improve the performance of Gaussian mixture model. Firstly, we estimate the Gaussian mixture modeling by using new update equations with expected statistics data:





1 1 1

|

t t t

i i i t i

w w p X w

t   _  _   _  _ (1)













1 1 1 | | i t

t t t

i i t t i

i j j p X X p X           



 











1 1 1 1

1

| |

T

i t

t t t t t

i i t t i t i i

i j j p X X X p X  _{ }             _     _  



(2)

where wit is the weight parameter of the ith Gaussian component,

t i

 _{is the mean of the ith Gaussian}

component, ti is the covariance of the ith Gaussian component, p(i|Xt)1 when i is the first

match Gaussian component; otherwise, p(_i|X_t)0, and i is the ith Gaussian distribution of

background component. When the first Mframes are processed, the update equations change to:





1 1 1

|

t t t

i i i t i

w w p X w

M   _ _   _  _ (3)  









1 1 1 | 1 | i t

t t t

i i t t i

t

i i j

j

p X

X M

w p X

                     



  











1 1 1 1

1 | 1 | T i t

t t t t t

i i t t i t i i

t

i i j

j

p X

X X

M

w p X

              _{ }      _     _    





 ₍₄₎

At the beginning, a good estimation of the background model can be obtained by using the new

update equations. After the first M frames are processed, the M recent frames that combine the

Multiple Objects Tracking

The particle filter based tracking methods are widely used in video object tracking applications due to the deterministic sampling approach [9] and its ability of managing multiple hypotheses simultaneously. However, if we apply the particle filters to track multiple targets directly, the particles of a standard particle filter might be collapsed to one same mode. This situation is usually referred to as coalescence or particle “hijacking” problem. Moreover, a joint particle filter has exponential complexity as the number of tracking objects increases. These drawbacks can be overcome by using MPFs. The MPFs contains independent particle filter for each target and it utilizes interaction factor which modifies the weight of particles to overcome the coalescence problem.

The algorithm constructs an initial sample set s(0) of N particles by random sampling and the

particles of the initial sample set have the same weight. In each iteration, the new samples s_i(t) are

obtained by selecting and replacing N samples s' (t 1)

i from s(t1) using the state transition

model. The value of weight parameter _i(t) _{of each particle is computed using the new observations,}

which can be obtained from p(z(t)|si(t)), and the interaction factor i(t). The interaction factor is

used to modify the final weight of a particle according to the distance between it and the rest of targets.

As we can see from the above procedure, the weight parameter of each particle relies on

)) ( | ) ( (zt s t

p _i . In order to improve tracking performance, we propose a multi-feature based MPFs which

adopt color, edge, and depth information of the observations to compute p(z(t)|s_i(t))_.

Color Information

In this work, we use the HSV color space which is invariable to illumination changes. A HSV color histogram contains m bins. The value m can be computed as:

h s v mn n n

(5)

where nh,ns,nvrepresent bins for hue, saturation and luminance, respectively. The color density

distribution for each bin q(u) can be calculated as:

 

 

1 1 n

j j

q u b p u

n  

 





_  _

(6)

where parameter K is the Kronecker delta function and function b():2{1,...,m} _{associates the index}

) (p_j

b _{of the histogram bin.}

The Bhattacharyya distance d_ci(t) can be calculated as:

     *

1 ,

i c

d t   q u q u _

  

  

 

* 1

q u   q u q u

   

*

1 q u q u,

_{  }  

 ₍₇₎

where ρ is the Bhattacharyya similarity measure of two color models. The probability distribution

function of d_ci(t) is calculated as:

     

  2

2

1

| exp

i c i

c i

c c

d t p d t s t

   

 _{ } 

   

   _ _

 

 

  ₍₈₎

where s_i(t) _{is the ith sample of the particle filter sample set, and} _c0.3 _{is determined by}

Depth Information

To compute the probability distribution function, depth information is assumed to be Gaussian

distribution and is managed by the variable i_z(t)~NZ_i(t),Z_i(t) which represents the center of mass

of the tracking object [14]:

 

 

*  *

 

* 1

1 n ti

i i

z z j

j i

t I p t

n t     



_{ } (9) As the depth image usually contains undefined points which do not have depth value. The equation

is only calculated for pixels that have valid depth information. n_i*(t) is the total number of pixels with

valid depth information within target area and pi_j*(t) is the location of pixels with valid depth

information. During tracking process, pixels without depth value may lead variable iz(t) _uncertain.

We use the confidence measure _i(t) _{to tackle this problem. It indicates the proportion of pixels with}

valid depth value in the sample set:

 

i*

 

 

i i n t t n t   (10)

The probability distribution function of zi(t) is computed as:

    _{ }       2 2 1 | exp 2 ₂ i z i i

z i i _i

z _z

t Z t

p u t s t

t _t      _{  }         _ _   _{ }  _{ }          ,   z i i z i i Z t t t t            (11)

where s_i(t) is the ith sample of the particle filter sample set and  is a parameter indicating the

minimum amount of valid depth information that a sample set must have. In this work we use  0.05.

Edge Information

Each pixel has two valuesM(x,y) andA(x,y)_{, where} M(x,y) _{is the gradient amplitude of a pixel and}

K y x

A（ , ） is the gradient direction. The gradient direction angle is divided into K level and the value

of K in our experiment is 8. The edge direction histogram He is:

     

,

, , ,

e x y

H 



M x y _ x y A x y _

(12)

By cumulating the value of each bin in the histogram, the edge direction vector E(e_1,,e_k) _{can be}

computed as an edge model. The similarity of two edge model is obtained by using the Euclidean distance measurement.

2 * 1

K

e i i i

D   e e

(13)

We set a distance threshold Dt 1 to determine when to update edge model. The probability

distribution function of edge model can be computed as

 



2



| exp

i

e k i e e

p _x s t  _  D

(14)

where si(t) is the ith sample of the particle filter sample set,

i k

x _{is edge information of the sample}

Particle Filters with Multiple Features

In this work, each particle filter represents the 3D position of the tracking targets and rotation about the camera’s optic axis:

       , , ,  

i i i i i

s t _X t Y t Z t  t _

(15) The weight of particles is computed by analyzing the color, edge and depth information. As

)] ( | [x s t

p_e i_{k i} , p[d_ci(t)|s_i(t)] and p[uzi(t)|si(t)] can be assumed independently, particles are weighted using

the following equation:

             

     

* | * | ,

|

* | * | * , else

i i

z i c i i

i _{i i}

e k i c i

A p u t s t p d t s t t p z t s t

A p x s t p d t s t

 



     

    



  

  _{   }

 _{   }

 ₍₁₆₎

where A is defined to ensure the sum up of p[z(t)|s_i(t)] is equal to one,



is the parameter which

enforces a smooth transition between the two parts of the equation and is calculated as:

 

1

=

2

i z t 

  zi

 

t z

 

0

 

 

(17) The weight equation is divided into two parts. The first one is applied to particles when the

proportion of valid depth information of pixel is larger than  . The second part is applied to the

situation in which the proportion of valid depth information of pixel is lower than  . In this case the

depth information is considered as unreliable and the MPFs uses color and edge features of tracking objects. The final weight of particle is calculated with the interaction factor:

 

 

*



   

|



i t i t p z t s ti

 

  _{ }

min

1 exp

, 1

i t

d t t





  

 _ _

 

  ₍₁₈₎

In the above equation, dmin(t,t1) is the distance between the particle and the nearest target in time

1



t , and



is a small constant to avoid _i(t) to be 0. When the particle moves to the position of

other targets, _i(t) tends to 0 and decreases the particle weight to avoid the coalescence problem. If

) (t

i

 tend to 1, it means the particle is further from other targets.

Experimental Results

Evaluation of Object Detection

To evaluate performance of the proposed moving objects detection methods, we compare our detecting approach with three different methods: Frame Difference method, Average background method and the original Gaussian mixture model method. The results of different method are shown in Fig. 2.

(a) (b) (c)

[image:5.595.164.431.593.758.2]

(d) (e) (f)

Figure 2(b)-(c) show the result of Frame Difference method. Figure 2(b) is obtained from two frames whereas another image is computed using three frames. Although Frame Difference method can detect moving objects, it cannot get the complete area of each moving objects. Figure 2(d) is the result of average background method. This method is unstable especially in the case that targets are moving fast. Figure 2(e) illustrates the result of original Gaussian mixture model method. Although it is better than the methods analyzed above, this method cannot distinguish shadows from the moving targets. Fig. 2(f) shows the result of our method with morphological operation and connecting space operation. The M recent frame based weight parameter updating algorithm enhances the robustness of the proposed method, even in the case of illumination change.

In some frames, the original Gaussian mixture model method marks the lower left pedestrians as one object due to the shadows and illumination change. The proposed method can adapt faster in the case of sharp scene changing and, therefore, it can deal with this problem. By comparing the results of all methods, the proposed method can acquire better moving objects detecting results.

Evaluation of Object Tracking

We compare our tracking method with three multiple particle filter based tracking methods with different features. These methods use color information, edge information, and depth-color information, respectively. The tracking results of color-based method, color-edge based method and the proposed method are illustrated in Fig. 3 and Fig. 4 (both test data are downloaded from PETS ([15],[16])).

(a)

(b)

[image:6.595.192.404.364.571.2]

(c)

(a)

(b)

[image:7.595.189.408.67.291.2] [image:7.595.205.393.448.569.2]

(c)

Figure 4. Results of color-based method(a), color-edge based method(b) and our method(c).

The results of color-based method and the edge-based method show that these methods can deal with multiple targets tracking without occlusion. However the color-based method only tracks part of the target when it has similar color distribution to the whole target. In this case, edge-based method is better than color-based method. When occlusion occurs, tracking performance of these two methods becomes worse.

Fig.3 (c) and Fig.4 (c) shows that the proposed method has better performance in the case of multiple targets tracking. Because of the depth information, the system can track occluded targets accurately. Moreover, as the proposed method is adaptive to video scenes with or without depth information, it can be applied to track objects in various situations and complex background.

Figure5. Center error measure.

Figure 5 shows the center error measurement of these methods which is the distance between center of the true target area and center of the target indicated by tracking methods. We can find that when the number of particles exceeds 150, there is no significant improvement in tracking performance of all methods. The color-based method suffers the biggest center error, while our method has the best tracking performance.

Conclusions

accurate object tracking than original MPFs approach. The proposed method can be applied to solve the problem of multiple targets tracking, even when targets are getting close or in the case of partial occlusion between tracking targets.

Acknowledgement

This research was supported by the National Natural Science Foundation of China under Grant 61461039.

References

[1] C. Stauffer, W.E.L. Grimson, Adaptive background mixture models for real-time tracking. IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2 (1999), 246-252. [2] H. Alizadeh, A. Khoshrou and A. Zuquete, Traffic classification and verification using unsupervised learning of Gaussian Mixture Models, IEEE International Workshop on Measurements & Networking (2015), 1-6.

[3] K. Vishwajeet and P. Singla, Adaptive splitting technique for Gaussian mixture models to solve Kolmogorov Equation, American Control Conference (2014), 5186-5191.

[4] H. Lin, J. Chuang and T. Liu, Regularized Background Adaptation: A Novel Learning Rate Control Scheme for Gaussian Mixture Modeling, IEEE Transactions on Image Processing 20(3) (2011), 822-836.

[5] D. Mukherjee, Q. M. J. Wu and T. M. Nguyen, Multiresolution Based Gaussian Mixture Model for Background Suppression, IEEE Transactions on Image Processing 22(12) (2013), 5022-5035. [6] S. Toral, M. Vargas, F. Barrero and M. G. Ortega, Improved sigma-delta background estimation for vehicle detection, Electronics letters 45(1) (2009), 32-34.

[7] A. Manzanera and J. C. Richefeu, A new motion detection algorithm based on Σ–Δ background estimation, Pattern Recognition Letter, 28( 3) (2007), 320-328.

[8] M. Vargas, J. Milla, S. Toral and F. Barrero, An enhanced background estimation algorithm for vehicle detection in Urban traffic scenes, IEEE Transactions on Vehicular Technology 59(8) (2010), 3694-3709.

[9] M. Isard and A. Blake, Condensation-Conditional Density Propagation for Visual Tracking, International Journal of Computer Vision 29(1) (1998), 5-28.

[10] A. Yao, X. Lin, G. Wang and S. Yu, A compact association of particle filtering and kernel based object tracking, Pattern Recognition 45(7) (2012), 2584-2597.

[11] G. Kitagawa, Monte Carlo filter and smoother for non-Gaussian nonlinear state space models, Journal of Computational and Graphical Statistics 5(1) (2012), 1-25.

[12] W. Du and J. Piater, Tracking by cluster analysis of feature points using a mixture particle filters, IEEE Conference on Advanced Video and Signal Based Surveillance (2005), 165-170.

[13] Z. Khan, T. Balch and F. Dellaert, MCMC-based particle filtering for tracking a variable number of interacting targets, IEEE Transactions on Pattern Analysis and Machine Intelligence 27(11) (2005), 1805-1819.

[14] R. Muñoz-Salinas, E. Aguirre, M. García-Silvente and A. Gonzalez, A multiple object tracking approach that combines colour and depth information using a confidence measure, Pattern Recognition Letters 29(10) (2008), 1504-1514.