Fast Moving Object Segmentation Based On Active Contours

(1)

Fast Moving Object Segmentation Based On

Active Contours

Hui-min Zhang 1,2_{, Zhi-yong Duan}1,2_{, Zhong-jie Zhu} 1,2+_{, Yuer Wang}2

1.Physical Engineering College, Zhengzhou University, Zhengzhou 450001, China 2.Ningbo Key Lab of DSP, Zhejiang Wanli University, Ningbo 315100, China

(+:correspondence author, E-mail: [email protected])

Abstract—Active contour method is widely used in the image processing field. Recently, it has been used in object segmentation and has attracted great attention. However, most of the existing object segmentation methods based on active contours are complex and time-consuming. They cannot be used in some real-time applications. Hence, in this paper, a fast and efficient moving object segmentation algorithm based on an improved active contour model is proposed. Firstly, moving regions are detected based on frame differences. Then their boundaries are extracted and used as the coarse objects’ contours for the next segmentation based on active contour model. Thirdly, an improved active contour method is introduced to extract the final objects with the coarse objects’ contours as initial values. Experiments are implemented and the results show that the proposed algorithm can segment and track moving targets quickly and correctly.

Index Terms—moving object segmentation, frame difference, active contour model, fast algorithm.

I. INTRODUCTION

Automatic segmentation of objects from video sequences is required for a variety of applications ranging from video compression to object recognition. For example, video compression standard MPEG-4 need automatic segmentation tools to support the object-based coding and description of video contents. Numerous techniques for video object segmentation have been developed. Representative approaches included optical flow method [1], frame difference method [2,3], Markov random filed model-based method [4,5] and the active contour method[6]. Optical flow method and Markov random filed model-based method are complex and time consuming. So they can hardly be used in real-time applications [7]. Frame difference method is fast and simple. However, its segmentation accuracy is poor.

Recently, active contour based on object segmentation has attracted great attention due to its good segment results[8,9]. However, most of the methods are firstly proposed for image segmentation and cannot be directly used for moving object segmentation. In addition, for an active contour method, the initial contours of objects should be given, whose accuracy have a great impact on the segment results. In traditional algorithms, the initial contours are usually designated randomly or by hand,

which usually cost much time and increase the algorithm’s complexity.

In order to overcome the above-mentioned drawbacks of traditional active contour methods, a novel algorithm for fast moving objects segmentation is proposed. Firstly, frame differences are used to detect initial moving regions. Secondly, boundaries of the moving regions are extracted and are designated as the initial object contours. Finally, an improved active contour model is employed to implement object segmentation.

In the proposed algorithm, because the initial contours are detected based on frame-difference, which are usually close to the final objects’ boundaries, the following segmentation can be only performed in a narrow band near the initial contours. Hence, the iteration times and computational load can be greatly reduced during active contour based segmentation. In addition, our algorithm is automatic without the need of man-interference. Hence, compared with traditional methods, our novel algorithm is more fast and efficient. The diagram of the proposed method is shown in Fig.1. The remainder of this paper is structured as follows. Section 2 introduces the process of motion detection. Section 3 describes the spatial segmentation based on an improved active contour method. Partial experimental results are given in Section 4. Section 5 concludes the paper.

II. MOTIONDETECTION BASED ON FRAME

DIFFERENCE

The aim of motion detection is to detect moving regions and determine initial objects’ contours. In this paper, we employ the fast and simple frame difference method to detect moving regions. In order to extract complete moving regions, multiple frames are used. The steps are shown in Fig.2 where the value of

N

is set to 4

in this paper. Firstly, frame differences

d

_k

(

x

,

y

)

,

1 (

k

=

N

between two consecutive frames are calculated:

N

k

y

x

I

y

x

I

y

x

d

_k

(

,

)

=

|

_k₊₁

(

,

)

−

_k

(

,

)

|

=

,1

,

(1)

(2)

case is not always true [10]. Hence, a threshold T is given to binarize d_k(x,y)

⎭

⎬

⎫

⎩

⎨

⎧

≤

>

=

T

y

x

d

T

y

x

d

y

x

b

k k k

)

,

(

,

0 )

,

(

,1

)

,

(

(2)

If bk(x,y)=1it means the kth pixel belongs to moving regions. Generally, the moving regions detected from a single frame difference

d

k

(

x

,

y

)

often contain holes and

are incomplete. Therefore, in this paper, multiple binary masks

b

_k

(

x

,

y

)

are merged to acquire a more complete

result:

)

,

(

)

,

(

)

,

(

)

,

(

)

,

(

3 2 1

y

x

b

y

x

b

y

x

b

y

x

b

y

x

B

k

∪

⋅⋅

⋅

=

(3)

where

B

(

x

,

y

)

is the detected moving regions based on

multiple frame-differences.

To further enhance the detected results, morphological post-pressing is implemented. Two operations, dilation and erosion, are employed to smooth the moving regions, which are defined as[10]:

}

)

,

(,

)

,

(

),

,

(

max{

)

,

)(

(

β

∈

−

=

⊕

b

a

D

b

y

a

x

b

y

a

x

B

y

x

B

B (4)

}

)

,

(

,

)

,

(

),

,

(

min{

)

,

)(

(

β

∈

+

=

Θ

b

a

D

b

y

a

x

b

y

a

x

B

y

x

B

B (5)

where

D

B is the domain of image

B

(

x

,

y

)

and

β

is the

structuring element which is a 3×3 rectangle in this paper.

III. SPATIAL SEGMENTATION BASED ON IMPROVED

ACTIVE CONTOUR MODEL

A. Improved Active Contour Model

Active contour models were firstly introduced by Kass, Witkin and Terzopoulos[11]. In this paper, the region-based C-V model is used[12], which has the following form:

∫

Ω Ω

−

+

−

+

⋅

+

⋅

=

2 1 2 2 2 2 1 1 2 1

|

)

,

(

|

)

,

(

|

))

(

)

(

)

,

(

dxdy

c

y

x

I

dxdy

c

y

x

I

C

inside

Area

C

Length

C

c

F

λ

ν

μ

(6)

where

C

is the closed evolving curve,

Length

(

C

)

is its length,

Area

(

inside

(

C

))

is the area inside the curve,

μ

,

ν

,

λ

₁ ,

λ

₂ are constants and

0 ,

,

0 ,

ν

≥

λ

1

λ

2

>

μ

,

Ω

₁ and

Ω

₂ are pixel subsets of the inside and outside the closed curve, respectively.

The above C-V model does not take gradient information into account. Though it can segment some images precisely, its segmentation accuracy is poor for some images whose boundaries are not obvious. To address this issue, we add an edge term into the energy function, which is defined as:

2

|

)

,

(

*

|

1

1 )

,

(

y

x

I

G

y

x

g

σ

∇

+

=

(7)

where G_σ is a Gaussian function whose variance is

σ

,

* is convolution operator, and

∇

is the spatial gradient

Image sequence

Moving region detection based on frame differences

Initial comtours

Spatial segmentation based on improved active contour method

Moving objects

Object tracking in following frames

end

Figure 1. Diagram of the proposed scheme

...

(3)

operator. The edge term is strictly positive in homogeneous regions and is near zero at the edges[13].

If the noise pollution is trivial, the equation (7) can be simplified as:

2

|

)

,

(

|

1

1 )

,

(

y

x

I

y

x

g

∇

+

=

(8)

By adding the edge term, the energy function can be formulized as: 2 2 2 2 2 1 1 2 1 | ) , ( * | 1 1 | ) , ( | | ) , ( | )) ( ( ) ( ) , , ( 2 1 y x I G dxdy c y x I dxdy c y x I C inside Area C Length C c c F σ

λ

ν

μ

∇ + + − + − + ⋅ + ⋅ =

∫

Ω Ω (9)

B. Level Set Solution to the improved C-V model

Level set method is firstly introduced by Osher and Sethian. It is a numerical approach that works on the topological changes of the evolving interface in an implicit form. In the level set method, a two-dimensional closed curve is viewed as the zero level set of a three-dimensional continuous surface

φ

(

x

,

y

)

called level set

function [14]. Based on level set formulation, the length of a curve C and its interior area can be described as:

∫

Ω Ω

Φ ∇Φ

= ∇

= H dxdy dxdy

C

Length( ) | |

δ

₍ ₎ | | (10)

∫

Ω

Φ

= H dxdy

C inside

Area( ( )) ( )

(11)

where H is the Heaviside function and

δ

(

x

)

is the Dirac

function. They are defined as:

⎩

⎨

⎧

<

≥

=

0 ,

0

0 ,

1 )

(

x

H

(12)

( )

H

(

x

)

dx

d

x

=

δ

(13)

Inserting equations (11) , (12) , (13) into (9) yields

∫

Ω Ω Φ + Φ ∇ Φ = dxdy H dxdy C c c F ) ( | | ) ( ) , , ( 1 2

ν

δ

μ

∫

Ω Ω Φ − − + Φ − + dxdy H c y x I dxdy H c y x I )) ( 1 ( | ) , ( | ) ( | ) , ( | 2 2 2 2 1 1

λ

2 | ) , ( * | 1 1 y x I G_σ ∇ +

+ (14)

where

c

₁ and

c

₂ are given by:

∫

Ω Ω Φ Φ = dxdy H dxdy H y x I c ) ( ) ( ) , ( 1 ε ε (15)

∫

Ω Ω Φ − Φ − = dxdy H dxdy H y x I c )) ( 1 ( )) ( 1 )( , ( 2 ε ε (16)

Based on functional theory, to get the optimal value of

F

with respect to

φ

, the following Euler-equation

should be satisfied[15]:

=

0 ∂

∂

φ

F

(17) where

φ

∂

F

can be calculated by:

] | ) , ( * | 1 1 ) ) , ( ( ) ) , ( ( | | )[ ( 2 2 2 2 2 1 1 y x I G c y x I c y x I F σ ε

λ

ν

μ

δ

φ

∇ + + − + − − − Φ ∇ Φ ∇ ⋅ ∇ Φ = ∂ ∂ (18)

Based on gradient descent method, the above problem equals to solving the following PDE equations:

⎪ ⎪ ⎪ ⎪ ⎪ ⎩ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎧ Φ = Φ ∇ + + − + − − − Φ ∇ Φ ∇ ⋅ ∇ Φ = ∂ Φ ∂ ) , ( ) , , 0 ( ] | ) , ( * | 1 1 ) ) , ( ( ) ) , ( ( | | )[ ( 0 2 2 2 2 2 1 1 y x y x y x I G c y x I c y x I t σ ε

λ

ν

μ

δ

(19) where

|

∇

Φ

∇

⋅

∇

is the curvature that can be calculated by: 2 / 3 2 2 2 2 ,

)

(

2 |

|

x y

x yy xy y x y xx j i

K

Φ

+

Φ

+

Φ

−

Φ

=

Φ

∇

Φ

∇

⋅

∇

=

(20)

Based on the finite differences theory, the above equation can be dispersed and yields the below iteration equation [16]: ] | ) , ( * | 1 1 ) ) , ( ( ) ) , ( ( [ ) ( 2 2 2 2 2 1 1 , , , 1 , y x I G c y x I c y x I K t y x n y x n y x n y x σ ε

λ

ν

μ

δ

∇ + + − + − − − Φ Δ + Φ = Φ + (21)

C. Steps of Segmentation and Tracking

(4)

(1) Detect moving regions based on frame differences

information and a binary motion mask map is acquired;

(2) The boundary of the binary map is extracted as the initial contours of moving objects;

(3) Calculate initial

Φ

0 based on the initial contour and set n=0;

(4) Calculate ₁

(

n

)

c

Φ

,

c

₂

(

Φ

n

)

,

δ

_ε

(

Φ

n

)

and

k

i,j

from the current level set function

_Φ

n

; (5) Calculate

Φ

n+1

within a narrow band based on (21) and set n=n+1;

(6) If n is smaller than the preset t meet the number of iteration times then return step (4) .

(7) Extract the final object’s boundary based on the zero level set.

(8) Tract the object in the next frame by using its current contour as initial value.

IV. EXPERIMENTAL RESULTS

In order to evaluate the performance of the proposed algorithm, experiments are conducted. The test images are randomly searched from Google image data set. In our experiments, four frames are used to detect moving region and some parameters are set as:

λ

1

=

1

,

λ

2

=

1

,

ε

=

1

,

μ

=

0 .

001 ×

255

2 ,

1 =

Δ

t

,

ν

=

0

.

Partial experimental results as well as some comparison results are given. Fig. 3 shows the original

images of Clare, Man and Lecture sequences. Fig.4 gives the experimental results of the method proposed by Zhao [17]. Fig.5 is the experimental results of the method proposed by Chan[12]. Fig.6 is the results of our method. Fig7 gives some tracking results of Claire and Man. Table 1 gives the consuming time of the three methods. where t1, t2 and t3 denote the consuming time of Chan’s

method, zhao’s method, and our method respectively. 1

t

Δ

and

Δ

t

₂ are the time reductions of our method

comparing with the other two methods, which are calculated by

3 1

1

t

=

−

Δ

,

Δ

t

₂

=

t

₂

−

t

₃

From the experimental results we can see that, compared with the other two methods, our proposed algorithm can segment the moving objects more accurately and effectively. In addition, we use the result of the frame difference as the initial contour of the active contour model which is close to final objects’ contours. It usually needs only a few iterations to get the precise contours which can save a lot of time. Hence, our method is more fast than the other two methods. Finally, since objects usually move slightly between two consequent frames, the segmentation result of the current frame can be used as the initial contour when we segment the objects of the next frame. That is, we can track objects easily among the sequences.

Figure 3. Original test images

(5)

Figure5. Segmentation results of Chan’s method

Figure6. Segmentation results of the proposed method

Figure7. Tracking results of the proposed method TABLE I.

SEGMENTATION TIMEOF THETHREEMETHODS

Sequence

1

t

₂

t

₃

Δ

t

₁

Δ

t

₂

Δ

t

1

/

t

1

Δ

t

₂

/

t

₂

Claire _{1.485 0.625 0.234 1.251 0.391 84.242 62.56}

Man _{3.143 1.75 0.213 2.93 1.537 93.223 87.828}

(6)

V. CONCLUSION

A fast moving object segmentation based on improved active contour method is proposed. Firstly, we employ frame-difference method to detect moving regions and extract their contours, which are used as the initial contours during object segmentation. Then an improved active contour method isperformed within a narrow band near the initial contours, which can greatly reduce the iteration times and computational load. In addition, an edge detection operator is incorporated into the PDE equation, which can improve the segment precision. Hence, compared with the existing techniques, our scheme demonstrated more effectiveness and efficiency. However, we also found some limitations of our algorithm. One major is that our segmentation results are not satisfactory for the images with complex background and multiple objects. These issues will be addressed in our future work.

ACKNOWLEDGMENT

This work was supported in part by the National Natural Science Foundation of China (No.60902066) ; the Natural Science Foundation of Zhejiang Province (No.Y107740) ; the Scientific Research Foundation of Zhejiang Provincial Education Department (No: Y201016875) .

REFERENCES

[1] X. W. Wang, and G. Ning, “A Modified Object Tracking Algorithm Based on Optical-Flow,” Academic meeting of national digital media and digital city. pp.351-353, March

2003.

[2] A. Cavallaro, and E. Touradj, “Accurate video object segmentation through change detection,” International Conference on Multimedia and ExPo, vol.l, pp.445-448,

2002.

[3] F. Dirk, “Automatic video-object segmentation employ multi-sprites with constrained delay,” 2006 International Conference on Consumer Electronics, pp.479-480.

[4] C. Alice, B. Laurent, and C. Jean-Marc, “Robust fast extraction of video objects combining frame differences and adaptive reference image,” 2001 International Conference on Image Processing, vol.2, pp.785- 88, 2001.

[5] F. Dirk, and E. Wolfgang, “Video-object segmentation using multi-sprite background subtraction,” 2004 International Conference on Multimedia and ExPo,vol.1,

pp.343-346, 2004.

[6] L. F. Jiang, and S. F. Gao, “Object segmentation and tracking algorithm based geometric active contour model,”

Shandong university. April. 2008.

[7] D. Y. Qiu, W. J. Zhang, B. Gu, and X. Y. Liu, “Application of Frame Difference Methods in Real-time Moving Target Tracking,” Journal of North China Institute of Water Conservancy and Hydroelectric Power.vol.30,

pp.45-46. June.2009.

[8] X. Y. Jiang, and Z. M. Zhao, “Research of the Image Sequence Segmentation Based on the Active Contour,”

Partment of ComPuter Science and Engineering Northwestern Polyteehnieal University . September.2002.

[9] Y. Zhang, and L Zheng, “Detecting and Tracking Moving Targets Based on Active Contour Model,”wuhan university of technology. April.2010.

[10]Z. J. Zhu, G. Y. Jiang, M. Yu, and X. W. Wu, “New Algorithm for Extracting Moving Object Based on Spatio-

temporal Information,” Journal of Image and Graphics,

vol.4, pp.422-426. April 2003..

[11]M. Kass, A. Witkin, and D. Terzopoulos, “Snakes - Active Contour Models,” International Journal of Computer Vision, vol.4, pp.321-331, 1987.

[12]T. Chan, and L, Vese, “Active contours without edges,”

IEEE Transactions on Image Processing, vol.2,

pp.266-277. October 2001.

[13]G. P. Zhu, and Q. S. Zeng, “Image Segmentation based on Active Contour Model,” Harbin Institute of Technology.

October2007.

[14]S. Oser, and J. A. Sethian. “Fronts Propagating with curvature dependent speed:Algorithms based on Hamilton-Jacobi formulation,” Journal of Computational Physics,

vol.1, pp.12-49. 1988.

[15]J. W. Liu, and H. Q. Feng, “Research of Level Set Method for image segmentation and its Applications in Medieal Images,” University of Seience and Technology of China.

April 2009.

[16]L. L. XU, J. S. Xiao, B. S. Yi, and L.J. Lou, “An Improved C-V Image Segmentation Method Based On Level Set Model,” 2008 IEEE DOI 10.1109/ICINIS.vol.127, pp.507-510. 2008

[17]H. Y. Zhao, Z. J. Liu, H. Zhang, “segmentation of multiple moving targets based on modified C-V model,”

Chinese Journal of Scientific Instrument, vol.31, No.5,

pp.1082-1089. May 2010.

Huimin Zhang received the B.S degree in electronic information science and technology from Zhengzhou University, china, in 2009. She is currently a master of Zhengzhou University, China. Her research interests include image segmentation and tracking and 3D image processing.

(7)

Zhongjie Zhu received the Ph.D degree in electronics science and technology from Zhejiang University, China, in 2004. He is currently a professor with Faculty of Electronics and Information Engineering, Zhejiang Wanli University, China. His research interests mainly include digital video compression and communication, watermarking and information hiding, 3D image processing, and image understanding.