2017 2nd International Conference on Computer Engineering, Information Science and Internet Technology (CII 2017) ISBN: 978-1-60595-504-9
Moving Object Detection via Color Optical Flow and
Visual Objectness Measurement
JIASHU DAI, NAN YAN, TAO LIU and JUN WANG
ABSTRACT
How to find the moving object in the dynamic scene is one of the core problems in computer vision. In order to accurately detect moving objects in the video surveillance scene, optical flow algorithm and visual saliency detection technology had been researched. A moving object detection algorithm via color optical flow and visual similarity measurement is proposed in this paper. Firstly, the optical flow is detected on the visual significant color channel according to the opponent color theory, and the rough position of moving object is located. Then, two objectness measurements are defined to consider whether a region is an object in the precise positioning stage. Finally, the sliding window is used to mark the boundary box of the moving object in the sequence image. Experimental results show that the proposed algorithm is better than the classical moving object detection algorithm.
KEYWORDS
Object Detection, Optical Flow, Objectness, Movement Scene.
INTRODUCTION
Moving object detection is the most basic problem in visual scene analysis, and is the basis of tracking, recognition and other issues. In the implementation of moving object detection problem, the input is a set of video frame images, and the output is the state of moving object. The mainstream methods include inter-frame difference, background estimation and optical flow. Inter-frame difference method is used to calculate the difference between consecutive two frames or multi frame sequence images, and then thresholding is carried on the difference image to obtain the moving object [1]. The background estimation method needs to estimate the background information in the scene [2-3], moving object can be obtained by the difference between the current frame image and the estimated background image. When the optical flow vector of all pixels are calculated, the moving target region can be extracted by threshold method [4,5]. Zhang et al.[6] proposed an algorithm based on pixel spatial sample difference consensus. Zhang et al.[7] proposed a Bayesian approach for detection.
This paper proposed a two stage algorithm for precise positioning the moving objects referring to the idea of optical flow and objectness. Comparative experiment suggests that the proposed algorithm can efficiently detect the moving object.
_________________________________________
MOVING OBJECT ROUGH POSITIONING GUIDED BY COLOR SALIENCY OPTICAL FLOW
The traditional optical flow detection technology [3,4] usually processing gray level images, while the images or video in the visual scene captured by camera are color images. These color information are effective helpful for obtaining motion information. Compared with gray level images, color images can provide discriminative color information.
Optical flow field with significant color
The famous Opponent color theories assume that the human brain visual cortex has a color competition mechanism with visual impact, this is also referred to as the "color competition system"[8]. In the receptive field of visual cortex, when the neuron is stimulated by a certain color, the other color will be ignored automatically when observing another color. As we observed the red color, we will consciously ignore the green information in the scene. In the visual cortex of the human brain, the color pairs with competitive relationship include red-green, green-red, blue-yellow and yellow- blue.
The Color image from acquisition device has three color channels which are the red (R), green (G) and blue (B). The other color information can obtained through the three color channels information. Red-Green color contrast space CRG, Blue-Yellow
color contrast space CBY and gray space Cgray can be calculated by the follow Eq. 1,
2, 3.
RG
C R G. (1)
2BY
C B R G . (2)
0.299*R+0.587* +0.114*
gray
C G B. (3)
Red-Green Green-Red Blue-Yellow Yellow - Blue Figure 1. Opponent color comparison.
obtain more detailed motion scenes, the optical flow in differential state are calculated on the Red-Green color contrast space CRG, Blue-Yellow color contrast space CBY
and gray space Cgray. Then, the three optical flow fields are fused to get a more
complete moving object optical flow field.
The energy function in the initial frame image of of Red - green color contrast space CRG is defined as follows.
2 2 2 2 2min ( ) ( ) ( ) ( )
RG i j
u u v v
E f u f v f didj
i j i j
. (4)The Eq.5 is equivalent to the solution of Euler equation in the Eq.4.
min , , , , ,
RG i i j j
E
F u v u v u v didj . (5)By simplifying, the corresponding Euler equation is
2 2 2 2 ii j t
j
i j t
f
u f u f v f
f
v f u f v f
. (6)
Where, 2 2 2i 2 2j . Pointing to the digital images, the continuous variables should be discretized, the energy function is simplified as smooth constraint.
2 2 2 2
, 1, , , 1 , 1, , , 1 ,
1 4
a b a b a b a b a b a b a b a b a b
S u u u u v v v v . (7)
Where, the
ua b, ,va b,
is the optical vector at the pixel
a b, .In order to calculate the optical flow vector
ua b, ,va b,
, the partial derivativerespectively carried on the ua b, ,va b, . Solving by iteration, the n+1 iteration formula is
( ) ( )
, ,
( 1) ( )
, , 2 2 2
( ) ( )
, ,
( 1) ( )
, , 2 2 2
n n
i a b j a b t
n n
a b a b i
i j
n n
i a b j a b t
n n
a b a b j
i j
f u f v f
u u f
f f
f u f v f
v v f
f f
Original image LK optical flow HS optical flow color saliency optical flow
Figure 2. Comparison of optical flow algorithms.
Similarly, the optical flow field of the Blue-Yellow color contrast space and the gray space can calculated in the same way. These three optical flow fields are weighted fusion into optical flow field which has a significant color. The new color saliency optical flow field is
( ) ( )
( ) ( )
RG BY
gray RG BY
RG BY RG BY
RG BY
gray RG BY
RG BY RG BY
S S
u u u u
S S S S
S S
v v v v
S S S S
. (9)
Figure 2 gives the results of color saliency optical flow field and other classical optical flow algorithms in visual scene. It can be seen from the figure that the HS optical flow and the LK optical flow are significant for the right object, while the left one is not obvious due to the dark light. The proposed optical flow field has highlighted dark objects due to the use of significant color contrast.
Rough positioning of moving object in optical flow field
The optical flow field previously obtained is the vector field which constructed by the velocity of every pixel. It was visualization with the length and direction. In order to highlight the moving target in the region, we need to convert it to gray image. Then, normalization and thresholding are carried to extract moving objects rough position.
The calculation of vector module of each pixel point
i j, in optical flow is2 , j ( , , ) i i j i j
M u v . (10)
The whole optical flow is quantized into a gray image G, the gray value of pixel
i j, is, j , j
min( )
max( ) min( )
i i
M M
G
M M
[image:4.612.195.416.302.368.2]Figure 3. Moving object rough localization.
The moving object region is extracted according to the gray image after thresholding. Figure 3 shows the rough positioning of moving object by color saliency optical flow. It can be seen from the figure, the rough region still contains a lot of background information. This will bring a lot of difficulties to the tracking, so we need to carry out further accurate positioning to the object.
MOVING OBJECT PRECISE POSITIONING BASED ON VISUAL OBJECTNESS MEASUREMENT
Alexe et al. [9] defined a generic measurement for determining whether an image window contained an object and established a large number of databases for training. Learn from the thought of objectness, from the human visual view-point, we must first determine whether the information contained in this region is different from its surrounding information. At the same time, the information contained in this image region should be complete, i.e. the edge should be more complete. From these two viewpoints, a visually objectness measurement is defined for accurate localization of moving object rough regions in previous color optical flow detection.
The visual significant objectness measurement
Mahadevan et al. [10] proposed a visual tracking algorithm based on the central surround saliency detection mechanism. In view of the idea of central surround saliency, an objectness measurement base on color contrast is defined. The color histogram in HSV space can effectively describe the color information of image blocks. The color distance between an image block area and its surrounding area is described by the Bhattacharyya distance between the window w and an external windowbw w r( , ). The color contrast objectness measurement ColorObjectnessis
( ( ), ( ( , )))
B hsv hsv
ColorObjectnessd h w h bw w r . (12)
Where, hhsv( ),w hhsv(bw w r( , )) are the color histogram of window w and bw w r( , )
in HSV space.
the image window and the length of the window. The edge integrity objectness measurement EdgeObjectnessis defined by
( )
( )
p w
Edge p EdgeObjectness
l w
. (13)Where, Edge p( )
0,1 is the result of Canny edge operator.Accurate positioning through sliding window
From the visual perception, as the sliding window continues to shrink, the proportion of object information is increasing, while the proportion of background information is reducing. When the window is just covering the object, the difference of color contrast should be in an extreme state theoretically. At the same time, the edge integrity of the window is also the maximum. In natural scenes, if the texture of the object surrounding area is complex, there may be some deviation.
The sliding window starts with the left corner of the original window, the top-left corner and the width and height of the window is changed gradually. The change rate is 2 pixels per time. The top- left vertex can only be moved to the center point of the original window. The width and height of the window change up to
width 2,height 2
. During the sliding process, the color contrast objectness measurement ColorObjectness and the edge integrity objectness measurementEdgeObjectness between the sliding window w and the initial outsourcing window ( , )
bw w r are respectively calculated. Then, the optimal object window is
( , ) ( )
arg max
( , ) ( )
optimal
w initial initial
ColorObjectness w r EdgeObjectness w w
ColorObjectness w r EdgeObjectness w
. (14)
Where, winitial is the result of rough positioning.
COMPARISON AND ANALYSIS OF EXPERIMENTS
The result of LK detection
The result of inter-frame difference
detection
The result of mixture Gauss model detection
[image:7.612.108.515.54.268.2]The proposed algorithm
Figure 4. Comparison of moving object discovering algorithms.
It can be seen from the Fig. 4, the detection results of the three comparison algorithm contains some background regions in the first row. While the detection result of the proposed algorithm has a two pixels different from the real object boundary box. Although this result contains a small amount of background information, it has been more accurate than other algorithms, and can meet the initialization requirements of the object tracking. The second row is the indoor surveillance scene and the object shadow is more obvious. The target region is detail analyzed in the precise positioning phase in the proposed algorithm. So the object region is accurate enough for visual tracking relative to other algorithms.
SUMMARY
The moving object detection problem in this paper is viewed as the two stage positioning process. In coarse positioning stage, the visual significant color optical flow is constructed according to the Opponent color theory. The optical flow field is normalized and other operations are used to mark the approximate region of the moving object in the visual scene. In the precise positioning stage, the definition of two kinds of objectness measurements are used to describe whether the image region is an object. These are color contrast and edge integrity measurement. Finally, the moving object bounding box is located by the sliding window. Experimental results show that the proposed object discovery algorithm can more accurate to mark the moving objects in the visual scene through the two stage positioning, it can provide automatic initialization for the moving target tracking algorithm.
ACKNOWLEDGEMENT
Foundation of Anhui Polytechnic University (No. 2015YQQ011), and Foundation of Key Laboratory of Computer Application Technology (No. JSJKF201603).
Jiashu Dai, [email protected], School of Computer and Information, Anhui Polytechnic University, Wuhu 241000, Anhui, China.
REFERENCES
1. Sengar S S, Mukhopadhyay S. Foreground Detection via Background Subtraction and Improved Three-Frame Differencing. Arabian Journal for Science & Engineering, 2017, 42(8):3621-3633. 2. Wren C.R., Azarbayejani A., Darrell T., et al. Pfinder: Real-time tracking of the human body. IEEE
Transactions on Pattern Analysis and Machine Intelligence, 1997, 19(7): 780-785.
3. Kim K., Chalidabhongse T.H., Harwood D., et al. Real-time foreground–background segmentation using codebook model. Real-time imaging, 2005, 11(3): 172-185.
4. Horn B.K., Schunck B.G. Determining optical flow. Proceedings of the 1981 Technical Symposium East, 1981, International Society for Optics and Photonics: 319-331.
5. Lucas B.D., Kanade T. An iterative image registration technique with an application to stereo vision. Proceedings of the IJCAI, 1981: 674-679.
6. Zhang C, Zheng J, Zhang Y, et al. Moving object detection algorithm based on pixel spatial sample difference consensus. Multimedia Tools & Applications, 2017(3):1-17.
7. Zhang X, Zhu C, Wang S, et al. A Bayesian Approach to Camouflaged Moving Object Detection. IEEE Transactions on Circuits & Systems for Video Technology, 2017, 27(9):2001-2013. 8. Zhang R, Liu X, Hu J, et al. A fast method for moving object detection in video surveillance
image. Signal Image & Video Processing, 2016:1-8.
9. Jain A., Healey G. A multiscale representation including opponent color features for texture recognition. IEEE Transactions on Image Processing, 1998, 7(1): 124-128.
10. Alexe B., Deselaers T., Ferrari V. Measuring the objectness of image windows. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11): 2189-2202.
