Performance Analysis of Spatial Color Information for Object Detection Using Background Subtraction

IERI Procedia 10 ( 2014 ) 63 – 69

2212-6678 © 2014 Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/).

Selection and peer review under responsibility of Information Engineering Research Institute doi: 10.1016/j.ieri.2014.09.092

ScienceDirect

P

a, Abst Obje the v meth Meth impr of di in th prese © 20 Sele Keyw 1. I D appl med impe parts the s * E

20

Performa

,b_{Department of C} c_Departm tract ect detection, w video of outdo hod frequently hod (EFDM) an rove the accurac ifferent spatial he analysis inclu

ent better result 014 S.Manipri ection and pee

words:Spatial Col

Introduction

Detecting obje lications such dical and mete erative to com s, and interpre state-of-the-ar Corresponding a E-mail address: m

014 Internat

ance Ana

Detect

S.M

Computer Science ment of Civil Eng

which is a prima or environmen used for in su nd Extended Hi cy and computa color informati udes RGB, HS ts with HSV spa iya et al. Publ r review unde or Information, B

ects, and its fe h as visual su eorological ap mprehend bas et the interact rt technologies author. [email protected]

tional Confe

alysis of

tion Usin

Manipriya

a

,

e and Engineerin gineering,Nationa

ary yet importa nt is influenced uch cases. In th istogram Differ ational efficien ion for the prop SV, CIELab, CI atial color infor lished by Else er responsibilit Background Subtr eatures, throug urveillance, ge pplications, etc sic computer v tions between s for these com

ference on F

f Spatial

ng Backg

, C.Mala

b,

*

ng,National Institu al Institute of Tec

ant step in appl d by illuminatio

his paper, enha rencing Method ncy of object de posed methods IELuv, YCrCb rmation. evier B.V. ty of Informat raction, Frame Di gh the frames estural human c., [12]. In or vision tasks s n people or ob mputer vision

Future Inform

Color In

ground S

*, Samson

ute of Technology chnology,Tiruchir lications includ on and weathe anced BS meth d (EHDM) usin etection in real are compared a color models. tion Engineer ifferencing, Histo s of any video n-machine int rder to unders such as motio bjects. Recent problems.

mation Eng

nformati

Subtract

Mathew

c y,Tiruchirappalli rappalli,620 015,

ding video surv er changes. Bac

hod namely Ex ng spatial color time and outdo and analyzed. T The results sh ing Research ogram Differencin o is a primary terface, video stand such hig on detection, years have se

gineering

ion for O

tion

i, 620 015,Tamil N Tamil Nadu,Indi veillance, gets d ckground Subtr xtended Frame information ar oor environmen The spatial info how that EFDM

Institute ng; y task in any i editing, mot gh level applic tracking targe een rapid dev

Object

Nadu,India a difficult when raction is the Differencing e proposed to nt. The effects ormation used M and EHDM image-based tion capture, cations, it is ets, labeling velopment in

© 2014 Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/3.0/).

Considering a video sequence from a stationary camera overlooking traffic in an outdoor environment, an object detecting algorithm should adapt to various levels of illumination at different times of the day and handle adverse weather conditions that modify the image content [10,11]. Use of different cameras affects the color values as well. Further movement of an object can cause blurring of colors.

Background Subtraction (BS) has been used for years in many computer vision systems as an initial preprocessing step for object detection and tracking [12]. The results of the existing algorithms are fairly good; in addition, many of them run in real-time [1-5], but are susceptible to both global and local illumination changes such as shadows and headlight glares. These cause the subsequent processes, e.g. tracking, recognition, etc., to fail. The accuracy and efficiency of the detection are clearly very crucial to those tasks.

There are several BS Techniques suited for different scenarios with each having their own draw backs [6]. The Frame Differencing methods of BS with pixel to pixel comparison of two images were proposed for detecting objects [1-4]. Muhammad Nawaz et al., [5], and Michel Mason et al., [7] considered histogram related operations for BS using region based method. Being able to detect shadows of object in the images are also very useful to many applications especially in Shadow detection. Advance BS techniques can also help in detecting and eliminating shadows of objects in images [10]. Most of the applications require BS in an outdoor environment such as vehicle detection, pedestrian detection, motion detection etc., In such applications, priority needs to be given for illumination changes caused in the images due to environmental factors. Bo Yang et al [11] proposed a BS that adheres to illumination changes. There are other techniques for object detection in addition to BS which use spatial or frequency information [14] of the image. Various methods have used the color information in their object detecting algorithms [7-9].

The remainder of the paper is organized as follows: Existing BS algorithms are discussed in Section 2. The proposed extended methods are given in Section 3. The performance analysis of spatial color information is presented in Section 4 and finally, the conclusion in Section 5.

2. Background subtraction for object detection

Any video is a consecutive sequence of images or frames from which objects in the video can be detected. BS is the commonly used object detection technique followed in any image or a segment of the image (Region of Interest) .The detection of motions can be achieved by taking a frame as background and comparing subsequent frames with it. This process is called Background Subtraction (BS) [1-2]. In general, Subtraction of any two images involves techniques to identify the change or variation in their intensity levels. Two images from the video used for subtraction are defined as background and foreground image. The background image is a reference image comprising of static scenes with which the foreground image, comprising of any type of moving objects, are compared. The resultant image of BS is called Differenced Image (DI).

2.1 Spatial Background Subtraction using Frame Differencing Method (FDM)

The spatial information of any image comprises of color information stored in three different components (color channels) for each pixel, which are interpreted as coordinates in some color space. Most of the existing methods for BS [1-6] convert this spatial color information of background and foreground image to either grey image of 0-255 intensity levels or binary image with two intensity levels [1-6].

The existing FDM of Pixel based BS [1, 2, 6] is modified and given in the Equation (1)

ܵሺ݅ǡ ݆ሻ ൌ ȁܤሺ݅ǡ ݆ሻ െ ܨሺ݅ǡ ݆ሻȁ

(1) where S(i,j) denotes pixel intensity of the DI at the ith _{row and j}th _{column of the image. B(i,j) and F(i,j)} denote the pixel intensity resolution of ith _{row and j}th _{column of the background and foreground images} respectively. The DI- ‘S’ has pixel intensities equal to the difference of the pixel intensities in the background and foreground image.

In order to view and analyze the object content of the images, the number of pixels with similar intensities are identified and plotted. This plot denotes the histogram of the spatial information exhibited for a specific region of the image. The equation for any histogram [15] is given by Equation (2)



ܪ൫ܫ

_௤

൯ ൌ ݊ሺݍሻ

- (2) where Iqdenotes the qth intensity and nqdenotes number of pixels having qthintensity level.

Similar to FDM, the color information are lost and the processing is done on the binary or grey level information [5, 7].

Objects in the foreground can be detected by segmenting the image into a matrix. The dimension of the segmented portion of the image is inversely proportional to the accuracy of the object detected. Using Equation (2) Histogram based differencing BS [1, 2] is adapted and defined by

ܦ

_௦

ሺܪ

_஻ǡ௦

ሺܫ

_௤

ሻǡ ܪ

_ிǡ௦

ሺܫ

_௤

ሻሻ  ൌ ൝

ͳǡ‹ˆหܪ

஻ǡ௦

൫

ܫ

ݍ

൯ െ ܪ

ிǡ௦

൫

ܫ

ݍ

൯ห ൐ ݄ܶ

ሺுሻ





Ͳǡܱݐ݄݁ݎݓ݅ݏ݁

(3)

ܵ

_௦

ሺ݅ǡ ݆ሻ ൌ  ܦ

_௦ (4) where Ds(HB,s(Iq),HF,s(Iq)) is the Boolean value for the DI of the Background (HB,s(Iq)) and foreground histogram (HF,s(Iq)) of qth _{intensity and s}th _{segment with threshold Th}

(H). The Differenced image Ss is

computed by finding Euclidean Distance Measure [12] in Equation (3) as Ds given in Equation (4).

The main drawback of the above methods is the loss of spatial color information that has most of the significant characteristic feature and changing illumination values. To overcome this drawback, novel methods are proposed in this paper to increase the performance, accuracy, efficiency and preserve spatial color information to the existing BS technique. In addition, the effects of different spatial color information such as RGB (Red, Green and Blue components), HSV (Hue, Saturation and Value), CIE Lab, CIE Luv (Lightness and Color Component) and YCrCb (luminescence, blue-difference and red-difference chromaticity components) are compared and analyzed.

3. Spatial Color Information In Object Detection

The traditional methods [1-6] suppress spatial domain knowledge of more than one intensity levels to a single level leading to loss of significant information. Other methods include the frequency perspective [14] in their detection of objects such as edge detection [14]. The proposed techniques include spatial color information for BS because they exhibit a drastic variability in making detection technique robust. Each spatial color information model comprises of at least three components with each one describing the content of the image.

Various expressions required for incorporating the spatial information to the resultant DI for the proposed methods namely Extended Frame Differencing Method (EFDM) and Extended Histogram Differencing Method (EHDM) are derived and discussed in Sections 3.1 and 3.2

3.1. Extended Frame Differencing Method (EFDM)

In order to improve the exactness and to reduce the computational complexity of object detection, the spatial color information of the image is converted to other color information. The pixel intensity of the kth color component of the background and foreground images are denoted by Bk(i,j) and Fk(i,j). FDM from Equation (1) is modified as

ܵ

௞

ሺ݅ǡ ݆ሻ ൌ ȁܤ

௞

ሺ݅ǡ ݆ሻ െ ܨ

௞

ሺ݅ǡ ݆ሻȁ

where 1 ” k ” 3 (5) where Sk(i,j) is the DI of each k individual spatial color component. The individual kth _{color component of}

the background and foreground image is subtracted separately and stored in Sk. In EFDM, the pixel intensity

of the final DI is calculated using the individual spatial color DI – ‘Sk’ . This pixel intensity is estimated as the

The Extended Background Subtraction (EBS) of the image is the resultant image adopting the proposed EFDM and is computed from the formulated Equation (6)

ܧܤܵ

௙

ሺ݅ǡ ݆ሻ ൌ

σ͵݇ൌͳܵ݇ሺ݅ǡ݆ሻ

݇

 

- (6) 3.2. Extended Histograms Differencing Method (EHDM)

The histograms from equation (2) are calculated based on the pixel intensities. Including the color information to histograms has an enhanced and added advantage in detecting objects. The histograms with color information show predominant variation in pixel intensities during illumination changes or any other variability in the image. The individual color components acquired from the color information of the segmented region using Equation (3) are identified for the background and foreground segmented images. These individual histograms are used for BS.

ܪ

஻ǡ௞ǡ௦

൫ܫ

௤

൯ ൌ ݊

௤ where 1 ” k ” 3 (7)

ܪ

_{ிǡ௞ǡ௦}

൫ܫ

_௤

൯ ൌ ݊

_௤ where 1 ” k ” 3 (8)

where HB,k,s (I (q) ), HF,k,s(I (q) ) and n (q) denotes the histograms of background , foreground region and the number of pixel in the qth _{intensity with k}th _{component of s}th_{segmented region respectively. EBS for the}

Histograms Differencing Method (EHDM) using Equation (3), (7) and (8) is proposed as

ܦ

_௞ǡ௦

ሺܪ

_{஻ǡ௞ǡ௦}

ሺܫ

_௤

ሻǡ ܪ

_{ிǡ௞ǡ௦}

ሺܫ

_௤

ሻሻ  ൌ ൝

ͳǡ‹ˆหܪ

_஻ǡ௞

൫

ܫ

ݍ

൯ െ ܪ

ிǡ௞

൫

ܫ

ݍ

൯ห ൐ ݄ܶ

ሺுሻ





Ͳǡܱݐ݄݁ݎݓ݅ݏ݁

_

₍₉₎

Dk,s(HB,k,s(Iq),HF,k(Iq)) denotes the difference of the Background Histogram (HB,k,s(Iq)) and Foreground Histogram (HF,k,s(Iq)) of the kth component , qth intensity and sthsegmented region. For each segmented region the DI- ‘Sk,s(i,j)’ is computed from the Equation (4)and Equation (9) and coined in Equation (10)

(10) EBS of the image is the resultant image adopting the proposed EHDM and is given by

ܧܤܵ_݄

ሺ

݅ǡ ݆

ሻ

ൌ

ڀ

͵_݇ൌͳܵ_݇ሺ݅ǡ ݆ሻ (11) Equation (11) calculates the final binary DI’s pixel intensity EBSh (i,j) by using logical AND operation on each k differenced binary pixel intensity Sk(i,j) values. EHDM vanquishes HDM by improving the exactness of the object to be detected.

4. Spatial Color Information And Their Performance Analysis

The Spatial color information in the form of color models such as RGB, HSV, CIE LAB, CIE LUV and YCrCb were used for evaluating their performance in EFDM and EHDM. The sample set of images with illumination changes in an outdoor environment used for the analysis and evaluation are images from traffic surveillance cameras. The original background and foreground surveillance images are shown in the Fig1 (a) and (b) respectively. The images are converted to different color models and analyzed using OpenCV Library. The results for the EFDM are shown in the Fig 3 (a-d) which corresponds to different color models. Fig5.(a-d) shows the result of EHDM and histograms of the background and the foreground image with Th(H) as128 and with the segment of 64 x 64. It is evident from the Fig. 3 and Fig. 5 that the proposed methods EFDM and EHDM detect more pixels that exhibit even minute change in pixel intensity when compared with Fig. 2 and Fig. 4 of FDM and HDM for object detection.

(a) Background Image (b) Foreground Image (a) Grey Image (b) Binary Image

Fig. 1. Background and Foreground Image Fig. 2. Differenced images using FDM

The acquired results of the BS are evaluated and compared based on the number of pixels in the object to be detected for an assortment of color models. Fig. 6 (a) shows the performance of the binary and grey color model in FDM. Fig. 6 (b) show the performance evaluation of color models in RGB, HSV, LAB, LUV, YCrCb for the proposed EFDM. It is also observed that HSV color model outperforms all other spatial color models for the proposed EFDM. Due to the illumination variability, HSV color components are sensitive to even the diminutive changes in the image which are predominantly evident from the tested image of Fig. 3 and Fig. 4.

The binary and the grey component illustrate similar results of FDM in HDM as shown in Fig 7.Similar to EFDM, the analysis of different spatial information for the detected and undetected region of the object for the EHDM is carried out. The graphs are shown in Fig 8.(a) and (b).

0 10000 20000 1 7 13 Num ber of Pixel Detected

Aggregated Pixel Intensity

Grey Binary 0 10000 20000 1 7 13 Num ber of Pixel Detected

Aggregated Pixel Intensity

RGB HSV LAB LUV YCrCb

0 0.1

Undetected Region Detected Region

Norm alized No. of Pixels Grey Binary

Fig. 7. Performance Evaluation of Detected and Undetected regions using HDM (a)Grey and Binary image using FDM (b) Spatial color information using

EFDM Fig. 6. Performance Evaluation

(a) RGB Image (b) HSV Image

(c) CIE Lab Image (d) CIE Luv Image

Fig3.Differenced images using EFDM

(a) RGB Image (b) HSV Image

(c) CIE Lab Image (d) CIE Luv Image Fig5. Differenced images using HDM (a) Grey Image

(b) Binary Image Fig4. Differenced

Fig. 8. Performance Evaluation Fig. 9. Effectiveness of EFDM and EHDM

The effectiveness of EFDM, FDM, EHDM and HDM are compared based on the number of pixels being detected for illumination changes in an outdoor environment and the results are shown in Fig 9. Based on the number of pixel of the object detected and the pixel intensity, the Fig.9 (a) and (b) show that EFDM and EHDM with HSV spatial information detects objects in the foreground image in an enhanced way compared to the normal grey information of the image.

5. Conclusion

The principal task in any image based application is identifying object of interest using BS. Most of the existing methods diminish the drastically varying spatial color intensity of the pixel for processing leading to loss of information. The proposed method namely Extended Frame Differencing Method (EFDM) and Extended Histogram Differencing Method (EHDM) overcomes this drawback by accounting the spatial color information. The results show that the proposed methods enhance the detection of objects way better than existing methods based on the number of pixel of the object detected acceptably. It was also inferred that HSV color model acquire better performance than RGB,CIE LAB,CIE LUV, YCrCb color models for the proposed method.

References

[1] Y. Benezeth P.M,. Jodoin B., Emile H. And Laurent C. Rosenber. Review and Evaluation of Commonly-Implemented Background Subtraction Algorithms. International Conference on Pattern Recognition, pp.1-4, 2008.

[2] Mohamad Hoseyn Sigari, Naser Mozayani and Hamid Reza Pourreza. Fuzzy Running Average and Fuzzy Background Subtraction: Concepts and Application. IJCSNS International Journal of Computer Science and Network Security, VOL.8 No.2, February 2008.

[3] Haiying Zhang1 and Kun Wu1. A Vehicle Detection Algorithm Based on Three-frame Differencing and Background Subtraction. Fifth International Symposium on Computational Intelligence and Design, pp.148-151, 2012.

[4] Shahrizat Shaik Mohamed, Nooritawati Md Tahir and Ramli Adnan. Background Modelling and Background Subtraction Performance for Object Detection. 6th International Colloquium on Signal Processing & Its Applications (CSPA), pp.1-6, 2010.

[5] Muhammad Nawaz, John Cosmas, Awais Adnan, Muhammad Inam Ul Haq and Eman Alazawi. Foreground detection using background subtraction with histogram. IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB),pp.1-5, 2013.

[6] Amali Therese Jenifa.R. , Akila.C. , and Kavitha.V. Rapid Background Subtraction from Video Sequences. International Conference on Computing, Electronics and Electrical Technologies (ICCEET), pp. 1077 – 1086, 2012. 0.86 0.9 0.94 0.98 Norm alized No. of Pixels Spatial Information Undetecte d Region -0.02 0.01 0.04 0.07 0.1 Norm alized No. of Pixels Spatial Information Detected Region 0 10000 20000 1 7 13 Nu m b er of P ixel Detected

Aggregated Pixel Intensity RGB HSV LAB LUV YCrCb 0 0.5 1

Undetected Region Detected Region

Norm alized No. of Pixels HSV in EHDM Grey in HDM

(b) Detected region of spatial color information using EHDM (a) Undetected region of spatial

color information using EHDM

(a)Comparision of Efficiency of EFDM and FDM

(b)Comparision of Efficiency of EFDM and FDM

[7] Michel Mason and Zoran Duric. Using Histograms to Detect and Track Objects in Color Video. IEEE Transaction on Applied Imagery Pattern Recognition, 2001.

[8] Jiebo Luo, Crandall.D. Color Object Detection Using Spatial-Color Joint Probability Functions. IEEE Transcations on Image Processing, pp.1443-1453, 2006.

[9] P. P´erez, C. Hue, J. Vermaak, and M. Gangnet. Color-Based Probabilistic Tracking. Springer-Verlag Berlin Heidelberg, ECCV 2002, LNCS 2350, pp. 661–675, 2002.

[10] Chulhee Lee, Sangwook Lee, Jiheon Ok and Jaeho Lee. Shadow Removal for Background Subtraction Using Illumination Invariant Measures. 4th International Conference on Intelligent Systems, Modelling and Simulation pp.237-239, 2013.

[11] Bo Yang, Yunlong Guo, Yangyang Ming and Aidong Men. An Effective Background Subtraction under a Continuously and Rapidly Varying Illumination. Second International Conference on Future Networks, pp.16-19, 2010.

[12] Jun-Wei Hsieh, Shih-Hao Yu, Yung-Sheng Chen, and Wen-Fong Hu. An Automatic Traffic Surveillance System for Vehicle Tracking and Classification. IEEE Transactions on Intelligent Transportation Systems, Vol. 7, No. 2, 175-187, 2006.

[13] Hafner, J, Sawhney, H.S. , Equitz and W. , Flickner, M. Efficient Color Histogram Indexing For Quadratic Form Distance Functions. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol 17 , Issue 7 ,pp.729 – 736, 1995.

[14] Ahmed Elgammal, Ramani Duraiswami, Davids Harwood, and Larry s. Davis. Background and Foreground Modelling Using Nonparametric Kernel Density Estimation for Visual Surveillance. Proceedings of the IEEE, vol. 90, no. 7, July 2002.

[15] Rafael C. Gonzalez and Richard E. Woods. Digital Image Processing (3rd _{Ed.). Pearson International}