Survey on Multiple Shadow Detection and Removal Techniques

6 Available online at www.ijiere.com

International Journal of Innovative and Emerging

Research in Engineering

e-ISSN: 2394 – 3343 p-ISSN: 2394 – 5494

Survey on Multiple Shadow Detection and Removal

Techniques

Vihangi N. Patel, Dr. Dharshak G. Thakore, Mrs. Mahasweta Joshi

E- mail: [email protected]_{, [email protected]}2_{, [email protected]}3 Department of Computer Engineering, Birla Vishvakarma Mahavidyalaya, Anand, Gujarat, India.

ABSTRACT

Shadow Removal from images have attracted researchers recently to reduce problems like Detection of object in Aerial Remote Sensing, Find human object in Surveillance Scenario and also in Traffic Analysis. In image, there must be a single shadow or multiple shadows, therefore Challenges occurs to remove that shadows will be discussed in this paper. This paper reviews few popular techniques such as segmentation, Histogram Matching, Image Matting, Paired Regions for shadow detection and Shadow matting, Morphological operations are used for shadow removal and summarizes them in a deliberate basic assessment seat. All techniques under review perform Detection and removal of shadow on images acquired through camera.

Keywords:

Shadow Detection; Histogram matching; Shadow Removal; Challenges in shadow removal; Removal Validation

I INTRODUCTION

Shadow is a dark area created when a source of light is blocked. Shadows exist in images where 3D objects are illuminated by solid directional lights. There are two types of shadow: Umbra and Penumbra. Umbra is a part of shadow surface in which the direct light source is completely obscured by occluding object. Penumbra is a part of surface where the light source is only partially occluded.

Figure 1. Umbra and penumbra [4]

In penumbra condition, detection of shadow is difficult in different situations like Dark background, Overlap boundary of object and shadow, object is inside the shadow. In Aerial Remote Sensing, Shadows exists with high resolutions. The properties of a shadow, such as size, shape, and direction, are important elements when recreating a 3-D model of the relating object, e.g., a building. However, the loss of radiance in the shadow area causes problems in mapping, object detection. A fundamental problem in Surveillance Scenarios is to detect objects of interest in a given scene. Surveillance domain needs to manage shadows to stay away from bends when distinguishing moving objects.

There are some challenges occurs in shadow removal: [4]

A. Physical Phenomena a. Shadow Intensity:

7 Non-uniform intensity and it is occurs when ambient light source and occluding object is closed to shadow region. Therefore, it is difficult to detect that shadow region.

b. Umbra and Penumbra:

Shadow can be classified in two types: Umbra and Penumbra. Umbra detection is difficult in dark background and Penumbra detection and removal is difficult in light background as well as in image where object and shadow are intersect with each other.

c. The Light Source:

Light source have different type and shades. In some images, one or more light source exist. Thus, shadow removal is difficult to do.

B. Scene Characteristics

a. Self-shadows and Shading:

In images, Self-shadow occur due to the direct light source and after removing that shadow region the information like shadow cues are lost from original image.

b. Complexity of Shadow Surface:

The surface of shadow can be dark, light, and textured surfaces. So the complexity to remove the existing shadow in that surfaces are increased.

c. Intersection of Shadow:

In some shadow images, the shadow and object is intersect with each other in complex manner so that to differentiate them and remove the shadow is a challenging task.

II SHADOW DETECTION TECHNQUES

A. Segmentation:

Segmentation is a common method to detect an object from image or video. It consists background and foreground extraction, boundary tracing, thresholding and at the end filtering to enhanced output image. To subtract foreground and background, we can use different models such as Invariant color cone model, Invariant gradient model.

Figure 2. Flow diagram [8]

B. Image Division:

Image Division techniques is a simple method to removal shadow. It is used to remove shadow because it highlights the attributes of shadow.

8

ImgDiv (x, y) =𝑜𝑏(𝑥, 𝑦)

𝑏𝑘(𝑥, 𝑦)∗ 100 , ∀𝑥 ∈ 𝑋, ∀𝑦 ∈ 𝑌

Resultant image (ImgDiv(x,y)) is multiplied with a constant for increasing the signal intensity. Here, the constant value is 100 as shown in above equation. After that thresholding is performed to decide the shadow’s blob in the resultant image of image division process. [1]

ImgTh = {1, tmin≤ ImgDiv ≤ tmax , ∀(x, y)

0 , otherwise

In [1], the authors sets pixel values range of tmin = 50, tmax = 80 as belonging to shadow pixels. However, the range is highly dependent on the illumination in the scene. But different scenes also produce different level at illumination.

Filtering is done to enhance the resultant image after the thresholding. The boundary tracing is performed to check each boundary pixel and its neighbor, to detect as shadow region.

C. Histogram Matching:

In this method [9], we have an image with single texture shadow. Now, separate that image into two parts called shadowed and unshaded area. Calculate Histogram of that two images, so author can conclude that the expectation (µ) and variance (σ) values of a shadowed piece’s histogram is declines.

(a) (b)

(c)

Figure 3. (a) Piece of shadowed area; (b) piece of unshaded area; (c) histogram of (a) and (b) respectively. [9]

To find whether a pixel of an input image is belongs to shadow or not can be judged by its gray value. If gray value is within the scale of [µ-3σ,µ+3σ] in the histogram of shadow then it is called a shadow pixel.

D. Image matting:

9 Image matting is used to detect a soft shadow or penumbra. The goal is to extract a foreground object based on limited user inputs and estimating the foreground alpha matte. The color of the ith _{pixel is assumed to be linear combination of the} foreground and background colors. [7]

Ii = αiGi+ (1 − αi)βi , αϵ[0,1] [7]

Where, Gi , βi = foreground and background respectively αi = pixel’s alpha matte

For detecting soft shadow, suppose that the shadow is foreground and non-shadow is background. Thus, matte can locate soft shadow relatively and normalized entry of each pixel.

III SHADOW REMOVAL TECHNQUES

A. Using Morphological Operations:

Algorithm to remove shadow form input image has following steps: [3]

1. Calculate erosion of input image by 3*3 square. The structuring image is S1 = [010,010,010], S2 = [000,111,000], S3 = [001,010,100], S4 = [010,111,010]

EI1 = IS ϴ S1, EI2 = IS ϴ S2, EI3 = IS ϴ S3, EI4 = IS ϴ S4

2. EIA = EI1 + EI2 +EI3 +EI4

3. Added eroded image is dilated by 3*3 structuring element. S5 = [111,111,111]

DI = EIA ϴ S5

4. Dilated image is subtracted from input image to get exact edge of the object. CSR = EI - DI

B. HSI color model recovery:

HSI color model includes Hue, Saturation and Intensity which are occur due to lack of lights in shadow. Owing to less light, the intensity is proposed to decline and the hue is expected to change from the actual hue to a darker one. [9]

To calculate the changes in all factors of HIS color model from shadow to unshaded area, find the subtraction of the mean value of all factors of shadowed area from the mean value of corresponding factor of unshaded area.

Now, add changes of each factor’s mean value from shadowed area to unshaded area to recover the image from shadow. But there are some small differences still exists. So, shadow removal is also done by moving histogram of shadowed area to where the histogram of unshaded area lies.

C. Using texture anchor point:

This method is using SVM and MRF to detect shadow region. After that the shadow scale factors in the umbra and penumbra are calculated using intensity surface approximation method and directional smoothing. The shadow in image is removed by adding channel image with scale factors image and y combing it with original non-shadow pixels. Final shadow free image is performed by applying shadow-free region enhancement algorithm. [4]

D. Automotive Shadow removal framework:

Figure shows the method to remove the shadow. [10]

10 Figure 4. Shadow removal framework [10]

IV LITERATURE SURVEY

In this section a literature survey is carried out for various Shadow detection and removal techniques. A survey of around 10 papers is shown here which includes criteria such as various techniques related to shadow detection, shadow removal with merits and demerits, Removal Validation, find the accurate and automated method.

With the survey of these papers, various automated method to detect shadow is found and some challenges to remove shadow are also come in view.

Table 1. Literature review

Sr. no.

Title & Author Publication Methods Merits De-Merits

1 “Object’s Shadow Removal with Removal

Validation” By M.A. As’ari, U.U. Sheikh, S.A.R. Abu-Bakar

2007 IEEE International Symposium on Signal

Processing and Information Technology

Image division technique,

Background subtraction, Thresholding, Boundary Removal

• Efficient

• Improves object detection

• For outdoor images removal is very Challenging

2 “Detection and Removal of Chromatic Moving Shadows in Surveillance Scenarios” By Ivan Huerta, Michael Holte, Thomas

Moeslund, and Jordi Gonz`alez

2009 IEEE 12th International Conference on Computer Vision (ICCV)

Foreground

Segmentation, Shadow Intensity Reduction, Bluish effect, Shadow texture detection, Shadow edge removal

• Robust

• Accurate

• Improve colour & gradient model

• Hard to understand

11 3 “Cast Shadow

Detection and Removal in a Real-time

Environment” By S. Lakshmi and V. Sankaranarayanan

2010 IEEE Boundary Detection, Morphological Operations (Erosion & Dialation) for removal

• Easy to understand

• Easy to implement ation

• Implementation in complex

environment is difficult

4 “Shadow

Removal Using Intensity Surfaces and Texture Anchor Points” By Eli Arbel and Hagit Hel-Or

IEEE

Transaction on Pattern

Analysis and Machine Intelligence, Vol. 33, No. 6, June 2011

Shadow mask

derivation using SVM, Region Growing phase, Anchor points, Shadow scale factor

• High-quality shadow-free result

• Increase algorithm robustness

• Take more time to complete

• Difficult for coincide boundaries

5 “Shadow

identification for digital imagery using colour and texture cues” By R. McFeely, M. Glavin, E. Jones

IET Image Process., 2012, Vol. 6, Iss. 2, pp. 148–159

Shadow Segmentation using Tree-structured histogram, Gabor filter

• Robust

• Effective segmentati on

• Automate d method

• Requires less time to give output

• Less prone to over-segmentation

• Hard to understand

6 “Paired Regions for Shadow Detection and Removal” By Ruiqi Guo, Qieyun Dai, Derek Hoiem

IEEE

Transaction on Pattern

Analysis and Machine Intelligence, Vol. 35, No. 12, December 2013

Mean Shift Algorithm, Trained Classifier, find Illumination Pairs, Construct relational Graph, Shadow Matting

• Novel Approach

• Effective

• Algorithm may fail in the case of multiple light sources

7 “An Adaptive Nonlocal

Regularized Shadow Removal Method for Aerial Remote Sensing Images” By Huifang Li, Liangpei Zhang, Huanfeng Shen

IEEE

Transactions on Geoscience And Remote Sensing, Vol. 52, No. 1, January 2014

Image Matting, NL Regularized Shadow Compensation

Method, Improves NL method: SA-NLSC

• Novel Method

• Effective

• Efficient

• Need to Improve restoration

8 “Efficient Shadow Removal

Technique for Tracking Human Objects” By Aniket K Shahade and Gajendra Y Patil

2014 IEEE Background Modeling, Background

subtraction and Foreground

Extraction, Object Tracking,

Morphological Process

• Easy to understand

• Easy to implement

12

V CONCLUSION

Shadow Detection and Removal is very needful in real-time application for detecting objects. Segmentation is common method to detect the shadow. Shadow removal is complex thing to do because as we discuss it has some challenges and after removing shadow the boundary of shadow is slightly visible as transparent line. Thus, to get a pure shadow free image is a challenging task.

REFERENCES

[1] M.A. As'ari, U.U. Sheikh, S.A.R. Abu-Bakar, “Object's Shadow Removal with Removal Validation” IEEE International Symposium on Signal Processing and Information Technology, 2007.

[2] Ivan Huerta, Michael Holte, Thomas Moeslund, and Jordi Gonz`alez, “Detection and Removal of Chromatic Moving Shadows in Surveillance Scenarios” IEEE 12th International Conference on Computer Vision (ICCV), 2009.

[3] S. Lakshmi and V. Sankaranarayanan, “Cast Shadow Detection and Removal in a Real-time Environment” IEEE Journal, 2010.

[4] Eli Arbel and Hagit Hel-Or, “Shadow Removal Using Intensity Surfaces and Texture Anchor Points” IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 33, No. 6, June 2011.

[5] R. McFeely, M. Glavin, E. Jones, “Shadow identification for digital imagery using colour and texture cues” IET Image Process., 2012, Vol. 6, Iss. 2, pp. 148–159.

[6] Ruiqi Guo, Qieyun Dai, Derek Hoiem, “Paired Regions for Shadow Detection and Removal” IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol. 35, No. 12, December 2013..

[7] Huifang Li, Liangpei Zhang, Huanfeng Shen, “An Adaptive Nonlocal Regularized Shadow Removal Method for Aerial Remote Sensing Images” IEEE Transactions on Geoscience And Remote Sensing, Vol. 52, No. 1, January 2014.

[8] Aniket K Shahade and Gajendra Y Patil, “Efficient Shadow Removal Technique for Tracking Human Objects” IEEE Journal, 2014.

[9] Zhang Yali, Zhao Yang, Xiao Pan, Yuan Yule, Yang Xi, Lu Yawei, “Shadow Removal of Single Texture Region Using Histogram Matching and Color Model Recovery.” ICSP2014 Proceedings.

[10]S. H. Khan, M. Bennamoun, F. Sohel and R. Togneri, “Automatic Shadow Detection and Removal from a Single Image” IEEE Transactions on Pattern Analysis and Machine Intelligence and Journal of Letex Class Files, Vol. 6, No. 1, July 2015.

9 “Shadow

Removal of Single Texture Region Using Histogram Matching and Color Model Recovery.” By Zhang Yali, Zhao Yang, Xiao Pan, Yuan Yule, Yang Xi, Lu Yawei

ICSP2014 Proceedings

Histogram Matching, HSI color Model Recovery

• Good Performan ce

• Acceptabl e results

• Effective

• Work with limited Datasets

10 “Automatic Shadow Detection and Removal from a Single Image” By S. H. Khan, M. Bennamoun, F. Sohel and R. Togneri

IEEE

Transactions on Pattern

Analysis and Machine Intelligence and Journal of Letex Class Files, Vol. 6, No. 1, July 2015

Bilateral Filtering, Boundary Extraction, Imbalance Removal (SMOTE), Shadow Localization, CRF Model, Bayesian Shadow Removal, Shadow Matting

• Effective

• Robust

• Efficient

• Does not perform on curved Surface