4561
INACCURACY CORRECTION METHOD FOR
MOVING SHAPES AND SHADOWS IN VIDEO
CODING OBJECT
Kanmani P, Priya V, Yuvaraj N, Sudhakar S, Sriram V PAbstract— The Internet is based on the best-effort distribution approach and the network is converted to packets. The Internet network, therefore, does not guarantee the transmission of data. Several strategies have been developed to mitigate the loss and mask the loss of information. MPEG-4 compression is used in object-based error concealment, which is required to acquire the shape and texture information of objects in the video along with their movement. On the other hand, in order to understand relative object location and size in a scene, Shadows plays major roles. For example, without a cast shadow, it is difficult to accurately determine an object's position in space. Shadows can also help us to understand a complex receiver's geometry. In images and videos, the current error concealment algorithms are not considered as shadow's function. Therefore, if the video has shadows, the precision of the error concealment can be that. The video is considered a crucial factor in overcoming this shadow. The error is hidden and applied using calculation and correction of global motion. The shadows are obtained through HSV color space while assigning the threshold value by standard deviation. To calculate the relative standard deviation, the discrete wavelet transformation is used. The output binary planes of the algorithm to conceal error and the algorithm to detect shadows are combined to recreate the damaged frame with shadows.
Index Terms— Error Concealment, Shadow objects concealment, object based error concealment.
—————————— ——————————
1 INTRODUCTION
THE use of real-time video streaming applications has been greatly increased due to the Internet at the moment. Because internet networks are the best-effort network, data is transmitted as a continuous signal in discrete chunks called packets. Packet switched networks are difficult for real-time video sharing or streaming. As more and more video is transmitted over the Internet and in higher resolutions as well, the packet loss can occur. Faulty equipment, congested routers or other issues can result in complete packet loss [8]. Several methods have been developed to mitigate the loss and conceal the loss of information. The techniques of error concealment can be classified into Forward Error Concealment, which is performed at the decoder, Post Processing Error Concealment, Interactive error concealment in both the encoder and the decoder. To evaluate MPEG-4 algorithms like object-based video coding object-based error concealment, Motion-based error concealment, and Edge-based error concealment are suggested. MPEG-4 compression, which is implemented by coding the shape information [2] of video objects along with their movement and texture information as per Cheng et al. (2005), is used as an object-based representation. DCT transformation and ROI coefficients are applied in Region of Interest (RoI) error concealment method. This method is used primarily for still images. Edge Directed Error Concealment (EDEC) algorithm[18] covers the frames of video sequences which is encoded by variable macroblock ordering. Originally, while referring previous and consequent frames, the strong edges of the broken frame are marked based on the contour.On the other hand, in order to understand relative object location and size in a scene, Shadows plays major roles. For example, without a cast shadow, it is impossible to easily determine an object's position in space [6]. Shadows can also help us to understand a complex receiver's geometry. Shadows can show occluded geometry or even out-of-view geometry. If the video and its artifacts have shadows, the existing methods do not understand the shadows ' properties. Shadows play an vital role in video remote sensing, security and tracking, as well as 2D, 3D animated videos [5].
In images and videos, the current error concealment algorithms are not considered as shadow's function. Therefore, if the video has shadows, the precision of the error concealment can be that. The video is considered a critical element in removing this shadow. The error is hidden and applied using calculation and correction of global motion. The shadows are observed in HSV color space by adding relative standard deviation as a threshold [3]. For measure the relative standard deviation, the discrete wavelet transform is used. The output binary planes of the algorithm to conceal error and the algorithm to detect shadows are combined to recreate the damaged frame with shadows.
2
LITERATURE
SURVEY
Any work is a continuation of the previous analysis and its emanation. Cheng et al. (2005) suggested a MPEG-4 object-based video coding error concealment technique [2] for the error correction among frames in the video. Reconstruction of missing pixels is achieved using the regional motion correction method[2]. The damaged design of the video object is converted to binary image, then the boundary is removed. After that, the weakened boundary is rebuilt using the global motion compensation. Eventually, restrictions are applied to ————————————————
Kanmani P , Assistant Professor, Dept.. of CSE, K.S Rangasamy
College of Technology, Namakkal, Tamil Nadu, India,
Priya V, Associate Professor, Dept. of CSE, Mahendra Institute of
Technology, Namakkal, Tamil Nadu, India,
Yuvaraj N , Postdoctoral Researcher, Kyungil University, Gyeongbuk, South Korea. [email protected]
Sudhakar S, Professor, Dept. of CSE, Sree Sakthi Engineering College, Tamil Nadu, India, [email protected]
4562 the reconstruction of the damaged plane of the video frame.
The missing edge regions are then replicated using adjacent spatial and temporal pixels. Shih-Chang et al. (2010) proposed the technique of concealment of errors based on [22] Region-Of–Interest (ROI). The ROI picture has been categorized into four sub-bands namely LL, HL, LH, HH. Sub-band blocks are subject to the DCT transformation. The ROI picture coefficients are contained in the initial block HH. Forward error correction is applied to the received bitstream at the receiver end, then reverse DCT is used without error to get the ROI image.On the basis of a combination of global and local motion correction, Soares et al. (2006) suggested an initial temporal form error concealment technique [25]. This approach is useful in error-prone environments for object-based video applications. The corrupted object shape is in the form of a binary alpha plane and some of the shape data is missed due to the errors in channel. To solve the error the decoder follows the a global movement method that reflects the shape changes in successive instants of time to conceal the corrupted form.Kaushik et al. (2014) proposed [12] a basic system for detecting and extracting shadows from photographs using the color space Yellow, Chroma-blue, Chroma-red (YCbCr). The Intensity of the frame is used to detect shadows in the YCbCr color space. The shadows in the frame are removed by renewing and correcting each and every pixel in the color space of red-green-blue (RGB) YCbCr. Manish et al (2014) suggested Discrete Wavelet Transform (DWT) [17] in video sequences to find and erase the shadows. This method is very supple and depends on the coefficients of the wavelet. The discrete transformation of the wavelet generates four sub-images in which LL is the low pass filtered in horizontal and vertical directions. HSV color model is used in this research in the Discrete Wavelet Transform domain. As the Hue value for shadow pixels varies quickly, components of saturation and brightness are used to locate the shadow. The relative standard deviation to detect shadow in the sequences is used to calculate the video threshold value.A comprehensive literature review and background analysis on existing methods of concealing errors and shadow detection was performed in this research. The current algorithm for concealing error does not fix the shadows. The existing methods of concealing error do not take into account shadow properties and importance. Now the internet and social media of a day are growing rapidly, so it is important to examine the concealment of errors for all forms of content. Shadow plays an important role while considering the object's position and geometry, in attracting the viewer's attention, and in particular animated videos, it gives meaning to the object. Several algorithms for shadow detection and elimination are already being proposed. But the gap exists to rebuild or maintain the shadow blocks that were lost. Shadows are created by using various lighting variables in animated videos and the shadows are very necessary to create the look and feel of the real world. Failure concealment must be investigated for animated videos.
3 OBJECT
BASED
ERROR
CONCEALMENT
The role of shadow in images and videos is not considered by previous error concealment algorithms. Shadow is considered a critical element in the video in order to solve this. The plan is applied in three main stages such as concealing errors,
detecting shadows and integrating concealing errors and detecting shadows. Using global motion calculation and compensation, the damaged shapes are rebuilt in error concealment. For shadow detection, shadowed artifacts are eliminated using Discrete Wavelet Transform (DWT) [6] and relative mean square for both reference and damaged frame wavelet coefficients. Finally, to select the correct shadow pixels, the output from error concealment and shadow detection is combined and then the color filling is applied to the resulting binary image.In error concealment, using global motion estimation, the following steps are applied in order to recover the image. Initially, damaged and reference frame boundaries are eliminated using the 8-Connected pixel process. Then the calculation of global motion parameters such as scale, centroid, and angle of orientation. According to Cheng et al. (2005), the zoom or scale can be defined as [2] the total amount to which the existing video object plane has expanded or shrunk with respect to the VOP reference and the scale can be calculated by
(1)
where
b2i,j- pixel value in the ith row and jth column of the binary mask of VOP2
b1i,j pixel value in the ith row and jth column of the binary mask of VOP1
The centroid of the every object obtained by
(3)
where
( x̅, y ̅) are the coordinates of the centroid of the VOP relative to the top left position of the VOP.
The orientation angle of an object in a binary image can be calculated by
(4)
where
where
(x_(i,j),y_(i,j))- represents the coordinates of the pixel in the ith row and jth column of the binary mask. The rotation angle parameter is calculated by
∆Ө = Өcur - Өref (5) Where
Өcur -The orientation angle of damaged frame for an object
4563 angle of the damaged pixel are used to calculate the boundary
pixel present in the reference VOP with that of the damaged VOP and vice versa.
+ (6)
+ (7)
For shadow detection, the HSV Color model reflects both the reference frame and the damaged frame. Saturation and value variable are determined after that the absolute difference between current and damaged frames Hue. The Discrete Wavelet Transform (DWT) is applied to the saturated value and the corresponding wavelet coefficients are removed. Also, it measures the relative standard deviation of the pixel coefficients of the wavelet. The resultant shadow state is eventually used to distinguish shadowed objects.
Since HSV's hue component is changed with respect to the shadow strength, Saturation and Value components to detect shadows. Wavelet decomposition is then performed using DWT on the modified image of the value component and the saturated component (ΔV and ΔS). The coefficients of the wavelet are given by WΔV and WΔS. The difference between the value variable of the current frame and the reference context frame is given by ΔS. It is also necessary to fix the optimum threshold value for effective shadow detection. Many current algorithms need manual threshold adjustment in terms of specific parameters that are predefined. Guan has proposed relative standard deviation as one of the criterions for detecting the shadow and elimination in the DWT domain. Relative standard deviation is suggested here as a new threshold for shadow detection and elimination in the DWT domain. The relative standard deviation defined as the standardized dispersion measure of a probability distribution and is given by standard deviation (σ) with respect to the mean (μ) value. Relative standard deviation represented by the unit of measurement independent. Due to the capability of our visual system to sense an object in a uniform context, the purpose of using relative standard deviation (also known as contrast ratio) depend on its size and contrast ratio. This is calculated by σ/μ. Mean (μ) represents the object's average luminance in the wavelet domain, and π denotes the object's standard luminance deviation added with wavelet domain surrounding.
4 SHADOW
DETECTION
For shadow detection, the HSV color model replicates both the reference frame and the damaged frame. The absolute gap is measured for the reference variable hue, saturation and meaning, and damaged objects. The Discrete Wavelet Transform (DWT) is applied to the saturated value and the corresponding wavelet coefficients are then extracted. For shadow power, saturation and value components used only for shadow detection, the hue function of HSV is modified very dramatically. The decomposition of the wavelet is carried out on variance images of value and saturation (ΔV and ΔS) and it is represented by wavelet coefficients denoted by WΔV and
WΔS. ΔV is calculated by Value component of current frame to the reference background frame. ΔS is the gap between the saturation component.
Manish et al. (2014) used relative standard deviation in the DWT domain as a threshold for shadow detection and removal. The relative standard deviation is also known as the standard deviation ratio (σ) with the mean (μ) Which is independent of the measurement unit. Finally, the quality of the shadow is used to classify shadowed objects. The shadowed foreground object detects the condition at every pixel (i, j) is defined as
(8)
Where
WΔV-Discrete wavelet transform coefficients of ΔV (σ/μ )_(W_∆v )- Relative standard deviation of WΔV.
The moving object identified with shadow may not be correct as some shadow points may be misclassified as points of the foreground object. The obtained binary image from calculating with error concealment is combined with the resulting binary image from shadow detection I order to select the appropriate shadow pixel and object pixel. Using scanline object filling algorithm, the restored damaged frame is finally filled. The PSNR value is checked for different video sequences and the error concealment is successfully done even though the video has shadows.
5 SIMULATION
RESULTS
The proposed algorithm overshadows the shadowed object picture. Real time videos and regular videos are used as datasets under various lighting, climatic conditions and with various object size. The videos are checked and the corresponding PSNR values are measured in different bitrate. The results are presented for four representative video sequences in different lighting and objects. Packet loss is simulated for encoded information on shape and texture, and the packet sequence may be lost during packet loss periods. The damaged frame and its reference frames are identified after the simulation of the error. Table 1 includes the detailed descriptions of the three representative video sequences. Peak Signal to Noise Ratio commonly referred to as PSNR and used to assess the image reconstruction efficiency. After that the damaged frame is repaired and the noise is hidden by the mistake.
TABLE. 1 DESCRIPTION OF THE VIDEO SEQUENCES
Video
Sequence
Information about the
objects
Information about
the shadow
Object size
Object speed
Noise level
Shadow Strength
4564 Shadow
Drone Small 10-15 High Low Low Baby Large 10-15 Normal High Normal
Peak Signal to Noise Ratio, commonly referred to as PSNR, used to calculate the image reconstruction accuracy. During the restoration of the broken element, the error concealment removes the noise. The higher PSNR shows the higher quality of the concealment. Using Mean Squared Error (MSE) to describe PSNR
(9)
Where,
I - Undamaged original frame
K- Reconstructed frame after error concealment m, n - Height and Width of the frame
Running Video
Reference Frame
Damaged Frame
Binary Image Boundary Recovery
Shadow Detection using Wavelet Transform in HSV
𝜇ΔWv = 2.0934
𝜎ΔWv = 10.8592
= 5.1873
HSV Color Model
ΔS ΔV
After Discrete Wavelet Trnasform
Concealed Damaged Frame with Shadows
FIGURE 1.ERROR CONCEALMENT WITH SHADOWS -RUNNING VIDEO
The PSNR is defined by
(10)
(11)
The input video files are converted into different bitrate ranges various from 110 Kbps to 150 Kbps and the PSNR value for different bitrate is calculated and it is represented in the table2.
Dancing Shadow
Reference Frame
Damaged Frame Binary Image Boundary Recovery
Shadow Detection using Wavelet Transform in HSV
𝜇ΔWv = 1.1374
𝜎ΔWv = 7.3249
= 6.4398
HSV Color Model
ΔS ΔV
After Discrete Wavelet Trnasform
Concealed Damaged Frame with Shadows
FIGURE 2.ERROR CONCEALMENT WITH SHADOWS -SHADOW DANCING VIDEO
TABLE. 2 DESCRIPTION OF THE VIDEO SEQUENCES
4565 (Kbps)
Running Dancing Shadow
110 3.78 4.0859
115 3.85 3.2618
120 3.91 2.9299
125 3.94 3.8153
130 3.92 3.7537
135 3.91 3.9145
140 2.85 3.9266
145 3.78 4.6258
150 3.65 4.1375
FIGURE 3.PSNRVS BITRATE FOR THE INPUT VIDEOS
6 CONCLUSION
AND
FUTURE
WORK
Due to network issues, a packet loss can occur when a video is transmitted over the network. Network latency, packet order, and packet loss are the limitations of video transmission over the Internet. The paths of the network are not static. When, for some reason, a path is inaccessible, the packets may be diverted to a longer route or a more latentious route. This packet loss can result in the loss of a video frame's important information. This research work masks the shadow pixels. The current algorithms for concealing errors are performed at the encoder and the shadow s decoder is assumed to be the context.The boundaries of the artifacts in the affected frame and reference frame are retrieved if there is any loss due to network problems. The affected objects are then replicated using global motion compensation based on artifacts. Comparing with the reference frame reconnects the discontinuous boundaries. In HSV color model, the shadow is detected by selecting the relative standard deviation calculated by discrete value component wavelet transformation. If the objects have shadows, this method will yield better results than existing methods of concealing errors. The videos selected are checked at various bit rates and measured for the respective PSNR values. The PSNR values are high, and if the corrupted videos have shadows, the quality of restoration will be increased. For complex and dark objects, this technique
can be improved in future work and with is shadows. Since this work is only implemented for object-based video coding, it is possible to implement further improvements for block-based video coding. The possibility of integrating animated videos ' shadow properties into object-based video coding.
7 REFERENCES
[1] Amit Phadikar, Santi P. Maity ―ROI Based Error Concealment of Compressed Object Based Image using QIM Data Hiding and Wavelet Transform‖ IEEE Transactions on Consumers Electronics, vol.56, no.2, pp. 971-979,2010.
[2] Cheng Huang and Paul Salama, "Error Concealment for Shape in MPEG-4 Object-Based Video Coding", IEEE Transactions on Image Processing, Vol.14, No. 4, pp.389-396, 2005.
[3] Chung.K.L, Lin.Y.R, and Huang.Y.H, ―Efficient Shadow Detection of Color Aerial Images based on Successive Thresholding Scheme,‖ IEEE Transactions on Geosciences and Remote Sensing, vol. 47, no. 2, pp. 671–682, Feb. 2009.
[4] Criminisi Prez & Toyama, K, ‗Region Filling and Object Removal by Exemplar-Based Image Inpainting‘, IEEE Transactions on Image Processing, vol. 13, pp. 1200-1212, 2004.
[5] Csaba Benedek and Tamás Szirányi,‖Study on Color Space Selection for Detecting Cast Shadows In Video Surveillance‖, International Journal of Imaging Systems and Technology, Volume 17, Issue 3, pp 190–201, 2007. [6] Dr. Zainab M. Hussain, Ali Ataeemah Jebur,‖ Shadow Detection and Removal in Video Sequence using Color-Based Method‖, Journal of Kerbala University, Vol. 11 No.4 Scientific, 2013.
[7] Evaggelia, T, Lisimachos, PK & Aggelos, KK, ‗Shape Error Concealment Based on a Shape-Preserving Boundary Approximation‘, IEEE Transactions on Image Processing, vol. 21, pp. 457-460, 2012.
[8] Guido M. Schuster, and Aggelos K. Katsaggelos, Fellow, IEEE‖ Motion Compensated Shape Error Concealment‖, IEEE Transactions on Image Processing, VOL. 15, NO. 2, Feb 2006.
[9] Henri N,‖Scalable Video Compression Scheme for Tele-Surveillance Applications Based On Cast Shadow Detection and Modelling‖, Image Processing, vol: 3, 2005.
[10] S. Sudhakar, V. Vijayakumar, C. Sathiya Kumar, V. Priya, Logesh Ravi, V. Subramaniyaswamy, 2019 Unmanned Aerial Vehicle (UAV) based Forest Fire Detection and monitoring for reducing false alarms in forest-fires, Elsevier- Computer Communications 149
(2020) 1–16,
https://doi.org/10.1016/j.comcom.2019.10.007.
[11] Jiangyan, D, Dianyuan, H & Xiaowei, Z, ‗Effective Moving Shadow Detection using Statistical Discriminate Model‘, Elsevier, vol. 126, pp. 5398-5406, 2010.
[12] Jie Li and Wenxuan Shi, ―Shadow Detection in Color Aerial Images based on HSI Space and Color Attenuation Relationship‖ Journal on Advances in Signal Processing, vol. 4, pp. 145–158,2012.
4566 Efficient Agent-Based Intrusion Detection System for
Detecting Malicious Nodes in MANET Routing‘, International Review on Computers and Software (I.RE.CO.S.), Vol. 7, No. 6, pp. 3037-304.
[14] Kuo-Liang Chung, Yi-Ru Lin and Yong-Huai Huang‖ Efficient Shadow Detection of Color Aerial Images based on Successive Thresholding Scheme‖ IEEE Transactions on Geosciences and Remote Sensing, vol.47, no.2, pp. 671-682, 2009.
[15] L. D. Soares and F. Pereira, "Temporal Shape Error Concealment by Global Motion Compensation with Local Refinement", IEEE Transactions on Image Processing, vol. 15, no. 6, pp. 1331-1348, 2006.
[16] L. D. Soares, F. Pereira, "Spatial Scene Level Shape Error Concealment for Segmented Video", IEEE Transactions on Multimedia, vol. 2, no. 3, pp. 185-190, 2000.
[17] S.Sudhakar, N.Satheesh, S.Balu, Amireddy Srinish Reddy, G.Murugan, 2019 Optimizing Joins in a Map-Reduce for Data Storage and Retrieval Performance Analysis of Query Processing in HDFS for Big Data, International Journal of Advanced Trends in Computer Science and Engineering , (IJATCSE), Vol.8 No 5, pp: 2062-2067.
https://doi.org/10.30534/ijatcse/2019/33852019
[18] Manish, K, Rajneesh, KS & Ashish, K, ‗Moving Shadow Detection and Removal – A Wavelet Transform Based Approach‘, The Institution of Engineering and Technology, vol. 8, pp. 1701-717, 2014.
[19] Mengyao Ma, Oscar, CA, Gary Chan, SH & Ming-Ting, S, ‗Edge-Directed Error Concealment‘, IEEE Transactions on Circuit Systems for Video Technology, vol. 20, pp. 382-395, 2010.
[20] Miao Qia, Jiangyan Daia,b, Qi Zhanga, Jun Konga,, ―Cascaded Cast Shadow Detection Method in Surveillance Scenes‖, Elsevier,Volume 125, No 3, pp 1396-1400,2014.
[21] Q. Zhang, W. Zhu and Y.-Q. Zhang ―Concealment Method For Shape Information In MPEG-4 Coded Video Sequences‖, IEEE Transactions on Multimedia, vol. 2, no. 3, pp. 185–190, Sep. 2000.
[22] Saritha Murali and Govindan.V.K, ―Shadow Detection and Removal from a Single Image using LAB Color Space‖, Cybernetics and Information Technologies, Sofia ,vol 13, no.1, 2013.
[23] Shih-Chang Hsia, Shyi-Chyi Cheng and Shih-Wen Chou ―Efficient Adaptive Error Concealment Technique for Video Decoding System‖ IEEE Transactions on Multimedia, vol.7, no. 5, 2005.
[24] Shih-Chang Hsia, Szu-Hong Wang and Ming-Huei Chen ―High Performance Region-of –Interest Image Error Concealment with Hiding TechniqueJournal of Electrical and Computer Engineering, vol. 10 pp. 971 - 979, 2010.
[25] Shiping Zhu, Zhichao Guo and Li Ma ―Shadow Removal with Background Difference Method based on Shadow Position and Edge Attributes‖ Eurasip Journal on Image and Video Processing, 2012.
[26] Sudhakar Sengan, Chenthur Pandian S, 2016, ‗Hybrid Cluster based Geographical Routing Protocol to
Mitigate Malicious Nodes in Mobile Ad Hoc Network, International Journal of Ad Hoc and Ubiquitous Computing, ISSN online: 8233; ISSN print: 1743-8225,Vol.21,No.4,pp:224-236,
DOI: 10.1504/ijahuc.2016.076358
[27] Wen-Jiin Tsai and Jian-Yu Chen ―Joint Temporal and Spatial Error Concealment for Multiple Description Video Coding‖ IEEE Transactions on Circuits and System for Video Technology, vol.20, no.12, pp. 1822-1833, 2010. [28] Yi Guo, Ying Chen, Miska M. Hannuksela ,"Error
Resilient Coding and Error Concealment in Scalable Video Coding", IEEE Transactions On Circuits And Systems For Video Technology, VOL. 19, NO. 6, pp 781-795, 2009.
[29] Jiandong Tian, Jing Sun, and Yandong Tang ‗Tricolor Attenuation Model for Shadow Detection‘, IEEE Transactions on Image Processing, vol. 18, no. 10,2009.
[30] Zelmira Tothova and Jarosalav Polec― echnique of Error Concealment Method Selection in
Texture Image using ALBP Classifier‖, Radio Engineering, vol. 19, no. 2, pp.331-337, 2010.
