REVIEW ON VIDEO ENHANCEMENT TECHNIQUES

(1)

Bhagya H.K and Keshaveni N ijesird, Vol. III, Issue II, August 2016/134

REVIEW ON VIDEO ENHANCEMENT TECHNIQUES

¹Bhagya H.K, ²Keshaveni N

¹Asso.Prof., ²Professor ECE Dept., K.V.G.C.E., Sullia

1[email protected], ²[email protected] ABSTRACT: Video enhancement is one of the most important

and difficult components in video research. The aim of video enhancement is to improve the visual appearance of the video, or to provide a better transform representation for future automated video processing, such as analysis, detection, segmentation, recognition, surveillance, traffic, criminal justice systems. In this paper overview of video enhancement processing techniques and analysis algorithms are described. The study is further going on to find a technique so that more accuracy can be obtained in 3D Video enhancement.

Keywords: Video enhancement, Self-enhancement, Frame-based fusion enhancement, Contrast enhancement, Histogram equalization.

1. INTRODUCTION

Video enhancement problem can be formulated as follows: given an input low quality video and the output high quality video for specific applications.

How can we make video more clearer or subjectively better? Digital video has become an integral part of everyday life. The prime intention of video enhancement is to bring out detail information that is hidden in video [1]. There are numerous applications where digital video is acquired, processed and used, such as surveillance, general identity verification, traffic, criminal justice systems, civilian or military video processing etc.

Carrying out video enhancement with low quality video is a challenging problem because of the following reasons

(i) Due to low contrast; we cannot clearly extract moving objects from the dark background.

(ii) The signal to noise ratio is usually very low due to high ISO (ISO is the number indicating camera sensors sensitivity to light). Using a high ISO number can produce visible noise in digital photos. Low ISO number means less sensitivity to light.

(iii)The information carrying video signal is a degraded version of a source or original.

(iv) Environmental information affects the way people perceive and understand what has happened. Hence, dealing with moving tree, fog rain, and behavior of people in night time videos are difficult because they lack background context due to poor illumination.

(v) Inter frame coherence must also be maintained, that is the moving objects regions as weights in successive images should change smoothly.

(vi) The poor quality of the video device used and lack of expertise of the operator.

The existing techniques of video enhancement can be classified into two main methods, namely spatial- based domain and frequency- based domain. Spatial based refers to the image plane itself and direct manipulation of pixels in an image.

Frequency- based domain processing techniques are based on modifying the spatial frequency spectrum of the image as obtained by transform. The main advantage of spatial- based domain technique is that they are conceptually simple to understand and the time complexity of these techniques is low which favors real time implementations. But these techniques generally lacks in providing adequate robustness and imperceptibility requirements. The advantage of transform- based video enhancement include (i) Low complexity of computation (ii) Ease of viewing and manipulating the frequency composition of the image (iii) the easy applicability of special transformed domain properties. The basic limitation including (i) it cannot simultaneously enhance all parts of the image very well and (ii) it is difficult to automate the image enhancement procedure.

The existing techniques of video enhancement can be classified into two broad categories. Self- enhancement and frame based fusion enhancement.

Self enhancement is a technique in which video frames can enhance themselves automatically as

(2)

Bhagya H.K and Keshaveni N ijesird, Vol. III, Issue II, August 2016/135 shown in figure 1. The advantage of this one is

likely to be straightforward in implementation;

however its drawback is that in case of extremely dark original videos, it is difficult to get the high quality video because some details might be lost during the processing.

Context-based fusion enhancement is another technique in which a night – time video with low quality will be improved by combining with a high quality day time image. As a result, this method can give a high quality output, but the algorithm which is used for fusion is very complex. Instead of this method uncomplicated algorithms and time effective processing methods on self – enhancement are used for video enhancement. Figure 2 shows an example of context-based fusion technique.

The methods are implemented in MATLAB software in order to analyze the performance analysis. The future work is to design and develop a technology which will give better efficiency in 3D video enhancement

2. STUDY OF SELF-ENHANCEMENT TECHNIQUES

2.1.ENHANCEMENT OF LOW LIGHTING VIDEO

Xuan Dong, et.al has proposed a novel and effective video enhancement algorithm for low lighting video. The algorithm is first inverting an input low lighting video and then applying an optimized image de-haze algorithm on the inverted video [2]. To facilitate faster implementation, temporal correlations between subsequent frames are utilized. Simulation results show excellent enhancement results and 4xspeed up as compared with the frame-wise enhancement algorithms.

2.2.SPATIO TEMPORAL VIDEO CONTRAST ENHANCEMENT

This paper presents a video contrast adjusting using spatio-Temporal Histogram specification (HS) method. They apply the conventional 2-D HS to the temporal domain to adjust the spatio- temporal contrast of video signals. The increase of the temporal contrast does not ensure the enhancement of the video quality. However, for the applications such as visual surveillance, the

temporal contrast as well as the spatial contrast needs to be adjusted to enable discriminability of slight changes of the video sequences. Experimental results show the effectiveness and suitable for real time contrast enhancement applications [3].

2.3. VIDEO ENHANCEMENT USING TONE ADJUSTMENT

D. BakkiyaLakeshmi et.al has proposed a video enhancement framework consisting of bilateral Tone adjustment and saliency – weighted contrast enhancement. This method integrates the saliency map with a simple contrast enhancement and also performs more enhancements in regions that humans give more attention. This work shows that SWCE achieves greater performance using luminance component [4]. The enhanced frame has significantly higher signal to noise ratio (SNR) relative to the original frames of both gray scale and color videos.

2.4. AN EFFECTIVE NIGHT VIDEO ENHANCEMENT ALGORITHM

This paper presents an approach to enhance the context of night-time video. There are several problems of existing techniques for night video enhancement. The authors described a simple enhancement algorithm for night video surveillance applications which will give better results. The algorithm uses an additive enhancement term with foreground object extraction and constrained low- passed object illumination to avoid light-inversion and sensitivity problems and to reduce ghost patterns introduced by illumination ratio variations.

Simulation results have demonstrated the effectiveness of the proposed algorithm [5].

2.5 DEPTH GUIDED ADAPTIVE CONTRAST ENHANCEMENT USING 2D HISTOGRAMS

Mohammad Moinul Islam et.al has proposed a depth-guided Contrast enhancement (CE) algorithm using 2D histograms. They first introduced a depth-guided 2D histogram, which assigns high priorities to foreground pixels in order to stretch the gray-level differences of adjacent foreground pixels more strongly than those of adjacent background pixels. They derived the foreground and

(3)

Bhagya H.K and Keshaveni N ijesird, Vol. III, Issue II, August 2016/136 background transformations separately and

combined them adaptively according to pixel depths. Experimental results demonstrated that the proposed algorithm outperforms the conventional CE algorithms by enhancing salient foreground objects efficiently and preserving background details faithfully [6].

2.6. AN EFFICIENT VIDEO ENHANCEMENT METHOD USING LA*B* ANALYSIS

This paper presents an efficient technique for real time enhancement of video containing inconsistent and complex conditions like non uniform and insufficient lighting. This method provided a better approach to enhance a video in low lighting conditions without any loss of color information. This approach was based on histogram manipulation on La*b* color model of the video frame. The algorithm provides an effective way for video enhancement with simple computational procedures and makes real time enhancement for homeland security applications.

The results are analyzed on videos which are taken under bad lighting conditions [7].

2.7. SUPER –RESOLUTION ENHANCEMENT TECHNIQUE FOR LOW RESOLUTION VIDEO

In this paper, a novel technique for video super- resolution using kernel regression is presented. The kernel regression method uses different weights to neighboring pixels and reduces the blur effect and it is computationally efficient and hence suitable for use in small processor devices like cell phone and surveillance camera to increase resolution.

Simulation results showed that the proposed algorithm performs better than the other state of the art resolution enhancement techniques [8].

2.8. REAL-TIME VIDEO ENHANCEMENT ON FPGA BY SELF-ENHANCEMENT TECHNIQUE

Nguyen Thanh Sang et.al has proposed two efficient methods towards real-time applications that aim to be launched in dedicated hardware platform. After analyzing and evaluating the results of these proposed methods on MATLAB, they chose the best one in RGB Enhancement, to implement the design in a pipeline hardware

structure on FPGA chip and establish an experimental system to demonstrate the reliability of the proposed method. The experimental results show that the appearances of output videos are enhanced significantly with almost objects that may not be observed in the original videos being recognized. This suggests that the proposed algorithm and the proposed hardware structure are feasible and flexible to Integrate into current cameras, serving applications such as in military, medicine, traffic monitoring, driving assistance, and video surveillance system [9].

2.9. A NOVEL FRAMEWORK FOR

EXTREMELY LOW-LIGHT VIDEO

ENHANCEMENT

In this paper, they proposed a novel framework for enhancement of very low-light video. For noise reduction, motion adaptive temporal filtering based on the Kalman structured updating is presented.

Dynamic range of denoised video is increased by adaptive adjustment of RGB histograms. Finally, remaining noise is removed using Non-local means (NLM) denoising. The proposed method exploits color filter array (CFA) raw data for achieving low memory consumption [10].

2.10. CONTRAST LIMITED ADAPTIVE HISTOGRAM EQUALIZATION BASED ENHANCEMENT FOR REAL TIME VIDEO SYSTEM

Garima Yadav et.al proposed a CLAHE enhancement method for improving the video quality in real time system. Adaptive histogram equalization (AHE) is different from normal histogram equalization because AHE uses several methods, each corresponding to different parts of image and used them to redistribute the brightness value of the image and in case of CLAHE

„Distribution‟ parameter are used to define the shape of histogram, which produce the better quality result than adaptive histogram equalization [11].

Drawback

(4)

Bhagya H.K and Keshaveni N ijesird, Vol. III, Issue II, August 2016/137 (i) AHE can work over homogeneous fog but

CLAHE applied over both homogeneous and heterogeneous fog and single image and video system.

(ii) AHE uses „cumulation function‟ which can be applied over only gray level image but CLAHE uses both colored and gray level images.

2.11 DAY COLOR TRANSFER BASED NIGHT

VIDEO ENHANCEMENT FOR

SURVEILLANCE SYSTEM

Soumya T et.al presented a Day color transfer method for night video enhancement. The algorithm initially estimated the night video background and illumination map. A sample based color transfer is applied to the frame fusion video. Objective measures are applied to evaluate the performance of their method. The experimental results showed that more visual perception compared to existing night video enhancement techniques. They are planned to extend their work using “fuzzy C means clustering”

for clustering different illuminated regions [12].

2.12. A PIECEWISE-BASED CONTRAST ENHANCEMENT FRAMEWORK FOR LOW LIGHTING VIDEO

Dongsheng Wang et.al presented the algorithm is based on a piecewise stretch on the brightness component extracted with Retinex theory in HSV space to improve the visuality of the image. By dividing the brightness component into dark and bright part, nonlinear transformations with different distribution assumption were performed respectively. All the model parameters were estimated automatically according to the illumination conditions. Experimental results showed that the algorithm can achieve satisfactory effect for night time image or video enhancement by comparing with some state-of-the-art approaches [13].

2.13. CONTRAST ENHANCEMENT IN VIDEO SEQUENCES USING VARIABLE BLOCK SHAPE ADAPTIVE HISTOGRAM

EQUALIZATION

In this paper they presented a new method of contrast enhancement in high dynamic range images. This technique is a generalization of the contrast limited adaptive histogram equalization, working on image segments obtained by the mean shift clustering technique, instead of working on rectangular image blocks. A new image interpolation technique is designed in order to avoid artifacts if segment borders do not coincide with object borders. With the proposed contrast enhancement method, meaningful parts in the image are preserved and enhanced effectively.

Comparative tests revealed performance improvements over the traditional contrast limited adaptive histogram equalization [14].

2.14. DEPTH VIDEO ENHANCEMENT BASED ON WEIGHTED MODE FILTERING

Dongbo Min et.al presented a novel approach for providing high-quality depth video in a system that consists of a color and a depth camera. First, the low-quality depth maps, which are of low-resolution and noisy, are up sampled by the proposed WMF method. It provides the results that have better edge-preserving performance. The experimental results show that the performance of the proposed method is superior to the existing methods. Since the computational complexity does not depend on the number of depth candidates, the proposed method is very efficient [15].

2.15. 3D VIDEO ENHANCEMENT BASED ON HUMAN VISUAL SYSTEM

CHARACTERISTICS

A. Neri et.al presented an enhancement method for 3D videos based on specific characteristics of human visual perception. The spatial and temporal organization of the observed scene is employed to control the employed adaptive multi-resolution edge enhancement process. The performed subjective experimental tests show that improved quality and greater realism for 3D representation can be obtained when applying the proposed approach [16].

(5)

Bhagya H.K and Keshaveni N ijesird, Vol. III, Issue II, August 2016/138 (a) (b)

Fig 1 Example of Self-enhancement technique (a) Original image (b) processed image

(a) (b) (c)

Fig 2 Example of Context-based technique (a) A frame of night-time video (b) a high quality day-time image (c) an enhancement result

(6)

Bhagya H.K and Keshaveni N ijesird, Vol. III, Issue II, August 2016/139 3. COMPARISON ANALYSIS

Table-1 Comparison of various Enhancement Techniques

4. CONCLUSION

This paper presents a survey of different types of methods and technologies that have been used for video enhancement. But the low contrast and noise remains a barrier to visually pleasing videos in low light conditions. In that condition, to find out a more accuracy in video enhancement process there is a need to detect and measure the intensity

level of individual pixel channel as well as have to present an appropriate enhancement factor for enhancement purpose, so that effective and efficient video enhancement process will be created. In future the 3D video enhancement process will measure the intensity level of individual pixels channels and decide the best enhancement factor which might be random or constant depends on the requirement of video enhancement algorithm and measure the performance parameters.

Author Year Operating domain Enhancement Techniques

Advantages/ disadvantages Application Mohammad 2010 Spatial domain Super resolution

technique

Uses Different weights to neighboring pixels &

reduces the blur effect and inexpensive

Suitable for use in small processor

devices Xuan dong 2011 Spatial domain Image inverting

model

4x speedups is achieved with no visible of critical

information

Traffic monitoring, medical image.

Dongo Min 2011 Spatial domain Weighted mode filtering

The performance is superior to the existing

methods

3D TV, 3D object modeling, robot vision and tracking.

D.

BakkiyaLakshmi

2012 Spatial domain Tone adjustment Higher signal to noise ratio relative to original frames of both gray scale and color

videos.

Identifying people, license plates etc.

Jun Jee 2014 Spatial domain ACE using 2D histogram

The algorithm Outperform the CE algorithms by

enhancing salient foreground objects efficiently & preserving

back ground details.

Depth video enhancement

Garima yadav 2014 Spatial domain CLAHE AHE works is work over homogeneous fog but CLAHE applied for both

homogeneous and heterogeneous fog and single image and video

system.

Used in video Real time system.

Dongsheng wang

2014 Frequency domain Piecewise based CE

It can effectively improve the brightness of nighttime video and enhance its details to higher visibility

Field of our daily life application like production quality inspection, security

monitoring, emergency alert etc.

Somya T 2015 Spatial domain Day color Transfer based

night video

More visual perception compared to night video enhancement techniques.

Various video analysis operations.

Nguyen Thanh sang

2015 Spatial domain Real time video enhancement on

FPGA

Implementing the video enhancement in real time.

The proposed method and hardware structure are

effective and feasible

Military, medicine, traffic monitoring, driving assistance,

and video surveillance system.

(7)

Bhagya H.K and Keshaveni N ijesird, Vol. III, Issue II, August 2016/140 REFERENCES

1. Hiding Yunbo Rao, Leiting Chen, “A Survey of Video Enhancement Techniques, Journal of Information and Multimedia Signal Processing”, 2012 ISSN 2073-4212 Volume 3, Number 1, January 2012.

2. Xuan Dong, Jiangtao (Gene) Wen, Senior Member, IEEE, Weixin Li, Yi (Amy) Pang, Guan Wang,” An Efficient and Integrated Algorithm for Video Enhancement in Challenging Lighting Conditions”. ArXiv: 1102.3328v1 [cs.GR] 16 Feb 2011.

3. Turgay Celik^1,2,3 “ Spatio- temporal video contrast enhancement”, IET Image process, 2013. Vol.7 Iss. 6, pp.543- 555.

5. Yanbo Rao, Zhongho Chen, Ming- Ting Sun, Yu-Feng Hsu, Zhengyou Zhang, “ An effective night video enhancement algorithm”.

6. Jun- Tae Lee, Chulwoo Lee, Jae-Young Sim and Chang-Su Kim, “Depth Guided Adaptive Contrast Enhancement Using 2D Histograms”, 978-1-4799-5751-4/14/$31.00@2014 IEEE.

7. Gaurav Mittal, Sushrutha Locharam & Sreela Sasi Glenn R.

Shaffer & Ajith K. Kumar, “An Efficient video enhancement method using La*b* analysis” Proceedings of the IEEE International Conference on Video and Signal Based Surveillance 0-7695-2688-8/06$20.00@2006 IEEE.

8. Mohammad Moinul Islam, Mohammed Nazrul Islam, “Super Resolution Enhancement Technique for Low Resolution Video”

0098 3063/10/$20.00@ 2010 IEEE.

9. Nguyen Thanh Sang, Vu Nguye Binth, Truong Quang Vinh, Vu Duc Lung , “ Real time Video Enhancement on FPGA by Self- enhancement Technique” 2015 International Conference on Advanced Technologies for communication, 978-1-4673- 8374-5/15/$31.00@ 2015 IEEE.

10. Minjae Kim, Dubok Park, David k. Han and Hanseok Ko, “A Novel Framework for Extremely Low – light Video Enhancement”, 2014 IEEE International Conference on Consumer Electronics 978-1-4799-1291-9/14/$31.00 @2014 IEEE.

11. Garima Yadav, Saurabh Maheshwari, Anjali Agarwal, “ Contrast Limited Adaptive Histogram Equalization based Enhancement for Real Time Video System” , 2014 International Conference on Advances in Computing , Communications and Informatics, 978-1-4799-3080-7/14/$31.00 @ 2014 IEEE.

12. Soumya T, Sabu M Thampi, “Day Color Transfer Based Night Video Enhancement for Surveillance System”, 978-1-4799- 1823-2/15/$31.00 @2015 IEEE.

13. Dongsheng Wang, Xin Niu and Yong Dou, “A piecewise based Contrast Enhancement Framework for Low Lighting Video”.

2014 IEEE International Conference on Security, Pattern Analysis, and Cybernetics, 978-1-4799-5353- 0/14/$31.00@2014 IEEE.

14. Bernd Hillers, Vasile Gui, Axel Graeser, “ Contrast enhancement in Video sequences using variable block shape Adaptive Histogram equalization”.

15. Dongbo Min, Jiangbo Lu, and Minh N. Do, “Depth Video Enhancement based on Weighted Mode Filtering”, IEEE

Transaction on Image Processing, VOL. 21, No.3, March 2012.

16. A. Neri, P. Campisi, E. Maiorana, F. Battisti “3 D Video enhancement based on human visual system characteristics”.