International Journal of Research in Information Technology (IJRIT)
www.ijrit.com www.ijrit.comwww.ijrit.com
www.ijrit.com ISSN 2001-5569
Image-Based Rain Streaks Removal
S.Senthil kumar M.E Dr R.S.Rajesh Ph.D.,M.E
PG scholar
Department of computer science Engineering
Head of the Department
Department of computer science Engineering Manonmaniam sundaranar university Manonmaniam sundaranar university
Tirunelveli,Tamilnadu India Tirunelveli,Tamilnadu,India
Abstract
Rain removal from a video is a challenging problem and has been recently investigated extensively. Nevertheless, the problem of rain removal from a single image was rarely studied in the literature, where no temporal information among successive images can be exploited, making the problem very challenging. In this paper, input video is converted into JPEG image with constant size. We propose a single-image-based rain removal framework via properly formulating rain removal as an image decomposition problem based on bilateral filter and k means algorithm. Instead of directly applying a conventional image decomposition technique, the proposed method first decomposes an image into the low- and high-frequency (HF) parts using a bilateral filter. The HF part is removed in rain image then decomposed into a “k means algorithm”. As a result, the rain component can be successfully removed from the image while preserving most original image details. Finally images converted into video format. Experimental results demonstrate the efficacy of the proposed algorithm
Index Terms – k means,bilateral filter.
I. Introduction
Different weather conditions such as rain, snow, haze or fog will cause complex visual effects of spatial or temporal domains in images or videos[1]. Such effects may significantly degrade the performances of outdoor vision systems relying on image/video feature extraction or visual attention modeling, such as image registration, event detection, object detection, tracking, and recognition, scene analysis and classification, image indexing and retrieval, and image copy/near-duplicate detection[2]. To the best of our knowledge, current approaches are all based on detecting and removing rain streaks in a video. This paper is among the first to specifically address the problem of removing rain streaks in a single image[2]. Note that rain removal in an image may also fall into the category of the problem about image noise removal or image restoration. Hence, in the following subsections, we first briefly review current vision-based (video-based) rain removal approaches and image noise removal, followed by presenting our motivations of single-image-based rain streak removal and contribution of the proposed method[8].
2. Rain removing using Bilateral Filter
A bilateral filter is a non-linear, edge-preserving and noise-reducing smoothing filter for images[25]. The intensity value at each pixel in an image is replaced by a weighted average of intensity values from nearby pixels.[3] This weight can be based on a Gaussian distribution. Crucially, the weights depend not only on Euclidean distance of pixels, but also on the radiometric differences (e.g. range differences, such as color intensity, depth distance, etc.). This preserves sharp edges by systematically looping through each pixel and adjusting weights to the adjacent pixels accordingly. Adobe Photoshop implements a bilateral filter in its surface blur tool. GIMP implements a bilateral filter in its Filters-->Blur tools; and it is called Selective Gaussian Blur'.[11]
The bilateral filter is defined as
where the normalization term
Filtering is perhaps the most fundamental operation of image processing and computer vision. In the broadest sense of the term "filtering",[11] the value of the filtered image at a given location is a function of the values of the input image in a small neighborhood of the same location. For example, Gaussian low-pass filtering computes a weighted average of pixel values in the neighborhood, in which the weights decrease with distance from the neighborhood center. Although formal and quantitative explanations of this weight fall-off can be given, the intuition is that images typically vary slowly over space, [10]so near pixels are likely to have similar values, and it is therefore appropriate to average them together. The noise values that corrupt these nearby pixels are mutually less correlated than the signal values, so noise is averaged away while signal is preserved. The assumption of slow spatial variations fails at edges, which are consequently blurred by linear low-pass filtering.
3. Rain removal identification using Kmeans Algorithm
The intensity histogram of a pixel in a video taken by a stationary camera exhibits two peaks. K-means clustering algorithm can be used to identify the two peaks[25]. For each pixel in the image, its intensity over the entire video is collected to compute its intensity histogram. Then, K-means clustering with K = 2 is applied. The two initial cluster centers wb for background and wr for rain are initialized to be the smallest and the largest intensities of the histogram. The distance d between the intensity I of pixel p and cluster center w is computed as:
During K-means clustering, pixel p is distributed to the background cluster if d(Ip, wb) < d(Ip, wr); otherwise, it is distributed to the rain cluster. After distributing the intensities of the pixels, the center of cluster C is updated as follows:
K-means clustering is performed until converging to identify the clusters of background and rain intensities. The above method is appropriate for videos of stationary scenes taken by a stationary camera. For videos of stationary scenes taken by a moving camera, the method can still be applied after performing video stabilization to remove camera motion. Video stabilization is performed by warping every video frame to align with the first frame. [7]After removing rain, the stabilized video is 17 destabilized by performing inverse warping to restore the original camera motion
There are some possible ways to further improve the visual qualities of rain-removed images. In addition to collecting training exemplar patches from some training nonrain images for learning, we may extract training patches from the same or neighboring camera(s) when extending the proposed method to video rain removal. That is, we may extract exemplar patches from the neighboring rain-removed frames captured by intra/inter cameras in the same scene. Then, we can integrate the geometric sub dictionary obtained from the HF part itself and the extended dictionary learned from the recollected training patches to form the final geometric sub dictionary. Moreover, the shared-private factorization scheme proposed in may be used to further improve the performance of image decomposition. In two best leverage the information contained in each view of an image represented by multiple views/ modalities,[25] inspired by structured spare coding, the authors proposed an approach to learning factorized representations of multiview data in which the information is correctly factorized into components that are shared across several views and private to each view. The concept of shared-private factorization may be applied to further improve our work in two aspects.[18] First, a rain image can be segmented into several local regions with different local characteristics, which can be viewed as the multiple views.
We may apply the multiview learning to learn a latent space that can separate the information (rain atoms) shared among several views from the information (unique nonrain atoms) private to each view. Then, the two dictionaries for rain removal can be accordingly identified. Second, rather than learning two disjoint private dictionaries without any 26 common atoms for the rain and nonrain components, the two dictionaries may share some common atoms (i.e., the shared dictionary). Soft clustering (e.g., the fuzzy C-means) rather than hard clustering, or shared-private factorization, can be used to obtain better sparse representations.
4. K-means Clustering Algorithm for Rain Detection
The intensity histogram of each pixel in video presents to be the Mixture Gauss Model.[6] Considering that K-means Clustering is
raindrops, background, and even the moving objects[3]. In order to accelerate the processing rate of rain detection in dynamic scenes, concise two peak histogram model than three-peak one is established.[15]
5. Training K-means clustering for Stationary Scenes
In stationary scenes, the intensity of each fixed pixel over the entire video is stored to compute the histogram[15]. Then the obtained histograms are trained with Kmeans clustering, where K equals 2. As to each histogram, two cluster centers ωb for background and ωr for raindrops are initialized to be the minimum and maximum of the intensity. The Euclid distance d standing for the distance between the intensity I in pixel p and the cluster center ω is computed as the following equation,
Pixel p in frame n is divided in raindrop cluster, if d(Ip,ωr) is smaller than d(Ip,ωb); otherwise it is divided in background cluster.
After that, the center of cluster C is updated as follows:
K-means Clustering continues until the cluster centers converge. Considering the difference for recognition of raindrops between lower intensity background and higher one, we believe that higher intensity background appears, if distance between the higher ω and the maximum of the intensity is within the limit θl. In that case, the lower ω stands for ωb and the higher one stands forω r.
The contrary case also works. When cluster centers are confirmed, pixel p in each frame can be considered to be the background or rain. If temporal property of histogram model is taken into account, the obtained cluster centers can be applied to further frames in video
6 .Exiting methods
We conducted experiments on several videos with rain to demonstrate the effectiveness of our algorithms[2]. In all experiments, we chose the photometric threshold c = 3 and the spatio-temporal correlation was computed using (11×11) neighborhoods over f = 30 frames. The rain segmentation has a temporal lag since 30 frames were used to compute spatio- temporal correlation. Where a person is moving and speaking over the phone. Rain is visible through the window. The camera moves and zooms in on the person[2]. The detection task is challenging due to the fast moving textures. Despite these complexities, our algorithm robustly detects only pixels with rain. Note that we are unable to detect rain in pixels with a bright 22 background (white wall) because the changes in intensities produced by rain are very low. and the differences between the derained and original frames. Similar results are shown for a clip from the movie. Despite the fast back and forth motion of the person‟s head and the book, our algorithm detects only pixels affected by rain. Results for a scene with raindrops falling and forming a pool of water. The ripples of water may be viewed as a temporal texture with frequencies similar to those produced by rain. Even in this case, the algorithm detects only pixels with rain. These examples demonstrate that our algorithm is effective for scenes with complex motions and at the same time is insensitive to time-varying textures that have temporal frequencies similar to those due to rain. We have developed a comprehensive model for the visual appearance of rain. [3]Based on this model, we presented efficient algorithms for the detection and removal of rain from videos. Note that simple temporal filtering methods are not effective in removing rain since they are spatially invariant and hence degrade the quality of the image in regions without rain.
In contrast, our method explicitly detects pixels effected by rain and removes the contribution of rain only from those pixels, preserving the temporal frequencies due to object and camera Motions. The proposed algorithm can be used in a wide range of applications including video surveillance, video/movie editing, and video indexing/retrieval. Our models also have implications for efficient and realistic rendering of rain. Currently, we do not handle the steady effects of rain and we do not remove severely defocused rain streaks. In future work, we wish
To address these issues and also extend our analysis to other types of dynamic weather conditions such as snow and hail.
7. The Application of Histogram on Rain Detection in Video
We used JVC GR-HD1 camera to take a video with different intensity of background, including moving objects. The former frames are of stationary scenes, while the latter frames contain moving objects. Mat lab 7.0 is applied to store the data, simulate the clustering and calculate the boundary[25]. Main parameters are estimated and filled in the following table,
Based on the established histogram model and the algorithms of K-Means Clustering, rains in video are well detected and substituted by background information.
Figure: Illustrates our results on rain detection and removal.
In order to detect rain in video, improved histogram models containing temporal and spatial properties for stationary and dynamic scenes are established.[25] K-Means Clustering is applied to obtain the 24 centers of background and rain; moreover, a parameterized method is introduced in. Further research will be on real-time treatment using physical directional property and background segmentation, considering the misinformation of winged insects from motion-blurred rain and low-time processing[3][25].
8 .Experimental result
9. Conclusion
It have proposed a single-image-based rain streak removal framework by formulating rain removal as an bilateral filtering and k means algorithms. The k means and bilateral filter of the proposed method is fully automatic and self-contained where no extra training samples are required in the sample image. We have also provided an optional scheme to further enhance the performance of rain removal by introducing an extended dictionary of nonfan atoms learned from nonfan training images. Our experimental results show that the proposed method achieves comparable performance with state-of-the-art video-based rain removal algorithms without the need of using temporal or motion information for rain streak detection and filtering among successive frames.
For future work, the performance of our method may be further improved in terms of computational complexity and visual quality by enhancing the bilateral filter, k means processes. More specifically, since the dictionary learning and sparse coding consume most of execution time, the input image can be segmented into several local regions with different local characteristics such that the online dictionary learning for individual regions can be performed in parallel to accelerate the two processes by taking advantage of current multi core processor technology.[25] In addition, with the localized k means and bilateral filter, the number of patches of each local region for k means and the number of Atoms for bilateral will be significantly fewer than those for the whole-image-based removed, thereby further reducing the computational complexities of the two processes. Nevertheless, the impact of localized learning and sparse coding on rain removal performance needs in-depth investigation
10. REFERENCES
[1] P. C. Barnum, S. Narasimhan, and T. Kanade, “Analysis of rain and snow in frequency space,” Int. J. Comput. Vis., vol. 86, no. 2/3, pp.256–274, Jan. 2010.
[2] K. Garg and S. K. Nayar, “Detection and removal of rain from videos,”in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2004, vol. 1,pp. 528–535.
[3] K. Garg and S. K. Nayar, “When does a camera see rain?,” in Proc.IEEE Int. Conf. Comput. Vis., Oct. 2005, vol. 2, pp. 1067–
1074.
[4] K. Garg and S. K. Nayar, “Vision and rain,” Int. J. Comput. Vis., vol.75, no. 1, pp. 3–27, Oct. 2007.
[5] K. Garg and S. K. Nayar, “Photorealistic rendering of rain streaks,”ACM Trans. Graph., vol. 25, no. 3, pp. 996–1002, Jul.
2006.
[6] X. Zhang, H. Li, Y. Qi, W. K. Leow, and T. K. Ng, “Rain removal in video by combining temporal and chromatic properties,”
in Proc.IEEE Int. Conf. Multimedia Expo., Toronto, ON, Canada, Jul. 2006,pp. 461–464.
[7] N. Brewer and N. Liu, “Using the shape characteristics of rain to identify and remove rain from video,” Lecture Notes Comput.
Sci., vol.5342/2008, pp. 451–458, 2008.
[8] J. Bossu, N. Hautière, and J. P. Tarel, “Rain or snow detection in image sequences through use of a histogram of orientation of streaks,” Int. J.Comput. Vis., vol. 93, no. 3, pp. 348–367, Jul. 2011.[9] M. S. Shehata, J. Cai, W. M. Badawy, T. W. Burr, M. S.
Pervez, R.
J. Johannesson, and A. Radmanesh, “Video-based automatic incident detection for smart roads: The outdoor environmental challenges regarding false alarms,” IEEE Trans. Intell. Transp. Syst., vol. 9, no. 2, pp. 349–360, Jun. 2008.
[10] M. Roser and A. Geiger, “Video-based raindrop detection for improved image registration,” in IEEE Int. Conf. Comput.
Vis.Workshops, Kyoto, Sep. 2009, pp. 570–577.
[11] J. C. Halimeh and M. Roser, “Raindrop detection on car windshields using geometric–photometric environment construction and intensitybased correlation,” in Proc. IEEE Intell. Veh. Symp., Xi’an, China, Jun.
2009, pp. 610–615.
[12] Google Goggles [Online]. Available: http://www.google.com/mobile/ goggles/
[13] D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis., vol. 60, no. 2, pp. 91–110, Nov. 2004.
[14] H. Baya, A. Essa, T. Tuytelaarsb, and L. V. Gool, “Speeded-up robust features (SURF),” Comput. Vis. Image Understand., vol. 110, no. 3, pp. 346–359, Jun. 2008.
[15] N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., San Diego, CA, Jun. 2005, vol. 1, pp. 886–893.
[16] O. Ludwig, D. Delgado, V. Goncalves, and U. Nunes, “Trainable classifier-fusion schemes: An application to pedestrian detection,” in Proc. IEEE Int. Conf. Intell. Transp. Syst., St. Louis, MO, Oct. 2009, pp. 1–6.
[17] S. Maji, A. C. Berg, and J. Malik, “Classification using intersection kernel support vector machines is efficient,” in Proc.
IEEE Conf. Comput. Vis. Pattern Recognit., Anchorage, AK, Jun. 2008, pp. 1–8.
[18] L. Itti, C.Koch, and E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254–1259, Nov. 1998.
[19] A. Buades, B. Coll, and J. M. Morel, “A review of image denoising algorithms, with a new one,” Multisc. Model. Simul., vol.
4, no. 2, pp. 490–530, 2005.
[20] M. Elad and M. Aharon, “Image denoising via sparse and redundant representations over learned dictionaries,” IEEE Trans.
Image Process., vol. 15, no. 12, pp. 3736–3745, Dec. 2006.
[21] M. Aharon, M. Elad, and A. M. Bruckstein, “The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representation,” IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311–4322, Nov. 2006.
[22] J. Mairal, M. Elad, and G. Sapiro, “Sparse representation for color image restoration,” IEEE Trans. Image Process., vol. 17, no. 1, pp. 53–69, Jan. 2008.
[23] M. Elad, Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. New York:
Springer-Verlag, 2010.
[24] J. Mairal, F. Bach, and J. Ponce, “Task-driven dictionary learning,” IEEE Trans. Pattern Anal. Mach. Intell., to be published, to be published.
[25] Y. H. Fu, L. W. Kang, C. W. Lin, and C. T. Hsu, “Single-frame-based rain removal via image decomposition,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., Prague, Czech Republic, May 2011,
