2017 2nd International Conference on Artificial Intelligence and Engineering Applications (AIEA 2017)
ISBN: 978-1-60595-485-1
Seam Carving Algorithm for Maintaining the Shape
Structure of Significant Objects
ZHENGHONG GUO and JUNHUA ZHANG
ABSTRACT
Seam carving algorithm can cause the loss of the important content and the destruction of the shape of significant objects in an image. We propose a method that maintains the important content of the image and the shape of significant objects in image scaling. This method combines the gradient graph of an image and the significant graph to redefine the significant energy, which can highlight the important content of the image and maintain the shape of significant objects. The experimental results show that the improved algorithm can better protect the important content in image scaling, and can better maintain the shape structure of the object.
KEYWORDS
Seam Carving Algorithm, Significant Objects, Gradient Graph, Significant Energy, Image Scaling.
INTRODUCTION
Normally, an image has a fixed resolution, but it needs to be adapted to the needs of various display devices in different situations. If the aspect ratio of the image is different from the available display area, the image should be adjusted to fit the corresponding layout. The process of changing the image size to display the best image is generally referred to as image scaling1.
The traditional image scaling technology mainly includes two methods of uniform scaling and cutting technology2. Both of these methods are easy to implement, but both methods only consider the geometric constraints of the scaled image, which is prone to the loss of important image content. For example, the uniform scaling technique does not take into account the content of the image when the image is scaled, especially when the aspect ratio is relatively large, which causes the image visual subject to be deformed. The cutting technique requires the appropriate window, since the main content of the image will be lost when the scale size is significantly reduced. As shown in Figures.1 (b), the traditional image scaling technology makes the little girl deform, the effect is poor.
In order to overcome these problems, Avidan ET al.3 proposed a Seam Carving algorithm that preserves the content of image, using the gradient graph as an energy graph, finding the optimal seam for the image by dynamic programming, removing or inserting the optimal seam to achieve image scaling.
_________________________________________
(a) Original image (b) Traditional scaling (c) Seam carving Figures 1. Comparison of Traditional Scaling Technique and Seam Carving scaling effect.
Compared with the traditional image scaling technology, Seam Carving algorithm has achieved better results. However, this method only uses the gradient value of the pixel as the energy value, the
Gradient calculation reflects the edge information of the image. As shown in Figures.1(c), when the visual subject of the image contains a large amount of low energy information, if the texture of the visual subject is not very rich, the cutting line will pass through the visual subject, and it is easy to cause the visual subject to deform when scaling the image.
In view of the shortcomings of Seam Carving algorithm, many scholars at home and abroad have proposed many different improved methods4-7. Rubinstein et al.7 improved the seam cutting method by introducing forward energy, which overcomes the scaling distortion caused by the energy change of the original method. Zhao et al.8 improved the Seam Carving algorithm by changing the energy function. Lin et al.9 proposed an image scaling method based on seam cutting and deformation, which avoids the structural damage of the important content of the image. Jia et al.10 proposed task driven saliency detection for image retargeting. Zou et al.11 used Gaussian difference to perform corner detection on the image, and proposed an image scaling algorithm based on the subject region preserving.
In order to highlight the important content of the image, based on the Seam Carving image scaling algorithm, this paper redefines the significant energy by combining the gradient graph of the image with the significant graph generated by the graphically significant analysis algorithm12. The algorithm highlights the important content of the image, better maintaining the shape structure of the significant object in the image.
IMPROVED SEAM CARVING
Calculation of gradient energy graphs
AssumingIis a given image, the size isn m , and then the gradient energy function of the image is defined as:
( )
e I I I
x y
(1)
That is, the sum of the absolute values of the gradient in the x-direction and the y-direction is the energy value of the pixel. The larger the energy value of the image pixel, indicating that the pixel is the high energy pixel value that needs to be maintained, the greater the importance of the image, and vice versa the low energy pixel value, the less important the image.
Calculation of saliency graphs
Significant energy can measure the importance of different regions in the image, and the higher the importance of the region, the higher the energy value in the significant graph. The graph-based Visual Saliency algorithm is a visual significance model constructed by the bottom-up method considering the local features. It mainly includes two steps: firstly, the activation graph is formed on the characteristic channel, that is, from the input image to calculate a significant map of the original image, and then from the image direction, color and brightness and other characteristics, the activation diagram of the image from the significant graph calculated. Secondly, the activation map of the image is normalized in a prominent way, and then averaged from the saliency of each significant region, and the saliency values of all significant regions were added. The GBVS model uses the Markov chain, and the information in the image area is obtained by defining the dissimilarity. Assuming that the characteristics of nodes i j, andp q, areM i j, andM p q, respectively, the dissimilarity of these two nodes can be defined as:
, ,
log
,
,M i j
d i j p q
M p q
(2)
At the same time we can define the weight coefficients of nodes that are fully connected between nodes i j, andp q, according to formula (2):
1 , , , , , ,
w i j p q d i j p q F ip j q (3)
Where:
2 2
2
, exp
2
a b
F a b
(4)
The construction of a new energy map
, , ,
E i j a e i j b g i j (5)
Whereaandbare image weighting free parameters, the range is [0, 1] and satisfies
1
a b . The new energy function is constructed to take into account the global
information of the image, and the important pixel points in the image are enhanced. The experimental results show that the Seam Carving algorithm to improve the effect is better when the range of a is in the range of [0.08, 0.1], b is in the range of [0.9, 0.92]. In order to facilitate the experiment, the experimental value isa0.1,b0.9.
The determination of the cutting line
Assuming that the size of the imageIisn m , and define a vertical seem to be:
1 1 , , , . , 1 1
n n
x x
i i x i i i s t i x i x i
s s (6)
Wherexis a mappingx:[1,2,…,n]→[1,2,…,m], that is, pixels from top to bottom contains only one pixel per line in the image. Similarly define a horizontal seem to be:
1 1 , , , . , 1 1
m m
y y
j j j y j j s t j y j y j
s s (7)
Whereyis a mappingy:[1,2,…,m]→[1,2,…,n], that is, pixels from left to right contains only one pixel per column in the image.
Take the vertical seam
{ }
s
i as an example, and the pixels of the path of seamswilltherefore be
1
,
1n n
s i i
i
I I s I x i i
. Similar to removing a row or column from an image, removing the pixels of a seam from an image has only a local effect. Assuming that we remove a vertical crop line from an image, the crop line is determined by the smoothness of the image from top to bottom. The vertical crop lineshas the following selection of the pixels in the adjacent two lines: above, top left and top right of the three cases.
If an energy function e is given, we can define the cost of a seam as:
1
ns i i
E s E I
e I s. Then we look for the optimal seamsthat minimizes this seam cost:
1
min min n i
s s
i
s E s e I s
(8)Take vertical seam as an example, we can use the dynamic programming method to find the optimal seam. The first step is to traverse the image from the second row to the last row and compute the cumulative minimum energyMfor all possible connected
M i j
, e i j, min
M i
1, 1 ,j
M i1, ,j M i
1, 1j
(9)At the end of this process, the minimum value of the last row inMwill indicate the end of the minimal connected vertical seam. Hence, in the second step we backtrack from this minimum entry onMto find the path of the optimal seam. The definition of
Mfor horizontal seams is similar.
RESULTS AND DISCUSSION
In order to further illustrate the experimental results, several different images are selected, and the algorithm is compared with the traditional scaling technique and the Seam Carving algorithm.
Figures.2 is the image scaling effects of two groups of original images which are reduced by the same number of columns with three different algorithms respectively. As can be seen from Figures.2(b), when the image is horizontally scaled, the traditional scale method causes the little girl in the red box and the building in the green box to be seriously deformed; As shown in Figures.2(c), the Seam Carving method destroys the shape of the little girl and the building in the image, indicating
(a) Original image 1 (b) Traditional scaling 1 (c) Seam carving 1 (d) our result 1
(a) Original image (b) Traditional scaling
(c) Seam carving (d) our result Figures 3. Comparison of vertical scaling effects.
(a) Original image (b) Traditional scaling
(c) Seam carving (d) our result Figures 4. Comparison of horizontal and vertical scaling effects.
that the cutting line passes through the image subject area during the cutting process, resulting in damage to the shape of the little girl and the building in the image, the overall visual effect of the image is poor; Figures.2 (d), which is narrowed by this algorithm, not only preserves the important contents of the image, but also maintains the shape structure of the little girl and the building. The overall scale effect is obviously better than the previous methods.
TABLE 1.COMPARISON OF THE PROPERTIES OF DIFFERENT THREE IMAGE SCALING METHODs.
Image scaling algorithm Content of the imageKeep the important
Maintain the shape structure of a significant object
Traditional scaling method No No
Seam carving algorithm Yes No
Our algorithm Yes Yes
Thus the scaling effect is not satisfactory; as can be seen From Figures.3 (d), the structure of the bus of the image by this algorithm remains unchanged, and the overall
Visual effect is better than Figures.3 (b) and Figures.3(c), can effectively maintain the important content and the shape structure of the bus in the image.
Figures.4 is the effect of the original image with three different algorithms at the same time the image width and height scaling, that is, vertical and horizontal scale. In this image, we are more likely to focus on the building in the image, which is an important part of the image. As can be seen from the red rectangle in Figures.4(b), the building is severely deformed in the image scaled by the traditional scale method; As can be seen from Figures.4(c), the overall effect of the image scaled with Seam Carving is better than Figures.4(b), but it can be seen that the building of the red rectangle can be found to be more obvious shrink than the original image; As can be seen From Figures.4(d), the image scaled by our algorithm can maintain the shape structure of the building in the image well, and the overall visual effect is obviously better than the previous methods.
In order to better illustrate the experimental results, we have several scaling methods in the image scaling to maintain the shape structure of significant object of the image, to maintain the important content of the image in the form of a table for comparative analysis, the results shown in TABLE 1.
CONCLUSION
On the basis of the Seam Carving algorithm, this paper redefines the significant energy by the combination of the gradient graph of the image and the significant graph. This method not only highlights the important content of the image but also maintains the shape of the obvious object in the image. It is further ensured that the cutting lines rarely pass through the high-energy visual subject area when inserting or removing the cutting lines, thus protecting the important contents of the image. The experimental results show that the improved algorithm has better scaling effect than the Seam Carving algorithm in image scaling, which not only protects the important content of the image, but also preserves the shape structure of significant object of the image.
ACKNOWLEDGEMENTS
REFERENCES
1. Wang Q., Yuan Y. High quality image resizing [J]. Neurocomputing, 2014,131 (9): 348-356. 2. Xiao L.,Yang S., Li-zhuang M.Image Resizing Based on Shape and Structure preserving of Salient
Objects [J]. Computer Science, 2014, 41(12): 288-292.
3. Avidan S., Shamir A. Seam Carving for Content-Aware Image Resizing [J]. Acm Transactions on Graphics, 2007, 26(3): 10.
4. He Z., Gao M., Yu H., et al. A new improved seam carving content aware image resizing method [C]// Industrial Electronics and Applications. IEEE, 2013: 738-741.
5. Yin T., Yang G., Li L., et al. Detecting seam carving based image resizing using local binary patterns [J]. Computers & Security, 2015, 55(C): 130-141.
6. Ye J., Shi Y Q. An effective method to detect seam carving [J]. Journal of Information Security & Applications, 2017, 35: 13-22.
7. Rubinstein M., Shamir A., Avidan S. Improved seam carving for video retargeting. [J]. Acm Transactions on Graphics, 2008, 27(3): 16.
8. Zhao W.W., Zhang J.H., Wang X. Seam carving with improved energy function for image resizing [J]. Journal of Yunnan University (Science and Technology). 2014, 36(2): 181-186.
9. Xiao L., Xiao-yu Z., Li-zhuang M. Image Resizing Based on Seam Carving and Warping [J]. Computer Science, 2015, 42(9): 289-292.
10. Jia X.X., Feng X.C., Wang W.W. Task driven saliency detection for image retargeting [J]. Optik International Journal for Light and Electron Optics, 2016, 127(20): 8306-8316.
11. Panpan Z., Ping L., Hengliang Z. An Novel Image Resizing Method Based on Important Area Maintain [J]. Journal of Graphics, 2016, 37(2): 230-236.
