Image Fusion Based on Combined Multi scale Decomposition and Improved Sparse Representation

2018 International Conference on Modeling, Simulation and Optimization (MSO 2018) ISBN: 978-1-60595-542-1

Image Fusion Based on Combined Multi-scale Decomposition and

Improved Sparse Representation

Meng-yun JIANG, Lin BAI

_{and Zhi-tong HUANG}

Beijing city Haidian District Xitucheng Road No. 10 *Corresponding author

Keywords: Image fusion, MSD-ISR, Multi-scale decomposition, Sparse representation.

Abstract. A new method of combined multi-scale decomposition and improved sparse representation (MSD-ISR) used in image fusion is proposed in this paper which has three advantages. As is known that it is hard to determine the decomposition level and it may achieve low contrast in multi-scale decomposition. The proposed MSD-ISR method can solve this problem and on the other hand, we get a higher efficiency in the process of sparse decomposition as well as better preserve the information of the source image. Experimental results show that the proposed MSD-ISR method can achieve better fusion results.

Introduction

A method of image fusion based on the combination of multi-scale decomposition and improved sparse representation is proposed. On one hand, the sparse representation method is applied in image fusion based on multi-scale transform which can combine the advantages of sparse representation and multi-scale decomposition. It can solve the problem of decomposition level as well as low contrast of transform base image fusion [1, 2]. As a result, the proposed method can better preserve the information in the source images. On the other hand, a whole image is used in the process of sparse decomposition but not blocks of image, which can raise the efficiency of sparse decomposition [3]. Five indicators are selected namely information entropy (E), standard deviation (SD), mutual information (MI), mean gradient (AG) and QABF as the objective evaluation index. Results show that the proposed method performed in both the objective and subjective index.

The Proposed MSD-ISR Method

The Framework of MSD-ISR

A new method of combined multi-scale decomposition and improved sparse representation (MSD-ISR) used in image fusion is proposed in this paper. We choose the coefficients which have larger gradient in the process of choosing low frequency and choose the larger coefficients of high frequency. The flowchart is shown as follow (Refer with: Figure 1). The fusion steps are as following:

Step 1: Get thetrain_im g. Choose the pixels which have larger area gradient between source images as the pixels of the train image.

Step 2: Decompose both the source images and train image. Using multi-scale decomposition methods to decomposition the source images and train image respectively to get the high frequency and low frequency.

Step 4: Fuse the low frequency reconstruction coefficients and the high frequency reconstruction coefficients. The fusing rules are used as the same as in step 1.

Step 5: Obtain the fused image by using inverse multi-scale decompose method.

Image A Multi-scale decomposition Fused low frequency coefficients Image B Multi-scale decomposition Multi-scale reconstruction Fused high frequency coefficients Fused image Low frequency reconstruction coefficients Sparse Reconstruction High frequency reconstruction coefficients Low frequency reconstruction coefficients High frequency reconstruction coefficients Fusion Rule 1 Fusion Rule 2 Sparse Reconstruction Sparse Reconstruction

Figure 1. The framework of MSD-ISR.

Advantages of the Proposed MSD-ISR Method. There are three advantages of the proposed MSD-ISR method compared with other methods. Firstly, it can achieve good fusion results regardless of the decomposition level of the multi-scale decomposition method and raise the contrast of the fused image. Secondly, using the preliminary fusion image instead of the source image as the train image can retain more source information. Finally, a whole image is used in the process of sparse decomposition but not blocks of image, which can raise the efficiency of sparse decomposition.

Experimental Results and Analysis

Experimental Results

This experiment is simulated on MATLAB R2017. The source images used are the following four groups, namely lab image, clock image, pepsi image and cup image. The purpose of the experiment is to prove that the proposed MSD-ISR method is superior to the general multi-scale decomposition method. We choose NSCT method as a representation of multi-scale decomposition method. The following are the fusion rules and decomposition layers of the three algorithms in the simulation (Refer with: Table 1).

Table 1. The fusion rules and decomposition layer of the algorithm.

Decomposition layer Fusion rule of low frequency Fusion rule of high frequency NSCT-2 layer 2 Area gradient based Coefficients based NSCT-3 layer 3 Area gradient based Coefficients based The prosed method 2 Area gradient based Coefficients based

     Figure 2. The fused (left) and local magnified lab image (right) of NSCT-2 layer method.

Figure 3. The result (left) and local magnified lab image (right) of NSCT-3 layer method.

Figure 4. The result (left) and local magnified lab image (right) of the proposed method.

     Figure 6. The result (left) and local magnified clock image (right) of NSCT-3 layer method.

Figure 7. The result (left) and local magnified clock image (right) of the proposed method.

Figure 8. The result (left) and local magnified pepsi image (right) of NSCT-2 layer method.

     Figure 10. The result (left) and local magnified pepsi image (right) of the proposed method.

Figure 11. The result (left) and local magnified cup image (right) of NSCT-2 layer method.

Figure 12. The result (left) and local magnified cup image (right) of NSCT-3 layer method.

Figure 13. The result (left) and local magnified cup image (right) of the proposed method.

Table 2. Result of objective evaluation.

EN1 SD MI AG QABF

Lab image

NSCT-2 layer 6.942 45.4 4.465 3.67 0.655 NSCT-3 layer 6.988 45.899 4.501 3.199 0.61 The prosed method 7.0839 48.626 4.882 3.992 0.696

Pepsi image

NSCT-2 layer 7.105 44.807 4.0084 4.33 0.604 NSCT-3 layer 7.104 44.501 4.0892 3.705 0.535 The prosed method 7.455 58.0011 4.09 5.733 0.647

Clock image

NSCT-2 layer 7.025 40.338 4.628 2.52 0.514 NSCT-3 layer 7.039 40.156 4.47 2.53 0.541 The prosed method 7.25 46.432 4.847 3.105 0.593

Cup image

NSCT-2 layer 7.472 53.705 4.836 4.938 0.708 NSCT-3 layer 7.48 53.791 4.466 4.263 0.593 The prosed method 7.502 54.653 4.119 4.454 0.652

Analysis and Discussions

Subjective Evaluation. In order to make a better contrast, we make local amplifications to the fusion images. It can be seen that the fusion images of the proposed MSD-ISR method are clearer in the edge and have more texture information and less artifacts. Compared with the other two algorithms, the fusion results are obviously improved. At the same time, the results of the proposed MSD-ISR method make the human eye more comfortable and more like the source image.

Objective Evaluation. As shown in table 2, the proposed MSD-ISR method can get better results in the above mentioned five indexes, namely information entropy (E), standard deviation (SD), mutual information (MI), average gradient (AG) and edge based image fusion objective evaluation index(QABF). Even if the decomposition level of NSCT method is raised, the proposed MSD-ISR method can still get better results of the five indexes.

Summary

The proposed MSD-ISR method is compared with the NSCT decomposition base method with different decomposition layers in both subjective and objective evaluations. The experimental results show that the fusion images of the proposed MSD-ISR method have clearer edges and more texture information than the compares methods. And the proposed MSD-ISR method achieves better performance on the evaluation indexes. Therefore, we can get the conclusion that the proposed method is an effective method of image fusion.

Acknowledgement

Thanks for great support of NSFC (No.61771073), P.R. China.

References

[1] Liu Y., Liu S., Wang Z. A general framework for image fusion based on multi-scale transform and sparse representation [J]. Information Fusion, 2015, 24: 147-164.

[2] Zhang B., Lu X., Pei H., et al. Multi-focus image fusion based on sparse decomposition and background detection [J]. Digital Signal Processing, 2016, 58: 50-63.

[4] Jia Y., Rong C., Wang Y., et al. A multi-focus image fusion algorithm using modified adaptive PCNN model [C]// International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery. IEEE, 2016: 612-617.