1
2018 International Conference on Computer Science and Software Engineering (CSSE 2018) ISBN: 978-1-60595-555-1
Compressed Sensing MRI Reconstruction
Based on Generative Adversarial Nets
Tao Jiang, Jinxu Tao, Zhongfu Ye, Bensheng Qiu
and Jinzhang Xu
ABSTRACT
Compressed sensing magnetic resonance imaging(CS-MRI) achieves shorter acquisition time by reducing the sampling rate of k-space data. This not only cuts down the scanning cost, but also reduces the artifacts and alising caused by the patient's movement and improves the imaging quality. However, the iterative reconstruction of MRI based on compressed sensing needs very expensive computation. Recently, some works about CS-MRI reconstruction based on deep learning have achieved great success. In this paper, we proposed a novel network named ResGAN to improve imaging quality while significantly reducing MRI scan time. The proposed network, ResGAN, is a deep generative adversarial network, which learns a mapping from the under-sampled MRI data to fully sampled data by the generative network. In particular, we adopt the training strategy of residual learning to achieve lower convergence loss as well as avoiding over fitting. Moreover, despite of the pixel-wise L1-loss, the feed-forward generative network also utilizes frequency domain loss to maintain data consistency. Our method obtained promising experimental results on both imaging quality and reconstruction speed for 1.5T brain MRI while only 30 percent of the raw k-space data are sampled using radial sampling pattern.
Tao Jiang1, Jinxu Tao1, Zhongfu Ye1, Bensheng Qiu1, Jinzhang Xu3 1
Department of Electronic Engineering and Information Science University of Science and Technology of China, Hefei, China 2
Department of Electronic Science and Technology
University of Science and Technology of China, Hefei, China 3
INTRODUCTION
As an indispensable part of modern medical imaging technology (X-ray imaging, ultrasound imaging, CT and MRI, etc.), magnetic resonance imaging (MRI) has unparalleled advantages over other bio-imaging technologies. First of all, MRI does not have ionizing radiation, which is one of the few medical imaging methods that has no harm to the human body. Secondly, it is very sensitive to soft tissue detection and can provide precise anatomical information of human structure. Moreover, MRI can obtain images that are difficult to be displayed by other imaging methods in the position of the lesion in the coronal, sagittal and axial directions and use multiple imaging parameters to perform multiple functional imaging. However, there are obvious deficiencies in MRI. It requires very long data acquisition time due to limitation of hardware, which leads that MRI has not been widely used in clinical practice [1, 2].
MRI includes two steps: signal acquisition and image reconstruction. In classical signal processing theory, the sampling frequency must be at least twice as high as the maximum signal frequency in order to fully reconstruct the sampled signal. On account of the limitation of physical mechanism and the Nyquist sampling theorem, the imaging speed of MRI is slower than that of other imaging techniques. In order to obtain higher spatial resolution, static-volume technique must collect enough data as possible, which will cost lots of scan time as well as introducing the motion artifacts, the data acquisition process is a big challenge for critically ill patients. The dynamic imaging, such as: heart perfusion imaging and interventional imaging has strict requirements for reconstruction of real-time which must limit the sampled data of each frame image. This will lead to comparatively lower spatial resolution. To this end, many scholars are devoted to the study of rapid imaging of MRI.
The application of compressive sensing theory [3] is widely used in mathematics, machine learning and medical imaging. It proves that if a signal is sparse on the spatial domain or in some transform domain, it can be used to reconstruct the original signal with the data that is far lower than that required by the Nyquist sampling theorem. Based on this theory, we can greatly reduce the sampling data of k-space to shorten the scan time. Spares-MRI [4] is the earliest CS-MRI framework put forward by Lustig. It uses the classical convex optimization algorithm: Conjugate Gradient Descent Algorithm With Backtracking Line Search (NLCG), but computational complexity of this kind of algorithm is higher and have slower convergence, which can't meet the demand of clinical imaging. Iterative Shrinkage Thresholding (IST) [5] is a quick operator solving the Ll norm optimization, the algorithm is suitable for temporal resolution demanding applications for its simple structure and low computational complexity.
improved algorithms based on FTVd. For example, Constrained Spilt Augmented Lagrangian Shrinkage Algorithm (CSALSA) [7] use auxiliary variable to build separable structure, then the whole problem is decomposed into several simpler sub-problems, the reconstruction of speed and quality have more obvious improvement. Afterwards, it occurs a series of this type algorithm, such as Reconstruction from Partial Fourier Data (RecPF) [8], Composite Splitting Algorithm (CSA) [9], Inexact Alternating Direction Proximal Method (IADPM) [10] and TVAL3 [11].
In the past few years, deep learning [12] has become very popular with the improvement of computing power. In particular, the convolutional neural networks (CNNs) has great advantages in handling large-scale visual tasks, such as image recognition [13], classification [14], super resolution [15, 16] and denoising [17]. Wang used CNNs for the first time in MRI reconstruction [18]. The network learned the mapping between zero-filled and full-sampled k-space data. In [19], the residual network is proposed for MRI resolution. The framework has obtained higher image quality by using multiple input data acquired from different viewing planes. The proposed network reconstructs MR image from zero-filled images based on compressed sensing method. Specially, we utilize generative adversarial networks (GANs) [19, 20]. GANs is a generative model whose basic idea is to learn the probability distribution of the training cases from a data-base. And the way to do that is to let the two networks compete with each other. The generator network constantly captures the probability distribution of real images. The discriminator network can observe both real and fake data at the same time to determine whether the data is true or not. This powerful combination has been used for generating Computed Tomography (CT)-like images from MRIs [21] and reconstructing the entire k-space grid from under-sampled data. In this paper, we use the generator to reconstruct MR images from zero-filled data. Its loss includes both spatial and the frequency domain loss which improves image quality and ensures the data consistency. Our method obtained promising experiment results on both imaging quality and reconstruction speed for brain MRI data.
The paper’s overall structure is as follows: Section 2 introduces the basic theory of CS-MRI and our method. Section 3 presents detail architecture of the model and Section 4 describes the experimental results. Conclusions and the future research direction are shown in Section 5.
THEORY
CS-MRI
= (1)
Where is a measurement matrix denoting the undersampling mask, represents the Fourier encoding matrix which is normalized as = , is the real image. denotes an orthonormal 2D IFFT operator. Then is equal to the partial two-dimension FT resulting in undersampled k-space data . So the zero-filled images is directly obtained from the inverse transform of measurement data by following equation:
= (2)
The reconstructed MR images suffer from severe alising and artifacts. It is shown in Fig. 1 while sampling 30 percent of the raw k-space data.
Reconstructing real image from undersampled k-space data using compressed sensing theory can be described as the follows:
‖ − ‖+ (3)
Where represents regularization for ill-posed optimization problems. It can be many traditional regularizations studied by the previous scholars, such as total variation[6,11,22], wavelets[23], etc. However, it needs time-consuming iterative process to solve such optimization by minimizing the equation above. Therefore, we proposed a deep feed-forward neural network for modeling L1 norm CS-MRI formulations to reduce the scan time and improve the imaging quality.
Objective
Our goal is to train a model with the set of parameters Θ, which can learn an end-to-end map from zero-filled images to the ground truth images. As the following objective:
ℒ ; Θ, (4)
[image:4.612.107.494.554.658.2](a) Real MRI (b) raw k-space (c) sample mask (d) zero-filled MRI
Where : represents the loss function. L1 norm has been used to filter the noise from the natural images as it does appropriately penalize the low-intensity frequency noise. Moreover, applying L1 norm to the model will not blur the generated images while L2 norm will slightly do that. In addition, we adopt the frequency domain loss which measures the difference between the generated and ground truth images. That is to say, we should guarantee the reconstructed images multiplying by undersampled operator are as much possible as close to the ground truth images multiplying by .
We trained our model by the adversarial strategy described in [20], The generator G takes zero-filled complex value = as input to generate images that are close to . Simultaneously, discriminator D scores one the real images drawn from the distribution , and scores zero the generated image ̅. The training aims to optimize G by maximizing "1 − log '() *+ while D can get better prediction at the same time. Once the training converges, the images generated by G will be close to the ground truth images and D cannot distinguish between and ̅. Therefore, D will dive 0.5 to the probability for both real and generated images.
The overall optimization process can be described as the following minimax problem:
,- ℒ), ' (5)
Where
ℒ), ' = ./∈1"1 − log '() *+ + . ∈23log '4 (6)
To strengthen the mapping between and ̅, the loss of G is designed to be a combination of spatial and frequency domain loss as flows:
ℒ, = ./∈1"1 − log '() *+ + ‖) − ‖5+ ‖) − ‖ (7)
Where λ is a hyper-parameter setting to be 0.01 by cross-validation.
MODEL ARCHITECTURE
Figure 2. (a) Generator; (b)(c) Basic building blocks of the proposed network architecture.
normalization and ReLU activation. After the nine residual blocks, another convolutional layers is added to perfect the network. Generator learns the mapping relations between zero-filled images and the ground truth while ensuring the data consistency at the same time.
The discriminator takes both the output of the generator and the ground truth images as input. Inspired by the architecture referred to in [20] comprised by convolutional layers, batch normalization and ReLU, we design a specialized network with six convolutional layers . The first four layers adopt 3 by 3 kernel size with stride 2 followed by batch normalization and ReLU. Instead, kernel size of the last two layers is only 1 by 1 while stride is equal to 1. Specially, the feature maps are doubled from 8 to 64 in the first four layers without pooling. Finally, discriminator output is averaged for binary classification.
EXPERIMENTAL RESULTS
In this section, we show the promising results of our proposed model through quantitative and qualitative analysis. The training data consists of 581 patients 3D fully sampled brain MR images (T1) acquired from 1.5T MR scanner. We extracted 10 slices from each patient’s 3D MR image and flipped each of them in order to augment dataset. Eight thousand slices were used to train our model and the rest for testing. To obtain zero-filled images as input, we applied the operator
to the ground truth images. Note that, the k-space data were sampled using
radial binary mask whose sample rate is 30%.
The momentum parameter of Adam solver is set to be 0.9 and mini-batch size is 4. Learning rate is reduced by half every 5000 iterations with initial value
1089. Epoch is allowed to be 10 for training with TensorFlow framework on a
Tesla K40 GPU, 12GB RAM, which cost about 11 hours.
[image:6.612.184.431.84.223.2]compared results of our model to zero-filled images and obtained from traditional compressed sensing method[4]. As shown in Fig. 3, there are almost no artifacts and alising in the reconstructed MR images based on our method. Moreover, it can not only preserve the overall profile but also reproduce the texture features of the original image better. In addition, we adopted reconstruction time (secs) and PSNR(dB) to measure imaging speed and quality presented in Table I. Results indicate that our method outperforms the others while strategy of L1 loss is better than L2 loss.
[image:7.612.126.472.217.535.2]
Zero-filled CS-MRI L1-ResGAN L2- ResGAN Ground truth
Figure 3. Reconstructed brain MR images from zero-filled images, which sample rate is 30%.
TABLE I. AVERAGED PSNR(DB) AND RECONSTRUCTION TIME (SECS).
Method PSNR Reconstruct time Zero-filled 29.45 10-4
CS-MRI[4] 35.73 2.31
L1-ResGAN 36.96 0.02
[image:7.612.150.437.585.663.2]CONCLUSIONS
In this paper, we designed a novel deep generative adversarial network based on residual learning to reconstruct the compressed sensing MR images, which improved the imaging quality and accelerated the construction speed. The experimental results have shown a promising potential of deep neural network for MRI application since it’s able to recover lost details and fine structures in zero-filled image. In the future, we will focus on the 3D MRI reconstruction as well as dynamic imaging by modifying and improving our ResGAN.
ACKNOWLEDGMENTS
This work was financially supported by the National Natural Science Foundation of China (No. 81527802).
REFERENCES
1. Lustig M, Pauly J M. SPIRiT: Iterative self-consistent parallel imaging reconstruction from arbitrary k-space[J]. Magnetic resonance in medicine, 2010, 64(2): 457-471.
2. Knoll F, Clason C, Bredies K, et al. Parallel imaging with nonlinear reconstruction using variational penalties[J]. Magnetic Resonance in Medicine, 2012, 67(1): 34-41.
3. Donoho D L. Compressed sensing[J]. IEEE Transactions on information theory, 2006, 52(4): 1289-1306.
4. Lustig M, Donoho D, Pauly J M. Sparse MRI: The application of compressed sensing for rapid MR imaging[J]. Magnetic resonance in medicine, 2007, 58(6): 1182-1195.
5. Beck A, Teboulle M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems[J]. SIAM journal on imaging sciences, 2009, 2(1): 183-202.
6. Wang Y, Yang J, Yin W, et al. A new alternating minimization algorithm for total variation image reconstruction[J]. SIAM Journal on Imaging Sciences, 2008, 1(3): 248-272.
7. Afonso M V, Bioucas-Dias J M, Figueiredo M A T. An augmented Lagrangian approach to the constrained optimization formulation of imaging inverse problems[J]. IEEE Transactions on Image Processing, 2011, 20(3): 681-695.
8. Yang J, Zhang Y, Yin W. A fast alternating direction method for TVL1-L2 signal reconstruction from partial Fourier data[J]. IEEE Journal of Selected Topics in Signal Processing, 2010, 4(2): 288-297.
10. Xiao Y H, Song H N. An inexact alternating directions algorithm for constrained total variation regularized compressive sensing problems[J]. Journal of Mathematical Imaging and Vision, 2012, 44(2): 114-127.
11. Li C, Yin W, Jiang H, et al. An efficient augmented Lagrangian method with applications to total variation minimization[J]. Computational Optimization and Applications, 2013, 56(3): 507-530.
12. LeCun Y, Bengio Y, Hinton G. Deep learning[J]. nature, 2015, 521(7553): 436.
13. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint arXiv:1409.1556, 2014.
14. Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks[C]//Advances in neural information processing systems. 2012: 1097-1105. 15. Kim J, Kwon Lee J, Mu Lee K. Accurate image super-resolution using very deep
convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 1646-1654.
16. Dong C, Loy C C, He K, et al. Learning a deep convolutional network for image super-resolution[C]//European Conference on Computer Vision. Springer, Cham, 2014: 184-199.
17. Burger H C, Schuler C J, Harmeling S. Image denoising: Can plain neural networks compete with BM3D?[C]//Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on. IEEE, 2012: 2392-2399.
18. Wang S, Su Z, Ying L, et al. Accelerating magnetic resonance imaging via deep learning[C]//Biomedical Imaging (ISBI), 2016 IEEE 13th International Symposium on. IEEE, 2016: 514-517.
19. Oktay O, Bai W, Lee M, et al. Multi-input cardiac image super-resolution using convolutional neural networks[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2016: 246-254.
20. Radford A, Metz L, Chintala S. Unsupervised representation learning with deep convolutional generative adversarial networks[J]. arXiv preprint arXiv:1511.06434, 2015. 21. Nie D, Trullo R, Lian J, et al. Medical image synthesis with context-aware generative
adversarial networks[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Cham, 2017: 417-425.
22. Chen C, Li Y, Huang J. Calibrationless parallel MRI with joint total variation regularization[C]//International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, Berlin, Heidelberg, 2013: 106-114.
