A Survey on Deep Learning techniques in Image fusion
DOI:
https://doi.org/10.5281/zenodo.10444476Keywords:
Image fusion, deep learning, CNN, GAN, AEAbstract
In the ever-evolving field of image fusion, the integration of deep learning techniques has led to remarkable advancements in the quality and applicability of fused images. This review work provides a comprehensive overview of state of art deep learning based image fusion techniques. We delve into the fundamental concepts, methodologies and challenges that have emerged in this domain. This work covers various aspects of deep learning-based image fusion, including multi-modal, multi-scale fusion, and cross modality fusion. This work offers insights into the practical applications of deep learning based image fusion across various domains. We highlight the potential benefits and limitations in this dynamic field.
References
Pajares, G., & De La Cruz, J. M. (2004). A wavelet-based image fusion tutorial. Pattern recognition, 37(9), 1855-1872.
Li, S., Yang, B., & Hu, J. (2011). Performance comparison of different multi-resolution transforms for image fusion. Information Fusion, 12(2), 74-84.
Mo, Y., Kang, X., Duan, P., Sun, B., & Li, S. (2021). Attribute filter based infrared and visible image fusion. Information Fusion, 75, 41-54.
Li, S., Kang, X., & Hu, J. (2013). Image fusion with guided filtering. IEEE Transactions on Image processing, 22(7), 2864-2875.
Liu, Y., Liu, S., & Wang, Z. (2015). A general framework for image fusion based on multi-scale transform and sparse representation. Information fusion, 24, 147-164.
Harsanyi, J. C., & Chang, C. I. (1994). Hyperspectral image classification and dimensionality reduction: An orthogonal subspace projection approach. IEEE Transactions on geoscience and remote sensing, 32(4), 779-785.
Han, J., Pauwels, E. J., & De Zeeuw, P. (2013). Fast saliency-aware multi-modality image fusion. Neurocomputing, 111, 70-80.
Ma, J., Chen, C., Li, C., & Huang, J. (2016). Infrared and visible image fusion via gradient transfer and total variation minimization. Information Fusion, 31, 100-109.
Zhang, H., Xu, H., Xiao, Y., Guo, X., & Ma, J. (2020, April). Rethinking the image fusion: A fast unified image fusion network based on proportional maintenance of gradient and intensity. In Proceedings of the AAAI conference on artificial intelligence (Vol. 34, No. 07, pp. 12797-12804).
Zhang, Y., Liu, Y., Sun, P., Yan, H., Zhao, X., & Zhang, L. (2020). IFCNN: A general image fusion framework based on convolutional neural network. Information Fusion, 54, 99-118.
Chen, X., & Konukoglu, E. (2018). Unsupervised detection of lesions in brain MRI using constrained adversarial auto-encoders. arXiv preprint arXiv:1806.04972.
Jian, L., Yang, X., Liu, Z., Jeon, G., Gao, M., & Chisholm, D. (2020). SEDRFuse: A symmetric encoder–decoder with residual block network for infrared and visible image fusion. IEEE Transactions on Instrumentation and Measurement, 70, 1-15.
Long, Y., Jia, H., Zhong, Y., Jiang, Y., & Jia, Y. (2021). RXDNFuse: A aggregated residual dense network for infrared and visible image fusion. Information Fusion, 69, 128-141.
Ram Prabhakar, K., Sai Srikar, V., & Venkatesh Babu, R. (2017). Deepfuse: A deep unsupervised approach for exposure fusion with extreme exposure image pairs. In Proceedings of the IEEE international conference on computer vision (pp. 4714-4722).
Isola, P., Zhu, J. Y., Zhou, T., & Efros, A. A. (2017). Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1125-1134).
Ma, J., Yu, W., Chen, C., Liang, P., Guo, X., & Jiang, J. (2020). Pan-GAN: An unsupervised pan-sharpening method for remote sensing image fusion. Information Fusion, 62, 110-120.
Li, H., & Zhang, L. (2018, October). Multi-exposure fusion with CNN features. In 2018 25th IEEE International Conference on Image Processing (ICIP) (pp. 1723-1727). IEEE.
Li, J., Guo, X., Lu, G., Zhang, B., Xu, Y., Wu, F., & Zhang, D. (2020). DRPL: Deep regression pair learning for multi-focus image fusion. IEEE Transactions on Image Processing, 29, 4816-4831.
Guo, X., Nie, R., Cao, J., Zhou, D., Mei, L., & He, K. (2019). FuseGAN: Learning to fuse multi-focus image via conditional generative adversarial network. IEEE Transactions on Multimedia, 21(8), 1982-1996.
Zhang, H., Le, Z., Shao, Z., Xu, H., & Ma, J. (2021). MFF-GAN: An unsupervised generative adversarial network with adaptive and gradient joint constraints for multi-focus image fusion. Information Fusion, 66, 40-53.
Zhang, H., Xu, H., Tian, X., Jiang, J., & Ma, J. (2021). Image fusion meets deep learning: A survey and perspective. Information Fusion, 76, 323-336.
Jian, L., Yang, X., Liu, Z., Jeon, G., Gao, M., & Chisholm, D. (2020). SEDRFuse: A symmetric encoder–decoder with residual block network for infrared and visible image fusion. IEEE Transactions on Instrumentation and Measurement, 70, 1-15.
Ahmed, S. T., Kumar, V., & Kim, J. (2023). AITel: eHealth Augmented Intelligence based Telemedicine Resource Recommendation Framework for IoT devices in Smart cities. IEEE Internet of Things Journal.
Li, H., Wu, X. J., & Kittler, J. (2018, August). Infrared and visible image fusion using a deep learning framework. In 2018 24th international conference on pattern recognition (ICPR) (pp. 2705-2710). IEEE.
Lahoud, F., & Süsstrunk, S. (2019, July). Zero-learning fast medical image fusion. In 2019 22th international conference on information fusion (FUSION) (pp. 1-8). IEEE.
Xu, H., Ma, J., Le, Z., Jiang, J., & Guo, X. (2020, April). Fusiondn: A unified densely connected network for image fusion. In Proceedings of the AAAI conference on artificial intelligence (Vol. 34, No. 07, pp. 12484-12491).
Zhao, C., Wang, T., & Lei, B. (2021). Medical image fusion method based on dense block and deep convolutional generative adversarial network. Neural Computing and Applications, 33, 6595-6610.
Ma, J., Jiang, X., Fan, A., Jiang, J., & Yan, J. (2021). Image matching from handcrafted to deep features: A survey. International Journal of Computer Vision, 129, 23-79.
Yang, J., Ma, Y., Yao, W., & Lu, W. T. (2008). A spatial domain and frequency domain integrated approach to fusion multifocus images. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 37(PART B7).
Thouheed Ahmed, S., & Sandhya, M. (2019). Real-time biomedical recursive images detection algorithm for Indian telemedicine environment. In Cognitive Informatics and Soft Computing: Proceeding of CISC 2017 (pp. 723-731). Springer Singapore.
Swamy, R., Ahmed, S. T., Thanuja, K., Ashwini, S., Siddiqha, S., & Fathima, A. (2021, January). Diagnosing the level of Glaucoma from Fundus Image Using Empirical Wavelet Transform. In Proceedings of the First International Conference on Advanced Scientific Innovation in Science, Engineering and Technology, ICASISET 2020, 16-17 May 2020, Chennai, India.
Ambika, B. J., Guptha, N. S., & Siddiqha, S. A. (2023). Anaemia Estimation for Patients Using Lasso And Ridge Regression Algorithms. Milestone Transactions on Medical Technometrics, 1(2), 53-63.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2023 Vasudha G S, Kusuma Kumari B M
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
