Research progress on polarimetric imaging technology in complex environments based on deep learning (invited)

被引:0
|
作者
Hu H. [1 ,2 ]
Huang Y. [2 ]
Zhu Z. [1 ]
Ma Q. [1 ]
Zhai J. [1 ]
Li X. [1 ]
机构
[1] School of Marine Science and Technology, Tianjin University, Tianjin
[2] School of Precision Instrument and Optoelectronics Engineering, Tianjin University, Tianjin
基金
中国国家自然科学基金;
关键词
deep learning; image denoising; image descattering; polarization imaging;
D O I
10.3788/IRLA20240057
中图分类号
学科分类号
摘要
Significance Polarization information, as one of the fundamental physical characteristics of light waves, can provide information about the intrinsic properties of the target. Polarimetric imaging technology digitizes the polarization information of the measured target field through digital processing. This approach effectively reduces the interference from the light propagation environment, thereby improving the imaging quality of the target and enhancing perception of its characteristics. In complex environments, polarimetric imaging has significant advantages. However, in complex environments such as scattering and low illumination, the degradation mechanism of polarized images exhibits nonlinear characteristics, leading to high complexity in polarized information interpretation methods. Deep learning methods possess powerful feature extraction and learning capabilities, enabling the recovery of polarized information by learning the mapping rules hidden in the large-scale collected data. This approach is particularly suitable for complex signal processing problems like polarimetric imaging, which involves multiple dimensions and interrelated signals. Progress First, the basic theory of the polarimetric imaging is introduced, including the principles of polarimetric imaging and a macroscopic description of polarimetric imaging issues in complex environments. Next, the general workflow of deep learning polarization imaging technology in complex environments is introduced. Based on deep learning, polarimetric imaging technology in complex environments uses the multidimensional polarimetric parameters collected by the polarimetric imaging system as input data. It leverages the nonlinear feature-fitting capabilities of neural networks to obtain image restoration results. Essentially, this approach transforms the nonlinear inverse problem of polarimetric imaging restoration in complex environments into a pseudo-forward problem, avoiding the challenges associated with solving nonlinear inverse problem algorithms. The representative developments of research in deep learning polarimetric imaging technology in response to scattering and noise, two of the most representative complex imaging environments, have been elaborated. From the inception of research in this field, the developmental trajectory of the field has been systematically outlined. In the early stages, polarimetric imaging technology in complex environments based on deep learning primarily relied on supervised training. Due to the challenges in collecting real-world data, researchers explored solutions using unsupervised, self-supervised, transfer learning, and simulation algorithms. Researchers also delved into the incorporation of prior knowledge and physical models into networks, leading to training approaches embedded with physical models or guided by prior knowledge. Overall, these representative works have made significant contributions to addressing the difficulties in constructing large-scale datasets, enhancing the generalization performance of networks, and exploring the interpretability of the networks. To better illustrate the connections and distinctions among research works, and to streamline the developmental process in this field for reader convenience, a summary has been compiled in the form of a table. The table provides task types, training methods, and characteristics of representative works for easy reference. Conclusions and Prospects With the rapid development of deep learning, polarimetric imaging technology in complex environments has achieved remarkable research progress. Existing studies indicate that, due to the multiple parameters and inherent correlations in polarized information, this multi-dimensional and interrelated signal processing problem is well-suited for the application of deep learning. The combination of deep learning and polarimetric imaging technology enables further improvement in optical imaging quality, meeting the imaging demands of complex environments and demonstrating more prominent advantages. The generalization ability, interpretability, and parameter lightweighting of deep learning technology remain areas that require further in-depth research. There is a continued need for refinement in multimodal fusion strategies, exploration of the underlying principles of network polarimetric parameter image restoration, and the design of network structures tailored for polarized multidimensional data to enhance real-time performance. Further efforts are essential to consolidate the feasibility of deep learning models in polarimetric imaging within complex environments, to enhance the adaptability of models to changes in complex environmental conditions, and to make them more universally applicable across different scenarios. © 2024 Chinese Society of Astronautics. All rights reserved.
引用
收藏
相关论文
共 81 条
  • [1] Goldstein D H., Polarized Light, (2017)
  • [2] Liu X, Zhang L, Zhai X, Et al., Polarization lidar: Principles and applications, Photonics. MDPI, 10, 10, (2023)
  • [3] Li X, Yan L, Qi P, Et al., Polarimetric imaging via deep learning: A review, Remote Sensing, 15, 6, (2023)
  • [4] Breugnot S, Clemenceau P., Modeling and performances of a polarization active imager at λ= 806 nm, Optical Engineering, 39, 10, pp. 2681-2688, (2000)
  • [5] Marino A, Dierking W, Wesche C., A depolarization ratio anomaly detector to identify icebergs in sea ice using dual-polarization SAR images, IEEE Transactions on Geoscience and Remote Sensing, 54, 9, pp. 5602-5615, (2016)
  • [6] Dong Y, Wan J, Wang X, Et al., A polarization-imaging-based machine learning framework for quantitative pathological diagnosis of cervical precancerous lesions, IEEE Transactions on Medical Imaging, 40, 12, pp. 3728-3738, (2021)
  • [7] Ramella-Roman J C, Novikova T., Polarized Light in Biomedical Imaging and Sensing: Clinical and Preclinical Applications, (2022)
  • [8] Novikova T, Ramella-Roman J C., Is a complete Mueller matrix necessary in biomedical imaging, Optics Letters, 47, 21, pp. 5549-5552, (2022)
  • [9] Sun M, He H, Zeng N, Et al., Characterizing the microstructures of biological tissues using Mueller matrix and transformed polarization parameters, Biomedical Optics Express, 5, 12, pp. 4223-4234, (2014)
  • [10] Liu F, Wei Y, Han P, Et al., Polarization-based exploration for clear underwater vision in natural illumination, Optics Express, 27, 3, pp. 3629-3641, (2019)