3D Shape Reconstruction from Multi Depth of Field Images: Datasets and Models

被引：0
|
作者：
Zhang J.-F. [1 ,2 ]
Yan T. [1 ,2 ,3 ,4 ]
Wang K.-Q. [1 ,2 ]
Qian Y.-H. [1 ,3 ,5 ]
Wu P. [1 ,5 ]
机构：
[1] Institute of Big Data Science and Industry, Shanxi University, Taiyuan
[2] School of Computer and Information Technology, Shanxi University, Taiyuan
[3] Engineering Research Center for Machine Vision and Data Mining of Shanxi Province, Shanxi University, Taiyuan
[4] Chongqing Research Institute of Harbin Institute of Technology, Chongqing
[5] Key Lab. of Computational Intelligence and Chinese Information Processing of Ministry of Education, Shanxi University, Taiyuan
来源：
Jisuanji Xuebao/Chinese Journal of Computers | 2023年 / 46卷 / 08期
关键词：
3D shape reconstruction; deep learning; image sequence datasets; kernel function; multi-focus images;
D O I：
10.11897/SP.J.1016.2023.01734
中图分类号：
学科分类号：
摘要：
Limited by the multi-source heterogeneity of data acquisition and the expensive annotation of 3D reconstruction results, the existing 3D shape reconstruction methods based on multi-depth of field image focus information usually need to be designed according to specific application scenes, resulting in a lack of scene adaptability. This paper proposed a theory and method of constructing multi-depth-of-field image datasets, and designed a robust deep network model on this basis. The constructed Multi-Depth-of-Field Image Datasets (MDFI Datasets) aimed at stripping the strong correlation between the actual semantics of images and the depth information. The shape kernel function nonlinear spatial mapping method was proposed to extend the multidimen-sionality and diversity of the datasets by combining the texture-rich characteristics of the input image sequences with the inherent homogeneity and step properties of the 3D shape. The Deep Shape from Focus Net (DSFF-Net) was designed with U-Net as the base network, and Deformable ConvNets v2 was added to enhance the feature extraction capability of the network, and the newly designed Local-Global Relationship Coupling (LGRCB) module helped to improve the aggregation capability of the global focus information of the model. To verify the cross-scene applicability of MDFI Datasets and the robustness as well as generalization of DSFF-Net model, this paper conducted experimental comparative analysis from four different aspects. The results of the experiments show that compared with the state-of-the-art Robust Focus Volume Regularization in Shape from Focus (RFVR-SFF) and All-in-Focus Depth Net ( AiFDepth-Net), the DSFF-Net model proposed in this paper decreases 15% and 29% in the Root Mean Square Error (RMSE) index, while the experiments on large depth-of-field scenes show that the datasets construction method proposed in this paper can adapt to real application scenes. © 2023 Science Press. All rights reserved.
引用
收藏
页码：1734 / 1751
页数：17
相关论文
共 54 条
  • [1] Zhang Shun, Gong Yi-Hong, Wang Jin-Jim, The development of deep convolutional neural networks and its applications on computer vision, Chinese Journal of Computers, 42, 3, pp. 453-482, (2019)
  • [2] Song Wei, Zhu Meng-Fei, Zhang Ming-Hua, Et al., A review of monocular depth estimation techniques based on deep learning, Journal of Image and Graphics, 27, 2, pp. 292-328, (2022)
  • [3] Li Hai-Sheng, Wu Yu-Juan, Zheng Yan-Ping, Et al., A survey of 3D data analysis and understanding based on deep learning, Chinese Journal of Computers, 43, 1, pp. 41-63, (2020)
  • [4] Han Lei, Xu Meng-Xi, Wang Xin, Et al., Depth estimation from multiple cues based light-field cameras, Chinese Journal of Computers, 43, 1, pp. 107-122, (2020)
  • [5] Wang Jun, Zhu Li, 3D building facade reconstruction based on image matching-point cloud fusing, Chinese Journal of Computers, 35, 10, pp. 2072-2079, (2012)
  • [6] Yan Tao, Qian Yu-Hua, Li Fei-Jiang, Et al., Intelligent microscopic 3D shape reconstruction method based on 3D time-frequency transformation, SCIENTIA SINICA Infor-mationis, 53, 2, pp. 282-308, (2023)
  • [7] Ali U, Mahmood M T., Robust focus volume regularization in shape from focus, IEEE Transactions on Image Processing, 30, pp. 7215-7227, (2021)
  • [8] Zhang Rui, Li Jin-Tao, A survey on algorithm research of scene parsing based on deep learning, Journal of Computer Research and Development, 57, 4, pp. 859-875, (2020)
  • [9] Yang Fengting, Huang Xiaolei, Zhou Zihan, Deep depth from focus with differential focus volume, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 12642-12651, (2022)
  • [10] Wang Ning-Hsu, Wang Ren, Liu Yu-Lun, Et al., Bridging unsupervised and supervised depth from focus via all-in-focus supervision, Proceedings of the IEEE International Conference on Computer Vision, pp. 12621-12631, (2021)
123456