Miper-MVS: Multi-scale iterative probability estimation with refinement for efficient multi-view stereo

Multi-view stereo reconstruction aims to construct 3D scenes from multiple 2D images. In recent years, learning-based multi-view stereo methods have achieved significant results in depth estimation for multi-view stereo reconstruction. However, the current popular multi-stage processing method cannot solve the low-efficiency problem satisfactorily owing to the use of 3D convolution and still involves significant amounts of calculation. Therefore, to further balance the efficiency and generalization performance, this study proposed a multi-scale iterative probability estimation with refinement, which is a highly efficient method for multi-view stereo reconstruction. It comprises three main modules: 1) a high-precision probability estimator, dilated-LSTM that encodes the pixel probability distribution of depth in the hidden state, 2) an efficient interactive multi-scale update module that fully integrates multi-scale information and improves parallelism by interacting information between adjacent scales, and 3) a Pi-error Refinement module that converts the depth error between views into a grayscale error map and refines the edges of objects in the depth map. Simultaneously, we introduced a large amount of high-frequency information to ensure the accuracy of the refined edges. Among the most efficient methods (e.g., runtime and memory), the proposed method achieved the best generalization on the Tanks & Temples benchmarks. Additionally, the performance of the Miper-MVS was highly competitive in DTU benchmark. Our code is available at https://github.com/zhz120/Miper-MVS.(c) 2023 Elsevier Ltd. All rights reserved.