Self-adaptive SURF for image-to-video matching

Speeded up robust feature (SURF) is one of the most popular feature-based algorithms handling image matching. Compared to emerging deep learning neural network-based image matching algorithms, SURF is much faster with comparable accuracy. Currently, it is still one of the dominant algorithms adopted in majority of real-time applications. With the increasing popularity of video-based computer vision applications, image matching between an image and different frames of a video stream is required. Traditional algorithms could fail to deal with live video because spatiotemporal differences between frames could cause significant fluctuation in the results. In this study, we propose a self-adaptive methodology to improve the stability and precision of image–video matching. The proposed methodology dynamically adjusts threshold in feature points extraction to control the number of extracted feature points based on the content of the previous frame. Minimum ratio of distance (MROD) matching is integrated to preclude false matches while keeping abundant sample sizes. Finally, multiple homography matrix (H-Matrix) are estimated using progressive sample consensus (PROSAC) with various reprojection errors. The model with lowest mean square error (MSE) will be selected for image-to-video frame matching. The experimental results show that the self-adaptive SURF offers more accurate and stable results while balancing single frame processing time in image-video matching.