Dynamically Removing False Features in Pyramidal Lucas-Kanade Registration

Pyramidal Lucas-Kanade (LK) optical flow is a real-time registration technique widely employed by a variety of cutting edge consumer applications. Traditionally, the LK algorithm is applied selectively to image feature points that have strong spatial variation, which include outliers in textured areas. To detect and discard the falsely selected features, previous methods generally assess the goodness of each feature after the flow computation is completed. Such a screening process incurs additional cost. This paper provides a handy (but not obvious) tool for the users of the LK algorithm to remove false features without degrading the algorithm's efficiency. We propose a confidence predictor, which evaluates the ill-posedness of an LK system directly from the underlying data, at a cost lower than solving the system. We then incorporate our confidence predictor into the course-to-fine LK registration to dynamically detect false features and terminate their flow computation at an early stage. This improves the registration accuracy by preventing the error propagation and maintains (or increases) the computation speed by saving the runtime on false features. Experimental results on state-of-the-art benchmarks validate that our method is more accurate and efficient than related works.