Efficient probability based macroblock mode selection in H.264/AVC

To get high compress efficiency, the latest H.264/AVC international video coding standard introduced many more advanced tools than previously used standards, such as using sophisticated prediction and rate-distortion (RD) mode selection. Although these new coding tools can significantly improve video coding efficiency, the improvement in performance comes at substantially higher computational complexity as a result of more complicated mode decision. To reduce the complexity of H.264 encoding, various fast mode decision algorithms have been proposed. Many of these algorithms take advantage of motion and video content classification so as to reduce the number of modes tried during mode decision. When used for video sequences with high motion or detailed visual information, video coding efficiency can be significantly compromised as a result of imperfect classification. In this paper, an efficient Probability Based Macroblock Mode Selection (PBMMS) algorithm is proposed. Each MB is predicted to different probability MB based on spatial and special correlation, which then leads to 50-60% computation savings and some compression performance improvements over existing H.264 fast mode decision algorithms.