Fast prediction mode selection for H.264/AVC video coding

The H.264 video coding standard provides considerably higher coding efficiency than previous standards, whereas its complexity is significantly increased at the same time. In an H.264 encoder, the most time-consuming component is variable block-size motion estimation. To reduce the complexity of motion estimation, this paper proposed a novel Fast prediction mode selection (FPMS) algorithm using spatial/temporal correlation of video sequence. Firstly, a frame is partitioned into three sets of background region, sub- background region, and motion region. Macroblocks in different motion regions may have different prediction mode space. Secondly, using spatial/temporal correlation, we can calculate the motion region to which a macroblock of inter-frame belongs. Then, the prediction mode of various motion regions can be used to greatly simplify the prediction mode selection of the macroblock, and the computational complexity in encoding inter-frame is also reduced. Two key parameters may be updated adaptively to further reduce the computational complexity and avoid the accumulation of prediction error. Simulation results show that the proposed FPMS method can reduce efficiently the computational cost by 2 to 3 times while the PSNR (Peak signal to noise ratio) degrades 0.1dB to 0.2dB and the bit rates increase less than 2%.