Efficient bit-rate estimation for mode decision of H.264/AVC

CAVLC is the entropy coding method adopted in H.264/AVC baseline profile. RDO is the mode decision method commended by JVT, which can improve the performance of the encoder markedly. However, during the process of RDO, entropy coding should be performed for every candidate coding, mode, which increases the computation complexity of video encoding largely. To avoid the expensive computation of Lagrange costs, in this paper, we propose a transform-domain bit-rate estimation method for CAVLC based on the analyses of its adaptive characteristic and apply it in a fast mode decision algorithm, in which bit-rate estimation is adopted instead of actual entropy coding in RDO mode decision, thus reducing the computing time for mode decision. Simulation results show that the fast algorithm based on proposed bitrate estimation method for CAVLC saves much encoding time while the losses of compression performance are negligible.