Efficient rate-distortion estimation for H.264/AVC coders

In video coders, the optimal coding mode decision for each coding block can be achieved by exhaustively calculating the rate-distortion cost, which simultaneously considers the distortion performance and the coding bit consumption of all possible modes. The best mode is chosen from the one with the minimum Lagrange cost. To avoid the expensive computation of Lagrange costs, in this paper, we propose transform-domain bit-rate estimation and distortion measures, based on quantized and inverse quantized integer transform coefficients, for the inter-mode decision in H.264/AVC coders. With the proposed scheme, entropy coding, inverse transform, and pixel-reconstructions are not required in the process. With ignorable degradation in coding performance, simulations demonstrate that the proposed estimation method achieves about 40% reduced computation time of rate-distortion cost for the inter-mode decision and saves about 17% total encoding time while combining with fast motion estimation and fast mode decision algorithms.