Spatio-temporal rate allocation for hybrid video coding

State-of-the-art video coders (e.g.(1)) feature a high level of adaptivity to meet the properties of their input signals. Video signal statistics are apt to sometimes discontinuously vary in all three dimensions: spatially and temporally. Consequently for video compression, different coding tools are employed at different coordinates of the signal to adaptively and maximally reduce redundancy. Since the decision for a certain tool and for the apropriate parameters is highly signal-dependent, a video coder forms a non-linear system and optimization is not trivial to perform. Lagrange Rate/Distortion-Optimization (RDO) has become an important tool for video encoding. It has achieved high gains in coding efficiency when applied to the independent encoding of macroblocks (MBs).(2) For a chosen quantizer, bit rate is allocated to each MB's coefficients and prediction parameters subject to minimization of a Lagrange cost function of rate and distortion. In effect, the cheaper option is chosen to either use a more precise prediction, i.e. motion vectors and block tiling, or to spend more bit rate on the coefficients. Often, the video coding objective is that of constant quality. This is approximately achieved by a constant quantizer for all MBs and RDO within each MB. Due to the recursive structure of hybrid video coders though, MBs are dependent on each other, demanding for dependent RDO. This has early been formulated,(3) but is hardly solvable due to the high dimensionality of this problem. In the following, possible simplified ways to take dependencies into account are explored. The variations in all three dimensions of the signal properties are differentiated and simplified models used for a dependent optimization.