Packet loss resilient MPEG-4 compliant video coding for the Internet

Targeting multimedia communications over the Internet, this paper describes a set of complementary techniques in the direction of both improved packet loss resiliency of video-compressed streams and efficient usage of available network resources. Aiming first at a best trade-off between compression efficiency and packet loss resiliency, a procedure for adapting the video coding modes to varying network characteristics is introduced. The coding mode selection is based on a rate-distortion procedure with global distortion metrics incorporating channel characteristics under the form of a two-state Markov model. This procedure has been incorporated in an MPEG-4 video encoder. It has been observed that, in error-free environments, the channel adaptive mode selection technique does not bring any penalty in terms of compression, with respect to the initial MPEG-4 encoder, while allowing a significant gain with respect to simple conditional replenishment. On the other hand, Finder the same loss conditions, it is shown that this procedure significantly improves the encoder's performance with respect to the original MPEG-4 encoder, to approach the robustness of conditional replenishment mechanisms. This intrinsic robustification of the encoder allows to minimize the effects of packet losses on the visual quality of the received video; however, it does not avoid losses. A rate-based flow control mechanism is then developed and introduced into the encoder, in order to match the bandwidth requirements of the source to the bandwidth available over the path of the connection, for both 'social' and 'individual' benefits. The control mechanism developed combines an RTT-based control loop allowing early reaction to congestion and a TCP-friendly rate prediction model getting into play under lossy conditions. This hybrid control mechanism allows full rate control (even in loss-free conditions) and smooth rate variations together with high responsiveness. The introduction of the rate control in the MPEG-4 compliant encoder allows to maintain a stable PSNR and visual quality while decreasing significantly the source throughput, hence reducing congestion and loss provoked by the same video source at a constant bit-rate. (C) 1999 Elsevier Science B.V. All rights reserved.