Methodologies used for evaluation of video tools and algorithms in MPEG-4

MPEG-4 issued two calls for proposals requesting submission of algorithms and tools relevant to standardization of MPEG-4. This paper reports on the evaluation of tools submitted for evaluation in November 1995 and January 1996. Complete video coding schemes submitted in January 1996 are also covered. The goal of the evaluation was to cluster the tools according to the technical areas they address, to evaluate them according to the issues relevant to the standardization process, and finally to suggest areas of core experiments to improve a video verification model (VM) as soon as the VM becomes available. Altogether, MPEG evaluated 87 tools and 19 complete coding algorithms, most of them highlighted in this paper. During the evaluation, 19 areas for core experiments were identified. Each core experiment is targeted at different functionalities like compression efficiency, content-based coding, error resilience, scalability. This definition of core experiments caused close collaboration and supported mutual fertilization between organizations working on similar tools, which allowed the VM to progress much faster than expected. (C) 1997 Elsevier Science B.V.