Tests on MPEG-4 audio codec proposals

During December 1995, subjective tests were carried out by members of the Moving Picture Experts Group (MPEG, ISO/JTC1/SC29/WG11) to select the proposed technology for inclusion in the audio part of the new MPEG-4 standard. The new standard addresses coding for more than just the functionality of data rate compression. Material coded at very low bit-rates is also included. Thus, different testing methodologies were applied, according to ITU-R Rec. BS 1116 for a bit-rate of 64 kbit/s per channel and according to ITU-T Rec, P.80 for lower bit-rates or functionalities other than data rate compression. Proposals were subjectively tested for coding efficiency, error resilience, scalability and speed change: a subset of the MPEG-4 'functionalities'. This paper describes how two different evaluation methods were used and adjusted to fit the different testing requirements. This first major effort to test coding schemes at low bit-rates proved successful. Based on the test results, decisions for MPEG-4 technology were made. This was the first opportunity for MPEG members to carry out tests on the submitted functionalities. In the process, much was learnt, As a result, some suggestions are made to improve the way new functionalities can be subjectively evaluated. (C) 1997 Elsevier Science B.V.