Vector quantization and minimum description length

In this paper we address the problem of finding the optimal number of reference vectors in vector quantization from the point of view of the Minimum Description Length (MDL) principle. We formulate the VQ in terms of the MDL principle, and then derive depending on the coding procedure different instantiations of the algorithm. Moreover, we develop an efficient algorithm (similar to EM-type algorithms) for optimizing the MDL criterion. In addition we can use the MDL principle to increase the robustness of the training algorithm. In order to visualize the behavior of the algorithm, well illustrate our approach on 2D clustering problems and present applications on image coding. Finally we outline various ways to extend the algorithm.