Communication Architectures and Algorithms for Media Mixing in Multimedia Conferences

Advances in computer and communication technologies have stimulated the integration of digital video and audio with computing, leading to the development of computer-assisted multimedia conferencing. We address the problem of media mixing which arises in teleconferencing applications such as teleorchestra. We present a mixing algorithm which minimizes the difference between generation times of the media packets that are being mixed together in the absence of globally synchronized clocks, but in the presence of jitter in communication delays on packet switched networks. The algorithm is shown to be complete: given that there are no other message exchanges except media data between mixers and media sources, there cannot be any other algorithm that succeeds when our algorithm fails. Mixing can be accomplished by several different communication architectures. In order to support applications such as teleorchestra, which involve a large number of participants, we propose hierarchical mixing architectures and show that they are an order of magnitude more scalable compared to purely centralized or distributed architectures. Furthermore, we present mechanisms for minimizing the delays incurred by mixing in various communication architectures. We have implemented the mixing algorithms on a network of workstations connected by Ethernets, and have experimentally evaluated the performance of various mixing architectures. These experiments revealed interesting results such as the maximum number of participants that can be supported in a conference.