Multimedia retrieval from a large number of sources in a ubiquitous environment

A system for multimedia retrieval and summarization in a ubiquitous environment is presented. Hierarchical clustering of data from pressure-based floor sensors is followed by video handover to retrieve video sequences showing the movement of each person in the environment. Audio handover is implemented to dub these sequences. Several methods for extracting key frames from the video sequences were implemented and evaluated by experiments. An adaptive spatio-temporal sampling algorithm based on the rate of footsteps yielded the best performance. The measured accuracy of key frame extraction within a difference of 3 seconds is approximately 80%. The system consists of a graphical user interface that can be used to retrieve video summaries interactively using simple queries.