Personalized multimedia summarization

Summarization and abstraction are our survival tools in this age of information explosion and availability. Ability to summarize will be seen as essential part of intelligent behavior of the consumer devices. We introduce the notion of video summarization, and provide definitions of the different flavors of summaries: Video skim, highlights, and structured multimedia summary. We present different features and methods for automatic video content analysis for the purpose of summarization: color analysis using Super-histograms, transcript extraction and analysis, superimposed and overlaid text detection, face detection, commercial detection. All these different types of video content analysis are used to produce multimedia summaries. We use audio, visual, and text information for selecting important elements of news stories for multimedia news summary. We present a method for surface level summarization and its applications: a talk show browser and a content-based video recorder called Video Scout. We also discuss a method for news story segmentation and summarization as an example of structured video summarization. Summaries presented to users should be personalized. Our initial study shows that personal preferences include implicit features such as viewing history, summary usage history, and explicit preferences such as topics, location, age, gender, profession, hobbies, allocated time, consumption preferences (auditory vs. visual). We believe that personalized summarization will provide essential tools for access to relevant information everywhere at any time in the context of ambient intelligence.