A Knowledge-driven Approach to Interactive Event Recognition for Semantic Video Understanding

Since early 1990, event recognition has been one of the most attractive research topics for video understanding, in company with object recognition. Most studies on video event recognition, which are based on data-driven approaches, should train a model for a newly-added event without using human knowledge and existing models for similar events. Because it is impossible to define all events required for video understanding in advance, this paper proposed a hierarchical recognition method for general events based on dynamic spatial relations between two objects and specialized events determined by the related objects. The general events are useful for describing interactions between objects of interest regardless of video domain. The specialized events can be provided to users as familiar terms in video interpretation or visual question answering for user-friendly interaction. For two general events and their specialized four events, the proposed recognition method performed the F-score of 82.31% and 88.61% based on object-based and region-based event matching, respectively.