Vision in the real world:: Finding, attending and recognizing objects

In this paper we discuss the notion of a "seeing" system that uses vision to interact with its environment. The requirements on such a system depend on the tasks it is involved in and should be evaluated with these in mind. Here we consider the task of finding and recognizing objects in the real world. After a discussion of the needed functionalities and issues about the design we present an integrated real-time vision system capable of finding, attending and recognizing objects in real settings. The system is based on a dual set of cameras, a wide field set for attention and a foveal one for recognition. The continuously running attentional process uses top-down object characteristics in terms of hue and 3D size. Recognition is performed with objects of interest foveated and segmented from its background. We describe the system structure as well as the different components in detail and present experimental evaluations of its overall performance. (C) 2007 Wiley Periodicals, Inc.