ACOUSTIC AND VIDEO SENSOR INTEGRATION FOR OBJECT RECOGNITION

Through our studies of methods for measuring the shapes of objects in three-dimensional object recognition, we have developed a method for constructing detailed solid object images that employs the fusion of the data from acoustic sensors and a charge-coupled device (CCD) camera. This method uses a matrix of ultrasonic sensors to obtain data on the position and height of the object. These data are used to automatically extract the two-dimensional images of the object from gray-scale camera images. By combining the results with distance information, a detailed solid image of the object is obtained. This method produces markedly better resolution than using acoustic data alone. Thus, by using it in combination with a neural network recognition mechanism, it is possible to automatically recognize small objects that am difficult to distinguish by means of acoustic sensing alone, even if they can be detected. This paper reports the newly developed sensor fusion mechanism, presents the results of experiments on an experimental system, and discusses the features of the method.