Combining convolutional neural networks with unsupervised learning for acoustic monitoring of robotic manufacturing facilities

For automated robotic manufacturing, a key aspect of monitoring is the identification and segmentation of core actuation processes captured in sensor logs. Once segmented, the behavior of an industrial system during a particular actuation can be tracked to detect signs of degradation. This study presents a technique for performing such an analysis through a combination of machine learning techniques designed to work with an acoustic monitoring system. A spectrogram-based convolutional neural network (CNN) is first trained to identify and segment primary motion classes from acoustic data. Unsupervised clustering and feature-space analysis are then employed to further separate the data into motion sub-classes beyond the capabilities of the CNN. This approach was evaluated on acoustic recordings of a Selective Compliance Assembly Robot Arm (SCARA) system. The developed CNN performed primary robotic motion segmentation with a maximum actuation identification accuracy of 87% when compared to validation data. The unsupervised clustering process had mixed success in distinguishing more fine-grained motion sub-classes due to strong variances in signal energy for some sub-classes. Further refinement is required for improved segmentation accuracy as well as automatic feature generation. The application of this process for life-cycle system monitoring is discussed as well.