End-to-end online quality prediction for ultrasonic metal welding using sensor fusion and deep learning

In industrial-scale production applications of ultrasonic metal welding (UMW), there is a strong need for predicting joint quality quickly, reliably, and non-destructively. State-of-the-art quality assessment methods such as destructive tensile testing and binary quality classification cannot meet such requirements. This paper develops a novel end-to-end online quality prediction method for UMW based on sensor fusion and deep learning. This method first preprocesses 1-dimensional signals from multiple sensors including an acoustic emission sensor, a linear variable differential transformer, and a microphone, and transforms them to 2 -dimensional images using wavelet transform. Then, these images are fed into ResNet20, which is a 20-layer convolutional neural network, to automatically generate feature maps and predict joint strength. The proposed method offers important advantages compared to state-of-the-art approaches, including automatic feature generation and good robustness to UMW tool conditions. The effectiveness of the developed method is demonstrated using real-world data generated from an UMW process with four different tool conditions. Additionally, we propose three feature fusion strategies (early fusion, middle fusion, and late fusion) and present a comparative case study to compare their performance. It is found that the late fusion strategy achieves the best prediction performance. Towards interpretability and explainability in deep learning, we perform a correlation analysis to reveal the connection between ResNet-generated features and features that are manually extracted based on UMW process physics. It is shown that many manual features are strongly correlated with ResNet features, proving that ResNet is able to resemble physical knowledge. The proposed method is readily applicable to industrial-scale UMW processes to enable accurate online quality prediction.