Monitoring process stability in robotic wire-laser directed energy deposition based on multi-modal deep learning

Robotized wire-laser directed energy deposition (DED) is a highly anticipated technology with excellent material utilization and deposition efficiency for fabricating complex-structure metal parts. Nevertheless, the immaturity of the monitoring and control techniques for the deposition process stability are the main challenges in achieving automatic and repeatable manufacturing of metal parts. This study proposes a novel approach to monitor the process stability, i.e., the intersection point between the laser beam and the wire to the top layer distance (IPTD), in robotic wire-laser DED based on a designed multi-modal IPTD estimation model and coaxial visual sensing. The novelty is that directly extracting deposition height features from coaxial molten pool images is attempted based on deep learning. In particular, the designed multi-modal IPTD estimation model, consisting of a convolutional neural network (CNN) part and a fully connected network, combines the features of molten pool images and process parameters to estimate the IPTD state. Six model architectures and three image pixel sizes are used in the IPTD classification tasks to determine the CNN part architecture and the input image pixel size of the multi-modal deep learning model based on classification results. The regression performances of established single-modal and multi-modal IPTD estimation models are studied and discussed. Compared to other model architectures, the ResNet-18 model possesses the highest convergence rate during training and the best classification accuracy of 0.9975 on the testing dataset with the image pixel size of 180 x 105. The fitting accuracy and generalization performance of the multi-modal IPTD estimation model are marked superior to the singlemodal IPTD estimation model. Validation experiments reveal the effectiveness of the proposed monitoring approach of process stability. This study will lay a solid foundation for the future control of process stability in robotic wire-laser DED.