An automatic unsafe states reasoning approach towards Industry 5.0's human-centered manufacturing via Digital Twin

The safety management of manufacturing workshops is crucial for ensuring human- centered Industry 5.0. To infer the unsafe state of a manufacturing workshop, an integrated Digital Twin (DT) and ontology method was used. However, two challenges arise from this approach. First, creating and labeling virtual-real mixed datasets for unsafe state detection typically requires manual effort. Secondly, recognizing semantic relations of instances of unsafe states using an ontology is time-consuming. To address these challenges, an automatic semantic reasoning framework for unsafe state in a Digital Twin Workshop (DTW) is proposed by integrating instance segmentation, relationship detection and ontology technology. For the first challenge, the Mask R-CNN algorithm is applied to generate automatically a semi-virtual dataset in a DTW, which is then mixed with the real datasets to form a virtual-real mixed dataset. This dataset detects object instances of unsafe states. For the second challenge, a relationship detection model is constructed to predict the semantic relationship of detected object instances. The predicted semantic relationship and detected instances are then mapped to unsafe states in an ontology to be used for automatic reasoning. Finally, an experiment in a welding shop demonstrates that the proposed approach can alleviate the two challenges.