An Empirical Study on Fault Diagnosis in Robotic Systems

Fault diagnosis in robotic systems is challenging due to their complex and heterogeneous structures and complex interactions with physical environments. Given the complexities and uncertainties, we think it may be helpful to diagnose faults of a robotic system by understanding its behaviors from the perspective of observability. In this paper, we conduct an empirical study to explore the efficacy of combining different kinds of common observability data (i.e., logs, traces, and trajectories) for fault diagnosis in robotic systems. In the study, we investigate root causes of 398 bug cases in robotic systems to understand their characteristics. Furthermore, we replicate 23 bugs out of them and perform a fault diagnosis study in which participants diagnose each of the replicated bug with only observability data and record how useful observability data is. The bug case analysis study revealed that the root causes of bugs in robotic systems originate from various levels, including physical environment interaction (11.81%), hardware usage (14.82%), software implementation (49.25%), and system configuration (24.12%). The fault diagnosis study shows the combination of trace and trajectory data improves the fault diagnosis success rate by 58.33% and 8.33%, respectively, compared to using only logs. Our study promotes the vision of observability-based fault diagnosis in robotic systems.