An actor-critic learning framework based on Lyapunov stability for automatic assembly

With the continuous improvement of the reinforcement learning (RL) algorithm, the algorithm has achieved excellent performance in an increasing number of automatic control tasks. However, there are still some challenges when applying the algorithm to realistic automatic assembly. The most significant challenge is that the stability of these model-free RL methods cannot be effectively guaranteed. Stability is the most critical characteristic of a control system, and stability is closely related to reliability and safety. To ensure the stability of the system, we reconstruct the RL algorithm based on the Lyapunov stability theory of the stochastic system proposed in this paper. An actor-critic learning framework based on Lyapunov stability (LSAC) is proposed for automatic assembly. In addition, this paper proposes a median Q-value theory to alleviate the Q-value estimation deviation that restricts the performance of the RL algorithm. To allow RL agents to better complete the automatic assembly task, this paper designs an adaptive impedance control algorithm. This impedance algorithm executes the actions output by the LSAC framework. Finally, a realistic experiment on automatic assembly is carried out to verify the robustness and superiority of the proposed strategy.