HaGAN: Hierarchical Attentive Adversarial Learning for Task-Oriented Dialogue System

Task-oriented dialogue system is commonly formulated as a reinforcement learning problem. A reward served as a learning objective is offered at the end of the generated dialogue to help optimize the system. As fulfilling a specific task often takes many turns between the system and the user, a scalar reward signal after this long process can be delayed and sparse. To address the above problems in the reinforcement learning (RL) based task-completion system, we propose a novel hierarchical attentive adversarial network HaGAN which features a cascaded attentive generator CAG that explores a state-action space to generate a dialogue and global-local attentive discriminators GLAD to give a relevant reward at multi-scale dialogue states. Specifically, after every turn of the dialogue generation, the turn-based discriminator tests the current turn and give a local reward representing the generator's current generating ability. When the dialogue finishes, the dialogue-based discriminator gives a global reward concerns the whole dialog. Finally, a synthesized reward computed by combining global and local reward is returned to the generator. By doing so, the generator is able to generate globally and locally fluent and informative dialogues. Through experiments on two public benchmark datasets demonstrate the superiority of our HaGAN over other representative state-of-the-art methods.