Deep relational self-Attention networks for scene graph generation

Scene graph generation (SGG) aims to simultaneously detect objects in an image and predict relations for these detected objects. SGG is challenging that requires modeling the contextualized relationships among objects rather than only considering relationships between paired objects. Most existing approaches ad -dress this problem by using a CNN or RNN framework, which can not explicitly and effectively models the dense interactions among objects. In this paper, we exploit the attention mechanism and introduce a relational self-attention (RSA) module to simultaneously model the object and relation contexts. By stack -ing such RSA modules in depth, we obtain a deep relational self-attention network (RSAN), which is able to characterize complex interactions thus facilitating the understanding of object and relation semantics. Extensive experiments on the benchmark Visual Genome dataset demonstrate the effectiveness of RSAN. (c) 2021 Elsevier B.V. All rights reserved.