Cycling topic graph learning for neural topic modeling

Topic models aim to discover a set of latent topics in a textual corpus. Graph Neural Networks (GNNs) have been recently utilized in Neural Topic Models (NTMs) due to their strong capacity to model document representations with the text graph. Most of the previous works construct the text graph by considering documents and words as nodes and document embeddings are learned through the topology structure of the text graph. However, while conducting graph learning on topic modeling, sorely considering document- word propagation will lose the guidance of topic relevance and the graph propagation cannot reflect the true relationship at the topic level which will result in inaccurate topic extraction. To address the above-mentioned issue, we propose a novel neural topic model based on Cycling Topic Graph Learning (CyTGL). Specifically, we design a novel three-party topic graph for document-topic-word to incorporate topic propagation into graph-based topic models. In the three-party topic graph, the topic layer is latent and we recursively extract the topic layer through the learning process. Leveraging this topic graph, we employ topic attention message passing to propagate topical information to enhance the document representations. What is more, the topic layer in the three-party graph can be regarded as the prior knowledge that offers guidance for the process of topic extraction. Crucially, the hierarchical relationships in the three-party graph are maintained during the learning process. We conduct experiments on several widely used datasets and the results show our proposed approach outperforms state-of-the-art topic models.