Learning continuous dynamic network representation with transformer-based temporal graph neural network

Continuous dynamic graph neural network (DGNN) methods have attracted increasing attention owing to their ability to learn fine-grained temporal representations. Real-world network events not only involve newly emerged edges but also affect larger regions owing to changes in the overall structure. However, most continuous DGNN methods only aggregate local neighborhood information and ignore the importance of global information, leading to the loss of structural and semantic information that is not aggregated during the evolution of networks. Therefore, we propose a continuous dynamic network representation learning model called the Transformer based Temporal Graph Neural Network (T-TGNN) to aggregate global event-driven information in continuous dynamic networks. Specifically, we first adopted ordinary neural differential equations to model continuous temporal changes in dynamic networks. Subsequently, we used transformer mechanisms to aggregate the temporal and structural information. To better utilize transformers to aggregate the impacts of large regional changes, we designed a spatial event coefficient based on an attention mechanism to describe the global range of event dependencies in dynamic networks. Finally, extensive experiments were conducted on dynamic network benchmark datasets to demonstrate the effectiveness of our model. The T-TGNN achieved the best performance over state-of-the-art baselines of network embedding.