Multi-Modal Pedestrian Trajectory Prediction for Edge Agents Based on Spatial-Temporal Graph

被引:11
|
作者
Zou, Xiangyu [1 ,2 ,3 ]
Sun, Bin [1 ,2 ,3 ]
Zhao, Duan [1 ,2 ,3 ]
Zhu, Zongwei [4 ]
Zhao, Jinjin [1 ,2 ,3 ]
He, Yongxin [1 ,2 ,3 ]
机构
[1] China Univ Min & Technol, Sch Informat & Control Engn, Xuzhou 221008, Jiangsu, Peoples R China
[2] China Univ Min & Technol, Internet Things Percept Mine Res Ctr, Xuzhou 221008, Jiangsu, Peoples R China
[3] Natl Joint Engn Lab Internet Appl Technol Mines, Xuzhou 221008, Jiangsu, Peoples R China
[4] Univ Sci & Technol China, Suzhou Inst Adv Study, Suzhou 215000, Peoples R China
来源
IEEE ACCESS | 2020年 / 8卷
关键词
Trajectory prediction; spatial-temporal graph; generative adversarial network; global node; scaled dot product attention; MODEL; ATTENTION;
D O I
10.1109/ACCESS.2020.2991435
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Edge agents, represented by socially-aware robots and autonomous vehicles, have gradually been integrated into human society. The safety navigation system in interactive scenes is of great importance to them. The key of this system is that the edge agent has the ability to predict the pedestrian trajectory in the dynamic scene, so as to avoid collision. However, predicting pedestrian trajectories in dynamic scenes is not an easy task, because it is necessary to comprehensively consider the spatial-temporal structure of human-environment interaction, visual attention, and the multi-modal behavior of human walking. In this paper, a scalable spatial-temporal graph generation adversarial network architecture (STG-GAN) is introduced, which can comprehensively consider the influence of human-environment interaction and generate a reasonable multi-modal prediction trajectory. First, we use LSTM nodes to flexibly transform the spatial-temporal graph of human-environment interactions into feed-forward differentiable feature coding, and innovatively propose the global node to integrate scene context information. Then, we capture the relative importance of global interactions on pedestrian trajectories through scaled dot product attention, and use recurrent sequence modeling and generative adversarial network architecture for common training, so as to generate reasonable pedestrian future trajectory distributions based on rich mixed features. Experiments on public data sets show that STG-GAN is superior to previous work in terms of accuracy, reasoning speed and rationality of trajectory prediction.
引用
收藏
页码:83321 / 83332
页数:12
相关论文
共 50 条
  • [1] Multimodal Pedestrian Trajectory Prediction Based on Relative Interactive Spatial-Temporal Graph
    Zhao, Duan
    Li, Tao
    Zou, Xiangyu
    He, Yaoyi
    Zhao, Lichang
    Chen, Hui
    Zhuo, Minmin
    [J]. IEEE ACCESS, 2022, 10 : 88707 - 88718
  • [2] Multi-modal Pedestrian Trajectory Prediction based on Pedestrian Intention for Intelligent Vehicle
    He, Youguo
    Sun, Yizhi
    Cai, Yingfeng
    Yuan, Chaochun
    Shen, Jie
    Tian, Liwei
    [J]. KSII TRANSACTIONS ON INTERNET AND INFORMATION SYSTEMS, 2024, 18 (06): : 1562 - 1582
  • [3] Graph based Spatial-temporal Fusion for Multi-modal Person Re-identification
    Zhang, Yaobin
    Lv, Jianming
    Liu, Chen
    Cai, Hongmin
    [J]. PROCEEDINGS OF THE 31ST ACM INTERNATIONAL CONFERENCE ON MULTIMEDIA, MM 2023, 2023, : 3736 - 3744
  • [4] Learning Pedestrian Group Representations for Multi-modal Trajectory Prediction
    Bae, Inhwan
    Park, Jin-Hwi
    Jeon, Hae-Gon
    [J]. COMPUTER VISION, ECCV 2022, PT XXII, 2022, 13682 : 270 - 289
  • [5] Temporal Pyramid Network With Spatial-Temporal Attention for Pedestrian Trajectory Prediction
    Li, Yuanman
    Liang, Rongqin
    Wei, Wei
    Wang, Wei
    Zhou, Jiantao
    Li, Xia
    [J]. IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING, 2022, 9 (03): : 1006 - 1019
  • [6] Trajectory prediction for autonomous driving based on multiscale spatial-temporal graph
    Tang, Luqi
    Yan, Fuwu
    Zou, Bin
    Li, Wenbo
    Lv, Chen
    Wang, Kewei
    [J]. IET INTELLIGENT TRANSPORT SYSTEMS, 2023, 17 (02) : 386 - 399
  • [7] STIGCN: spatial-temporal interaction-aware graph convolution network for pedestrian trajectory prediction
    Chen, Wangxing
    Sang, Haifeng
    Wang, Jinyu
    Zhao, Zishan
    [J]. JOURNAL OF SUPERCOMPUTING, 2024, 80 (08): : 10695 - 10719
  • [8] Trajectory prediction of cyclist based on spatial-temporal multi-graph network in crowded scenarios
    Li, Meng
    Chen, Tao
    Du, Hao
    [J]. ELECTRONICS LETTERS, 2022, 58 (03) : 97 - 99
  • [9] MSTCNN: multi-modal spatio-temporal convolutional neural network for pedestrian trajectory prediction
    Haifeng Sang
    Wangxing Chen
    Haifeng Wang
    Jinyu Wang
    [J]. Multimedia Tools and Applications, 2024, 83 : 8533 - 8550
  • [10] MSTCNN: multi-modal spatio-temporal convolutional neural network for pedestrian trajectory prediction
    Sang, Haifeng
    Chen, Wangxing
    Wang, Haifeng
    Wang, Jinyu
    [J]. MULTIMEDIA TOOLS AND APPLICATIONS, 2024, 83 (03) : 8533 - 8550