EvRepSL: Event-Stream Representation via Self-Supervised Learning for Event-Based Vision

被引：0
|
作者：
Qu, Qiang [1 ]
Chen, Xiaoming [2 ]
Ying Chung, Yuk [1 ]
Shen, Yiran [3 ]
机构：
[1] The University of Sydney, School of Computer Science, Sydney,NSW,2050, Australia
[2] Beijing Technology and Business University, School of Computer and Artificial Intelligence, Beijing,102401, China
[3] Shandong University, School of Software, Jinan,250100, China
来源：
IEEE Transactions on Image Processing | 2024年 / 33卷
关键词：
Event-stream representation is the first step for many computer vision tasks using event cameras. It converts the asynchronous event-streams into a formatted structure so that conventional machine learning models can be applied easily. However; most of the state-of-the-art event-stream representations are manually designed and the quality of these representations cannot be guaranteed due to the noisy nature of event-streams. In this paper; we introduce a data-driven approach aiming at enhancing the quality of event-stream representations. Our approach commences with the introduction of a new event-stream representation based on spatial-temporal statistics; denoted as EvRep. Subsequently; we theoretically derive the intrinsic relationship between asynchronous event-streams and synchronous video frames. Building upon this theoretical relationship; we train a representation generator; RepGen; in a self-supervised learning manner accepting EvRep as input. Finally; the event-streams are converted to high-quality representations; termed as EvRepSL; by going through the learned RepGen (without the need of fine-tuning or retraining). Our methodology is rigorously validated through extensive evaluations on a variety of mainstream event-based classification and optical flow datasets (captured with various types of event cameras). The experimental results highlight not only our approach's superior performance over existing event-stream representations but also its versatility; being agnostic to different event cameras and tasks. © 1992-2012 IEEE;
D O I：
10.1109/TIP.2024.3497795
中图分类号：
学科分类号：
摘要：
引用
收藏
页码：6579 / 6591
相关论文
共 50 条
  • [21] Self-Supervised Graph Representation Learning via Topology Transformations
    Gao, Xiang
    Hu, Wei
    Qi, Guo-Jun
    IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2023, 35 (04) : 4202 - 4215
  • [22] Self-Supervised Representation Learning via Latent Graph Prediction
    Xie, Yaochen
    Xu, Zhao
    Ji, Shuiwang
    INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 162, 2022,
  • [23] Self-Supervised Graph Representation Learning via Information Bottleneck
    Gu, Junhua
    Zheng, Zichen
    Zhou, Wenmiao
    Zhang, Yajuan
    Lu, Zhengjun
    Yang, Liang
    SYMMETRY-BASEL, 2022, 14 (04):
  • [24] Self-Supervised Visual Representation Learning via Residual Momentum
    Pham, Trung Xuan
    Niu, Axi
    Zhang, Kang
    Jin, Tee Joshua Tian
    Hong, Ji Woo
    Yoo, Chang D.
    IEEE ACCESS, 2023, 11 : 116706 - 116720
  • [25] Self-supervised graph representation learning via positive mining
    Lee, Namkyeong
    Lee, Junseok
    Park, Chanyoung
    INFORMATION SCIENCES, 2022, 611 : 476 - 493
  • [26] Event-Stream Representation for Human Gaits Identification Using Deep Neural Networks
    Wang, Yanxiang
    Zhang, Xian
    Shen, Yiran
    Du, Bowen
    Zhao, Guangrong
    Lizhen, Lizhen Cui Cui
    Wen, Hongkai
    IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2022, 44 (07) : 3436 - 3449
  • [27] CSTR: A Compact Spatio-Temporal Representation for Event-Based Vision
    El Shair, Zaid A.
    Hassani, Ali
    Rawashdeh, Samir A.
    IEEE ACCESS, 2023, 11 : 102899 - 102916
  • [28] Whitening for Self-Supervised Representation Learning
    Ermolov, Aleksandr
    Siarohin, Aliaksandr
    Sangineto, Enver
    Sebe, Nicu
    INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 139, 2021, 139
  • [29] Self-Supervised Representation Learning for CAD
    Jones, Benjamin T.
    Hu, Michael
    Kodnongbua, Milin
    Kim, Vladimir G.
    Schulz, Adriana
    2023 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2023, : 21327 - 21336
  • [30] Exploiting Lightweight Statistical Learning for Event-Based Vision Processing
    Shi, Cong
    Li, Jiajun
    Wang, Ying
    Luo, Gang
    IEEE ACCESS, 2018, 6 : 19396 - 19406
12345