EvRepSL: Event-Stream Representation via Self-Supervised Learning for Event-Based Vision

被引：0
|
作者：
Qu, Qiang [1 ]
Chen, Xiaoming [2 ]
Ying Chung, Yuk [1 ]
Shen, Yiran [3 ]
机构：
[1] The University of Sydney, School of Computer Science, Sydney,NSW,2050, Australia
[2] Beijing Technology and Business University, School of Computer and Artificial Intelligence, Beijing,102401, China
[3] Shandong University, School of Software, Jinan,250100, China
来源：
IEEE Transactions on Image Processing | 2024年 / 33卷
关键词：
Event-stream representation is the first step for many computer vision tasks using event cameras. It converts the asynchronous event-streams into a formatted structure so that conventional machine learning models can be applied easily. However; most of the state-of-the-art event-stream representations are manually designed and the quality of these representations cannot be guaranteed due to the noisy nature of event-streams. In this paper; we introduce a data-driven approach aiming at enhancing the quality of event-stream representations. Our approach commences with the introduction of a new event-stream representation based on spatial-temporal statistics; denoted as EvRep. Subsequently; we theoretically derive the intrinsic relationship between asynchronous event-streams and synchronous video frames. Building upon this theoretical relationship; we train a representation generator; RepGen; in a self-supervised learning manner accepting EvRep as input. Finally; the event-streams are converted to high-quality representations; termed as EvRepSL; by going through the learned RepGen (without the need of fine-tuning or retraining). Our methodology is rigorously validated through extensive evaluations on a variety of mainstream event-based classification and optical flow datasets (captured with various types of event cameras). The experimental results highlight not only our approach's superior performance over existing event-stream representations but also its versatility; being agnostic to different event cameras and tasks. © 1992-2012 IEEE;
D O I：
10.1109/TIP.2024.3497795
中图分类号：
学科分类号：
摘要：
引用
收藏
页码：6579 / 6591
相关论文
共 50 条
  • [41] DocMAE: Document Image Rectification via Self-supervised Representation Learning
    Liu, Shaokai
    Feng, Hao
    Zhou, Wengang
    Li, Houqiang
    Liu, Cong
    Wu, Feng
    2023 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, ICME, 2023, : 1613 - 1618
  • [42] METRICBERT: TEXT REPRESENTATION LEARNING VIA SELF-SUPERVISED TRIPLET TRAINING
    Malkiel, Itzik
    Ginzburg, Dvir
    Barkan, Oren
    Caciularu, Avi
    Weill, Yoni
    Koenigstein, Noam
    2022 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2022, : 8142 - 8146
  • [43] Self-Supervised Intensity-Event Stereo Matching
    Gu, Jinjin
    Zhou, Jinan
    Chu, Ringo Sai Wo
    Chen, Yan
    Zhang, Jiawei
    Cheng, Xuanye
    Zhang, Song
    Ren, Jimmy S.
    JOURNAL OF IMAGING SCIENCE AND TECHNOLOGY, 2022, 66 (06)
  • [44] Self-Supervised Representation Learning via Neighborhood-Relational Encoding
    Sabokrou, Mohammad
    Khalooei, Mohammad
    Adeli, Ehsan
    2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2019), 2019, : 8009 - 8018
  • [45] Self-supervised Health Representation Decomposition based on contrast learning
    Wang, Yilin
    Shen, Lei
    Zhang, Yuxuan
    Li, Yuanxiang
    Zhang, Ruixin
    Yang, Yongshen
    RELIABILITY ENGINEERING & SYSTEM SAFETY, 2023, 239
  • [46] Self-Distilled Self-supervised Representation Learning
    Jang, Jiho
    Kim, Seonhoon
    Yoo, Kiyoon
    Kong, Chaerin
    Kim, Jangho
    Kwak, Nojun
    2023 IEEE/CVF WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV), 2023, : 2828 - 2838
  • [47] A Dynamic GCN with Cross-Representation Distillation for Event-Based Learning
    Deng, Yongjian
    Chen, Hao
    Li, Youfu
    THIRTY-EIGHTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, VOL 38 NO 2, 2024, : 1492 - 1500
  • [48] Self-supervised Representation Learning on Document Images
    Cosma, Adrian
    Ghidoveanu, Mihai
    Panaitescu-Liess, Michael
    Popescu, Marius
    DOCUMENT ANALYSIS SYSTEMS, 2020, 12116 : 103 - 117
  • [49] Self-Supervised Relational Reasoning for Representation Learning
    Patacchiola, Massimiliano
    Storkey, Amos
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 33, NEURIPS 2020, 2020, 33
  • [50] Self-Supervised Learning for Specified Latent Representation
    Liu, Chicheng
    Song, Libin
    Zhang, Jiwen
    Chen, Ken
    Xu, Jing
    IEEE TRANSACTIONS ON FUZZY SYSTEMS, 2020, 28 (01) : 47 - 59
12345