Real-time panoptic segmentation with relationship between adjacent pixels and boundary prediction

被引:1
|
作者
Zhang, Xiaoliang [1 ]
Li, Hongliang [1 ]
Wang, Lanxiao [1 ]
Cheng, Haoyang [1 ]
Qiu, Heqian [1 ]
Hu, Wenzhe [1 ]
Meng, Fanman [1 ]
Wu, Qingbo [1 ]
机构
[1] Univ Elect Sci & Technol China, Sch Informat & Commun Engn, Chengdu 611731, Peoples R China
基金
国家重点研发计划;
关键词
Panoptic segmentation; Graph convolution; Fully convolution; Relationship between adjacent pixels; Boundary prediction;
D O I
10.1016/j.neucom.2022.07.078
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Panoptic segmentation has recently received increasing attention since it generates coherent scene segmentation by unifying semantic and instance segmentation. The most popular methods for panoptic segmentation are currently based on an instance segmentation framework with a semantic segmentation branch in parallel. However, these methods are too bloated for real-world applications. In this paper, we propose a simple yet effective fully convolutional network for fast panoptic segmentation. Instead of directly generating the mask for each instance, we leverage a simple graph convolutional layer to con-struct a pixel relationship head to predict the relationship between two adjacent pixels and determine whether they belong to the same instance. Besides, we leverage boundary information to enhance super-vision information and help our method distinguish adjacent objects. Combining predicted category labels for each pixel from the semantic segmentation branch, we can generate a unified panoptic segmen-tation mask in a parameter-free step. We demonstrate our method's effectiveness on MS COCO dataset and Cityscapes dataset, which obtain competitive results.(c) 2022 Elsevier B.V. All rights reserved.
引用
收藏
页码:290 / 299
页数:10
相关论文
共 50 条
  • [41] Real-time adaptive background segmentation
    Butler, D
    Sridharan, S
    Bove, VM
    2003 INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO, VOL III, PROCEEDINGS, 2003, : 341 - 344
  • [42] Classification boundary approximation by using combination of training steps for real-time image segmentation
    Mitéran, J
    Bouillant, S
    Bourennane, E
    MACHINE LEARNING AND DATA MINING IN PATTERN RECOGNITION, PROCEEDINGS, 2003, 2734 : 141 - 155
  • [43] BOUNDARY CORRECTED MULTI-SCALE FUSION NETWORK FOR REAL-TIME SEMANTIC SEGMENTATION
    Jiang, Tianjiao
    Jin, Yi
    Liang, Tengfei
    Wang, Xu
    Li, Yidong
    2022 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, ICIP, 2022, : 1886 - 1890
  • [44] Real-time Flutter Boundary Prediction Using Peak-Hold Method
    Ma, Yingwen
    Zhou, Li
    NONDESTRUCTIVE CHARACTERIZATION AND MONITORING OF ADVANCED MATERIALS, AEROSPACE, CIVIL INFRASTRUCTURE, AND TRANSPORTATION XII, 2018, 10599
  • [45] Urban Road Network Modeling and Real-Time Prediction Based on Householder Transformation and Adjacent Vector
    Deng, Shuo
    Hu, Jianming
    Wang, Yin
    Zhang, Yi
    ADVANCES IN NEURAL NETWORKS - ISNN 2009, PT 3, PROCEEDINGS, 2009, 5553 : 899 - 908
  • [46] Urban road network modeling and real-time prediction based on householder transformation and adjacent vector
    Department of Automation and TNList, Tsinghua University, Beijing 100084, China
    Lect. Notes Comput. Sci., 1600, PART 3 (899-908):
  • [47] Automatic sensor data stream segmentation for real-time activity prediction in smart spaces
    Korea Advanced Institute of Science and Technology, Korea, Republic of
    IoT-Sys - Proc. Workshop IoT Challenges Mob. Ind. Syst., (13-18):
  • [48] Using density and spatial cues for clustering image pixels in real-time
    Kravtchenko, V
    Little, JJ
    IMAGE AND VIDEO COMMUNICATIONS AND PROCESSING 2000, 2000, 3974 : 741 - 750
  • [49] BiSeNet: Bilateral Segmentation Network for Real-Time Semantic Segmentation
    Yu, Changqian
    Wang, Jingbo
    Peng, Chao
    Gao, Changxin
    Yu, Gang
    Sang, Nong
    COMPUTER VISION - ECCV 2018, PT XIII, 2018, 11217 : 334 - 349
  • [50] Relationship between FRW and AE for real-time quality evaluation of engine valve production and the creep life prediction
    Oh, SK
    Kim, SJ
    Park, HD
    PROCEEDINGS OF THE NINTH (1999) INTERNATIONAL OFFSHORE AND POLAR ENGINEERING CONFERENCE, VOL IV, 1999, 1999, : 222 - 230