Persistently trained, diffusion-assisted energy-based models

被引:0
|
作者
Zhang, Xinwei [1 ]
Tan, Zhiqiang [2 ]
Ou, Zhijian [3 ,4 ]
机构
[1] NYU, Dept Biostat, New York, NY USA
[2] Rutgers State Univ, Dept Stat, Piscataway, NJ 08854 USA
[3] Tsinghua Univ, Dept Elect Engn, Beijing, Peoples R China
[4] Tsinghua Univ, Dept Elect Engn, Beijing 100084, Peoples R China
来源
STAT | 2023年 / 12卷 / 01期
关键词
energy-based models (EBMs); out-of-distribution detection; score-based diffusion models; MONTE-CARLO; STATISTICAL-MODELS;
D O I
10.1002/sta4.625
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
Maximum likelihood (ML) learning for energy-based models (EBMs) is challenging, partly due to nonconvergence of Markov chain Monte Carlo. Several variations of ML learning have been proposed, but existing methods all fail to achieve both posttraining image generation and proper density estimation. We propose to introduce diffusion data and learn a joint EBM, called diffusion-assisted EBMs, through persistent training (i.e. using persistent contrastive divergence) with an enhanced sampling algorithm to properly sample from complex, multimodal distributions. We present results from a 2D illustrative experiment and image experiments and demonstrate that for the first time for image data, persistently trained EBMs can simultaneously achieve long-run stability, post-training image generation and superior out-of-distribution detection.
引用
收藏
页数:14
相关论文
共 50 条
  • [21] A thermodynamically consistent constitutive model for diffusion-assisted plasticity in Ni-based superalloys
    Barba, D.
    Alabort, E.
    Garcia-Gonzalez, D.
    Moverare, J. J.
    Reed, R. C.
    Jerusalem, A.
    INTERNATIONAL JOURNAL OF PLASTICITY, 2018, 105 : 74 - 98
  • [22] Simple super-resolution live-cell imaging based on diffusion-assisted Förster resonance energy transfer
    Sangyeon Cho
    Jaeduck Jang
    Chaeyeon Song
    Heeyoung Lee
    Prabhakar Ganesan
    Tae-Young Yoon
    Mahn Won Kim
    Myung Chul Choi
    Hyotcherl Ihee
    Won Do Heo
    YongKeun Park
    Scientific Reports, 3
  • [23] Model-Based Planning with Energy-Based Models
    Du, Yilun
    Lin, Toru
    Mordatch, Igor
    CONFERENCE ON ROBOT LEARNING, VOL 100, 2019, 100
  • [24] Fractional dynamics approach to diffusion-assisted reactions in disordered media
    Sung, JY
    Barkai, E
    Silbey, RJ
    Lee, S
    JOURNAL OF CHEMICAL PHYSICS, 2002, 116 (06): : 2338 - 2341
  • [25] Energy-based models for sparse overcomplete representations
    Teh, YW
    Welling, M
    Osindero, S
    Hinton, GE
    JOURNAL OF MACHINE LEARNING RESEARCH, 2004, 4 (7-8) : 1235 - 1260
  • [26] Towards understanding retrosynthesis by energy-based models
    Sun, Ruoxi
    Dai, Hanjun
    Li, Li
    Kearnes, Steven
    Dai, Bo
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 34 (NEURIPS 2021), 2021,
  • [27] Rate -Distortion via Energy-Based Models
    Li, Qing
    Kim, Yongjune
    Guyot, Cyril
    2023 DATA COMPRESSION CONFERENCE, DCC, 2023, : 351 - 351
  • [28] LOCAL AND NONLOCAL ENERGY-BASED COUPLING MODELS
    Acosta, Gabriel
    Bersetche, Francisco
    Rossi, Julio D.
    SIAM JOURNAL ON MATHEMATICAL ANALYSIS, 2022, 54 (06) : 6288 - 6322
  • [29] Implicit Generation and Modeling with Energy-Based Models
    Du, Yilun
    Mordatch, Igor
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 32 (NIPS 2019), 2019, 32
  • [30] RNA pseudoknot prediction in energy-based models
    Lyngso, RB
    Pedersen, CNS
    JOURNAL OF COMPUTATIONAL BIOLOGY, 2000, 7 (3-4) : 409 - 427