Approximation and sampling of multivariate probability distributions in the tensor train decomposition

被引:0
|
作者
Sergey Dolgov
Karim Anaya-Izquierdo
Colin Fox
Robert Scheichl
机构
[1] University of Bath,
[2] University of Otago,undefined
[3] University of Heidelberg,undefined
来源
Statistics and Computing | 2020年 / 30卷
关键词
Multivariate distributions; Surrogate models; Tensor decomposition; MCMC; Importance weights;
D O I
暂无
中图分类号
学科分类号
摘要
General multivariate distributions are notoriously expensive to sample from, particularly the high-dimensional posterior distributions in PDE-constrained inverse problems. This paper develops a sampler for arbitrary continuous multivariate distributions that is based on low-rank surrogates in the tensor train format, a methodology that has been exploited for many years for scalable, high-dimensional density function approximation in quantum physics and chemistry. We build upon recent developments of the cross approximation algorithms in linear algebra to construct a tensor train approximation to the target probability density function using a small number of function evaluations. For sufficiently smooth distributions, the storage required for accurate tensor train approximations is moderate, scaling linearly with dimension. In turn, the structure of the tensor train surrogate allows sampling by an efficient conditional distribution method since marginal distributions are computable with linear complexity in dimension. Expected values of non-smooth quantities of interest, with respect to the surrogate distribution, can be estimated using transformed independent uniformly-random seeds that provide Monte Carlo quadrature or transformed points from a quasi-Monte Carlo lattice to give more efficient quasi-Monte Carlo quadrature. Unbiased estimates may be calculated by correcting the transformed random seeds using a Metropolis–Hastings accept/reject step, while the quasi-Monte Carlo quadrature may be corrected either by a control-variate strategy or by importance weighting. We show that the error in the tensor train approximation propagates linearly into the Metropolis–Hastings rejection rate and the integrated autocorrelation time of the resulting Markov chain; thus, the integrated autocorrelation time may be made arbitrarily close to 1, implying that, asymptotic in sample size, the cost per effectively independent sample is one target density evaluation plus the cheap tensor train surrogate proposal that has linear cost with dimension. These methods are demonstrated in three computed examples: fitting failure time of shock absorbers; a PDE-constrained inverse diffusion problem; and sampling from the Rosenbrock distribution. The delayed rejection adaptive Metropolis (DRAM) algorithm is used as a benchmark. In all computed examples, the importance weight-corrected quasi-Monte Carlo quadrature performs best and is more efficient than DRAM by orders of magnitude across a wide range of approximation accuracies and sample sizes. Indeed, all the methods developed here significantly outperform DRAM in all computed examples.
引用
收藏
页码:603 / 625
页数:22
相关论文
共 50 条
  • [1] Approximation and sampling of multivariate probability distributions in the tensor train decomposition
    Dolgov, Sergey
    Anaya-Izquierdo, Karim
    Fox, Colin
    Scheichl, Robert
    STATISTICS AND COMPUTING, 2020, 30 (03) : 603 - 625
  • [2] On fusion of probability density functions using tensor train decomposition
    Ajgl, Jiri
    Straka, Ondrej
    2024 27TH INTERNATIONAL CONFERENCE ON INFORMATION FUSION, FUSION 2024, 2024,
  • [3] BLACK BOX APPROXIMATION IN THE TENSOR TRAIN FORMAT INITIALIZED BY ANOVA DECOMPOSITION
    Chertkov, Andrei
    Ryzhakov, Gleb
    Oseledets, Ivan
    SIAM JOURNAL ON SCIENTIFIC COMPUTING, 2023, 45 (04): : A2101 - A2118
  • [4] STREAMING TENSOR TRAIN APPROXIMATION
    Kressner, Daniel
    Vandereycken, Bart
    Voorhaar, Rik
    SIAM JOURNAL ON SCIENTIFIC COMPUTING, 2023, 45 (05): : A2610 - A2631
  • [5] Probabilistic Tensor Train Decomposition
    Hinrich, Jesper L.
    Morup, Morten
    2019 27TH EUROPEAN SIGNAL PROCESSING CONFERENCE (EUSIPCO), 2019,
  • [6] TENSOR-TRAIN DECOMPOSITION
    Oseledets, I. V.
    SIAM JOURNAL ON SCIENTIFIC COMPUTING, 2011, 33 (05): : 2295 - 2317
  • [7] Approximation of Unknown Multivariate Probability Distributions by Using Mixtures of Product Components: A Tutorial
    Grim, Jiri
    INTERNATIONAL JOURNAL OF PATTERN RECOGNITION AND ARTIFICIAL INTELLIGENCE, 2017, 31 (09)
  • [8] SPECTRAL TENSOR-TRAIN DECOMPOSITION
    Bigoni, Daniele
    Engsig-Karup, Allan P.
    Marzouk, Youssef M.
    SIAM JOURNAL ON SCIENTIFIC COMPUTING, 2016, 38 (04): : A2405 - A2439
  • [9] GRAPH REGULARIZED TENSOR TRAIN DECOMPOSITION
    Sofuoglu, Seyyid Emre
    Aviyente, Selin
    2020 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, 2020, : 3912 - 3916
  • [10] AN INCREMENTAL TENSOR TRAIN DECOMPOSITION ALGORITHM
    Aksoy, Doruk
    Gorsich, David J.
    Veerapaneni, Shravan
    Gorodetsky, Alex A.
    SIAM JOURNAL ON SCIENTIFIC COMPUTING, 2024, 46 (02): : A1047 - A1075