Optimal model inference for Bayesian mixture of experts

被引：0
|
作者：
Ueda, N [1 ]
Ghahramani, Z [1 ]
机构：
[1] NTT, Commun Sci Labs, Kyoto 6190237, Japan
来源：
NEURAL NETWORKS FOR SIGNAL PROCESSING X, VOLS 1 AND 2, PROCEEDINGS | 2000年
关键词：
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
We present an algorithm for inferring the parameter and model structure of a mixture of experts model (MoE) based on the variational Bayesian (VB) framework. First, in the VB framework, we show that the model parameter and structure of a MoE can be simultaneously optimized by maximizing an objective funtion derived in this paper. Next, we present a deterministic algorithm to find the optimal number of experts of a MoE while avoiding local maxima. Our experimental results demonstrate the practical usefulness of the method.
引用
收藏
页码：145 / 154
页数：10
相关论文
共 50 条
  • [31] Efficient Bayesian inference for dynamic mixture models
    Gerlach, R
    Carter, C
    Kohn, R
    [J]. JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION, 2000, 95 (451) : 819 - 828
  • [32] Optimal transport and variational Bayesian inference
    Bahraini, Alireza
    Sadeghi, Saeed
    [J]. INTERNATIONAL JOURNAL OF APPROXIMATE REASONING, 2023, 162
  • [33] An optimal approximation algorithm for Bayesian inference
    Dagum, P
    Luby, M
    [J]. ARTIFICIAL INTELLIGENCE, 1997, 93 (1-2) : 1 - 27
  • [34] Optimal approximation algorithm for Bayesian inference
    Stanford Univ Sch of Medicine, Stanford, United States
    [J]. Artif Intell, 1-2 (1-27):
  • [35] Exploiting Inter-Layer Expert Affinity for Accelerating Mixture-of-Experts Model Inference
    Yao, Jinghan
    Anthony, Quentin
    Shafi, Aamir
    Subramoni, Hari
    Panda, Dhabaleswar K.
    [J]. PROCEEDINGS 2024 IEEE INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM, IPDPS 2024, 2024, : 915 - 925
  • [36] A mixture-of-experts approach for gene regulatory network inference
    Shao, Borong
    Lavesson, Niklas
    Boeva, Veselka
    Shahzad, Raja Khurram
    [J]. INTERNATIONAL JOURNAL OF DATA MINING AND BIOINFORMATICS, 2016, 14 (03) : 258 - 275
  • [37] Non-parametric Bayesian inference for continuous density hidden Markov mixture model
    Bathaee, Najmeh
    Sheikhzadeh, Hamid
    [J]. STATISTICAL METHODOLOGY, 2016, 33 : 256 - 275
  • [38] Markov chain Monte Carlo simulation of a Bayesian mixture model for gene network inference
    Ko, Younhee
    Kim, Jaebum
    Rodriguez-Zas, Sandra L.
    [J]. GENES & GENOMICS, 2019, 41 (05) : 547 - 555
  • [39] Markov chain Monte Carlo simulation of a Bayesian mixture model for gene network inference
    Younhee Ko
    Jaebum Kim
    Sandra L. Rodriguez-Zas
    [J]. Genes & Genomics, 2019, 41 : 547 - 555
  • [40] Optimal inference with suboptimal models: Addiction and active Bayesian inference
    Schwartenbeck, Philipp
    FitzGerald, Thomas H. B.
    Mathys, Christoph
    Dolan, Ray
    Wurst, Friedrich
    Kronbichler, Martin
    Friston, Karl
    [J]. MEDICAL HYPOTHESES, 2015, 84 (02) : 109 - 117
12345