Optimal model inference for Bayesian mixture of experts

被引：0
|
作者：
Ueda, N [1 ]
Ghahramani, Z [1 ]
机构：
[1] NTT, Commun Sci Labs, Kyoto 6190237, Japan
来源：
NEURAL NETWORKS FOR SIGNAL PROCESSING X, VOLS 1 AND 2, PROCEEDINGS | 2000年
关键词：
D O I：
暂无
中图分类号：
TP18 [人工智能理论];
学科分类号：
081104 ; 0812 ; 0835 ; 1405 ;
摘要：
We present an algorithm for inferring the parameter and model structure of a mixture of experts model (MoE) based on the variational Bayesian (VB) framework. First, in the VB framework, we show that the model parameter and structure of a MoE can be simultaneously optimized by maximizing an objective funtion derived in this paper. Next, we present a deterministic algorithm to find the optimal number of experts of a MoE while avoiding local maxima. Our experimental results demonstrate the practical usefulness of the method.
引用
收藏
页码：145 / 154
页数：10
相关论文
共 50 条
  • [1] Optimal model inference for Bayesian mixture of experts
    Ueda, Naonori
    Ghahramani, Zoubin
    [J]. Neural Networks for Signal Processing - Proceedings of the IEEE Workshop, 2000, 1 : 145 - 154
  • [2] Model Selection of Bayesian Hierarchical Mixture of Experts based on Variational Inference
    Iikubo, Yuji
    Horii, Shunsuke
    Matsushima, Toshiyasu
    [J]. 2019 IEEE INTERNATIONAL CONFERENCE ON SYSTEMS, MAN AND CYBERNETICS (SMC), 2019, : 3474 - 3479
  • [3] Sparse Bayesian Hierarchical Mixture of Experts and Variational Inference
    Iikubo, Yuji
    Horii, Shunsuke
    Matsushima, Toshiyasu
    [J]. PROCEEDINGS OF 2018 INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY AND ITS APPLICATIONS (ISITA2018), 2018, : 60 - 64
  • [4] Robust Bayesian inference for the censored mixture of experts model using heavy-tailed distributions
    Mirfarah, Elham
    Naderi, Mehrdad
    Lin, Tsung-, I
    Wang, Wan-Lun
    [J]. ADVANCES IN DATA ANALYSIS AND CLASSIFICATION, 2024,
  • [5] Bayesian Inference on a Mixture Model With Spatial Dependence
    Cucala, Lionel
    Marin, Jean-Michel
    [J]. JOURNAL OF COMPUTATIONAL AND GRAPHICAL STATISTICS, 2013, 22 (03) : 584 - 597
  • [6] Bayesian inference for a mixture double autoregressive model
    Yang, Kai
    Zhang, Qingqing
    Yu, Xinyang
    Dong, Xiaogang
    [J]. STATISTICA NEERLANDICA, 2023, 77 (02) : 188 - 207
  • [7] Bayesian estimation of Dirichlet mixture model with variational inference
    Ma, Zhanyu
    Rana, Pravin Kumar
    Taghia, Jalil
    Flierl, Markus
    Leijon, Arne
    [J]. PATTERN RECOGNITION, 2014, 47 (09) : 3143 - 3157
  • [8] A similarity-based Bayesian mixture-of-experts model
    Tianfang Zhang
    Rasmus Bokrantz
    Jimmy Olsson
    [J]. Statistics and Computing, 2023, 33
  • [9] A similarity-based Bayesian mixture-of-experts model
    Zhang, Tianfang
    Bokrantz, Rasmus
    Olsson, Jimmy
    [J]. STATISTICS AND COMPUTING, 2023, 33 (04)
  • [10] SPARSE BAYESIAN HIERARCHICAL MIXTURE OF EXPERTS
    Mossavat, Iman
    Amft, Oliver
    [J]. 2011 IEEE STATISTICAL SIGNAL PROCESSING WORKSHOP (SSP), 2011, : 653 - 656
12345