Optimal model inference for Bayesian mixture of experts

被引：0

作者：

Ueda, N ^{[1
]}

Ghahramani, Z ^{[1
]}

机构：

[1] NTT, Commun Sci Labs, Kyoto 6190237, Japan

来源：

NEURAL NETWORKS FOR SIGNAL PROCESSING X, VOLS 1 AND 2, PROCEEDINGS | 2000年

关键词：

D O I：

暂无

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

We present an algorithm for inferring the parameter and model structure of a mixture of experts model (MoE) based on the variational Bayesian (VB) framework. First, in the VB framework, we show that the model parameter and structure of a MoE can be simultaneously optimized by maximizing an objective funtion derived in this paper. Next, we present a deterministic algorithm to find the optimal number of experts of a MoE while avoiding local maxima. Our experimental results demonstrate the practical usefulness of the method.

引用

页码：145 / 154

页数：10

共 50 条

[41] RETRACTED: Bayesian Inference of Mixture Model via Differential Evolution and Sampling (Retracted Article)
Guo, Peng
Li, Naixiang
[J]. 2ND IEEE INTERNATIONAL CONFERENCE ON ADVANCED COMPUTER CONTROL (ICACC 2010), VOL. 4, 2010, : 504 - 508
[42] Inference for the Number of Topics in the Latent Dirichlet Allocation Model via Bayesian Mixture Modeling
Chen, Zhe
Doss, Hani
[J]. JOURNAL OF COMPUTATIONAL AND GRAPHICAL STATISTICS, 2019, 28 (03) : 567 - 585
[43] Bayesian Inference for Finite Mixture Regression Model Based on Non-Iterative Algorithm
Shan, Ang
Yang, Fengkai
[J]. MATHEMATICS, 2021, 9 (06)
[44] Mixture of experts classification using a hierarchical mixture model
Titsias, MK
Likas, A
[J]. NEURAL COMPUTATION, 2002, 14 (09) : 2221 - 2244
[45] Nonlinear Time Series Analysis Using Bayesian Mixture of Experts
Baldacchino, Tara
Rowson, Jennifer
Worden, Keith
[J]. NONLINEAR DYNAMICS, VOL 2, 2014, : 111 - 121
[46] Bayesian inference for some mixture problems in quality and reliability
Nair, VN
Tang, B
Yu, LA
[J]. JOURNAL OF QUALITY TECHNOLOGY, 2001, 33 (01) : 16 - 28
[47] Inference of gene pathways using mixture Bayesian networks
Ko, Younhee
Zhai, ChengXiang
Rodriguez-Zas, Sandra
[J]. BMC SYSTEMS BIOLOGY, 2009, 3
[48] Streaming Variational Inference for Bayesian Nonparametric Mixture Models
Tank, Alex
Foti, Nicholas J.
Fox, Emily B.
[J]. ARTIFICIAL INTELLIGENCE AND STATISTICS, VOL 38, 2015, 38 : 968 - 976
[49] Bayesian Inference for the Loss Models via Mixture Priors
Deng, Min
Aminzadeh, Mostafa S.
[J]. RISKS, 2023, 11 (09)
[50] Bayesian Estimation of Beta Mixture Models with Variational Inference
Ma, Zhanyu
Leijon, Arne
[J]. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2011, 33 (11) : 2160 - 2173

← 1 2 3 4 5 →