Bayesian shrinkage in mixture-of-experts models: identifying robust determinants of class membership

被引：2
|
作者：
Zens, Gregor [1 ]
机构：
[1] Vienna Univ Econ & Business, Dept Econ, Welthandelspl 1, A-1020 Vienna, Austria
来源：
ADVANCES IN DATA ANALYSIS AND CLASSIFICATION | 2019年 / 13卷 / 04期
关键词：
Mixture-of-experts; Classification; Shrinkage; Bayesian inference; Normal gamma prior; VARIABLE SELECTION; FINITE MIXTURE; INFERENCE; REGRESSION; DISTRIBUTIONS; INEQUALITY; LIKELIHOOD;
D O I：
10.1007/s11634-019-00353-y
中图分类号：
O21 [概率论与数理统计]; C8 [统计学];
学科分类号：
020208 ; 070103 ; 0714 ;
摘要：
A method for implicit variable selection in mixture-of-experts frameworks is proposed. We introduce a prior structure where information is taken from a set of independent covariates. Robust class membership predictors are identified using a normal gamma prior. The resulting model setup is used in a finite mixture of Bernoulli distributions to find homogenous clusters of women in Mozambique based on their information sources on HIV. Fully Bayesian inference is carried out via the implementation of a Gibbs sampler.
引用
收藏
页码：1019 / 1051
页数：33
相关论文
共 38 条
  • [1] Bayesian shrinkage in mixture-of-experts models: identifying robust determinants of class membership
    Gregor Zens
    [J]. Advances in Data Analysis and Classification, 2019, 13 : 1019 - 1051
  • [2] Asymptotic properties of mixture-of-experts models
    Olteanu, M.
    Rynkiewicz, J.
    [J]. NEUROCOMPUTING, 2011, 74 (09) : 1444 - 1449
  • [3] A similarity-based Bayesian mixture-of-experts model
    Tianfang Zhang
    Rasmus Bokrantz
    Jimmy Olsson
    [J]. Statistics and Computing, 2023, 33
  • [4] A similarity-based Bayesian mixture-of-experts model
    Zhang, Tianfang
    Bokrantz, Rasmus
    Olsson, Jimmy
    [J]. STATISTICS AND COMPUTING, 2023, 33 (04)
  • [5] A Universal Approximation Theorem for Mixture-of-Experts Models
    Nguyen, Hien D.
    Lloyd-Jones, Luke R.
    McLachlan, Geoffrey J.
    [J]. NEURAL COMPUTATION, 2016, 28 (12) : 2585 - 2593
  • [6] GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
    Du, Nan
    Huang, Yanping
    Dai, Andrew M.
    Tong, Simon
    Lepikhin, Dmitry
    Xu, Yuanzhong
    Krikun, Maxim
    Zhou, Yanqi
    Yu, Adams Wei
    Firat, Orhan
    Zoph, Barret
    Fedus, Liam
    Bosma, Maarten
    Zhou, Zongwei
    Wang, Tao
    Wang, Yu Emma
    Webster, Kellie
    Pellat, Marie
    Robinson, Kevin
    Meier-Hellstern, Kathleen
    Duke, Toju
    Dixon, Lucas
    Zhang, Kun
    Le, Quoc V.
    Wu, Yonghui
    Chen, Zhifeng
    Cui, Claire
    [J]. INTERNATIONAL CONFERENCE ON MACHINE LEARNING, VOL 162, 2022,
  • [7] New estimation and feature selection methods in mixture-of-experts models
    Khalili, Abbas
    [J]. CANADIAN JOURNAL OF STATISTICS-REVUE CANADIENNE DE STATISTIQUE, 2010, 38 (04): : 519 - 539
  • [8] Hierarchical mixture-of-experts models for count variables with excessive zeros
    Park, Myung Hyun
    Kim, Joseph H. T.
    [J]. COMMUNICATIONS IN STATISTICS-THEORY AND METHODS, 2022, 51 (12) : 4072 - 4096
  • [9] Adaptive mixture-of-experts models for data glove interface with multiple users
    Yoon, Jong-Won
    Yang, Sung-Ihk
    Cho, Sung-Bae
    [J]. EXPERT SYSTEMS WITH APPLICATIONS, 2012, 39 (05) : 4898 - 4907
  • [10] Janus: A Unified Distributed Training Framework for Sparse Mixture-of-Experts Models
    Liu, Juncai
    Wang, Jessie Hui
    Jiang, Yimin
    [J]. PROCEEDINGS OF THE 2023 ACM SIGCOMM 2023 CONFERENCE, SIGCOMM 2023, 2023, : 486 - 498
1234