Not too little, not too much: a theoretical analysis of graph (over)smoothing

被引:0
|
作者
Keriven, Nicolas [1 ]
机构
[1] CNRS, GIPSA Lab, Grenoble, France
关键词
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
We analyze graph smoothing with mean aggregation, where each node successively receives the average of the features of its neighbors. Indeed, it has quickly been observed that Graph Neural Networks (GNNs), which generally follow some variant of Message-Passing (MP) with repeated aggregation, may be subject to the oversmoothing phenomenon: by performing too many rounds of MP, the node features tend to converge to a non-informative limit. In the case of mean aggregation, for connected graphs, the node features become constant across the whole graph. At the other end of the spectrum, it is intuitively obvious that some MP rounds are necessary, but existing analyses do not exhibit both phenomena at once: beneficial "finite" smoothing and oversmoothing in the limit. In this paper, we consider simplified linear GNNs, and rigorously analyze two examples for which a finite number of mean aggregation steps provably improves the learning performance, before oversmoothing kicks in. We consider a latent space random graph model, where node features are partial observations of the latent variables and the graph contains pairwise relationships between them. We show that graph smoothing restores some of the lost information, up to a certain point, by two phenomena: graph smoothing shrinks non-principal directions in the data faster than principal ones, which is useful for regression, and shrinks nodes within communities faster than they collapse together, which improves classification.
引用
收藏
页数:14
相关论文
共 50 条
  • [21] Too little mathematics - And too much
    Wilson, EB
    SCIENCE, 1928, 67 : 52 - 59
  • [22] Thrombopoietin: too much or too little
    Newland, Adrian
    HEMATOLOGY, 2012, 17 : S166 - S168
  • [23] Too little or too much are inadequate
    Pichard, Claude
    Attaix, Didier
    Baracos, Vickie E.
    CURRENT OPINION IN CLINICAL NUTRITION AND METABOLIC CARE, 2014, 17 (03): : 211 - 212
  • [24] TOO LITTLE WATER - AND TOO MUCH
    不详
    ECONOMIST, 1965, 215 (13): : 1523 - 1523
  • [25] WATER - TOO LITTLE AND TOO MUCH
    不详
    NATION, 1965, 201 (05) : 91 - 91
  • [26] Too much medicine? Or too little?
    Melton, LJ
    Heaney, RP
    BONE, 2003, 32 (04) : 327 - 331
  • [27] TOO MUCH MATHS, OR TOO LITTLE
    THOMPSON, D
    CROWTHER, K
    PHYSICS WORLD, 1993, 6 (09) : 21 - 21
  • [28] Too little Fruit, too little Wholegrain and too much Salt
    Meinrenken, Susanne
    DEUTSCHE MEDIZINISCHE WOCHENSCHRIFT, 2019, 144 (16) : 1104 - +
  • [29] How much responsiveness is too much or too little?
    Mulgan, Richard
    AUSTRALIAN JOURNAL OF PUBLIC ADMINISTRATION, 2008, 67 (03) : 345 - 356
  • [30] Failure mode and effects analysis: too little for too much?
    Franklin, Bryony Dean
    Shebl, Nada Atef
    Barber, Nick
    BMJ QUALITY & SAFETY, 2012, 21 (07) : 607 - 611