Selecting informative conformal prediction sets with false coverage rate control

被引:0
|
作者
Gazin, Ulysse [1 ,2 ]
Heller, Ruth [3 ]
Marandon, Ariane [4 ]
Roquain, Etienne [5 ,6 ]
机构
[1] Univ Paris Cite, F-75013 Paris, France
[2] Sorbonne Univ, CNRS, Lab Probabil Stat & Modelisat, F-75013 Paris, France
[3] Tel Aviv Univ, Dept Stat & Operat Res, Tel Aviv, Israel
[4] Alan Turing Inst, London, England
[5] Sorbonne Univ, F-75005 Paris, France
[6] Univ Paris Cite, CNRS, Lab Probabil Stat & Modelisat, F-75005 Paris, France
关键词
classification; false discovery rate; label shift; prediction interval; regression; selective inference; DISCOVERY RATE;
D O I
10.1093/jrsssb/qkae120
中图分类号
O21 [概率论与数理统计]; C8 [统计学];
学科分类号
020208 ; 070103 ; 0714 ;
摘要
In supervised learning, including regression and classification, conformal methods provide prediction sets for the outcome/label with finite sample coverage for any machine learning predictor. We consider here the case where such prediction sets come after a selection process. The selection process requires that the selected prediction sets be 'informative' in a well-defined sense. We consider both the classification and regression settings where the analyst may consider as informative only the sample with prediction sets small enough, excluding null values, or obeying other appropriate 'monotone' constraints. We develop a unified framework for building such informative conformal prediction sets while controlling the false coverage rate (FCR) on the selected sample. While conformal prediction sets after selection have been the focus of much recent literature in the field, the new introduced procedures, called InfoSP and InfoSCOP, are to our knowledge the first ones providing FCR control for informative prediction sets. We show the usefulness of our resulting procedures on real and simulated data.
引用
收藏
页数:21
相关论文
共 50 条
  • [21] Evidential Uncertainty Sets in Deep Classifiers Using Conformal Prediction
    Karimi, Hamed
    Samavi, Reza
    13TH SYMPOSIUM ON CONFORMAL AND PROBABILISTIC PREDICTION WITH APPLICATIONS, 2024, 230 : 466 - 489
  • [22] Applying Conformal Prediction to control an exoskeleton
    Eliades, Charalambos
    Papadopoulos, Harris
    CONFORMAL AND PROBABILISTIC PREDICTION AND APPLICATIONS, VOL 105, 2019, 105
  • [23] NESTED CONFORMAL PREDICTION SETS FOR CLASSIFICATION WITH APPLICATIONS TO PROBATION DATA
    Kuchibhotla, Arun K.
    Berk, Richard A.
    ANNALS OF APPLIED STATISTICS, 2023, 17 (01): : 761 - 785
  • [24] False Discovery Rate Control With Groups
    Hu, James X.
    Zhao, Hongyu
    Zhou, Harrison H.
    JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION, 2010, 105 (491) : 1215 - 1227
  • [25] FEATURE SELECTION IN OMICS PREDICTION PROBLEMS USING CAT SCORES AND FALSE NONDISCOVERY RATE CONTROL
    Ahdesmaeki, Miika
    Strimmer, Korbinian
    ANNALS OF APPLIED STATISTICS, 2010, 4 (01): : 503 - 519
  • [26] Using expert knowledge with control of false discovery rate to select regressors for prediction of soil properties
    Lark, R. M.
    Bishop, T. F. A.
    Webster, R.
    GEODERMA, 2007, 138 (1-2) : 65 - 78
  • [27] Leveraging conformal prediction to annotate enzyme function space with limited false positives
    Ding, Kerr
    Luo, Jiaqi
    Luo, Yunan
    PLOS COMPUTATIONAL BIOLOGY, 2024, 20 (05)
  • [28] Conformal PID Control for Time Series Prediction
    Angelopoulos, Anastasios N.
    Candes, Emmanuel J.
    Tibshirani, Ryan J.
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 36 (NEURIPS 2023), 2023,
  • [29] ONLINE RULES FOR CONTROL OF FALSE DISCOVERY RATE AND FALSE DISCOVERY EXCEEDANCE
    Javanmard, Adel
    Montanari, Andrea
    ANNALS OF STATISTICS, 2018, 46 (02): : 526 - 554
  • [30] Optimal weighting for false discovery rate control
    Roquain, Etienne
    van de Wiel, Mark A.
    ELECTRONIC JOURNAL OF STATISTICS, 2009, 3 : 678 - 711