Data-Driven Offline Decision-Making via Invariant Representation Learning

被引:0
|
作者
Qi, Han [1 ]
Su, Yi [1 ]
Kumar, Aviral [1 ]
Levine, Sergey [1 ]
机构
[1] Univ Calif Berkeley, Dept Elect Engn & Comp Sci, Berkeley, CA 94720 USA
关键词
MODEL;
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
The goal in offline data-driven decision-making is synthesize decisions that optimize a black-box utility function, using a previously-collected static dataset, with no active interaction. These problems appear in many forms: offline reinforcement learning (RL), where we must produce actions that optimize the long-term reward, bandits from logged data, where the goal is to determine the correct arm, and offline model-based optimization (MBO) problems, where we must find the optimal design provided access to only a static dataset. A key challenge in all these settings is distributional shift: when we optimize with respect to the input into a model trained from offline data, it is easy to produce an out-of-distribution (OOD) input that appears erroneously good. In contrast to prior approaches that utilize pessimism or conservatism to tackle this problem, in this paper, we formulate offline data-driven decision-making as domain adaptation, where the goal is to make accurate predictions for the value of optimized decisions ("target domain"), when training only on the dataset ("source domain"). This perspective leads to invariant objective models (IOM), our approach for addressing distributional shift by enforcing invariance between the learned representations of the training dataset and optimized decisions. In IOM, if the optimized decisions are too different from the training dataset, the representation will be forced to lose much of the information that distinguishes good designs from bad ones, making all choices seem mediocre. Critically, when the optimizer is aware of this representational tradeoff, it should choose not to stray too far from the training distribution, leading to a natural trade-off between distributional shift and learning performance.
引用
收藏
页数:12
相关论文
共 50 条
  • [41] 4D: DEVELOPING DASHBOARDS FOR DATA-DRIVEN DECISION-MAKING
    O'Donnell, C.
    Murphy, B.
    Hunter, B.
    11TH INTERNATIONAL CONFERENCE OF EDUCATION, RESEARCH AND INNOVATION (ICERI2018), 2018, : 8421 - 8425
  • [42] Data-driven decision-making with weights and reliabilities for diagnosis of thyroid cancer
    Min Xue
    Peipei Cao
    Bingbing Hou
    Weiyong Liu
    International Journal of Machine Learning and Cybernetics, 2022, 13 : 2257 - 2271
  • [43] Artificial Intelligence for data-driven decision-making and governance in public affairs
    Charles, Vincent
    Rana, Nripendra P.
    Carter, Lemuria
    GOVERNMENT INFORMATION QUARTERLY, 2022, 39 (04)
  • [44] Data-Driven Decision-Making in COVID-19 Response: A Survey
    Yu, Shuo
    Qing, Qing
    Zhang, Chen
    Shehzad, Ahsan
    Oatley, Giles
    Xia, Feng
    IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS, 2021, 8 (04) : 1016 - 1029
  • [45] Data-Driven Decision-Making in Cyber-Physical Integrated Society
    Sonehara, Noboru
    Suzuki, Takahisa
    Kodate, Akihisa
    Wakahara, Toshihiko
    Sakai, Yoshinori
    Ichifuji, Yu
    Fujii, Hideo
    Yoshii, Hideki
    IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, 2019, E102D (09): : 1607 - 1616
  • [46] Integrating expertise and parametric analysis for a data-driven decision-making practice
    Bernal, Marcelo
    Okhoya, Victor
    Marshall, Tyrone
    Chen, Cheney
    Haymaker, John
    INTERNATIONAL JOURNAL OF ARCHITECTURAL COMPUTING, 2020, 18 (04) : 424 - 440
  • [47] The role of optimization in some recent advances in data-driven decision-making
    Lennart Baardman
    Rares Cristian
    Georgia Perakis
    Divya Singhvi
    Omar Skali Lami
    Leann Thayaparan
    Mathematical Programming, 2023, 200 : 1 - 35
  • [48] Marketing analytics in 2024 conferences: AI and data-driven decision-making
    Maria Petrescu
    Anjala S. Krishen
    Journal of Marketing Analytics, 2024, 12 (4) : 743 - 745
  • [49] Critical review of data-driven decision-making in bridge operation and maintenance
    Wu, Chengke
    Wu, Peng
    Wang, Jun
    Jiang, Rui
    Chen, Mengcheng
    Wang, Xiangyu
    STRUCTURE AND INFRASTRUCTURE ENGINEERING, 2022, 18 (01) : 47 - 70
  • [50] The role of optimization in some recent advances in data-driven decision-making
    Baardman, Lennart
    Cristian, Rares
    Perakis, Georgia
    Singhvi, Divya
    Lami, Omar Skali
    Thayaparan, Leann
    MATHEMATICAL PROGRAMMING, 2023, 200 (01) : 1 - 35