Data-Driven Offline Decision-Making via Invariant Representation Learning

被引:0
|
作者
Qi, Han [1 ]
Su, Yi [1 ]
Kumar, Aviral [1 ]
Levine, Sergey [1 ]
机构
[1] Univ Calif Berkeley, Dept Elect Engn & Comp Sci, Berkeley, CA 94720 USA
关键词
MODEL;
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
The goal in offline data-driven decision-making is synthesize decisions that optimize a black-box utility function, using a previously-collected static dataset, with no active interaction. These problems appear in many forms: offline reinforcement learning (RL), where we must produce actions that optimize the long-term reward, bandits from logged data, where the goal is to determine the correct arm, and offline model-based optimization (MBO) problems, where we must find the optimal design provided access to only a static dataset. A key challenge in all these settings is distributional shift: when we optimize with respect to the input into a model trained from offline data, it is easy to produce an out-of-distribution (OOD) input that appears erroneously good. In contrast to prior approaches that utilize pessimism or conservatism to tackle this problem, in this paper, we formulate offline data-driven decision-making as domain adaptation, where the goal is to make accurate predictions for the value of optimized decisions ("target domain"), when training only on the dataset ("source domain"). This perspective leads to invariant objective models (IOM), our approach for addressing distributional shift by enforcing invariance between the learned representations of the training dataset and optimized decisions. In IOM, if the optimized decisions are too different from the training dataset, the representation will be forced to lose much of the information that distinguishes good designs from bad ones, making all choices seem mediocre. Critically, when the optimizer is aware of this representational tradeoff, it should choose not to stray too far from the training distribution, leading to a natural trade-off between distributional shift and learning performance.
引用
收藏
页数:12
相关论文
共 50 条
  • [41] Critical review of data-driven decision-making in bridge operation and maintenance
    Wu, Chengke
    Wu, Peng
    Wang, Jun
    Jiang, Rui
    Chen, Mengcheng
    Wang, Xiangyu
    [J]. STRUCTURE AND INFRASTRUCTURE ENGINEERING, 2022, 18 (01) : 47 - 70
  • [42] The role of optimization in some recent advances in data-driven decision-making
    Lennart Baardman
    Rares Cristian
    Georgia Perakis
    Divya Singhvi
    Omar Skali Lami
    Leann Thayaparan
    [J]. Mathematical Programming, 2023, 200 : 1 - 35
  • [43] Data-Driven Modeling of Evacuation Decision-Making in Extreme Weather Events
    Hancock, Matthew
    Halim, Nafisa
    Kuhlman, Chris J.
    Marathe, Achla
    Mozumder, Pallab
    Ravi, S. S.
    Vullikanti, Anil
    [J]. COMPLEX NETWORKS & THEIR APPLICATIONS X, VOL 1, 2022, 1015 : 681 - 692
  • [44] Data-Driven Decision-Making in COVID-19 Response: A Survey
    Yu, Shuo
    Qing, Qing
    Zhang, Chen
    Shehzad, Ahsan
    Oatley, Giles
    Xia, Feng
    [J]. IEEE TRANSACTIONS ON COMPUTATIONAL SOCIAL SYSTEMS, 2021, 8 (04) : 1016 - 1029
  • [45] Artificial Intelligence for data-driven decision-making and governance in public affairs
    Charles, Vincent
    Rana, Nripendra P.
    Carter, Lemuria
    [J]. GOVERNMENT INFORMATION QUARTERLY, 2022, 39 (04)
  • [46] 4D: DEVELOPING DASHBOARDS FOR DATA-DRIVEN DECISION-MAKING
    O'Donnell, C.
    Murphy, B.
    Hunter, B.
    [J]. 11TH INTERNATIONAL CONFERENCE OF EDUCATION, RESEARCH AND INNOVATION (ICERI2018), 2018, : 8421 - 8425
  • [47] Data-driven decision-making with weights and reliabilities for diagnosis of thyroid cancer
    Min Xue
    Peipei Cao
    Bingbing Hou
    Weiyong Liu
    [J]. International Journal of Machine Learning and Cybernetics, 2022, 13 : 2257 - 2271
  • [48] Integrating expertise and parametric analysis for a data-driven decision-making practice
    Bernal, Marcelo
    Okhoya, Victor
    Marshall, Tyrone
    Chen, Cheney
    Haymaker, John
    [J]. INTERNATIONAL JOURNAL OF ARCHITECTURAL COMPUTING, 2020, 18 (04) : 424 - 440
  • [49] Data-Driven Decision-Making in Cyber-Physical Integrated Society
    Sonehara, Noboru
    Suzuki, Takahisa
    Kodate, Akihisa
    Wakahara, Toshihiko
    Sakai, Yoshinori
    Ichifuji, Yu
    Fujii, Hideo
    Yoshii, Hideki
    [J]. IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, 2019, E102D (09): : 1607 - 1616
  • [50] Data-driven decision-making with weights and reliabilities for diagnosis of thyroid cancer
    Xue, Min
    Cao, Peipei
    Hou, Bingbing
    Liu, Weiyong
    [J]. INTERNATIONAL JOURNAL OF MACHINE LEARNING AND CYBERNETICS, 2022, 13 (08) : 2257 - 2271