Data-Driven Offline Decision-Making via Invariant Representation Learning

被引:0
|
作者
Qi, Han [1 ]
Su, Yi [1 ]
Kumar, Aviral [1 ]
Levine, Sergey [1 ]
机构
[1] Univ Calif Berkeley, Dept Elect Engn & Comp Sci, Berkeley, CA 94720 USA
关键词
MODEL;
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
The goal in offline data-driven decision-making is synthesize decisions that optimize a black-box utility function, using a previously-collected static dataset, with no active interaction. These problems appear in many forms: offline reinforcement learning (RL), where we must produce actions that optimize the long-term reward, bandits from logged data, where the goal is to determine the correct arm, and offline model-based optimization (MBO) problems, where we must find the optimal design provided access to only a static dataset. A key challenge in all these settings is distributional shift: when we optimize with respect to the input into a model trained from offline data, it is easy to produce an out-of-distribution (OOD) input that appears erroneously good. In contrast to prior approaches that utilize pessimism or conservatism to tackle this problem, in this paper, we formulate offline data-driven decision-making as domain adaptation, where the goal is to make accurate predictions for the value of optimized decisions ("target domain"), when training only on the dataset ("source domain"). This perspective leads to invariant objective models (IOM), our approach for addressing distributional shift by enforcing invariance between the learned representations of the training dataset and optimized decisions. In IOM, if the optimized decisions are too different from the training dataset, the representation will be forced to lose much of the information that distinguishes good designs from bad ones, making all choices seem mediocre. Critically, when the optimizer is aware of this representational tradeoff, it should choose not to stray too far from the training distribution, leading to a natural trade-off between distributional shift and learning performance.
引用
收藏
页数:12
相关论文
共 50 条
  • [21] Data-driven Decision-making Strategies for Electricity Retailers: A Deep Reinforcement Learning Approach
    Liu, Yuankun
    Zhang, Dongxia
    Gooi, Hoay Beng
    [J]. CSEE JOURNAL OF POWER AND ENERGY SYSTEMS, 2021, 7 (02): : 358 - 367
  • [22] Data-Driven Decision Making
    Jose Divan, Mario
    [J]. 2017 INTERNATIONAL CONFERENCE ON INFOCOM TECHNOLOGIES AND UNMANNED SYSTEMS (TRENDS AND FUTURE DIRECTIONS) (ICTUS), 2017, : 50 - 56
  • [23] BARRIERS TO DATA-DRIVEN DECISION-MAKING AMONG ONLINE RETAILERS
    Kemppainen, Tiina
    Frank, Lauri
    Makkonen, Markus
    Kallio, Antti
    [J]. 35TH BLED ECONFERENCE DIGITAL RESTRUCTURING AND HUMAN (RE)ACTION, BLED ECONFERENCE 2022, 2022, : 327 - 342
  • [24] Data-Driven Decision-Making in Support of Managing Pathology Laboratories
    Dahl, Julia
    Myers, Jeffrey L.
    Pantanowitz, Liron
    [J]. AJSP-REVIEWS AND REPORTS, 2022, 27 (04) : 158 - 163
  • [25] Data-Driven Decision-Making in Product R&D
    Fabijan, Aleksander
    Olsson, Helena Holmstrom
    Bosch, Jan
    [J]. AGILE PROCESSES, IN SOFTWARE ENGINEERING, AND EXTREME PROGRAMMING, XP 2015, 2015, 212 : 350 - 351
  • [26] THE IMPLICATIONS OF INTEGRATING ARTIFICIAL INTELLIGENCE INTO DATA-DRIVEN DECISION-MAKING
    Sutherns, Jessica
    Fanta, Getnet B.
    [J]. South African Journal of Industrial Engineering, 2024, 35 (03) : 195 - 207
  • [27] Data-driven decision-making challenges of local government in Indonesia
    Sayogo, Djoko Sigit
    Yuli, Sri Budi Cantika
    Amalia, Firda Ayu
    [J]. TRANSFORMING GOVERNMENT- PEOPLE PROCESS AND POLICY, 2024, 18 (01) : 145 - 156
  • [28] A data-driven approach to shared decision-making in a healthcare environment
    Singh, Sudhanshu
    Verma, Rakesh
    Koul, Saroj
    [J]. OPSEARCH, 2022, 59 (02) : 732 - 746
  • [29] Data-driven decision-making for precision diagnosis of digestive diseases
    Jiang, Song
    Wang, Ting
    Zhang, Kun-He
    [J]. BIOMEDICAL ENGINEERING ONLINE, 2023, 22 (01)
  • [30] Data-Driven Decision-Making Process: The Case of Polish Organizations
    Palonka, Joanna
    Begovic, Din
    [J]. PROCEEDINGS OF THE 13TH INTERNATIONAL CONFERENCE ON INTELLECTUAL CAPITAL KNOWLEDGE MANAGEMENT & ORGANISATIONAL LEARNING (ICICKM 2016), 2016, : 216 - 224