Multiuser, multimodal sensemaking cognitive immersive environment with a task-oriented dialog system

被引:0
|
作者
Briggs, Shannon [1 ]
Chabot, Sam [1 ]
Sanders, Abraham [2 ]
Peveler, Matthew [2 ]
Strzalkowski, Tomek [2 ]
Braasch, Jonas [1 ]
机构
[1] Rensselaer Polytech Inst, Cognit Immers Syst Lab, Troy, NY 12181 USA
[2] Rensselaer Polytech Inst, Comp Sci, Troy, NY USA
关键词
immersive interactive systems progressive dialog agent; sensemaking;
D O I
10.1109/HST56032.2022.10025454
中图分类号
TP39 [计算机的应用];
学科分类号
081203 ; 0835 ;
摘要
This paper is a conceptual paper that explores how the sensemaking process by intelligence analysts completed within a cognitive immersive environment might be impacted by the inclusion of a progressive dialog system. The tools enabled in the sensemaking room (a specific instance within the cognitive immersive environment) were informed by tools from the intelligence analysis domain. We explore how a progressive dialog system would impact the use of tools such as the collaborative brainstorming exercise [1]. These structured analytic techniques are well established in intelligence analysis training literature, and act as ways to access the intended users' cognitive schema as they use the cognitive immersive room and move through the sensemaking process. A prior user study determined that the sensemaking room encouraged users to be more concise and representative with information while using the digital brainstorming tool. We anticipate that addition of the progressive dialog function will enable a more cohesive link between information foraging and sensemaking behaviors for analysts.
引用
下载
收藏
页数:3
相关论文
共 50 条
  • [1] SIMMC 2.0: A Task-oriented Dialog Dataset for Immersive Multimodal Conversations
    Kottur, Satwik
    Moon, Seungwhan
    Geramifard, Alborz
    Damavandi, Babak
    2021 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP 2021), 2021, : 4903 - 4912
  • [2] SIMMC-VR: A Task-oriented Multimodal Dialog Dataset with Situated and Immersive VR Streams
    Wu, Te-Lin
    Kottur, Satwik
    Madotto, Andrea
    Azab, Mahmoud
    Rodriguez, Pedro
    Damavandi, Babak
    Peng, Nanyun
    Moon, Seungwhan
    PROCEEDINGS OF THE 61ST ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, ACL 2023, VOL 1, 2023, : 6273 - 6291
  • [3] Incremental Dialog Processing in a Task-Oriented Dialog
    Ghigi, Fabrizio
    Eskenazi, Maxine
    Ines Torres, M.
    Lee, Sungjin
    15TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION (INTERSPEECH 2014), VOLS 1-4, 2014, : 308 - 312
  • [4] Multimodal Hierarchical Reinforcement Learning Policy for Task-Oriented Visual Dialog
    Zhang, Jiaping
    Zhao, Tiancheng
    Yu, Zhou
    19TH ANNUAL MEETING OF THE SPECIAL INTEREST GROUP ON DISCOURSE AND DIALOGUE (SIGDIAL 2018), 2018, : 140 - 150
  • [5] UniTranSeR: A Unified Transformer Semantic Representation Framework for Multimodal Task-Oriented Dialog Systems
    Ma, Zhiyuan
    Li, Jianjun
    Li, Guohui
    Cheng, Yongjing
    PROCEEDINGS OF THE 60TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2022), VOL 1: (LONG PAPERS), 2022, : 103 - 114
  • [6] Paraphrase Augmented Task-Oriented Dialog Generation
    Gao, Silin
    Zhang, Yichi
    Ou, Zhijian
    Yu, Zhou
    58TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL 2020), 2020, : 639 - 649
  • [7] TamilATIS: Dataset for Task-Oriented Dialog in Tamil
    Ramaneswaran, S.
    Vijay, Sanchit
    Srinivasan, Kathiravan
    PROCEEDINGS OF THE SECOND WORKSHOP ON SPEECH AND LANGUAGE TECHNOLOGIES FOR DRAVIDIAN LANGUAGES (DRAVIDIANLANGTECH 2022), 2022, : 25 - 32
  • [8] An Efficient Framework for Development of Task-Oriented Dialog Systems in a Smart Home Environment
    Park, Youngmin
    Kang, Sangwoo
    Seo, Jungyun
    SENSORS, 2018, 18 (05)
  • [9] Combining Open Domain Question Answering with a Task-Oriented Dialog System
    Nehring, Jan
    Feldhus, Nils
    Ahmed, Akhyar
    Kaur, Harleen
    1ST WORKSHOP ON DOCUMENT-GROUNDED DIALOGUE AND CONVERSATIONAL QUESTION ANSWERING (DIALDOC 2021), 2021, : 38 - 45
  • [10] Evaluating System Sufficiency in a Multimodal, Multiuser Sensemaking Environment Designed for Intelligence Analysis
    Briggs, Shannon
    HCI INTERNATIONAL 2019 - LATE BREAKING PAPERS, HCII 2019, 2019, 11786 : 140 - 152