LEVA: Using Large Language Models to Enhance Visual Analytics

被引:2
|
作者
Zhao, Yuheng [1 ]
Zhang, Yixing [1 ]
Zhang, Yu [2 ]
Zhao, Xinyi [1 ]
Wang, Junjie [1 ]
Shao, Zekai [1 ]
Turkay, Cagatay [3 ]
Chen, Siming [1 ]
机构
[1] Fudan Univ, Sch Data Sci, Shanghai 200437, Peoples R China
[2] Univ Oxford, Dept Comp Sci, Oxford OX1 2JD, England
[3] Univ Warwick, Coventry CV4 7AL, England
关键词
Data visualization; Visual analytics; Task analysis; Data analysis; Grammar; Codes; Annotations; Insight recommendation; mixed-initiative; interface agent; large language models; visual analytics; GUIDANCE; VISUALIZATION; TYPOLOGY;
D O I
10.1109/TVCG.2024.3368060
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
Visual analytics supports data analysis tasks within complex domain problems. However, due to the richness of data types, visual designs, and interaction designs, users need to recall and process a significant amount of information when they visually analyze data. These challenges emphasize the need for more intelligent visual analytics methods. Large language models have demonstrated the ability to interpret various forms of textual data, offering the potential to facilitate intelligent support for visual analytics. We propose LEVA, a framework that uses large language models to enhance users' VA workflows at multiple stages: onboarding, exploration, and summarization. To support onboarding, we use large language models to interpret visualization designs and view relationships based on system specifications. For exploration, we use large language models to recommend insights based on the analysis of system status and data to facilitate mixed-initiative exploration. For summarization, we present a selective reporting strategy to retrace analysis history through a stream visualization and generate insight reports with the help of large language models. We demonstrate how LEVA can be integrated into existing visual analytics systems. Two usage scenarios and a user study suggest that LEVA effectively aids users in conducting visual analytics.
引用
收藏
页码:1830 / 1847
页数:18
相关论文
共 50 条
  • [41] JailbreakLens: Visual Analysis of Jailbreak Attacks Against Large Language Models
    Feng, Yingchaojie
    Chen, Zhizhang
    Kang, Zhining
    Wang, Sijia
    Zhu, Minfeng
    Zhang, Wei
    Chen, Wei
    arXiv,
  • [42] Visual In-Context Learning for Large Vision-Language Models
    Zhou, Yucheng
    Le, Xiang
    Wang, Qianning
    Shen, Jianbing
    FINDINGS OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS: ACL 2024, 2024, : 15890 - 15902
  • [43] Large Language Models and Multimodal Retrieval for Visual Word Sense Disambiguation
    Kritharoula, Anastasia
    Lymperaiou, Maria
    Stamou, Giorgos
    2023 CONFERENCE ON EMPIRICAL METHODS IN NATURAL LANGUAGE PROCESSING (EMNLP 2023), 2023, : 13053 - 13077
  • [44] Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Models
    Ma, Chuofan
    Jiang, Yi
    Wu, Jiannan
    Yuan, Zehuan
    Qi, Xiaojuan
    COMPUTER VISION - ECCV 2024, PT VI, 2025, 15064 : 417 - 435
  • [45] Experiencing Visual Captions: Augmented Communication with Real-time Visuals using Large Language Models
    Liu, Xingyu 'Bruce'
    Kirilyuk, Vladimir
    Yuan, Xiuxiu
    Chi, Peggy
    Chen, Xiang 'Anthony'
    Olwal, Alex
    Du, Ruofei
    ADJUNCT PROCEEDINGS OF THE 36TH ANNUAL ACM SYMPOSIUM ON USER INTERFACE SOFTWARE & TECHNOLOGY, UIST 2023 ADJUNCT, 2023,
  • [46] Language Models Meet World Models: Embodied Experiences Enhance Language Models
    Xiang, Jiannan
    Tao, Tianhua
    Gu, Yi
    Shu, Tianmin
    Wang, Zirui
    Yang, Zichao
    Hu, Zhiting
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 36 (NEURIPS 2023), 2023,
  • [47] Collaborative Visual Analytics of Large Radio Surveys
    Vohl, D.
    Fluke, C. J.
    Hassan, A. H.
    Barnes, D. G.
    Kilborn, V. A.
    ASTRONOMICAL DATA ANALYSIS SOFTWARE AND SYSTEMS XXVI, 2019, 521 : 264 - 267
  • [48] A VISUAL ANALYTICS FRAMEWORK FOR LARGE TRANSPORTATION DATASETS
    Zhong, Chen
    Arisona, Stefan Muller
    Schmitt, Gerhard
    PROCEEDINGS OF THE 19TH INTERNATIONAL CONFERENCE ON COMPUTER-AIDED ARCHITECTURAL DESIGN RESEARCH IN ASIA (CAADRIA 2014): RETHINKING COMPREHENSIVE DESIGN: SPECULATIVE COUNTERCULTURE, 2014, : 223 - 232
  • [49] Using Large Language Models to Improve Sentiment Analysis in Latvian Language
    Purvins, Pauls
    Urtans, Evalds
    Caune, Vairis
    BALTIC JOURNAL OF MODERN COMPUTING, 2024, 12 (02): : 165 - 175
  • [50] Self-Training Large Language Models for Improved Visual Program Synthesis With Visual Reinforcement
    Khan, Zaid
    Kumar, Vijay B. G.
    Schulter, Samuel
    Fu, Yun
    Chandraker, Manmohan
    2024 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2024, : 14344 - 14353