Assessing the Impact of GPT-4 Turbo in Generating Defeaters for Assurance Cases

被引:2
|
作者
Shahandashti, Kimya Khakzad [1 ]
Sivakumar, Mithila [1 ]
Mohajer, Mohammad Mahdi [1 ]
Belle, Alvine B. [1 ]
Wang, Song [1 ]
Lethbridge, Timothy C. [2 ]
机构
[1] York Univ, Toronto, ON, Canada
[2] Univ Ottawa, Ottawa, ON, Canada
关键词
Large Language Models; assurance cases; assurance defeaters; system certification; FM for Requirement Engineering;
D O I
10.1145/3650105.3652291
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Assurance cases (ACs) are structured arguments that allowverifying the correct implementation of the created systems' non-functional requirements (e.g., safety, security). This allows for preventing system failure. The latter may result in catastrophic outcomes (e.g., loss of lives). ACs support the certification of systems in compliance with industrial standards, e.g., DO-178C and ISO 26262. Identifying defeaters -arguments that challenge these ACs - is crucial for enhancing ACs' robustness and confidence. To automatically support that task, we propose a novel approach that explores the potential of GPT-4 Turbo, an advanced Large Language Model (LLM) developed by OpenAI, in identifying defeaters within ACs formalized using the Eliminative Argumentation (EA) notation. Our preliminary evaluation assesses the model's ability to comprehend and generate arguments in this context and the results show that GPT-4 turbo is very proficient in EA notation and can generate different types of defeaters.
引用
收藏
页码:52 / 56
页数:5
相关论文
共 50 条
  • [1] Using GPT-4 Turbo To Automatically Identify Defeaters In Assurance Cases
    Shahandashti, Kimya Khakzad
    Belle, Alvine Boaye
    Mohajer, Mohammad Mahdi
    Odu, Oluwafemi
    Lethbridge, Timothy C.
    Hemmati, Hadi
    Wang, Song
    32ND INTERNATIONAL REQUIREMENTS ENGINEERING CONFERENCE WORKSHOPS, REW 2024, 2024, : 46 - 56
  • [2] GPT-4 Turbo with Vision fails to outperform text-only GPT-4 Turbo in the Japan Diagnostic Radiology Board Examination
    Hirano, Yuichiro
    Hanaoka, Shouhei
    Nakao, Takahiro
    Miki, Soichiro
    Kikuchi, Tomohiro
    Nakamura, Yuta
    Nomura, Yukihiro
    Yoshikawa, Takeharu
    Abe, Osamu
    JAPANESE JOURNAL OF RADIOLOGY, 2024, 42 (08) : 918 - 926
  • [3] Assessing the medical reasoning skills of GPT-4 in complex ophthalmology cases
    Milad, Daniel
    Antaki, Fares
    Milad, Jason
    Farah, Andrew
    Khairy, Thomas
    Mikhail, David
    Giguere, Charles-Edouard
    Touma, Samir
    Bernstein, Allison
    Szigiato, Andrei-Alexandru
    Nayman, Taylor
    Mullie, Guillaume A.
    Duval, Renaud
    BRITISH JOURNAL OF OPHTHALMOLOGY, 2024, 108 (10) : 1398 - 1405
  • [4] GPT-3.5 Turbo and GPT-4 Turbo in Title and Abstract Screening for Systematic Reviews
    Oami, Takehiko
    Okada, Yohei
    Nakada, Taka-aki
    JMIR MEDICAL INFORMATICS, 2025, 13
  • [5] GPT-4 turbo with vision fails to outperform text-only GPT-4 turbo in the Japan diagnostic radiology board examination: correspondence
    Kleebayoon, Amnuay
    Wiwanitkit, Viroj
    JAPANESE JOURNAL OF RADIOLOGY, 2024, 42 (10) : 1213 - 1213
  • [6] Toward Improved Radiologic Diagnostics: Investigating the Utility and Limitations of GPT-3.5 Turbo and GPT-4 with Quiz Cases
    Kikuchi, Tomohiro
    Nakao, Takahiro
    Nakamura, Yuta
    Hanaoka, Shouhei
    Mori, Harushi
    Yoshikawa, Takeharu
    AMERICAN JOURNAL OF NEURORADIOLOGY, 2024, 45 (10) : 1506 - 1511
  • [7] Assessing GPT-4 multimodal performance in radiological image analysis
    Brin, Dana
    Sorin, Vera
    Barash, Yiftach
    Konen, Eli
    Glicksberg, Benjamin S.
    Nadkarni, Girish N.
    Klang, Eyal
    EUROPEAN RADIOLOGY, 2025, 35 (04) : 1959 - 1965
  • [8] An AAC Application for Generating Japanese Response Phrases Using GPT-4
    Kitayama, Suzuna
    Hirotomi, Tetsuya
    COMPUTERS HELPING PEOPLE WITH SPECIAL NEEDS, PT II, ICCHP 2024, 2024, 14751 : 144 - 152
  • [9] Assessing the Performance of GPT-3.5 and GPT-4 on the 2023 Japanese Nursing Examination
    Kaneda, Yudai
    Takahashi, Ryo
    Kaneda, Uiri
    Akashima, Shiori
    Okita, Haruna
    Misaki, Sadaya
    Yamashiro, Akimi
    Ozaki, Akihiko
    Tanimoto, Tetsuya
    CUREUS JOURNAL OF MEDICAL SCIENCE, 2023, 15 (08)
  • [10] An AAC Application for Generating Japanese Response Phrases Using GPT-4
    Kitayama, Suzuna
    Hirotomi, Tetsuya
    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2024, 14751 LNCS : 144 - 152