Automatically learning usage behavior and generating event sequences for black-box testing of reactive systems

被引:2
|
作者
Kirac, M. Furkan [1 ]
Aktemur, Baris [1 ]
Sozer, Hasan [1 ]
Gebizli, Ceren Sahin [2 ]
机构
[1] Ozyegin Univ, Comp Sci, Istanbul, Turkey
[2] Vestel Elect, Manisa, Turkey
关键词
Test case generation; Black-box testing; Recurrent neural networks; Long short-term memory networks; Learning usage behavior; MODEL;
D O I
10.1007/s11219-018-9439-1
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
We propose a novel technique based on recurrent artificial neural networks to generate test cases for black-box testing of reactive systems. We combine functional testing inputs that are automatically generated from a model together with manually-applied test cases for robustness testing. We use this combination to train a long short-term memory (LSTM) network. As a result, the network learns an implicit representation of the usage behavior that is liable to failures. We use this network to generate new event sequences as test cases. We applied our approach in the context of an industrial case study for the black-box testing of a digital TV system. LSTM-generated test cases were able to reveal several faults, including critical ones, that were not detected with existing automated or manual testing activities. Our approach is complementary to model-based and exploratory testing, and the combined approach outperforms random testing in terms of both fault coverage and execution time.
引用
收藏
页码:861 / 883
页数:23
相关论文
共 50 条
  • [1] Automatically learning usage behavior and generating event sequences for black-box testing of reactive systems
    M. Furkan Kıraç
    Barış Aktemur
    Hasan Sözer
    Ceren Şahin Gebizli
    Software Quality Journal, 2019, 27 : 861 - 883
  • [2] AN ALGORITHM FOR AUTOMATICALLY GENERATING BLACK-BOX TEST CASES
    Xu Baowen Nie Changhai Shi Qunfeng Lu Hong (Department of computer Science & Engineering
    JournalofElectronics(China), 2003, (01) : 74 - 77
  • [3] Automated Requirements-Based Testing of Black-Box Reactive Systems
    Narizzano, Massimo
    Pulina, Luca
    Tacchella, Armando
    Vuotto, Simone
    NASA FORMAL METHODS (NFM 2020), 2020, 12229 : 153 - 169
  • [4] Generating expected results for automated black-box testing
    Schroeder, PJ
    Faherty, P
    Korel, B
    ASE 2002: 17TH IEEE INTERNATIONAL CONFERENCE ON AUTOMATED SOFTWARE ENGINEERING, 2002, : 139 - 148
  • [5] Black-box testing of grey-box behavior
    Tyler, B
    Soundarajan, N
    FORMAL APPROACHES TO SOFTWARE TESTING, 2004, 2931 : 1 - 14
  • [6] Integration of Learning-Based Testing and Supervisory Control for Requirements Conformance of Black-Box Reactive Systems
    Zhang, Huimin
    Feng, Lei
    Wu, Naiqi
    Li, Zhiwu
    IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, 2018, 15 (01) : 2 - 15
  • [7] Generating Test Data for Black-Box Testing using Genetic Algorithms
    Fischer, Marten
    Toenjes, Ralf
    2012 IEEE 17TH CONFERENCE ON EMERGING TECHNOLOGIES & FACTORY AUTOMATION (ETFA), 2012,
  • [8] FACTS: Automated Black-Box Testing of FinTech Systems
    Wang, Qingshun
    Gu, Lintao
    Xue, Minhui
    Xu, Lihua
    Niu, Wenyu
    Dou, Liang
    He, Liang
    Xie, Tao
    ESEC/FSE'18: PROCEEDINGS OF THE 2018 26TH ACM JOINT MEETING ON EUROPEAN SOFTWARE ENGINEERING CONFERENCE AND SYMPOSIUM ON THE FOUNDATIONS OF SOFTWARE ENGINEERING, 2018, : 839 - 844
  • [9] Black-Box Testing and Auditing of Bias in ADM Systems
    Krafft, Tobias D.
    Hauer, Marc P.
    Zweig, Katharina
    MINDS AND MACHINES, 2024, 34 (02)
  • [10] Challenges of Explaining the Behavior of Black-Box AI Systems
    Asatiani, Aleksandre
    Malo, Pekka
    Nagb, Per Radberg
    Penttinen, Esko
    Rinta-Kahila, Tapani
    Salovaara, Antti
    MIS QUARTERLY EXECUTIVE, 2020, 19 (04) : 259 - 278