Do automated program repair techniques repair hard and important bugs?

被引:0
|
作者
Manish Motwani
Sandhya Sankaranarayanan
René Just
Yuriy Brun
机构
[1] University of Massachusetts,College of Information and Computer Science
来源
关键词
Automated program repair; Repairability;
D O I
暂无
中图分类号
学科分类号
摘要
Existing evaluations of automated repair techniques focus on the fraction of the defects for which the technique can produce a patch, the time needed to produce patches, and how well patches generalize to the intended specification. However, these evaluations have not focused on the applicability of repair techniques and the characteristics of the defects that these techniques can repair. Questions such as “Can automated repair techniques repair defects that are hard for developers to repair?” and “Are automated repair techniques less likely to repair defects that involve loops?” have not, as of yet, been answered. To address such questions, we annotate two large benchmarks totaling 409 C and Java defects in real-world software, ranging from 22K to 2.8M lines of code, with measures of the defect’s importance, the developer-written patch’s complexity, and the quality of the test suite. We then analyze relationships between these measures and the ability to produce patches for the defects of seven automated repair techniques —AE, GenProg, Kali, Nopol, Prophet, SPR, and TrpAutoRepair. We find that automated repair techniques are less likely to produce patches for defects that required developers to write a lot of code or edit many files, or that have many tests relevant to the defect. Java techniques are more likely to produce patches for high-priority defects. Neither the time it took developers to fix a defect nor the test suite’s coverage correlate with the automated repair techniques’ ability to produce patches. Finally, automated repair techniques are less capable of fixing defects that require developers to add loops and new function calls, or to change method signatures. These findings identify strengths and shortcomings of the state-of-the-art of automated program repair along new dimensions. The presented methodology can drive research toward improving the applicability of automated repair techniques to hard and important bugs.
引用
收藏
页码:2901 / 2947
页数:46
相关论文
共 50 条
  • [31] Poster: Automated Program Repair with Canonical Constraints
    Hill, Andrew
    Pasareanu, Corina S.
    Stolee, Kathryn T.
    PROCEEDINGS 2018 IEEE/ACM 40TH INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING - COMPANION (ICSE-COMPANION, 2018, : 339 - 341
  • [33] Automated Program Repair in an Integrated Development Environment
    Pei, Yu
    Furia, Carlo A.
    Nordio, Martin
    Meyer, Bertrand
    2015 IEEE/ACM 37TH IEEE INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING, VOL 2, 2015, : 681 - 684
  • [34] Large Language Models for Automated Program Repair
    Ribeiro, Francisco
    COMPANION PROCEEDINGS OF THE 2023 ACM SIGPLAN INTERNATIONAL CONFERENCE ON SYSTEMS, PROGRAMMING, LANGUAGES, AND APPLICATIONS: SOFTWARE FOR HUMANITY, SPLASH COMPANION 2023, 2023, : 7 - 9
  • [35] How to Measure the Performance of Automated Program Repair?
    Qi, Yuhua
    Liu, Wenhong
    Zhang, Weixiang
    Yang, Deheng
    2018 5TH INTERNATIONAL CONFERENCE ON INFORMATION SCIENCE AND CONTROL ENGINEERING (ICISCE 2018), 2018, : 246 - 250
  • [36] Automated patch assessment for program repair at scale
    Ye, He
    Martinez, Matias
    Monperrus, Martin
    EMPIRICAL SOFTWARE ENGINEERING, 2021, 26 (02)
  • [37] The Impact of Search Algorithms in Automated Program Repair
    Assiri, Fatmah Yousef
    Bieman, James M.
    PROCEEDINGS OF THE 2015 INTERNATIONAL CONFERENCE ON SOFT COMPUTING AND SOFTWARE ENGINEERING (SCSE'15), 2015, 62 : 65 - 72
  • [38] High-Quality Automated Program Repair
    Motwani, Manish
    2021 IEEE/ACM 43RD INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING: COMPANION PROCEEDINGS (ICSE-COMPANION 2021), 2021, : 309 - 314
  • [39] Static Automated Program Repair for Heap Properties
    van Tonder, Rijnard
    Le Goues, Claire
    PROCEEDINGS 2018 IEEE/ACM 40TH INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING (ICSE), 2018, : 151 - 162
  • [40] BugsPHP: A dataset for Automated Program Repair in PHP
    Pramod, K.D.
    De Silva, W.T.N.
    Thabrew, W.U.K.
    Shariffdeen, Ridwan
    Wickramanayake, Sandareka
    arXiv,