ComplexCodeEval: A Benchmark for Evaluating Large Code Models on More Complex Code

被引：0
|
作者：
Feng, Jia [1 ]
Liu, Jiachen [2 ]
Gao, Cuiyun [2 ]
Chong, Chun Yong [3 ]
Wang, Chaozheng [4 ]
Gao, Shan [5 ]
Xia, Xin [5 ]
机构：
[1] University of Electronic Science and Technology of China, Shenzhen, China
[2] Harbin Institute of Technology, Shenzhen, China
[3] HUAWEI, Hong Kong, Hong Kong
[4] The Chinese University of Hong Kong, Hong Kong, Hong Kong
[5] Huawei, Shenzhen, China
来源：
Proceedings - 2024 39th ACM/IEEE International Conference on Automated Software Engineering, ASE 2024 |
关键词：
Compendex;
D O I：
暂无
中图分类号：
学科分类号：
摘要：
Information leakage
引用
收藏
页码：1895 / 1906
相关论文
共 50 条
  • [11] Large Language Models of Code Fail at Completing Code with Potential Bugs
    Tuan Dinh
    Zhao, Jinman
    Tan, Samson
    Negrinho, Renato
    Lausen, Leonard
    Zha, Sheng
    Karypis, George
    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 36 (NEURIPS 2023), 2023,
  • [12] Evaluating and Optimizing the Effectiveness of Neural Machine Translation in Supporting Code Retrieval Models: A Study on the CAT Benchmark
    Phan, Hung
    Jannesari, Ali
    PROCEEDINGS OF THE 32ND ACM INTERNATIONAL CONFERENCE ON INFORMATION AND KNOWLEDGE MANAGEMENT, CIKM 2023, 2023, : 2055 - 2064
  • [13] CodeLMSec Benchmark: Systematically Evaluating and Finding Security Vulnerabilities in Black-Box Code Language Models
    Hajipour, Hossein
    Hassler, Keno
    Holz, Thorsten
    Schoenherr, Lea
    Fritz, Mario
    IEEE CONFERENCE ON SAFE AND TRUSTWORTHY MACHINE LEARNING, SATML 2024, 2024, : 684 - 709
  • [14] Evaluating and Optimizing the Effectiveness of Neural Machine Translation in Supporting Code Retrieval Models: A Study on the CAT Benchmark
    Phan, Hung
    Jannesari, Ali
    arXiv, 2023,
  • [15] Exploring and Evaluating Personalized Models for Code Generation
    Zlotchevski, Andrei
    Drain, Dawn
    Svyatkovskiy, Alexey
    Clement, Colin B.
    Sundaresan, Neel
    Tufano, Michele
    PROCEEDINGS OF THE 30TH ACM JOINT MEETING EUROPEAN SOFTWARE ENGINEERING CONFERENCE AND SYMPOSIUM ON THE FOUNDATIONS OF SOFTWARE ENGINEERING, ESEC/FSE 2022, 2022, : 1500 - 1508
  • [16] Exploring and Evaluating Personalized Models for Code Generation
    Zlotchevski, Andrei
    Drain, Dawn
    Svyatkovskiy, Alexey
    Clement, Colin
    Sundaresan, Neel
    Tufano, Michele
    arXiv, 2022,
  • [17] Will they like this? Evaluating Code Contributions With Language Models
    Hellendoorn, Vincent J.
    Devanbu, Premkumar T.
    Bacchelli, Alberto
    12TH WORKING CONFERENCE ON MINING SOFTWARE REPOSITORIES (MSR 2015), 2015, : 157 - 167
  • [18] Evaluating Social Bias in Code Generation Models
    Ling, Lin
    COMPANION PROCEEDINGS OF THE 32ND ACM INTERNATIONAL CONFERENCE ON THE FOUNDATIONS OF SOFTWARE ENGINEERING, FSE COMPANION 2024, 2024, : 695 - 697
  • [19] A bilingual benchmark for evaluating large language models
    Alkaoud, Mohamed
    PEERJ COMPUTER SCIENCE, 2024, 10
  • [20] Gotcha! This Model Uses My Code! Evaluating Membership Leakage Risks in Code Models
    Yang, Zhou
    Zhao, Zhipeng
    Wang, Chenyu
    Shi, Jieke
    Kim, Dongsun
    Han, Donggyun
    Lo, David
    IEEE TRANSACTIONS ON SOFTWARE ENGINEERING, 2024, 50 (12) : 3290 - 3306
12345