EaTVul: ChatGPT-based Evasion Attack Against Software Vulnerability Detection


Shigang Liu, CSIRO's Data61 and Swinburne University of Technology; Di Cao, Swinburne University of Technology; Junae Kim, Tamas Abraham, and Paul Montague, DST Group, Australia; Seyit Camtepe, CSIRO's Data61; Jun Zhang and Yang Xiang, Swinburne University of Technology


Recently, deep learning has demonstrated promising results in enhancing the accuracy of vulnerability detection and identifying vulnerabilities in software. However, these techniques are still vulnerable to attacks. Adversarial examples can exploit vulnerabilities within deep neural networks, posing a significant threat to system security. This study showcases the susceptibility of deep learning models to adversarial attacks, which can achieve 100% attack success rate. The proposed method, EaTVul, encompasses six stages: identification of important adversarial samples using support vector machines, identification of important features using the attention mechanism, generation of adversarial data based on these features, preparation of an adversarial attack pool, selection of seed data using a fuzzy genetic algorithm, and the execution of an evasion attack. Extensive experiments demonstrate the effectiveness of EaTVul, achieving an attack success rate of more than 83% when the snippet size is greater than 2. Furthermore, in most cases with a snippet size of 4, EaTVul achieves a 100% attack success rate. The findings of this research emphasize the necessity of robust defenses against adversarial attacks in software vulnerability detection.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

Liu Paper (Prepublication) PDF