Accepted author manuscript, 436 KB, PDF document
Available under license: CC BY: Creative Commons Attribution 4.0 International License
Final published version
Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Conference contribution/Paper › peer-review
Publication date | 26/10/2022 |
---|---|
Host publication | Proceedings - International Workshop on Automated Program Repair, APR 2022 |
Place of Publication | New York |
Publisher | ACM |
Pages | 46-52 |
Number of pages | 7 |
ISBN (electronic) | 9781450392853 |
<mark>Original language</mark> | English |
Event | APR '22: Proceedings of the Third International Workshop on Automated Program Repair - Pittsburgh, United States Duration: 19/05/2022 → … https://dl.acm.org/doi/proceedings/10.1145/3524459 |
Workshop | APR '22: Proceedings of the Third International Workshop on Automated Program Repair |
---|---|
Abbreviated title | APR '22 |
Country/Territory | United States |
City | Pittsburgh |
Period | 19/05/22 → … |
Internet address |
Workshop | APR '22: Proceedings of the Third International Workshop on Automated Program Repair |
---|---|
Abbreviated title | APR '22 |
Country/Territory | United States |
City | Pittsburgh |
Period | 19/05/22 → … |
Internet address |
Defects4J is a popular dataset against which many Java Automatic Program Repair (APR) tools benchmark their performance. However, recent evidence suggests that some APR tools overfit to Defects4J, producing plausible patches which are incorrect. What we do not currently know is whether there is any commonality in the features of these plausible patches that turn out not to be correct. We compare the features of Defects4J's human written patches in terms of those correctly patched by existing APR tools and those incorrectly patched. We found that 48.4% of Defects4J v1.5 have been automatically patched by existing APR tools; of which only 28.9% have been correctly patched leaving 19.5% incorrectly patched. We found patches of defects that added a method call, added a variable, or wrapped existing code with new code, such as a try/catch block were significantly associated with incorrect patches. Editing only a single line was significantly associated with correct patches. Our results suggest that current tools are weak at generating multi-line patches and synthesising new code especially when wrapping existing code. Our results highlight potential future areas of development for new APR approaches, such as developing a tool that effectively repairs defects that require a try/catch block. Our replication Package is available online11Replication Package available at: https://github.com/IncorrectDefects/ReplicationPackage.