Home > Research > Publications & Outputs > Some automatically generated patches are more l...

Electronic data

  • Some Automatically Generated Patches are More Likely to be Correct than Others An Analysis of Defects4J Patch Features

    Accepted author manuscript, 436 KB, PDF document

    Available under license: CC BY: Creative Commons Attribution 4.0 International License

Links

Text available via DOI:

View graph of relations

Some automatically generated patches are more likely to be correct than others: an analysis of Defects4J patch features

Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSNConference contribution/Paperpeer-review

Published
Publication date26/10/2022
Host publicationProceedings - International Workshop on Automated Program Repair, APR 2022
Place of PublicationNew York
PublisherACM
Pages46-52
Number of pages7
ISBN (electronic)9781450392853
<mark>Original language</mark>English
EventAPR '22: Proceedings of the Third International Workshop on Automated Program Repair - Pittsburgh, United States
Duration: 19/05/2022 → …
https://dl.acm.org/doi/proceedings/10.1145/3524459

Workshop

WorkshopAPR '22: Proceedings of the Third International Workshop on Automated Program Repair
Abbreviated titleAPR '22
Country/TerritoryUnited States
CityPittsburgh
Period19/05/22 → …
Internet address

Workshop

WorkshopAPR '22: Proceedings of the Third International Workshop on Automated Program Repair
Abbreviated titleAPR '22
Country/TerritoryUnited States
CityPittsburgh
Period19/05/22 → …
Internet address

Abstract

Defects4J is a popular dataset against which many Java Automatic Program Repair (APR) tools benchmark their performance. However, recent evidence suggests that some APR tools overfit to Defects4J, producing plausible patches which are incorrect. What we do not currently know is whether there is any commonality in the features of these plausible patches that turn out not to be correct. We compare the features of Defects4J's human written patches in terms of those correctly patched by existing APR tools and those incorrectly patched. We found that 48.4% of Defects4J v1.5 have been automatically patched by existing APR tools; of which only 28.9% have been correctly patched leaving 19.5% incorrectly patched. We found patches of defects that added a method call, added a variable, or wrapped existing code with new code, such as a try/catch block were significantly associated with incorrect patches. Editing only a single line was significantly associated with correct patches. Our results suggest that current tools are weak at generating multi-line patches and synthesising new code especially when wrapping existing code. Our results highlight potential future areas of development for new APR approaches, such as developing a tool that effectively repairs defects that require a try/catch block. Our replication Package is available online11Replication Package available at: https://github.com/IncorrectDefects/ReplicationPackage.