Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Conference contribution/Paper › peer-review
Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Conference contribution/Paper › peer-review
}
TY - GEN
T1 - Code cleaning for software defect prediction
T2 - A cautionary tale
AU - Shippey, T.
AU - Bowes, D.
AU - Counsell, S.
AU - Hall, T.
PY - 2018/8/29
Y1 - 2018/8/29
N2 - In this paper, we describe our experience of developing a new technique to improve defect prediction (code cleaning) which performed very encouragingly on the first two systems on which we evaluated it (both systems had their origins in one company). Code cleaning also worked well on an additional open source system (Eclipse). But our code cleaning technique then performed disappointingly on all 69 subsequent open source systems on which we evaluated it. Without our round twoevaluations on these 69 open source systems we would have published misleading prediction results. We discuss the need for performance evaluations to be performed on carefully selected samples of systems if reliable conclusions are to be drawn.
AB - In this paper, we describe our experience of developing a new technique to improve defect prediction (code cleaning) which performed very encouragingly on the first two systems on which we evaluated it (both systems had their origins in one company). Code cleaning also worked well on an additional open source system (Eclipse). But our code cleaning technique then performed disappointingly on all 69 subsequent open source systems on which we evaluated it. Without our round twoevaluations on these 69 open source systems we would have published misleading prediction results. We discuss the need for performance evaluations to be performed on carefully selected samples of systems if reliable conclusions are to be drawn.
U2 - 10.1109/SEAA.2018.00047
DO - 10.1109/SEAA.2018.00047
M3 - Conference contribution/Paper
SP - 239
EP - 243
BT - 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA)
PB - IEEE
ER -