Home > Research > Publications & Outputs > Authors' reply to 'comments on 'researcher bias...

Links

Text available via DOI:

View graph of relations

Authors' reply to 'comments on 'researcher bias: The use of machine learning in software defect prediction''

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Published

Standard

Authors' reply to 'comments on 'researcher bias: The use of machine learning in software defect prediction''. / Shepperd, M.; Hall, T.; Bowes, D.

In: IEEE Transactions on Software Engineering, Vol. 44, No. 11, 01.11.2018, p. 1129-1131.

Research output: Contribution to Journal/MagazineJournal articlepeer-review

Harvard

APA

Vancouver

Shepperd M, Hall T, Bowes D. Authors' reply to 'comments on 'researcher bias: The use of machine learning in software defect prediction''. IEEE Transactions on Software Engineering. 2018 Nov 1;44(11):1129-1131. Epub 2017 Jul 24. doi: 10.1109/TSE.2017.2731308

Author

Shepperd, M. ; Hall, T. ; Bowes, D. / Authors' reply to 'comments on 'researcher bias: The use of machine learning in software defect prediction''. In: IEEE Transactions on Software Engineering. 2018 ; Vol. 44, No. 11. pp. 1129-1131.

Bibtex

@article{eab4f7cc2f3a48b6bc169fbfeb931fc3,
title = "Authors' reply to 'comments on 'researcher bias: The use of machine learning in software defect prediction''",
abstract = "In 2014 we published a meta-analysis of software defect prediction studies [1] . This suggested that the most important factor in determining results was Research Group, i.e., who conducts the experiment is more important than the classifier algorithms being investigated. A recent re-analysis [2] sought to argue that the effect is less strong than originally claimed since there is a relationship between Research Group and Dataset. In this response we show (i) the re-analysis is based on a small (21 percent) subset of our original data, (ii) using the same re-analysis approach with a larger subset shows that Research Group is more important than type of Classifier and (iii) however the data are analysed there is compelling evidence that who conducts the research has an effect on the results. This means that the problem of researcher bias remains. Addressing it should be seen as a matter of priority amongst those of us who conduct and publish experiments comparing the performance of competing software defect prediction systems.",
author = "M. Shepperd and T. Hall and D. Bowes",
year = "2018",
month = nov,
day = "1",
doi = "10.1109/TSE.2017.2731308",
language = "English",
volume = "44",
pages = "1129--1131",
journal = "IEEE Transactions on Software Engineering",
issn = "0098-5589",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "11",

}

RIS

TY - JOUR

T1 - Authors' reply to 'comments on 'researcher bias: The use of machine learning in software defect prediction''

AU - Shepperd, M.

AU - Hall, T.

AU - Bowes, D.

PY - 2018/11/1

Y1 - 2018/11/1

N2 - In 2014 we published a meta-analysis of software defect prediction studies [1] . This suggested that the most important factor in determining results was Research Group, i.e., who conducts the experiment is more important than the classifier algorithms being investigated. A recent re-analysis [2] sought to argue that the effect is less strong than originally claimed since there is a relationship between Research Group and Dataset. In this response we show (i) the re-analysis is based on a small (21 percent) subset of our original data, (ii) using the same re-analysis approach with a larger subset shows that Research Group is more important than type of Classifier and (iii) however the data are analysed there is compelling evidence that who conducts the research has an effect on the results. This means that the problem of researcher bias remains. Addressing it should be seen as a matter of priority amongst those of us who conduct and publish experiments comparing the performance of competing software defect prediction systems.

AB - In 2014 we published a meta-analysis of software defect prediction studies [1] . This suggested that the most important factor in determining results was Research Group, i.e., who conducts the experiment is more important than the classifier algorithms being investigated. A recent re-analysis [2] sought to argue that the effect is less strong than originally claimed since there is a relationship between Research Group and Dataset. In this response we show (i) the re-analysis is based on a small (21 percent) subset of our original data, (ii) using the same re-analysis approach with a larger subset shows that Research Group is more important than type of Classifier and (iii) however the data are analysed there is compelling evidence that who conducts the research has an effect on the results. This means that the problem of researcher bias remains. Addressing it should be seen as a matter of priority amongst those of us who conduct and publish experiments comparing the performance of competing software defect prediction systems.

U2 - 10.1109/TSE.2017.2731308

DO - 10.1109/TSE.2017.2731308

M3 - Journal article

VL - 44

SP - 1129

EP - 1131

JO - IEEE Transactions on Software Engineering

JF - IEEE Transactions on Software Engineering

SN - 0098-5589

IS - 11

ER -