Rights statement: ©2020 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Accepted author manuscript, 375 KB, PDF document
Available under license: CC BY-NC: Creative Commons Attribution-NonCommercial 4.0 International License
Final published version
Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Conference contribution/Paper › peer-review
Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Conference contribution/Paper › peer-review
}
TY - GEN
T1 - A Survey of Methodology in Self-Adaptive Systems Research
AU - Porter, Barry
AU - Rodrigues Filho, Roberto
AU - Dean, Paul
N1 - ©2020 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
PY - 2020/9/15
Y1 - 2020/9/15
N2 - Major research venues on autonomic and self-adaptive systems have been active for 16 years, exploring and building on the seminal vision of autonomic computing in 2003. We study the current trajectory and progress of the research field towards this vision, surveying the research questions that are asked by researchers and the methodological practice that they employ in order to answer these questions. We survey contributions under this lens across the three main venues for primary research in autonomic and self-adaptive systems work: ICAC, SASO, and SEAMS. We examine the last three years of contributions from each venue, totalling 210 publications, to gain an understanding of the dominant current research questions and methodological practice - and what this shows us about the progress of the field. Our major findings include: (i) most research questions still focus one level below the highest autonomy level vision; (ii) methodological practice is split almost evenly between real-world experiments and simulation; (iii) a high level of positive results bias exists in publications; and (iv) there are low levels of repeatability across most contributions.
AB - Major research venues on autonomic and self-adaptive systems have been active for 16 years, exploring and building on the seminal vision of autonomic computing in 2003. We study the current trajectory and progress of the research field towards this vision, surveying the research questions that are asked by researchers and the methodological practice that they employ in order to answer these questions. We survey contributions under this lens across the three main venues for primary research in autonomic and self-adaptive systems work: ICAC, SASO, and SEAMS. We examine the last three years of contributions from each venue, totalling 210 publications, to gain an understanding of the dominant current research questions and methodological practice - and what this shows us about the progress of the field. Our major findings include: (i) most research questions still focus one level below the highest autonomy level vision; (ii) methodological practice is split almost evenly between real-world experiments and simulation; (iii) a high level of positive results bias exists in publications; and (iv) there are low levels of repeatability across most contributions.
U2 - 10.1109/ACSOS49614.2020.00039
DO - 10.1109/ACSOS49614.2020.00039
M3 - Conference contribution/Paper
SP - 168
EP - 177
BT - 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS)
PB - IEEE
ER -