Final published version
Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Conference contribution/Paper › peer-review
Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSN › Conference contribution/Paper › peer-review
}
TY - GEN
T1 - The impact of fault models on software robustness evaluations
AU - Winter, S.
AU - Sârbu, C.
AU - Suri, Neeraj
AU - Murphy, B.
PY - 2011/5/21
Y1 - 2011/5/21
N2 - Following the design and in-lab testing of software, the evaluation of its resilience to actual operational perturbations in the field is a key validation need. Software-implemented fault injection (SWIFI) is a widely used approach for evaluating the robustness of software components. Recent research [24, 18] indicates that the selection of the applied fault model has considerable influence on the results of SWIFI-based evaluations, thereby raising the question how to select appropriate fault models (i.e. that provide justified robustness evidence). This paper proposes several metrics for comparatively evaluating fault models's abilities to reveal robustness vulnerabilities. It demonstrates their application in the context of OS device drivers by investigating the influence (and relative utility) of four commonly used fault models, i.e. bit flips (in function parameters and in binaries), data type dependent parameter corruptions, and parameter fuzzing. We assess the efficiency of these models at detecting robustness vulnerabilities during the SWIFI evaluation of a real embedded operating system kernel and discuss application guidelines for our metrics alongside. © 2011 ACM.
AB - Following the design and in-lab testing of software, the evaluation of its resilience to actual operational perturbations in the field is a key validation need. Software-implemented fault injection (SWIFI) is a widely used approach for evaluating the robustness of software components. Recent research [24, 18] indicates that the selection of the applied fault model has considerable influence on the results of SWIFI-based evaluations, thereby raising the question how to select appropriate fault models (i.e. that provide justified robustness evidence). This paper proposes several metrics for comparatively evaluating fault models's abilities to reveal robustness vulnerabilities. It demonstrates their application in the context of OS device drivers by investigating the influence (and relative utility) of four commonly used fault models, i.e. bit flips (in function parameters and in binaries), data type dependent parameter corruptions, and parameter fuzzing. We assess the efficiency of these models at detecting robustness vulnerabilities during the SWIFI evaluation of a real embedded operating system kernel and discuss application guidelines for our metrics alongside. © 2011 ACM.
KW - fault injection
KW - fault models
KW - robustness testing
KW - Bit-flips
KW - Data type
KW - Device Driver
KW - Embedded operating systems
KW - Fault injection
KW - Fault model
KW - Function parameters
KW - Relative utility
KW - Robustness evaluation
KW - Software component
KW - Random access storage
KW - Software engineering
KW - Software testing
KW - Models
U2 - 10.1145/1985793.1985801
DO - 10.1145/1985793.1985801
M3 - Conference contribution/Paper
SN - 9781450304450
SP - 51
EP - 60
BT - Proceedings of the 33rd International Conference on Software Engineering
PB - ACM
ER -