Home > Research > Publications & Outputs > Straggler root-cause and impact analysis for ma...

Electronic data

  • tsc2016b

    Rights statement: © 2019 IEEE. This is an author produced version of a paper published in IEEE Transactions on Services Computing. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Uploaded in accordance with the publisher's self-archiving policy.

    Accepted author manuscript, 2.11 MB, PDF document

    Available under license: CC BY: Creative Commons Attribution 4.0 International License

Links

Text available via DOI:

View graph of relations

Straggler root-cause and impact analysis for massive-scale virtualized cloud datacenters

Research output: Contribution to journalJournal articlepeer-review

Published
Close
<mark>Journal publication date</mark>1/01/2019
<mark>Journal</mark>IEEE Transactions on Services Computing
Issue number1
Volume12
Number of pages14
Pages (from-to)91-104
Publication StatusPublished
Early online date20/09/16
<mark>Original language</mark>English

Abstract

Increased complexity and scale of virtualized distributed systems has resulted in the manifestation of emergent phenomena substantially affecting overall system performance. This phenomena is known as “Long Tail”, whereby a small proportion of task stragglers significantly impede job completion time. While work focuses on straggler detection and mitigation, there is limited work that empirically studies straggler root-cause and quantifies its impact upon system operation. Such analysis is critical to ascertain in-depth knowledge of straggler occurrence for focusing developmental and research efforts towards solving the Long Tail challenge. This paper provides an empirical analysis of straggler root-cause within virtualized Cloud datacenters; we analyze two large-scale production systems to quantify the frequency and impact stragglers impose, and propose a method for conducting root-cause analysis. Results demonstrate approximately 5% of task stragglers impact 50% of total jobs for batch processes, and 53% of stragglers occur due to high server resource utilization. We leverage these findings to propose a method for extreme straggler detection through a combination of offline execution patterns modeling and online analytic agents to monitor tasks at runtime. Experiments show the approach is capable of detecting stragglers less than 11% into their execution lifecycle with 95% accuracy for short duration jobs.

Bibliographic note

© 2019 IEEE. This is an author produced version of a paper published in IEEE Transactions on Services Computing. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Uploaded in accordance with the publisher's self-archiving policy.