Home > Research > Publications & Outputs > ROSE

Electronic data

Links

Text available via DOI:

View graph of relations

ROSE: Cluster Resource Scheduling via Speculative Over-subscription

Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSNConference contribution/Paperpeer-review

Published

Standard

ROSE: Cluster Resource Scheduling via Speculative Over-subscription. / Sun, Xiaoyang; Hu, Chunming; Yang, Renyu et al.
38th IEEE International Conference on Distributed Systems Computing Systems. IEEE, 2018. p. 949-960 (2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS)).

Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSNConference contribution/Paperpeer-review

Harvard

Sun, X, Hu, C, Yang, R, Garraghan, P, Wo, T, Xu, J, Zhu, J & Li, C 2018, ROSE: Cluster Resource Scheduling via Speculative Over-subscription. in 38th IEEE International Conference on Distributed Systems Computing Systems. 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), IEEE, pp. 949-960. https://doi.org/10.1109/ICDCS.2018.00096

APA

Sun, X., Hu, C., Yang, R., Garraghan, P., Wo, T., Xu, J., Zhu, J., & Li, C. (2018). ROSE: Cluster Resource Scheduling via Speculative Over-subscription. In 38th IEEE International Conference on Distributed Systems Computing Systems (pp. 949-960). (2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS)). IEEE. https://doi.org/10.1109/ICDCS.2018.00096

Vancouver

Sun X, Hu C, Yang R, Garraghan P, Wo T, Xu J et al. ROSE: Cluster Resource Scheduling via Speculative Over-subscription. In 38th IEEE International Conference on Distributed Systems Computing Systems. IEEE. 2018. p. 949-960. (2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS)). Epub 2018 Apr 4. doi: 10.1109/ICDCS.2018.00096

Author

Sun, Xiaoyang ; Hu, Chunming ; Yang, Renyu et al. / ROSE : Cluster Resource Scheduling via Speculative Over-subscription. 38th IEEE International Conference on Distributed Systems Computing Systems. IEEE, 2018. pp. 949-960 (2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS)).

Bibtex

@inproceedings{3f23d24ff7ee45e4b30502cc6fc06e15,
title = "ROSE: Cluster Resource Scheduling via Speculative Over-subscription",
abstract = "A long-standing challenge in cluster scheduling is to achieve a high degree of utilization of heterogeneous resources in a cluster. In practice there exists a substantial disparity between perceived and actual resource utilization. A scheduler might regard a cluster as fully utilized if a large resource request queue is present, but the actual resource utilization of the cluster can be in fact very low. This disparity results in the formation of idle resources, leading to inefficient resource usage and incurring high operational costs and an inability to provision services. In this paper we present a new cluster scheduling system, ROSE, that is based on a multi-layered scheduling architecture with an ability to over-subscribe idle resources to accommodate unfulfilled resource requests. ROSE books idle resources in a speculative manner:instead of waiting for resource allocation to be confirmed by the centralized scheduler,it requests intelligently to launch tasks within machines according to their suitability to oversubscribe resources. A threshold control with timely task rescheduling ensures fully-utilized cluster resources without generating potential tasks tragglers. Experimental results show that ROSE can almost double the average CPU utilization, from 36.37% to 65.10%, compared with a centralized scheduling scheme, and reduce the workload makespan by 30.11%, with an 8.23% disk utilization improvement over other scheduling strategies.",
author = "Xiaoyang Sun and Chunming Hu and Renyu Yang and Peter Garraghan and Tianyu Wo and Jie Xu and Jianyong Zhu and Chao Li",
year = "2018",
month = jul,
day = "2",
doi = "10.1109/ICDCS.2018.00096",
language = "English",
series = "2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS)",
publisher = "IEEE",
pages = "949--960",
booktitle = "38th IEEE International Conference on Distributed Systems Computing Systems",

}

RIS

TY - GEN

T1 - ROSE

T2 - Cluster Resource Scheduling via Speculative Over-subscription

AU - Sun, Xiaoyang

AU - Hu, Chunming

AU - Yang, Renyu

AU - Garraghan, Peter

AU - Wo, Tianyu

AU - Xu, Jie

AU - Zhu, Jianyong

AU - Li, Chao

PY - 2018/7/2

Y1 - 2018/7/2

N2 - A long-standing challenge in cluster scheduling is to achieve a high degree of utilization of heterogeneous resources in a cluster. In practice there exists a substantial disparity between perceived and actual resource utilization. A scheduler might regard a cluster as fully utilized if a large resource request queue is present, but the actual resource utilization of the cluster can be in fact very low. This disparity results in the formation of idle resources, leading to inefficient resource usage and incurring high operational costs and an inability to provision services. In this paper we present a new cluster scheduling system, ROSE, that is based on a multi-layered scheduling architecture with an ability to over-subscribe idle resources to accommodate unfulfilled resource requests. ROSE books idle resources in a speculative manner:instead of waiting for resource allocation to be confirmed by the centralized scheduler,it requests intelligently to launch tasks within machines according to their suitability to oversubscribe resources. A threshold control with timely task rescheduling ensures fully-utilized cluster resources without generating potential tasks tragglers. Experimental results show that ROSE can almost double the average CPU utilization, from 36.37% to 65.10%, compared with a centralized scheduling scheme, and reduce the workload makespan by 30.11%, with an 8.23% disk utilization improvement over other scheduling strategies.

AB - A long-standing challenge in cluster scheduling is to achieve a high degree of utilization of heterogeneous resources in a cluster. In practice there exists a substantial disparity between perceived and actual resource utilization. A scheduler might regard a cluster as fully utilized if a large resource request queue is present, but the actual resource utilization of the cluster can be in fact very low. This disparity results in the formation of idle resources, leading to inefficient resource usage and incurring high operational costs and an inability to provision services. In this paper we present a new cluster scheduling system, ROSE, that is based on a multi-layered scheduling architecture with an ability to over-subscribe idle resources to accommodate unfulfilled resource requests. ROSE books idle resources in a speculative manner:instead of waiting for resource allocation to be confirmed by the centralized scheduler,it requests intelligently to launch tasks within machines according to their suitability to oversubscribe resources. A threshold control with timely task rescheduling ensures fully-utilized cluster resources without generating potential tasks tragglers. Experimental results show that ROSE can almost double the average CPU utilization, from 36.37% to 65.10%, compared with a centralized scheduling scheme, and reduce the workload makespan by 30.11%, with an 8.23% disk utilization improvement over other scheduling strategies.

U2 - 10.1109/ICDCS.2018.00096

DO - 10.1109/ICDCS.2018.00096

M3 - Conference contribution/Paper

T3 - 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS)

SP - 949

EP - 960

BT - 38th IEEE International Conference on Distributed Systems Computing Systems

PB - IEEE

ER -