Home > Research > Publications & Outputs > Memory handling in the ATLAS submission system ...

Research

Associated organisational unit

Links

Text available via DOI:

View graph of relations

Memory handling in the ATLAS submission system from job definition to sites limits

Research output: Contribution to Journal/Magazine › Journal article › peer-review

Published

Standard

Memory handling in the ATLAS submission system from job definition to sites limits. / Forti, A. C.; Walker, R.; Maeno, T. et al.
In: Journal of Physics: Conference Series, Vol. 898, No. 3, 052004, 23.11.2017.

Research output: Contribution to Journal/Magazine › Journal article › peer-review

Harvard

Forti, AC, Walker, R, Maeno, T, Love, PA, Rauschmayr, N, Filipcic, A & Di Girolamo, A 2017, 'Memory handling in the ATLAS submission system from job definition to sites limits', Journal of Physics: Conference Series, vol. 898, no. 3, 052004. https://doi.org/10.1088/1742-6596/898/5/052004

APA

Forti, A. C., Walker, R., Maeno, T., Love, P. A., Rauschmayr, N., Filipcic, A., & Di Girolamo, A. (2017). Memory handling in the ATLAS submission system from job definition to sites limits. Journal of Physics: Conference Series, 898(3), Article 052004. https://doi.org/10.1088/1742-6596/898/5/052004

Vancouver

Forti AC, Walker R, Maeno T, Love PA, Rauschmayr N, Filipcic A et al. Memory handling in the ATLAS submission system from job definition to sites limits. Journal of Physics: Conference Series. 2017 Nov 23;898(3):052004. doi: 10.1088/1742-6596/898/5/052004

Author

Forti, A. C. ; Walker, R. ; Maeno, T. et al. / Memory handling in the ATLAS submission system from job definition to sites limits. In: Journal of Physics: Conference Series. 2017 ; Vol. 898, No. 3.

Bibtex

@article{ca19909c47494fac9ac3cd47f30a3c20,
title = "Memory handling in the ATLAS submission system from job definition to sites limits",
abstract = "In the past few years the increased luminosity of the LHC, changes in the linux kernel and a move to a 64bit architecture have affected the ATLAS jobs memory usage and the ATLAS workload management system had to be adapted to be more flexible and pass memory parameters to the batch systems, which in the past wasn't a necessity. This paper describes the steps required to add the capability to better handle memory requirements, included the review of how each component definition and parametrization of the memory is mapped to the other components, and what changes had to be applied to make the submission chain work. These changes go from the definition of tasks and the way tasks memory requirements are set using scout jobs, through the new memory tool developed to do that, to how these values are used by the submission component of the system and how the jobs are treated by the sites through the CEs, batch systems and ultimately the kernel.",
author = "Forti, {A. C.} and R. Walker and T. Maeno and Love, {Peter Allan} and N. Rauschmayr and A. Filipcic and {Di Girolamo}, A.",
note = "Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.",
year = "2017",
month = nov,
day = "23",
doi = "10.1088/1742-6596/898/5/052004",
language = "English",
volume = "898",
journal = "Journal of Physics: Conference Series",
issn = "1742-6588",
publisher = "IOP Publishing Ltd.",
number = "3",

}

RIS

TY - JOUR

T1 - Memory handling in the ATLAS submission system from job definition to sites limits

AU - Forti, A. C.

AU - Walker, R.

AU - Maeno, T.

AU - Love, Peter Allan

AU - Rauschmayr, N.

AU - Filipcic, A.

AU - Di Girolamo, A.

N1 - Content from this work may be used under the terms of the Creative Commons Attribution 3.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.

PY - 2017/11/23

Y1 - 2017/11/23

N2 - In the past few years the increased luminosity of the LHC, changes in the linux kernel and a move to a 64bit architecture have affected the ATLAS jobs memory usage and the ATLAS workload management system had to be adapted to be more flexible and pass memory parameters to the batch systems, which in the past wasn't a necessity. This paper describes the steps required to add the capability to better handle memory requirements, included the review of how each component definition and parametrization of the memory is mapped to the other components, and what changes had to be applied to make the submission chain work. These changes go from the definition of tasks and the way tasks memory requirements are set using scout jobs, through the new memory tool developed to do that, to how these values are used by the submission component of the system and how the jobs are treated by the sites through the CEs, batch systems and ultimately the kernel.

AB - In the past few years the increased luminosity of the LHC, changes in the linux kernel and a move to a 64bit architecture have affected the ATLAS jobs memory usage and the ATLAS workload management system had to be adapted to be more flexible and pass memory parameters to the batch systems, which in the past wasn't a necessity. This paper describes the steps required to add the capability to better handle memory requirements, included the review of how each component definition and parametrization of the memory is mapped to the other components, and what changes had to be applied to make the submission chain work. These changes go from the definition of tasks and the way tasks memory requirements are set using scout jobs, through the new memory tool developed to do that, to how these values are used by the submission component of the system and how the jobs are treated by the sites through the CEs, batch systems and ultimately the kernel.

U2 - 10.1088/1742-6596/898/5/052004

DO - 10.1088/1742-6596/898/5/052004

M3 - Journal article

VL - 898

JO - Journal of Physics: Conference Series

JF - Journal of Physics: Conference Series

SN - 1742-6588

IS - 3

M1 - 052004

ER -