HPC Resource Accounting: Progress Against Allocation—Lessons Learned
LISA: Where systems engineering and operations professionals share real-world knowledge about designing, building, and maintaining the critical systems of our interconnected world.
The LISA conference has long served as the annual vendor-neutral meeting place for the wider system administration community. The LISA14 program recognized the overlap and differences between traditional and modern IT operations and engineering, and developed a highly-curated program around 5 key topics: Systems Engineering, Security, Culture, DevOps, and Monitoring/Metrics. The program included 22 half- and full-day training sessions; 10 workshops; and a conference program consisting of 50 invited talks, panels, refereed paper presentations, and mini-tutorials.
Ken Schumacher, Fermi National Accelerator Laboratory
The stakeholders who fund and oversee our HPC facilities need to know how our resources are utilized. Based on usage, we manage priorities and quotas so all users can get their fair share. Each year we allocate normalized cpu-core hours across our clusters. I will describe our usage reporting including how to incorporate additional ""charges"" for on-line storage, off-line storage and dedicated processors. You will learn how we deal with credits for failed jobs and periods of reduced performance due to load shed events. And I'll describe the usefulness of calculating burn rates for reprioritizing batch queues.
Ken Schumacher, Fermi National Accelerator Laboratory
Ken Schumacher is a 35 year computing professional who has spent the last 17 years at Fermi National Accelerator Lab. He currently helps support several HPC compute clusters, monitoring and reporting resource usage against allocations. Previously he worked with teams supporting lab wide Unix systems, farm and grid systems as well as data storage systems (currently over 300 PB of tape).
Open Access Media
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.
author = {Ken Schumacher},
title = {{HPC} Resource Accounting: Progress Against {Allocation{\textemdash}Lessons} Learned},
year = {2014},
address = {Seattle, WA},
publisher = {USENIX Association},
month = nov
}
