HPC Cluster Basics

High-Performance-Computing with SLURM at HPI

About this course
This course consists of video recordings made by the former Digital Health Compute Lab (DHC Lab). HPI has since established a central Scientific Compute Infrastructure (SCI), superseding the DHC Lab. The videos still refer to the DHC Lab and its clusters. The concepts they teach apply directly to the current clusters of the SCI and the AI Service Center (AISC). For the exact commands, partitions and settings on these clusters, use the official documentation at docs.sc.hpi.de.
Why is this topic relevant?
When a computation needs more resources than a personal computer can provide, it runs on an HPC cluster. At HPI, the SCI and the AISC operate such clusters. To use them efficiently, every user needs to understand how a cluster works and how to schedule jobs well. This raises the overall throughput of the clusters and gets your results back sooner.
What will participants learn?
SLURM (Simple Linux Utility for Resource Management) is an open-source system for cluster management and job scheduling. This course teaches the basics of using SLURM.
Participants who have not worked with the Linux command line before start with a short introduction to it. The course then covers the basics of SLURM. Later modules cover advanced topics and include master classes on optimizing HPC workloads.
What is it useful for?
The course is not mandatory, but we recommend it to everyone who wants to work with the SCI or AISC compute resources. It covers the basic knowledge that helps you avoid common problems, such as failed or stuck jobs, and makes your work on the shared clusters more efficient.

If you have any questions, contact us at sc-helpdesk@hpi.de.

High-Performance-Computing with SLURM at HPI

Why is this topic relevant?

Whenever computations require more resources than a normal personal computer can supply, workloads are shifted on an HPC cluster. For the cluster to be used efficiently, all researchers need to understand how the cluster works and how their workloads can be scheduled most efficiently. This not only increases the overall throughput, but also speeds up your research!

What will participants learn?

SLURM (Simple Linux Utility and Resource Manager) is an open source cluster management and job scheduling system for compute clusters. In this course, we will teach the basics of how to use it.

For those who never worked with the Linux command line before, we will start with a short overview of how it works. We will then move on to the basics of using the SLURM. Later modules will focus on advanced topics as well as provide master classes in optimizing HPC workloads.

What is it useful for?

To raise the overall efficiency of our clusters, we expect every user to have successfully passed this course before working with our resources. Thereby, we ensure that our entire user base has the same base knowledge and abilities to carry out their work optimally.

If you have any questions, please do not hesitate to contact us via sc-helpdesk@hpi.de - thanks!

This course was developed with contributions from the AI Service Center Berlin-Brandenburg at the Hasso Plattner Institute. The AI Service Center is funded by the Federal Ministry of Research, Technology and Space (BMFTR) under the funding code 16IS22092.