Course on Parallel I/O, organized by NHR@Göttingen
Topic: Holistic HPC I/O Evaluation (H2IO)
Agenda:
Evening program for own design
Lecturers
Kevin
Lüdemann (GWDG, IO)
Mohammad Hossein Biniaz (GWDG, ML)
This day
will provide an introduction to storage types with an emphasis on parallelism.
Participants will get to know distributed filesystems and experiment with
storage API on them. Again, strong emphasis will be on how to optimize an
application to utilize the underlying system to its maximum. Example applications will include standard MPI applications, but special attention will be
given to machine learning-type applications.
After the lunch break, benchmarking and performance engineering will be
introduced and directly used to analyse the popular benchmarking suit IO500. The knowledge gained will be applied to a theoretical roofline model and
compared with data collected before the course from several NHR systems.
The course is rounded off by exploring alternative storage APIs like MPI-IO,
HDF5, and are compared with the standard API called POSIX.
Pre-Requisites
A computer or laptop with SSH access to the cluster is required to
take part in the hands-on section of this course.
Time series and benchmark results of the full system will be provided, and a
scaled down version of the IO500 will be set up for running the highly IO-intensive benchmark.
Update 20.05.2024
The content for this day is uploaded except for the part we are developing new. These slides will change in appearance, but the content stays.
Update 05.06.2024
We updated the timetable since we dropped the analysis tool section and increased the API part to include a longer hands on section. Additionally, we drop all the breaks, since there is no time to waste ... Just kidding (maybe). Further updates of the content will be done in the following days.
Update 08.06.2024
We updated the API slides. For the exercise, you need to know python. It is also enough if you know someone, who is there and know python. We will work in groups.
