Course: Core-Level Performance Engineering

General

While many developers put a lot of effort into optimizing large-scale parallelism, they often neglect the importance of an efficient serial code. Even worse, slow serial code tends to scale very well, hiding the fact that resources are wasted because no definite hardware performance limit (“bottleneck”) is exhausted. This tutorial conveys the required knowledge to develop a thorough understanding of the interactions between software and hardware on the level of a single CPU core and the lowest memory hierarchy level (the L1 cache). We introduce general out-of-order core architectures and their typical performance bottlenecks using modern x86-64 (Intel Ice Lake) and ARM (Fujitsu A64FX) processors as examples. We then go into detail about x86 and AArch64 assembly code, specifically including vectorization (SIMD), pipeline utilization, critical paths, and loop-carried dependencies. We also demonstrate performance analysis and performance engineering using the Open-Source Architecture Code Analyzer (OSACA) in combination with a dedicated instance of the well-known Compiler Explorer. Various hands-on exercises will allow attendees to make their own experiments and measurements and identify in-core performance bottlenecks. Furthermore, we show real-life use cases to emphasize how profitable in-core performance engineering can be.

Prerequisites:
a) It is recommended for attendees to have a basic understanding of the Roofline model. You can find some information here (lecture slides) and here (publication by S. Williams).
b) It is recommended for attendees to have some experience in using the Compiler Explorer. You can find a 30 min tutorial video (two parts) here:

Part 1

Part 2

This a half-day tutorial at SC24 in Atlanta, GA.

Lecturers: Jan Laukemann and Dr. Georg Hager

Course date: November 18, 2024

Course program:

Introduction
Basic processor and core architecture
- Intel Ice Lake (Server) architecture
- Scheduling in an out-of-order backend
Terminology and code execution on out-of-order CPUs
- Throughput, Latency, Critical Path and Loop-carried Dependencies
x86 ISA introduction
- Understanding scalar and vectorized assembly code
Performance analysis of simple kernels
- STREAM Triad
- Dot product
- PI by integration
OSACA introduction
- How to use OSACA
- How to use the Compiler Explorer
- Analyze kernels using OSACA to find potential bottlenecks
AArch64 ISA introduction
In-core performance engineering using OSACA
- Sparse Matrix-Vector (SpMV) Multiplication on A64FX
- Lattice Quantum Chromodynamics (QCD) on A64FX (optional)
- 2D Gauss-Seidel on ICX