PTfS26: Assignment 9: Code optimization, performance counters, barriers

Opened: Tuesday, 30 June 2026, 12:00 PM

Due: Wednesday, 15 July 2026, 10:05 AM

Optimizing loop code. Look at the following loop code.
```
int N = 16384;
double mat[N][N],s[N][N];
// ...
srand(1);  // random seed
for(i=0; i<N ; ++i) { 
   val = (double)rand(); // random number
   for(j=0; j<N; ++j) {
      mat[j][i] = s[j][i] * sqrt(val);
   }
 }
```
(a) (15 credits) What is wrong with this code (in terms of performance)? Suggest optimizations that will improve the performance (but will leave the numerical results intact, of course)! Which of them do you think may be employed by a compiler automatically?
(b) (10 credits) Construct a Roofline model for your optimized code for one socket of the Fritz cluster! You know the memory bandwidth and that a sqrt operation takes 3 cycles. According to your model, what is the expected upper performance limit and what is the expected bottleneck?
Hardware performance counters. Performance counters let us count certain events in the hardware. Here you will investigate the data transfers through the cache hierarchy for the "STREAM Triad" loop: (a[:] = b[:] + s * c[:]).

Note: Make sure to allocate your job with --constraint=hwperf to be able to use the performance counters!

(a) (10 crd) You can use your code from an earlier assignment as a starting point. Parallelize the benchmark loop with OpenMP and instrument it with LIKWID marker calls. Place the marker calls outside of the repetition loop (why?)
(b) (20 crd) Run the code with 1 thread and with 18 threads, using a loop length of 10⁸ on one ccNUMA domain of a Fritz CPU. Observe the overall data traffic to and from memory for both cases (using the likwid-perfctr MEM group). Does the measured data volume match your expectations? What is the qualitative difference between the two cases? What is the reason for this?
Hint: It helps to break down the traffic in terms of bytes per loop iteration.
Note: Do not be alarmed by the fact that almost all cores have zero event counts for memory traffic. This is because there is only one counter for these events, and one core gets to count all the events.
(c) (10 crd) For the same problem size and 18 threads, measure the data traffic to and from the L3 and L2 caches and compare with the memory traffic. What is so special about the L3 cache?
Barrier pain. Barrier synchronization is a major performance problem in many OpenMP programs. Here we want to quantify the cost of a barrier.

(a) (20 crd) Write a benchmark that measures the time it takes to execute an OpenMP barrier in cycles. You are free to choose any strategy you like. Surprise us, but make sure you check your results for plausibility (e.g., the barrier cost should generally go up with increasing thread count).
(b) (15 crd) Plot the barrier time in cycles on 1..72 cores on one Fritz node using the default Intel compiler and a recent GNU compiler (available as a module "gcc/14.2.0"). Use the "-O3" optimization level. Note that you need the option "-fopenmp" to activate OpenMP with gcc. Set the clock frequency to 2 GHz to ensure comparability of results.