High Performance Computing — University of Basel, FS2026

SciCORE HPC Cluster · NVIDIA L40S / A100

A comprehensive study of parallel computing across four algorithms, six parallelisation strategies, and two GPU architectures — with real benchmark results from a production university HPC cluster.

---

What's in this repo

Directory	Contents
index.html	Interactive results dashboard (GitHub Pages)
src/matmul/	Matrix multiplication — Seq, OpenMP, MPI, CUDA, OpenACC, Hybrid
src/nbody/	N-Body simulation — Seq, OpenMP, MPI, CUDA, OpenACC, Hybrid
src/merge_sort/	Merge sort — Sequential, OpenMP, MPI
src/emd/	Earth Mover's Distance — Seq, OpenMP, MPI, CUDA, MPI+CUDA
slurm/	Production Slurm job scripts used on SciCORE
docker/	CPU-fallback demo runnable without an HPC account

---

Results at a Glance

Matrix Multiplication (4096 × 4096, -O3)

Method	Runtime	Speedup
Sequential	53.65 s	1×
OpenMP 16 threads	8.54 s	6.3×
MPI 16 processes	5.47 s	9.8×
OpenACC (L40S GPU)	1.064 s	50×
CUDA Tiled (L40S)	0.0985 s	545×
MPI + CUDA (8×GPU)	0.0126 s	🏆 4,245×

N-Body Gravitational Simulation (N = 5,000 particles, 50 steps)

Method	Runtime	GFLOP/s	Speedup
Sequential	~7.74 s	3.2	1×
OpenMP 20T	0.273 s	91.6	18.6×
CUDA L40S (N=10K)	0.874 s	228.7	35×
CUDA L40S (N=50K)	8.73 s	572.9	—

Earth Mover's Distance (N = 100,000,000 samples, compute phase)

Method	Compute Time	Speedup
OMP 1 thread	0.172 s	1×
OMP 64 threads	0.026 s	6.6×
MPI 256 ranks	0.0032 s	54×
CUDA A100	0.00235 s	🏆 73×

Merge Sort (N = 1,000,000 integers) + Energy

Method	Time	Speedup	Energy
Sequential	0.109 s	1×	—
OpenMP 16T	0.028 s	3.87×	8.74 J
MPI 8P	0.131 s	0.83×	799 J

OpenMP uses 91× less energy than MPI for this problem size.

---

Architecture Overview

Four Kernels × Six Parallelism Models
══════════════════════════════════════════════
              │ Seq │ OMP │ MPI │ CUDA │ OAcc │ Hybrid
──────────────┼─────┼─────┼─────┼──────┼──────┼───────
MatMul        │  ✓  │  ✓  │  ✓  │  ✓   │  ✓   │ OMP+MPI, CUDA+MPI
N-Body        │  ✓  │  ✓  │  ✓  │  ✓   │  ✓   │ OMP+MPI
Merge Sort    │  ✓  │  ✓  │  ✓  │      │      │
EMD           │  ✓  │  ✓  │  ✓  │  ✓   │      │ CUDA+MPI

---

SciCORE Cluster

All benchmarks ran as Slurm batch jobs on the University of Basel's SciCORE cluster:

Partition	Hardware	Used for
l40s	NVIDIA L40S (44 GB, SM 8.9)	MatMul CUDA/OpenACC, N-Body CUDA
a100	NVIDIA A100 (80 GB, SM 8.0)	EMD CUDA
scicore	AMD EPYC / Intel Xeon	OpenMP + MPI jobs

Software stack: GCC 13.3, OpenMPI 5.0.7, CUDA 12.6/13.1, NVIDIA nsys profiler, PMT for energy measurement (RAPL + NVML).

---

Run Locally (No HPC Account Needed)

Docker (easiest)

git clone https://github.com/Denxhinjo/hpc-parallel-computing-unibasel
cd hpc-parallel-computing-unibasel
docker compose up --build

Runs matrix multiplication on your CPU in sequential + OpenMP modes and prints a speedup comparison.

Native (Linux / WSL2)

# Install dependencies
sudo apt install build-essential libopenmpi-dev

# Matrix multiplication benchmark
cd src/matmul && make all && ./run_local.sh

# Merge sort
cd src/merge_sort && make all
./merge_sort_omp 1000000 16  # 1M elements, 16 threads

# N-Body
cd src/nbody && make seq omp
./nbody_omp -n 5000 -s 100 -dt 0.01

---

Key Observations

When CUDA wins (and when it doesn't)

• CUDA dominates compute-bound kernels (N-Body, MatMul) — up to 545× speedup
• For EMD, CUDA compute is 73× faster, but GPU setup (host→device transfer + GPU sort) takes 1.66 s vs OMP setup of 0.49 s. MPI at 256 ranks wins end-to-end (0.009 s)

MPI vs OpenMP for sorting

• Merge Sort is communication-unfriendly: MPI scatter/gather moves O(N) data and is actually slower than sequential at N=1M
• OpenMP task parallelism maps naturally onto divide-and-conquer; 3.87× speedup with 16 threads

Energy efficiency

• OpenMP uses 91× less energy than MPI for merge sort at N=1M
• GPU energy efficiency (J/op) dominates at large N for compute-bound work

Hybrid parallelism

• MPI+CUDA achieves 4,245× over sequential on MatMul — combining node-level (MPI) and device-level (CUDA) parallelism
• Requires careful data partitioning: each MPI rank owns a strip of rows, each GPU handles that strip

---

Profiling

NVIDIA Nsight Systems (nsys) was used to profile GPU kernels:

nsys profile --output profile.nsys-rep ./nbody_cuda -n 10000 -s 100
nsys stats profile.nsys-rep

Energy measured with PMT (Performance Measurement Tools):

• CPU: Intel RAPL (Running Average Power Limit)
• GPU: NVML (NVIDIA Management Library)

---

Citation / Acknowledgement

Benchmarks conducted on the SciCORE scientific computing core facility at the University of Basel.
Hardware: NVIDIA L40S (SMd 8.9), NVIDIA A100 (SM 8.0), AMD EPYC.
Software: CUDA 12.6/13.1, GCC 13.3, OpenMPI 5.0.7.