Skip to content

Latest commit

 

History

6 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

HPC Kernel Optimization

Three compute kernels, each optimized step by step with a different parallel-programming model — every version is benchmarked against the same correctness-checked baseline. Best result: scaled-dot-product attention from 241 s to 3.2 s (75.6×) with MPI + cache blocking + AVX-512 on a 4-node cluster.

Kernels

Kernel Optimization stages Tech
attention/ serial → MPI row partitioning → + cache-blocked matmul → + AVX-512 intrinsics MPI, _mm512 FMA intrinsics
cuda-transpose/ naive → shared-memory tiling → + bank-conflict avoidance via padding CUDA, Nsight Compute
conv2d/ serial → OpenMP collapse(2) → + cache tiling OpenMP

Results

Attention (Q·Kᵀ → softmax → ·V), 16384×16384, d=1024

Measured on a 4-node RDMA cluster, 16 processes per node; timings recorded in the source header with correctness verified per run:

Version Time Speedup
serial 240.99 s
MPI 6.68 s 36.1×
MPI + blocked matmul 5.70 s 42.3×
MPI + AVX-512 5.15 s 46.8×
MPI + blocked + AVX-512 3.19 s 75.6×

The report also examines process-count scaling (1→16 processes: 6.3× → 37.7× on the largest input) and why node count does not scale monotonically (communication vs. computation trade-off).

Complex-matrix conjugate transpose (CUDA), N=4096, median of 5 runs

Version Time vs. naive
naive (global memory) 627 µs
shared-memory tiles (32×32) 224 µs 2.8×
+ bank-conflict avoidance (padding) 176 µs 3.6×

run_experiments.sh sweeps 12 matrix sizes × 100 repetitions; Nsight Compute counters (l1tex__data_bank_conflicts_*) confirm the padding removes the 32-way conflicts caused by the 8-byte complex element stride (full analysis in the report).

2D convolution (OpenMP), up to 4096×4096 × 9×9 kernels

3–4× speedup at 4–8 threads on a 4-core CPU; the report analyzes why scaling degrades beyond the physical core count (threading overhead, false sharing) and the effect of cache tiling (measurements in the report).

Build & Run

Each kernel directory has its own Makefile:

# conv2d (gcc + OpenMP)
cd conv2d && make
./conv-openmp MATRIX_FILE KERNEL_FILE ANSWER_FILE

# attention (OpenMPI; AVX-512-capable CPU for the avx512 targets)
cd attention && make
mpirun --hostfile hosts -npernode 16 ./attention-mpi-blocked-avx512 TESTCASE.bin

# cuda-transpose (CUDA toolkit)
cd cuda-transpose && make
./run_experiments.sh          # full benchmark sweep
./ncu-profiling.sh            # Nsight Compute profiling

Test inputs are not committed (course-provided binaries up to 513 MB); input formats are documented in each kernel's source.

Context

Assignment work for High-Performance Big Data and AI Systems at NTU (Spring 2025), taught by Prof. Shih-Hao Hung. The convolution and CUDA-transpose kernels are my individual work. The attention kernel was a four-person group assignment; I was responsible for the AVX-512 and cache-blocking optimizations and the experiments. Cleaned up and documented for release.

About

Step-by-step kernel optimization with OpenMP, MPI + AVX-512, and CUDA — attention kernel from 241s to 3.2s (75.6x) on a 4-node cluster

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages