Skip to content

Benchmark compiler optimization flags (clang + g++/gcc); add opt-in OSQP_OPT_FAST_MATH - #4

Merged
mmmarinho merged 2 commits into
mainfrom
benchmark-osqp-optflags
Oct 1, 2026
Merged

mmmarinho merged 2 commits into
mainfrom
benchmark-osqp-optflags

Conversation

@mmmarinho

@mmmarinho mmmarinho commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Benchmarks the compiler optimization flags applied to the solver core (OSQP's ADMM loop + built-in CSC algebra + the qdldl/AMD LDL^t factorizer — the code that dominates solve time) on both clang and g++/gcc, and adds a small, opt-in CMake option for the one flag that can pay off.

This was generated by an AI agent (OpenHands) on behalf of the user.

What was measured

A new bench/ harness compiles exactly the C sources the package links against (OSQP + qdldl, same sources/defines/include layout as the CMake build) with a chosen flag set, then times two deterministic dense-SPD QP workloads (n=800 and n=2500) with cold-started solves. Compiling the C core directly guarantees that only the optimization flags vary between runs. The matrix was run under both Apple clang 21 and Homebrew GCC 16 (gcc-16/g++-16).

Variants: -O0, -O1, -O2, -O3, -Ofast, -O3 -march=native, -O3 -march=native -flto, -Ofast -march=native -flto.

Full results are in BENCHMARK.md. Headline (Apple M2, arm64):

clang 21 vs -O3 GCC 16 vs -O3
-O2 / -O1 ~1.00× (no gain) ~1.00× (no gain)
-O0 ~2.9× slower ~3.6× slower
-march=native (± -flto) ~1.00× (no gain) ~1.00× (no gain)
-Ofast / -ffast-math ~0.70× (≈30% faster) ~1.17× (≈15–18% slower)

All variants produce the same objective values and status; no solution changes.

Key findings

  1. -O3 is already the sweet spot. The vendored osqp/ and qdldl/ submodules force -O3 in their own CMake files, and the wrapper C++ follows the top-level build type (Release → -O3). Both compilers agree there's nothing to gain by changing the level.

  2. -march=native and -flto add nothing measurable on this workload, and -march=native would break manylinux portability of the wheel — so neither is a candidate for a default.

  3. The only effective lever is fast-math — and it is compiler-specific. This is the headline cross-compiler result:

    • clang 21 is ~30% faster with -Ofast/-ffast-math.
    • GCC 16 is ~15–18% slower with the same flags.

    Two subtleties verified experimentally:

    • OSQP/qdldl append an -O3 after any injected flag, and the last -O wins, so a bare -Ofast gets overridden and does nothing. -ffast-math is independent of the -O level and does survive.
    • The flag must reach the vendored C code, not just the wrapper.

    Because fast-math helps clang but hurts GCC, it's a gamble, not a universal win.

  4. It's numerically safe here. OSQP runs to a tolerance; -ffast-math doesn't change the reported solutions. All 46 tests pass on both the default and the fast-math build, and the example prints identical output.

Change

  • CMakeLists.txt: new option OSQP_OPT_FAST_MATH (OFF by default). When ON it appends -ffast-math (or /fp:fast on MSVC) to the base C/CXX flags before add_subdirectory(osqp), so it reaches OSQP, qdldl, and the wrapper. The default build is byte-for-byte the IEEE-conformant -O3 — the shipped wheel and CI are unaffected.
  • BENCHMARK.md: full methodology, per-compiler results, and reproduction steps.
  • bench/: the reproducible harness (osqp_bench.c, build.sh, run.sh, aggregate.py, scan_flags.py).
  • .gitignore: ignore the harness's output dirs (bench/{bin,gen,results}/).

Enable it per-consumer (on a clang toolchain) if the relaxed FP contract is acceptable:

cmake -B build -GNinja -DCMAKE_BUILD_TYPE=Release -DOSQP_OPT_FAST_MATH=ON
# or, for the wheel:
CMAKE_ARGS="-DOSQP_OPT_FAST_MATH=ON" python setup.py bdist_wheel

Note: setup.py reuses its build/temp/ CMake cache, and CMake option() doesn't reset an already-cached variable — delete build/ between wheel builds when flipping this flag.

Validation

  • BENCHMARK.md numbers reproduced twice from a clean checkout (same ratios within noise), under both clang and GCC.
  • python -m pytest from tests/: 46 passed on the default wheel and on the OSQP_OPT_FAST_MATH=ON wheel.
  • example_osqp prints identical results (x = [0.2, 1.0], obj = -0.68) for both builds.
  • CI (ubuntu-latest + ubuntu-24.04-arm, Python 3.12/3.13) builds the wheel successfully with the default (fast-math OFF) configuration.

Murilo Marinho and others added 2 commits October 1, 2026 17:58
Benchmarks g++/gcc/clang optimization levels for the solver core (OSQP
ADMM loop + built-in algebra + qdldl/AMD factorizer) and documents the
findings in BENCHMARK.md, with a reproducible harness under bench/.

Findings (Apple M2 arm64):
- -O3 is already the sweet spot; -O2/-O1 are within noise, -O0 ~3x slower.
- -march=native and -flto give no measurable gain on this workload.
- -ffast-math speeds up the C OSQP core ~25-30% (and ~4% end-to-end
  through the Python wrapper, whose dense->CSC conversion dominates).

Change: add an opt-in OSQP_OPT_FAST_MATH CMake option (OFF by default, so
the shipped wheel stays IEEE-conformant -O3). It appends -ffast-math
(/fp:fast on MSVC) to the base C/CXX flags before add_subdirectory(osqp),
so it reaches the vendored OSQP/qdldl as well as the wrapper. All 46
tests pass on both the default and the fast-math build, with identical
solutions.

Co-authored-by: openhands <openhands@all-hands.dev>
…GCC)

Cross-compiler result: clang 21 is ~30% faster with -Ofast/-ffast-math,
but GCC 16 is ~15-18% slower. This is the strongest argument for keeping
OSQP_OPT_FAST_MATH opt-in and OFF by default.

Co-authored-by: openhands <openhands@all-hands.dev>
@mmmarinho mmmarinho changed the title Benchmark compiler optimization flags; add opt-in OSQP_OPT_FAST_MATH Benchmark compiler optimization flags (clang + g++/gcc); add opt-in OSQP_OPT_FAST_MATH Oct 1, 2026

@mmmarinho mmmarinho left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@mmmarinho
mmmarinho merged commit 2f6b2bb into main Oct 1, 2026
11 checks passed
@mmmarinho
mmmarinho deleted the benchmark-osqp-optflags branch October 1, 2026 18:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant