Skip to content

COORDINATION without a neighbor list: faster, threaded, thread-indepe… - #1642

Merged
carlocamilloni merged 3 commits into
masterfrom
coordination-nolist
Nov 18, 2026
Merged

carlocamilloni merged 3 commits into
masterfrom
coordination-nolist

Conversation

@carlocamilloni

Copy link
Copy Markdown
Member

…ndent

CoordinationBase (COORDINATION, DHENERGY, GHBFIX):

  • pairCutoff2(): pairs beyond the cutoff of the switching function are skipped before pairing(); derivatives and virial are added only for the pairs with a nonzero derivative.
  • Without a neighbor list the pairs are taken from the groups in blocks of rows (the larger group), without storing the list of all the pairs. Each block adds the derivatives of its rows directly and stores those of the columns, which are summed over the blocks in order: the result does not depend on the number of threads.
  • With NLIST/NLISTCELLS/PAIR, the per-thread buffers are kept between steps and summed in parallel, in the order of the threads, instead of in a critical section.

Core: for actions with many atoms, retrieveAtoms, setForcesOnAtoms (when no atom is requested twice), checkForForces (few values with many derivatives) and clearDerivatives use the threads; the results are identical.

Claude-Session: https://claude.ai/code/session_016vwxtWQypT9j2SfDguLb95

COORDINATION without a neighbour list (2026-11-17)

Follows coordination-nolist-kickoff.md.

Status:

  • Implemented the four changes proposed after the profile (listed in "What changed"). Carlo agreed to results independent of the thread count, and to the core threading in this branch.
  • Committed as 168703a on branch coordination-nolist (from master 33b78e7) in the Mac worktree Claude outputs/coordination-nolist/wt; not pushed. The same diff is in Claude outputs/coordination-nolist/coordination-nolist.patch.

Setup

  • Mac worktree: Claude outputs/coordination-nolist/wt, branch coordination-nolist from master 33b78e7 (= origin/master; openaccPTM-multigpu not merged yet).
  • Cloud build:
    • a clone of GitHub master built with -O2 -g -fno-omit-frame-pointer, CXX="ccache mpicxx", --enable-modules=all --disable-doc --disable-python;
    • about 35 min on 2 cores; a change to core/Value.h rebuilds almost everything (~25 min);
    • master kernel saved for A/B with PLUMED_KERNEL=... plumed-runtime driver (lost with the container).
  • perf in the cloud: /usr/lib/linux-tools/6.8.0-149-generic/perf (apt-get install linux-tools-generic; the perf wrapper refuses the running kernel). Profile with plumed benchmark --ixyz conf --nsteps N and stacks.py (root PlumedMain::performCalc); perf annotate -l -s "<symbol> [clone ._omp_fn.0]" for line level. Leaf libc calls (memset) lose their direct caller with frame-pointer stacks.
  • Bench: Claude outputs/coordination-nolist/bench/run.sh (COORDINATION + RESTRAINT, driver timers, best of 3; configs from neighbor-sums/bench-one-row/gen.py).

Results (cloud, 2 cores, -O2, fwd+bwd ms/step with forces, best of 3; master → branch)

input | pairs | atoms | 1 thread | 2 threads -- | -- | -- | -- | -- 1 atom vs 10k | 10k | 10k | 0.33 → 0.22 | 0.29 → 0.18 1 atom vs 100k | 100k | 100k | 4.92 → 3.66 | 5.03 → 1.97 200 × 500 | 100k | 700 | 1.90 → 0.85 | 1.02 → 0.46 GROUPA 500 | 125k | 500 | 2.36 → 1.07 | 1.25 → 0.56 1000 × 2000 | 2M | 3000 | 39.3 → 18.6 | 21.0 → 9.4 NLIST, 1 vs 100k, stride 10 |   |   | 0.058 → 0.055 | 0.125 → 0.059 NLIST, 1000 × 2000, stride 10 |   |   | 2.49 → 2.14 | 1.28 → 1.13
  • With a stride (skin), the lists always win: from 1.6× in the most compact case to 60× for one atom against 100k.
  • Against the exact list rebuilt every step (cells1), no list wins for one atom against many (3×), and is about even for 100×100.
    • cells1 with one atom against 100k is 10.9 ms on master too: the cells of all the atoms are built every step.
    • cellsMinimumGroup on openaccPTM-multigpu (no cells for few rows) should address it; not checked.

Next

  • Committed on the Mac: 168703a on coordination-nolist (from master 33b78e7), Carlo's identity + Co-Authored-By. Not pushed.
  • Leonardo 8/32 threads for c1-100k and for 1000×2000.
  • Optional: vectorizable orthorhombic Pbc::apply.
# COORDINATION without a neighbour list (2026-11-17)

Follows [coordination-nolist-kickoff.md](coordination-nolist-kickoff.md).

Status:

  • Implemented the four changes proposed after the profile (listed in "What changed"). Carlo agreed to results independent of the thread count, and to the core threading in this branch.
  • Committed as 168703a on branch coordination-nolist (from master 33b78e7) in the Mac worktree Claude outputs/coordination-nolist/wt; not pushed. The same diff is in Claude outputs/coordination-nolist/coordination-nolist.patch.

Setup

  • Mac worktree: Claude outputs/coordination-nolist/wt, branch coordination-nolist from master 33b78e7 (= origin/master; openaccPTM-multigpu not merged yet).
  • Cloud build:
    • a clone of GitHub master built with -O2 -g -fno-omit-frame-pointer, CXX="ccache mpicxx", --enable-modules=all --disable-doc --disable-python;
    • about 35 min on 2 cores; a change to core/Value.h rebuilds almost everything (~25 min);
    • master kernel saved for A/B with PLUMED_KERNEL=... plumed-runtime driver (lost with the container).
  • perf in the cloud: /usr/lib/linux-tools/6.8.0-149-generic/perf (apt-get install linux-tools-generic; the perf wrapper refuses the running kernel). Profile with plumed benchmark --ixyz conf --nsteps N and stacks.py (root PlumedMain::performCalc); perf annotate -l -s "<symbol> [clone ._omp_fn.0]" for line level. Leaf libc calls (memset) lose their direct caller with frame-pointer stacks.
  • Bench: Claude outputs/coordination-nolist/bench/run.sh (COORDINATION + RESTRAINT, driver timers, best of 3; configs from neighbor-sums/bench-one-row/gen.py).

Results (cloud, 2 cores, -O2, fwd+bwd ms/step with forces, best of 3; master → branch)

input pairs atoms 1 thread 2 threads
1 atom vs 10k 10k 10k 0.33 → 0.22 0.29 → 0.18
1 atom vs 100k 100k 100k 4.92 → 3.66 5.03 → 1.97
200 × 500 100k 700 1.90 → 0.85 1.02 → 0.46
GROUPA 500 125k 500 2.36 → 1.07 1.25 → 0.56
1000 × 2000 2M 3000 39.3 → 18.6 21.0 → 9.4
NLIST, 1 vs 100k, stride 10 0.058 → 0.055 0.125 → 0.059
NLIST, 1000 × 2000, stride 10 2.49 → 2.14 1.28 → 1.13
  • Sharing data (0.3–0.5 ms at 100k atoms) is not included in the table and is unchanged.
  • Per pair: 19 → 8.5 ns.
  • Pbc::apply is now half of the per-pair cost (NOPBC: 4 ns/pair). On this gcc the orthorhombic loop is scalar (cvttsd2si/cvtsi2sd, x+=100 trick), also when Pbc.cpp is built with -O3. A possible separate item: a vectorizable rounding, which would change the last digits everywhere.

What changed

  1. Pair kernel (CoordinationBase, all CVs):
    • virtual double pairCutoff2() const, +inf by default. Coordination returns dmax²·(1+1e-12), a margin for the rounding of dmax² and of the sqrt in calculateSqr. GHBFIX returns its dmax_squared.
    • Pairs beyond the cutoff are skipped before the virtual pairing. Derivatives and virial are added only when dfunc!=0.
  2. No list (calculateBlocks): no NeighborList is built, so the pairs are no longer stored (8 B/pair).
    • The rows are the larger group (or GROUPA for a single group: pairs i<j). Blocks have rowsPerBlock = max(16, ceil(512/ncols)) rows; it is raised if nblocks·ncols > 2^20.
    • Each block adds its rows' derivatives to the Value and writes its column contributions to blockColumns[b]. The columns are then summed over the blocks in order (parallel over the columns), and value and virial per block are summed in order.
    • Results identical for any number of threads; vs master, last-digit differences (≤ 7e-15 relative).
    • The self-pair test runs only if the groups share atoms (checked once).
    • MPI: a contiguous range of blocks per rank, mpiDeriv + comm.Sum.
  3. With NLIST/NLISTCELLS/PAIR (calculateList): persistent per-thread buffers summed in thread order, in parallel over the atoms, instead of the critical.
    • threadUsed copes with a smaller team (without OpenMP getNumThreads() is 2 but the team has 1 thread).
    • Still depends on the number of threads; run to run it is reproducible.
  4. Core:
    • ActionAtomistic::retrieveAtoms: the positions are copied with threads (≥ 2×16384 atoms per group).
    • ActionAtomistic::setForcesOnAtoms: the forces are added with threads, only if no atom is requested twice. The new member atomsAreUnique is set in requestAtoms from unique before/after removeDuplicates.
    • ActionWithValue::checkForForces: with few values and ≥ 16 chunks of 4096 derivatives, the threads share the derivatives. The same operations as the serial loop are used, so the results are identical.
    • ActionWithValue::clearDerivatives: values fewer than threads → Value::clearDerivatives(force, nt) (new, in Value.cpp, with OpenMP in the .cpp and not in the header).
  • CHANGES/v2.11.md: two entries in "Core, performance and input".

Verification

  • Cloud, -O2 with MPI and OpenMP, PLUMED_NUM_THREADS=2: all 1169 regtests pass.
  • clang without OpenMP and without MPI, -O1: basic, colvar, symfunc, adjmat and multicolvar (606 tests) pass, with no new warnings.
  • Bitwise thread independence checked with %.17g:
    • 10 small inputs: two groups, overlapping groups, a duplicated atom, a single group, PAIR, NOPBC, legacy R_0, NLIST, NLISTCELLS;
    • full MD forces (--dump-forces) of 1 atom vs 100k: identical on 1 and 2 threads.
  • NLIST differs between 1 and 2 threads only in the last digits (expected).

Profile before the change (master)

  • ab (200×500, pair dominated): 65% loop body, 22% Pbc::apply. The Tensor outer product and the derivative updates ran for all pairs, but only 2.4% of the pairs are within D_MAX.
  • c1-100k:
    • 1 thread: about half of the forward time was O(N_atoms) (deriv zero/copy, Colvar::apply serial, retrieveAtoms);
    • 2 threads: zeroing omp_deriv took 23% and the critical 21%.

No list vs NLIST / NLISTCELLS (branch, cloud, fwd+bwd ms/step, 1t / 2t)

D_MAX=1.2; NLIST and NLISTCELLS with NL_CUTOFF=1.5 NL_STRIDE=10; "cells1" = NLISTCELLS NL_CUTOFF=1.2 NL_STRIDE=1 (exact, no skin).

input no list NLIST s10 NLISTCELLS s10 cells1
1 vs 10k 0.21 / 0.18 0.022 / 0.022 0.022 / 0.022 0.63 / 0.49
1 vs 100k 3.5 / 1.9 0.054 / 0.053 0.055 / 0.054 10.7 / 7.2
50 vs 100k 42.9 / 23.2 1.54 / 1.32 1.53 / 1.31 11.9 / 7.5
100 × 100 0.091 / 0.055 0.015 / 0.016 0.015 / 0.016 0.060 / 0.051
200 × 500 0.86 / 0.46 0.11 / 0.081 0.11 / 0.081 0.27 / 0.21
GROUPA 500 1.07 / 0.55 0.13 / 0.077 0.13 / 0.077 0.50 / 0.33
1000 × 2000 18.4 / 9.4 2.14 / 1.12 1.99 / 1.04 2.52 / 1.47
GROUPA 3000 42.0 / 21.2 4.80 / 2.45 4.50 / 2.28 6.45 / 3.77
compact 229×229 (1.5 nm sphere, 47% of pairs within 1.5) 0.88 / 0.47 0.56 / 0.29 0.53 / 0.28 0.64 / 0.39
compact 1089×1089 (2.5 nm sphere) 13.2 / 6.7 4.0 / 2.0 3.7 / 1.9 4.0 / 2.3
  • With a stride (skin), the lists always win: from 1.6× in the most compact case to 60× for one atom against 100k.
  • Against the exact list rebuilt every step (cells1), no list wins for one atom against many (3×), and is about even for 100×100.
    • cells1 with one atom against 100k is 10.9 ms on master too: the cells of all the atoms are built every step.
    • cellsMinimumGroup on openaccPTM-multigpu (no cells for few rows) should address it; not checked.

Next

  • Committed on the Mac: 168703a on coordination-nolist (from master 33b78e7), Carlo's identity + Co-Authored-By. Not pushed.
  • Leonardo 8/32 threads for c1-100k and for 1000×2000.
  • Optional: vectorizable orthorhombic Pbc::apply.

carlocamilloni and others added 2 commits November 17, 2026 22:19
…ndent

CoordinationBase (COORDINATION, DHENERGY, GHBFIX):
- pairCutoff2(): pairs beyond the cutoff of the switching function are
  skipped before pairing(); derivatives and virial are added only for the
  pairs with a nonzero derivative.
- Without a neighbor list the pairs are taken from the groups in blocks of
  rows (the larger group), without storing the list of all the pairs. Each
  block adds the derivatives of its rows directly and stores those of the
  columns, which are summed over the blocks in order: the result does not
  depend on the number of threads.
- With NLIST/NLISTCELLS/PAIR, the per-thread buffers are kept between steps
  and summed in parallel, in the order of the threads, instead of in a
  critical section.

Core: for actions with many atoms, retrieveAtoms, setForcesOnAtoms (when no
atom is requested twice), checkForForces (few values with many derivatives)
and clearDerivatives use the threads; the results are identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016vwxtWQypT9j2SfDguLb95
The value and the virial were accumulated through references (in the
lambda of calculateBlocks) or in a Tensor in memory, one store and load per
pair. They are now summed per batch in local variables, with the six
components of the symmetric virial. Without D_MAX (all the pairs computed),
the loop is now 6% faster than on master instead of 3% slower; NLIST is
6-10% faster. The virial changes in the last digits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016vwxtWQypT9j2SfDguLb95
Comment thread src/colvar/CoordinationBase.cpp Fixed
Comment thread src/colvar/CoordinationBase.cpp Fixed
@codecov

codecov Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.53552% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.97%. Comparing base (33b78e7) to head (1f9da95).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
src/colvar/CoordinationBase.cpp 96.29% 5 Missing ⚠️
src/core/ActionWithValue.cpp 58.33% 5 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##           master    #1642    +/-   ##
========================================
  Coverage   88.97%   88.97%            
========================================
  Files         651      652     +1     
  Lines       81815    81939   +124     
========================================
+ Hits        72793    72908   +115     
- Misses       9022     9031     +9     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

The local reference to the Pbc was unused in the instantiations without
pbc (CodeQL).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016vwxtWQypT9j2SfDguLb95
@carlocamilloni
carlocamilloni merged commit d6ad776 into master Nov 18, 2026
66 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants