Skip to content

STRIDE - BUG FIX - Allocate threadprivate spline work arrays per thread instead of COPYIN in sing_asymp - #368

Open
SimonPinches wants to merge 1 commit into
PrincetonUniversity:developfrom
SimonPinches:stride-threadprivate-copyin
Open

SimonPinches wants to merge 1 commit into
PrincetonUniversity:developfrom
SimonPinches:stride-threadprivate-copyin

Conversation

@SimonPinches

Copy link
Copy Markdown
Collaborator

Fixes #273.

The sing_asymp parallel region received its THREADPRIVATE spline work arrays (sq_s_f*, fmats_s_f*, locstab_s_f) through COPYIN. Their values are never used, since every spline evaluation overwrites them. The COPYIN was only relied on, implicitly, to allocate them on threads of the nested vac_parallel team that didn't run sing_parallel_alloc.

With ifx, COPYIN of allocated allocatable THREADPRIVATE arrays leaves the threads sharing storage (reproducer in #273). The singular-surface matrices therefore race: di changes from run to run, and STRIDE stops with ZVODE1 integration failure whenever nThreads > 1.

Changes, all in stride/sing.F:

  • new sing_thread_alloc: allocates each of the calling thread's work arrays if not yet allocated (same pattern as ode_run);
  • sing_parallel_alloc calls it, replacing its ALLOCATEs and COPYIN;
  • the sing_asymp region calls it before the loop and copies in only locstab_s_ix, sq_s_ix, fmats_s_ix.

Testing: docs/examples/DIIID_ideal_example, nThreads=32, five runs per compiler:

compiler before after
ifx 2025.2 + MKL ZVODE1 integration failure (5/5) Normal termination, total energy 7.769E-01 (5/5)
gfortran 14.3 + FlexiBLAS 7.806E-01 (5/5) 7.806E-01 (5/5), identical

…ad instead of COPYIN in sing_asymp

The parallel loop over singular surfaces in sing_asymp got its
threadprivate spline work arrays (sq_s_f*, fmats_s_f*, locstab_s_f) via
COPYIN. Their values are never needed (every spline evaluation
overwrites them); COPYIN was only relied on, implicitly, to allocate
them on threads of the nested team that did not take part in
sing_parallel_alloc.

With ifx (2025.2), COPYIN of allocated allocatable threadprivate arrays
leaves the threads sharing storage, so the Mercier/asymptotic matrices
race: di differs from run to run and STRIDE stops with
"ZVODE1 integration failure" whenever nThreads > 1 (PrincetonUniversity#273).

Add sing_thread_alloc, which allocates the calling thread's arrays if
needed (the same pattern ode_run already uses), call it in
sing_parallel_alloc and at the start of the sing_asymp parallel region,
and COPYIN only the spline interval indices.

Tested on the DIIID_ideal_example with nThreads=32, five runs each:
ifx 2025.2 now terminates normally every time with identical energies;
gfortran 14.3 results are bit-identical to before the change.
@matt-pharr
matt-pharr self-requested a review November 13, 2026 15:32
@logan-nc logan-nc added bug Hot Fix Critical bug fix needing urgent attention to merge into both master and develop labels Nov 13, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Hot Fix Critical bug fix needing urgent attention to merge into both master and develop

Projects

None yet

Development

Successfully merging this pull request may close these issues.

STRIDE parallel-FM ZVODE fails to converge at higher thread counts on near-separatrix equilibrium

3 participants