ngspice on Euler produces different result #282

Activity

1. linear commented on Oct 5, 2026

   

   linearbot

   on Oct 5, 2026 – with Linear

   More actions

   SOH-148
2. g-braeunlich commented on Oct 5, 2026

   

   g-braeunlich

   on Oct 5, 2026

   CollaboratorAuthor

   More actions

   @markfuge FYI
3. markfuge commented on Oct 6, 2026

   

   markfuge

   on Oct 6, 2026

   Member

   More actions

   @SoheylM can best comment on this as he has been dealing with the cross platform issues
4. markfuge

   assigned

   SoheylM

   on Oct 6, 2026
5. SoheylM commented on Oct 7, 2026

   

   SoheylM

   on Oct 7, 2026

   Contributor

   More actions

   @g-braeunlich Could you have a look at #279? We have already investigated these backend differences. Our own PowerElectronics v0 comparisons used ngspice 44.2, whereas the Euler module in your report is 44. The v0 regression test also passes in our macOS CI with 44.2.

   For Euler, we compiled ngspice 44.2 from a checksum-verified source archive and packaged it in a fixed Linux x86_64 Apptainer image. We ran the v1 pilot twice with identical outputs, then used the same image to generate the full v1 dataset.

   For canonical v1 runs on Euler, the reference runtime is this image, pinned by its OCI digest and verified with the SIF checksum. This fixes the simulator binary and its dependencies, making the environment reproducible beyond the version number alone. The pull and verification instructions are included in the PR.

   v1 defines a new benchmark with corrected objectives and regenerated labels; the historical v0 reference assertion shown here remains a separate comparison. We have not established that the 44 versus 44.2 difference alone explains this discrepancy.

   The image is already stored in GHCR. I’m waiting for #279 to be merged before making the package public.
6. g-braeunlich commented on Oct 12, 2026

   

   g-braeunlich

   on Oct 12, 2026

   CollaboratorAuthor

   More actions

   Hey @SoheylM I had a look.
   In the meantime, I installed an Euler module for ngspice 44.2 in /cluster/work/fuge/cluster-modules.
   This still produces different versions:

   SUBFAILED[verify objective values] tests/test_problem_implementations.py::test_python_problem_impl[PowerElectronics] - AssertionError: 
   Not equal to tolerance rtol=5e-05, atol=0
   Mismatched elements: 2 / 2 (100%)
   Mismatch at indices:
    [0]: 0.0107207 (ACTUAL), -0.00809848 (DESIRED)
    [1]: 1.304891 (ACTUAL), -0.914915 (DESIRED)
   Max absolute difference among violations: 2.219806
   Max relative difference among violations: 2.42624288
    ACTUAL: array([0.010721, 1.304891])
    DESIRED: array([-0.008098, -0.914915])
   FAILED tests/test_problem_implementations.py::test_python_problem_impl[PowerElectronics] - contains 1 failed subtest
   = 2 failed, 195 passed, 1 skipped, 2 warnings, 23 subtests passed in 3908.67s (1:05:08) =
   
7. SoheylM commented on Oct 13, 2026

   

   SoheylM

   on Oct 13, 2026

   Contributor

   More actions

   Hi @g-braeunlich, following our corridor discussion about FMA, I investigated the discrepancy you reported. Here is what I tested, what I found, and how you can reproduce it.

   I first reproduced your module’s values, then compared it with my original ngspice 44.2 SIF on the same Euler Linux x86_64 compute node, using the exact same v0 design, generated netlist and measurement window. Each backend reproduced its own results across repeated runs:

   • Your module: gain +0.0107207, signed ripple +1.304891.
   • My original SIF: gain −0.00809852, signed ripple −0.914900.

   These are the original v0 definitions, not the proposed v1 objectives. Both executables use ngspice 44.2; this is not a 44-versus-44.2 comparison. Equalizing startup settings, thread count and matrix solver did not remove the discrepancy.

   To separate compiler effects from the different installations, I rebuilt the same 44.2 source archive inside the original container environment, using GCC 12.2.0 and otherwise matching settings:

   Unpatched build	Gain	Signed ripple
   Generic -O2	−0.00809852	−0.914900
   -O2 -march=x86-64-v3, FP contraction enabled	+0.0107207	+1.304891
   Same x86-64-v3 build, -ffp-contract=off	−0.00809852	−0.914900

   The FMA-disabled build reproduced the original SIF’s numeric waveform payload exactly. The FMA-capable build reproduced your printed measurements, although its full waveform was not bit-identical to your GCC 15.2.0 build.

   FMA means fused multiply-add: multiplication and addition/subtraction can be evaluated with one rounding instead of separate roundings. GCC’s -ffp-contract setting controls whether this contraction is permitted. GCC documentation

   The defect is in the periodic PWL phase calculation, not in FMA itself. In vsrcload.c, ngspice uses:

   time -= period * floor(time / period);

   At certain period boundaries, separate rounding makes the wrapped time slightly exceed the final PWL point—by approximately 1.36e-20 s in this case. The subsequent segment search then misses the intended segment.

   That tiny arithmetic difference becomes an actual switching error: my original SIF produces events where the complementary gate commands GS2 and GS4 are both zero, near 545, 625 and 1010 µs. Your module does not show those events for this case. The incorrect switching changes the subsequent circuit trajectory, rather than just the last digits of the reported objectives.

   I also isolated the effect by recompiling only vsrcload.c with FMA-capable flags. That changed the generic build to your printed measurements. However, changing only that file in the opposite direction produced a third trajectory. So simply enabling or disabling FMA is not a validated general fix.

   I prepared a two-file candidate patch using fmod for periodic phase reduction in vsrcload.c and vsrcacct.c. With that patch, both controlled builds—FP contraction enabled and disabled—give your printed measurements for this case, without the gate violations. Synthetic PWL tests and targeted period-boundary checks also pass. This remains a proposed local patch, not an accepted upstream fix.

   This corrects my earlier interpretation: my original SIF passes the historical assertion, but its PWM waveform is defective for this case. A pinned container preserves an execution; it does not prove that execution is correct. Conversely, these checks do not establish that every result from your module is physically correct.

   I placed the exact reproducer on Euler:

   /cluster/work/fuge/smassoudi/issue282-collaboration-20261013.KQFta7
   

   Your account brgerhar has read access to the original ZIP in reference/, and read/write access to editable/issue282-reproducer/.

   For a clean reproduction, extract the ZIP into your own scratch:

   case_root=$(mktemp -d /cluster/scratch/$USER/issue282-repro.XXXXXX)
   unzip /cluster/work/fuge/smassoudi/issue282-collaboration-20261013.KQFta7/reference/geri-issue282-reproducer-validated.zip \
     -d "$case_root"
   cd "$case_root/issue282-reproducer"
   sha256sum -c SHA256SUMS

   Then follow the Slurm commands in README.txt, after loading your 44.2 module and Apptainer. The runner repeats the module/SIF comparison and performs a separately labelled full-transient gate audit on one compute node. No EngiBench/EngiOpt installation or dataset download is needed. The package includes the exact input, patch, logs and build fingerprints, but no patched binary or replacement SIF.

   After the correction, I also ran a targeted 300-design pilot. The PWM checks passed, but 77/300 designs still had an objective difference between the two patched builds beyond the comparison tolerance. Subsequent convergence checks exposed additional sensitivity and transient failures; at least one failure also reproduces with unmodified 44.2 sources. I have not established a second solver bug. A separate ngspice report is drafted but not submitted.

   The existing v0/v1 datasets, original SIF and your module remain unchanged. I am holding off freezing a replacement runtime or generating a new dataset until those remaining issues are understood.

   Could you:

   1. Run the supplied comparison and confirm the module/SIF values and gate events?
   2. Inspect the periodic PWL calculation and proposed patch?
   3. Help assess the correction before I change the production runtime or regression reference?

   There is no need to rerun the full hour-long suite for this first check.