Repository navigation
processSEG 0.1.0: the processing library - #1
Merged
Merged
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ChunkMeta gains live length, window origin, full geometry and signed offsets; document the ORC/ptr threading rule, plan generations, edge handling, SEG-D Rev 1/2 geometry via SPS, and a fuller P2 signature. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
processSEG.nimble requires the parseSEG branch processseg-integration; nim parseSEGTest runs parseSEG's own suite in deps/parseSEG, and nim parseSEGBranch re-attaches the checkout after atlas install. proposedChanges.md is removed: C1-C8 shipped in parseSEG 0.7.0, and the follow-ups are now processSEG tasks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- src/processseg.nim -> src/processSEG.nim (D9), build/test/docs tasks updated. - `nim fmt` formats every .nim file under src/, tests/ and bench/. - CI fmt check walks subdirectories (globstar) and reports every diff. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- tests/tester.nim compiles and runs each t*.nim in its own process, passing its arguments through as build flags (debug, -d:release, -d:danger, -d:addressSanitizer), and fails listing every failing file. - tests/config.nims holds the shared paths and the ASan switches (-d:useMalloc, -d:noSignalHandler). - Placeholder Context API removed; tumbrella.nim is the smoke test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Signals added in place into var openArray[T]: sinusoid, spike, Ricker, DC, linear drift, seeded Gaussian noise. - SynthGather[T]: parseSEG TraceKeys with geometry (signed offset, source/ receiver/midpoint x, elevations, t0Ms, intervalUs, live status), per-row live lengths, a zero-padded [rows, cols] tensor, and a TraceRows view. - Shot (split-spread, end-on) and CMP gathers; hyperbolic and linear events with optional per-trace shifts; elevation statics. - tests/tsynth.nim self-tests every generator. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- bench/benchutil.nim: warm-up + timed samples (each >= 20 ms), median and minimum per iteration, GB/s; prints CPU, OS, Nim, C compiler and the build flags with every run. - bench/bmemory.nim: copy and in-place scale of a 2000 x 2001 float32 panel. - `nim bench` runs bench/b*.nim in a danger build with recorded flags; results recorded in bench/README.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…data model - ChunkMeta is a zero-copy wrapper over parseSEG's TraceRows (D12): per-row values from the view, liveness rule (tsLive/tsUnknown, not failed), and the three view lifetimes. - Geometry from TraceKeys (inline/crossline from the layout), traceStatics, and keysRevision for anything derived from keys. - Threads: the three ways to run, with the persistent public ProcessPool (D6); batch jobs in the prefetcher transform hook (D13); pure-Nim FFT (D2). - "Proposed parseSEG changes" becomes "parseSEG 0.7.0 and the patch branch" (P1-P4 landed; F1-F3 on processseg-integration). Viewer checklist uses c.rows, uniform = true and keysRevision. Pool open question closed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
parseSEG 6780373 adds keysAt(rows, row), the non-raising form of traceKeys
for {.raises: [].} kernels and ChunkTransforms (tested there; parseSEG's
whole suite passes through `nim parseSEGTest`).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- integration doc: derived offsets (SEG-D Rev 3, SEG-2, survey joins) are unsigned distances, so split-spread side comes from the coordinates; kernels read keys with keysAt (F1 marked done); the adapter's interval check is a pass over the rows, not O(1). - synth: unsignOffsets() models derived offsets; the rows() doc forbids temporaries. - tester: runs the compiler with startProcess (no shell quoting). - tests/config.nims: ASan flags for gcc/clang or MSVC; MinGW is an error. - bench: the CPU target comes from config.nims (-d:cpuTarget) instead of being re-derived; clock read once per doubling batch; a compiler memory barrier after every body run (without it gcc merged batched in-place runs and reported 2x the real bandwidth). - atlas.lock: parseSEG with the review's test fix. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…or tests Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nting and reject tests Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
NaN/Inf rows left out of averaged spectra; always-on FFT buffer length checks; short windows tapered and scaled per row; default-plan assertion; workspaceSize(batched); more lane, NaN and float64 tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ed and benched Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…otch, impulse length Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…; warped FD Butterworth Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dget Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… margin, zeroDc) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ip by histogram TraceStats/traceStatsInto, PanelStats/panelStats (65536-bin float32-bit histogram, 0.4 % relative), exact percentileAbs; Workspace gains a uint32 region. Bench in bench/bgain.nim. Also applies nim fmt to the Phase 3 filter files (layout only). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
scaleDb; NormalizePlan (normMax/normRms/normPercentile, perTrace/perPanel); AgcPlan (agcRms/agcMeanAbs, centred window, real counts at the ends, epsilon relative to the trace RMS, live length, kept gains + removeGain). Window sums from block suffix + prefix sums: no subtraction, exact to a few ulps at any dynamic range, O(n) whatever the window. tvectorized groups a kernel's GCC parts and clones. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GainCurve (knotGain linear in dB and flat outside, powerGain, exponentialGain), GainCurvePlan tabulated on the dt grid; applyGainCurve multiplies each live row by the table at its own times (t0Ms, firstSample), linear between grid points for a fractional shift, the curve itself outside the table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n in CI Workers start once, sleep on a condition variable and are joined by =destroy; =copy and =dup are errors. apply validates on the caller's thread, splits the rows into one range per worker, and runs a per-panel factor on worker 0 between parts. Identical to the single-thread apply for 1-8 workers; ASan and TSan clean. Bench: 6-12 us per job, 7.4x at 8 workers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…out) lmoTime/nmoTime, Moveout (linearMoveout, hyperbolicMoveout), and moveoutTimesInto (ms from a TraceRows view) / moveoutColumnsInto (block columns from a ChunkMeta). Offsets keep their sign in the keys; the formulas use |x|. NaN offsets or t0 give NaN. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… real API tests/utils/viewersim.nim acts out a viewer: uniform prefetch into a raw cache, ProcessPool into processed buffers tagged by generation, a checking draw, slider edits, zoom windows with requiredMargin, moveout ticks tagged with keysRevision and made stale by applySurvey. tviewerloop drives it on synthetic data and parseSEG's demo files: stale buffers always reprocessed, 0 allocations per steady-state frame, ASan clean. The doc now names the real API (ops, Pipeline, apply, pool, moveout) and says what the margin promises: relative to the strongest nearby signal; AGC after a filter lifts the leak in quiet windows; whole-trace ops use the window's statistics. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… pipeline checks) Two fresh reviews: a docs-only API review that wrote its own viewer loop (kept as tests/tapireview.nim) and a diff review. - Processed buffers are stale when the plan's generation or the file's keysRevision changes (mutes read offsets, gain curves t0, every op the status); viewersim tags both and gains refresh(f); new viewer-loop test. - pipeline: factors checked against the sample type; checkApply(src, dst) rejects partial overlaps; runPart/panelFactorOf assert their preconditions (kept public for viewers with their own threads); muteTopOp/muteBottomOp; requiredMargin(flow, dtMs) without compiling; decodeWindow. - pool: workers of a moved-from pool is 0, ping asserts, a failed createThread joins only started threads. - moveout: hyperbolic t0Ms >= 0; moveoutTimesInto(meta, ...). - The umbrella re-exports parseSEG. Integration doc: cached-chunk lifetimes, what allocates, zoom windows on the traces' peak, batch marked planned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
import processSEG no longer re-exports parseSEG: the two packages stay separate, and callers import both (import parseSEG, processSEG). parseSEG is on every build's path through Atlas's nim.cfg. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
models/pool_handoff.nif: two workers, up to two jobs, destroy at any idle point, spurious wake-ups. Every job runs exactly once per worker before the caller returns, and no wake-up is lost. Base passes (235 states), the witness is reachable, and all five mutants fail (no recheck after a wake, wait ignoring stop, caller waiting once, no done signal, signal instead of broadcast). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…L/CSV picks Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ifts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rses, zero knot, negative gradient) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ctional shifts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stretch mute Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…r inline/crossline Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…malization, merge Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… NMO, per-worker stacks, progress and cancel Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…and intervals skipped, live-sized stack, worker early stop, overflow guard, rcvDepth, header statics column) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…: traces from keys) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… offsets, midpoint, per-trace flags Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ppens; minimum-phase band-pass, LMO-guided window, SNR confidence Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… residuals, neighbour cross-correlation, symmetry; swapped receivers flagged Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… t0 picks, degenerate and long-spread fits, midpoint flag out of badGeometry, writeStack sorting and docs, detection limits) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…range values tests/tproperty.nim: random pipelines on random gathers (sizes, dt, live lengths, dead/auxiliary/failed rows, 1e-6..1e6 amplitudes): compile raises only ValueError, apply is finite, leaves non-live samples alone and equals src -> dst; a NaN/Inf sample reaches no other row; FFT and IIR filter linearity; AGC and normalize scale invariance; zero in, zero out; shifts, NMO and first-break picks with extreme parameters; velocity parsers on mutated text. tests/fuzz: libFuzzer + ASan + UBSan targets for the velocity parsers and for pipelines built from bytes (raw float parameters and samples), with seeds and a `nim fuzz` task. UBSan findings fixed: the impulse-length grid of a near-zero ramp (an 18 s run on a garbage FFT size), the NMO taper start for a huge taperMs, and first-break and shot-fit correlation windows of huge finite lengths. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hmark) - first breaks: a band-passed plan filters fftLanes rows at a time through the IIR lanes kernel into workspace copies (was one scalar trace at a time): 5-60 Hz picking 100/86/91 -> 44/31/35 ms per 2000-trace panel; unfiltered plans pick from the rows in place; - NMO: the RMS velocity column is reused across rows with the same start time (and location), and the input columns are computed by a vectorized block loop (moveoutCols, -fno-math-errno in the nmo module): linear 36 -> 17.5 ms, cubic 55 -> 25 ms; - shot fit: l1Fit normal equations in local sums (vectorized), golden section reuses one point per step: 38 -> 21 ms on 2000 traces; - tvectorized checks moveoutCols and l1Fit; bench/README records the run. nim test and nim testRelease pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- README: overview, a compiled quick start, links; - docs/guide/: getting started, pipelines (zooms, the pool, own threads), gain, velocity and stack, QC; every example is a program; - docs/api.md: conventions, then every exported name by module; - tests/tdocs.nim: compiles and runs the guide and README examples in one build, and checks api.md names all 370 exported names (jsondoc); - docs/integration.md: status for the release candidate, QC workspace, and the zoom margin's measured worst case (2.5 %, not 1 %); - doc comments: nim docs is warning-free. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Defects: - mute kernels: opposite infinite bounds made a NaN ramp bound and the boundChecks-off loops ran from int.low (heap overflow under ASan); an infinite taper is now no taper, the column helpers are NaN-safe, and a subnormal dtMs is refused by initChunkMeta; - intervalVelocity with a first time of 0 changed the first layer (a knot at 0 makes Vrms linear from it): the zero-thickness layer is dropped; - float32 overflow to Inf turned zeros into NaN: gain-curve tables and scaleDb refuse such gains, block scale refuses a factor Inf in T, and off-table curve samples keep zeros zero; - FFT/IIR filter and averaged-spectrum block forms now refuse rows whose interval is not the plan's (they filtered at the wrong frequencies); - first breaks: an infinite t0 is no time axis (was a NaN-to-int); - shot fit: golden-section refinement kept within the candidates' range; fitShot no longer infers raises: [Exception] (forward declaration); - rowBlock(data, rows, cols) refuses an overflowing size; decodeWindow takes a negative margin as 0 and cannot overflow; - velocity parsers accept a UTF-8 byte-order mark. API: pick(plan, x, ws, offset = NaN, t0Ms = 0) (ws no longer after the defaults). Docs: zero-mean claim of withZeroDc, asymmetryMs sign, lag choice, MER weight, pool exceptions, perPanel in bruteStack is per chunk, the zero-offset tolerance; doc comments from the docs pass. Tests for each fix, an NMO reference test, allocation-free filtered pickInto. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The patch branch is merged upstream and released as parseSEG 0.8.0 (v0.8.0, 7be5c47); atlas.lock pins it, so CI's atlas rep resolves a pushed commit again. Integration doc: parseSEG section for 0.8.0, and the zoom margin's worst case as an open question for the viewer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nimpretty's default 80 columns rewrapped the 100-column sources and, on one wrap, joined `and p.lagToleranceMs` into `andp.lagToleranceMs`. CI and `nim fmt` now pass --maxLineLen:100; the remaining diffs are comment alignment. tfilterapply's spec list is hoisted out of the for header (a comment inside it crashed nimpretty), and two long test lines are rewrapped by hand. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the CI runner (gcc 15, x86-64-v3) the release build differs from gcc 12 in the last bits: tpipeline's plan-vs-ops check asked for bit equality (off by 6.7e-16), and tproperty's NaN-containment check compared a near-zero sample (2e-11) relative to itself. Both tolerances are now relative to the data's scale (the output's maximum, the row's maximum). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The first release of processSEG: seismic processing in Nim on top of parseSEG (>= 0.8.0), built for an interactive viewer. Kernels work in place on parseSEG's sample tensors, chunk by chunk, with no hidden allocation and no hidden threads.
Contents
ProcessPoolruns a plan across cores.Testing
35 test files pass locally in debug, release, danger and ASan builds; property and fuzz tests; benchmarks in
bench/.🤖 Generated with Claude Code