Repository navigation
Journal every backtest run: RunID directory with run.json and events.jsonl (part of #702) - #716
Conversation
…jsonl (part of #702) order.record tells a run-scoped journal of every order transition, the broker's own exits included, so the journal holds the whole sequence in the order it happened, the final frame's transitions too. internal/results makes the run directory (named by the RunID) and finalizes run.json by temp file and rename; service.RunBacktest journals every run, failed and cancelled ones too, under a results root given by --results-root, TRADER_RESULTS_ROOT, results.root or /srv/trading/backtest/results. A journal that cannot write ends the run with its error without changing what is traded. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Review of PR #716 against #702. Good architecture overall: Must fix before merge
Clarify scope (no extra framework)
Simplification / maintainability
Verdict: request the two concrete error-path fixes, document the |
…failure (part of #702) Finish always flushes and closes and adds a journal failure to the run's own error instead of dropping it, classifying the status from the run's error first. A journal writer that writes less than a line and reports no error is a failed journal (io.ErrShortWrite). RunOptions documents which entry points journal. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
Addressed in 8a69cff (not merged).
|
|
Re-review at |
Part of #702: PR 1, the journal and the in-process path, as approved on the issue. #702 stays open for the remote EMAX run,
examples/emaxand the parity test (PR 2).What
Every backtest run now leaves a directory under a results root, named by its RunID (the ported
internal/id, #714), holding two files:run.json: RunID, kind, strategy (emax,in-process), source, interval, instruments, range, wall-clockstarted_at/ended_at,status(runningfrom the start, thencompleted/failed/cancelled),error, and for a completed run the engine'sresult.events.jsonl: one JSON line per order transition, in the order it happened.Example (scenario
b2-stop-cross, an open, its broker stop, the stop's fill):Integration point (the one place transitions are written)
order.recordis the single writer of every order transition, so it also tells the run'sjournal(journal.go, unexported), through one new fieldorder.journal. A nil journal is a no-op, so engine tests and the EMAX md5 are unchanged. No polling, no second record of the run: it copies what the engine writes, when it writes it.Owner points on the plan, each tested:
recordreturns no error and its signature is unchanged. The journal keeps its first write error and stops writing;trader.onFramechecks it after each frame andRunSeriesWithafter the last, and the run ends withjournal: .... Test: a writer that fails after 3 writes ends the run with that error and nothing more is written.requestedline is never lost. The journal pointer is set before the firstrecordfor strategy orders (handleOrder) and for every broker-made order: protective stop and take profit (placeExit, which now takes the trail so the stop's requested line has it, and which an amend uses too) and margin closeouts (closePosition). Test: every order's first line isrequested, including the broker's exits, and a target added by an amend.service.RunBacktestalways creates and finalizes the run, andResultsRootis required (the cmd resolves the default). Which failures come before a run exists: only an unsupported source, a missing or unusable results root, or a directory that cannot be made, since there is then nowhere to record it. Everything after is a run with a status: bars that cannot be loaded, a request the engine refuses (for example an interval it does not support), a cancellation. Tests for failed, cancelled, refused-by-engine, and for the three pre-run refusals (which create nothing).run.jsonis minimal. The result is the engine's own summary (Result.Summary(): the numbersWritealready prints, exact, read from the result, nothing calculated again; money and prices as decimal strings), and the orders areevents.jsonl. No parallel order model.run_<ULID>(Port RunID from the archive; add internal/clock (part of #702) #714). The directory is made withMkdir, so a clash is an error, never an overwrite.Layering
The engine still opens no file:
RunSeriesWith(ctx, RunOptions{Journal: io.Writer}, ...).servicemay not importos(a layering test), so the files are opened by a new foundational package,internal/results(Start, Journal, Finish;run.jsonwritten to a temp file, synced and renamed, so a reader sees the old record or the new, never half; a journal that cannot be flushed turns a would-be completed run into a failed one).service.RunBacktestreturns*BacktestRun(the engine's*Resultembedded, soWriteis unchanged, plusRunIDandDir); the error of a failed run says where the run is.cmd/traderstays transport only.Results root
--results-root, thenTRADER_RESULTS_ROOT, thenresults.rootinetc/trader.yml, then/srv/trading/backtest/results: the same precedence as the data root. The command ends its output withrun <RunID>: <dir>. Every test uses temporary directories and never touches/srv/trading.Tests
journal_test.go(engine): every transition in order with contiguousseq; the open's four states with units, fill price and id where they belong; a protective stop's requested and fill lines (cause: stop loss); an order requested on the last frame is in the journal although noOnBarscould ever tell the strategy; a sizing refusal isrequestedthenrejectedwith no id; a target added by an amend; same run, same lines; journaling changes nothing the run trades, for all six scenarios (summary andSummary()equal with and without); a failing writer ends the run with its error.internal/results: directory and files, running record, every refusal (no root, root that is a file, an existing run directory is left untouched), finish as completed / failed (partial journal kept) / cancelled, a flush failure, no temp file left.service: reads the run from the files only, with no trader object: an opening request, sized, submitted, filled, a closing fill, status completed,resultagreeing with the journal, RunID in the name and inrun.json, its time nearstarted_at; a failed load, an engine-refused request and a cancellation are journaled with their status and error; nothing runs without somewhere to record it; two runs get distinct sorted directories with the same events.cmd/trader: the summary ends with the run's directory; root resolved flag, then environment, then file; a failed run's error names its directory;results.rootparses.make checkpasses; the EMAX md5 is unchanged (195e457a…).Not in this PR (PR 2 and later)
The remote EMAX run (
RunSeriesWithwith a worker address,--worker),examples/emaxand the parity test against the internal baseline, REST (#700), data provenance and build digests, crash-recovery beyond temp-file-and-rename, retention.🤖 Generated with Claude Code