Skip to content

Trader: journal every run with durable RunID results (first increment) #702

Description

@rustyeddy

Goal — Trader-owned journaling for every run

Trader must persist inspectable results for every run, regardless of how it was initiated or whether the strategy runs in-process or externally over TCP/JSON. Journaling belongs to Trader's run/execution machinery, not the remote strategy package.

This issue is the first small durable-results increment toward Strategy Protocol milestone #27; it is not the entire reporting or persistence system. Part of #690. Preserve the earlier decision to defer separate trade persistence.

Owner decision (2026-11-21)

  • Journaling applies to all runs (backtests, studies, and live runs where those run kinds are supported), not only external strategies. Implement and verify a backtest path first; don't invent unsupported run-kind machinery.
  • Trader owns the journal, RunID, metadata and final status. An external strategy neither writes nor authoritatively modifies results.
  • Reuse existing run identity, order history, broker/account events, and reporting patterns before adding types or storage abstractions. Do not create a second authoritative trading state store.
  • Results live under a configurable results root, defaulting to /srv/trading/backtest/results, grouped by RunID, for Investor or other clients to inspect after the run. Use the existing configuration mechanism rather than introducing a new configuration subsystem.

First implementation increment

  1. Inspect existing Trader RunID generation and report/journal facilities. Propose the smallest integration point that is shared by in-process and remote backtests, and describe it in the PR before broad refactoring.
  2. Persist one run directory, e.g.:
    /srv/trading/backtest/results/<run_id>/
      run.json
      events.jsonl
    
    The leading path is the default, not a hardcoded location: configuration may override the results root. Exact filenames are flexible if they fit existing conventions.
  3. Include minimal metadata: RunID, run kind, strategy identity (where known), instrument(s), requested historical range/data identity where available, start/end timestamps, terminal status and error when applicable.
  4. Append enough authoritative execution records to reconstruct the sequence of requested orders, Trader admission/rejections, order transitions and fills, including broker-generated protective exits. Capture the final callback's outcomes even if no next OnBars occurs.
  5. Finalize a successful or failed run with an inspectable status. A failed/disconnected remote run should preserve the partial events that were already recorded.
  6. Use EMAX as the official end-to-end example, placed under examples/emax/ with a runnable entry point and a short README covering invocation, parameters, and journal inspection. The remote example must use the public Go SDK and TCP/JSON protocol rather than reimplementing transport plumbing or importing private Trader internals.
  7. Run the remote EMAX backtest against the existing internal EMAX backtest used as our regression baseline, with identical historical data, time range, parameters, spread/fill assumptions, and initial account state. Compare their strategy decisions, accepted orders, fills/exits, final positions/account results, and resulting journals. Normalize expected differences (RunID, transport/strategy identity, wall-clock timestamps and paths); any trading-result mismatch is a regression to explain and fix, not a tolerated protocol difference.

Acceptance

  • A completed in-process backtest and an equivalent external TCP/JSON backtest each produce a run directory and consistent metadata/events.
  • The results root defaults to /srv/trading/backtest/results; an explicit configuration override writes beneath the configured root. Tests use isolated temporary directories and do not require access to /srv/trading.
  • A test reads the files without in-memory trader objects and verifies an opening request, admission/fill, closing or protective exit, and final status.
  • A failed/disconnected run preserves a meaningful partial journal and records why it ended.
  • Final-frame order events are present even when they were never delivered back to the strategy via OnBars feedback.
  • examples/emax/ is independently runnable and documented as the canonical external-strategy example; it demonstrates how to locate the run journal.
  • An automated EMAX parity test compares remote TCP/JSON execution to the pre-existing internal EMAX baseline using the same inputs, and asserts equivalent order/execution sequence, P&L, final holdings and journaled trading events after normalizing non-semantic fields. Keep the existing internal test/fixture as the reference; do not rewrite it to match the remote result.
  • Existing backtest results remain unchanged when journaling is enabled; go test -race ./... and make check pass.

Defer to later increments / focused reviews

  • Full study/live-run integration where the requisite run orchestration does not yet exist.
  • Comprehensive data provenance/build digests, replay engine, derived reports and UI.
  • Exact crash-recovery/atomic-finalization protocol, concurrency/retention policy, and interrupted-write recovery beyond safe basic file handling.
  • REST run management (Strategy Protocol: read/control completed and active runs via REST #700), service registration/RunID negotiation, remote strategy registry and asynchronous feedback.
  • A separate persistent trade store or duplicate Account/Broker state.

Development cadence

Build → test → review → decide. Claude should first post the proposed journal integration point and minimal JSON/JSONL shape for review; implement only this increment and open a PR before expanding it.

Activity

  1. added this to the Strategy Protocol milestone on Nov 20, 2026
  2. changed the title [-]Strategy Protocol: durable RunID journal and reproducible results[/-] [+]Trader: journal every run with durable RunID results (first increment)[/+] on Nov 21, 2026
  3. rustyeddy commented on Nov 21, 2026

    @rustyeddy
    OwnerAuthor

    Proposed integration point, file shape and staging for #702 (for review; nothing is implemented yet)

    I read #702, the run path (service.RunBacktest → trader.RunSeries → trader / backtest.runSeriesFrom), order.record, the broker, the config loader and internal/logging. What I found, then the proposal.

    What exists, and what does not

    • There is no RunID generator. internal/logging only names the attribute (run_id). RunSeries has no identity or sink; Result is a summary printed by Write.
    • order.record is the single writer of every order transition (requested, sized, submitted, filled, rejected, cancelled), called from trader.handleOrder, account.size, the broker (enqueue, fill, placeExit, closePosition, cancelWhere, amend, trailStops) and reject. Each order carries its own history. The broker's protective stop, take profit and closeout orders are ordinary orders made in placeExit / closePosition.
    • The engine "opens no file" (historical.go). Files are opened by the application layer (service), which cmd/trader calls. Config precedence is flag → environment → etc/trader.yml → default (see --data-root).
    • RunSeries hardcodes EMAX (emaxEach); there is no exported way to run a remote strategy through it yet.

    Smallest shared integration point

    A journal that every order reports its transitions to, at the one place they are written. Because order.record is the single writer, in-process and remote runs, strategy orders and broker exits, and the final frame's outcomes are all captured with no polling and no dependence on feedback:

    • root journal type (unexported) with record(o *order, e orderEvent); it appends one JSON line per transition to an io.Writer and gives each order an ordinal on first sight.
    • order gets one unexported field, journal *journal; order.record calls it when non-nil.
    • It is set in three places: trader.handleOrder (strategy orders), broker.placeExit (inherits the parent's) and broker.closePosition (the broker's own journal pointer). A nil journal costs nothing, so existing runs, tests and the EMAX md5 are unchanged.
    • The engine receives only an io.Writer; it opens nothing. service creates the directory and files and passes the writer in, which keeps the layering rule that the engine does no file I/O.

    This is the append-only sequence the issue asks for. It is not a second state store: it only records transitions the engine already makes, in the order they happen.

    Run directory

    <results root>/<run_id>/ with:

    run.json, written at start with "status":"running" (so a crash leaves evidence) and rewritten at finalization by write-temp-then-rename:

    {"run_id":"run_01KB5Q3V7T2N4XG9YB6HFA1V7Z","kind":"backtest",
     "strategy":{"name":"emax","transport":"in-process"},      // or {"transport":"tcp","addr":"127.0.0.1:9100"}
     "source":"oanda","interval":"h1","instruments":["EURUSD"],
     "from":"2020-01-01T00:00:00Z","to":"2021-01-01T00:00:00Z",
     "account":{"id":"sim123","currency":"USD","initial_balance":"10000"},
     "started_at":"...","ended_at":"...","status":"completed","error":"",
     "result":{"bars":8640,"orders":{"submitted":..,"filled":..,"rejected":..,"cancelled":..},"trades":..,"balance":"...","equity":"..."}}

    status is running, completed, failed or cancelled; error carries why (a worker disconnect, a validation failure). started_at / ended_at are wall clock and the only run-varying fields besides run_id and strategy.

    events.jsonl, one line per order transition, flushed at finalization and on failure so a failed run keeps what was recorded:

    {"seq":1,"time":"2020-01-06T00:00:00Z","order":1,"id":"","state":"requested","instrument":"EURUSD","action":"open","direction":"long","type":"market","cause":"strategy","price":"1.1","stop_loss":"1.09"}
    {"seq":2,"time":"2020-01-06T00:00:00Z","order":1,"id":"sim-1","state":"sized","units":"666"}
    {"seq":3,"time":"2020-01-06T00:00:00Z","order":1,"id":"sim-1","state":"submitted"}
    {"seq":4,"time":"2020-01-07T00:00:00Z","order":1,"id":"sim-1","state":"filled","units":"666","fill_price":"1.10215"}
    {"seq":9,"time":"2020-01-11T00:00:00Z","order":3,"id":"sim-2","state":"filled","cause":"stop loss","action":"close","units":"666","fill_price":"1.092"}

    time is the simulated step time, so the file has no wall-clock and two equivalent runs produce identical lines. The requested line carries what was asked (instrument, action, direction, type, price, stop, target); later lines carry only what changed (ids, units, fill price, reason text for a rejection or cancel). Prices and sizes are decimal strings (ADR-004). A refused order shows requested then rejected with the reason and never gets an id, exactly as today.

    RunID (updated: ported from the archive, PR #714)

    run_<ULID> from the ported internal/id (id.RunID, Generator, ParseRunID), not my earlier <UTC>-<hex>. Run directories are named by it, so they sort by start time, cannot realistically collide, and ParseRunID tells a real run directory from a stray folder. The generator takes a func() time.Time; production passes the new internal/clock.Clock (one place that reads the time, which can become Trader's own clock later), tests pass a fixed time with id.NewDeterministic. The directory is still created with Mkdir, so a clash is an error, never an overwrite. The ULID holds wall-clock time, but no RunID appears in events.jsonl, so parity comparisons are unaffected.

    Results root (existing config mechanism)

    Same precedence as --data-root: --results-root, then TRADER_RESULTS_ROOT, then results.root in etc/trader.yml (a new small results: section in fileConfig), then the default /srv/trading/backtest/results. cmd/trader stays transport only; the path goes into the request to service.RunBacktest. Tests use temporary directories and never touch /srv/trading.

    The remote run and EMAX parity

    • trader.RunSeries becomes a wrapper over a new RunSeriesWith(ctx, RunOptions, series...), where RunOptions adds an optional worker address (a remoteStrategy replaces emaxEach) and the journal writer. Defaults are today's behaviour. trader backtest gains --worker host:port and --results-root.
    • examples/emax/: a worker using only the public SDK (strategy.ListenAndServe, Order, Event, num, marketdata), implementing the EMAX rule (EMA 2/4, flip = close then open, 1% stop rounded to the tick). A README covers starting the worker, running trader backtest --worker …, and reading run.json / events.jsonl.
    • Parity test: the internal EMAX baseline (RunSeries as it is today) and the remote EMAX worker are run on identical series, range, balance and fill assumptions. I compare the journals line by line (they have no wall-clock, so no normalization is needed in events.jsonl), and run.json after normalizing run_id, strategy, started_at, ended_at, plus the result summary. The existing internal EMAX tests and fixtures are not touched.
    • Risks to flag: (1) EMAX's EMA uses float64 (indicators.go); the example must reproduce the same arithmetic, so it copies that small function rather than inventing a variant. (2) The worker needs the instrument's tick for the stop; the SDK has no catalog, so the example takes a -tick flag (default 0.00001, EURUSD) and the parity test passes the same value. (3) A mismatch is a regression to explain, per the issue.

    Tests

    • Journal unit tests: nil journal is a no-op; ordinals and seq; every transition recorded in occurrence order; refusal has no id; broker exits (stop loss, take profit, closeout) recorded with their cause; an io.Writer error surfaces as a failed run, not silently dropped.
    • Read-only-from-disk test (no trader objects): parse run.json and events.jsonl and assert an opening request, admission and fill, a protective-stop exit and status: completed.
    • Final-frame: an order requested on the last frame appears in the journal although no later OnBars could ever feed it back.
    • Failure: a worker that disconnects mid-run leaves a failed run with the reason and the events recorded so far.
    • Config: default root, override by flag, environment and file; temp dirs only.
    • RunID: parse of a created directory name, injected clock and entropy, collision refused (the ID package itself is covered by Port RunID from the archive; add internal/clock (part of #702) #714).
    • Existing results unchanged with journaling on (summary and the EMAX md5 195e457a…), go test -race ./..., make check.

    Proposed staging (the issue's "implement only this increment")

    It is sizeable, so I suggest three PRs, all Part of #702 except the last:
    0. RunID port (#714, open for review).

    1. Journal, RunID, results root and the in-process path: the journal hook, run.json / events.jsonl, config, service and trader backtest wiring, and the tests above that need no worker.
    2. Remote run, examples/emax and the parity test: RunSeriesWith with a worker address, --worker, the example and README, the failure/disconnect journal test and the EMAX parity test. Closes #702.

    Deferred (as the issue says)

    Data provenance and build digests, replay, derived reports, crash-recovery and atomic-finalization beyond temp-file-and-rename, retention and concurrency policy, REST (#700), registration/RunID negotiation, a separate trade store, studies and live runs.

    Defaults I'll assume unless you say otherwise

    1. The journal hook is order.journal, called from order.record (exact order, covers broker exits), rather than polling histories the way feedback does.
    2. One events.jsonl line per order transition (including requested, sized, submitted), not a reduced set.
    3. RunID is the ported run_<ULID> (PR Port RunID from the archive; add internal/clock (part of #702) #714), generated with internal/clock.Clock.
    4. Three PRs, as above (0 is the RunID port).
    5. The file-writing code lives in service (the application layer); the engine only receives an io.Writer.
  4. rustyeddy commented on Nov 21, 2026

    @rustyeddy
    OwnerAuthor

    Reviewed Claude's #702 plan against current order.go, historical.go, service/backtest.go, and indicators.go. Overall: good direction; approve the two-increment staging with the following simplifications/corrections before coding. The order.record hook is the strongest part: it already owns the transition history, and service should own disk I/O.

    Must resolve

    1. Journal write failures must propagate without altering trading outcomes. Today (*order).record returns no error and is called throughout broker/account/trader code. Adding io.Writer there cannot magically return an error to the run. Avoid a sweeping signature change solely for journaling: a run-scoped journal can retain its first write error, checked at safe boundaries and on finalization, with tests proving that a failed writer marks the run failed. Document when a partially failed journal is detected.
    2. Capture the first requested event. Attach the journal before the first call to order.record for both strategy orders and broker-generated exits. Test this explicitly. A journal pointer assigned in handleOrder after an earlier record would silently lose it.
    3. Do not let nil journaling become the production default. The owner decision is every run. Existing direct engine tests can use an optional/no-op sink, but service-initiated runs must always create/finalize the journal, including failures after run creation. Define clearly which input-validation/load failures occur before a run is established.

    Simplify / avoid duplication

    1. Prefer a standard sortable ID over bespoke timestamp-8hex plus clock/random injection. We previously discussed ULID/time-ordered IDs. Use any existing ID helper if present; otherwise a small established ULID/UUIDv7 implementation or standard-library-supported approach, without building a configurable ID subsystem. Tests need uniqueness and format, not forced deterministic IDs if they can compare normalized output.
    2. Avoid copying the private EMA math into examples/emax. That creates two drifting strategy implementations and weakens parity as a regression test. Prefer an existing public indicator helper or a tiny shared function only if it genuinely reduces duplication. If neither exists, call out the temporary duplication and protect it with explicit numerical parity tests; don't introduce a broad new indicator framework for Trader: journal every run with durable RunID results (first increment) #702.
    3. Keep run.json minimal and reuse existing Result / order data. Do not create a second result calculation or full parallel order model. Record what already exists. The example JSON is illustrative, not a mandate to introduce every field and converter at once.
    4. Keep remote wiring minimal. A second PR is sensible, but RunSeriesWith, RunOptions, worker selection and a new CLI flag are justified only by the actual remote EMAX exercise. Reuse the current strategy client/server and CLI/config patterns; no additional protocol or registry abstractions.

    Parity acceptance

    Use the unchanged internal EMAX baseline as oracle. Compare deterministic order transitions, fill/exit prices, units, final balances/P&L and journal events for identical inputs. Normalize run metadata (and any legitimately nondeterministic IDs), instead of requiring byte-for-byte equality as an end in itself. Include final-frame and broker-protective exits. Ensure the remote example uses public APIs and lives at examples/emax/.

    Verdict: Proceed with PR 1 (journal + in-process path), then review before PR 2 (remote EMAX). No need for more planning layers; small code, real tests, inspect the resulting files.

  5. rustyeddy commented on Nov 21, 2026

    @rustyeddy
    OwnerAuthor

    Reopened: PR #714 was only the RunID step (part of this issue). It closed #702 on merge because its branch was linked to the issue with gh issue develop, not because of a keyword. #702 stays open for the run directories, events, EMAX example and parity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions