Skip to content

Remote EMAX over TCP: --worker, examples/emax and parity with the built-in EMAX (#702) - #717

Open
rustyeddy wants to merge 1 commit into
mainfrom
feat/remote-emax-parity
Open

rustyeddy wants to merge 1 commit into
mainfrom
feat/remote-emax-parity

Conversation

@rustyeddy

Copy link
Copy Markdown
Owner

Closes #702. PR 2 of 2 (PR 1 is #716, the journal and the in-process path; the RunID port was #714).

What

  • trader backtest --worker host:port trades with an external strategy process instead of the built-in EMAX. In the engine this is RunOptions.Worker: RunSeriesWith dials the worker (loopback only, the existing strategy client and strategy_remote.go adapter) in place of emaxEach. Nothing else changes: the same loop, account sizing, broker and journal. service.BacktestRequest.Worker carries it; run.json records strategy: {name: "remote", transport: "tcp", addr}.
  • examples/emax: the canonical external strategy. main.go is the program (flags -addr, -short, -long, -stop, -tick, then strategy.ListenAndServe); emax/emax.go is the strategy and the EMA, written against the public SDK only (strategy, marketdata, num; a test keeps it that way). The README says how to run it, what the flags mean, how to compare with the built-in run and how to read the journal.
  • No new protocol, no registry. Reused as is: the TCP/JSON client and server, the order DTO, the onBars error path. The one engine addition besides Worker is the strategy's name in Summary (x.strategy, from strategy.name()).

Parity (the oracle is the unchanged built-in EMAX)

The same series, settings (windows 2 and 4, 1% stop), tick, fill assumptions and account (10,000 USD) are run with the built-in strategy and with the worker over loopback TCP, each journaled. Required equal: the whole journal (every order transition, fill and protective exit, in order, to the price), the summary (orders, trades, balance, equity, open positions) and the text a person reads. Only what names the strategy, the RunID and the wall clock may differ. The existing internal EMAX tests and fixtures are untouched.

  • Engine level (trader_test.go): the six EMAX scenarios; a 1,500-bar, two-instrument seeded walk (over 100 fills, over 20 closed trades, stop-outs, shorts, stops cancelled by closes, so it is not a vacuous comparison).
  • Application level (service/backtest_test.go): a real 600-hour month for two instruments built through the raw store and canonical build, with an odd spread so the mids are half ticks the engine rounds; RunBacktest with and without a worker; run.json compared after removing run_id, strategy, started_at, ended_at and the result's strategy name; events.jsonl compared line by line.
  • CLI (cmd/trader): the example built and run as a separate process; trader backtest --worker prints the same summary as the built-in run.
  • The comparison can fail: a worker with a 2% stop instead of 1% is caught (TestEMAXParity_ADifferentWorkerIsCaught).
  • Disconnect: a worker that dies on its 251st group leaves a failed run with the error (strategy remote: reading orders: EOF), no result, and a journal that is exactly the first N lines of the full run's: nothing lost, nothing invented. A worker that is not listening, or on a non-loopback address, is a failed run too.

Findings and decisions to flag

  • The EMA is a copy, as the review anticipated (point 5). A worker cannot import the root package's ema, and no public indicator helper exists. I did not add a package for it. The duplication is marked in emax.go's package doc and protected two ways: the parity tests above, and TestEMAXParity_TheWorkersEMAIsTheBuiltInEMA, which feeds the same closes to both for windows 1, 2, 4, 10, 50 and 200 and requires the same float64 to the last bit. The stop arithmetic (MulRate, RoundDown to the tick, at least one tick) and the skip when a stop is above the largest price are copies too and are covered by the order comparison.
  • The worker needs the tick. The SDK has no catalog, so -tick is a flag (default 0.00001, EURUSD and GBPUSD, the instruments the engine trades today). Another tick would need that flag.
  • A worker is not warmed up. The built-in EMAX is prefilled with the month before --from; the engine does not show those bars to a strategy that is not a warmer, and the service skips loading them for a worker. A run of the same range therefore agrees only when the month before --from is not stored (or --from is the start of the data); the parity tests use data with no earlier bars, a test shows the worker is not sent the bars before tradeFrom, and the README and Users Guide say so. Warm-up over the wire would be a protocol change (a warm message), left for its own issue.
  • Small guard: newResult tolerates a trader with no strategy (an existing unit test builds one).

Acceptance (#702)

  • A completed in-process run and an equivalent external TCP/JSON run each produce a run directory with consistent metadata and events (PR 1, and the parity tests here).
  • Results root defaults to /srv/trading/backtest/results, override by flag, environment or file; tests use temporary directories (PR 1).
  • A test reads the files without in-memory trader objects and checks an opening request, admission/fill, a closing or protective exit and the final status (PR 1, service).
  • A failed or disconnected run preserves a meaningful partial journal and records why (here, remote).
  • Final-frame order events are in the journal though never fed back (PR 1).
  • examples/emax/ is independently runnable and documented, and shows how to find the journal (README; the CLI test runs the binary).
  • An automated EMAX parity test compares remote execution with the internal baseline on the same inputs (above).
  • Existing results unchanged (EMAX md5 195e457a…); make check (including -race) passes.

Not here

Warm-up over the wire, data provenance and build digests, REST (#700), registration, a Trader-side listener, retention.

🤖 Generated with Claude Code

…lt-in EMAX (#702)

RunOptions.Worker swaps the built-in strategy for a worker dialed over loopback;
everything else in the run is unchanged, journal included. examples/emax is the
EMAX strategy written against the public SDK only, with a README. Parity tests run
the built-in EMAX and the worker from the same data, settings and account (the six
scenarios, a long two-instrument walk, and a real month through the store) and
require identical journals, summaries and final positions. A worker that dies
leaves a failed run with the events recorded until then.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 21, 2026 21:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Trader: journal every run with durable RunID results (first increment)

2 participants