Skip to content

Add opt-in bounded-scan traversal for MCTS search trees - #129

Open
sylvesterkaczmarek wants to merge 1 commit into
google-deepmind:mainfrom
sylvesterkaczmarek:fix/107-opt-in-bounded-scan-tree-traversal-20261011
Open

sylvesterkaczmarek wants to merge 1 commit into
google-deepmind:mainfrom
sylvesterkaczmarek:fix/107-opt-in-bounded-scan-tree-traversal-20261011

Conversation

@sylvesterkaczmarek

Copy link
Copy Markdown

Addresses #107.

Summary

  • Add an experimental, opt-in bounded-scan implementation of MCTS forward tree traversal and backward value propagation. Set MCTX_USE_BOUNDED_SCAN=true before tracing a policy; optionally set MCTX_BOUNDED_SCAN_UNROLL (default 4) to control JAX scan unrolling.
  • The default remains unchanged and uses jax.lax.while_loop.
  • The bounded scan uses the statically allocated tree capacity as an upper bound. Each iteration conditionally executes the original body only while the traversal remains active, preserving early exit semantics and avoiding extra updates or random key consumption.
  • Document the opt-in flag, JIT tracing, compilation trade-offs, and benchmark caveats.
  • Add tests for early completion, no-op initial conditions, invalid unroll values, and MuZero policy/search-tree parity across different depth limits and batched inputs.

Validation

  • Complete MCTX test suite in standard mode: 30 passed.
  • Complete MCTX test suite with MCTX_USE_BOUNDED_SCAN=true and unroll=2: 30 passed.
  • Six new bounded-loop regression tests passed, including JIT and multiple MuZero depth limits.
  • Ruff, Python compilation and diff checks passed.

Scope and follow-up

This experiment targets dynamic while-loop overhead on accelerators, with a configurable static unroll alternative. It is conceptually separate from upstream PR #116, which reduces full-buffer copies during backward propagation; if that PR merges first, this change may need a rebase. GPU performance improvements have not been measured locally; compare both modes using representative hardware before enabling by default.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant