Repository navigation
Conversation
backward() was an unbatched function under jax.vmap, and its while_loop carries the whole tree. Under vmap the loop predicate is batched, so every iteration selects between the old and the new carry for every array of the tree: a backup step costs O(num_simulations) and a whole search O(num_simulations^2). The batched version uses a scalar predicate, lets finished rows rewrite their current values and only scatters the entries on the paths. The trees are unchanged; the new tests compare them with those built by the previous vmapped implementation, including rows at different depths.
yoch
force-pushed
the
pr/batched-backward
branch
from
October 14, 2026 18:31
07629cd to
4a2cb60
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
backward()is an unbatched function underjax.vmap, and itswhile_loopcarries the whole tree. Undervmapthe loop predicate becomes batched, so JAX turns every iteration into aselectbetween the old and the new carry for every array of the tree. A backup step then costs O(num_simulations) instead of O(path length), and a whole search O(num_simulations²).This version processes the batch in one loop with a scalar predicate (
any(index != ROOT)). Rows that have reached the root rewrite their current values and are otherwise left alone; the other rows scatter the entries on their path. The trees are identical to the current ones (the new tests compare against the previous implementation).Measured on CPU (4 cores),
gumbel_muzero_policywith a trivial recurrent function, best of 5 calls:On a Colab T4 (jax 0.11.1), median of 5 calls, same benchmark:
The gain grows with the number of actions, which is what the diagnosis predicts (the selects are over arrays of size [num_simulations, num_actions]); with 81 actions it is within the noise of a shared runtime. In every configuration, on CPU, GPU and TPU, the trees are identical to those of
main.On a Colab TPU v5e (jax 0.7.2, the oldest version
pyproject.tomlallows), median of 5 calls:So on TPU this is never slower than
mainand gains from 1.1x to 1.5x with many actions. #116 was slower thanmainin all four of my runs there, which matches the caution in its description about TPU; I only ran it as it stands, without tuning. A real network will dilute all of these numbers, since the cost of the model call is not in this benchmark.Related: this touches the same function as #116, which attacks the same cost with a different structure (it keeps the vmap and records the path in a small carry, then does one scatter per array after the loop). On the same benchmarks the two are close for small trees and small action sets, and this one is ahead where #116's own description measures (batch 128, 1729 actions: 7.5x against 6.6x at 64 simulations on the T4; on CPU the difference grows with the number of simulations, 5.5x against 3.6x at 400), and on TPU only this version avoids a slowdown (table above). The two PRs conflict textually, and so do #128 and #129 (they edit the same region). I am happy to rebase on whichever lands first, or to help combine them if you prefer.
Relevant to #107 (
loop_fninbackwardslow as the tree gets deeper) only partly: this reduces the cost of each iteration, it does not reduce the number of iterations or kernel launches, which is what matters most on GPU. I would not close that issue with it.Tests: pytest from
_testing/: 29 passed (24 existing plus 5 new), pylint clean on the changed files.