Skip to content

FE-1341: Give experiment compute backends one interface - #9178

Draft
kube wants to merge 1 commit into
cf/fe-1237-place-token-capacityfrom
cf/fe-1341-swappable-experiment-backends
Draft

FE-1341: Give experiment compute backends one interface#9178
kube wants to merge 1 commit into
cf/fe-1237-place-token-capacityfrom
cf/fe-1341-swappable-experiment-backends

Conversation

@kube

@kube kube commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

🌟 What is the purpose of this PR?

Choosing a compute backend is about to stop being a single choice. Today ExperimentsProvider hardcodes it — try one, and if it declines, use the other and put the reason in a notification — which works for exactly two backends and has to be edited to gain a third.

This introduces the contract both backends satisfy, so the WebGPU backend stacked above plugs into a seam rather than rewiring the provider.

Pure refactor: no user-visible behaviour changes, and no changeset.

🔗 Related links

🚫 Blocked by

🔍 What does this change?

The runtime half of swappability already worked and is untouched: a backend produces a MonteCarloExperiment, and consumers drive one with no branching. What was missing is asking a backend whether it can run a net, and choosing when one declines.

A new @hashintel/petrinaut-core/experiments entry point adds:

  • ExperimentRequestwhat to compute, as plain serializable data. Anything describing how belongs to the backend's construction or to per-call options. That rule is what stops this becoming a union of every backend's knobs, and it is why signal and note callbacks are not on it — for a future out-of-process backend the request is the request body.
  • ExperimentAssessment — refusal as a value, not an exception, because a subset engine declining a net is ordinary operation. Blockers carry a code, an optional itemId, and an origin (model / configuration / environment / capacity) so a UI can attribute a problem to the transition that caused it. capacity is separate from environment because a device momentarily out of memory warrants "use fewer runs", not "hide the option".
  • ExperimentBackend — two phases. Assessment settles the net and configuration without acquiring a device or worker pool, so assessing while the user edits is cheap, and hands back an instantiate() closure carrying that work forward.
  • selectExperimentBackend — walks registrations in preference order, records why each declined, and treats a failed instantiation as a refusal so a runnable net still falls back when a resource cannot be acquired.
flowchart TD
  R[registrations, preference order] --> L[load backend]
  L --> A{assess request}
  A -->|blockers| N[record refusal] --> R
  A -->|eligible| I{instantiate}
  I -->|environment / capacity| N
  I -->|handle| D[run, recording which backend]
Loading

Registrations carry a deferred load, so a heavy backend can be registered without pulling its implementation into a bundle that never uses it.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • are internal and do not require a docs change

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

⚠️ Known issues

Only one backend is registered at this point, so assess never refuses and the blocker machinery is unexercised until the WebGPU backend lands above. That is the cost of introducing the seam before its second implementation; the alternative was a GPU PR that both adds a backend and rewires the provider.

Resource sharing across experiments — a reused worker pool, a shared GPUDevice — was considered and deliberately left out. Worker reuse depends on the worker fully resetting frame and metric state, and if it does not, results stay plausible rather than visibly wrong.

🐾 Next steps

Providing backends through a React context, so an embedder can supply its own. The contract is shaped for it: a provider builds the backends its environment supports and publishes the registration list.

🛡 What tests cover this?

select-experiment-backend.test.ts covers the walk: first-accepting wins without loading later registrations, fallback records the refusal, a failed instantiation falls through, blockers are ordered by actionability, an unavailable backend is skipped without assessment, a backend whose module fails to load does not sink the experiment, and requests are built once per distinct artifact requirement.

❓ How to test this?

  1. Check out the branch and run an experiment.
  2. Confirm it behaves exactly as before — this changes how the backend is chosen, not what it does.

Choosing a backend was hardcoded in ExperimentsProvider. The runtime
half of swappability already worked — a backend produces a
MonteCarloExperiment and consumers drive one with no branching — so
this adds only the missing part: asking a backend whether it can run a
net, and choosing between backends when one declines.

A request is plain serializable data; anything describing how to
compute belongs to the backend's construction or to per-call options.
Refusal is a value carrying structured blockers with a code, an item
id and an origin, so a UI can attribute a problem to the item that
caused it. Assessment settles the net without acquiring a device or
worker pool and hands back an instantiate() closure.

Pure refactor: no user-visible behaviour changes. Registrations carry
a deferred load so a heavy backend need not enter the bundle.
@vercel

vercel Bot commented Aug 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview Aug 8, 2026 1:11am
petrinaut Ready Ready Preview Aug 8, 2026 1:11am
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
hashdotdesign-tokens Ignored Ignored Aug 8, 2026 1:11am

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/libs Relates to first-party libraries/crates/packages (area) type/eng > frontend Owned by the @frontend team

Development

Successfully merging this pull request may close these issues.

1 participant