WIP: ChainBuddy 2.0, an in-app assistant that proposes flow changes

[ianarawjo]

Work in progress: a first version of ChainBuddy 2.0, plus two fixes found along the way.

ChainBuddy 2.0

A chat panel in the bottom-right corner of the canvas. You ask for a flow or a change; ChainBuddy reads the canvas and proposes changes, drawn on the canvas for you to accept or reject:

• New nodes and connections are dashed; nodes it would change or remove are outlined.
• A card in the chat lists each change. Edits show before and after; for lists, only the items added or removed.
• Nothing changes until you accept, and it never runs anything.

It supports the Prompt, TextFields and JavaScript Evaluator nodes for now, and uses the model chosen under AI Support, on OpenRouter or Ollama.

Everything lives in src/chainbuddy/, kept apart from the rest of the app. The app itself only changes by mounting the panel in App.tsx, plus a webpack rule in craco.config.js that bundles the knowledge files as text.

Folder	Role
knowledge/	What ChainBuddy knows, and the rules it follows, written for people to read. Start with its README.
flowApi/	ChainBuddy's actions, and the checks on every proposal
adapters/	The only code that touches the store; translates node data to and from ChainBuddy's settings
model/, runtime/	A small streaming, tool-calling client for OpenRouter and Ollama, built on the openai SDK already in use, and the loop that runs tools
ui/	The panel and the proposal card

Commits

1. Fix evaluators that return objects with several responses per prompt. Object scores failed as "Unsupported types" whenever a prompt collected more than one response. This affected JavaScript evaluators and in-browser Python evaluators.
2. Stop the settings dialog updating the store during render. Removes a React warning, and a possible double save of settings.
3. Add ChainBuddy 2.0.
4. Read the canvas before proposing, and start over on a new flow. After New Flow, ChainBuddy used to work from its memory of the old flow.
5. Remove proposed nodes left over from a saved flow.
6. Pin @mantine/hooks to 6.0.21 so npm ci works again. Add the frontend packages the code imports but package.json left out #457 added @mantine/hooks at ^6.0.22, but the locked @mantine/core, dates, dropzone and prism (all 6.0.21) require exactly 6.0.21, so npm ci failed on main. The lockfile also gains the peer packages npm installs automatically, which it lacked. A plain npm ci, the tests and a production build all pass.
7. Apply each proposal once, and all or nothing. Fixes from code review: a double-click on Accept (or a Reject meanwhile) could apply a proposal twice; a node deleted since the proposal left it half-applied; and accepting a proposal that replaced every node reset the conversation as if another flow had been opened.
8. Let a loading flow finish before removing leftover nodes. After rebasing onto main, removing leftover proposed nodes as a flow loaded kept main's FlowLoadGuard (Don't autosave an empty flow over the saved one while a flow is loading #460) from ever seeing the load finish, so every save was refused for the rest of the session. The cleanup now waits for the canvas to settle; a test drives the real guard.
9. Refuse {{double-brace}} variables. ChainForge reads {{country}} as a variable named {country, so the model would be sent a stray brace.
10. One NodeKind per node type, in a registry. Everything ChainBuddy knows about a node type (settings, checks, how to read and write its data) is in one object in src/chainbuddy/nodes/; adding a type means adding one kind to NODE_KINDS. A test registers a stand-in Items Node kind to prove it.
11. Move the conversation into a session hook, and notice new flows. ChainBuddyPanel only draws; useChainBuddySession runs the model and tracks proposals. Opening another flow now starts a new conversation and says so.
12. Always bring proposals into view, and label them on the canvas. Proposed nodes are fitted into the canvas left of the chat panel once they have been measured. Additions, changes and removals get a stronger outline and a label, so a proposed edit is hard to miss.
13. Show edits to unfinished nodes filled in. After New Flow, filling in the blank nodes used to look like nothing happened. An edit to an unfinished node now shows on the node, labelled "Proposed contents"; rejecting it (or reloading mid-proposal) restores the node. Edits to finished nodes are still only outlined.
14. Keep proposals readable when they're wider than the free canvas. The view zooms out no further than 0.65; a wider proposal starts from its left and runs under the chat panel.
15. Keep each message and proposal card separate in the chat. Text after a tool call was sometimes joined onto the previous message with no space, and a canvas created again (e.g. by hot reload) reused proposal ids, so a new proposal got no card.
16. Decide connections by what outputs give and inputs accept. Each kind's output names what it gives (values, responses, scored_responses) and accepts lists what its inputs take, replacing per-kind lists of target node types. A new node type no longer means editing the others.
17. Make the knowledge files describe only what exists. Planned features (running, results, undo, more actions) move to a Planned section; guides stop advising on runs; required settings are marked and tested.
18. Gather model code in adapters/models.ts, and fold nodeData.ts away. Kinds use ChainForge's template parser directly (no more CanvasPort.inputsFor or simplified test parser). Also makes redrawing a node fail safe: an error while redrawing used to lose the node silently.

Testing

• 1,350 front-end tests pass. The ChainBuddy tests include round trips through every supported node in the example flows, and a check that the knowledge files agree with the code.
• Checked by hand in the app with Claude Haiku 4.5 on OpenRouter:
  • Building a flow from scratch, then running it.
  • Editing existing nodes, rejecting, and accepting.
  • Filling in the blank nodes of a New Flow.
• A live harness, src/chainbuddy/__test__/liveAgent.test.ts, runs the tools against a real model. It's skipped unless CHAINBUDDY_LIVE is set, e.g. CHAINBUDDY_LIVE=openrouter:anthropic/claude-haiku-4.5.

Not done yet

• No undo after accepting a change set.
• An edited node that had already run can still show as up to date.
• No reading of results, running of nodes, or asking the user questions yet.
• Proposed nodes are real nodes until you decide. They can be autosaved while waiting, though they're now removed on reload.

🤖 Generated with Claude Code