Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,9 @@ defines its own SDK-agnostic `pkg/llm` types (`Provider`, `Message`, `Tool`,
parameterized by base URL + API key + model, serves **OpenAI, OpenRouter, Ollama,
and Gemini's OpenAI-compatible endpoint** via `openai-go` — four providers from
one implementation. Native **Anthropic** gets its own implementation in Phase C.
A registry, built from configuration at startup, resolves providers by name.
A registry resolves providers by name (built from configuration; since
[ADR-0031](0031-settings-secrets-and-environments.md) it builds providers per execution so keys /
base URLs can resolve from the secret store per environment — see Consequences).

This fits the actor model (we own the reasoning loop as our own code rather than
embedding a second orchestration framework), keeps dependencies minimal, and
Expand All @@ -54,8 +56,11 @@ substituted later without touching callers.
- Bad: we maintain the provider mapping and the agent loop ourselves (no framework
doing it for us).
- Bad: Anthropic's distinct protocol needs a separate mapping (deferred to Phase C).
- Neutral: the registry is built once at startup, so keys/base URLs are static per
process — revisited by [ADR-0008](0008-settings-environments-and-secrets-management.md).
- Neutral: the registry was originally built once at startup, so keys/base URLs were static per
process. [ADR-0031](0031-settings-secrets-and-environments.md) since made provider construction
dynamic — `provideLLMRegistry` now builds per-provider `llm.ProviderFactory` closures that
resolve `{{secret:NAME}}` / `{{credential:ID.FIELD}}` references per execution, scoped by the
workflow's environment; fully-static config is still built once (fast path).

## Pros and Cons of the Options

Expand Down Expand Up @@ -89,6 +94,10 @@ substituted later without touching callers.
- Native **Anthropic** provider (Phase C): `internal/llm/providers/anthropic/` via
`anthropic-sdk-go`, registered under the `anthropic` key in `internal/app/di/llm.go`. Specified
under `specs/002-anthropic-provider/`. (Streaming and prompt caching remain Phase C follow-ups.)
- Per-context provider keys: [ADR-0031](0031-settings-secrets-and-environments.md) made the
registry dynamic (per-provider factories resolving secret/credential references per environment),
superseding the original static-at-startup construction.
- Related: [ADR-0005](0005-ai-agents-as-workflow-nodes-phased-roadmap.md),
[ADR-0007](0007-agent-reasoning-loop-and-tools-from-functions.md),
[ADR-0008](0008-settings-environments-and-secrets-management.md).
[ADR-0031](0031-settings-secrets-and-environments.md) (supersedes the original reference to the
now-superseded [ADR-0008](0008-settings-environments-and-secrets-management.md)).
16 changes: 13 additions & 3 deletions docs/adr/0031-settings-secrets-and-environments.md
Original file line number Diff line number Diff line change
Expand Up @@ -138,13 +138,23 @@ drivers with the smallest, most incremental change and gives B and C a stable se
ai/chat & ai/agent functions via `ExecutionInfo.Environment` (carried on
`messaging.ExecuteFunctionMessage`); resolution runs in the function's async goroutine and the
resolved key is `Reveal()`'d only into the SDK client, never logged. Different workflows in one
process can now use different provider credentials per environment. Credential objects (Phase 2)
and Infisical (the read-only external backend) remain scheduled.
process can now use different provider credentials per environment.
- **Phase 2 — credential objects shipped**: typed, centrally-managed credentials referenced by id.
A `Credential` is metadata (`pkg/workflow/credential.go`: id, free-form type, field names) in a
`CredentialRepository` (memory + Postgres, migration `000011_create_credentials`); its field
**values** live in the `SecretStore` under the reserved `cred/<id>/<field>` name, per environment
— one custody point, reusing Phase-1 encryption (the `/` separator is outside the `{{secret:}}`
charset, so the namespaces cannot collide). `CredentialService` writes values via a
`ManagedSecretStore`; CRUD at `/v1/credentials` (values never returned on reads) and a
`fuse credentials` CLI manage them. Three reference forms — input-mapping `source:"credential"`,
the `{{credential:id.field}}` token (`pkg/secrets/credential.go`), and
`LLM_<PROVIDER>_CREDENTIAL=<id>` — all resolve through the existing environment-scoped resolver.
Only **Infisical** (the read-only external backend, option C) remains scheduled.
- Supersedes [ADR-0008](0008-settings-environments-and-secrets-management.md) (which recorded the
problem and deferred the decision).
- Current state this changes: `internal/app/config/config.go` (env-var secrets),
`internal/app/di/llm.go` (static-at-startup provider registry), `internal/workflow/edge_schema.go`
(`SourceSchema`/`SourceFlow` input mapping — gains `SourceSecret`), `pkg/strutil/strings.go`
(`SourceSchema`/`SourceFlow` input mapping — gains `SourceSecret` and `SourceCredential`), `pkg/strutil/strings.go`
(`ReplaceTokens` precedent), `pkg/workflow/execution_info.go` (stays a clean contract — no secret
field).
- Related: [ADR-0005](0005-ai-agents-as-workflow-nodes-phased-roadmap.md) (agents drive per-context
Expand Down
104 changes: 104 additions & 0 deletions docs/adr/0032-sub-workflow-composition.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# 0032. Sub-workflow composition: child workflows as first-class instances linked by a durable ref

- Status: Accepted
- Date: 2026-06-03
- Deciders: FUSE maintainers

## Context and Problem Statement

A workflow often needs to invoke **another workflow** as a step — to reuse a published pipeline,
decompose a large graph into modular pieces, or give a sub-pipeline its own schema version,
concurrency limit, and timeout. This needs a composition primitive that fits the actor model
([ADR-0002](0002-ergo-actor-model-for-workflow-execution.md)) and the durable-execution model
([ADR-0010](0010-durable-execution-journal-and-replay.md)): the parent must be able to launch a
child, optionally wait for it, survive a restart mid-flight, and work in a multi-node cluster
([ADR-0018](0018-high-availability-and-clustering.md)) where parent and child may live on different
nodes. This decision records the model already implemented; it was previously only mentioned in
passing by ADR-0010.

## Decision Drivers

- **Reuse & modularity** — invoke an existing workflow schema by id from within another workflow.
- **Isolation** — the child is its own execution with its own journal, threads, versioning,
concurrency, and timeout, not code nested inside the parent's actor.
- **Durability & replay** — the parent↔child link and the start/complete events must survive
restart and replay deterministically.
- **HA-friendly** — the child is triggered like any workflow, so it can run on any node; the link
must be discoverable across nodes.
- **Reuse existing machinery** — lean on the trigger path, journal, and supervision tree rather
than building a second nested-execution engine.

## Considered Options

- **A — Nested in-process execution.** Run the child graph inside the parent's `WorkflowHandler`.
- **B — Child as a first-class workflow instance linked by a persisted `SubWorkflowRef`** (chosen).
- **C — External orchestration only** (no in-engine composition; callers chain via the API).

## Decision Outcome

Chosen: **B.** A system function (`fuse/pkg/system/subworkflow`, `system.SubWorkflowFullFunctionID`)
emits a `RunSubWorkflowAction`; the parent's `WorkflowHandler.handleSubWorkflowAction`
(`internal/actors/workflow_handler.go`):

1. Mints a fresh child `workflow.ID` and persists a **`SubWorkflowRef`**
(`internal/workflow/subworkflow.go`: `ParentWorkflowID`, `ParentThreadID`, `ParentExecID`,
`ChildWorkflowID`, `ChildSchemaID`, `Async`) via `WorkflowRepository.SaveSubWorkflowRef`
(Postgres table `sub_workflow_refs`, migration `000001`).
2. Appends a `subworkflow:started` journal entry (`JournalSubWorkflowStarted`) for replay.
3. Triggers the child through the **normal trigger path**
(`messaging.NewTriggerWorkflowWithEnvMessage`), so the child is an ordinary workflow instance
with its own actor tree, journal, version, concurrency, and timeout — and **inherits the
parent's environment** ([ADR-0031](0031-settings-secrets-and-environments.md)).

On completion, the child's handler calls `notifyParentIfSubWorkflow`: it looks up its
`SubWorkflowRef` and sends a `SubWorkflowCompletedMessage` to the parent handler, which resumes
the waiting parent thread (`handleMsgSubWorkflowCompleted`) and records `subworkflow:completed`.
The **`Async` flag** distinguishes a **synchronous** sub-workflow (the parent thread blocks until
the child completes, then continues with the child's output) from an **asynchronous** one
(fire-and-forget; the parent does not wait). Because the link is persisted, recovery can find
in-flight children (`FindActiveSubWorkflows`) and the completion notification works across nodes.

### Consequences

- Good: full reuse and isolation — children are independently versioned, rate-limited, timed out,
observed, and replayed; the parent graph stays small.
- Good: durable and HA-safe — the persisted `SubWorkflowRef` + journal entries survive restart and
let parent/child live on different nodes.
- Good: minimal new machinery — composition rides the existing trigger, journal, and supervision
paths.
- Bad: extra coordination surface — completion notification by message, ref persistence, and the
possibility of orphaned children if a parent is cancelled mid-flight (mitigated by recovery
sweeps).
- Neutral: cross-node completion relies on actor messaging by name; a lost message must be healed
by recovery rather than a synchronous call.

## Pros and Cons of the Options

### A — Nested in-process execution
- Good: no cross-instance coordination; simplest happy path.
- Bad: no isolation (shared journal/threads), no independent versioning/concurrency/timeout, and
replay of a nested graph inside the parent is far more complex. Rejected.

### B — First-class instance + durable ref (chosen)
- Good: isolation, reuse, durability, HA; reuses trigger/journal/supervision.
- Bad: coordination + persistence overhead; orphan handling.

### C — External orchestration only
- Good: zero engine complexity.
- Bad: pushes composition to every caller; no in-engine durability/replay for the parent↔child
relationship. Rejected as the in-engine primitive.

## More Information

- Code: `internal/workflow/subworkflow.go` (`SubWorkflowRef`), `internal/actors/workflow_handler.go`
(`handleSubWorkflowAction`, `notifyParentIfSubWorkflow`, `handleMsgSubWorkflowCompleted`,
`handleSystemSubWorkflow`), `internal/workflow/journal.go` (`subworkflow:started|completed`),
`internal/workflow/workflowactions` (`RunSubWorkflowAction`), Postgres `sub_workflow_refs`
(`FindSubWorkflowRef`, `FindActiveSubWorkflows`, `SaveSubWorkflowRef`).
- Related: [ADR-0010](0010-durable-execution-journal-and-replay.md) (journal/replay the start/complete
events ride on), [ADR-0011](0011-threading-model-and-foreach.md) (the parent thread that blocks on
a sync child), [ADR-0018](0018-high-availability-and-clustering.md) (cross-node parent/child),
[ADR-0023](0023-timeout-enforcement-model.md) (each instance times out independently),
[ADR-0031](0031-settings-secrets-and-environments.md) (children inherit the parent's environment),
and [ADR-0027](0027-async-tool-invocation-sub-execution-channel.md) (a distinct, agent-driven
async sub-execution channel — not the same as workflow-graph sub-workflows).
94 changes: 94 additions & 0 deletions docs/adr/0033-dependency-injection-and-app-composition.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# 0033. Dependency injection & application composition with uber-go/fx modules

- Status: Accepted
- Date: 2026-06-03
- Deciders: FUSE maintainers

## Context and Problem Statement

FUSE boots a sizable graph of subsystems — config, structured logging, metrics/tracing, the ergo
node and its actor factories, repositories (memory or Postgres), an object store, services, HTTP
handlers, the secret store, the LLM registry, idempotency, concurrency limiters, the event bus —
with real wiring constraints: a strict construction order, **driver-selected backends** (memory vs
Postgres, fs vs S3), **optional dependencies** (no DB pool under the memory driver), **lifecycle**
(open/close pools, start/stop the node), and the need to build **partial graphs** for CLI
subcommands (`fuse migrate`, `fuse secrets`, `fuse credentials`, `fuse workflow`) and tests. This
records how that composition root is built. ADR-0002 chose ergo for the actor *runtime*; it does
not cover how the application is *assembled*.

## Decision Drivers

- An explicit, inspectable dependency graph instead of hand-ordered constructor calls in `main`.
- First-class **optional** dependencies (e.g. a `*pgxpool.Pool` that is nil under `DB_DRIVER=memory`).
- **Lifecycle** hooks (resource open/close, migrations, node start/stop) tied to app start/stop.
- **Driver selection** at provide time (one provider returns the memory or Postgres implementation).
- **Composability** — boot a subset of modules for a CLI command or a test without the full server.
- A single composition root, keeping wiring out of business logic.

## Considered Options

- **A — Manual constructor wiring in `main`** (plain Go).
- **B — uber-go/fx** (runtime DI container with modules, lifecycle, and parameter/result objects)
(chosen).
- **C — google/wire** (compile-time DI via code generation).

## Decision Outcome

Chosen: **uber-go/fx.** Each layer is an `fx.Module` — `CommonModule` (config, loggers,
metrics, tracing), `DatabaseModule`, `RepoModule`, `ServicesModule`, `WorkerModule` (HTTP handler
factories), `ActorModule` (ergo actor factories), `SecretsModule`, `LLMModule`, `ObjectStoreModule`,
`IdempotencyModule`, `ConcurrencyModule`, `EventsModule`, `PackageModule`, `FuseAppModule` — composed
into **`AllModules`** (`internal/app/di/di.go`). Conventions:

- **`fx.Provide`** for constructors; **`fx.Invoke`** for side-effecting initialization that must run
(e.g. forcing logger init, starting the node, registering internal packages).
- **Driver selection inside providers**: e.g. `provideWorkflowRepository` /
`provideSecretStore` return the Postgres implementation when `DB_DRIVER=postgres` and a pool
exists, else the memory one (`internal/app/di/repos.go`, `secrets.go`).
- **Optional dependencies via `fx.In` parameter structs with `optional:"true"`** — the
`*pgxpool.Pool` is provided through an `fx.Out` result and consumed optionally, so memory-driver
runs and CLI commands work with a nil pool (`internal/app/di/database.go`).
- **Lifecycle via `fx.Lifecycle` hooks** — pool open/close, migrations at provide time, node
start/stop.
- **fx's own logs routed through zerolog** (`logging.NewFxLogger`) so DI diagnostics match the app
log format ([ADR-0020](0020-observability-metrics-tracing-execution-traces.md)).
- **Partial graphs**: CLI subcommands assemble only the modules they need (e.g. `CommonModule +
DatabaseModule + SecretsModule`) via `fx.New(...)`, reusing the same providers as the server.

### Consequences

- Good: the dependency graph is explicit and centralized; adding a subsystem is a new module, not a
surgical edit to `main`.
- Good: optional/driver/lifecycle patterns are first-class, which the memory-vs-Postgres and CLI
paths rely on heavily.
- Good: the same providers back the server, the CLI, and tests — no duplicate wiring.
- Bad: wiring errors surface at **runtime** (app start), not compile time; a missing/ambiguous
provider fails on boot.
- Bad: fx adds reflection and a learning curve; module boundaries must be maintained deliberately.
- Neutral: fx is now a load-bearing, costly-to-reverse framework choice on par with ergo.

## Pros and Cons of the Options

### A — Manual wiring in `main`
- Good: no framework, compile-time safety, zero reflection.
- Bad: verbose and brittle as the graph grows; hand-managed ordering, optional deps, lifecycle, and
per-driver branching become error-prone; partial graphs for CLI/tests duplicate wiring. Rejected.

### B — uber-go/fx (chosen)
- Good: explicit graph, modules, lifecycle, optional deps, partial graphs, shared providers.
- Bad: runtime (not compile-time) errors; reflection; learning curve.

### C — google/wire
- Good: compile-time, no reflection, generated code is debuggable.
- Bad: codegen step in the build; less ergonomic for runtime driver selection and `optional`
dependencies, which are pervasive here. Rejected.

## More Information

- Code: `internal/app/di/di.go` (`AllModules`, `CommonModule`, `PackageModule`, `FuseAppModule`,
`Run`), and the per-layer modules in `internal/app/di/*.go` (`database.go`, `repos.go`,
`services.go`, `actors.go`, `secrets.go`, `llm.go`, `objectstore.go`, …).
- Related: [ADR-0002](0002-ergo-actor-model-for-workflow-execution.md) (the actor runtime fx wires),
[ADR-0003](0003-in-memory-repositories-by-default.md) (the memory/Postgres driver selection fx
expresses), [ADR-0020](0020-observability-metrics-tracing-execution-traces.md) (fx logs via
zerolog).
12 changes: 12 additions & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,3 +58,15 @@ copying [`template.md`](template.md).
| 0029 | [LLM cost & token-usage tracking and budgets](0029-llm-cost-and-usage-tracking-and-budgets.md) | Proposed | 2026-06-02 |
| 0030 | [Structured/JSON output enforcement for ai nodes](0030-structured-output-enforcement.md) | Proposed | 2026-06-02 |
| 0031 | [Settings, secrets & environments: a SecretStore seam](0031-settings-secrets-and-environments.md) | Accepted | 2026-06-02 |
| 0032 | [Sub-workflow composition: child workflows as first-class instances](0032-sub-workflow-composition.md) | Accepted | 2026-06-03 |
| 0033 | [Dependency injection & app composition with uber-go/fx](0033-dependency-injection-and-app-composition.md) | Accepted | 2026-06-03 |

### Proposed backlog (not yet implemented)

ADRs **0026–0030** are a cohesive **agent-capabilities** series exploring the next phase of the AI
agent ([ADR-0005](0005-ai-agents-as-workflow-nodes-phased-roadmap.md)) — orchestrator mode (0026),
async tool invocation (0027), prompt/context & memory (0028), cost/usage tracking & budgets (0029),
and structured-output enforcement (0030). They cross-reference one another and remain `Proposed`
(none implemented yet). **0025** (browser-automation package) is an independent, parallel stream,
not part of that series. These stay `Proposed` until scheduled; when one is implemented its status
moves to `Accepted` and its "More Information" records what shipped (as 0031 does).
Loading