Sandbox templates: content-keyed records, derived-image strategies, template forks, GC - #16
Open
czpython wants to merge 8 commits into
Open
Sandbox templates: content-keyed records, derived-image strategies, template forks, GC#16czpython wants to merge 8 commits into
czpython wants to merge 8 commits into
Conversation
Templates are keyed by content — unique (provider, base_image, requirements_hash) — with status building -> available -> failed. POST /templates answers 202 and the caller polls; the build runs as a background task against the TemplateCapability a provider may implement (implementations land next). Concurrent identical creates resolve through the unique index: the losing racer re-reads the winner. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The TemplateCapability input is (base_image, setup_script) — no live host: the strategy synthesizes FROM <base> plus the script as a bake-time root step, builds with the docker CLI, and the resulting ref is the handle. docker keeps the local tag, docker-sbx builds into the daemon sandboxd reads templates from, and exe pushes to a registry exe.dev can pull (exe.dev has no snapshot verb — the image is its snapshot; registry credentials are EXE_TEMPLATE_REGISTRY/_USERNAME/_PASSWORD). Providers stay dumb: the handle rides host.image and create_vm is unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
HostCreate accepts template — an id or a requirements hash — and the resolution is explicit image > template handle > provider default, with the resolved handle riding host.image into the unchanged provision path. A hash matching several bases prefers the available ones, newest first. Naming a template that is not available fails 409 with the status (and last_error for failed builds); drukbox never builds on miss — the caller owns when to build. A template request is a customization, so it never claims a warm pool host, and each fork stamps last_used_at for GC. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Templates cost money independent of hosts, so python -m templates.janitor gets its own cron entry. Three sweeps: an unfinished build older than TEMPLATE_BUILD_TIMEOUT_MINUTES becomes failed (the only exit for a build whose process died); failed templates keep their diagnostics for TEMPLATE_FAILED_RETENTION_HOURS, then go; available templates unleased for TEMPLATE_UNUSED_TTL_DAYS age out through the strategy's delete -- which is also how a superseded hash dies, since edits mint a new one. Deletes re-validate under the row lock so a fresh lease or status change spares the row, hosts-janitor style. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
create_template replaces materialize_template on TemplateCapability and
its three providers, and the DockerCLI attribute is docker_cli. The docs
and comments this branch added are rewritten to ASD-STE100 shape: short
sentences, active voice, one name per concept ("persistent" replaces
"durable"; "template" replaces "artifact").
exe.dev assumes a public image, and templates land in a private
registry, so create_vm now passes --registry-auth when the host image
lives on the configured template registry — and only then, so the
credentials never reach another registry (exe.dev docs: private-image).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The requirements hash is script-only, so one hash can match two base images on the same provider — hash leasing had to guess with a newest-available tie-break. The guess is gone: template is a UUID, the lookup is exact, and the wire boundary validates the type. Callers do not lose content addressing — POST /templates is idempotent by content and always returns the current record with its ID. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The repost-as-lookup suggestion blurred the contract: POST /templates is the only path that builds, and naming a template on POST /hosts never creates one. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reaping is one concern, and every resource type adding its own cron entry is operator sprawl. The reapers stay in their packages; the janitor package is the single cron entry that runs both. The pool maintainer stays its own command because it creates resources instead of reaping them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the drukbox half of the sandbox-needs design (ENG-877, ENG-878, ENG-879, ENG-880 — children of ENG-527). Boot-and-bake providers (ENG-881) come in a later PR.
Verification: ruff, ruff format, pyright, pytest (447) all green per commit.