Repository navigation
Conversation
…ush without Gradle and serve dev packs off localhost - management: each SetAssetHead deletes the project's revisions that are not the head, one of its 10 newest ready revisions, under an hour old, or pinned by an environment's active, unfinished or 10 newest deployments, then every blob no remaining revision holds from the database and the release store. A per-project advisory lock, shared by uploads and deployments, keeps collection from racing them. ReleaseStore gains delete. - core removes asset revisions and blobs no resident or loading deployment pins; chunk-jvm keeps only the revision it launches. - core, chunk-jvm and chunk assets pull download four blobs at once. - chunk assets push builds only the assets revision, without Gradle. - chunk dev takes --pack-bind and --pack-url for the pack server. - dashboard: deployments show their asset revision, and projects list their revisions. Closes #328
…o phases and refresh protection on repeated uploads - management: collection also keeps revisions pinned by a superseded deployment within its environment's drain deadline. - SetAssetHead checks the revision and moves the head under the shared asset lock, in one transaction. - Collection deletes rows and records blobs in asset_deletions in one transaction, then deletes the store objects of unreferenced keys in a second, retrying earlier failures. - uploadAssets sets protected_until on the revision it declares again, ready or not, replacing the create_time rule. - core records the kept set only after a prune fully succeeds, and invalidates it whenever a load finishes. Refs #328
…evisions for the deadline they were drained under, and count loads once their inserts finish - completeAssetUpload runs in one transaction holding the shared project lock, so collection can't delete the revision or its blobs between verification and the READY transition. eachLimited now waits for every started run before it rejects. - A deployment records drain_until when it is superseded; collection keeps its revision until the later of that and update_time plus the environment's current deadline, so shortening the setting removes nothing early. - Migration 0004 protects existing revisions for an hour, and 0005 backfills drain_until. - core's asset load accounting waits for the blocking inserts a cancelled or failed load left running. - management README notes the remaining window of a store delete outliving its lock. Refs #328
…set, wait out store deletes and register blobs in digest order - core reports every deployment resident in its backend, including one loading, in ReportStatusRequest.resident_deployment_ids; management stores the set of each accepted report on the environment. - collection keeps the revisions of superseded deployments that can still be activated and of deployments in the reported resident set, replacing the drain deadline rules and drain_until (migration 0005 now adds the resident set). - activateDeployment takes the shared asset lock, so activation and collection don't interleave. - the S3 store's delete waits for both of its deletes to settle before it fails. - completions verify blobs concurrently and register them in one insert sorted by digest, so concurrent completions can't deadlock. Refs #328
…esident set An unknown resident set looked like an empty one: a fresh core whose first read timed out reported nothing, and the next collection deleted the revision of a draining deployment. - ReportStatusRequest.resident_deployments_known is true only when the observation read the resident set. Core no longer repeats a cached set. - Management stores the IDs of reports that read them and records the flag on the environment, resetting it on each new lease. - Collection keeps the revisions of all of an environment's deployments while the flag is false. Refs #328
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #328. Since asset revisions landed (#327), nothing is ever deleted. Every pushed revision and blob stays in management's store, and every revision core or a JVM downloaded stays on its disk. This PR adds retention, plus the other #328 follow-ups.
Retention
Management collects on every
SetAssetHead, so every push triggers it. A revision is kept while any of these holds:UploadAssetssetsprotected_untilto now + 1 h, whether the revision is UPLOADING or READY;resident_deployment_idsandresident_deployments_knownonReportStatus. Until a report has actually read that set, which includes every new lease, all of the environment's deployments count as resident.Concurrency:
asset_deletions. In a second locked transaction it deletes those blobs from the store, skipping any a new upload references again. A failed delete stays recorded and is retried on the next collection.Core prunes revisions,
apps/<rev>directories and blobs that no resident or loading deployment pins. chunk-jvm keeps only the revision its launch runs.Other follow-ups
assets_pb.ts.chunk assets pullfetch 4 blobs at once. Size and SHA-256 checks and staging cleanup are unchanged.chunk assets push: it builds the revision directly, without Gradle.chunk dev: new--pack-bindand--pack-urloptions.--pack-urlis required when packs bind to all addresses.Known limits
SetAssetHead, and the limits are constants.Checks
cargo teston chunk-build, chunk-jvm, chunk-environment and chunk-cli, plus clippy and fmt.buf lint protoandbuf format.oxfmtandoxlinton touched files.🤖 Generated with Claude Code