feat: per-study delete versions and duplicate-upload pre-flight - #2
Closed
JoshuaHarris391 wants to merge 2 commits into
Closed
feat: per-study delete versions and duplicate-upload pre-flight#2JoshuaHarris391 wants to merge 2 commits into
JoshuaHarris391 wants to merge 2 commits into
Conversation
Release ladders diverge per study in practice, which with one global
--version meant one job per study, each with its own confirmation and
failure log. Each --studies entry may now be written name:version, with
--version as the default for bare entries. Design notes carried from
the reference implementation:
- partition(':') so a trailing colon is rejected, never silently
defaulted; the whole list validates before anything dispatches
- uniform versions canonicalise back to the legacy '--studies a,b
--version X' argv, keeping newer CLIs compatible with older installed
service scripts
- one 'all' anywhere in the list forces the prompt even with --yes
- the failure log records the version, since one job can now delete two
versions of the same study
The shell wrapper parses both shapes into parallel arrays, keeps the
exit-3 skip semantics, and picks the worker per study. New bash-driven
tests pin the routing with a stubbed python3.
metadata upload is purely additive: re-running one silently doubles the project's records at that version — observed live as an audit table holding exactly 2x the per-run row count for several study/version pairs. Before submitting, the worker asks the audit table whether this project + version + endpoint has already been uploaded and refuses with exit 2 if so, naming --force-reupload as the deliberate override. A missing audit table is a first run, not an error. The flag threads through metadata upload, upload-all and the bulk wrapper.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1. Two capabilities from the acdc learnings ledger:
name:versionspecs fordelete metadata— one job can retire different versions per study (--studies "ausdiab:0.7.5,cdah:0.8.1"). Uniform versions canonicalise back to the legacy argv so newer CLIs stay compatible with older installed service scripts; a trailing colon is rejected, the whole list validates before dispatch, and oneallanywhere forces the confirmation prompt even with--yes. New bash-driven tests pin the worker routing with a stubbed python3.--force-reuploadis the deliberate override.Known pre-existing limitation (unchanged): a remote
--on ec2delete containingallre-prompts on the box and aborts — same behaviour as before this PR.324 tests pass.