UN-4008 [FEAT] Add the unstract CLI to run extractions and API deployments from the terminal

[chandrasekharan-zipstack]

What

unstract — one command that takes a document to structured JSON: LLMWhisperer extraction, an API deployment run, and the polling in between.

• unstract config init | list | get | set | doctor
• unstract whisper extract | status | retrieve | detail | highlights | usage, whisper webhook create | get | update | delete
• unstract docstudio deployment run | status
• unstract --discover groups | summary | full

Why

Both products are reachable today only from Python. This makes them scriptable from a shell and drivable by an agent: one output envelope, one exit-code table, and a JSON description of the whole surface so a caller can construct a command without a second round trip.

How

• Config — profiles for both products, resolved flag > env > profile > default. Discovery is --config → $UNSTRACT_CONFIG → a project-local .unstract.toml found by searching upward (stopping at $HOME) → ~/.unstract/config.toml. Values may indirect through env:VAR, so a config file can be committed without a key in it. Files are written 0600. Deployments are named aliases inheriting org and key from their profile.
• Trust — a discovered .unstract.toml may not supply api_key or base_url. A checkout the user did not write would otherwise hand the CLI a key or point it at another host; a path the user names explicitly is a deliberate choice and is honoured. Withheld values are kept and written back to their own file, never carried into another one. Config writes refuse a symlinked target and land through mkstemp + os.replace, so a planted symlink cannot redirect config set onto some other file. Routing (org_id, api_name, profile selection) stays repo-controllable by design.
• Output — every command prints {ok, data, error, meta} on stdout, in JSON by default whether or not stdout is a TTY, so a script gets the same bytes as a terminal. --output table wraps rather than truncates.
• Errors — a fixed exit-code table (auth, not-found, validation, rate-limited, timeout, server, already-consumed), so a caller branches on the code without parsing text. Secrets are scrubbed from anything rendered. A result that can be read only once is written to disk before it is printed.
• Poll — one wait-for-completion loop for both products. It never sleeps past the deadline — --timeout 30 returns at 30s — and a timeout carries the handle out so a caller can resume rather than restart.
• Flags are derived from the committed OpenAPI specs, intersected with what the pinned client's signature accepts: a spec parameter the client cannot name would raise TypeError at the call rather than reach the API, so it is not offered, and tests/test_contract.py records which ones those are so the gap widens on purpose or not at all. An unpassed flag is not sent, so the server default applies rather than one pinned here; only None counts as absent, so 0, false and "" travel.
• Discovery — --discover answers what --help answers, as JSON, read back from Click itself, so a described command cannot drift from the one the parser accepts. full adds every flag with its type, choices and default plus the exit-code table.

Can this PR break any existing features

No. New repository, nothing depends on it yet, and it is not published. It reads the two clients through their public APIs only.

Notes on Testing

230 tests, offline by design: no network and no credentials — the clients are replaced at the factory, so what is asserted is which arguments a command hands the client, and what a caller sees on stdout and in the exit code. CI runs ruff and pytest, then the suite a second time against the newest click the pin allows — uv run resolves from the lockfile, and an install in the wild does not. Live round trips are a manual pre-release step.

Related Issues or PRs

• UN-4008
• UN-4009 [MISC] Generate and commit the API deployment OpenAPI spec in-repo unstract#2237, UN-4010 [FEAT] Support every API deployment request parameter via a generated transport unstract-python-client#27, UN-4011 [FEAT] Support every extraction parameter via a generated transport llm-whisperer-python-client#35 — the specs and clients this CLI is built on

Pins

Built on unstract-client==1.6.0 and llmwhisperer-client==2.9.0, both from PyPI. The vendored specs are byte-identical copies of the ones those releases were generated from, recorded with their source commit and sha256 in src/unstract_cli/specs/provenance.json and checked by tests/test_specs.py. Neither client installs a console script, so unstract is this CLI's alone; unstract-cli remains as a second name.

Release

.github/workflows/release.yml — a manual dispatch that bumps the version, lints, tests, builds, publishes to PyPI with uv publish through a Trusted Publisher, and only then commits the bump, tags it and cuts the GitHub release. The version lives only in src/unstract_cli/__init__.py, read through hatch, so the bump edits one file, and the committed value names the last stable release.

pre_release publishes a PEP 440 release candidate: the target version is computed as usual, then rcN is appended, counting up from the rc tags already published for that target, and __version__ is left alone because a candidate is not a release. Promoting is the same dispatch with pre_release off. version_bump: none targets the version already in the repo, so the first cut is none + pre_release → 0.1.0rc1, and none alone promotes it to 0.1.0.

Owner-side setup, none of which exists yet on this repo:

• PyPI pending publisher for a project that does not exist yet — PyPI → Your projects → Publishing → Add a pending publisher (GitHub): PyPI Project Name unstract-cli, Owner Zipstack, Repository name unstract-cli, Workflow name release.yml, Environment name blank (the workflow declares no environment).
• PUSH_TO_MAIN_APP_ID (Actions variable) and PUSH_TO_MAIN_APP_PRIVATE_KEY (Actions secret) must reach this repo. Neither exists at repo level here; both are presumably org-level, and an org-level secret still has to list unstract-cli among the repositories it is visible to.
• The GitHub App behind those credentials must be installed on Zipstack/unstract-cli — the workflow requests a token scoped to that repository by name.

🤖 Generated with Claude Code

https://claude.ai/code/session_014f9oEEYspPH4fmPULTnLkJ

[ritwik-g]

Standardized PR Review — INITIAL

Run under the team's standardized 19-lens rubric. Base 13cb3f77 -> head f46b650c, 46 files, +14,071/-0; scope reconciled against GitHub's own file count. Reported at a must-flag floor by request — Lows and style are suppressed throughout, so absence of a nit here is not evidence of one.

Verdict: BLOCK

Critical: 0 · High: 9 · Medium: 13 · Lenses run: 19/19

The engineering is careful and the reasoning is written down — redaction is applied before rendering, the poll deadline uses monotonic with an injected clock, config writes are mkstemp+fsync+os.replace, and the trust boundary on a discovered .unstract.toml survived adversarial probing. The architecture is not what blocks this. Nine High-severity defects do, all at seams between components, most proved by reproduction rather than argued. Each has a stated local fix.

Findings are threaded inline at their file:line. Highs are #1-#9.

---

Lens checklist — 19/19

	Lens	Result
1	Spec & intent	See unanchored — repo states no title convention, so nothing to judge the title against
2	Architectural fit	Clean
3	Correctness & edge cases	See #1, #4, #9, #10, #11, #12, #18
4	Security	Clean — vendored security deep-dive completed (not degraded); config trust boundary adversarially probed and holds
5	Data integrity	See #2, #17
6	Concurrency	N/A — single-threaded; the one module global (secrets registry) is documented and test-reset
7	API & contract compatibility	See #3, #6, #13, #15, #21, #22
8	Reliability & resilience	See #19
9	Performance & cost	Clean — cold import 0.18-0.31s, specs @cached
10	Observability	See #16
11	Operational safety	See #5, #20
12	LLM/agent	N/A — no prompts, model config, or agent loops
13	Testing	See #7, #14, and the coverage entry below
14	Dependencies & build	Clean — clients pinned exactly, lockfile resolves 159 hashes from PyPI only
15	Code quality	Clean — pinned gate passes at ruff 0.16.2, the lockfile's version
16	Doc & comment accuracy	See #8, #13
17	Cross-document rule consistency	Clean — branch current with origin/main, no merge-result sweep needed
18	On-prem deployment experience	N/A — no chart, on-prem values, schema, build path, or customer docs
19	Scaling & workload lifecycle	N/A — no pooled worker, autoscaling, or lifecycle config

Lenses 4, 5, 6, 8, 11, 12, 14, 17, 18, 19, lens 7's consumer search and lens 15's pinned gate were assessed by the orchestrator rather than a dispatched agent. A data-flow boundary sweep traced 11 output sinks: clean.

---

Coverage gaps — folded into one entry, all mutation- or coverage-proven

Not padding, and not a request for more tests where coverage exists — each is a failure path with none:

• translated()'s Timeout branch is unexercised, and only clause order keeps ConnectTimeout (whose MRO is ConnectTimeout -> ConnectionError -> Timeout) out of the ConnectionError arm. Reordering silently changes a published exit code. core/clients.py:213-220
• clone's PlatformAPIError / CloneError / report.aborted translation is entirely untested — PlatformAPIError is the shape every non-2xx Platform API answer arrives in. commands/clone_cmd.py:182-199, 221-222, 263
• whisper webhook update and delete are untested, including the remember_secret call that stops a fresh --auth-token echoing back. The identically-shaped webhook get has that test. commands/whisper_cmd.py:381-386, 411-415
• __main__'s BrokenPipeError and OSError arms are untested. unstract ... | head -1 is ordinary usage. __main__.py:73-75, 79-84

Suite coverage is 94% (1698 statements, 104 missed). Defensive branches are deliberately not filed.

---

Unanchored findings

• The PR description asserts a mechanism that does not exist. It states "Config writes use O_NOFOLLOW, so a planted symlink cannot redirect config set onto some other file." O_NOFOLLOW appears nowhere in the repository. The security property does hold — os.replace acts on the final link, not its target — so this is a wrong claim rather than a vulnerability. Flagging it because it is the sentence a reviewer would rely on to skip checking the write path.
• redact_headers is dead code, referenced only by tests.
• The spec pin is self-certifying. tests/test_specs.py compares each vendored spec against a sha256 in provenance.json, and both move in the same edit — so it catches a corrupted copy, not a hand-edited one, and nothing verifies the repo/commit/path it names. The bump-client-pins skill states this limit correctly and explicitly; src/unstract_cli/specs/README.md:13-14 reads as stronger protection than exists.
• Nothing ties the client pin to the spec commit at release time. Bumping a client without re-syncing its spec fails no test; the CLI just silently stops offering a parameter the new client gained. That direction only loses a feature, so it is the benign half.

Evidence and its limits

Every High was reproduced or verified against the pinned clients in isolated virtualenvs outside the worktree; the review worktree was never written to. One exception, stated in the finding itself: #9 depends on service behaviour nobody could settle without live credentials.

Verdict posted as a comment, not as REQUEST_CHANGES — the merge gate is the maintainers' call, not the review's.

[ritwik-g]

Standardized PR Review — FOLLOWUP (rounds 2 and 3, combined)

Scope: f46b650c..1ac518e7 — the 26 commits since my first review. Base unmoved (13cb3f77), branch current. Scope change: NO — no new files, dependencies, persisted fields or network calls.

Verdict: REQUEST CHANGES

Critical: 0 · High: 3 · Medium: 5 · Prior findings resolved: 21 of 22

Down from BLOCK. Every High below is a regression introduced by a fix, or a guard on a fix that cannot fail — not a surviving defect from the original PR. Each has a one-line remedy.

Prior findings — reconciliation

RESOLVED (21): #1-#8, #10-#19, #21, #22, the coverage entry, and all four unanchored items.

Three were re-proved by mutation rather than taken on trust — #7 (clients.py, both original mutations now fail), #14 (test_workflows.py, and its anti-vacuity guard is itself load-bearing), and #9.

#9 deserves a specific note. It was the one High left open pending a live run, and 1ac518e settles it: "The service reports a batch as COMPLETED even when a document inside it failed." So it was real, not void. The fix walks per-file results on both run and status, sets verbatim_details=True so the successful documents survive the error path, names the failed files in extra, and updates the exit-code table. I verified the --save-before-raise ordering on both paths myself (poll.py:266-272 inside wait_for_completion; docstudio_cmd.py:285 before :287). Good fix.

NOT RESOLVED — deliberately, with reasoning I accept (1): #20, tracked as issue #7 — nothing on PyPI to pin to until the release workflow runs once.

Two claims I investigated and rejected

Recorded because passing them on would have wasted your time:

• "preflight no longer rejects a directory --save target." False positive. All three --save options declare click.Path(dir_okay=False); Click raises BadParameter before preflight runs. Reproduced by calling preflight() directly, which bypasses type conversion.
• "PUSH_TO_MAIN_APP_CLIENT_ID may not be set." Already satisfied — it is available to this repo org-wide, and in use in three other Zipstack repos.

Release-path status, since it came up in Slack

All three prerequisites are now met. The org variable and secret were already shared with this repo, the App was already installed org-wide (repository_selection: "all"), and the main ruleset now carries zipstack-push-to-main (actor_id 1199465, Always) in its bypass list — verified, with all four rules intact. Only step 3 ever actually needed doing.

Lens checklist — 19/19

Unchanged from round 1 except: 3 → #1, #2, #8; 7 → #1; 11 → #4; 13 → #3, #7, #8; 16 → #5, #6. 4 Security — Clean, with the open question below. 15 — Clean; pinned gate passes at ruff 0.16.2 (the lockfile's version) on 1ac518e7. 17 — Clean, branch current. 6, 12, 18, 19 — N/A.

Coverage caveat, stated rather than hidden: lens 3 over commit 1ac518e was assessed by me directly, not by a dispatched specialist — that agent terminated on a rate limit and returned a fragment. I read _failed_files / _raise_for_failed_files and both call sites in full and found the logic sound, but this line had one reader rather than two.

Open question for the author

verbatim_details=True means a credential the run never resolved, appearing under a secret-looking key in a server-authored body, now reaches stdout where it was redacted before. Nobody could trace a concrete route — FileResult.metadata is untyped in the vendored spec, which is the gap. Does include_metadata on a deployment status response ever carry adapter or connector credentials? If it can, poll.py:172/:205 and docstudio_cmd.py:207 need redaction on everything except the result-bearing key rather than being skipped wholesale.

Unanchored observation

_failed_files treats any per-file status that is not casefolded "success" as a failure, while the execution-level poll in the same file accepts two success spellings (terminal_success=("COMPLETED", "SUCCESS")). The spec types FileResult.status as a bare string with no enum. The new tests do cover case variation ("Success", "SUCCESS") and the all-succeeded direction, and you confirmed the shape against the live service — so this is a robustness note, not a defect. Worth a comment naming the spelling you observed, since the spec does not pin it.

Posted as a comment, not REQUEST_CHANGES — the merge gate is the maintainers' call.

[ritwik-g]

[High] [Lens 3/7] — the fix for finding #1 was reasoned from one product and applied to both: -o raw now prints a job handle where text belongs, and loses a one-shot retrieve

Found independently by four of six reviewers, each reproducing it by execution.

The guard moved from is not None to not in (None, ""). The premise is real but docstudio-specific: unstract-client==1.6.0 spells a not-yet-available extraction_result as "" and flips pending: True for SUCCESS and not extraction_result, so docstudio never surfaces a terminal empty result. llmwhisperer-client==2.9.0 does neither — whisper_retrieve returns the body verbatim, so result_text == "" on the whisper path means exactly one thing: a document that genuinely extracted to no text (blank page, image-only PDF under a text mode, an out-of-range page selection).

Two consequences, both on the primary -o raw path:

1. whisper retrieve <hash> -o raw — RAW_TEXT = ("result_text",) is a single-field tuple, so raw_value finds nothing and raises. render() runs before emit_text writes, so stdout carries an error envelope and never the extraction. That read is the acknowledging one-shot; without --save the result is gone. Exit 1, where it previously printed an empty line and exited 0.
2. whisper extract doc.pdf -o raw — EXTRACT_RAW = ("result_text", "whisper_hash"), so it falls through and prints the hash. unstract whisper extract blank.pdf -o raw > out.txt writes a hash into the text file, exit 0, nothing on stderr.

The comment deleted in this same commit named the outcome exactly: "skipping it would print the next field -- a handle where the caller expects text." The replacement tests are docstudio-only; the ACK/PENDING_STATUS fixtures were edited from None to "", and no whisper fixture covers an empty result_text.

Fix: make "empty counts as absent" a property of the field list rather than of raw_value — or collect the first non-empty hit and fall back to a present-but-empty one before raising. Either keeps the docstudio pending case and restores an empty extraction as a real answer on both whisper paths.

[ritwik-g]

[High] [Lens 3] — the fix for finding #6 turned prose into a paste-ready command, and the command drops --save

The old hint was prose; this one is something a user copies verbatim:

Resume with `unstract docstudio deployment status <target> <id>` rather than resubmitting the document.

No --save. Trace the path: deployment run <target> doc.pdf --save out.json preflights the save, polls, times out. Nothing has been written — deliver (poll.py:266-272) only runs on terminal success. The caller pastes the hint, and by this CLI's own model the status call is the acknowledging read — errors.py:401: "a deployment serves its result exactly once; re-running the status call cannot recover it."

So a caller who explicitly asked for --save spends the one-shot read to a terminal with nothing on disk. deployment status has its own --save (docstudio_cmd.py:204) that would have kept it.

Fix: append the flag when the run carried one — f"... status {target} {found} --save {save}"; save is already in scope at this point. Otherwise mention that --save keeps the result.

[ritwik-g]

[High] [Lens 13] — the only assertion on the rescued payload cannot fail when the payload is redacted

verbatim_details=True (docstudio_cmd.py:207) is the line that keeps the one-shot result whole. Without it CLIError.to_dict runs redact_value(self.details), which collapses any key whose name segments hit a credential word.

The fixture's result bodies are {"total": 1}, None and {"total": 3} — and names_a_secret("total") is False. So the redacted and verbatim payloads are byte-identical here, and assert error["details"] == PARTIAL_FAILURE passes either way.

Mutation-proven: deleting verbatim_details=True at docstudio_cmd.py:205-207 leaves 398 passed, 1 skipped — the full baseline, no test named. The comment on line 1481 asserts exactly the property the assertion does not pin.

The consequence of a future edit dropping the opt-out is silent: a caller's extracted field named policy_key, access_key, token or plain key comes back ***REDACTED***, and because the status read is one-shot that value survives nowhere else. Confirmed: redact_value({'result': {'policy_key': 'X', 'total': 1}}) → {'result': {'policy_key': '***REDACTED***', 'total': 1}}.

Fix — one word in the fixture: give one succeeded entry a result key the redactor matches, e.g. "result": {"policy_key": "PK-1", "total": 1}. The existing assertion then fails the moment the opt-out is lost.

The code is correct today; this is the guard on it that cannot fail.

[ritwik-g]

[Medium] [Lens 11] — the reorder fixes finding #5 but trades one unrecoverable state for another, and the comment overstates what survives

The order is now push-main → tag → release → publish. That does fix the original hazard. But uv publish is the step most likely to fail transiently (PyPI outage, OIDC hiccup, network), and when it does you are left with a public GitHub release for a version that is not installable: main carries the bump, tag vX exists, the release is live, PyPI has nothing.

The retry is blocked. version_bump: none recomputes X and exits at the existing-tag guard (:105-108 — fetch-depth: 0 at :51 means the tag is visible), while patch publishes X+1 and leaves X permanently absent behind a live release.

The comment at :131-134 says "a tag, a branch and a release can all be deleted" — omitting the fourth artifact, the bump commit already pushed to the default branch, which needs a revert push rather than a delete.

Fix: create the release as a draft and un-draft it after uv publish succeeds; or let a retry skip the tag/push steps when vX already exists rather than exiting 1. If the order is intentional as-is, at minimum correct the comment to name the commit on main and record the manual cleanup sequence.

[ritwik-g]

[Medium] [Lens 16] — warn()'s docstring states an import relationship that does not exist

The docstring justifies the sink indirection with "The config, overlay and credential registries are all imported by it [the output layer], so they cannot import it back."

output.py imports only errors — output.py:32-38 is its entire first-party import list. Neither config.py nor overlay.py is imported by the output layer, and neither would create a cycle importing it (output → errors → stdlib). Both import from errors, not into it.

A maintainer reading this believes an architectural constraint binds three modules when it binds one, and will preserve or extend the held-note machinery for modules that could call diagnostic directly.

Fix — prefer deletion over a rewrite: drop the second sentence, keeping "this is the seam that keeps their notes subject to the same --quiet as every other diagnostic." The rule is already stated correctly and scoped correctly at app.py:176-177 — "Modules the output layer imports cannot import it back" — which is true of errors.py, the module that actually has the constraint.

[ritwik-g]

[Medium] [Lens 16] — the pin guard is described as catching something it cannot

"tests/test_specs.py fails … if the pin in pyproject.toml has moved without the spec being re-synced."

The test compares provenance.json's client string against the pyproject.toml pin — two hand-edited strings. Bump the pin, edit client to match, leave docstudio.json untouched: all three tests pass. The sha256 test passes because the file is unchanged, the entry-set test passes, and the pin test passes because the two strings now agree.

The guard is a prompt to re-sync, not proof that re-syncing happened. This paragraph's whole purpose is to say what the check does and does not catch, and the following sentence never corrects it because that one is about the hash only.

Fix: delete the over-strong clause and let the sentence stop at the hash. .claude/skills/bump-client-pins/SKILL.md:55-58 already states the same guard accurately — "fails if a vendored file stops matching its recorded sha256, if its client no longer equals the pin in pyproject.toml, or if a spec has no provenance entry" — naming the mechanism without promising it detects a stale copy.

[ritwik-g]

[Medium] [Lens 13] — the only BrokenPipeError coverage cannot fail on the behaviour it claims

This is the sole test of the arm at __main__.py:73-79, and it exercises none of it. Under capsys, sys.stdout is a CaptureIO over a BytesIO, so sys.stdout.fileno() raises io.UnsupportedOperation — which subclasses OSError and is therefore swallowed by the enclosing contextlib.suppress(OSError). The os.dup2(…devnull…) redirect never executes.

The trailing print("still writable") then writes to an in-memory buffer that was never broken — the BrokenPipeError came from a monkeypatched load_config, not from stdout. What survives is only the exit-code assertion.

Mutation-proven: replacing both lines of the suppressed block with pass leaves 394 passed, 1 skipped, fully green. grep -rn BrokenPipe tests/ returns only this line, so nothing else covers it. unstract whisper extract … | head -1 is ordinary usage for this CLI, and the redirect is what stops a second BrokenPipeError during interpreter-exit flush.

Fix: use capfd instead of capsys — it gives stdout a real file descriptor, so fileno() succeeds and dup2 runs — then assert the redirect happened, or spy on os.dup2.

[ritwik-g]

[Medium] [Lens 13] — the status is None branch is the one shape of the defect no test exercises

This is a second detection rule, for a per-file entry carrying an error but no status key — a real shape, since the vendored spec lists only file as required on FileResult.

No entry in PARTIAL_FAILURE omits status, so the rule is never reached.

Mutation-proven: replacing the branch body with pass leaves 398 passed, 1 skipped, the full baseline. Contrast the sibling rule one line below, where dropping .casefold() fails 3 named tests — so the coverage is real for one rule and absent for the other.

If this rule is wrong or gets removed, a batch whose failures arrive in that shape goes back to exit 0 with a document's output missing — the exact defect this commit exists to fix — and nothing notices.

Fix: add a fourth entry to PARTIAL_FAILURE with "file": "noStatus.pdf", no status key and a non-empty error, and extend the two failed_files assertions to ["bad.pdf", "noStatus.pdf"]. That covers both rules from the fixture already in place.