Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
{
"name": "brief2ship",
"description": "Search and score existing code before greenfield AI builds, then ship with proof",
"version": "0.7.0",
"version": "0.8.0",
"source": "./",
"author": {
"name": "Michael Costea",
Expand Down
2 changes: 1 addition & 1 deletion .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "brief2ship",
"description": "Repo-first discovery and verified delivery for AI coding agents",
"version": "0.7.0",
"version": "0.8.0",
"author": {
"name": "Michael Costea",
"url": "https://michaelcostea.com/"
Expand Down
15 changes: 15 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,20 @@
# Changelog

## 0.8.0 - Unreleased

- Preserve domain intent and record structured requirement evidence; low keyword overlap no longer authorizes a clean build.
- Expand bounded PyPI retrieval using index-confirmed concept hints and observed metadata relevance.
- Validate malformed manifests, isolate candidate inspection failures and preserve bounded source-order README evidence.
- Save provider checkpoints and support explicit retries with freshness and scope validation.
- Add automatic output directories, two default CLI static inspections, optional text summaries and stderr progress.
- Preserve status, pinned identities, required checks and next actions in shared handoff output and agent instructions.
- Add realistic synthetic mismatch/abstention regressions. Human-reviewed live-provider accuracy remains unmeasured.
- Harden integration-review counterexamples: preserve alternative-platform uncertainty and domain-language intent, validate hydrated fields before mutation, retain unknown aggregate dependencies after incomplete inspection, validate nested Poetry declarations, filter irrelevant hints, protect exact package hydration and prioritize topic terms over generic artifact words.
- Keep detected negated/optional target requirements and unresolved dependency includes or unsupported declarations unknown rather than authorizing false negative decisions.
- Include pinned version/commit, original source observation times, and structured requirement results/evidence in human-readable handoffs.

This source package is an unpublished update based on v0.7.0; no v0.8.0 tag or PyPI publication is claimed.

## 0.7.0 - 2026-09-05

- separate discovery health, decision status, and reuse disposition; provider failures, empty retrieval and missing inspections now yield `inconclusive` with CLI exit 5, never a false clean-build recommendation
Expand Down
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,23 @@ Brief2Ship is a dependency-free Python CLI and portable Agent Skill for **repo-f

*Historical v0.6.2 dogfood receipt (not a v0.7.0 result): `local/brief2ship`, 70.50/100, inspected, `selective-reuse`. [Read the source transcript](https://github.com/five0nit/brief2ship/blob/v0.6.2/docs/assets/demo-transcript.txt) or [see more decision cases](https://github.com/five0nit/brief2ship/blob/v0.6.2/docs/case-studies.md).*

## v0.8.0 (unreleased)

This checkout includes requirement-aware decisions, stronger package retrieval,
inspection failure isolation, retry checkpoints and a simpler first run.
Install this update from the repository checkout or extracted source directory:

```text
uv tool install .
brief2ship discover "Python retry library with exponential backoff" --sources github,pypi --text
```

The CLI creates an output directory and performs up to two static inspections.
Use `--summary` for JSON, or `--progress` to report progress on stderr.
See [the update and migration guide](docs/update-0.8.0.md) for retry behavior,
compatibility changes, limitations and validation. The versioned installation
commands below install the previous published GitHub release, v0.7.0.

## v0.7.0 — reliable reuse decisions

This release adds reliable reuse decisions: explicit `inconclusive` outcomes,
Expand Down
10 changes: 9 additions & 1 deletion ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Brief2Ship stays deliberately lean. Roadmap items must improve repo-first decisions, portable installation, evidence quality, or verified delivery without turning the project into a general-purpose agent framework.

## Now — v0.6.x distribution and adoption
## Distribution and adoption (carried forward from v0.6.x distribution and adoption)

- publish signed, tagged Python distributions to PyPI through GitHub Trusted Publishing;
- validate and publish `skills/brief2ship` through GitHub's Agent Skills workflow;
Expand All @@ -21,6 +21,14 @@ Brief2Ship stays deliberately lean. Roadmap items must improve repo-first decisi

Included in v0.7.0. PyPI publication and broader host validation remain separate work.

## 0.8.0 source update

Implemented locally: requirement evidence and safer abstention, bounded PyPI
metadata retrieval with concept hints, manifest failure isolation, resumable
provider checkpoints, portable first-run CLI and consistent agent handoff.
See [the update guide](docs/update-0.8.0.md). Publication and hosted CI remain
separate; human-reviewed quality measurement is still outstanding.

## Next evidence-quality work

- human-reviewed real-world briefs and provider recall benchmarks, beyond synthetic regression fixtures;
Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
0.7.0
0.8.0
137 changes: 137 additions & 0 deletions docs/update-0.8.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# 0.8.0 update and migration guide

Status: unpublished source update based on v0.7.0 commit
`2119cd8de44104641c13aa39c80580dc0293c5a4`. The source version is 0.8.0;
this does not imply a GitHub tag, hosted CI result or PyPI release exists.

## Run the updated CLI

From the extracted source directory:

```text
uv tool install .
brief2ship doctor
brief2ship discover "Python retry library with exponential backoff" --sources github,pypi --text
```

The discovery command works in PowerShell and POSIX shells. Without `--output`,
it creates a fresh system temporary directory, performs up to two static
inspections and prints a readable decision. Normal discovery does not execute
candidate code or install candidate dependencies. Provisional and inconclusive
results are evidence states, not a test failure.

Use `--summary` for JSON, `--progress` for stderr progress, and `--inspect-top 0`
for an intentional search-only run. Progress is automatic on an interactive
stderr. Existing `--output DIR` calls still print one receipt path unless
`--summary` or `--text` is requested. The Python API retains its explicit
`DiscoveryConfig.inspect_top` default of zero for compatibility.

## Retry an incomplete search

Each completed provider writes `checkpoint.json` before vulnerability checks
and repository inspection. Retry with the same query, sources, local roots and
per-source limit:

```text
brief2ship discover "Python retry library with exponential backoff" --sources github,pypi --summary --output first-run
brief2ship discover "Python retry library with exponential backoff" --sources github,pypi --summary --resume first-run --output retry-run
```

Successful public-source observations up to 24 hours old may be reused. Failed,
partial, expired and local sources are fetched again. Vulnerability checks,
ranking, requirements and inspections run again. Display, inspection and timeout
budgets may change on retry. Use a fresh output directory; inspection clones
are not reused. An interruption during the first provider can still leave no
completed checkpoint. Later failures retain completed provider observations.

Checkpoints are bounded to 10 MB when read, typed and checked against the exact
source scope. They are unsigned local evidence; resume only checkpoints you
trust. Individual source observation times remain visible and do not become
newer on reuse. A retry can inspect a newer upstream commit than the cached
metadata; review the new pinned identity and outstanding checks.

## Requirements and decisions

Candidates add structured `requirement_checks` with pass/fail/unknown status
and evidence. Existing string checks remain. Domain phrases retain meaningful
terms; supported explicit contradictions prevent selection. Unsupported
constraints remain unknown. Declaration-based matches are not runtime
verification: language metadata does not prove bindings, offline behavior or
compatibility with a target device.

The planner does not implement a Boolean requirement grammar. Compound `or`
and `and/or` requirements, detected negation and optional/preferred targets remain
unknown for manual review rather than becoming mandatory constraints. This is a
conservative heuristic, not a complete natural-language grammar. `parser for Java`
retains Java as domain intent; explicit
implementation cues such as `written in Java` can establish a runtime requirement.

A low keyword score alone no longer authorizes `build-clean`. It can leave a
candidate and the run `inconclusive`. Review explicit negative evidence before
concluding none of the evaluated candidates can be reused. This is a bounded
comparison, not proof about every package or repository.

The full receipt schema remains `brief2ship-discovery-v2`; new fields are
additive. The summary now includes selected identity/version/commit,
requirement evidence, source observation times and a `next_action`. Human text
uses that same summary contract. Consumers must handle candidate-level
`inconclusive` and retain provisional checks even when the process exits zero.
Exit codes retain their v0.7 meanings: 0 supported/provisional, 5 inconclusive,
and 2/3/4 input/policy/operational errors.

## Retrieval and inspection

PyPI retrieval supplements name matching with a small transparent concept-to-name
hint list confirmed against the observed Simple index. It hydrates a bounded
pool, ranks observed name/summary/keyword/description evidence, and records
retrieval provenance. This is not an ecosystem-wide semantic index. A cold
Simple-index fetch can still be slow; no paid search service or model call is
introduced. The retrieved set is not exhaustive.

Exact package identity receives priority before the hydration budget is applied.
When meaningful topic terms exist, generic artifact words such as `library` and
`package` do not crowd them out; ignored terms remain visible in the receipt.
Hint-only candidates require matching observed metadata even when result slots
are empty. These guards do not establish broad search recall or accuracy.

Manifest parsing distinguishes invalid declarations from verified zero
dependencies. Candidate metadata/I/O failures produce inspection evidence
instead of losing the whole result. README terms retain bounded source order
instead of alphabetically discarding terms after the first 200. Limits and
partial evidence remain explicit. Known blocked candidates no longer consume
inspection slots ahead of otherwise eligible candidates just through keyword fit.

Incomplete traversal and unreadable or unsupported recognized manifests leave
the aggregate dependency count unknown, not a verified subtotal. A cap limited
to README feature evidence does not invalidate otherwise complete dependency
inspection. Hydrated metadata is type-checked before candidate fields are changed.
Requirements-file includes/options/markers and unsupported Go module declarations
remain unknown; only the supported static declaration subset is counted. No
included files or package-manager commands are executed to resolve uncertainty.

## Agent instructions and delivery

The skill focuses on choosing reusable implementations. Existing canonical
work, supplied-source reports and direct page extraction can use reduced or
conditional workflows. Detailed design/report/scraping guidance is loaded
only when relevant. Portable examples use automatic output directories.
The build receipt retains status, checks, identities, provenance and next actions.

## Verification and remaining work

Run `python scripts/validate-release.py` for documentation, unit/integration
tests and synthetic decision regressions. Save benchmark detail with
`python scripts/benchmark-discovery.py --output quality-report.json`.
The delivered ZIP includes exact local test/build logs and a validation report.
Local Windows results do not establish Linux sandbox behavior.

New mismatch, synonym and abstention cases are agent-authored synthetic
regressions. They do not establish human-reviewed real-world recall or accuracy.
A reviewed corpus of 30–50 real briefs and recorded provider responses remains
follow-up work. The hint list does not replace that evaluation.

Release publishing, PyPI trusted-publisher setup, hosted CI, real agent-host
installation trials and a new rendered demo require their own final verification.
Candidate-test runner improvements and writable Linux sandbox build/temp areas
were deferred pending Linux evidence. No production publication or account
configuration is part of this source package.
2 changes: 2 additions & 0 deletions docs/windows-powershell.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Native Windows PowerShell quickstart

For the delivered 0.8.0 update, extract the ZIP and start in its `source/brief2ship` directory; skip the v0.7.0 clone below. Follow the isolated install instructions or see [the update guide](update-0.8.0.md).

Use PowerShell with Python 3.11, 3.12 or 3.13 installed. Start in a fresh
Brief2Ship v0.7.0 source checkout containing `pyproject.toml`:

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "brief2ship"
version = "0.7.0"
version = "0.8.0"
description = "Repo-first discovery for AI coding agents: search and score existing code before building, then ship with proof."
readme = "README.md"
requires-python = ">=3.11"
Expand Down
52 changes: 46 additions & 6 deletions scripts/benchmark-discovery.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT / "src"))

from brief2ship.discovery_decision import decide
from brief2ship.discovery_models import Candidate, InspectionResult, SourceReceipt
from brief2ship.discovery_scoring import rank_candidates
from brief2ship.render import atomic_write
from brief2ship.discovery_decision import decide # noqa: E402
from brief2ship.discovery_models import Candidate, InspectionResult, SourceReceipt # noqa: E402
from brief2ship.discovery_scoring import rank_candidates # noqa: E402
from brief2ship.render import atomic_write # noqa: E402


def run_benchmark() -> dict:
Expand Down Expand Up @@ -63,13 +63,53 @@ def run_benchmark() -> dict:
}
rows.append({"id": case["id"], "query": case["query"], "ranked": names,
"checks": checks, "passed": all(checks.values()), "ranking_ms": round(elapsed_ms, 3)})
# These close-competitor and abstention cases are also synthetic. They
# exercise decision semantics, not live retrieval recall or human accuracy.
for case in fixtures.get("adversarial_cases", []):
pool = []
for values in case["candidates"]:
url = "https://github.com/" + values["name"]
pool.append(Candidate(
source="github", url=url, repository_url=url, license="MIT",
updated_at=fixtures["clock"], dependency_count=0,
inspection=InspectionResult(
repository_url=url, status="inspected", commit="synthetic",
manifest_files=["pyproject.toml"], docs_files=["README.md"],
),
**values,
))
start = time.perf_counter()
ranked = rank_candidates(case["query"], pool, now=now)
elapsed_ms = (time.perf_counter() - start) * 1000
outcome = decide(ranked, [SourceReceipt("github", "ok", len(pool), returned=len(pool))])
selected = next((item for item in ranked if item.canonical_id == outcome.selected_id), None)
blocked = sorted(item.name for item in ranked if item.hard_blockers)
unknown = {check.requirement for item in ranked for check in item.requirement_checks if check.status == "unknown"}
names = [item.name for item in ranked]
checks = {
"selected": (selected.name if selected else None) == case["expected_selected"],
"decision_status": outcome.status == case["expected_status"],
"blocked_candidates": blocked == sorted(case["expected_blocked"]),
"unknown_requirements": set(case.get("expected_unknown", [])) <= unknown,
"no_false_clean_build": outcome.recommendation != "build-clean",
"no_blocked_top": not ranked[0].hard_blockers,
"no_mismatched_selection": not selected or not any(check.status == "fail" for check in selected.requirement_checks),
}
if case.get("expected_top"):
checks["top1"] = names[0] == case["expected_top"]
checks["top3"] = case["expected_top"] in names[:3]
rows.append({"id": case["id"], "query": case["query"], "ranked": names,
"decision_status": outcome.status, "checks": checks,
"passed": all(checks.values()), "ranking_ms": round(elapsed_ms, 3)})
times = sorted(row["ranking_ms"] for row in rows)
count = len(rows)
return {
"schema": "brief2ship-quality-report-v1", "scope": fixtures["provenance"],
"case_count": count, "passed_count": sum(row["passed"] for row in rows),
"top1_hits": sum(row["checks"]["top1"] for row in rows),
"top3_hits": sum(row["checks"]["top3"] for row in rows),
"ranking_case_count": sum("top1" in row["checks"] for row in rows),
"adversarial_case_count": len(fixtures.get("adversarial_cases", [])),
"top1_hits": sum(row["checks"].get("top1", False) for row in rows),
"top3_hits": sum(row["checks"].get("top3", False) for row in rows),
"false_clean_build_cases": sum(not row["checks"]["no_false_clean_build"] for row in rows),
"blocked_top_cases": sum(not row["checks"]["no_blocked_top"] for row in rows),
"ranking_median_ms": round(statistics.median(times), 3),
Expand Down
4 changes: 2 additions & 2 deletions scripts/validate-docs.py
Original file line number Diff line number Diff line change
Expand Up @@ -167,9 +167,9 @@
"crawl stayed same-origin",
],
"skills/brief2ship/SKILL.md": [
"sole repo-search skill",
"Decision contract",
"--local",
"agent-code entropy gate",
"Handoff and build receipt",
"exact repository/base",
"The 4 lanes",
"Report / Document",
Expand Down
Loading
Loading