Repository navigation
feat(bom): build BOMs without a security scan with bom repo and bom image (PPSC-1704) - #371
Draft
omriarnon-armis wants to merge 7 commits into
Draft
omriarnon-armis wants to merge 7 commits into
omriarnon-armis wants to merge 7 commits into
Conversation
Add a bom command that builds BOMs without a security scan. It uploads what scan repo and scan image upload, sends scan_type "bom" on POST /api/v1/ingest/scan, waits as a scan does, and downloads the BOMs through the --bom code. No findings are fetched or reported. - Flags: --type (sbom, aibom, cbom, all; default all, the combined BOM), --format (the --bom-format values) and --output (a directory or -, default .armis). The checks, file names, the safe .armis write, the refetch and the empty-BOM line are those of --bom. A refused pair with the default type says "--type all (the default)". A missing --output path that ends in .json is refused before upload, with a hint to use --output - (for one type) or a directory. - Exit codes: 0 when at least one BOM was written and none failed to download; 2 when no BOM was written or a built BOM could not be downloaded; 1 on errors; 130 on Ctrl+C. There is no --fail-on. - bom repo takes --include-files and --include-tests and sends the git hints scan repo sends, under the same conditions (wantGitHints), so the BOMs name the repository and commit. bom image takes --pull and --tarball and sends image_ref as scan image --bom does. Both take --scan-timeout, --upload-timeout and --no-progress, registered with scan's through one helper. - The request carries bom_types, bom_format and image_ref, with sbom_generate and vex_generate false and no custom_scans. - A 422 that rejects scan_type as a request validation error, which a server without BOM-only scans gives, exits 1 with "this Armis server does not support BOM-only scans yet; run: armis-cli scan repo <path> --bom <types> ...". The command after "run: " is the matching scan --bom command and pastes into a shell as printed: the target, the --tenant-id, --region and --dev flags typed, --include-files, --include-tests or --pull, the format and the output directory as given. Only values a shell would read differently are single-quoted, and a value with a control character is shown as a placeholder such as <path>. Credentials and values from the environment are left out. - A detail.code refusal prints the existing "the server refused the BOM request (...)" form, naming --type and --format. - The progress line while the server works reads "Building BOMs...". Shared code: NormalizeBOMFlags and ValidateBOMFlags take the flag names for their messages; scan image and scan repo target checks, the timeout check, the git-hint condition, the BOM completions and the artifact downloads moved into helpers that both commands call. scan output and requests are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nks (PPSC-1704) The CLI names the BOM files in a --bom-output or bom --output directory, as it does in the default .armis, but wrote them in place, so a symlink committed at <artifact>-bom-<type>.json in a scanned repository redirected the write. Write them as in .armis: a symlink or non-regular file at the name is refused, and the data goes to a 0600 temp file that is renamed over the old one. The typed directory itself is still opened as given, a symlinked one included; only the default .armis must be a plain directory. File paths from --sbom-output, --vex-output and --bom-output - are written in place as before, so /dev/stdout, FIFOs and NUL keep working. saveArtifact takes a write mode (in place, default directory, typed directory) instead of a bool; the replace code is shared by both directory modes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…errors (PPSC-1704) The scan status shown in the progress line for an unknown state, a failed scan's last error, the scanner-skip reason (warning and summary) and the fields of scan status were printed as the server sent them. Drop control characters, such as terminal escapes, with util.StripControl, as the BOM refusal and the raw /scan error already do. bom repo and bom image reach the same code. StripControl also dropped line breaks and tabs, so a multi-line server error ran its words together. A run of whitespace that holds a \n, \r or \t now becomes one space between words; other text, non-ASCII included, is kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- README: a "BOMs Without a Security Scan" section with the flags, defaults, exit codes, the .json output refusal and the older-server message, plus pointers from the global flags note, the --bom notes and the exit codes list. - docs/FEATURES.md: rows for --type, --format and --output in the flags table, and a short section on the bom commands. - docs/ci-examples/README.md: a note on bom for jobs that need only BOMs. - CHANGELOG: Added lines under Unreleased, one per topic. - CLAUDE.md: the new files in the command list. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…PSC-1704) TestScan_TempFileCleanedUpOnCancel globbed the shared temp directory for armis-repo-*.tar.gz and reported any file it had not seen before as a leak, so a repo scan running at the same time in another test or package failed it now and then under go test ./.... Point TMPDIR, TMP and TEMP at a directory of its own; it still fails when the scanner leaves its temp file behind. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ts (PPSC-1704) normalizeRepoName stripped the userinfo of an origin URL but not its query or fragment, so https://github.com/org/repo.git?access_token=X was sent as repo_name "org/repo.git?access_token=X". scan repo and bom repo send this hint. For http and https URLs, drop everything from the first ? or # before deriving owner/repo. git reads ? and # as path characters in ssh://, git:// and file:// URLs, scp-like remotes and local paths, so their names stay as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…704) A BOM type can come back generated or empty while bom_summary lists failed_tools, such as a tool that finds part of the components. The CLI wrote the file, exited 0 and said nothing, so the BOM could be partial without a word, and an empty one printed "No ... components found". After the BOM lines, print one warning when at least one requested type was written and failed_tools is not empty: "BOM tools failed on the server: <tools>; <types> may be incomplete". The tools have control characters stripped and are deduplicated and sorted; the types are the written ones, since a failed type that wrote nothing names the tools in its own warning. The exit code does not change. scan --bom and the bom commands share the code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| if unicode.IsControl(r) { | ||
| return -1 | ||
| var b strings.Builder | ||
| b.Grow(len(s)) |
Test Coverage Reporttotal: (statements) 75.7% Coverage by function |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related Issue
Type of Change
Problem
scan repo --bomandscan image --bomare the only way to get BOMs, and they run a full security scan first. A job that needs only the BOMs waits for findings it does not use. The server is gaining a BOM-only scan that builds the same BOMs and takes less time.Solution
Commands:
armis-cli bom repo [path]andarmis-cli bom image <name> | --tarball <file>.--type:sbom,aibom,cbom,all, comma-separated or repeated. Defaultall, the combined BOM: one file with every type.--type=is an error, as--bom=is.--format: the--bom-formatvalues and aliases, defaultcyclonedx-1.7. The same pairs are refused before upload:aibomwithspdx-2.3, andcbomorallwith any SPDX value. So--format spdxneeds--type sbomoraibom; without--type, the message says--type all (the default).--output: a directory or-, default.armis, with the--bom-outputrules. A path that does not exist and ends in.jsonis refused before upload, with a hint to use--output -(offered only for one type) or a directory; existing directories and-are unchanged.bom repoalso takes--include-files(with the "BOMs cover only the uploaded files" warning) and--include-tests.bom imagetakes--pulland--tarballwith thescan imagechecks. Both take--scan-timeout,--upload-timeout,--no-progressand the global auth flags.--fail-on,--format human|jsonor--group-by.--typeand--formatcomplete in the shell.Request: the upload, status polling and results calls are those of
scan repoandscan image.POST /api/v1/ingest/scancarriesscan_type: "bom",bom_types,bom_formatas typed, andimage_reffor images (same rules asscan image --bom).sbom_generateandvex_generateare false, andcustom_scansandsbom_formatare absent.bom reposends the git hintsscan reposends (repo_name,git_sha,origin_sha), under the same conditions, so the server names the BOM root after the repository and commit instead of a barerepository./normalized-resultsis never called.Output: stderr shows the same progress as a scan, with
Building BOMs...while the server works, then the--bomlines:<Label> saved to: <path>per file,<Label> written to stdoutwith--output -, the empty-BOM line, and the "not available" warnings. stdout holds a BOM only with--output -. No component count is printed, as--bomprints none.Exit codes:
0when at least one BOM was written and none failed to download (empty BOMs count; types not offered only warn);2when no BOM was written (scan completed but no BOM was written: <labels> not available), including--output -with nothing to write, or a built BOM could not be downloaded, as with--bom;1on errors;130on Ctrl+C, including during a download. Help, README, FEATURES, the CI note and the CHANGELOG use the same wording.Servers:
scan_type: "bom"with a 422 request validation error onscan_type. The command exits 1 withthis Armis server does not support BOM-only scans yet; run: armis-cli scan repo|image <target> --bom <types> [--bom-format <format>] [--bom-output <dir>]. The text afterrun:is the matchingscan repo|image --bomcommand, which runs a security scan as well, printed without outer quotes so it pastes into a shell as is. It leaves credentials and environment values out of the line, so it uses those of the shell it is pasted into. The target is the path, the image name or--tarball <file>as given, followed by the--tenant-id,--regionand--devflags typed on the command line, then--include-filesand--include-tests(repo) or--pull(image by name) when given, so the command reaches the same tenant and uploads the same content. Values from environment variables stay implicit, and credentials (--token,--client-id,--client-secret) are never repeated; the format and directory appear when they are not the defaults. Only values a shell would read differently are single-quoted (./appstays as is,my appbecomes'my app'), as is any value holding=or~(zsh expands them inside a word with MAGIC_EQUAL_SUBST) or#, with'written as'\'', so!,$, backticks and spaces paste back unchanged. A value that is not valid UTF-8 or holds a control character, such as a tab, a line break or a C1 byte, is shown as a placeholder (<path>,<image>,<file>,<dir>) rather than as a different path. A repo path that starts with-is printed as./-x, soscan repodoes not read it as a flag; an image name that starts with-is a placeholder. A test splits the printed command withshand gets the original arguments back.{"detail": {"code", "message"}}refusal prints the existing form, naming the bom flags:the server refused the BOM request (--type <types> --format <format>): <message> (<code>). This coversbom_types_required,custom_scans_not_allowed,sbom_vex_not_allowed,artifact_type_not_supportedandexport_format_unavailable. Any other 422 keeps the raw text.<Label> was requested but not available in results, and the command exits 2, since no BOM was written.Shared code: nothing is copied from the scan commands.
cmdutil.NormalizeBOMFlagsandValidateBOMFlagstake the flag names for their messages (cmdutil.ScanBOMFlags,cmdutil.BOMCommandFlags).scan.BOMOptions.Onlymarks a BOM-only scan. The repo and image scanners sendscan_typefrom it, showBuilding BOMs..., and return after the downloads without fetching findings.scan.DownloadArtifactsreplaces the download block both scanners had;scan.BOMOnlyResultbuilds the BOM-only result.scan imagetarget checks, thescan repopath and--include-fileschecks, the timeout check, the upload flags and the BOM completions moved into helpers that both commands call.OnRequest,OnScanRequest,ScanStatusandScanBodyfor the tests.Also in this PR (separate commits):
fix(scan): BOM files in a typed output directory (scan --bom-output <dir>,bom --output <dir>) are now written as in.armis: a symlink or non-regular file at the file name the CLI picks is refused, and the data goes to a 0600 temp file renamed over the old one. The directory itself is still used as given, a symlinked one included. File paths from--sbom-output,--vex-outputand--bom-output -are written in place as before, so/dev/stdout, FIFOs andNULkeep working. A link committed in a scanned repository could otherwise redirect the write.fix(scan): therepo_namegit hint drops the query and fragment of anhttporhttpsorigin URL, sohttps://github.com/org/repo.git?access_token=Xgivesorg/repoinstead of sending the token.bom reponow sends git hints too, so the shared helper is fixed.ssh://,git://,file://, scp-like remotes and local paths keep?and#, which git reads as path characters, so their names are the same as on main.feat(scan): a BOM written while BOM tools failed on the server (bom_summary.failed_toolsset on ageneratedoremptytype) now printsBOM tools failed on the server: <tools>; <types> may be incompleteafter the BOM lines, instead of passing silently. Tools are deduplicated, sorted and stripped of control characters; only written types are named, so afailedtype that already named the tools gets no second line. The exit code does not change;scan --bomandbomshare it.test(scan):TestScan_TempFileCleanedUpOnCancelglobbed the shared temp directory and failed now and then when another test or package scanned a repo at the same time; it now uses a temp directory of its own.fix(scan): control characters are stripped from server text the CLI prints: an unknown status in the progress line, a failed scan's last error, the scanner-skip reason, and every field ofscan status.bomreaches the same code asscan.util.StripControlnow turns a run of whitespace holding\n,\ror\tinto one space between words, so multi-line server errors no longer run words together; other C0/C1 controls are dropped, and non-ASCII text (Cyrillic, CJK, emoji) is kept.Docs: a README section "BOMs Without a Security Scan"; flag rows and a short section in
docs/FEATURES.md; a note indocs/ci-examples/README.md; CHANGELOG lines under Unreleased (Added, one per topic, and two Security lines); theCLAUDE.mdfile list.Testing
Automated Tests
What ran:
gofmt -l .: no files.go vet ./...andGOOS=windows go vet ./...: clean.golangci-lint run --config .golangci.yml ./internal/cmd/... ./internal/api/... ./internal/scan/... ./internal/testutil/... ./internal/output/... ./internal/model/... ./internal/util/...: 0 issues.go test -race -count=1 ./...: 23 packages pass.go test -count=1 -json ./...: 4,289 tests and subtests pass (1,294 top-level), 0 fail. Three existing tests skip (platform or mock-server limits); none is new.go test -race -count=3 -shuffle=on ./...: 23 packages pass, so the command-line tests leave no flag state behind.TestClient_StartIngest,TestSaveArtifactOutputPaths,TestSBOMVEXDownloader_DownloadBOMs,TestFormatScanStatus,TestStripControland the two completion tests; 164 tests and subtests in all. They are table-driven and use the mock API server:bom repoandbom image(scan_type, types, format, image_ref, git hints for a full upload of a checkout and none otherwise, no SBOM, VEX or custom scans, no findings call), throughRunEand through the real command line;.jsonoutput path, exit 1 with no request to the server, and a symlinked default.armis;shandzsh; placeholders for values with control characters;wantGitHintsforscan repoandbom repowith--include-filesand--changed; a region from the environment kept out of the hint; andrepo_namewithout a URL query or fragment;.armisand a typed directory with mode 0600, SPDX and CycloneDX 1.6 keys, stdout mode, types not offered, one of two written,--output -with nothing to write, an empty BOM;main.gomaps them;<typed dir>/<artifact>-bom-sbom.jsonrefused with its target untouched, throughscan repo --bom-outputandbom repo --output, and a symlinked typed directory still written through;scan statusfield; multi-line text joined with single spaces, and UTF-8 text unchanged;--typeand--formatcompletions.-count=2. Both now start each run from the flag defaults.Manual Testing
Built the binary and ran it against a local stub of the ingest API:
bom repo <dir>: onescan_type: "bom"request withbom_types: ["all"],<dir>-bom-all.jsonwritten with mode 0600, exit 0, no findings call.bom image --tarball app.tar --type sbom --format spdxagainst a stub answering the older-server 422: the plain message naming thescan imagecommand, exit 1.export_format_unavailablerefusal: the--type ... --format ...form, exit 1. A failed download: exit 2.--type cbom --output -: the document on stdout only.scan repo --bomandscan image --tarball --bom: the main and branch builds sent the same/scanbody (scan_type: "full") and gave the same stdout, stderr, files and exit code.--helpofscan,scan repo,scan image,scan status,scan sbomandsupply-chain checkis the same on both.Real-world test
The branch binary ran against a development server with BOM-only scans and a staging server without them. Inputs were public repositories and images pinned by commit or digest, on two tenants whose scans the server routes differently.
bom repo .againstscan repo . --bom allon the same copy: mcp-quickstart 29 s against 296 s, and 40 s against 281 s; java-jwt 13 s against 75 s, and 39 s against 170 s.scan --bomone (467 and 13), and every file matched the server'sbom_summary..tarand.tar.gz(combined, 407). An AI-BOM of an image warns "not supported" and exits 2 with nothing written.--output -piped to a JSON parser;-with two types andx.jsonrefused before upload; a symlinked.armisrefused before upload; a link at a file name in a typed directory refused, its target unchanged; reruns replace files with mode 0600.bomscan is recorded withscan_typebomand no findings. The server logs show only the BOM tools.--tenant-id,--regionand--pull, and no token.auth0/java-jwt@1d589a4…, as withscan repo --bom. A copy without.gitand a run with--include-filessend none.The first round found two CLI bugs, both fixed here:
bom reposent no git hints, so the BOM root was a barerepository, and the hint dropped--tenant-id. It also found a server bug: on one scan path, two of three runs returned an incomplete BOM, once with a failed tool. That is fixed on the server. The CLI wrote that BOM without a word, which is why this PR adds the failed-tools warning.Reviews
Code, standards and security reviews ran on the full diff before the real-world test; their fixes are in the commits above. After the test, a targeted review ran on each of three fix rounds. Each found only low items, all fixed:
bom reposends the git hintsscan reposends, under the same rule (wantGitHints).--tenant-id,--regionand--dev. Credentials and environment values never appear.=or~(zsh expands them inside a word with MAGIC_EQUAL_SUBST) or a leading#is quoted. A value with a control character or invalid UTF-8 becomes a placeholder. A repo path that starts with-prints as./-x.--changedcheck, the image tenant flags, and a region from the environment.repo_namegit hint drops the query and fragment ofhttpandhttpsorigin URLs only. Every other remote keeps the name it has on main, so incremental baselines don't reset.-shufflewere fixed: one left colors on, the other globbed the shared temp directory.The security passes found no way for a credential to reach the request, the hint or the logs once item 5 is in.
Reviewer Notes
scan --bomkeeps working.--bomdoes.scan --bom-output <dir>too: a file replaced there now ends with mode 0600, as in.armis, and a symlink at the BOM's name is refused instead of written through.--typehas no flag default.allis applied when--typeis not given, so--type=stays an error. Help shows(default: all), as--outputshows(default: .armis).--outputdefaults to"", not.armis, so the default goes through the strict.armiswrite of--bom(plain directory, no symlinks, temp file and rename). A typed--output .armisis written as typed, as--bom-output .armisis.--format,--outputand-f/-omean other things onscan;bomregisters its own--formatand--outputand no short forms.Checklist
🤖 Generated with Claude Code