feat(RHIDP-14129): add intent-based CLI commands and backstage-cli pass-through - #156
Conversation
…tions related will use backstage-cli Signed-off-by: Stephanie <yangcao@redhat.com>
…nto intent-based-cli
Signed-off-by: Stephanie <yangcao@redhat.com>
yangcao77
changed the title
yangcao77
changed the title
- Statically import command modules in commands/index.ts instead of using require(), since the backstage-cli bundler only follows static ESM imports/dynamic import() and silently dropped the require()'d files from the packed dist, breaking every command once installed from npm (Cannot find module './backstage-passthrough'). - Fix TS2352 in intent-errors.ts by adding a safe getStderr() helper instead of casting Error directly to Record<string, unknown>. - Restrict the PATH used to resolve backstage-cli via `which` to directories that aren't group/other-writable, addressing the SonarCloud S4036 PATH-search security hotspot in lib/client.ts. - Extract shared runEntityListAction/runRawAction/runSearchAction helpers and a registerPassthroughCommand helper to remove the heavy code duplication SonarCloud flagged across catalog/api/template/ search/docs/backstage-passthrough command files. - Fix pre-existing lint (no-empty, func-names) and prettier issues so the Checks job can get past the linter/prettier steps. Co-authored-by: Cursor <cursoragent@cursor.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
…ctory Move catalog/api/search/docs/template/backstage-passthrough and their supporting client/format/intent-errors/helpers modules into src/commands/intent-based-actions/, mirroring the existing export-dynamic-plugin/ and package-dynamic-plugins/ layout, with a single registerIntentCommands() entry point. Co-authored-by: Cursor <cursoragent@cursor.com>
SonarCloud S4036 still flagged spawnSync('which', ...) even with a restricted PATH env, since it pattern-matches on shelling out to a path-search utility rather than analyzing the PATH value. Replace it with a direct filesystem walk over PATH entries (skipping group/other-writable directories) and an accessSync executability check, avoiding the flagged pattern entirely.
Co-authored-by: Cursor <cursoragent@cursor.com>
kadel
left a comment
kadel
left a comment
There was a problem hiding this comment.
Two concerns about how backstage-cli is resolved and surfaced.
Resolution: findBackstageCliOnPath walks system PATH looking for a backstage-cli binary, but backstage-cli is not something people typically install as a standalone global binary, so the PATH walk is unlikely to find it. This means the npx -y @backstage/cli fallback is effectively the default path. This fallback silently downloads whatever latest is on npm without user confirmation. rhdh-cli is built against 0.36.3, so the downloaded version could behave differently, and -y suppresses the install prompt. This is a supply chain concern for a CLI meant for production use.
@backstage/cli is already a declared dependency of this project at 0.36.3. Could we resolve the binary from the installed dependency instead of walking PATH or downloading via npx?
Leaking backstage-cli identity: The passthrough commands expose backstage-cli's own output directly. Commander intercepts --help before it reaches backstage-cli, so passthrough commands show empty help with no options. But backstage-cli itself has useful help that's being hidden. Compare:
rhdh-cli auth login --help:
Usage: rhdh-cli auth login [options]
Log in to a Backstage/RHDH instance
Options:
-h, --help display help for command
backstage-cli auth login --help:
Usage:
backstage-cli auth login [flags...]
Flags:
--backend-url <string> Backend base URL
-h, --help Show help
--instance <string> Name for this instance
--no-browser Do not open browser automatically
rhdh-cli actions execute --help:
Usage: rhdh-cli actions execute [options]
Execute an action
Options:
-h, --help display help for command
backstage-cli actions execute --help:
Usage:
backstage-cli actions execute [flags...] <action-id>
Flags:
-h, --help Show help
--instance <string> Name of the instance to use
The rhdh-cli versions hide --backend-url, --no-browser, --instance, and the <action-id> positional argument. Running without --help (e.g., rhdh-cli actions execute with no args) does forward to backstage-cli but then shows backstage-cli branding instead of rhdh-cli.
For human users this is confusing. For AI agents discovering the CLI through --help, it's a blocker since they see no options and can't tell which CLI to use.
|
Thanks for the review! Fixed both: Resolution: |
Signed-off-by: Stephanie <yangcao@redhat.com>
f58172c to
a504d15
Compare
kadel
left a comment
kadel
left a comment
There was a problem hiding this comment.
Do we need the intent-based commands (template, catalog, api, search, docs) in this PR, or should they be a follow-up?
Every intent-based command maps 1:1 to actions execute — for example, rhdh-cli template list is just rhdh-cli actions execute catalog:query-catalog-entities --query '{"kind":"Template"}'. The passthrough layer (auth, actions) already gives users and agents full access to the same functionality.
The intent-based commands add ~1,000 lines with no tests, and one of them (template dry-run) already has a bug where it passes an entity ref as templateYaml instead of actual YAML content. The impact is low — the command just fails with an error, it can't cause any damage — but it shows the risk of shipping this much code without test coverage.
If we want to keep those extra commands they need test coverage
Would it make sense to merge just the passthrough commands (auth, actions, actions sources) first — they're solid and already working — and add the intent-based layer in a follow-up with proper test coverage? The repo already has a Jest setup in src/lib/*.test.ts, and most of the new code (formatting, error handling, entity extraction) is pure functions that are straightforward to test.
Signed-off-by: Stephanie <yangcao@redhat.com>
|
@kadel On whether we need the intent-based commands in this PR: yes, they're the actual point of this story https://redhat.atlassian.net/browse/RHIDP-14129. The goal is that users and agents should never need to know internal action names or the I agree the test coverage is needed, I initially did not add any as I didn't see any tests coverage for all other cmds in this repo. I've add some unit tests as part of the PR, the integration tests I have created it as a QE item. https://redhat.atlassian.net/browse/RHIDP-14254 |
I don't think that for agentic use this matters that much. The original backstage cli commands are well designed for AI use. If the goal for new commands is mainly human usage, then we need to think a little bit harder about how they look like and what arguments they expose. For example expecting humans to type JSON strings in terminal as arguments for CLI command is bad UX. Agents can deal with it, but people can't. Even simple tasks like searching only in Components I have to type JSON something like this would be much better cli experience: In the template execution it can also get quite complicated. More natural CLI experience should be something like this. The errors are currently also not presented in a user-friendly way Another usability problem for human use is that there is no way to easily list what parameters a template requires. The only way to do it is using following command. Which forces people to parse JSON. (without pre-filtering it with command outputWhen testing this I also found bug in in json output it is fine To summarize this: |
Signed-off-by: Stephanie <yangcao@redhat.com>
Signed-off-by: Stephanie <yangcao@redhat.com>
|
@kadel so I pushed a new commit for your suggested UX improvement on the input format, so they no longer require raw JSON: here is the summary: Summary1.
|
|
@kadel Re showing the template params and also the user-friendly error, as long as the bug https://redhat.atlassian.net/browse/RHDHBUGS-3698 you created. I would like to use follow up PR to address those issue. here is a analysis from claude Problem A — No easy way to see a template's inputsToday the only way to discover what i.e. fetch the whole entity and hand-parse nested JSON. What to build: a new read-only subcommand, e.g.
Decision needed: command name ( Problem B — Validation errors are an unreadable JSON dump
What to build: parse that payload into a friendly message.
How they connectThey pair naturally: the friendly error in B points the user at the command from A. Build A first so B can reference Effort / risk / sequencing
PR recommendation: clean second PR — also satisfies "split into multiple PRs" point |
|
@kadel PTAL |
|
/fs-review |
|
🤖 Finished Review · ✅ Success · Started 11:16 AM UTC · Completed 11:57 AM UTC Commit: Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $8.30 |
Signed-off-by: Stephanie <yangcao@redhat.com>
Signed-off-by: Stephanie <yangcao@redhat.com>
Signed-off-by: Stephanie <yangcao@redhat.com>
|
@kadel I've addressed the consistency concerns you raised. Here's a point-by-point response:
Your Concern: Commands like docs Response: This is now documented in README and also CLI reference, I will also document this in the release doc. The Documentation:
You reported the Input: Output: Verification that actions are available: All TechDocs commands work correctly on RHDH instances with the plugin installed. Setup Documentation Added: The comprehensive CLI documentation (
2. Inconsistent Entity ReferencesYour Concern: Different reference formats across commands:
Response: Not changed - keeping current design. Rationale:
3. Flag InconsistencyYour Concern:
Response: ✅ FIXED Changes Made: a) Standardized flag naming
b) Removed redundant JSON input flags
Rationale: More CLI-idiomatic than JSON strings; reduces redundancy c) Added
|
| Change | Before | After |
|---|---|---|
| Flag naming | docs list --entity-type Component |
docs list --kind Component |
| Filter support | api list --type openapi (no filters) |
api list --type openapi --filter spec.owner=team-a |
| Filter support | template list (no filters) |
template list --filter metadata.tags=nodejs |
| Filter flags | --filter and --filters on catalog list |
Only --filter (repeatable) on all commands |
| Template value flags | --value and --values |
Only --value (repeatable), optional |
| Template secret flags | --secret and --secrets |
Only --secret (repeatable) |
| Limit support | docs list (missing --limit) |
docs list --limit 10 |
| Template values required | --value required |
--value optional |
✅ Verified Working
docs getworks correctly withtechdocs-mcp-extrasplugin (test results above)docs listworks correctly and now supports--limitand--kinddocs coverageworks correctly- All actions are available on RHDH instances with the required plugin
❌ Not Changed (By Design)
| Item | Reason |
|---|---|
| Entity reference formats | Each command maps to its underlying action's parameter format; changing would add unnecessary transformation logic |
| TechDocs RHDH-only requirement | Plugin is RHDH-specific by design; already documented extensively with setup instructions |
RHDH Branding Improvements
In addition to consistency fixes, we've improved RHDH branding:
- Auth URL flag:
auth loginnow accepts both--rhdh-urland--backend-url - Example URLs: All documentation uses
rhdh.example.cominstead ofbackstage.example.com - Error messages: Reference "RHDH instance" instead of "Backstage instance"
Example:
# Both work
rhdh-cli auth login --rhdh-url https://rhdh.example.com
rhdh-cli auth login --backend-url https://rhdh.example.comSigned-off-by: Stephanie <yangcao@redhat.com>
Signed-off-by: Stephanie <yangcao@redhat.com>
Signed-off-by: Stephanie <yangcao@redhat.com>
…nto intent-based-cli
I did not say that it doesn't work properly. It works. I made a suggestion that we also need to have a way to trigger TechDocs build from CLI. Otherwise, users or agents won't be able to get docs without going to web ui first.
It doesn't matter what action it uses. We are exposing completely new commands to users. The whole point of introducing new commands is to provide nice human friendly UX. If you keep it as it is and just pass flags down to actions than what is point of introducing new commands? Users can run those actions directly. |
this feels a bit redundant, |
You are right on that, I misread the comment.
ahh. I somehow thought it was also updated the entity references as suggested Replaced Before: After: Key Features:
Affected Commands:
|
Signed-off-by: Stephanie <yangcao@redhat.com>
5437902 to
4ab0ca4
Compare
Signed-off-by: Stephanie <yangcao@redhat.com>
Signed-off-by: Stephanie <yangcao@redhat.com>
kadel
left a comment
kadel
left a comment
There was a problem hiding this comment.
Thank you for implementing unified entity references it makes command much easier to use.
docs search without plugin-search-backend-module-techdocs still fails with error 400
❯ ./bin/rhdh-cli docs search "rhdh"
Error: Invalid input to action "search:query"; caused by [
Try:
rhdh-cli docs search "getting started"
Instead of throwing missleading 400 error it should match docs get/list behavior which correctly suggests that it might be missing plugin.
Error handling in docs get command is weird:
Trying to get docs for non-existing component says that TechDocs were not found, not that entity was not found and it suggests building docs for non-existing entity and on top of that as exit code it returns non-error 0, instead of expected 1
❯ ./bin/rhdh-cli docs get system:default/non-existing
TechDocs content not found for system:default/non-existing
The documentation may not have been built yet.
Trigger build with: rhdh-cli docs build system:default/non-existing
Or visit the TechDocs page in RHDH to trigger a build.
❯ echo $?
0
running it on entity that exists but doesn't have docs built:
❯ ./bin/rhdh-cli docs get system:default/rhdh-local
TechDocs content not found for system:default/rhdh-local
The documentation may not have been built yet.
Trigger build with: rhdh-cli docs build system:default/rhdh-local
Or visit the TechDocs page in RHDH to trigger a build.
❯ echo $?
0
the output is ok, but it returns non-error exit code, i would also expect that this should return error 1, because operation was not successful.
After triggering built it correctly returns docs with non-error exit code (0), this looks OK this is what I would expect.
❯ curl -H "Authorization: Bearer $(./bin/rhdh-cli auth print-token)" http://localhost:7007/api/techdocs/sync/default/system/rhdh-local
<output>
❯ ./bin/rhdh-cli docs get system:default/rhdh-local
<output>
❯ echo $?
0
|
Addressed these issues:
see below for the output of |
Signed-off-by: Stephanie <yangcao@redhat.com>
Assisted-by: GPT-5
|
|
@kadel can you help merge this PR? I do not have permission |
|
🤖 Finished Retro · ✅ Success · Started 2:30 PM UTC · Completed 2:44 PM UTC Commit: Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $3.43 |
Retro: PR #156 — Intent-based CLI commandsTimeline
Review quality deltaThe agent and human reviews are almost entirely complementary with near-zero overlap (1 partial overlap out of 35 combined findings). The agent excelled at static code analysis: runtime crash in The biggest quality gap: the agent missed every design-level finding (6 UX issues, 3 architecture issues). It treated the code as "is this implementation correct?" rather than "is this the right design?" — because AGENTS.md encodes architecture and code organization patterns but not CLI UX design principles. Rework25 commits over 41 days with 7+ addressing review feedback. The human suggested splitting the PR on Aug 19 (day 14); continuing as a monolith extended the review cycle. The agent review on Sep 8 (day 34) added mechanical findings but the highest-impact issues had already been identified by the human. Evidence for existing issues
Proposals filed
|
3 participants
https://redhat.atlassian.net/browse/RHIDP-14129
Adds intent-based subcommands and backstage-cli pass-through commands to rhdh-cli
Today, interacting with a running RHDH/Backstage instance from the CLI requires
backstage-cli actions execute <pluginId>:<actionName>with internal action IDs and raw JSON input. This is very bad experience for human operations.This change makes rhdh-cli the single entry point:
All intent-based commands support
--output jsonfor agent consumption and--instance <name>for multi-instance targeting.The local metadata file is still going to use the config file for
backstage-cli, so that existing backstage-cli user can migrate to userhdh-cliwith no extra effortsee recording:
rhdh-cli.mov