Skip to content

feat(cli): add --debug request tracing with credential redaction - #668

Merged
gulshngill merged 24 commits into
mainfrom
feat/debug-request-tracing
Sep 22, 2026
Merged

gulshngill merged 24 commits into
mainfrom
feat/debug-request-tracing

Conversation

@gulshngill

@gulshngill gulshngill commented Sep 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds an opt-in request trace so auth, response timing, retry, and response-shape failures can be diagnosed without mixing diagnostics into command output.

--debug (a global flag) or NANSEN_DEBUG=1 emits HTTP trace events to stderr. Events include method, sanitized URL, response status, time-to-response-headers (TTFB) in milliseconds, retry decisions (attempt, reason, delay), and the server request id when present. stdout remains JSON/CSV-only and safe to pipe:

nansen research token screener --chain solana --debug
nansen research token screener --chain solana 2>trace.log | jq .
[nansen:debug] http.request method=POST url=https://api.nansen.ai/api/v1/token-screener attempt=1/4
[nansen:debug] http.response method=POST url=https://api.nansen.ai/api/v1/token-screener status=429 duration_ms=182 request_id=6f1c0f2a-0000-4000-8000-0000000000aa attempt=1
[nansen:debug] http.retry method=POST url=https://api.nansen.ai/api/v1/token-screener status=429 attempt=1 reason=retry-after delay_ms=1100 retry_after_ms=1000
[nansen:debug] http.request method=POST url=https://api.nansen.ai/api/v1/token-screener attempt=2/4
[nansen:debug] http.response method=POST url=https://api.nansen.ai/api/v1/token-screener status=200 duration_ms=143 request_id=6f1c0f2a-0000-4000-8000-0000000000ab attempt=2

(All values above are examples.)

For http.response, duration_ms measures from the start of an attempt until response headers arrive (TTFB); it deliberately excludes downloading and parsing the response body. For http.error, it measures elapsed time until the transport failed before headers arrived.

Migration: DEBUG=1 no longer enables Nansen diagnostics. Use NANSEN_DEBUG=1 (or --debug) instead.

Redaction and output guarantees

  • Header values are never traced, including apikey, Authorization, and Payment-Signature.
  • Request and response bodies are never traced.
  • Credential-named query parameters and credential-shaped values are replaced with [redacted].
  • Embedded URLs and credential-shaped substrings in diagnostic messages are sanitized.
  • URL userinfo is removed, fragments are dropped, and secret-looking path segments are redacted.
  • Binary/typed-array values are summarized, and every serialized field is length-bounded.
  • Trace events are written only to stderr.

Coverage

The shared API client is traced end to end: every main request attempt, response, network failure, retry decision, x402 paid retry, and local cache hit. The AI-agent SSE stream, trading/bridge/limit-order transports, direct RPC calls, telemetry, and the background update check remain outside this PR because each uses a separate transport and error flow.

Changes

  • src/debug.js — opt-in switch (--debug / NANSEN_DEBUG), stderr writer, recursive redaction and bounding, URL/header sanitizers, and http.* helpers.
  • src/api.js — request/response/error/retry/cache tracing for the shared API loop and x402 paid retry.
  • src/cli.js — global --debug parsing and early trace activation, merged with current main's argument validation and --yes support.
  • src/privy.js, src/api.js — wallet-lookup diagnostics now use the shared trace helper.
  • src/schema.json, README.md — global option and usage documentation.
  • .changeset/debug-request-tracing.md — minor release note.
  • src/__tests__/debug.test.js, src/__tests__/api-debug-trace.test.js — redaction, bounding, switch, transport, retry, x402, cache, and CLI flag coverage.

The branch includes a merge of current main, including #669's CLI/help coverage and #667's trade/bridge --dry-run and --yes behavior. The only merge conflict was the valueless-flag registry; the resolution retains both --debug and --yes.

Validation

npm run lint
  clean (exit 0)

npx vitest run src/__tests__/debug.test.js src/__tests__/api-debug-trace.test.js
  Test Files  2 passed (2)
  Tests       29 passed (29)

npx vitest run src/__tests__/cli.internal.test.js src/__tests__/help-ux.test.js src/__tests__/execute-guard.test.js src/__tests__/bridge-execute-guard.test.js
  Test Files  4 passed (4)
  Tests       741 passed (741)

git diff --check
  clean

GitHub CI
  lint passed
  test (Node 20) passed
  test (Node 22) passed
  test (Node 24) passed
  Aikido security check passed
  automated PR reviewer passed

The previous full CI matrix passed on head b1237de; the matrix will rerun for the timing-documentation commit. Local full-suite execution in the review sandbox is not reported as passing: the existing subprocess smoke test attempts the real API and times out under restricted networking, while the wallet/keychain guard does not terminate under the isolated temporary HOME. Focused tests covering this PR and the timing semantics pass as shown above.

Checklist

  • Focused and merge-conflict tests pass
  • npm run lint passes
  • src/schema.json documents the new global option
  • README documents stderr behavior and redaction
  • User-facing changeset added
  • Full CI matrix rerun pending on the timing-documentation head

`--debug`, or `NANSEN_DEBUG=1`, prints one line per HTTP request to stderr:
method, URL, response status, latency in ms, retry decisions (attempt,
reason, delay) and the server request id when the response carries one.
stdout still carries only the JSON/CSV payload, so piped output stays
machine-parseable.

The trace never prints a credential. src/debug.js owns the single switch
and a redact() pass that blanks values under any key naming a key, token,
secret, signature, password or auth; strips inline Bearer/Payment
credentials; blanks private-key, mnemonic, JWT and long opaque shapes
wherever they appear; drops URL userinfo; and truncates long values.
Header values are never traced at all and bodies are never traced.
Request ids and public identifiers (addresses, chains) stay readable so
the trace is worth having.

The two x402 wallet-lookup notes that were keyed on a bare DEBUG env var
now route through the same helper, so there is one switch, one output
stream and one redaction pass instead of a parallel mechanism; DEBUG
keeps working as an alias.

Covers the shared API client: the main request loop (including cache
hits, which report that no request was made) and the x402 paid retry.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@nansen-pr-reviewer

nansen-pr-reviewer Bot commented Sep 19, 2026 •

Copy link
Copy Markdown

pr-reviewer Summary for #f7db304

📝 2 findings

Review completed. Please address the findings below.

Findings by Severity

Severity Count
🟡 Medium 1
🔵 Low 1

Review effort: 4/5 (Complex)

Summary

This is a solid, well-considered feature. The debug.js module is carefully designed: it's properly opt-in, writes exclusively to stderr, handles circular references, applies layered credential scrubbers in the right order, and is extensively tested. The stateful /g regex note is correctly documented. The loop refactor in api.js (attempt <= maxRetries → attempt < maxAttempts) preserves correct semantics and actually fixes a subtle display bug where retry:false could have produced attempt=2/1. Integration into privy.js and the CLI flag wiring are minimal and correct. No credentials can leak from any traced path via the code paths reviewed.

Findings

src/debug.js — SECRET_NAME matches auth substring, which is also a substring of common URL path segments

Severity: medium

SECRET_NAME includes auth, which matches any string containing that substring — including the query-parameter name retry_after_ms if it ever appeared as a standalone field value or key name. More importantly, it matches auth as a bare path segment (e.g. /api/v1/auth/login). The path-segment redaction condition guards this correctly with a 32-character minimum (decoded.length >= 32 && isSecretName(decoded)), so short segments like auth survive. However, a hypothetical endpoint like /api/v1/authenticate-and-refresh-now (38 chars, contains auth) would be silently redacted from the trace — making the URL look like hxxps://api[.]nansen[.]ai/api/v1/[redacted] and defeating the diagnostic purpose.

This is an acknowledged fail-closed design choice (the code comment says so), but the threshold of 32 chars is not obviously safe for every current or future Nansen API path. The token-screener, token-holders, and token-flow-intelligence endpoints contain token and are 14–25 chars, safely below 32. But if any endpoint name ever hits 32 chars and contains a secret-adjacent word, it will be silently hidden from traces with no error.

Suggested fix: Add a regression test asserting that the specific known Nansen API paths (/api/v1/token-screener, /api/v1/auth/login, etc.) survive redactUrl() unchanged. This documents the intended pass-through as a contract, and will catch any future threshold adjustment that breaks it. The test in debug.test.js already checks '/auth/login' via redactUrl, so extend it to verify the full set of first-class endpoints.


src/debug.js — looksLikeSecretValue uses module-level LONG_OPAQUE which overlaps with INLINE_SCHEME

Severity: low

looksLikeSecretValue calls INLINE_SCHEME.test(text) as its last check. INLINE_SCHEME is a non-/g regex (correctly), but it is also called from INLINE_SCHEME_ALL.source in redactString. There is a secondary concern: the LONG_OPAQUE pattern (/^[A-Za-z0-9_\-+/=.]{64,}$/) will match any base64-encoded value of 64+ chars, which includes encoded payment signatures and API keys — good. It also matches standard Solana transaction signatures (87 base58 chars) passed as values in error messages. This is explicitly documented and intentional ("fail closed"). However, looksLikeSecretValue is also called inside redactUrl for path segments, meaning a 64-char Solana transaction signature appearing in a path segment (e.g. a webhook callback URL) would also be redacted from the trace. This is fine given the security posture.

The real low-severity note: looksLikeSecretValue does not call MNEMONIC.test(text) in constant time — the MNEMONIC regex is exponential-backtrack-safe (alternations of fixed-width words with a space separator), but a crafted 500-word "mnemonic" passed in a query param could cause measurable latency before truncation fires. In practice this is theoretical — error messages from the Nansen API won't contain mnemonic-shaped strings — but if a future trace event exposes arbitrary user input, consider truncating before calling looksLikeSecretValue.

Suggested fix: Pre-truncate the string inside looksLikeSecretValue (or before calling it) to MAX_VALUE_LENGTH so the regex tests never run on excessively long strings, regardless of what future callers pass.


Token usage: 2,162 input, 14,597 output, 1,669,583 cache read, 53,966 cache write | Usage Guide

New pushes are reviewed automatically with a 10-minute cooldown between reviews. To request a review at any time, comment @nansen-pr-reviewer re-review.


Note: Some URLs in this summary were sanitized for security. Defanged links (hxxp/[.]) are not clickable by design.

`--debug` was documented as a global flag but was missing from
VALUELESS_FLAGS, so `nansen --debug research token screener` consumed
`research` as the flag's value: tracing stayed off and a different
command was dispatched. Placing a global flag first is the natural way
to type it, so this made the documented usage silently wrong.

Also narrow the inline-credential pattern to `Bearer`/`Basic` and
require the credential to be at least 8 characters. `Payment` and
`Signature` are ordinary English words in traced text, and a real
payment signature is already covered by header-name matching and the
value-shape rules, so including them only garbled prose.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@gulshngill

Copy link
Copy Markdown
Contributor Author

Both findings addressed in 8fba5e7.

--debug missing from VALUELESS_FLAGS (high) — confirmed and fixed. Reproduced first: nansen --debug research token screener --chain solana parsed research as the flag's value, so tracing never turned on and the run fell through to the deprecated nansen token screener alias instead of the command that was typed. Added 'debug' to the set, plus a regression test asserting parseArgs(['--debug','auth','status']) yields flags.debug === true with _ intact, and that runCLI enables tracing when the flag comes first. The test fails on the unfixed parser.

INLINE_SCHEME matching prose (medium) — took the stricter half of your suggestion: dropped Payment and Signature from the keyword list and required the credential to be 8+ characters, so the pattern is now /\b(Bearer|Basic)\s+[\w.~+/=-]{8,}/i. A real payment signature is still covered by header-name matching and by the value-shape rules, which is why the keywords were redundant rather than protective. Added tests pinning that payment outcome unknown, signed payment was transmitted and Signature verification failed for header pass through unchanged, that Bearer token is left alone, and that a genuine Bearer eyJ... is still redacted.

npm test — 78 files, 3224 passed, 2 skipped. npm run lint clean.

@gulshngill

Copy link
Copy Markdown
Contributor Author

Merge position: 8 of 9. Merge after #661, #662 and #666.

Shares src/api.js and README.md with #661 and #662, and src/cli.js plus src/schema.json with #662, #666 and #667.

One live conflict to expect: #651 edits the same retry branch this PR's retry trace hooks into. The trace reports whatever that PR decides, so it is a context conflict rather than a behavioural one.

@gulshngill

Copy link
Copy Markdown
Contributor Author

@codex review

@gulshngill

Copy link
Copy Markdown
Contributor Author

@nansen-pr-reviewer re-review

@gulshngill

Copy link
Copy Markdown
Contributor Author

@nansen-pr-reviewer re-review

@gulshngill

Copy link
Copy Markdown
Contributor Author

@nansen-pr-reviewer re-review

@gulshngill

Copy link
Copy Markdown
Contributor Author

@nansen-pr-reviewer re-review

@gulshngill

Copy link
Copy Markdown
Contributor Author

@nansen-pr-reviewer re-review

@gulshngill

Copy link
Copy Markdown
Contributor Author

@nansen-pr-reviewer re-review

@gulshngill

Copy link
Copy Markdown
Contributor Author

@nansen-pr-reviewer re-review

@gulshngill

Copy link
Copy Markdown
Contributor Author

@nansen-pr-reviewer re-review

@gulshngill

Copy link
Copy Markdown
Contributor Author

@nansen-pr-reviewer re-review

@gulshngill

Copy link
Copy Markdown
Contributor Author

@nansen-pr-reviewer re-review

@gulshngill

Copy link
Copy Markdown
Contributor Author

@nansen-pr-reviewer re-review

@gulshngill
gulshngill merged commit 77b7020 into main Sep 22, 2026
9 checks passed
@gulshngill
gulshngill deleted the feat/debug-request-tracing branch September 22, 2026 05:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant