From e33cf5ff96567f025017f56fecd16325d38d95b6 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 21 Sep 2026 05:18:15 +0000 Subject: [PATCH 1/4] feat(cli): log to ~/.rawback/logs and add the logs command group MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The CLI had no verbosity flag and no diagnostics: the only console output was the two sinks inside CommandOutput, so a user reporting a failure had nothing to send. Build on the SDK's logging subsystem. - Global `--log-level` and `-v`/`-vv`, declared beside `--env` and stashed in a new `src/log-level.ts` singleton. Its `@rawback/sdk` import is type-only and erases, so `cli.ts` stays off the SDK barrel and `rawback --help` keeps its startup cost — the trap `src/trace.ts` already documents. - `src/logging.ts` builds the one logger this process owns, lazily and memoized, behind the same boundary that keeps `--help` and `--version` from writing to a user's home. Neither path creates `~/.rawback/logs/`, which is asserted rather than assumed. - `runCli` records the command and its exit code, then flushes in a `finally` — a failed command is exactly the one whose records need to reach the file. - `rawback logs path|show|purge`, modelled on the `config` group. `show` reads only the tail, so it stays fast on a ten-megabyte file, and shows a torn line rather than dropping it. `purge` covers Desktop's log too, since both clients share the directory; it matches only the names this tool writes and never removes the directory itself, because `logging.directory` is user-controlled. Logs go to the file and never to stdout, so raising verbosity cannot disturb a script parsing `--json`. Docs updated across all four surfaces: yargs help, README, the global options table and a `rawback logs` section in docs/commands.md, and a Logging section in docs/configuration.md. The directory path is rendered as a text block rather than a field: the shared fields renderer drops its label padding when a value is long, and a log directory is always a long absolute path. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HwoxuU7mv4yFQnecnKoaPY --- README.md | 24 ++++ docs/commands.md | 79 +++++++++- docs/configuration.md | 106 ++++++++++++++ src/cli.ts | 109 ++++++++++++++ src/command.ts | 14 ++ src/features/cli/runtime.ts | 44 +++++- src/log-level.ts | 41 ++++++ src/logging.ts | 73 ++++++++++ src/logs.ts | 247 ++++++++++++++++++++++++++++++++ test/commands-cli.test.ts | 99 ++++++++++++- test/log-level.test.ts | 44 ++++++ test/logs.test.ts | 277 ++++++++++++++++++++++++++++++++++++ 12 files changed, 1149 insertions(+), 8 deletions(-) create mode 100644 src/log-level.ts create mode 100644 src/logging.ts create mode 100644 src/logs.ts create mode 100644 test/log-level.test.ts create mode 100644 test/logs.test.ts diff --git a/README.md b/README.md index 23cfc14..149d706 100644 --- a/README.md +++ b/README.md @@ -394,6 +394,23 @@ rawback album article --help rawback shares list --help ``` +### When something goes wrong + +The CLI keeps a JSON log at `~/.rawback/logs/cli.log`, shared with Rawback +Desktop. At the default verbosity it records every failure — including the trace +and `cf-ray` IDs identifying the request on the server — and nothing else. + +```bash +rawback logs show --level warn # what recently failed, and why +rawback -v photos upload ~/raw # re-run the failing command with more detail +rawback logs path # where the files are, and how big +rawback logs purge # delete them, this app and Desktop +``` + +Logs never go to standard output, so `-v` cannot disturb a script parsing +`--json`. Tokens, passwords and authorization headers are redacted before +anything is written. + ## Files and security Rawback stores local state under `~/.rawback/`: @@ -404,6 +421,7 @@ Rawback stores local state under `~/.rawback/`: | `config.yml` | Environments, optional hosts, metadata workers, and SFTP | | `upload-state.json` | Shared upload queue, history, and trusted host keys | | `cameras.json` | Saved Canon cameras, shared with Rawback Desktop | +| `logs/` | Rolling JSON diagnostics, shared with Rawback Desktop | `cameras.json` is shared with the Rawback desktop app, so both can reach the same camera without pairing twice. It holds a camera password only when you pass @@ -421,6 +439,12 @@ replace it by hand or copy it from another machine. Run `rawback config init into issues and logs. `rawback config view` masks every `sftp.password` in both terminal and JSON output, including the ones inside `environments`. +Log files are written at mode `0600` too, and secret field and header names are +replaced with `[REDACTED]` before a record reaches disk. At `--log-level trace` +the log does record request and response bodies — file paths and album titles +among them — so turn that on to diagnose something rather than leaving it on. +See [Logging](docs/configuration.md#logging). + ## Development Install the pinned dependencies and run the CLI from source: diff --git a/docs/commands.md b/docs/commands.md index 65a0160..6671a37 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -7,14 +7,24 @@ English only. ## Global options -| Option | Description | -| ----------------- | --------------------------------------------- | -| `-h`, `--help` | Show help for the current command | -| `-V`, `--version` | Print the CLI version | -| `--env ` | Run against one environment from `config.yml` | +| Option | Description | +| --------------------- | ---------------------------------------------- | +| `-h`, `--help` | Show help for the current command | +| `-V`, `--version` | Print the CLI version | +| `--env ` | Run against one environment from `config.yml` | +| `--log-level ` | Verbosity of `~/.rawback/logs/cli.log` | +| `-v`, `--verbose` | Shorthand: `-v` for `debug`, `-vv` for `trace` | Running `rawback` without arguments shows top-level help. +`--log-level` takes `trace`, `debug`, `info`, `warn`, `error`, `fatal` or +`silent` and only affects the log file — **never** standard output, so `--json` +stays machine-readable at any verbosity. The default is `info`, at which a run +records every failure and nothing else. `RAWBACK_LOG_LEVEL` sets the same thing +for a shell session, and the flag beats it. See +[Configuration](configuration.md#logging) for the file layout and +[`rawback logs`](#rawback-logs) for reading and clearing it. + `--env` accepts any name under `environments:` in `~/.rawback/config.yml`, plus the reserved name `default` for the file's top-level settings. Without it, commands use the saved `current:` environment. An unknown name fails and lists @@ -1024,6 +1034,61 @@ rawback web The URL uses `webHost` from `~/.rawback/config.yml`, or `https://rawback.app` by default. +## `rawback logs` + +Read and clear the diagnostic log the CLI and the Desktop app write to +`~/.rawback/logs/`. One JSON object per line, so `jq` works directly on the +file. + +```bash +rawback logs path +rawback logs show --lines 100 --level warn +rawback logs purge --yes +``` + +### `rawback logs path` + +Prints the log directory and every file in it with its size and modification +time. + +| Option | Description | +| -------- | ---------------------------- | +| `--json` | Output machine-readable JSON | + +### `rawback logs show` + +Prints the most recent records, oldest first. Only the tail of the file is read, +so this stays fast on a ten-megabyte log. + +| Option | Description | +| ----------------- | ------------------------------------------------ | +| `--lines ` | How many records to show, 1–10000 (default `50`) | +| `--level ` | Show only records at this level or above | +| `--app ` | `cli` (default) or `desktop` | +| `--json` | Output machine-readable JSON | + +A line that cannot be parsed is shown as-is rather than dropped — a torn record +is evidence too. `--json` emits `{ "file", "lines" }`, where an unparseable line +appears as `{ "raw": "…" }`. + +### `rawback logs purge` + +Deletes the log files. Only files this tool writes are removed and the directory +itself is left in place, so a custom `logging.directory` holding other files is +safe. + +| Option | Description | +| -------------- | ------------------------------------ | +| `--app ` | `all` (default), `cli`, or `desktop` | +| `--yes` | Skip the confirmation prompt | +| `--json` | Output machine-readable JSON | + +Without `--yes` it asks first, and fails with a message naming `--yes` when the +terminal is not interactive. The default `--app all` includes the Desktop app's +log, because both share one directory. A file that cannot be deleted is reported +rather than thrown — on Windows the Desktop app holds `desktop.log` open — and +the command exits `1` when any file was left behind. + ## Scripting and exit behavior Use `--json` when available instead of parsing human-readable tables. JSON is @@ -1036,6 +1101,10 @@ indicators; redirected output is deterministic and contains no cursor-control sequences. JSON, `--content-only`, and version output are never decorated with icons or prose. +Diagnostic logs go to `~/.rawback/logs/cli.log`, never to standard output or +standard error, so raising the verbosity with `-v` cannot disturb a script +parsing `--json`. + The CLI exits with status `0` on success, `1` for validation, API, filesystem, or upload failures, and `130` when an interactive prompt is cancelled. Scripts should check the exit status before consuming output. diff --git a/docs/configuration.md b/docs/configuration.md index aac2d7a..2d57b86 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -62,6 +62,11 @@ webHost: https://rawback.app metadata: concurrency: 8 +# Optional. Diagnostic logging, shared with the Desktop app. See "Logging". +logging: + level: info + maxFiles: 3 + # Required only by `rawback photos upload` sftp: endpoint: sftp://ftp.rawback.app:23168 @@ -427,8 +432,109 @@ Verify the fingerprint through a trusted Rawback channel before pinning it. Do not delete the upload-state file merely to bypass a host-key mismatch; investigate the server or network change first. +## Logging + +The CLI and the Desktop app both write diagnostics to `~/.rawback/logs/`, beside +`config.yml` and `credentials.json`: + +| File | Written by | +| --------------------------- | ----------------------------- | +| `cli.log` | `rawback` | +| `desktop.log` | The Desktop app | +| `cli.1.log`, `cli.2.log`, … | Rolled archives, newest first | + +Each line is one JSON object, so the file is ordinary JSONL: + +```bash +tail -f ~/.rawback/logs/cli.log | jq -c '{time, level, event, ids}' +jq 'select(.levelValue >= 40)' ~/.rawback/logs/cli.log +``` + +Every record carries `time`, `level`, `levelValue`, `msg`, the writing app and +its version, the process ID, and the environment in use. API records add the +method, path, status and duration, plus the `x-trace-id`, `cf-ray` and +`x-request-id` values identifying that request server-side — those are the IDs +worth quoting in a support report. + +### Levels + +`trace` < `debug` < `info` < `warn` < `error` < `fatal`, plus `silent` to write +nothing. The default is `info`, at which a run records **every failure and +nothing else** — browsing the library does not fill the file with one line per +request. + +- `debug` adds the full request stream and request/response headers. +- `trace` adds request and response bodies, truncated. + +Secret field and header names — `authorization`, `cookie`, `password`, `token`, +`refreshToken` and friends — are replaced with `[REDACTED]` at every level, at +any nesting depth. Credential and authentication GraphQL operations never log +their variables or body even at `trace`. Query strings are stripped from logged +URLs, because they carry search terms. + +`trace` still records file paths, album titles and other content from request +bodies. Turn it on to diagnose something, not as a standing setting. + +### Configuration + +```yaml +logging: + level: info # trace|debug|info|warn|error|fatal|silent + file: true # set false to stop writing to disk entirely + directory: ~/.rawback/logs + maxFileSize: 10485760 # roll the active file at this many bytes + maxFiles: 3 # rolled archives kept, *besides* the active file + redact: [] # extra field names to blank out, added to the built-in list + stderr: false # also mirror records to standard error +``` + +`maxFiles` counts archives, not the total: the default keeps `cli.log` plus +`cli.1.log` through `cli.3.log`, about 40 MB per app at the default size. +`redact` only ever adds to the built-in list — a setting that could switch +redaction off is a setting that leaks tokens. + +Like every other section, `logging` can appear at the top level and inside a +named environment, where it merges key by key: a shared `level` applies +everywhere while one environment overrides just the `directory`. + +These environment variables override the file, and the `--log-level` flag +overrides them: + +| Variable | Effect | +| ----------------------- | --------------------------------------- | +| `RAWBACK_LOG_LEVEL` | One of the level names above | +| `RAWBACK_DEBUG=1` | Shorthand for `RAWBACK_LOG_LEVEL=debug` | +| `RAWBACK_LOG_FILE=0` | Stop writing to disk | +| `RAWBACK_LOG_DIR` | Write somewhere other than the default | +| `RAWBACK_LOG_MAX_SIZE` | Roll at this many bytes | +| `RAWBACK_LOG_MAX_FILES` | Keep this many rolled archives | +| `RAWBACK_LOG_STDERR=1` | Also mirror records to standard error | + +An unrecognised value is ignored rather than rejected, so a typo in a shell +profile cannot stop a command from running. + +### Reading and clearing + +```bash +rawback logs path # where the files are, and how big +rawback logs show --level warn # the recent records that mattered +rawback logs purge --yes # delete them, this app and Desktop +``` + +`purge` removes only the files listed above and leaves the directory itself +alone, so pointing `logging.directory` at a folder holding other things is safe. +See [`rawback logs`](commands.md#rawback-logs) for the full options. + +On Linux and macOS log files are created at mode `0600` inside a `0700` +directory, the same as `credentials.json`. Windows has no equivalent, so the +files inherit the directory's permissions there. + ## Troubleshooting +Whatever the symptom, `rawback logs show --level warn` is the fastest way to see +what actually failed, including the trace and `cf-ray` IDs to quote when +reporting it. Re-run the failing command with `-v` first if the log is empty. + ### Device authorization is temporarily unavailable The CLI retries temporary network and server failures while creating the device diff --git a/src/cli.ts b/src/cli.ts index 13213d1..467bbda 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -2,6 +2,7 @@ import yargs from 'yargs' import type { Argv } from 'yargs' import { setSelectedEnvironment } from './environment.ts' +import { LOG_LEVEL_CHOICES, setSelectedLogLevel } from './log-level.ts' import { expandHomePath } from './paths.ts' import { traceIdOf } from './trace.ts' import { CommandOutput } from './ui/output.tsx' @@ -178,8 +179,24 @@ export function createProgram(version: string, output = new CommandOutput()): Ar global: true, type: 'string', }) + .option('log-level', { + choices: LOG_LEVEL_CHOICES, + describe: 'verbosity of ~/.rawback/logs/cli.log; never affects stdout', + global: true, + type: 'string', + }) + .option('verbose', { + alias: 'v', + count: true, + describe: 'log more: -v for debug, -vv for trace', + global: true, + }) .middleware((args) => { setSelectedEnvironment(typeof args.env === 'string' ? args.env : undefined) + setSelectedLogLevel({ + ...(typeof args.logLevel === 'string' ? { level: args.logLevel } : {}), + ...(typeof args.verbose === 'number' ? { verbose: args.verbose } : {}), + }) }) .command( 'auth [subcommand]', @@ -2836,6 +2853,98 @@ export function createProgram(version: string, output = new CommandOutput()): Ar .strict(), () => {}, ) + .command( + 'logs', + 'inspect and clear the local diagnostic logs', + (command) => + command + .command( + 'path', + 'show where the logs are written and how large they are', + (path) => + path.option('json', { + default: false, + describe: 'output machine-readable JSON', + type: 'boolean', + }), + async (args) => { + if (process.exitCode !== undefined && process.exitCode !== 0) return + const { runLogsPath } = await import('./logs.ts') + await runCommand(() => runLogsPath({ json: args.json })) + }, + ) + .command( + 'show', + 'print the most recent log records', + (show) => + show + .option('lines', { + default: 50, + describe: 'how many records to show (1-10000)', + type: 'number', + }) + .option('level', { + choices: LOG_LEVEL_CHOICES, + describe: 'show only records at this level or above', + type: 'string', + }) + .option('app', { + choices: ['cli', 'desktop'] as const, + default: 'cli', + describe: 'which client wrote the records', + type: 'string', + }) + .option('json', { + default: false, + describe: 'output machine-readable JSON', + type: 'boolean', + }), + async (args) => { + if (process.exitCode !== undefined && process.exitCode !== 0) return + const { runLogsShow } = await import('./logs.ts') + await runCommand(() => + runLogsShow({ + app: args.app, + json: args.json, + lines: args.lines, + ...(args.level !== undefined ? { level: args.level } : {}), + }), + ) + }, + ) + .command( + 'purge', + 'delete the local log files', + (purge) => + purge + .option('app', { + choices: ['cli', 'desktop', 'all'] as const, + default: 'all', + describe: "which client's logs to delete", + type: 'string', + }) + .option('yes', { + default: false, + describe: 'skip the confirmation prompt', + type: 'boolean', + }) + .option('json', { + default: false, + describe: 'output machine-readable JSON', + type: 'boolean', + }), + async (args) => { + if (process.exitCode !== undefined && process.exitCode !== 0) return + const { runLogsPurge } = await import('./logs.ts') + await runCommand(() => + runLogsPurge({ app: args.app, json: args.json, yes: args.yes }), + ) + }, + ) + .demandCommand(1, 'Choose a logs command: path, show, or purge') + .strict(), + () => {}, + ) .command( 'web', 'open your Rawback profile in a web browser', diff --git a/src/command.ts b/src/command.ts index 7a726c3..b49af3f 100644 --- a/src/command.ts +++ b/src/command.ts @@ -1,5 +1,8 @@ +import type { Logger, LogLevel } from '@rawback/sdk' + import { type RawbackClient, createRawbackClient } from './client.ts' import { environmentName } from './config.ts' +import { commandLogger } from './logging.ts' import { CommandOutput, type CommandOutputOptions } from './ui/output.tsx' export interface ReadCommandDependencies extends CommandOutputOptions { @@ -8,6 +11,10 @@ export interface ReadCommandDependencies extends CommandOutputOptions { /** Environment from `~/.rawback/config.yml`; defaults to the `--env` flag. */ env?: string fetch?: typeof globalThis.fetch + /** Overrides the shared logger; tests inject a capturing one. */ + logger?: Logger + /** Overrides the `--log-level` flag. */ + logLevel?: LogLevel output?: CommandOutput } @@ -20,7 +27,14 @@ export async function createCommandClient( authenticated = true, ): Promise { const env = environmentName(dependencies) + const logger = await commandLogger({ + ...(dependencies.configPath !== undefined ? { configPath: dependencies.configPath } : {}), + ...(env !== undefined ? { env } : {}), + ...(dependencies.logger !== undefined ? { logger: dependencies.logger } : {}), + ...(dependencies.logLevel !== undefined ? { logLevel: dependencies.logLevel } : {}), + }) const client = await createRawbackClient({ + logger, ...(dependencies.configPath !== undefined ? { configPath: dependencies.configPath } : {}), ...(env !== undefined ? { env } : {}), ...(dependencies.credentialsPath !== undefined diff --git a/src/features/cli/runtime.ts b/src/features/cli/runtime.ts index 776a127..27a56d7 100644 --- a/src/features/cli/runtime.ts +++ b/src/features/cli/runtime.ts @@ -24,6 +24,21 @@ function isConfigInit(args: string[]): boolean { return args[0] === 'config' && args[1] === 'init' } +/** + * The command being run, without its arguments. + * + * Only the leading non-flag words: an option value can be a search term or a + * path from the user's disk, and neither belongs in a log record. + */ +function commandName(args: string[]): string { + const words: string[] = [] + for (const argument of args) { + if (argument.startsWith('-')) break + words.push(argument) + } + return words.join(' ') || '(none)' +} + export async function runCli( args: string[], version: string, @@ -45,9 +60,34 @@ export async function runCli( if (!isHelpRequest(parseArgs)) { // Creating the config here rather than in a yargs middleware is what keeps // `--help` and `--version` from writing to the user's home directory: both - // return above, and every help request takes the branch below. + // return above, and every help request takes the branch below. The logger + // is built lazily behind the same boundary, so neither path creates + // `~/.rawback/logs/` either. if (!isConfigInit(parseArgs)) await bootstrapConfig(output) - await program.parseAsync(parseArgs) + const { commandLogger, flushCommandLogger } = await import('../../logging.ts') + const command = commandName(parseArgs) + const logger = (await commandLogger()).child({ component: 'cli', command }) + const startedAt = performance.now() + logger.debug(`running ${command}`, { event: 'cli.command.start' }) + try { + await program.parseAsync(parseArgs) + } finally { + // `runCommand` turns a failure into an exit code rather than a throw, so + // the outcome is read from there rather than from a catch. + const exitCode = process.exitCode ?? 0 + const failed = exitCode !== 0 + logger[failed ? 'warn' : 'debug']( + failed ? `${command} exited ${String(exitCode)}` : `${command} finished`, + { + event: 'cli.command.done', + exitCode, + durationMs: Math.round(performance.now() - startedAt), + }, + ) + // A failed command is exactly the one whose records need to reach the + // file; the sink's exit drain is the backstop, not the mechanism. + await flushCommandLogger() + } return } diff --git a/src/log-level.ts b/src/log-level.ts new file mode 100644 index 0000000..e56a45d --- /dev/null +++ b/src/log-level.ts @@ -0,0 +1,41 @@ +import type { LogLevel } from '@rawback/sdk' + +/** + * The verbosity selected by the global `--log-level` and `-v` flags. + * + * Mirrors `src/environment.ts`: `runCli` sets it once per parse, including back + * to `undefined` when neither flag is present, so nothing leaks between parses + * in the same process. Command modules take `dependencies.logLevel` like every + * other injected value and that always wins. + * + * The `@rawback/sdk` import is **type-only** and erases at compile time. That + * matters: `cli.ts` imports this module and runs on every invocation, so a + * runtime import would drag the SDK barrel — ssh2 included — onto the + * `rawback --help` path, the same trap `src/trace.ts` documents. + */ +export const LOG_LEVEL_CHOICES = [ + 'trace', + 'debug', + 'info', + 'warn', + 'error', + 'fatal', + 'silent', +] as const satisfies readonly LogLevel[] + +let selected: LogLevel | undefined + +export function setSelectedLogLevel(input: { level?: string; verbose?: number }): void { + const named = LOG_LEVEL_CHOICES.find((choice) => choice === input.level?.trim()) + if (named) { + selected = named + return + } + // `-v` is debug and `-vv` is trace; an explicit `--log-level` beats both. + const verbosity = input.verbose ?? 0 + selected = verbosity >= 2 ? 'trace' : verbosity === 1 ? 'debug' : undefined +} + +export function selectedLogLevel(): LogLevel | undefined { + return selected +} diff --git a/src/logging.ts b/src/logging.ts new file mode 100644 index 0000000..fa18f76 --- /dev/null +++ b/src/logging.ts @@ -0,0 +1,73 @@ +import { + createAppLogger, + type Logger, + type LogLevel, + NOOP_LOGGER, + readEnvironment, + resolveLoggingOptions, + type RootLogger, +} from '@rawback/sdk' + +import packageJson from '../package.json' with { type: 'json' } +import { selectedLogLevel } from './log-level.ts' + +export interface LoggerDependencies { + configPath?: string + env?: string + logger?: Logger + logLevel?: LogLevel +} + +let root: RootLogger | undefined +let building: Promise | undefined + +/** + * The one logger this process writes through. + * + * Built lazily and memoized: a command that never talks to the network should + * not create `~/.rawback/logs/` on the way past, and one process wants one file + * handle and one exit hook rather than one per command module. + */ +export async function commandLogger(dependencies: LoggerDependencies = {}): Promise { + if (dependencies.logger) return dependencies.logger + if (root) return root + building ??= buildLogger(dependencies) + root = await building + return root +} + +async function buildLogger(dependencies: LoggerDependencies): Promise { + const level = dependencies.logLevel ?? selectedLogLevel() + const environment = await readEnvironment(dependencies.configPath, dependencies.env).catch( + () => undefined, + ) + return createAppLogger({ + app: 'cli', + version: packageJson.version, + options: resolveLoggingOptions({ + ...(level ? { overrides: { level } } : {}), + ...(environment?.logging ? { config: environment.logging } : {}), + }), + ...(environment?.name ? { environmentName: environment.name } : {}), + }) +} + +/** + * Writes out anything still buffered. + * + * Called from `runCli`'s `finally` so a command's records reach the file even + * when it failed; the sink's process-exit drain is the backstop, not the + * mechanism. + */ +export async function flushCommandLogger(): Promise { + if (!root) return + await root.flush() +} + +/** Forgets the memoized logger. Tests need this; nothing else should. */ +export function resetCommandLogger(): void { + root = undefined + building = undefined +} + +export { NOOP_LOGGER } diff --git a/src/logs.ts b/src/logs.ts new file mode 100644 index 0000000..eebb905 --- /dev/null +++ b/src/logs.ts @@ -0,0 +1,247 @@ +import { open } from 'node:fs/promises' + +import { + isLogApp, + LOG_APPS, + type LogApp, + type LogDirectoryListing, + type LogLevel, + listLogFiles, + logFileName, + LOG_LEVELS, + type PurgeLogsResult, + purgeLogs, + readEnvironment, + resolveLoggingOptions, +} from '@rawback/sdk' + +import { commandOutput, type ReadCommandDependencies } from './command.ts' +import { environmentName } from './config.ts' +import { logFilesDocument, logLinesDocument, purgeResultDocument } from './features/logs/view.ts' + +export interface LogsPrompts { + confirm(message: string): Promise +} + +export interface LogsCommandDependencies extends ReadCommandDependencies { + /** Overrides the directory resolved from `config.yml`; tests point it at a tmpdir. */ + logDirectory?: string + prompts?: LogsPrompts +} + +export interface LogsPathOptions { + json?: boolean +} + +export interface LogsShowOptions { + lines?: number + level?: string + app?: string + json?: boolean +} + +export interface LogsPurgeOptions { + app?: string + yes?: boolean + json?: boolean +} + +/** One line of the log file, parsed when it is valid JSON and kept raw when not. */ +export interface LogLine { + time?: string + level?: string + component?: string + event?: string + msg?: string + traceId?: string + /** Set instead of the rest when the line could not be parsed. */ + raw?: string +} + +/** + * Where the logs live for this invocation. + * + * Resolved the same way the logger itself resolves it — flag, then environment, + * then `config.yml`, then the default — so `rawback logs path` can never point + * at a different directory from the one being written to. + */ +export async function resolveLogDirectory( + dependencies: LogsCommandDependencies = {}, +): Promise { + if (dependencies.logDirectory) return dependencies.logDirectory + const environment = await readEnvironment( + dependencies.configPath, + environmentName(dependencies), + ).catch(() => undefined) + return resolveLoggingOptions(environment?.logging ? { config: environment.logging } : {}) + .directory +} + +function requireApp(value: string | undefined): LogApp | undefined { + if (value === undefined || value === 'all') return undefined + if (!isLogApp(value)) { + throw new Error(`--app must be one of ${LOG_APPS.join(', ')}, or all`) + } + return value +} + +export async function runLogsPath( + options: LogsPathOptions = {}, + dependencies: LogsCommandDependencies = {}, +): Promise { + const listing = await listLogFiles(await resolveLogDirectory(dependencies)) + const output = commandOutput(dependencies) + + if (options.json) { + output.json(serializeListing(listing)) + return + } + output.document(logFilesDocument(listing)) +} + +function serializeListing(listing: LogDirectoryListing) { + return { + directory: listing.directory, + totalBytes: listing.totalBytes, + files: listing.files.map((file) => ({ + name: file.name, + path: file.path, + bytes: file.bytes, + modifiedAt: file.modifiedAt, + })), + } +} + +/** How much of the tail to read. Enough for a large `--lines`, never the whole file. */ +const TAIL_BYTES_PER_LINE = 1_024 +const MAX_TAIL_BYTES = 4 * 1024 * 1024 + +/** + * Reads the last `count` lines of a file. + * + * Positioned rather than whole-file: the active log can be ten megabytes, and + * `rawback logs show` must not pull all of it into memory to print fifty lines. + */ +export async function readTail(path: string, count: number): Promise { + const handle = await open(path, 'r').catch(() => undefined) + if (!handle) return [] + try { + const { size } = await handle.stat() + const span = Math.min(size, Math.min(count * TAIL_BYTES_PER_LINE, MAX_TAIL_BYTES)) + const buffer = Buffer.alloc(span) + await handle.read(buffer, 0, span, size - span) + const text = buffer.toString('utf8') + // A partial first line is likely when the read started mid-record. + const lines = text.split('\n').filter((line) => line.trim().length > 0) + if (span < size && lines.length > 1) lines.shift() + return lines.slice(-count) + } finally { + await handle.close() + } +} + +function parseLine(line: string): LogLine { + try { + const record = JSON.parse(line) as Record + const ids = record.ids as { traceId?: string } | undefined + return { + ...(typeof record.time === 'string' ? { time: record.time } : {}), + ...(typeof record.level === 'string' ? { level: record.level } : {}), + ...(typeof record.component === 'string' ? { component: record.component } : {}), + ...(typeof record.event === 'string' ? { event: record.event } : {}), + ...(typeof record.msg === 'string' ? { msg: record.msg } : {}), + ...(ids?.traceId ? { traceId: ids.traceId } : {}), + } + } catch { + // A torn line is worth showing rather than dropping — it is evidence too. + return { raw: line } + } +} + +function atLeastLevel(line: LogLine, minimum: LogLevel | undefined): boolean { + if (minimum === undefined) return true + const level = line.level as LogLevel | undefined + if (level === undefined || !(level in LOG_LEVELS)) return true + return LOG_LEVELS[level] >= LOG_LEVELS[minimum] +} + +export async function runLogsShow( + options: LogsShowOptions = {}, + dependencies: LogsCommandDependencies = {}, +): Promise { + const count = options.lines ?? 50 + if (!Number.isSafeInteger(count) || count < 1 || count > 10_000) { + throw new Error('--lines must be an integer between 1 and 10000') + } + const level = options.level as LogLevel | undefined + if (level !== undefined && !(level in LOG_LEVELS)) { + throw new Error(`--level must be one of ${Object.keys(LOG_LEVELS).join(', ')}`) + } + + const directory = await resolveLogDirectory(dependencies) + const app = requireApp(options.app) ?? 'cli' + const path = `${directory}/${logFileName(app)}` + const lines = (await readTail(path, count)) + .map(parseLine) + .filter((line) => atLeastLevel(line, level)) + const output = commandOutput(dependencies) + + if (options.json) { + output.json({ file: path, lines }) + return + } + output.document(logLinesDocument(path, lines)) +} + +export async function runLogsPurge( + options: LogsPurgeOptions = {}, + dependencies: LogsCommandDependencies = {}, +): Promise { + const app = requireApp(options.app) + const directory = await resolveLogDirectory(dependencies) + const output = commandOutput(dependencies) + + if (!options.yes) { + const listing = await listLogFiles(directory, app) + if (listing.files.length === 0) { + if (options.json) output.json(emptyPurge(directory)) + else output.info(`No log files to delete in ${directory}.`) + return + } + const prompts = dependencies.prompts ?? defaultPrompts() + const confirmed = await prompts.confirm( + `Delete ${listing.files.length} log file${listing.files.length === 1 ? '' : 's'} ` + + `from ${directory}?`, + ) + if (!confirmed) { + output.info('Left the log files in place.') + return + } + } + + const result = await purgeLogs({ directory, ...(app ? { app } : {}) }) + if (options.json) { + output.json(result) + } else { + output.document(purgeResultDocument(result)) + } + // A file that could not be deleted is reported, not thrown: on Windows the + // Desktop app holds `desktop.log` open, and the rest should still clear. + if (result.failed.length > 0) process.exitCode = 1 +} + +function emptyPurge(directory: string): PurgeLogsResult { + return { directory, removed: [], failed: [], bytes: 0 } +} + +function defaultPrompts(): LogsPrompts { + return { + async confirm(message) { + if (!process.stdin.isTTY || !process.stdout.isTTY) { + throw new Error('Re-run with --yes to delete log files non-interactively.') + } + const { confirm } = await import('@inquirer/prompts') + return confirm({ default: false, message }) + }, + } +} diff --git a/test/commands-cli.test.ts b/test/commands-cli.test.ts index 422f330..df9d179 100644 --- a/test/commands-cli.test.ts +++ b/test/commands-cli.test.ts @@ -1,5 +1,5 @@ import { afterEach, describe, expect, test } from 'bun:test' -import { mkdtempSync, rmSync } from 'node:fs' +import { existsSync, mkdtempSync, readFileSync, rmSync } from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' @@ -377,3 +377,100 @@ describe('new command hierarchy', () => { } }) }) + +describe('logs commands', () => { + test('documents the logs group and each subcommand', () => { + const group = runCli('logs', '--help') + expect(group.exitCode).toBe(0) + for (const subcommand of ['path', 'show', 'purge']) { + expect(group.stdout).toContain(subcommand) + } + + const show = runCli('logs', 'show', '--help') + for (const flag of ['--lines', '--level', '--app', '--json']) { + expect(show.stdout).toContain(flag) + } + + const purge = runCli('logs', 'purge', '--help') + for (const flag of ['--app', '--yes', '--json']) { + expect(purge.stdout).toContain(flag) + } + }) + + test('requires a logs subcommand and rejects an unknown one', () => { + expect(runCli('logs').exitCode).toBe(1) + expect(runCli('logs').stderr).toContain('Choose a logs command') + expect(runCli('logs', 'tailf').exitCode).toBe(1) + }) + + test('documents the global verbosity options', () => { + const help = runCli('--help') + expect(help.stdout).toContain('--log-level') + expect(help.stdout.replace(/\s+/g, ' ')).toContain('--verbose') + }) + + test('rejects a log level that is not one of the known ones', () => { + const result = runCli('--log-level', 'verbose', 'logs', 'path') + expect(result.exitCode).toBe(1) + }) + + test('writes no log directory for --help or --version', () => { + // The logger is built lazily behind the same boundary that keeps the config + // bootstrap off these paths; a regression here would create files in the + // user's home just for asking for help. + for (const args of [['--help'], ['--version'], ['logs', '--help']]) { + const home = emptyHome() + const result = Bun.spawnSync([process.execPath, 'run', entrypoint, ...args], { + env: { ...process.env, HOME: home }, + stderr: 'pipe', + stdout: 'pipe', + }) + expect(result.exitCode).toBe(0) + expect(existsSync(join(home, '.rawback', 'logs'))).toBe(false) + } + }) + + test('records a failed command, with its trace-able exit code', () => { + const home = emptyHome() + const run = (...args: string[]) => + Bun.spawnSync([process.execPath, 'run', entrypoint, ...args], { + env: { ...process.env, HOME: home }, + stderr: 'pipe', + stdout: 'pipe', + }) + + // No credentials in an empty home, so this fails and should be recorded. + expect(run('photos', 'list').exitCode).toBe(1) + + const contents = readFileSync(join(home, '.rawback', 'logs', 'cli.log'), 'utf8') + const records = contents + .split('\n') + .filter(Boolean) + .map((line) => JSON.parse(line)) + expect(records.at(-1)).toMatchObject({ + level: 'warn', + event: 'cli.command.done', + exitCode: 1, + app: 'cli', + command: 'photos list', + }) + + // And the CLI can read back what it just wrote. + const shown = run('logs', 'show', '--json') + expect(shown.exitCode).toBe(0) + expect( + (JSON.parse(shown.stdout.toString()) as { lines: unknown[] }).lines.length, + ).toBeGreaterThan(0) + }) + + test('keeps --json output clean while logging verbosely', () => { + const home = emptyHome() + const result = Bun.spawnSync( + [process.execPath, 'run', entrypoint, '-vv', 'logs', 'path', '--json'], + { env: { ...process.env, HOME: home }, stderr: 'pipe', stdout: 'pipe' }, + ) + expect(result.exitCode).toBe(0) + // stdout is the machine-readable contract; log records never belong in it. + expect(() => JSON.parse(result.stdout.toString())).not.toThrow() + }) +}) diff --git a/test/log-level.test.ts b/test/log-level.test.ts new file mode 100644 index 0000000..2c77c3b --- /dev/null +++ b/test/log-level.test.ts @@ -0,0 +1,44 @@ +import { afterEach, describe, expect, test } from 'bun:test' + +import { LOG_LEVEL_CHOICES, selectedLogLevel, setSelectedLogLevel } from '../src/log-level.ts' + +afterEach(() => setSelectedLogLevel({})) + +describe('setSelectedLogLevel', () => { + test('takes the named level when one is given', () => { + setSelectedLogLevel({ level: ' warn ' }) + expect(selectedLogLevel()).toBe('warn') + }) + + test('maps -v to debug and -vv to trace', () => { + setSelectedLogLevel({ verbose: 1 }) + expect(selectedLogLevel()).toBe('debug') + setSelectedLogLevel({ verbose: 3 }) + expect(selectedLogLevel()).toBe('trace') + }) + + test('lets an explicit level beat the counted flag', () => { + setSelectedLogLevel({ level: 'error', verbose: 2 }) + expect(selectedLogLevel()).toBe('error') + }) + + test('reports nothing when neither flag was passed', () => { + // Reset to undefined between parses, so nothing leaks in a single process. + setSelectedLogLevel({ verbose: 0 }) + expect(selectedLogLevel()).toBeUndefined() + setSelectedLogLevel({ level: 'nonsense' }) + expect(selectedLogLevel()).toBeUndefined() + }) + + test('offers every level yargs advertises', () => { + expect([...LOG_LEVEL_CHOICES]).toEqual([ + 'trace', + 'debug', + 'info', + 'warn', + 'error', + 'fatal', + 'silent', + ]) + }) +}) diff --git a/test/logs.test.ts b/test/logs.test.ts new file mode 100644 index 0000000..42398b2 --- /dev/null +++ b/test/logs.test.ts @@ -0,0 +1,277 @@ +import { afterEach, describe, expect, test } from 'bun:test' +import { mkdtempSync, readdirSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' + +import { runLogsPath, runLogsPurge, runLogsShow } from '../src/logs.ts' + +const directories: string[] = [] + +afterEach(() => { + for (const directory of directories.splice(0)) rmSync(directory, { force: true, recursive: true }) + process.exitCode = undefined +}) + +function logDirectory(): string { + const directory = mkdtempSync(join(tmpdir(), 'rawback-cli-logs-')) + directories.push(directory) + return directory +} + +function record(level: string, message: string, extra: Record = {}): string { + return JSON.stringify({ + time: '2026-09-21T10:00:00.000Z', + level, + levelValue: 30, + msg: message, + ...extra, + }) +} + +function capture() { + const lines: string[] = [] + const errors: string[] = [] + return { + lines, + errors, + stdout: (message: string) => lines.push(message), + stderr: (message: string) => errors.push(message), + json: () => JSON.parse(lines.join('\n')) as T, + } +} + +describe('rawback logs path', () => { + test('reports every log file with its size', async () => { + const directory = logDirectory() + writeFileSync(join(directory, 'cli.log'), 'aaaa') + writeFileSync(join(directory, 'desktop.log'), 'bb') + const output = capture() + + await runLogsPath({ json: true }, { logDirectory: directory, stdout: output.stdout }) + + expect( + output.json<{ directory: string; totalBytes: number; files: Record[] }>(), + ).toEqual({ + directory, + totalBytes: 6, + files: [ + { + name: 'cli.log', + path: join(directory, 'cli.log'), + bytes: 4, + modifiedAt: expect.any(String), + }, + { + name: 'desktop.log', + path: join(directory, 'desktop.log'), + bytes: 2, + modifiedAt: expect.any(String), + }, + ], + }) + }) + + test('renders a document when JSON was not asked for', async () => { + const directory = logDirectory() + const output = capture() + await runLogsPath({}, { logDirectory: directory, stdout: output.stdout }) + expect(output.lines.join('\n')).toContain('No log files yet.') + }) +}) + +describe('rawback logs show', () => { + test('prints the most recent records, newest last', async () => { + const directory = logDirectory() + writeFileSync( + join(directory, 'cli.log'), + [record('info', 'one'), record('info', 'two'), record('info', 'three')].join('\n') + '\n', + ) + const output = capture() + + await runLogsShow({ json: true, lines: 2 }, { logDirectory: directory, stdout: output.stdout }) + + const shown = output.json<{ lines: { msg: string }[] }>() + expect(shown.lines.map((line) => line.msg)).toEqual(['two', 'three']) + }) + + test('filters to the given level and above', async () => { + const directory = logDirectory() + writeFileSync( + join(directory, 'cli.log'), + [record('debug', 'noisy'), record('warn', 'notable'), record('error', 'bad')].join('\n'), + ) + const output = capture() + + await runLogsShow( + { json: true, level: 'warn' }, + { logDirectory: directory, stdout: output.stdout }, + ) + + expect(output.json<{ lines: { msg: string }[] }>().lines.map((line) => line.msg)).toEqual([ + 'notable', + 'bad', + ]) + }) + + test('surfaces the trace ID so it can be quoted in a support report', async () => { + const directory = logDirectory() + writeFileSync( + join(directory, 'cli.log'), + record('warn', 'http failed', { event: 'http.request', ids: { traceId: 'abc123' } }), + ) + const output = capture() + + await runLogsShow({ json: true }, { logDirectory: directory, stdout: output.stdout }) + expect(output.json<{ lines: { traceId?: string }[] }>().lines[0]?.traceId).toBe('abc123') + }) + + test('shows a torn line rather than dropping the evidence', async () => { + const directory = logDirectory() + writeFileSync(join(directory, 'cli.log'), `${record('info', 'fine')}\n{"half":`) + const output = capture() + + await runLogsShow({ json: true }, { logDirectory: directory, stdout: output.stdout }) + expect(output.json<{ lines: { raw?: string }[] }>().lines[1]?.raw).toBe('{"half":') + }) + + test('reads only the tail of a large file', async () => { + const directory = logDirectory() + const many = Array.from({ length: 5_000 }, (_, index) => record('info', `line-${index}`)) + writeFileSync(join(directory, 'cli.log'), `${many.join('\n')}\n`) + const output = capture() + + await runLogsShow({ json: true, lines: 3 }, { logDirectory: directory, stdout: output.stdout }) + expect(output.json<{ lines: { msg: string }[] }>().lines.map((line) => line.msg)).toEqual([ + 'line-4997', + 'line-4998', + 'line-4999', + ]) + }) + + test('reports cleanly when the file does not exist yet', async () => { + const directory = logDirectory() + const output = capture() + await runLogsShow({ json: true }, { logDirectory: directory, stdout: output.stdout }) + expect(output.json<{ lines: unknown[] }>().lines).toEqual([]) + expect(process.exitCode).toBeUndefined() + }) + + test('rejects an out-of-range line count and an unknown level', async () => { + const directory = logDirectory() + await expect(runLogsShow({ lines: 0 }, { logDirectory: directory })).rejects.toThrow('--lines') + await expect(runLogsShow({ level: 'loud' }, { logDirectory: directory })).rejects.toThrow( + '--level', + ) + }) + + test('reads the app the caller asked for', async () => { + const directory = logDirectory() + writeFileSync(join(directory, 'desktop.log'), record('info', 'from desktop')) + const output = capture() + + await runLogsShow( + { app: 'desktop', json: true }, + { logDirectory: directory, stdout: output.stdout }, + ) + expect(output.json<{ lines: { msg: string }[] }>().lines[0]?.msg).toBe('from desktop') + }) +}) + +describe('rawback logs purge', () => { + test('deletes the log files and leaves everything else alone', async () => { + const directory = logDirectory() + writeFileSync(join(directory, 'cli.log'), 'aaa') + writeFileSync(join(directory, 'cli.1.log'), 'aa') + // `logging.directory` is user-controlled, so purging must stay surgical. + writeFileSync(join(directory, 'notes.txt'), 'keep me') + const output = capture() + + await runLogsPurge( + { json: true, yes: true }, + { logDirectory: directory, stdout: output.stdout }, + ) + + expect(output.json<{ removed: string[]; bytes: number }>()).toMatchObject({ bytes: 5 }) + expect(readdirSync(directory)).toEqual(['notes.txt']) + }) + + test('narrows to one app when asked', async () => { + const directory = logDirectory() + writeFileSync(join(directory, 'cli.log'), 'a') + writeFileSync(join(directory, 'desktop.log'), 'a') + const output = capture() + + await runLogsPurge( + { app: 'desktop', json: true, yes: true }, + { logDirectory: directory, stdout: output.stdout }, + ) + expect(readdirSync(directory)).toEqual(['cli.log']) + }) + + test('rejects an unknown app rather than deleting the wrong thing', async () => { + const directory = logDirectory() + await expect( + runLogsPurge({ app: 'server', yes: true }, { logDirectory: directory }), + ).rejects.toThrow('--app') + }) + + test('asks before deleting, and leaves the files when the answer is no', async () => { + const directory = logDirectory() + writeFileSync(join(directory, 'cli.log'), 'aaa') + const output = capture() + const asked: string[] = [] + + await runLogsPurge( + {}, + { + logDirectory: directory, + prompts: { + confirm: async (message) => { + asked.push(message) + return false + }, + }, + stdout: output.stdout, + }, + ) + + expect(asked[0]).toContain('Delete 1 log file') + expect(readdirSync(directory)).toEqual(['cli.log']) + expect(output.lines.join('')).toContain('Left the log files in place') + }) + + test('deletes once the prompt is answered yes', async () => { + const directory = logDirectory() + writeFileSync(join(directory, 'cli.log'), 'aaa') + const output = capture() + + await runLogsPurge( + {}, + { + logDirectory: directory, + prompts: { confirm: async () => true }, + stdout: output.stdout, + }, + ) + expect(readdirSync(directory)).toEqual([]) + }) + + test('does not prompt when there is nothing to delete', async () => { + const directory = logDirectory() + const output = capture() + + await runLogsPurge( + {}, + { + logDirectory: directory, + prompts: { + confirm: async () => { + throw new Error('should not have asked') + }, + }, + stdout: output.stdout, + }, + ) + expect(output.lines.join('')).toContain('No log files to delete') + }) +}) From c8ea5800ea5e9a0eacced4b67160188eacfcd137 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 21 Sep 2026 05:29:18 +0000 Subject: [PATCH 2/4] docs(cli): note the logs contract in the agent guide `~/.rawback/logs/` joins the files shared with Desktop, and `logs purge` clears Desktop's log too. Records the two rules a future change would otherwise trip over: diagnostics never go to stdout, and `src/log-level.ts` must stay SDK-free at runtime. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HwoxuU7mv4yFQnecnKoaPY --- AGENTS.md | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 0189829..aadd28a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -72,6 +72,10 @@ cannot run, report exactly which command was skipped and why. - Author shared GraphQL operations and transport behavior in `../sdk`; this repository should keep only CLI presentation and platform adapters. - Keep secrets in `~/.rawback/`, never in repository fixtures or documentation. +- Diagnostics go to `~/.rawback/logs/cli.log` through the SDK logger, **never** + to stdout: stdout is the `--json` contract. `src/log-level.ts` holds the + `--log-level`/`-v` singleton and must stay SDK-free at runtime (its import is + type-only) for the same startup-cost reason `src/trace.ts` documents. ## Implementation expectations @@ -122,9 +126,10 @@ platform support that the release configuration does not provide. ## Gotchas -- `~/.rawback/{config.yml,credentials.json,upload-state.json,cameras.json}` is - shared with the Desktop app at runtime. Changing a file's shape here breaks - Desktop, and the contract is owned by `@rawback/sdk`, not by this repo. +- `~/.rawback/{config.yml,credentials.json,upload-state.json,cameras.json}` and + `~/.rawback/logs/` are shared with the Desktop app at runtime. Changing a + file's shape here breaks Desktop, and the contract is owned by `@rawback/sdk`, + not by this repo. `rawback logs purge` clears Desktop's log too, by design. - `@rawback/sdk` and `@rawback/ccapi-js` are pinned to exact versions. Bumping one means regenerating the lockfile; CI installs `--frozen-lockfile`. - The binary's `--version` must equal `package.json`'s version — CI asserts it From d934668584aca05e188b6f37e02f33ce17987a5f Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 21 Sep 2026 06:12:18 +0000 Subject: [PATCH 3/4] refactor(logs): follow the SDK onto pino The SDK's logging core is now pino, so this tracks its API and its file naming. - Records use pino's argument order, `logger.info({ event }, 'message')`. - `pino-roll` numbers every file from `1`, so there is no `cli.log` to open by name. `logs show` asks the SDK for the active file, and reports cleanly when the app has not logged yet; `logs path` and `logs purge` needed no change, since both already worked off a pattern. - `--level` filtering keeps its own copy of pino's level numbers rather than importing a map the SDK no longer exports. - `flushCommandLogger` goes through the SDK's `flushLogger`: the rolling destination is asynchronous, and a CLI process exits long before it would drain on its own. Docs updated for the numbering and to name pino, since the format is now one `pino-pretty` reads directly. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HwoxuU7mv4yFQnecnKoaPY --- AGENTS.md | 6 ++++-- README.md | 3 +-- docs/commands.md | 13 ++++++------ docs/configuration.md | 13 +++++++----- src/features/cli/runtime.ts | 7 ++++--- src/logging.ts | 18 ++++++++--------- src/logs.ts | 39 +++++++++++++++++++++++++----------- test/commands-cli.test.ts | 8 ++++++-- test/logs.test.ts | 40 ++++++++++++++++++------------------- 9 files changed, 86 insertions(+), 61 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index aadd28a..ebfb0bf 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -72,8 +72,10 @@ cannot run, report exactly which command was skipped and why. - Author shared GraphQL operations and transport behavior in `../sdk`; this repository should keep only CLI presentation and platform adapters. - Keep secrets in `~/.rawback/`, never in repository fixtures or documentation. -- Diagnostics go to `~/.rawback/logs/cli.log` through the SDK logger, **never** - to stdout: stdout is the `--json` contract. `src/log-level.ts` holds the +- Diagnostics go to `~/.rawback/logs/` through the SDK's pino logger, **never** + to stdout: stdout is the `--json` contract. Records use pino's argument + order, `logger.info({ event }, 'message')`, and `pino-roll` numbers every + file, so use `activeLogFile()` rather than building a name. `src/log-level.ts` holds the `--log-level`/`-v` singleton and must stay SDK-free at runtime (its import is type-only) for the same startup-cost reason `src/trace.ts` documents. diff --git a/README.md b/README.md index 149d706..e4fc127 100644 --- a/README.md +++ b/README.md @@ -396,8 +396,7 @@ rawback shares list --help ### When something goes wrong -The CLI keeps a JSON log at `~/.rawback/logs/cli.log`, shared with Rawback -Desktop. At the default verbosity it records every failure — including the trace +The CLI keeps JSON logs in `~/.rawback/logs/`, shared with Rawback Desktop. At the default verbosity it records every failure — including the trace and `cf-ray` IDs identifying the request on the server — and nothing else. ```bash diff --git a/docs/commands.md b/docs/commands.md index 6671a37..f7246bc 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -1036,9 +1036,9 @@ The URL uses `webHost` from `~/.rawback/config.yml`, or ## `rawback logs` -Read and clear the diagnostic log the CLI and the Desktop app write to -`~/.rawback/logs/`. One JSON object per line, so `jq` works directly on the -file. +Read and clear the diagnostic logs the CLI and the Desktop app write to +`~/.rawback/logs/`. One JSON object per line — pino's format — so `jq` and +`pino-pretty` work on them directly. ```bash rawback logs path @@ -1049,7 +1049,8 @@ rawback logs purge --yes ### `rawback logs path` Prints the log directory and every file in it with its size and modification -time. +time. Files are numbered (`cli.1.log`, `cli.2.log`, …) and the highest number +is the one currently being written. | Option | Description | | -------- | ---------------------------- | @@ -1057,8 +1058,8 @@ time. ### `rawback logs show` -Prints the most recent records, oldest first. Only the tail of the file is read, -so this stays fast on a ten-megabyte log. +Prints the most recent records from the file currently being written, oldest +first. Only the tail is read, so this stays fast on a ten-megabyte log. | Option | Description | | ----------------- | ------------------------------------------------ | diff --git a/docs/configuration.md b/docs/configuration.md index 2d57b86..4950282 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -483,15 +483,14 @@ logging: file: true # set false to stop writing to disk entirely directory: ~/.rawback/logs maxFileSize: 10485760 # roll the active file at this many bytes - maxFiles: 3 # rolled archives kept, *besides* the active file + maxFiles: 3 # how many files to keep in total redact: [] # extra field names to blank out, added to the built-in list stderr: false # also mirror records to standard error ``` -`maxFiles` counts archives, not the total: the default keeps `cli.log` plus -`cli.1.log` through `cli.3.log`, about 40 MB per app at the default size. -`redact` only ever adds to the built-in list — a setting that could switch -redaction off is a setting that leaks tokens. +`maxFiles` is how many files are kept in total, so the defaults come to about +30 MB per app. `redact` only ever adds to the built-in list — a setting that +could switch redaction off is a setting that leaks tokens. Like every other section, `logging` can appear at the top level and inside a named environment, where it merges key by key: a shared `level` applies @@ -529,6 +528,10 @@ On Linux and macOS log files are created at mode `0600` inside a `0700` directory, the same as `credentials.json`. Windows has no equivalent, so the files inherit the directory's permissions there. +Logging is [pino](https://github.com/pinojs/pino) under the hood, so the output +is the line-JSON format `pino-pretty` and most log tooling already understands: +`rawback logs show --json | jq -c .lines[] | pino-pretty` works. + ## Troubleshooting Whatever the symptom, `rawback logs show --level warn` is the fastest way to see diff --git a/src/features/cli/runtime.ts b/src/features/cli/runtime.ts index 27a56d7..51c9b95 100644 --- a/src/features/cli/runtime.ts +++ b/src/features/cli/runtime.ts @@ -68,7 +68,7 @@ export async function runCli( const command = commandName(parseArgs) const logger = (await commandLogger()).child({ component: 'cli', command }) const startedAt = performance.now() - logger.debug(`running ${command}`, { event: 'cli.command.start' }) + logger.debug({ event: 'cli.command.start' }, `running ${command}`) try { await program.parseAsync(parseArgs) } finally { @@ -77,15 +77,16 @@ export async function runCli( const exitCode = process.exitCode ?? 0 const failed = exitCode !== 0 logger[failed ? 'warn' : 'debug']( - failed ? `${command} exited ${String(exitCode)}` : `${command} finished`, { event: 'cli.command.done', exitCode, durationMs: Math.round(performance.now() - startedAt), }, + failed ? `${command} exited ${String(exitCode)}` : `${command} finished`, ) // A failed command is exactly the one whose records need to reach the - // file; the sink's exit drain is the backstop, not the mechanism. + // file, and the rolling destination is asynchronous — without this the + // process exits before it drains. await flushCommandLogger() } return diff --git a/src/logging.ts b/src/logging.ts index fa18f76..8e97df7 100644 --- a/src/logging.ts +++ b/src/logging.ts @@ -1,11 +1,11 @@ import { createAppLogger, + flushLogger, type Logger, type LogLevel, NOOP_LOGGER, readEnvironment, resolveLoggingOptions, - type RootLogger, } from '@rawback/sdk' import packageJson from '../package.json' with { type: 'json' } @@ -18,15 +18,15 @@ export interface LoggerDependencies { logLevel?: LogLevel } -let root: RootLogger | undefined -let building: Promise | undefined +let root: Logger | undefined +let building: Promise | undefined /** * The one logger this process writes through. * * Built lazily and memoized: a command that never talks to the network should - * not create `~/.rawback/logs/` on the way past, and one process wants one file - * handle and one exit hook rather than one per command module. + * not create `~/.rawback/logs/` on the way past, and one process wants one + * rolling destination rather than one per command module. */ export async function commandLogger(dependencies: LoggerDependencies = {}): Promise { if (dependencies.logger) return dependencies.logger @@ -36,7 +36,7 @@ export async function commandLogger(dependencies: LoggerDependencies = {}): Prom return root } -async function buildLogger(dependencies: LoggerDependencies): Promise { +async function buildLogger(dependencies: LoggerDependencies): Promise { const level = dependencies.logLevel ?? selectedLogLevel() const environment = await readEnvironment(dependencies.configPath, dependencies.env).catch( () => undefined, @@ -56,12 +56,12 @@ async function buildLogger(dependencies: LoggerDependencies): Promise { if (!root) return - await root.flush() + await flushLogger(root) } /** Forgets the memoized logger. Tests need this; nothing else should. */ diff --git a/src/logs.ts b/src/logs.ts index eebb905..164c934 100644 --- a/src/logs.ts +++ b/src/logs.ts @@ -1,20 +1,31 @@ import { open } from 'node:fs/promises' import { + activeLogFile, isLogApp, LOG_APPS, type LogApp, type LogDirectoryListing, type LogLevel, listLogFiles, - logFileName, - LOG_LEVELS, + LOG_LEVEL_NAMES, type PurgeLogsResult, purgeLogs, readEnvironment, resolveLoggingOptions, } from '@rawback/sdk' +/** pino's numbering, so `--level warn` can mean "warn and above". */ +const LEVEL_ORDER: Record = { + trace: 10, + debug: 20, + info: 30, + warn: 40, + error: 50, + fatal: 60, + silent: 70, +} + import { commandOutput, type ReadCommandDependencies } from './command.ts' import { environmentName } from './config.ts' import { logFilesDocument, logLinesDocument, purgeResultDocument } from './features/logs/view.ts' @@ -161,8 +172,10 @@ function parseLine(line: string): LogLine { function atLeastLevel(line: LogLine, minimum: LogLevel | undefined): boolean { if (minimum === undefined) return true const level = line.level as LogLevel | undefined - if (level === undefined || !(level in LOG_LEVELS)) return true - return LOG_LEVELS[level] >= LOG_LEVELS[minimum] + // A line without a usable level is shown rather than filtered away: it is + // more likely a torn record than something the caller meant to hide. + if (level === undefined || !(level in LEVEL_ORDER)) return true + return LEVEL_ORDER[level] >= LEVEL_ORDER[minimum] } export async function runLogsShow( @@ -174,23 +187,25 @@ export async function runLogsShow( throw new Error('--lines must be an integer between 1 and 10000') } const level = options.level as LogLevel | undefined - if (level !== undefined && !(level in LOG_LEVELS)) { - throw new Error(`--level must be one of ${Object.keys(LOG_LEVELS).join(', ')}`) + if (level !== undefined && !(level in LEVEL_ORDER)) { + throw new Error(`--level must be one of ${LOG_LEVEL_NAMES.join(', ')}`) } const directory = await resolveLogDirectory(dependencies) const app = requireApp(options.app) ?? 'cli' - const path = `${directory}/${logFileName(app)}` - const lines = (await readTail(path, count)) - .map(parseLine) - .filter((line) => atLeastLevel(line, level)) + // `pino-roll` numbers every file, so the current one is the highest-numbered + // rather than a fixed name. + const path = await activeLogFile(directory, app) + const lines = path + ? (await readTail(path, count)).map(parseLine).filter((line) => atLeastLevel(line, level)) + : [] const output = commandOutput(dependencies) if (options.json) { - output.json({ file: path, lines }) + output.json({ file: path ?? null, lines }) return } - output.document(logLinesDocument(path, lines)) + output.document(logLinesDocument(path ?? directory, lines)) } export async function runLogsPurge( diff --git a/test/commands-cli.test.ts b/test/commands-cli.test.ts index df9d179..b385a4c 100644 --- a/test/commands-cli.test.ts +++ b/test/commands-cli.test.ts @@ -1,5 +1,5 @@ import { afterEach, describe, expect, test } from 'bun:test' -import { existsSync, mkdtempSync, readFileSync, rmSync } from 'node:fs' +import { existsSync, mkdtempSync, readFileSync, readdirSync, rmSync } from 'node:fs' import { tmpdir } from 'node:os' import { join } from 'node:path' @@ -442,7 +442,11 @@ describe('logs commands', () => { // No credentials in an empty home, so this fails and should be recorded. expect(run('photos', 'list').exitCode).toBe(1) - const contents = readFileSync(join(home, '.rawback', 'logs', 'cli.log'), 'utf8') + // pino-roll numbers every file, so discover it rather than assume a name. + const logs = join(home, '.rawback', 'logs') + const [file] = readdirSync(logs) + expect(file).toMatch(/^cli\.\d+\.log$/) + const contents = readFileSync(join(logs, file!), 'utf8') const records = contents .split('\n') .filter(Boolean) diff --git a/test/logs.test.ts b/test/logs.test.ts index 42398b2..51b01bb 100644 --- a/test/logs.test.ts +++ b/test/logs.test.ts @@ -43,8 +43,8 @@ function capture() { describe('rawback logs path', () => { test('reports every log file with its size', async () => { const directory = logDirectory() - writeFileSync(join(directory, 'cli.log'), 'aaaa') - writeFileSync(join(directory, 'desktop.log'), 'bb') + writeFileSync(join(directory, 'cli.1.log'), 'aaaa') + writeFileSync(join(directory, 'desktop.1.log'), 'bb') const output = capture() await runLogsPath({ json: true }, { logDirectory: directory, stdout: output.stdout }) @@ -56,14 +56,14 @@ describe('rawback logs path', () => { totalBytes: 6, files: [ { - name: 'cli.log', - path: join(directory, 'cli.log'), + name: 'cli.1.log', + path: join(directory, 'cli.1.log'), bytes: 4, modifiedAt: expect.any(String), }, { - name: 'desktop.log', - path: join(directory, 'desktop.log'), + name: 'desktop.1.log', + path: join(directory, 'desktop.1.log'), bytes: 2, modifiedAt: expect.any(String), }, @@ -83,7 +83,7 @@ describe('rawback logs show', () => { test('prints the most recent records, newest last', async () => { const directory = logDirectory() writeFileSync( - join(directory, 'cli.log'), + join(directory, 'cli.1.log'), [record('info', 'one'), record('info', 'two'), record('info', 'three')].join('\n') + '\n', ) const output = capture() @@ -97,7 +97,7 @@ describe('rawback logs show', () => { test('filters to the given level and above', async () => { const directory = logDirectory() writeFileSync( - join(directory, 'cli.log'), + join(directory, 'cli.1.log'), [record('debug', 'noisy'), record('warn', 'notable'), record('error', 'bad')].join('\n'), ) const output = capture() @@ -116,7 +116,7 @@ describe('rawback logs show', () => { test('surfaces the trace ID so it can be quoted in a support report', async () => { const directory = logDirectory() writeFileSync( - join(directory, 'cli.log'), + join(directory, 'cli.1.log'), record('warn', 'http failed', { event: 'http.request', ids: { traceId: 'abc123' } }), ) const output = capture() @@ -127,7 +127,7 @@ describe('rawback logs show', () => { test('shows a torn line rather than dropping the evidence', async () => { const directory = logDirectory() - writeFileSync(join(directory, 'cli.log'), `${record('info', 'fine')}\n{"half":`) + writeFileSync(join(directory, 'cli.1.log'), `${record('info', 'fine')}\n{"half":`) const output = capture() await runLogsShow({ json: true }, { logDirectory: directory, stdout: output.stdout }) @@ -137,7 +137,7 @@ describe('rawback logs show', () => { test('reads only the tail of a large file', async () => { const directory = logDirectory() const many = Array.from({ length: 5_000 }, (_, index) => record('info', `line-${index}`)) - writeFileSync(join(directory, 'cli.log'), `${many.join('\n')}\n`) + writeFileSync(join(directory, 'cli.1.log'), `${many.join('\n')}\n`) const output = capture() await runLogsShow({ json: true, lines: 3 }, { logDirectory: directory, stdout: output.stdout }) @@ -166,7 +166,7 @@ describe('rawback logs show', () => { test('reads the app the caller asked for', async () => { const directory = logDirectory() - writeFileSync(join(directory, 'desktop.log'), record('info', 'from desktop')) + writeFileSync(join(directory, 'desktop.1.log'), record('info', 'from desktop')) const output = capture() await runLogsShow( @@ -180,8 +180,8 @@ describe('rawback logs show', () => { describe('rawback logs purge', () => { test('deletes the log files and leaves everything else alone', async () => { const directory = logDirectory() - writeFileSync(join(directory, 'cli.log'), 'aaa') - writeFileSync(join(directory, 'cli.1.log'), 'aa') + writeFileSync(join(directory, 'cli.1.log'), 'aaa') + writeFileSync(join(directory, 'cli.2.log'), 'aa') // `logging.directory` is user-controlled, so purging must stay surgical. writeFileSync(join(directory, 'notes.txt'), 'keep me') const output = capture() @@ -197,15 +197,15 @@ describe('rawback logs purge', () => { test('narrows to one app when asked', async () => { const directory = logDirectory() - writeFileSync(join(directory, 'cli.log'), 'a') - writeFileSync(join(directory, 'desktop.log'), 'a') + writeFileSync(join(directory, 'cli.1.log'), 'a') + writeFileSync(join(directory, 'desktop.1.log'), 'a') const output = capture() await runLogsPurge( { app: 'desktop', json: true, yes: true }, { logDirectory: directory, stdout: output.stdout }, ) - expect(readdirSync(directory)).toEqual(['cli.log']) + expect(readdirSync(directory)).toEqual(['cli.1.log']) }) test('rejects an unknown app rather than deleting the wrong thing', async () => { @@ -217,7 +217,7 @@ describe('rawback logs purge', () => { test('asks before deleting, and leaves the files when the answer is no', async () => { const directory = logDirectory() - writeFileSync(join(directory, 'cli.log'), 'aaa') + writeFileSync(join(directory, 'cli.1.log'), 'aaa') const output = capture() const asked: string[] = [] @@ -236,13 +236,13 @@ describe('rawback logs purge', () => { ) expect(asked[0]).toContain('Delete 1 log file') - expect(readdirSync(directory)).toEqual(['cli.log']) + expect(readdirSync(directory)).toEqual(['cli.1.log']) expect(output.lines.join('')).toContain('Left the log files in place') }) test('deletes once the prompt is answered yes', async () => { const directory = logDirectory() - writeFileSync(join(directory, 'cli.log'), 'aaa') + writeFileSync(join(directory, 'cli.1.log'), 'aaa') const output = capture() await runLogsPurge( From 2159d78f53f8689d05d0bc92a7769b6859420806 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 21 Sep 2026 06:35:53 +0000 Subject: [PATCH 4/4] docs(cli): note that log retention must be at least one file The SDK now rejects `maxFiles: 0` rather than silently rewriting it to 1; a rolling log always has a file open, so `file: false` is how writing to disk is switched off. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01HwoxuU7mv4yFQnecnKoaPY --- docs/configuration.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/configuration.md b/docs/configuration.md index 4950282..ec06bc9 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -483,13 +483,14 @@ logging: file: true # set false to stop writing to disk entirely directory: ~/.rawback/logs maxFileSize: 10485760 # roll the active file at this many bytes - maxFiles: 3 # how many files to keep in total + maxFiles: 3 # how many files to keep in total, at least 1 redact: [] # extra field names to blank out, added to the built-in list stderr: false # also mirror records to standard error ``` `maxFiles` is how many files are kept in total, so the defaults come to about -30 MB per app. `redact` only ever adds to the built-in list — a setting that +30 MB per app. It must be at least `1` — a rolling log always has a file open, +so set `file: false` rather than `maxFiles: 0` to stop writing to disk. `redact` only ever adds to the built-in list — a setting that could switch redaction off is a setting that leaks tokens. Like every other section, `logging` can appear at the top level and inside a