Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,12 @@ cannot run, report exactly which command was skipped and why.
- Author shared GraphQL operations and transport behavior in `../sdk`; this
repository should keep only CLI presentation and platform adapters.
- Keep secrets in `~/.rawback/`, never in repository fixtures or documentation.
- Diagnostics go to `~/.rawback/logs/` through the SDK's pino logger, **never**
to stdout: stdout is the `--json` contract. Records use pino's argument
order, `logger.info({ event }, 'message')`, and `pino-roll` numbers every
file, so use `activeLogFile()` rather than building a name. `src/log-level.ts` holds the
`--log-level`/`-v` singleton and must stay SDK-free at runtime (its import is
type-only) for the same startup-cost reason `src/trace.ts` documents.

## Implementation expectations

Expand Down Expand Up @@ -122,9 +128,10 @@ platform support that the release configuration does not provide.

## Gotchas

- `~/.rawback/{config.yml,credentials.json,upload-state.json,cameras.json}` is
shared with the Desktop app at runtime. Changing a file's shape here breaks
Desktop, and the contract is owned by `@rawback/sdk`, not by this repo.
- `~/.rawback/{config.yml,credentials.json,upload-state.json,cameras.json}` and
`~/.rawback/logs/` are shared with the Desktop app at runtime. Changing a
file's shape here breaks Desktop, and the contract is owned by `@rawback/sdk`,
not by this repo. `rawback logs purge` clears Desktop's log too, by design.
- `@rawback/sdk` and `@rawback/ccapi-js` are pinned to exact versions. Bumping
one means regenerating the lockfile; CI installs `--frozen-lockfile`.
- The binary's `--version` must equal `package.json`'s version — CI asserts it
Expand Down
23 changes: 23 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -394,6 +394,22 @@ rawback album article --help
rawback shares list --help
```

### When something goes wrong

The CLI keeps JSON logs in `~/.rawback/logs/`, shared with Rawback Desktop. At the default verbosity it records every failure — including the trace
and `cf-ray` IDs identifying the request on the server — and nothing else.

```bash
rawback logs show --level warn # what recently failed, and why
rawback -v photos upload ~/raw # re-run the failing command with more detail

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use --path in the troubleshooting upload example

The documented command cannot rerun an upload: photos upload defines the directory as the demanded --path option and has no positional path, so copying this example produces a validation error instead of collecting verbose upload diagnostics. Change it to rawback -v photos upload --path ~/raw.

AGENTS.md reference: AGENTS.md:L124-L125

Useful? React with 👍 / 👎.

rawback logs path # where the files are, and how big
rawback logs purge # delete them, this app and Desktop
```

Logs never go to standard output, so `-v` cannot disturb a script parsing
`--json`. Tokens, passwords and authorization headers are redacted before
anything is written.

## Files and security

Rawback stores local state under `~/.rawback/`:
Expand All @@ -404,6 +420,7 @@ Rawback stores local state under `~/.rawback/`:
| `config.yml` | Environments, optional hosts, metadata workers, and SFTP |
| `upload-state.json` | Shared upload queue, history, and trusted host keys |
| `cameras.json` | Saved Canon cameras, shared with Rawback Desktop |
| `logs/` | Rolling JSON diagnostics, shared with Rawback Desktop |

`cameras.json` is shared with the Rawback desktop app, so both can reach the same
camera without pairing twice. It holds a camera password only when you pass
Expand All @@ -421,6 +438,12 @@ replace it by hand or copy it from another machine. Run `rawback config init
into issues and logs. `rawback config view` masks every `sftp.password` in both
terminal and JSON output, including the ones inside `environments`.

Log files are written at mode `0600` too, and secret field and header names are
replaced with `[REDACTED]` before a record reaches disk. At `--log-level trace`
the log does record request and response bodies — file paths and album titles
among them — so turn that on to diagnose something rather than leaving it on.
See [Logging](docs/configuration.md#logging).

## Development

Install the pinned dependencies and run the CLI from source:
Expand Down
80 changes: 75 additions & 5 deletions docs/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,24 @@ English only.

## Global options

| Option | Description |
| ----------------- | --------------------------------------------- |
| `-h`, `--help` | Show help for the current command |
| `-V`, `--version` | Print the CLI version |
| `--env <name>` | Run against one environment from `config.yml` |
| Option | Description |
| --------------------- | ---------------------------------------------- |
| `-h`, `--help` | Show help for the current command |
| `-V`, `--version` | Print the CLI version |
| `--env <name>` | Run against one environment from `config.yml` |
| `--log-level <level>` | Verbosity of `~/.rawback/logs/cli.log` |
| `-v`, `--verbose` | Shorthand: `-v` for `debug`, `-vv` for `trace` |

Running `rawback` without arguments shows top-level help.

`--log-level` takes `trace`, `debug`, `info`, `warn`, `error`, `fatal` or
`silent` and only affects the log file — **never** standard output, so `--json`
stays machine-readable at any verbosity. The default is `info`, at which a run
records every failure and nothing else. `RAWBACK_LOG_LEVEL` sets the same thing
for a shell session, and the flag beats it. See
[Configuration](configuration.md#logging) for the file layout and
[`rawback logs`](#rawback-logs) for reading and clearing it.

`--env` accepts any name under `environments:` in `~/.rawback/config.yml`, plus
the reserved name `default` for the file's top-level settings. Without it,
commands use the saved `current:` environment. An unknown name fails and lists
Expand Down Expand Up @@ -1024,6 +1034,62 @@ rawback web
The URL uses `webHost` from `~/.rawback/config.yml`, or
`https://rawback.app` by default.

## `rawback logs`

Read and clear the diagnostic logs the CLI and the Desktop app write to
`~/.rawback/logs/`. One JSON object per line — pino's format — so `jq` and
`pino-pretty` work on them directly.

```bash
rawback logs path
rawback logs show --lines 100 --level warn
rawback logs purge --yes
```

### `rawback logs path`

Prints the log directory and every file in it with its size and modification
time. Files are numbered (`cli.1.log`, `cli.2.log`, …) and the highest number
is the one currently being written.

| Option | Description |
| -------- | ---------------------------- |
| `--json` | Output machine-readable JSON |

### `rawback logs show`

Prints the most recent records from the file currently being written, oldest
first. Only the tail is read, so this stays fast on a ten-megabyte log.

| Option | Description |
| ----------------- | ------------------------------------------------ |
| `--lines <n>` | How many records to show, 1–10000 (default `50`) |
| `--level <level>` | Show only records at this level or above |
| `--app <name>` | `cli` (default) or `desktop` |
| `--json` | Output machine-readable JSON |

A line that cannot be parsed is shown as-is rather than dropped — a torn record
is evidence too. `--json` emits `{ "file", "lines" }`, where an unparseable line
appears as `{ "raw": "…" }`.

### `rawback logs purge`

Deletes the log files. Only files this tool writes are removed and the directory
itself is left in place, so a custom `logging.directory` holding other files is
safe.

| Option | Description |
| -------------- | ------------------------------------ |
| `--app <name>` | `all` (default), `cli`, or `desktop` |
| `--yes` | Skip the confirmation prompt |
| `--json` | Output machine-readable JSON |

Without `--yes` it asks first, and fails with a message naming `--yes` when the
terminal is not interactive. The default `--app all` includes the Desktop app's
log, because both share one directory. A file that cannot be deleted is reported
rather than thrown — on Windows the Desktop app holds `desktop.log` open — and
the command exits `1` when any file was left behind.

## Scripting and exit behavior

Use `--json` when available instead of parsing human-readable tables. JSON is
Expand All @@ -1036,6 +1102,10 @@ indicators; redirected output is deterministic and contains no cursor-control
sequences. JSON, `--content-only`, and version output are never decorated with
icons or prose.

Diagnostic logs go to `~/.rawback/logs/cli.log`, never to standard output or
standard error, so raising the verbosity with `-v` cannot disturb a script
parsing `--json`.

The CLI exits with status `0` on success, `1` for validation, API, filesystem, or
upload failures, and `130` when an interactive prompt is cancelled. Scripts
should check the exit status before consuming output.
Expand Down
110 changes: 110 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,11 @@ webHost: https://rawback.app
metadata:
concurrency: 8

# Optional. Diagnostic logging, shared with the Desktop app. See "Logging".
logging:
level: info
maxFiles: 3

# Required only by `rawback photos upload`
sftp:
endpoint: sftp://ftp.rawback.app:23168
Expand Down Expand Up @@ -427,8 +432,113 @@ Verify the fingerprint through a trusted Rawback channel before pinning it. Do
not delete the upload-state file merely to bypass a host-key mismatch; investigate
the server or network change first.

## Logging

The CLI and the Desktop app both write diagnostics to `~/.rawback/logs/`, beside
`config.yml` and `credentials.json`:

| File | Written by |
| --------------------------- | ----------------------------- |
| `cli.log` | `rawback` |
| `desktop.log` | The Desktop app |
| `cli.1.log`, `cli.2.log`, … | Rolled archives, newest first |

Each line is one JSON object, so the file is ordinary JSONL:

```bash
tail -f ~/.rawback/logs/cli.log | jq -c '{time, level, event, ids}'
jq 'select(.levelValue >= 40)' ~/.rawback/logs/cli.log
Comment on lines +449 to +450

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reference the numbered active log file

These troubleshooting commands target cli.log, but the configured pino-roll sink numbers every file and the implementation explicitly discovers the highest-numbered cli.N.log as the active file. A user copying either command therefore gets a missing-file error even while CLI logs exist; make the example discover the active numbered file or direct users through rawback logs path.

AGENTS.md reference: AGENTS.md:L75-L78

Useful? React with 👍 / 👎.

```

Every record carries `time`, `level`, `levelValue`, `msg`, the writing app and
its version, the process ID, and the environment in use. API records add the
method, path, status and duration, plus the `x-trace-id`, `cf-ray` and
`x-request-id` values identifying that request server-side — those are the IDs
worth quoting in a support report.

### Levels

`trace` < `debug` < `info` < `warn` < `error` < `fatal`, plus `silent` to write
nothing. The default is `info`, at which a run records **every failure and
nothing else** — browsing the library does not fill the file with one line per
request.

- `debug` adds the full request stream and request/response headers.
- `trace` adds request and response bodies, truncated.

Secret field and header names — `authorization`, `cookie`, `password`, `token`,
`refreshToken` and friends — are replaced with `[REDACTED]` at every level, at
any nesting depth. Credential and authentication GraphQL operations never log
their variables or body even at `trace`. Query strings are stripped from logged
URLs, because they carry search terms.

`trace` still records file paths, album titles and other content from request
bodies. Turn it on to diagnose something, not as a standing setting.

### Configuration

```yaml
logging:
level: info # trace|debug|info|warn|error|fatal|silent
file: true # set false to stop writing to disk entirely
directory: ~/.rawback/logs
maxFileSize: 10485760 # roll the active file at this many bytes
maxFiles: 3 # how many files to keep in total, at least 1
redact: [] # extra field names to blank out, added to the built-in list
stderr: false # also mirror records to standard error
```

`maxFiles` is how many files are kept in total, so the defaults come to about
30 MB per app. It must be at least `1` — a rolling log always has a file open,
so set `file: false` rather than `maxFiles: 0` to stop writing to disk. `redact` only ever adds to the built-in list — a setting that
could switch redaction off is a setting that leaks tokens.

Like every other section, `logging` can appear at the top level and inside a
named environment, where it merges key by key: a shared `level` applies
everywhere while one environment overrides just the `directory`.

These environment variables override the file, and the `--log-level` flag
overrides them:

| Variable | Effect |
| ----------------------- | --------------------------------------- |
| `RAWBACK_LOG_LEVEL` | One of the level names above |
| `RAWBACK_DEBUG=1` | Shorthand for `RAWBACK_LOG_LEVEL=debug` |
| `RAWBACK_LOG_FILE=0` | Stop writing to disk |
| `RAWBACK_LOG_DIR` | Write somewhere other than the default |
| `RAWBACK_LOG_MAX_SIZE` | Roll at this many bytes |
| `RAWBACK_LOG_MAX_FILES` | Keep this many rolled archives |
| `RAWBACK_LOG_STDERR=1` | Also mirror records to standard error |

An unrecognised value is ignored rather than rejected, so a typo in a shell
profile cannot stop a command from running.

### Reading and clearing

```bash
rawback logs path # where the files are, and how big
rawback logs show --level warn # the recent records that mattered
rawback logs purge --yes # delete them, this app and Desktop
```

`purge` removes only the files listed above and leaves the directory itself
alone, so pointing `logging.directory` at a folder holding other things is safe.
See [`rawback logs`](commands.md#rawback-logs) for the full options.

On Linux and macOS log files are created at mode `0600` inside a `0700`
directory, the same as `credentials.json`. Windows has no equivalent, so the
files inherit the directory's permissions there.

Logging is [pino](https://github.com/pinojs/pino) under the hood, so the output
is the line-JSON format `pino-pretty` and most log tooling already understands:
`rawback logs show --json | jq -c .lines[] | pino-pretty` works.

## Troubleshooting

Whatever the symptom, `rawback logs show --level warn` is the fastest way to see
what actually failed, including the trace and `cf-ray` IDs to quote when
reporting it. Re-run the failing command with `-v` first if the log is empty.

### Device authorization is temporarily unavailable

The CLI retries temporary network and server failures while creating the device
Expand Down
Loading
Loading