Skip to content

Download control: several connections, file names, headers and retries - #18

Merged
iStorry merged 4 commits into
mainfrom
feat/download-control-2
Oct 21, 2026
Merged

iStorry merged 4 commits into
mainfrom
feat/download-control-2

Conversation

@iStorry

@iStorry iStorry commented Oct 21, 2026

Copy link
Copy Markdown
Owner

Finishes download control after #17. There are three new optional settings on download, one commit each.

What's new

  • connections: int (244a3fa) — the first connection asks for Range: bytes=0-. When the server answers 206 with the file's size, the file is split into parts of at least 64 KiB, with at most 16 connections. Each part is fetched by its own connection into its own place in the .part file, and each connection opens the file separately. Each outlet has room for its part's end. Pausing records how far each part got, and resuming opens a connection for each unfinished part. A 200 answer, or one with no size, keeps one connection.
  • name: string (9fb959d) — this goes through the same sanitising as a name from the address (now name::safe_file_name), then Partial::create, so it can't leave the folder and never overwrites a file. "", the default, names the file after the address.
  • headers: [string] and retries: int (993b2d8) — each header is written Name: value and sent on every request through http::get. A malformed header, or Range, fails the download before it asks the server anything. Retries only cover transient errors: a broken body read, a connect timeout, or a 5xx. The wait starts at 1 s and doubles up to 30 s, using recv_timeout on the event receiver so pause and cancel still work during it. While every unfinished part is waiting, status shows Retrying….

Every surface

  • Checker table rows, HIR, interpreter, and compiled ABI: DownloadDesc gains name, connections, headers and retries, and ABI_VERSION goes from 4 to 5 (bumped once for all three).
  • In codegen, download now holds each borrowed setting across all later expressions, not just the next one (new hold_across).
  • Docs: functions.md, cheat sheet, syntax decisions 25–28 in language.md, and the roadmap (download control moves to Supported).
  • The VS Code grammar test covers the new labels. The downloader.nf fixture gains a Connections slider, retries: 3, and Retrying… in is_running.

Tests

All tests run against the local test server and are deterministic, with no sleeps. The test server now supports closed ranges, bytes=a-b.

  • Interpreter:
    • splitting into parts, checking the range each part asked for
    • falling back to one connection when the server can't send parts
    • pause and resume across parts
    • cancel across parts
    • a chosen name with no-overwrite, and an unsafe name
    • headers on every connection
    • a malformed header
    • retry after a 503
    • running out of retries
    • no retry on a 404
    • cancel during a retry wait
    • pause and resume during a retry wait
  • Compiled: the fixture splits a large file; a chosen name; headers plus a retry.
  • Unit tests: part splitting, range headers, header parsing, which errors are transient, and the backoff.

make check passes, including clippy for x86_64-pc-windows-msvc. The two retry tests wait out the real 1 s backoff.

Checked by hand

  • macOS: only through the tests above. Nothing has been run by hand in a real app yet.
  • Windows: not checked. Still to do:
    • a real multi-connection download
    • pause and resume with parts
    • cancelling while several handles have the .part file open (Windows file locking)
    • a 503 retry

No backend code changed.

Follow-up

Body::dialog in codegen has the same gap the download had: each borrowed argument is held only across the next expression. hold_across covers the fix; I've left it out of this PR.

download takes an optional connections: int. The first connection asks for
the whole file as a range; when the server answers 206 with the file's size,
the file is split into parts of at least 64 KB, each fetched by its own
connection into its own place in the .part file, opening the file
separately. A server that sends all of it, or no size, keeps one connection.

Each outlet now has room for its part's end, pausing records how far each
part got, and resuming opens a connection for each unfinished part. Adds
connections to DownloadDesc and bumps ABI_VERSION to 5.
download takes an optional name: string. It goes through the same
sanitising as a name from the address, so it can't leave the folder, then
Partial::create, so it keeps the no-overwrite numbering. "", the default,
names the file after the address. Adds name to DownloadDesc, within ABI 5.
download takes optional headers: [string], each written Name: value, sent
with every request through http::get; a malformed header or Range fails
the download before it asks the server. retries: int tries a connection
again after a failure that may pass (a broken-off body, a connect timeout
or a 5xx), waiting 1 s, doubling to 30 s. The wait is spent in
recv_timeout on the event receiver, so pausing or cancelling still works
at once; status says Retrying… while every unfinished part waits.

Download hold()s each borrowed setting across all later expressions, not
just the next. Adds headers and retries to DownloadDesc, within ABI 5, and
moves download control to Supported on the roadmap.
@iStorry
iStorry merged commit 14b61be into main Oct 21, 2026
5 checks passed
@iStorry
iStorry deleted the feat/download-control-2 branch October 21, 2026 08:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant