Skip to content

Read web pages with fetch, and search and cut strings - #20

Open
iStorry wants to merge 2 commits into
mainfrom
feat/fetch
Open

iStorry wants to merge 2 commits into
mainfrom
feat/fetch

Conversation

@iStorry

@iStorry iStorry commented Oct 21, 2026 •

Copy link
Copy Markdown
Owner

What

  • fetch(url, status: $s, text: $page, code: $c, response_headers: $h, headers: […]) { … } reads an address in the background, writes the response into state, then runs its block with the names around the call as they were when it was made (syntax decision row 29).
  • json_text(json, path) reads one value out of JSON as text (row 31).
  • Strings gain contains, starts_with, ends_with, find, slice, split, replace and trim, counting characters as len does; find gives -1 when nothing is found and slice outside the text is a runtime error, like a list position (row 30).

Every surface

  • Checker: fetch and json_text rows in FUNCTIONS (functions gain a block field, as elements have); string methods in METHODS with a receiver and typed parameters. return in an action block gets its own message.
  • Interpreter and compiled apps: shared implementations in nifty-runtime/src/text.rs and nifty-download/src/fetch.rs. In compiled apps the block is compiled as an action over a copy of the caller's frame, with for aliases read into it, defined once the enclosing function is. ABI_VERSION is now 6.
  • LSP: completion inside a fetch block offers what an action does; function snippets add a block when one is needed; hover shows method parameters.
  • Highlighters: Tree-sitter corpus, Zed, TextMate (plus a syntax test) and the docs highlighter.
  • Docs: functions.md (fetch, json_text), a string methods section in values.md, language.md, cheat-sheet.md, the source-code map and the roadmap. The new fixtures/preview.nf is included in the docs and run by tests both ways.

Built-in calls move out of the interpreter's eval and codegen's expr. On main, an interpreted call took about 15.5 KB of stack in debug builds, so the 128-deep recursion test sat at 1.98 MB of a 2 MB thread; it now takes about 14.4 KB.

Checked

  • cargo fmt --all --check, cargo clippy --workspace --all-targets -D warnings (also --target x86_64-pc-windows-msvc), cargo build --workspace, cargo test --workspace, Tree-sitter and VS Code tests, and mdbook build docs all pass on macOS.
  • Against real TikTok on macOS, through the interpreter: tiko, a downstream app built on this (local, not yet on GitHub), fetched a public video's page, read its data with json_text and the string methods, and saved it with download and the page's cookies. The HD (1080p HEVC, decodes cleanly with ffmpeg), SD, watermarked and sound (.m4a) files were all saved. Removed videos, profile links and non-TikTok links gave their messages. The same app compiles with nf build.
  • Not checked by hand: clicking through the GUI on macOS (computer use wasn't available to me), and anything on Windows beyond CI. No backend crate changed; fetch uses the same ureq + native TLS client as download.

`fetch(url, status: $s, text: $page, code: $c, response_headers: $h, headers: […]) { … }`
reads an address in the background, writes the response into state, then runs its
block with the names around the call as they were when it was made. `json_text(json,
path)` reads a value out of JSON, and strings gain `contains`, `starts_with`,
`ends_with`, `find`, `slice`, `split`, `replace` and `trim`, counting characters as
`len` does.

Every surface has them: the checker tables, the interpreter, compiled apps (the block is
compiled as an action over a copy of the caller's frame), the language server, the four
highlighters, the reference docs and syntax decisions, the roadmap, and a `preview.nf`
fixture that a test runs both ways. Built-in calls move out of the interpreter's `eval`
and codegen's `expr`, which also leaves each interpreted call using less stack than
before.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant