Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 50 additions & 0 deletions Docs/04-advanced-features/web-servers.md
Original file line number Diff line number Diff line change
Expand Up @@ -1194,6 +1194,56 @@ curl -k https://localhost:8443/

⚠️ Self-signed certificates are for development only. In production, use a certificate from a real authority (e.g. Let's Encrypt via certbot).

### Multiple domains on one HTTPS port

Add `for "hostname"` to a certificate/key pair, then join additional pairs with
`and certificate`. WFL selects the certificate using the DNS name sent by the
client in its TLS handshake (Server Name Indication, or SNI):

```wfl
// CI-SKIP: Requires certificate and key files covering the configured domains.
listen on port 8443 secured with
certificate "one.pem" and key "one.key" for "one.example.com"
and certificate "two.pem" and key "two.key" for "two.example.com" as secure_server
```

Hostnames and paths may also be text variables or parenthesized expressions.
Each clause evaluates its certificate path, key path, then hostname exactly once
in source order, stopping at the first invalid operand.
The listener supports up to 128 named entries. Names are matched exactly and
case-insensitively. Use DNS names (international names in ASCII/Punycode form),
without a URL scheme, port, trailing dot, IP address, or wildcard. A wildcard
certificate can cover a configured exact name, but `for "*.example.com"` is not
a wildcard routing rule. Repeat the pair for each exact name it should serve.

WFL loads and validates every pair before opening the listening socket. Invalid
DNS names, duplicate names (including different capitalization), unreadable or
malformed files, mismatched keys, and certificates that do not cover their
configured hostname stop startup with an error. Certificates are loaded once;
restart the listener after replacing files to pick up renewals. WFL does not
issue or automatically renew certificates.

A listener containing only named pairs rejects clients with unknown or missing
SNI during the TLS handshake. It does not silently use the `.wflcfg` certificate
as a fallback. To provide a fallback explicitly, put an unnamed pair first:

```wfl
// CI-SKIP: Requires certificate and key files covering the configured domains.
listen on port 8443 secured with
certificate "default.pem" and key "default.key"
and certificate "one.pem" and key "one.key" for "one.example.com" as secure_server
```

The default pair is used when SNI is absent or does not match a named entry.
Clients still verify that the selected certificate covers the name they used.
Existing single-certificate statements and bare `secured` configuration behave
as before.

SNI selects the TLS certificate; it does not dispatch application handlers or
authorize access to a site. Route requests by `header "Host" of req` and their
path in your WFL application, and reject unknown HTTP hosts. An HTTP Host header
can differ from the TLS name; WFL does not enforce equality between them.

### Production notes

- The private key file must be readable by the WFL process — protect it with file permissions (`chmod 600 key.pem`).
Expand Down
5 changes: 5 additions & 0 deletions Docs/reference/configuration-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -628,6 +628,11 @@ listen on port 8443 secured as s

A plain `listen` (without `secured`) **always** serves HTTP — putting cert paths in `.wflcfg` never silently upgrades HTTP to HTTPS.

For multiple certificates on the same port, use the named certificate clauses
on `listen` described in [Multiple domains on one HTTPS port](../04-advanced-features/web-servers.md#multiple-domains-on-one-https-port).
These default configuration paths apply to bare `secured` statements only;
named-only listeners do not inherit a fallback certificate from configuration.

#### `web_server_tls_key_file`

Default TLS private key (PEM) for bare `listen … secured`.
Expand Down
4 changes: 4 additions & 0 deletions Docs/reference/keyword-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -370,6 +370,10 @@ connect to database at "sqlite://app.db" as db
- 5 appear contextual but are actually always reserved

### "What about `secured`, `certificate`, `key`, `redirecting`, `content_type`, `transaction`, `schema`, `changes`, `without following redirects`?" → Not keywords

For HTTPS, `certificate ... and key ... for "example.com"` associates a pair
with a DNS name using the existing `for` keyword. Join more named pairs with
`and certificate`. See [multiple-domain HTTPS](../04-advanced-features/web-servers.md#multiple-domains-on-one-https-port).
These words are recognized purely by position — inside `listen` / `respond` statements, in `in transaction on db:` / `in transaction on db for schema changes:` / `end transaction`, or as the `and without following redirects` clause in `open url` — and are **never reserved**. Use them as variable names freely. `raise_error` is a standard-library function, not a keyword. See [Marker Words That Are Not Keywords](reserved-keywords.md#marker-words-that-are-not-keywords-at-all).

---
Expand Down
5 changes: 5 additions & 0 deletions Docs/reference/reserved-keywords.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,11 @@ A few words have special meaning in exactly one statement position but are **not
- `schema`, `changes` - optional SQLite transaction mode in `in transaction on db for schema changes:`
- `without following redirects` - per-request redirect clause in `open url at address and without following redirects and read response as reply`; the individual words and the complete phrase remain ordinary variable names outside that clause position

TLS pairs can also use the existing `for` keyword to select a DNS name:
`certificate "cert.pem" and key "key.pem" for "example.com"`. Additional named
pairs begin with `and certificate`; this adds no reserved words. See
[multiple-domain HTTPS](../04-advanced-features/web-servers.md#multiple-domains-on-one-https-port).

`raise_error` is an ordinary standard-library function called with `call
raise_error with message` or `raise_error of message`; it is not a keyword.

Expand Down
156 changes: 156 additions & 0 deletions Engineering/evidence/2026-09-23-tls-sni.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# TLS SNI certificate selection

## Scope and risk

Risk class **R3**: TLS certificate selection, untrusted ClientHello input,
language syntax, and backward compatibility. One secured listener can select
among at most 128 named certificate/key pairs, with an optional explicit default
pair. Named-only listeners reject missing and unknown SNI. No HTTP routing,
client authentication, certificate issuance, renewal, or configuration-file
format changes are included.

The existing connection transport, queues, request budgets, streaming behavior,
and shutdown ownership are unchanged. Rustls handles ClientHello decoding and
hostname/certificate verification. Selection performs an immutable map lookup;
there is no file I/O or configuration mutation during a handshake.

## Test-first history

- Published base: `9c7c8f55057789c1c5f0a275989b433ed0d8438b` (`main`).
- Published Red: `6ad75a47` (`test: specify multi-certificate TLS SNI behavior (red)`).
- Published Green: `b1f8d747` (`feat: select HTTPS certificates by TLS server name`),
a direct descendant of the Red commit.
- The original development lineage was base `9bf3fa48`, Red `43b4889b`,
Green `341099ee`. Before publication, only the three SNI commits were rebased
onto current `main` to exclude the unrelated, unmerged ES256 feature present
in the starting checkout. The original validation below describes that local
development lineage; post-rebase validation and PR CI are recorded separately.
- Before any production change, `cargo test --test web_server_sni_test -- --nocapture`
compiled and failed all five initial tests: `for` and additional certificate
clauses were rejected by the existing parser; the real binary exited before
creating a listener. These are the missing language capability under test.
- During implementation, the multiline case exposed the need to consume line
boundaries between certificate clauses. The operand test also needed its
fixture variable changed from reserved `server` to `secure_server` before its
analyzer/typechecker assertions could execute. That fixture failure is not
counted as evidence of an operand-validation defect.
- The final suite broadens the initial contracts to seven tests, including
malformed/oversized lists and runtime operand checks.

## Acceptance criteria and coverage

| Contract | Evidence |
| --- | --- |
| Two DNS names share one TLS port and receive the correct distinct certificate | `sni_binary_selects_two_certs_rejects_unknown_and_survives_stalled_clients`: real WFL child process, generated self-signed certificates added to the client trust store, hostname verification enabled, exact peer certificate bytes and HTTP response asserted |
| Unknown/missing SNI is rejected without a default | Same test; unknown SNI must return a server TLS alert, rather than merely fail client hostname verification |
| An explicit unnamed first pair handles unknown/missing SNI; named entries take precedence | `sni_explicit_default_handles_unknown_and_missing_sni` |
| Configuration does not silently add a fallback to named-only listeners | `sni_runtime_checks_dynamic_operands_and_ignores_implicit_default` |
| Invalid DNS names, case-insensitive duplicates, SAN mismatch, missing files and key mismatch fail startup | `sni_invalid_configuration_fails_before_binding`; existing malformed PEM/key cases in `web_server_tls_test` |
| Literals, variables, multiline declarations, semantic/type checks and runtime text checks work | `sni_syntax_accepts_named_certificates_and_variables`, `sni_operands_are_analyzed_and_typechecked`, and runtime operand test |
| Incomplete, ambiguous and oversized lists are rejected; 128 entries parse | `sni_parser_rejects_incomplete_ambiguous_and_oversized_lists`; retained `fuzz/seeds/fuzz_parser/seed_tls_sni*.wfl` inputs |
| Slow/failed handshakes do not stop either domain; closing the listener cancels idle established/partial connections | Real-binary SNI test runs two domain requests concurrently beside partial and idle clients, checks close/error rather than post-close bytes, and verifies the port stops accepting |
| Single-certificate/configured TLS, HTTP/2, HTTP/1.1, TLS 1.2/1.3, redirects, peer IP, and lifecycle remain compatible | All 27 existing TLS parser/runtime tests plus web integration and the full Rust suite |
| WFL startup errors remain catchable | Both tests in `TestPrograms/tls_sni/startup.test.wfl` with the release binary |

Existing cancellation, timeout, disconnect, bounded-queue/backpressure,
resource-limit and handler-isolation tests passed in the full Rust suite. This
change does not introduce a new queue, task, handshake scheduler or streaming
path. The existing parser fuzz target builds against the change; no extended
fuzz campaign or quantitative coverage run is claimed.

## Verification

Local platform: Windows x86-64, Rust 1.98.1. No dependencies or lockfiles changed.

- `cargo fmt --all -- --check`: passed.
- `cargo clippy --all-targets --all-features -- -D warnings`: passed.
- `cargo test --all --no-fail-fast`: passed, 2,431 tests across 177 result groups;
27 pre-existing ignored cases remain unchanged. All seven new tests ran.
- `cargo build --release`: passed.
- `cargo build --locked -p wfl-lsp`: passed.
- `cargo check --locked --manifest-path fuzz/Cargo.toml`: passed.
- `scripts/run_web_tests.ps1`: passed all three scenarios, including native TLS
and HTTP-to-HTTPS redirection.
- `scripts/run_integration_tests.ps1 -TestOnly`: Rust integration tests passed;
the WFL sweep reported **165 passed, 1 failed, 24 pre-existing exclusions**.
The new `tls_sni/startup.test.wfl` passed. The failure was
`file_io_comprehensive.wfl` scanning `.` recursively and encountering
`target/test-artifacts/config-review/snapshot-5n0sbodd`, an inaccessible
artifact directory created September 17, before this change. A diagnostic
rerun from the repository root reproduced `Access is denied (os error 5)` at
line 123. Python directory traversal independently identified that directory.
The unchanged program passed from an isolated directory under
`target/test-artifacts/tls-sni-file-io/`. The original suite failure remains
recorded; this is not a claim that the original full sweep passed. Three
source-root test files left by its failed cleanup were removed afterward.
- `target/release/wfl --execution-timeout 330 --test tests/fixtures/cli_budget/long-run.test.wfl`:
passed the real 305-second execution-budget check.
- `python -X utf8 scripts/validate_docs_examples.py --ci --force`: passed all
38 registered examples. The external-certificate example is explicitly
non-executable and checked at layers 1–4; real TLS execution is tested above.
- The built `wfl-lsp --mcp` passed `parse_wfl`, `analyze_wfl`, `typecheck_wfl`
and `lint_wfl` on both new WFL files, with no diagnostics/type/lint errors.
- `python scripts/check_repo_hygiene.py --mode static`: passed.

Raw local reports are ephemeral under `target/reports/tls-sni/`. The first
sandboxed full test run failed in the existing Git patch round-trip test because
Git could not access the temporary directory. Running the unchanged suite with
that filesystem restriction removed passed. The first documentation run also
hit Python's Windows text-decoding error; explicit UTF-8 fixed the environment.
Neither failure was hidden through skipped tests or relaxed assertions. The
validator still emits existing warnings for its manifest schema metadata keys.

## PR review follow-up (2026-09-23)

[PR #746](https://github.com/WebFirstLanguage/wfl/pull/746) publishes only the
SNI changes. After rebasing onto `main`, all 34 initial TLS tests passed locally.

The operand-order finding was reproduced in test-only commit `02972e3a`:
`sni_evaluates_operands_once_in_source_order` observed `host;cert;key;` instead
of `cert;key;host;`. The follow-up fix assigns named result fields in source
order and short-circuits on an invalid operand. All 36 TLS tests (nine SNI,
14 original parser, 13 original runtime) then passed. The added negative case
also verifies that an invalid certificate operand prevents key/hostname side
effects.

The fallback finding was investigated against the actual locked Rustls 0.23.45
resolver: `ResolvesServerCertUsingSni::resolve` only looks up the supplied name;
it does not filter the key by signature schemes. The new
`sni_incompatible_named_key_does_not_select_compatible_default` test passed
before any runtime fix. It gives both an ECDSA named certificate and an Ed25519
default certificate the same DNS name, then restricts a TLS 1.3 client to
Ed25519. Configured SNI receives a server fatal alert; absent SNI succeeds with
the exact default certificate; an unrestricted named request receives the exact
named certificate. No resolver behavior change was needed.

A subsequent review found that the separate unused-variable pass did not visit
TLS operands. Test-only commit `70745701` records two failing regressions: the
real `wfl --analyze` CLI incorrectly reported all five default/named TLS
variables unused, and the analyzer unit test reported six unused variables
instead of only its deliberately unused sentinel. The fix visits both default
paths and each named certificate's certificate, key and hostname expressions.
All 38 analyzer tests and all 37 TLS tests (ten SNI, 14 parser, 13 runtime)
then passed, retaining detection of genuinely unused variables.

The initial PR head's Linux/Windows WFL-program suites, build/test/Clippy,
fuzz compilation, repository hygiene, database, release scripts, config lint and
Docker jobs passed. Its integration jobs reached the long-budget step without
failures. These results are superseded by final-head checks after pushing the
review fix; final CI links and review disposition are recorded on the PR.

## Release boundaries and recovery

No deployment or merge is claimed. The original local validation did not include
Linux execution or independent R3 review; the PR now supplies platform CI and
independent review. Clean final-head Windows/Linux CI and resolved review findings
remain merge/release gates. Extension host tests were not run locally because
no VS Code extension code, dependencies, or protocol changes were made; the PR's
Build, Test, Clippy job runs the extension checks.
The repository-root file-I/O sweep failure needs a clean test workspace (or
repair of the old inaccessible artifact directory) before that gate can pass.

Revert the feature commit to remove the new syntax. Existing single-certificate
programs require no migration. Programs using the new named syntax must return
to a single certificate or proxy TLS before running on an older binary. No
persistent state, certificate files, or external server configuration is changed
by this implementation task.
36 changes: 36 additions & 0 deletions History/dev-diary/2026/2026-09-23-tls-sni.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Multiple TLS certificates on one listener

WFL now accepts named certificate/key pairs on secured listeners using
`for "hostname"` and additional `and certificate` clauses. The original unnamed
pair remains an optional explicit fallback; bare `secured` still uses the
existing configuration paths. Named-only listeners reject unknown or absent
SNI instead of inheriting a default from configuration.

The runtime builds a Rustls SNI resolver before binding, validates DNS names and
certificate coverage, and rejects duplicate names and invalid key pairs. The
existing per-connection handshake and cancellation transport is unchanged.
Certificate selection does not add HTTP virtual-host routing or Host/SNI
authorization checks. Certificate renewal still requires a listener restart.

Tests exercise the real WFL binary with generated certificates and verified TLS
clients, assert the selected certificate bytes and HTTP responses for two names,
and cover fallback, rejection, startup validation, partial handshakes, and
shutdown. Parser, analyzer, typechecker, and WFL startup regression cases cover
the language boundary. See the corresponding engineering evidence record for
commands, outcomes, and remaining release gates.

## Review follow-up (2026-09-23)

PR review identified that named-clause operands ran in hostname-first order.
A failing regression observed `host;cert;key;` instead of the source order
`cert;key;host;`. The runtime now evaluates each operand once in source order
and stops on an invalid operand. A separate verified TLS regression confirms
that a named ECDSA key incompatible with an Ed25519-only client fails the
handshake rather than falling back to a compatible default certificate covering
the same hostname; the pinned Rustls resolver already preserves that behavior.

A further review caught a separate static-analysis visitor that did not count
TLS operands as variable uses. Unit and real-CLI regressions reproduced false
unused-variable diagnostics for certificate paths, key paths, and hostnames.
That visitor now traverses both default and named pairs while retaining warnings
for genuinely unused variables.
6 changes: 6 additions & 0 deletions TestPrograms/docs_examples/_meta/manifest.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,10 @@
{
"docs_examples/tls_sni/multiple_domains.wfl": {
"doc_section": "Docs/04-advanced-features/web-servers.md#multiple-domains-on-one-https-port",
"type": "snippet",
"validate_layers": [1, 2, 3, 4],
"skip_execution": true
},
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://wfl.example.com/schemas/docs-examples-manifest.json",
"title": "WFL Documentation Examples Manifest",
Expand Down
10 changes: 10 additions & 0 deletions TestPrograms/docs_examples/tls_sni/multiple_domains.wfl
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
// CI-SKIP: Requires certificate and key files covering the configured domains.
listen on port 8443 secured with
certificate "one.pem" and key "one.key" for "one.example.com"
and certificate "two.pem" and key "two.key" for "two.example.com" as secure_server
close server secure_server

listen on port 8443 secured with
certificate "default.pem" and key "default.key"
and certificate "one.pem" and key "one.key" for "one.example.com" as fallback_server
close server fallback_server
Loading
Loading