feat(secure): add NoiseHFS - post-quantum hybrid Noise (Noise_XXhfs_25519+MLKEM768_ChaChaPoly_SHA256)

[paschal533]

Draft: opened for early feedback rather than as something ready to merge - see "What's left" below.

What this adds

A post-quantum hybrid variant of the Noise handshake, NoiseHFS, implementing Noise_XXhfs_25519+MLKEM768_ChaChaPoly_SHA256 under protocol id /noise-mlkem768-hfs/0.2.0. It applies the Noise Hybrid Forward Secrecy extension (the e1/ekem1 tokens) to the classical XX pattern, so a hybrid-capable node keeps the same three-message handshake structure as plain /noise, just with an extra ML-KEM-768 key encapsulation mixed into the chaining key alongside the existing X25519 DH operations:

-> e, e1
<- e, ee, ekem1, s, es
-> s, se

The session stays confidential if X25519 is later broken by a quantum computer (the KEM covers that), and stays confidential if ML-KEM-768 turns out to have a classical weakness (the DH tokens cover that) - neither side's failure weakens the other's contribution to the final keys.

Full wire format, the reasoning behind picking raw ML-KEM-768 over a composite KEM like X-Wing, and the exact ekem1 token ordering (which matters - swapping two steps there silently produces divergent chaining keys) are written up in libp2p/protocols/secure/NOISE_HFS_SPEC.md. The profile itself is specified in libp2p/specs#727, a Stage 1A Working Draft by @royzah. My own draft, libp2p/specs#716, was closed on 2026-09-18 in favour of #727 so that one document rather than two carries the proposal; its text stays readable at https://github.com/paschal533/specs/tree/master/noise-pq, and the material it covered (test vectors, the interoperability matrix, wire sizes and the downgrade analysis) was offered to #727.

Breaking change (2026-09-17): the suite was renamed from Noise_XXhfs_25519+ML-KEM-768_ChaChaPoly_SHA256, because Noise (revision 34, §8.2) allows only alphanumerics and / in algorithm names. The name is hashed into the handshake hash, so peers on the old and new names cannot complete a handshake, and NoiseHFSCodec moved from /noise-mlkem768-hfs/0.1.0 to /noise-mlkem768-hfs/0.2.0. Message sizes are unchanged. /noise-mlkem768-hfs/0.2.0 is the id this branch ships, not a spec-endorsed one: #727 writes /noise-mlkem768-hfs/0.1.0 and lists the identifier string as the first of its open issues, so this will follow whatever #727 settles on.

Why raw ML-KEM-768 specifically

This wire format is meant to match Noise_XXhfs_25519+MLKEM768_ChaChaPoly_SHA256 as described in a research paper I've been working on ("Post-Quantum Cryptography Integration into the Noise Protocol"), which covers the TypeScript, Python, Rust and Nim implementations of the same profile. The design rationale is that the XXhfs pattern's three DH tokens already provide classical security on their own, so wrapping the KEM slot in a composite primitive like X-Wing would just be a redundant extra X25519 operation.

Where the ML-KEM-768 implementation comes from

Rather than pulling in a separate PQC library, libp2p/crypto/mlkem768.nim binds directly to the MLKEM768_* C API that BoringSSL already ships (crypto/mlkem/mlkem.cc), which nim-libp2p already compiles in through its boringssl dependency for TLS. It's the same MLKEM768_* API that BoringSSL's own TLS stack calls for its X25519MLKEM768 key share. No new C dependency, no new nimble dependency.

How it's wired in

NoiseHFS reuses noise.nim's SymmetricState/CipherState/KeyPair and message framing as-is (I exported what was needed rather than duplicating it - see the "export primitives" commit, which is purely additive visibility changes plus one new optional parameter with a default that keeps the classical Noise class's behaviour unchanged). Only the e1/ekem1 token handling and the top-level connection encrypter are new.

It's registered as a second SecureProtocol variant alongside the existing one, with a withNoiseHFS() builder method, meant to be mounted next to plain Noise so a hybrid-capable node still falls back to classical /noise transparently against peers that don't support it - no coordination needed, multistream-select handles the negotiation.

What's verified

I didn't have a Nim toolchain locally, so I set one up from scratch (portable Nim + mingw-w64 + a from-source BoringSSL build) specifically so I could actually run this rather than just eyeball it.

Unit and integration tests (tests/libp2p/protocols/test_noisehfs.nim, 8 tests, all passing at 2de86ac): the protocol id and name, MLKEM768 encapsulate/decapsulate round-trips, malformed-length rejection, wrong-key implicit rejection (FIPS 203 6.4 - this must not crash), key uniqueness, and a full two-node NoiseHFS handshake over a real TCP connection with peer identity verification, plus a peer-id-mismatch rejection case.

Cross-language interop, all four implementations, both directions. interop/noise-pq/interop_listen.nim and interop_dial.nim always exchange one encrypted greeting each way after the handshake and follow a stdout contract shared with the TypeScript, Python and Rust harnesses. On 2026-09-19 a neutral runner ran every ordered listener/dialer pairing of this branch, js-libp2p-noise PR #665 (TypeScript), py-libp2p PR #1310 (Python) and rust-libp2p PR #6481 by @royzah (Rust, with the harness from royzah/rust-libp2p PR #1), each implementation against itself included, three times each, with Nim at b3f203d: 48 runs, 48 passed.

listener \ dialer	JS	Python	Nim	Rust
JS	3/3	3/3	3/3	3/3
Python	3/3	3/3	3/3	3/3
Nim	3/3	3/3	3/3	3/3
Rust	3/3	3/3	3/3	3/3

A run passes only if both sides exit cleanly, each side reports the other's actual peer id, and each side decrypts the other's greeting, which exercises both split() cipher states. All runs were over loopback TCP on one Windows 11 machine. The harnesses start the handshake directly on TCP, so multistream-select negotiation is not covered. Results, versions.txt and all 96 logs: pq-noise-artifacts, run 20260919T223056Z, at Nim b3f203d, TypeScript 0c55599, Python c8d16e63 and Rust bdd417e. Negative controls (an old-name build that fails against every other implementation, and a fabricated identity that the cross-check catches): interop/negative-controls.

The unit tests ran at 2de86ac and the matrix at the current head b3f203d. 2de86ac is 2b8efb5 plus two documentation commits (bb75dcb, e3d2d14), a merge of master (48fcf892), and a one-line fix (2de86ac) that follows #3105's rename of Opt.withValue to ifValue, without which the branch does not compile after the merge. The commits since are 2c275cd, 824cce0 and b3f203d, which only update documentation, and 5a756f5, a second merge of master. Every NoiseHFS source, test, harness and benchmark file is byte-identical between 2de86ac and b3f203d: the only differences in the NoiseHFS area are NOISE_HFS_SPEC.md and interop/noise-pq/README.md.

A stale citation in the branch. NOISE_HFS_SPEC.md at b3f203d still cites the previous matrix run, 20260919T053615Z at Nim 824cce0. The run above supersedes it. That is a one-line documentation fix and I have left it rather than push another commit to a branch that is otherwise settled; say the word and I will fold it into the next push.

What changed in the newest run, and why nothing here changed for it. The Python implementation switched its ML-KEM-768 backend from the pure-Python kyber-py to a C-backed one through the cryptography package, so this is the first matrix run with that backend on the other side of the wire. Nothing in this branch changed, and 48 of 48 still pass, which is direct evidence that a peer's choice of KEM implementation is not observable in the wire format. That change is Python's alone; none of the Nim figures in this description are affected by it.

Earlier results, corrected. An earlier version of this description reported a live run against py-libp2p's scripts/interop_listen_mlkem768.py (2026-07-11, HANDSHAKE_OK), on the hyphenated name and /noise-mlkem768-hfs/0.1.0, and said Rust and JavaScript interop hadn't been run. That run covered the handshake only, with no transport messages exchanged, and predates the rename; the matrix above replaces it. The note on why the harnesses use Ed25519 identity keys rather than the crypto module's default ECDSA (at the time, py-libp2p's key-type deserializer didn't cover ECDSA) is still in interop/noise-pq/README.md, alongside the older runs, which are labelled there as a superseded historical record.

What's left

• Not wired into libp2p.nim's public re-exports - kept the surface area small for this first pass; happy to add that plus any other builder ergonomics reviewers want if the general approach looks right.
• Peer identity is still classical (Ed25519/etc via the existing signature scheme) - this only addresses the key exchange, matching the scope of the underlying research. ML-DSA identity keys would be a separate, later piece of work.
• Protocol id negotiation is not covered by the interop runs, which start the handshake directly on TCP.

Files

• libp2p/crypto/mlkem768.nim - raw ML-KEM-768 via BoringSSL
• libp2p/crypto/mlkem768layout.nim - BoringSSL MLKEM768_* key struct sizes, with a compile-time layout guard
• libp2p/protocols/secure/noise.nim - exports the primitives NoiseHFS reuses (no behaviour change)
• libp2p/protocols/secure/noisehfs.nim - the NoiseHFS connection encrypter
• libp2p/protocols/secure/NOISE_HFS_SPEC.md - wire format, design rationale, interop status
• libp2p/builders.nim - SecureProtocol.NoiseHFS / withNoiseHFS()
• tests/libp2p/protocols/test_noisehfs.nim - unit + integration tests
• benchmarks/bench_noisehfs.nim - classical XX vs NoiseHFS handshake and ML-KEM-768 benchmarks
• interop/noise-pq/ - dial/listen interop harnesses and their README

[AkshayaMani]

Cross-checked against the spec. Looks good. Left a couple of (non-blocking) suggestions inline.

[AkshayaMani]

Might be worth extending with a compile-time assert to guard against a future BoringSSL bump silently writing past the fixed-size buffers.

[AkshayaMani]

Suggestion:
sharedSecret is no longer needed. Binding as var and adding burnMem(sharedSecret) keeps defensive zeroization consistent (same pattern as burnMem(hs) below).

[AkshayaMani]

Same pattern as comment at line 112: encapRes.sharedSecret here would benefit from the same burnMem.

[paschal533]

Thanks for the review @AkshayaMani ! I addressed all three... struct size guard: added a _Static_assert against sizeof(struct MLKEM768_public_key)/private_key from the real BoringSSL header, so a future bump that changes the opaque layout fails the C build instead of silently writing past our buffers. Had to put it in its own file (mlkem768layout.nim) rather than inline... including <openssl/mlkem.h> in mlkem768.nim itself collided with the loosely-typed (byte*) importc declarations we already have there for the same BoringSSL functions, since gcc won't accept two different prototypes for the same symbol in one translation unit. burnMem on the decap shared secret: done, right after mixKey consumes it in the outbound handshake. burnMem on the encap shared secret: same, in the inbound handshake.
Verified all of this compiles and test_noisehfs.nim still passes (8/8).

[AkshayaMani]

LGTM on the fixes, thanks!

[paschal533]

Two updates: the interop coverage this PR was missing is now done, and I have benchmark numbers.

Interop

The README previously listed the JavaScript and Rust pairings as not yet run. Both pass now, on 2026-09-05:

Pair	Roles tested	Result
nim ↔ js-libp2p-noise (PR #665)	both — nim listening, and nim dialling	PASS
nim ↔ rust-libp2p (royzah/rust-libp2p PR #1)	nim dialling only	PASS

With the earlier py-libp2p test that puts all six pairwise combinations across the four implementations of this profile at a verified live handshake, with no protocol changes needed anywhere.

Two things worth calling out. In the direction where the JS side listens, the peer id we report as remotePeer is exactly the id the JS listener printed for itself — so the Ed25519 signature over the handshake hash verified and both sides derived the same h. And interop_dial now takes an opt-in --chat flag that reads one post-handshake message and replies. Completing the handshake only proves both sides agreed on the handshake hash and the KEM shared secret; it says nothing about whether the two cipher states came out of split() with the same orientation. A swapped cs1/cs2 still prints HANDSHAKE_OK and only fails on the first data frame. Since split() gives initiator and responder opposite states, the two directions exercise different keys, so one direction alone leaves one transport key untested. Default behaviour is unchanged, so the existing Python and Rust pairings still work as before.

The Rust pair is one-directional because rust-libp2p has a listener example but no dialer.

Benchmarks

New benchmarks/bench_noisehfs.nim. Windows 11 Pro x64, Nim 2.2.10, -d:release, medians over five runs, in-memory bridgedConnections(), Ed25519 identity keys:

Operation	ops/s	ms/op
generateKeyPair	9,615	0.104
encapsulate	9,434	0.106
decapsulate	9,009	0.111
KEM round-trip	3,115	0.321

Protocol	ms/handshake
Noise_XX_25519_ChaChaPoly_SHA256	2.838
Noise_XXhfs_25519+ML-KEM-768_ChaChaPoly_SHA256	3.245

The hybrid handshake costs about 14% more than the classical one, and the KEM is only ~9.9% of it. For comparison the same protocol costs +5.0x in the JavaScript implementation and +12.9x in the Python one, where the KEM is 48% and 63% of handshake time respectively.

The part that might interest you more than the PQC work

The reason the KEM is such a small slice here is not only that BoringSSL's ML-KEM is fast. It is that everything around it is comparatively slow. A 2.8 ms classical Noise XX handshake is a lot for compiled code, and it breaks down like this: X25519 and ChaCha20-Poly1305 come from BearSSL, which is deliberately compact rather than optimised; SHA-256 and HMAC come from nimcrypto in pure Nim; and Ed25519 is our own pure-Nim port of the SUPERCOP ref10 code, which is the reference implementation. ML-KEM-768 is the only primitive in the handshake that goes through BoringSSL.

So nim-libp2p currently pays more for the classical half of a post-quantum handshake than for the post-quantum half. That is a pre-existing cost, unrelated to this PR, but the benchmark makes it visible and it seems worth flagging. If the classical primitives were routed through BoringSSL too, both numbers would drop noticeably.

One methodology note in case anyone reruns this. I originally measured 500 classical handshakes and then 500 hybrid ones, and got overhead ratios ranging from 0.99x to 1.27x across nine runs — 0.99x meaning the hybrid handshake came out faster, which is impossible. Absolute latencies drift by ~0.6 ms between runs while the effect is ~0.35 ms, so the phase separation was putting the drift straight into the ratio. Interleaving the two protocols within each iteration makes the drift common-mode; the paired ratio then held at 1.134–1.140 across five runs. The harness does it the paired way.

[paschal533]

A short follow-up to the benchmark numbers I posted earlier. The Nim figures held up; the ones I posted for the other implementations did not, and the corrected comparison is more flattering to this PR than the original.

Nim, re-measured

Five serial passes on a quiet machine, medians:

	value
ML-KEM-768 round-trip (BoringSSL)	0.274 ms
classical Noise_XX	2.943 ms
Noise_XXhfs	3.356 ms
paired overhead	1.13x (range 1.117–1.141)
KEM share of handshake	~8%

Essentially unchanged from what I posted before (1.14x), which is what I would hope for, and the paired ratio held to 2% across passes even though absolute latencies on this machine varied by more than a factor of two between sessions. The pairing in benchmarks/bench_noisehfs.nim is doing its job.

Rust, for comparison

I finally ran rust-libp2p's benches/noise_hfs.rs, which turned out to have existed all along and never been executed. Criterion, 100 samples:

	Rust	Nim
KEM library	RustCrypto ml-kem	BoringSSL
classical Noise_XX	1.531 ms	2.943 ms
Noise_XXhfs	1.928 ms	3.356 ms
overhead	1.30x	1.13x

Two things worth drawing out of that.

Rust's classical handshake is 1.531 ms against our 2.943 ms — for the same protocol, same parameters. That is the BearSSL and pure-Nim classical stack showing up in the measurement, exactly as the earlier discussion of mlkem768.nim predicted it would: ML-KEM-768 is the only primitive in the handshake that reaches BoringSSL, while X25519 and ChaCha20-Poly1305 come from BearSSL, SHA-256 and HMAC from nimcrypto, and Ed25519 from our own pure-Nim ref10 port. This is a pre-existing property of nim-libp2p rather than anything this PR introduces, but the benchmark makes it legible.

And note the direction of the ratios. Rust has the faster classical stack and the higher overhead ratio — 1.30x against our 1.13x — despite an equally quick KEM. That is the arithmetic working as expected: a smaller denominator makes the same numerator look larger. I mention it because 1.13x is a flattering number and I would rather not have it quoted without that context. It is partly a consequence of our classical baseline being slower than it needs to be.

What changed elsewhere

The JavaScript and Python figures I quoted in my earlier comment were both wrong. Both benchmark harnesses were still measuring X-Wing months after that migration, and the JavaScript one was additionally comparing a native-backend classical handshake against a pure-JavaScript hybrid one, so it was attributing a whole change of crypto backend to the KEM. Corrected, the four implementations look like this:

	Python	JavaScript	Rust	Nim
Overhead	10.5x	1.47x	1.30x	1.13x

Three of four sit between 1.1x and 1.5x. Only Python is an order of magnitude out, and only because kyber-py does its lattice arithmetic in pure Python.

Nothing here needs action on this PR — the implementation and the interop results are unchanged. I am posting it because the numbers in my previous comment are now superseded and I would rather not leave incorrect figures standing in the thread.

[paschal533]

Pushed the protocol rename, the id bump and always-on bidirectional interop harnesses. This is a breaking change on the wire.

Rename. ProtocolXXHFSName is now Noise_XXhfs_25519+MLKEM768_ChaChaPoly_SHA256. Noise (revision 34, §8.2) allows only alphanumerics and / in algorithm names, so ML-KEM-768 was never a valid name (@royzah spotted it). The name is hashed into the handshake hash h, so peers on the old and new names can't complete a handshake. Message sizes are unchanged. The PR title still has the old name.

Id. NoiseHFSCodec is now /noise-mlkem768-hfs/0.2.0, so builds that can't interoperate don't advertise the same id. NOISE_HFS_SPEC.md is updated.

Interop, 48/48. interop/noise-pq/interop_listen and interop_dial now always exchange one encrypted greeting each way (--chat is gone) and follow a stdout contract shared with the TypeScript, Python and Rust harnesses. A neutral runner ran every ordered listener/dialer pairing of the four, Nim against itself included, three times each, on 2026-09-17 with Nim at 2b8efb5 (the two later commits on this branch, bb75dcb and e3d2d14, only update documentation). A run passes only if both sides exit cleanly, each reports the other's actual peer id, and each decrypts the other's greeting, which exercises both split() cipher states. Everything ran over loopback TCP on one machine, with no multistream-select. Results and logs.

Negative controls. A TypeScript build with only the old name fails against Nim in both roles, with the dialer failing tag authentication on message B (control A, re-run with the matrix). A fake peer id is caught by the identity cross-check (control B, from the earlier run 20260917T015709Z).

Corrections to my 5 September comment.

• "the Ed25519 signature over the handshake hash verified". The libp2p identity signature covers the Noise static key, not h. Agreement on h follows from the AEAD instead: h is the associated data, so the static key and payload only decrypt if both sides computed the same h. The interop README now says this.
• "The Rust pair is one-directional because rust-libp2p has a listener example but no dialer." That listener was ours, from test(noise): MLKEM768 HFS interop listener and dialer, protocol id tests and criterion benchmarks royzah/rust-libp2p#1, not part of feat(noise): hybrid post-quantum handshake (X25519 + ML-KEM-768) libp2p/rust-libp2p#6481, which ships no examples. [WIP] Do not merge! Chat and further improvements. #1 now has a dialer too, and the matrix covers Nim↔Rust in both roles. That 5 September Rust run also used a snow pinned at a June commit that still spelled the suite ML-KEM-768. royzah renamed it on 17 August 2026, and the rename was on the #6481 branch by 22 August at the latest, so that run was already behind his PR.
• The 2026-07-11 run against py-libp2p covered the handshake only and predates the rename. The matrix replaces it.

Benchmarks. 2026-09-17 re-run on the renamed suite: SUMMARY.md. For Nim: 1.19x. That's the ratio of each pass's hybrid median to its classical median (5 passes × 500 interleaved iterations), medianed over the passes, with a range of 1.16–1.21x. Absolute latencies from that session aren't comparable with earlier sessions (see the file).

[paschal533]

Two new matrix runs that bear on this branch, both at the current head f15881df rather than the b3f203d the description cites.

Regression, mechanism off: 48 of 48. Same 4x4x3 shape as the run in the description, re-run on 2026-09-25 against the current heads of all four implementations. Nim is unchanged since b3f203d in the NoiseHFS area, so this is a re-validation rather than new coverage, but it does mean the cited result now also holds at the head this PR points at: 20260925T123204Z.

Extension variant, 48 of 48, with Nim deliberately unbound. The TypeScript and Python branches now implement transcript-bound security protocol negotiation, a defence against the plaintext multistream-select downgrade. In that run only those two were bound; the Nim and Rust harnesses have no flag for it and ran unbound: 20260925T124411Z.

That is the useful part for this branch: it is direct evidence that Nim needs no changes to interoperate with peers that use the extension. The mechanism carries its protocol list in NoiseExtensions, so an implementation that does not know the fields ignores them and the handshake completes. Nothing here is asking for work on the Nim side.

If you do want it, the shape is small: one extension field pair, a canonical length-prefixed encoding of the offered protocol list, and a comparison of the negotiated protocol against what the two signed lists imply. Wire format and both variants are in TRANSCRIPT_BINDING_SPEC.md on the js branch, with cross-implementation vectors that pin the bytes. Happy to add the flag to interop/noise-pq myself if that is useful, or leave it.

On the stale citation I mentioned earlier: NOISE_HFS_SPEC.md still points at 20260919T053615Z, and there are now two newer runs. Still a one-line fix, and still yours to take or leave.