refactor(labels): read symbols through the demangle package

[r0ny123]

What this changes

ItaniumDemangler, MsvcDemangler, MachoDemangler and rust_demangler keep every public function the rest of smda and the tests import, and become thin wrappers over the demangle package: one pure-Python dependency in place of the pycxxfilt build, the vendored 1,386-line MSVC demangler, the vendored Rust demangler and the swift demangle subprocess. Swift names are now read in-process, so a Linux host without a Swift toolchain labels them too. primeSwiftSymbols stays as a no-op for callers. NOTICE drops the section for the demangler behaviour that no longer ships and keeps a trimmed Team bi0s section for the Rust entry point and exception classes that still do.

Under src/: +85 / -2,514 lines.

Measured on the bundled fixtures

All 34 test binaries, the 12 Mach-O images and both PDBs, through the PE, ELF, Mach-O, PDB and Rust providers and LanguageAnalyzer's C++ checks: 8,793 demangler calls, 6,705 distinct names.

names	identical	changed
Itanium (PE/ELF), 2,595	2,595	0
Rust, 2,228	2,228	0
MSVC (PE/PDB), 7	7	0
msvc_reference_corpus.txt, 609	609	0
Mach-O, 3,963	3,767	196 (all Swift, previously left mangled without a toolchain)

Every C++ and Rust answer agrees with llvm-cxxfilt 18; every MSVC answer with llvm-undname 18, except that malformed names it reads past are refused (below). Of the 7,937 labels the fixtures store, 32 change, all Swift (27 in JokerSpy, 5 in HLoader), e.g. _$s7HLoader4mainyyF -> HLoader.main() -> (). is_itanium_cpp_symbol / is_msvc_cpp_symbol answer the same for every name. PE-decorated C names (_name@8, __imp_..., ordinals) stay unchanged.

Labelling the fixtures end to end: 381 ms -> 297 ms cold (median of 7). Per name, C++ is slower than the native build, Rust faster.

Label changes outside the fixtures

None of these names occurs in the bundled fixtures; each is a test expectation that changed.

Rust follows rustc-demangle:

• _RNvC3foo3bar.llvm.1234567890abcdef: foo::bar -> foo::bar.llvm.1234567890abcdef (rustc-demangle drops only an uppercase-hex .llvm. suffix)
• _RIC1aKh_E: raised UnableTov0Demangle -> a::<0>
• _ZN3$u$E: refused -> $u$
• _ZN17h0000000000000000E: "" -> raises UnableToLegacyDemangle. It is no longer Rust evidence, so demangle_itanium_symbol / demangle_macho_symbol leave a hash-only legacy name mangled rather than reading it as C++ h0000000000000000.

MSVC: demangle_msvc_symbol refuses malformed names before calling the library, which follows llvm-undname in reading past them: a string literal whose byte count disagrees with its recorded length (MSVC's 32-byte truncation allowed), a non-hexadecimal unnamed-namespace discriminator, a ? scope fragment that is not ?A, ?$, ?? or a numbered local scope, and a conversion operator with parameters. Of the nine names the vendored demangler refused and llvm-undname reads, the seven malformed ones are refused again; void f(...) (?f@@YAXZZ) and operator co_await (??__L_deg@@YAHO@Z) are well-formed and are read. Over the 897 MSVC names in the tests and msvc_reference_corpus.txt, nothing the vendored demangler read is refused, and no fixture name reaches the new checks.

Tests

Full suite: 2,211 passed, 1 skipped. Well-formed shapes the vendored demangler refused are in REFERENCE_READS with llvm-undname's spelling (variadic, co_await, ?a..?z escapes, a wide literal, a truncated literal); the malformed ones are in MALFORMED and must come back unchanged. A hash-only legacy Rust name is checked on the Itanium and Mach-O paths.

Dependency

demangle==0.5.0, pinned exactly: every label comes from it, so a change in its output arrives only as a reviewed bump here.

The pin moved to demangle==0.5.1, which loads no plugin unless asked, in #383.

[danielplohmann]

Thanks for this. Getting rid of the pycxxfilt build and the swift demangle subprocess is a real improvement, and labels for 431 mangled names from the test files came out the same as on master, apart from the cases below. A few things before it can go in.

Must fix

1. NOTICE drops the Team bi0s rust_demangler (MIT) section, but src/smda/common/labelprovider/rust_demangler/rust.py and main.py still ship and are derived from it (same RustDemangler / determine_type structure, same TypeNotFoundError message). The UnableTo*Demangle classes kept in rust_legacy.py and rust_v0.py come from there too. Please keep a trimmed bi0s section. Dropping the Ghidra section is fine.

Please disclose or reconsider

2. The description and CHANGELOG say Rust labels are unchanged, but these expectations changed in tests/testRustSymbolProvider.py / tests/testPdbSymbolProvider.py:

   • _RNvC3foo3bar.llvm.1234567890abcdef: foo::bar → foo::bar.llvm.1234567890abcdef
   • _RIC1aKh_E: raised UnableTov0Demangle → a::<0>
   • _ZN3$u$E: refused → $u$
   • _ZN17h0000000000000000E: "" → raises UnableToLegacyDemangle

   Most of these follow rustc-demangle and are fine, but please list them in the CHANGELOG.

3. A side effect of the last one: is_rust_language_evidence is now False for a hash-only name, so demangle_itanium_symbol / demangle_macho_symbol label _ZN17h0000000000000000E as C++ h0000000000000000. Before, it stayed mangled. Rare, but it is a new wrong label. Can the Itanium path leave hash-only names alone?

4. Several of the nine MSVC names moved from DECLINED to REFERENCE_READS are not well-formed: ??_C@_05ABCDEFGH@hi?$AA@ (the length says 5, the literal is shorter) and ?x@?Q@@3HA (invalid scope, read as int ?Q::x). The old demangler refused anything it did not fully understand, and I would like to keep that for malformed input. Labels feed into MCRIT, so a confident wrong name costs more than a mangled one.

The dependency

5. demangle becomes the one place every smda and MCRIT label comes from, so please pin it exactly: demangle==0.5.0. With the floor-only pin, any new release would change labels on new installs without an smda release, and stored labels would then drift between installs. With an exact pin, every change in demangling output arrives as a deliberate bump in an smda PR that we review like any other code change.
6. core/registry.py loads plugins from the demangle.languages entry-point group, and a plugin can replace a built-in scheme such as itanium. For smda I would rather that did not happen implicitly. Is there a way to use only the built-ins, or could the library make plugin loading opt-in?

Small

7. The description still says "Draft until 0.5 is on PyPI". 0.5.0 has been out since 2026-10-07.

[r0ny123]

Thanks for going through it this carefully.

1. NOTICE has a trimmed Team bi0s section again, naming what still ships from it: the RustDemangler entry point and its dispatch, and the three exception classes. The README credit line mentions it too.
2. The four Rust changes are listed in the CHANGELOG and the description.
3. demangle_itanium_symbol now leaves a legacy name that is only a hash (_ZN17h<16 hex>E, with or without the Mach-O underscore) mangled, so it is no longer labelled h0000000000000000. Test added for both paths.
4. Agreed. demangle_msvc_symbol now refuses malformed names before calling the library: a literal whose byte count disagrees with its recorded length (MSVC's 32-byte truncation allowed), a non-hex unnamed-namespace discriminator, a ? scope fragment that is not ?A, ?$, ?? or a numbered local scope, and a conversion operator with parameters. Seven of the nine are refused again. ?f@@YAXZZ (void f(...)) and ??__L_deg@@YAHO@Z (operator co_await) are well-formed, so they stay read. Over the 897 MSVC names in the tests and the reference corpus, nothing the old demangler read is now refused, and no fixture name reaches the new checks.
5. Pinned demangle==0.5.0, with a comment saying why.
6. Not in 0.5.0: the built-ins register first and the demangle.languages entry points can then replace one. I'll make plugin loading opt-in in demangle, so only built-ins answer unless a caller asks for plugins, and move the pin here to that release.
7. Description updated.

[r0ny123]

Point 6: demangle 0.5.1 is on PyPI and only reads demangle.languages entry points when a caller calls load_plugins(), which smda never does. This PR merged with the 0.5.0 pin, so the bump is in #383, with a test showing that a fake plugin advertising itanium, msvc, rust and swift no longer replaces the built-ins. Labels are unchanged; the schemes are the same in both releases.