Conversation
Introduces a DebugInfoModule trait (src/traits/mod.rs) unifying the existing Windows PDB backend and a new DwarfModule (src/dwarf/mod.rs), so callers can resolve symbols/lines for a loaded module into the same SymbolInfo/LineInfo interval-tree shape regardless of whether its debug info came from a PDB or from DWARF embedded in an ELF sidecar (as produced by the EDK2 GCC5 toolchain for UEFI/SMM BIOS modules). - traits::DebugInfoModule: fn intervals(&mut self, &SourceCache) -> Result<Vec<Element<u64, SymbolInfo>>>, implemented for Module/ProcessModule by delegating to their existing inherent intervals() methods (no behavior change), and for the new DwarfModule. - dwarf::DwarfModule<'data>: takes an already-parsed object::File plus a runtime base address; hand-rolls a gimli DIE/line-program walk (no addr2line -- its point-lookup-oriented API doesn't obviously support the proactive "enumerate every function/line" use case this needs, which is left as an open question) to find every DW_TAG_subprogram and its lines, keyed by [base + link_addr, base + link_addr + size). Runtime address translation is a flat `base + addr` addition, same as the PDB backend's `base + rva`. - SourceCache::lookup_dwarf: mirrors lookup_pdb, trying a DWARF5 DW_LNCT_MD5 file checksum first and falling back to the existing format-agnostic lookup_file_name_components. - Added gimli/object dependencies (object chosen over stretching goblin, which is only used elsewhere for PE-specific parsing; object is gimli's standard companion crate for ELF). Out of scope for this milestone (by design): wiring up a caller (Simics-side UEFI/SMM module discovery and the SMM trigger question), the test fixture, and performance at scale. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds a synthetic ELF+DWARF fixture (tests/fixtures/dwarf/X509CertVerify.c, compiled with gcc/ld to X509CertVerify.debug with .text linked at a non-zero VMA 0x240, mirroring the real EDK2 GCC5 per-module .debug convention) and an integration test (tests/dwarf_fixture.rs) that exercises DwarfModule::intervals() end-to-end against it: parses the ELF with `object`, resolves DWARF via DwarfModule, and asserts the resolved X509VerifyCert SymbolInfo's address range (base + link-time addr), size, and per-line LineInfo against ground truth independently confirmed with objdump/nm/readelf. Since tests/ integration tests are a separate crate and this crate had no public API at all (everything pub(crate)), widens exactly `dwarf` and `source_cov` to `pub` and re-exports SymbolInfo/LineInfo/ DebugInfoModule from tsffs::dwarf, instead of widening `os` (Windows kernel/PDB internals) or `traits` (also holds the unrelated TracerDisassembler trait). Full `cargo test`/`cargo build` still fails at the link step on this Windows dev machine (link.exe LNK1107, linking directly against libsimics-common.dll) regardless of Simics package version (reproduces on 7.84.0 and 7.70.0) and independent of this change (reproduces with a trivial assert_eq! test too) -- a pre-existing, environmental MSVC/Simics-packaging issue also flagged by the prior commit's cargo build --lib failure. `cargo check --tests` and `cargo clippy --tests --no-deps` are green and are the strongest signal available on this machine. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
get_object("tsffs") was called unconditionally (guarded only with
`if let Ok(o) = ...`) purely to emit a debug log line. Calling any
SIM_* API entry point with no Simics kernel initialized hard-aborts
the process from inside libsimics-common.dll itself, before the FFI
call can return Err, so the guard never actually helped. This broke
the dwarf_fixture integration test, which is specifically designed to
validate the DWARF parsing chain offline with no live Simics session.
Remove the non-essential debug log call so SourceCache::new works
offline as intended. Also updates the test's stale doc comment, which
claimed the crate could not link/build at all on Windows (true only
for the default MSVC host target; building with the GNU host target
per CI's build_windows job works, and now the test passes end to end).
Verified with:
cargo test --target x86_64-pc-windows-gnu --test dwarf_fixture
-> dwarf_module_intervals_resolves_synthetic_smm_handler ... ok
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
… 1-2) Simics has no native C interface for UEFI module discovery (no osa_target_info, confirmed against Simics 6/7 headers); the only mechanism is the uefi_fw_tracker component's list-modules CLI command via run_command(), returning a dynamic AttrValue/AttrValueType tagged union. This adds the two pieces of that milestone testable fully offline: - src/uefi/mod.rs: parse_module_list() parses the assumed list-modules AttrValueType shape (documented in the module doc comment, since no live Simics session is available here to confirm it) into (name, base, size, embedded_path) tuples, and UefiOsInfo::resolve() resolves each module's real local debug-info path against a build-root directory via longest- matching-path-suffix, falling back to bare-stem search and "fail open" (log + take the first candidate) on unresolvable ambiguity. UefiOsInfo holds a flat Vec (unlike WindowsOsInfo's per-CPU HashMap), matching UEFI/SMM's single flat address space. - src/util/path_suffix_index.rs: the shared "path-suffix index" primitive factored out of SourceCache's prefix_lookup/lookup_file_name_components, so the UEFI resolver doesn't pay for SourceCache::new's content-hashing on .debug/.efi binaries. Adds an "unambiguous" lookup variant (only returns a match when exactly one local path shares the longest matching suffix) needed by the resolver's disambiguation logic; SourceCache keeps its original best-effort (never-ambiguous) lookup behavior unchanged. - tests/uefi_module_discovery_fixture.rs: offline integration test (a separate cargo target, since [lib] test = false) against a hand-built AttrValueType fixture modeling 7 modules, including the 3 real observed duplicate-name cases (AcpiVTD.efi, MicrocodeUtilityDxe.efi, SataController.efi) each appearing twice with distinct embedded paths and base addresses. Confirms parsing, confirms suffix-matching resolves each duplicate pair to its correct distinct local file (via fabricated PkgA/PkgB local directory fixtures), and confirms the bare-stem fallback + WARN-level ambiguity logging (captured via a tracing subscriber) fires when suffix-matching can't disambiguate. AttrValueType (not AttrValue) is used throughout because constructing an owned AttrValue::List/Dict allocates through real SIM_alloc_attr_list/ dict FFI calls, which hard-abort with no live Simics session -- the same class of bug already found and fixed once in SourceCache::new (see f31492a on the sibling DWARF branch). AttrValueType's variants are plain, FFI-free Rust construction, safe for offline fixtures. Per the milestone's explicit scope, this does not wire into src/haps/mod.rs/HARNESS_START, add a self.uefi attribute, touch the OS enum, or call run_command for real anywhere. Verified with: cargo test --target x86_64-pc-windows-gnu --test uefi_module_discovery_fixture -> 3 passed; 0 failed Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…-M2 M2) Validated the assumed AttrValueType shape of uefi_fw_tracker list-modules command against a real, live Simics session (a live checkpoint, 2026-09-16) instead of the prior guess. The assumption was wrong in every particular: - Real shape is a positional List of List rows (Module, Loaded Address, Size, Adjusted Address, Adjusted Size), not a List of Dict rows keyed by name/base/size. - The Module column is only ever the bare basename (e.g. DxeCore.efi), confirmed both by the live capture and by cross-referencing every installed Simics-Base version own list-modules implementation (simmod/uefi_fw_tracker/module_load.py). list-modules never returns a full embedded build-machine path. - A real duplicate-name case (BootScriptExecutorDxe.efi, two entries, two addresses, no other distinguishing info) confirms path-suffix disambiguation can never fire for list-modules-sourced input; fail-open bare-stem fallback is the expected outcome for duplicates, not an edge case. Rewrote parse_module_list/parse_module_row for the confirmed positional-list shape, updated the module doc comment to state the shape as confirmed (with the live evidence), and rebuilt the offline test fixtures around the real captured data (including the real duplicate-name and "<unknown>" cases). UefiOsInfo::resolve generic path-suffix disambiguation is now tested directly against hand-built full-path tuples, since real list-modules output never supplies one itself. cargo test passes for tests/uefi_module_discovery_fixture.rs (3/3) on a native Linux build. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A follow-up live investigation in a test session (real Simics 6.0.189 session,
real checkpoint past DXE dispatch, 68 real loaded UEFI modules) found a
richer, confirmed-real data source than list-modules: the uefi_fw_tracker
component's underlying C object's `maps` attribute, reached via the same
run_command FFI path already used elsewhere (e.g.
`tracker_obj->maps`). Unlike list-modules (which
calls basename() on the underlying data before returning it, so it only
ever exposes a bare filename), tracker_obj->maps returns 7-element rows
that carry the full embedded build-machine path:
[loaded_address, loaded_size, <bool>, adjusted_address, adjusted_size,
<bool>, full_path_string].
- Rewrite parse_module_list/parse_module_row for the 7-element positional
row shape instead of the previously confirmed 5-element list-modules
shape. A row's "name" is now derived from the full path via
Path::file_name when present, or the existing UNKNOWN_MODULE_NAME
("<unknown>") placeholder for genuinely pathless rows.
- Make path-suffix disambiguation (PathSuffixIndex) the primary
resolution path in UefiOsInfo::resolve, since a full embedded path is
now the common case rather than a rare bonus that real list-modules
output could never actually supply. Bare-stem search is now purely a
fallback for when suffix-matching finds nothing; a genuinely pathless
row fails explicitly instead of guessing.
- Rebuild the fixture tests around the real 7-element shape: the real
BootScriptExecutorDxe.efi duplicate (identical embedded path for both
instances) now resolves cleanly with no ambiguity warning, since it
turns out not to be genuinely ambiguous once the real path is
available; a fabricated different-path AcpiVTD.efi duplicate proves
suffix disambiguation still works when real ambiguity exists; and a
pathless row confirms graceful naming/fallback (a clean Err, not a
panic) rather than a crash or silent misparse.
cargo test --target x86_64-pc-windows-gnu --test uefi_module_discovery_fixture
passes 5/5 on this machine.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Validated the 7-element parser against a real, full 68-row tracker_obj->maps capture in a live test session (Simics 6.0.189, same checkpoint used to confirm the row shape). 67 of 68 rows matched the implementation's assumptions exactly (unsigned addresses, positive ints, real full paths). The one genuinely pathless row (a real unresolved/unknown module) is Python None, which simics::AttrValueType::from(AttrValue) converts to AttrValueType::Nil via its is_nil() check -- not AttrValueType::String(String::new()) as the offline fixture had assumed. The old list_get_string only matched String and hard-errored on Nil, so parse_module_list would fail on real data despite passing all offline fixture tests. list_get_string is now list_get_string_or_nil, returning Option<String> and accepting both Nil and (defensively) an empty String as no path. Updated the fixture test's pathless row to build AttrValueType::Nil instead of an empty string, matching the confirmed real shape. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
# Conflicts: # src/source_cov/mod.rs
…S_START Integrates UCOV-M1 (DWARF/ELF debug info parsing) with UCOV-M2 (UEFI/SMM module discovery via tracker_obj->maps) into a single feature: at HARNESS_START, query the configured UEFI tracker object, resolve each discovered module's local build path, derive its EDK2 GCC5 .debug ELF sidecar (same directory, extension swapped from .efi), parse its DWARF info, and merge the resulting symbols into the same per-processor symbol lookup tree Windows uses, gated by new `uefi`, `uefi_tracker_object`, and `uefi_debug_info_directory` attributes (mirroring `windows`/`debuginfo_download_directory`). Unlike Windows's CR3-write-triggered refresh, UEFI/SMM has no per-process address-space switch to key off, so this collects once at HARNESS_START (all tracked modules are loaded by then) rather than on a recurring trigger.
A live fuzzing validation run surfaced a case not covered by the offline fixtures: when HARNESS_START fires early in DXE dispatch (before all modules have loaded), tracker_obj->maps can return a list containing AttrValueType::Invalid entries -- reserved but not-yet- populated slots in the tracker's underlying storage -- alongside well-formed 7-element rows for the modules loaded so far. The parser treated any non-List top-level entry as fatal, aborting the whole batch. Skip Invalid entries instead, distinct from a genuinely pathless module row (a well-formed 7-element list with a Nil path).
…en it does save_symbolic_coverage's directory-creation check was inverted (create_dir_all only ran when the directory already existed), so Records::to_html failed with "Node not found" the first time a real coverage run tried to write output to a directory that had never been created. Surfaced by a live UEFI source-coverage validation run, the first time this environment exercised symbolic_coverage=true with real coverage data end to end.
…n error Records::to_html seeds its output-tree graph entirely from the records themselves, so when zero source lines were ever recorded (e.g. a short run whose covered code never lands inside a symbolicated module -- observed for real with UEFI/SMM coverage, whose HARNESS_START can fire before every module is loaded), it never creates a graph node for the output directory and its root-node lookup fails with NodeNotFound on that exact path. That's an empty-coverage outcome, not a real error -- report and skip it instead of propagating it through the simulation-stopped HAP callback, which panics via .expect().
Live validation in a test session (a real boot with the compiled-in
harness) exposed a genuine bug: querying maps via the
CLI string-command path (run_command("tracker_obj->maps")) returned a
stale/near-empty result (3 elements, all Invalid) at a point in boot
where a direct attribute read of the exact same object already
returned the real, fully-populated list (67 real rows, confirmed via
a live Python probe at the same virtual time). The data was always
there -- the CLI string-command round-trip was the bug. Switch to
get_object + get_attribute (SIM_get_attribute), a direct FFI read with
no CLI round-trip.
Investigating why source coverage stays empty even after switching tracker_obj->maps reads from run_command to get_attribute: need to see the real row counts as observed by collect_symbols itself, since an unrelated "AttrValue(Sim_Val_List, 3, ...)" console line (not emitted by this code) turned out to be a red herring -- it didn't change at all after the get_attribute switch, so it isn't actually reporting this function's data.
…ain .into() Root cause of the persistently-empty UEFI source coverage: plain AttrValueType::from(AttrValue) (equivalently `.into()`) recurses into nested lists via AttrValue::as_list::<AttrValueType>, which requires every element of a list to share the same private_kind before converting any of them -- despite the crate's own doc comment suggesting heterogeneous lists are supported via a *different* method (as_heterogeneous_list). The outer `maps` list is homogeneous (every row is itself a List) so converting it this way works, but each row is [Unsigned, Unsigned, Bool, Unsigned, Unsigned, Bool, String] -- genuinely heterogeneous -- so plain `.into()` silently converted every real row to AttrValueType::Invalid. Confirmed in a live test session: a real, fully-populated 67-row maps parsed as 0 module rows, no errors, no warnings, just silently wrong -- and completely unrelated to the tracker-timing/run_command issues investigated (and fixed) earlier. Now: extract each row as a raw AttrValue first (as_list::<AttrValue>, an identity conversion -- fine, since the outer list genuinely is homogeneous), then convert each row with as_heterogeneous_list, which has no such check.
… batch UefiOsInfo::resolve propagated the first per-module resolution error via `?`, aborting the entire batch. Confirmed in a live test session: a real tracker_obj->maps capture always has at least one genuinely pathless "<unknown>" module (mixed in with dozens of resolvable ones), so that one module was discarding source coverage for every other resolvable module in the same capture -- the exact opposite of "fail gracefully," just failing the whole batch loudly instead of one entry quietly. Now logs a warning and skips just that module, keeping the rest. Updates the existing offline test's asserted semantics (resolving a pathless-only batch is now Ok with zero results, not Err) and adds a new mixed-batch test guarding the actual real-world case: resolvable and unresolvable modules together in one capture.
…ry requirement Removes the TSFFS_DIAG2 logging added to isolate the previous three fixes (get_attribute, as_heterogeneous_list, per-module resolve skip). With all three applied plus debuginfo_source_directory pointed at the real local EDK2 source tree, end-to-end UEFI source coverage is confirmed working in a live test session: a real fuzzing campaign against the compiled-in harness produces a real, non-empty HTML coverage report (e.g. MdePkg/Library/StackCheckLibNull/StackCheckLibNullGcc.c.html) from 66 resolved real UEFI modules and 29 collected DWARF interval elements. Documents on the `uefi` attribute that debuginfo_source_directory must also be set for source *lines* to resolve -- symbols resolve independently of it, but every one has zero lines without it, since DWARF line entries resolve against debuginfo_source_directory, not uefi_debug_info_directory (which only locates each module's own .debug file, not its original source).
…cation Real EDK2 GCC5 debug info universally emits the abstract/concrete instance DWARF split: every concrete DW_TAG_subprogram DIE (100% of 114,310 checked across 914 real .debug files from an internal validation corpus) has no direct DW_AT_name, only DW_AT_abstract_origin pointing at the DIE that carries the real name. The abstract origin is frequently in a different compilation unit (DW_FORM_ref_addr/DebugInfoRef, not a same-unit UnitRef), since GCC emits each abstract instance once and shares it across every CU that inlines/instantiates it. The previous code skipped any DIE without a direct DW_AT_name, silently dropping every real function and leaving symbolic coverage permanently empty with no error anywhere downstream. intervals() now pre-parses all units up front and resolves names across all of them. Validated against all 914 real .debug files from an internal validation run: 0/914 -> 914/914 producing symbols, 114,310 total (matches the exact concrete-DIE count from live investigation), names spot-checked correct. tests/dwarf_fixture.rs (direct DW_AT_name, no indirection) still passes.
Wenzel
force-pushed
the
feat/uefi-source-coverage
branch
3 times, most recently
from
September 21, 2026 14:38
d716869 to
7ec6f40
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds UEFI/SMM source-level (line + function) code coverage to TSFFS, built from two backends wired together at
HARNESS_START:src/dwarf/mod.rs): parses.debugsidecar files to resolve addresses to source file/line/function info, feeding the existingsource_covHTML/lcov reporting.src/uefi/mod.rs): reads the Simics UEFI firmware tracker'stracker_obj->mapsattribute to enumerate loaded PE/COFF modules and their debug info paths during a live session.Real bugs found and fixed while validating against real-world EDK2 firmware builds (not just synthetic fixtures):
tracker_obj->mapsreads neededget_attribute, notrun_command(stale/wrong data via the latter).AttrValue::as_list::<T>()'s homogeneity check breaks on the tracker's heterogeneous 7-element rows; switched toas_heterogeneous_list().parse_module_listdidn't filterInvalid/Nil-path top-level tracker entries.save_symbolic_coveragehad an inverted directory-existence check and crashed (instead of no-op) onlcov2::error::Error::NodeNotFoundwhen zero coverage was recorded.DwarfModule::intervals()silently produced zero symbols against real EDK2 GCC5 DWARF, becauseunit_symbolsrequired a directDW_AT_nameon each subprogram DIE. Real EDK2 GCC5 output universally uses the standard abstract-instance/concrete-instance split instead — the name lives on a separate DIE referenced viaDW_AT_abstract_origin(frequently cross-compilation-unit). Fixed by pre-parsing all units up front and resolving names across units (falling back toDW_AT_specification), handling both same-unit (UnitRef) and cross-unit (DebugInfoRef) reference forms.Validation
.debugfiles: 0/914 files produced any symbol before the fix; 914/914 do after (114,310 symbols total, names spot-checked correct:CopyMem,AllocateZeroPool,GetHobList,_ModuleEntryPoint, etc.). The existing synthetic fixture (tests/dwarf_fixture.rs) still passes unmodified.Prior attempts on other internal test platforms had produced either a single irrelevant page with 0/0 hits, or zero files at all — this is the first time this feature has produced a working, real coverage report end-to-end.
Test plan
tests/dwarf_fixture.rs(synthetic DWARF fixture) passes unmodifiedtests/uefi_module_discovery_fixture.rscovers module discovery/resolution edge cases (Invalid/Nil entries, heterogeneous rows, per-module skip-on-failure).debugfiles (0/914 → 914/914 producing symbols)🤖 Generated with Claude Code