Skip to content

irverify: run every walking check from one traversal of the document #962

Closed

Description

@OmarAlJarrah

Problem

irverify.Verify walks the whole document 17 times: 16 ir.WalkValues walks and one ir.ValueCycles. On the golden spec at c387e25c (the Cloudflare API schema, cloudflare/api-schemas openapi.json at 911ad78, compiled to 41,592 types), it makes:

  • 121.8 M visits;
  • 132.9 M allocations;
  • 17.1 GiB allocated;

and takes about 7.4 s, longer than the 7.1 s compile. The 17 walks cost 0.4 to 0.5 s each, so no single check dominates. Path-string building (fieldPath) is about half of the walk's CPU (#813). Verify runs in every harness sweep and corpus test, so this is test and CI time.

Direction

A proposal, measured on a throwaway prototype:

  • Each walking check becomes a handler over the nodes it cares about (kinds, visit, cycle, finish).
  • A 64-line dispatcher runs all of them from one traversal.
Before After
visits 121.84 M 7.29 M
allocations 132.93 M 9.50 M
allocated 17.1 GiB 1.22 GiB
time about 7.4 s about 1.0 s

Across the 20 largest corpus documents, allocations fell 13 to 15×. The output was identical, []Violation equal under cmp.Equal, on all 1,839 compiled documents and in 3,000 trials that planted random defects into 60 of them. The copied irverify suite passes. Dropping any one handler reddens it.

Two costs:

  • The change adds 74 code lines (about 5%).
  • Cycle detection needs a cycle callback on the ir walker, about 8 lines.

The 10 checks that loop over registries rather than walking stay as they are. pass.Validate's 3 walks could share the same traversal later; that is not measured.

Done when Verify makes one traversal, with a counted-work test pinning it.

Out of scope

Merging irverify into pass.Validate. Only 7 of Validate's 17 checks overlap, the saving is in the traversal rather than the package boundary, and the two report on different channels (our bugs versus spec problems).

Related: #813, #859

Priority: P4, rule 1 (a refactor that changes no output). Count ir.WalkValues calls during one irverify.Verify of golden.ir.json → 16, plus one ir.ValueCycles (spec @911ad78, morphic @c387e25c).

Activity

  1. added
    type:choreRefactor or cleanup, no behavior change
    performanceLatency, throughput, allocation, or footprint
    priority:P4Refactors, tests, CI, hostile-input hardening, cosmetic diagnostics, other formats
    on Oct 19, 2026
  2. OmarAlJarrah commented on Oct 26, 2026

    @OmarAlJarrah
    MemberAuthor

    Re-measured at 6c7bf102 on the golden spec (cloudflare/api-schemas openapi.json at 708fba78), compiled with morphic compile -o golden.ir.json, by a program that decodes that IR and times each call with runtime.MemStats deltas around it:

    call time allocations allocated findings
    irverify.Verify 8.4 to 8.6 s 134.9 M 16.9 GiB 0
    pass.Validate 1.75 to 1.83 s 24.8 M 3.2 GiB 0
    ir.DeclaredIDs 0.56 s 8.2 M 1.0 GiB 140,046 declarations
    • One walk of this document visits 7.41 M values: 1.13 M structs, 2.40 M strings, 1.25 M pointers, 1.09 M slices and 0.30 M maps. Verify walks it 17 times.
    • Each visit builds the value's path string, a concatenation per field and an fmt.Sprintf per index and map entry, whether or not the visitor reads it (ir: build WalkValues paths only when a visitor reads them #813). With no violation to report, Verify reads none of them.
    • The CPU profile of one Verify (go tool pprof -top) puts runtime.gcDrain at 35% of samples and runtime.concatstrings at 18%, cumulative.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    performanceLatency, throughput, allocation, or footprintpriority:P4Refactors, tests, CI, hostile-input hardening, cosmetic diagnostics, other formatstype:choreRefactor or cleanup, no behavior change

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions