Skip to content

harness: the validate pass never runs under the harness #682

Closed

Description

@OmarAlJarrah

Problem

internal/harness compiles each spec through its compile seam (internal/harness/harness.go:96), which calls the OpenAPI compiler alone. engine.Run also runs pass.Validate, and the harness doesn't, so a spec whose lowered IR the pass rejects sweeps ok.

Concretely, #593 fails morphic compile with pass/discriminator-missing-variant and passes the harness.

Expected

An oracle, in the existing order, that runs pass.Validate on the compiled document and fails on any error the compiler didn't report itself.

Activity

  1. OmarAlJarrah commented on Oct 13, 2026

    @OmarAlJarrah
    MemberAuthor

    Now that #805 gives the engine an ordered pass list, the fix should compile through the engine rather than call pass.Validate beside the compiler. Every oracle then judges what morphic compile returns, including the output of any transform that joins the list, and a pass's error finding reaches the error-diagnostic oracle the way the compiler's do.

    engine.Run takes a path while Check holds bytes. Run already builds a compilers.Source from the file it reads, so the change is a source-taking entry point that Run delegates to and the harness's compile seam calls, rather than a temporary file per input. It is the single-source case of the entry point #74 plans.

  2. added
    priority:P4Refactors, tests, CI, hostile-input hardening, cosmetic diagnostics, other formats
    on Oct 19, 2026
  3. OmarAlJarrah commented on Oct 19, 2026

    @OmarAlJarrah
    MemberAuthor

    Still true at c387e25c. internal/harness also compiles with openapi.New() alone, so a second compiler gets none of the oracles either (#968).

    What the missing Validate run hides is small today. Of 1,839 compiled documents, only 3 produce any Validate diagnostic: 5 discriminator-missing-variant in all.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority:P4Refactors, tests, CI, hostile-input hardening, cosmetic diagnostics, other formatstype:testTests only

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions