docs/emitter-design: say what an SDK emitter does with each kind of Unmodeled entry #951

Problem

docs/emitter-design.md gives no rule for Unmodeled entries other than vendor extensions. §3.4 says Reason and Provenance are what a policy layer selects on. §10 says SDK and docs emitters ignore validation_only entries. Nothing covers no_ir_home, degraded_lowering or out_of_scope, so a refiner that meets one has no rule for whether to ignore it, warn or refuse.

These entries reach most operations of the golden spec. The golden spec here is the Cloudflare API schema (cloudflare/api-schemas openapi.json at 911ad78), compiled with go run ./cmd/morphic compile openapi.json -o golden.ir.json at c387e25c. That command exits 1 on four unrelated errors and still writes the document.

The golden spec holds 25,983 entries:

• 94.45% are vendor extensions;
• 1,443 are not: 734 no_ir_home, 699 degraded_lowering, 10 validation_only and 0 out_of_scope.

The non-vendor entries sit directly on only 28 operations (0.74%) and 444 models (2.43%). But they sit on 1,323 types, so through the types they use, 2,581 of the 3,761 operations (68.6%) and 24.0% of models reach one. The reach is concentrated:

• the 25 most-reached types reach 27% of operations, and the 100 most-reached reach 48%;
• the most-reached are the response-envelope schemas each product section of the spec repeats: an openapi:type kept on the failure envelope's result, and an openapi:uniqueItems kept on the messages items.

Across 1,534 compiled corpus specs that have operations, 410 have 5% or more of their operations reaching a non-vendor entry.

Two keys make up much of the volume: readOnly (360) and default (339). ir-design §12 keeps a declaration-site copy of each verbatim after pushing the fact down to its use sites. For an emitter those copies restate a fact it already has, and no rule tells it so.

import json, sys
d = json.load(open(sys.argv[1])); types = d["types"]
NV = {"no_ir_home", "degraded_lowering", "validation_only", "out_of_scope"}
def walk(x, refs, hits):
    if isinstance(x, dict):
        for k, v in x.items():
            if k == "unmodeled" and isinstance(v, dict):
                hits[0] += sum(e.get("reason") in NV for e in v.values()); continue
            walk(v, refs, hits)
    elif isinstance(x, list):
        for v in x: walk(v, refs, hits)
    elif isinstance(x, str) and x in types: refs.add(x)
edges, carries = {}, {}
for tid, t in types.items():
    refs, hits = set(), [0]; walk(t, refs, hits); refs.discard(tid)
    edges[tid], carries[tid] = refs, hits[0] > 0
def reaches(roots):
    seen, stack = set(), list(roots)
    while stack:
        x = stack.pop()
        if x in seen: continue
        seen.add(x)
        if carries[x]: return True
        stack.extend(edges[x])
    return False
def ops(groups):
    for g in groups:
        yield from g.get("operations") or []; yield from ops(g.get("groups") or [])
n = hit = 0
for s in d["services"]:
    for o in ops(s.get("groups") or []):
        refs, hits = set(), [0]; walk(o, refs, hits); n += 1
        hit += bool(hits[0]) or reaches(refs)
print(hit, "of", n)

Direction

A proposal, in §3.4 or §4: a table of what an SDK emitter does with each reason.

• Name the keys that restate a fact lowered elsewhere (the pushed-down readOnly and default copies), so an emitter can skip them without reading their values. Any rule that reads a key's name depends on Unmodeled key names are an unversioned contract that consumers are currently forced to read #429.
• Done when the table is in the design with the diagnostic code each row emits, if any, and §13 has a test in which an entry of each reason reaches an emitter and the emitter does what the table says.

Out of scope

Promoting any key to a typed field. Each gap that should become a field is its own issue (#285 and its siblings).

Related: #429, #712, #722, #285

Priority: P1, rule 5 (direct impact: an emitter-plan contract decision on the golden spec's output). python3 reach.py golden.ir.json (above) → 2581 of 3761 operations (spec @911ad78, morphic @c387e25c).