Repository navigation
Method-level marks: @experimental is name-keyed, which is unusable where the surface is the dispatch table #2
Description
The gap
A mark is keyed by a name, so @experimental paints every method behind that name at once.
For a package whose public surface is its dispatch table, that granularity is unusable.
Measured on QAtlas.jl, 2026-09-01:
QAtlas.fetch 570 methods (all defined by QAtlas / AbstractQAtlas)
audit(QAtlas) public=310 documented=239 foreign=71 unaccounted=0
570 : 1. Marking fetch says nothing useful about any particular oracle.
The second line is the worse one. The audit reports no findings for QAtlas while never
looking at a single one of those 570 methods. For an oracle library the surface that matters is
"which model × quantity × boundary condition returns a value you can trust", and that lives
entirely in method space. A green that is computed over a surface the package does not actually
expose is the failure mode this package exists to remove, so having it here is not acceptable
long-term.
It works — prototype, not speculation
Three things needed checking and all three hold (Julia 1.12.2):
marks survived precompilation: 2
oracle(::User.B) @ User .../User.jl:6 -> this dispatch path only
oracle(x::T) where T<:Integer @ User:7 -> the where-form too
which(oracle, User.A) is marked: false <- the sibling method is not painted
which(oracle, User.B) is marked: true
which(oracle, Int64) is marked: true
Methodobjects do survive into the consumer's precompile cache image, the same way the
current name-keyedVector{Mark}does.- The
where-form is capturable. which(f, Tuple{...})gives an exact per-path answer, and does not leak onto neighbours.
Capture is by (file, line) taken from the definition expression's own LineNumberNode, not
from __source__ — otherwise the continuation style used throughout the README
(@experimental "…" \ newline function f(x)) points at the wrong line. Two methods on one
physical line must error, not be guessed at, consistent with how the macro already refuses
definitions it cannot read a name out of.
Sketch
@experimental "the r,s branch is provisional" \
function fetch(m::MinimalModel, ::ConformalWeights; r::Integer, s::Integer, kwargs...)
...
endisexperimental(which(fetch, Tuple{MinimalModel, ConformalWeights})) # exact, per path
mark(which(fetch, Tuple{MinimalModel, ConformalWeights})).reasonThat is the call a downstream test wants before trusting a reference value.
Open questions — these are the reason this is not a one-afternoon change
-
auditdoes not extend naturally. There is no method-level analogue ofnames(m), so
there is no surface to diff against. Requiring all 570 methods to be documented-or-declared is
not a rule anyone would adopt. Docstrings are signature-keyed (Docs.meta(m)[binding].docs
is keyed by signatureTuple), so a method-level audit is technically computable — but most
packages document one signature and carry twenty methods, so it would be noise.Current inclination: push declaration and query down to methods; leave the exhaustive audit
at name level. Method marks are then "queryable if declared", not "checked for completeness".
Needs a decision, and the decision should be written down before the code. -
Revise / re-inclusion. Redefining a method makes the captured
Methodstale. The
name-keyed registry dedups byname; the method-keyed one needs to dedup by(file, line)
or by signature, and to decide what a stale entry means. -
API shape.
experimental(M)currently returnsVector{Mark}. Does it return method marks
too, or is there a separateexperimental_methods(M)? DoesMarkgrow a
Union{Symbol,Method}field, or is there a second type? A union field makes every consumer
branch; a second type duplicates the accessors. -
snapshot/compare. A method signature is exactly the thingcomparecurrently admits
it cannot see (see the "Names, not signatures" warning in the docs). Method marks would make
part of that visible — worth doing, but it changes the snapshot schema, which is itself
declared@experimentalfor this reason.
Why this matters beyond one package
At name level this package overlaps heavily with what already exists: Docs.undocumented_names
is public API in Base since 1.11, and Aqua.test_undocumented_names ships the test. At method
level nothing in the ecosystem offers a stability declaration — and Aqua structurally cannot,
because a mark has to be written in src/ while Aqua is a test-only dependency.
So this is not a refinement of the current feature. It is the part that is not already covered.
Not urgent
Filed as a TODO. No timeline attached.
Activity
sotashimozono commentedon Sep 9, 2026
Implemented in #13 and verified again just now:
The mark records the signature it attached to, so
auditreads it by name andreachreads it by method.contributed_methods/unaccounted_methodscover the other half of the issue — a method contributed to somebody else's generic, whichnames(m)cannot see.compare_methodsdiffs at the same granularity at release time.