Context. Layer 5 § Open Questions: "Who defines the reference task behind acmp:price_basis, and who may publish or version such a catalogue?"
Layer 5 advertises an indicative price per capability (acmp:price_cu_indicative), measured on a reference task named by acmp:price_basis. Without a shared definition of that task, two providers' figures describe different work — and the field is no more comparable than the per-token price it replaced.
Your task. Work out who could own such a catalogue, and what it would take:
- Who publishes it? The working group, an independent body, competing catalogues?
- What goes in it? What makes a task a fair reference for a capability like
sentiment-analysis — input size, difficulty spread, scoring?
- Versioning. The example value is
acmp:reference-tasks/v1. What happens to advertised prices when v2 lands?
- Prior art. How do MLPerf, SPEC, or benchmark suites in other markets govern their definitions?
This is the discovery-side face of the CU tier-definition question (RFC-0001 §7, question 2) — its candidate Assessor role maintains benchmark definitions, and a reference task is one. An answer here may constrain that one.
Done = a comment or PR weighing at least two governance models against ACMP's principle of provider-agnostic simplicity (P3).
Context. Layer 5 § Open Questions: "Who defines the reference task behind
acmp:price_basis, and who may publish or version such a catalogue?"Layer 5 advertises an indicative price per capability (
acmp:price_cu_indicative), measured on a reference task named byacmp:price_basis. Without a shared definition of that task, two providers' figures describe different work — and the field is no more comparable than the per-token price it replaced.Your task. Work out who could own such a catalogue, and what it would take:
sentiment-analysis— input size, difficulty spread, scoring?acmp:reference-tasks/v1. What happens to advertised prices when v2 lands?This is the discovery-side face of the CU tier-definition question (RFC-0001 §7, question 2) — its candidate Assessor role maintains benchmark definitions, and a reference task is one. An answer here may constrain that one.
Done = a comment or PR weighing at least two governance models against ACMP's principle of provider-agnostic simplicity (P3).