Skip to content

skills: add Public231 C1, remeasure gate, and new-model integration - #172

Merged
hsliuustc0106 merged 3 commits into
mainfrom
cursor/repo-measurement-skills-19e8
Oct 25, 2026
Merged

hsliuustc0106 merged 3 commits into
mainfrom
cursor/repo-measurement-skills-19e8

Conversation

@hsliuustc0106

@hsliuustc0106 hsliuustc0106 commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

Required human contributor answers

Source: owner Hongsheng, this task (draft three repo skills; draft PR only). Later room clarification: production GPU identity is H200 / SM90, not a hard-coded L20X / sm_89 label. Each tip must name its RSI phase and the primary versus alternatives, or it is not merge-clean.

  1. What outcome are you trying to achieve?
    Repo-level skills so agents measure Public231 C1, gate later optimization on a remeasure with the same protocol and the same recorded hardware, and bring in a new open model in a fixed order.

  2. What is the highest-leverage variable for that outcome?
    A written, repo-accurate recipe agents load before those tasks, instead of inventing a hardware label, stacking unmeasured tips, or bundling a model into a serving issue.

  3. How will this work move that variable rather than create busywork?
    Three SKILL.md files under the existing .agents/skills/ layout. No runtime code changes.

  4. Who is the customer or user, and what do they actually need?
    Maintainers and coding agents on system1-omni. They need the measurement, the remeasure gate, and the new-model sequence in the repo.

Is a PR needed? Yes. Owner Hongsheng approved a draft pull request for these three skills.

Bug-fix evidence: Not a bug fix. Review withhold: do not treat a non-matching SKU as INCONCLUSIVE. The documented production target is H200 / SM90. Evidence records the measured device string and compute capability. Paired runs must share that recorded identity.

Purpose

Add three skills agents can load for measurement and model bring-up:

  • .agents/skills/public231-c1/SKILL.md — Public231 C1: warmup 5, concurrency 1, three rounds, SHA, pass rate, p50/p95, the bench.py clock, both no-thinking treatments, metrics-only room posts. Every paste records device and compute capability. Examples name the production target as H200 / SM90. A different nvidia-smi string is recorded as printed and is not INCONCLUSIVE.
  • .agents/skills/optimize-remeasure-gate/SKILL.md — paste p50/p95 plus device and compute capability before editing; do not stack the next tip until that tip is remeasured with the same protocol on the same recorded identity; incomplete evidence is INCONCLUSIVE. Each tip states its RSI phase (bring-up/scheduling, parallelism/comms, kernel, or launch/post) and which alternatives were considered versus the primary, or it is not merge-clean. The PR stays draft until a separate owner yes.
  • .agents/skills/new-model-integration/SKILL.md — own GitHub issue, draft PR, recipe and operator checklist, then Public231 C1, then continuous optimization on that branch under the same gate. Serving-chain issues stay separate from the new-model RFC.

Placement matches the existing .agents/skills/ tree (add-new-model, self-review, precheck-pr). There is no .cursor/skills/ tree in this repo.

Test Plan

Docs-only skill files. No runtime code.

  • Confirm the three paths exist and frontmatter has name and description.
  • Check relative links resolve to current benchmarks/, recipe/, CONTRIBUTING.md, and sibling skills.
  • Confirm commands match benchmarks/bench.py (--warmup default 3, so C1 passes --warmup 5; clock post_to_body_json_and_schema_validation; three separate measured runs).
  • Confirm no skill gate marks a run INCONCLUSIVE for failing to match L20X, sm_89, H200, or SM90. Comparisons require the same protocol and the same recorded device plus compute capability.

Test Result

System1-Omni Version / Commit: c1dd645

Relative links in the three skills resolve. Claims were checked against benchmarks/README.md, benchmarks/bench.py, benchmarks/latency-leaderboard.md, and the open new-model vs serving-chain issue split (#61, #85, #125, #150 versus new model issues such as #114 and #136). No Rust or GPU run: the diff does not change runtime code.

Demo / evidence

N/A. Skill documents only; no inference, serving, or benchmark campaign in this PR.

Self-review

  • I have reviewed the full diff and addressed the issues I found.
  • I have checked that the change follows the project's architecture and stays focused on the stated purpose.
  • I have run the checks appropriate to this change and reported commands, results, and anything I could not verify above.
  • I have checked that the PR description, documentation, and any accuracy or performance claims match the implementation and available evidence.

Draft only. Do not merge and do not mark ready without a separate owner yes.

Open in Web Open in Cursor 

Document the lab L20X C1 measurement, the one-tip remeasure gate for
agent optimization loops, and the issue-then-draft sequence for a new
open model. Skill docs only.

Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
@hsliuustc0106

Copy link
Copy Markdown
Contributor Author

the skill should not be spefified to a particular hardware

cursoragent and others added 2 commits October 25, 2026 07:05
Stop treating a non-L20X device as inconclusive. Record the measured
device name and compute capability, and require that pair plus the same
measurement protocol on both sides of a comparison.

Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
Document H200 / SM90 as the production label without gating on it.
Every paste records the measured device string and compute capability,
paired runs must share that identity, and a tip is not merge-clean
without its RSI phase and the primary versus alternatives.

Co-authored-by: Hongsheng Liu <liuhongsheng4@huawei.com>
@hsliuustc0106
hsliuustc0106 marked this pull request as ready for review October 25, 2026 07:10
@hsliuustc0106
hsliuustc0106 merged commit 51f80ad into main Oct 25, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants