Repository navigation
feat: enable bounded Yibuapi Skill Eval execution - #181
Conversation
Signed-off-by: GUOGUO <xwdong1998@163.com> Agent-assisted-by: Codex (author)
Skill evaluationPLANNED: execution is not authorized by knowing this hash. Selected comparisons: 0 · Estimated task calls: 0. Applicable only while these revisions and inputs match the current PR. Evidence and logs · Retention: 30 days. Pipeline evidence only; offline fixed outputs do not establish skill efficacy. Plan bindings, budget and frozen examPlan SHA-256: Model/settings:
Uncovered / unsupported paths (9)
|
Signed-off-by: GUOGUO <xwdong1998@163.com> Agent-assisted-by: Codex (author)
Signed-off-by: GUOGUO <xwdong1998@163.com> Agent-assisted-by: Codex (author)
Signed-off-by: GUOGUO <xwdong1998@163.com> Agent-assisted-by: Codex (author)
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Dong1017
left a comment
There was a problem hiding this comment.
LGTM for the bounded software-delivery scope at head ffb5ce1 against base 38d5883. Independent static reviews covered cost reservation/runtime and credential/workflow isolation; the final delta rounds each request upward to cents before allocation and adds an additivity regression assertion. No verified correctness blocker remains in the inspected change.
Verified CI: all four Ready jobs passed: https://github.com/InferMatrixLab/InferRSI/actions/runs/40481781107 . Current declaration policy is still pending, so merge remains conditional on that prerequisite turning green.
The configured path uses the internal Python worker, preserves trusted publication/regrading and default Codex behavior, excludes provider credentials from the verifier and publication credentials from the worker, and bounds all attempted requests without automatic retries. This is software readiness, not evidence of live Yibuapi compatibility or skill efficacy. No paid call was made during review. The real smoke still needs an exact fresh frozen plan, applicable key-group billing facts and the owner's USD100 total bound; #101/#88 acceptance is not granted by this review.
Refs #101. Enables usable opt-in Yibuapi execution after #174.
provider_config connects the direct Messages client to an internal Python worker, the existing paired comparison runner and the hosted execute job. Complete reviewed pricing/bounds and the owner-provisioned Secret enable the path; no unconditional live blocker remains. Codex stays the default. No external agent executable/framework is added.
The frozen readiness record binds model/provider/group, effective currency/per-million input/output/cache-read/cache-write rates, request fee, expiry and all request/output/read/tool bounds. Reservation covers every possible attempted request across both arms/comparisons, using the full registered model-context input ceiling, the highest input/cache rate, full output ceiling and fee. Every request rounds upward to cents before allocation. Unknown/expired pricing, understated context, currency mismatch, over-budget totals or missing keys fail before calls; missing keys precede the local claim. Dispatch history remains durable and is not erased. No automatic retries, fallback model or extra paid probe; the first baseline request supplies compatibility evidence.
Only the conditional provider model step receives SKILL_EVAL_YIBUAPI_KEY. Worker/verifier environments exclude GitHub/Actions credentials, and the verifier receives no provider key. No key in argv, prompt, trace, artifact or plan/publish. Hosted permissions, trusted checkout, exact approval/source identity, one-shot history, artifact validation and the separate privileged publisher/regrade remain intact. New paid runs explicitly stop on unknown usage/interruption before later comparisons. Thinking/signature blocks are structurally validated and preserved opaquely without content logging; auto tool choice supports Opus5.5.
Head ffb5ce1; base 38d5883; tree e10902c81ee296ea5e3dba7f9f32515ba985793b. All nine remote files were read back equal to the inspected source.
Validation:
Owner's current one-smoke total is USD100, superseding CNY10; no retries or extra campaigns. Parent verified public Yibuapi per1M prices8/40/0.4/10 USD for input/output/cache-read/cache-write in default/claude-code/claude-code-pro. A two-trial, max3-requests/trial, max4096-output proposal reserves USD61.02 at the full1M input ceiling (explicit zero request fee under the published token-only schedule). Confirm the chosen group applies to the owner key before freezing. No API/key probe, spending, Secret read/configuration, login or execute dispatch occurred.
This PR proposes a narrowly scoped Secret exposure; parent checks the owner's action-time approval before merge, then prepares/approves a new exact trusted plan and coordinates the real smoke. Selected public inputs/skills go to Yibuapi; held-out answers/verifier/unrelated files do not. #101/#88 remain open.