Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# AGENTS.md — Tandem 开发纪律
# AGENTS.md — JevDuo 开发纪律

本文件是协作与代码合入的硬约束。此前那份仓库外的重构总计划已删除,
改为 **human in the loop 的开发与重构**:改动前先与维护者确认形态与取舍,
范围与命名以本文件 + `docs/CLUSTER_RUNTIME.md` 为准。

## 仓库定位

主体是 **Tandem harness**(快慢双模 CUA,`src/jevduo/`);`src/jevduo_eval/` 仅为适配集群 rjob 的测评薄适配层(worker 侧 6 文件 + 开发机侧 rjob.py / rjob_spec.py / rjob_status.py / config_schema.py)。
主体是 **JevDuo harness**(快慢双模 CUA,`src/jevduo/`);`src/jevduo_eval/` 仅为适配集群 rjob 的测评薄适配层(worker 侧 6 文件 + 开发机侧 rjob.py / rjob_spec.py / rjob_status.py / config_schema.py)。

## 五条硬纪律(CI/评审强制)

Expand Down
3 changes: 3 additions & 0 deletions DEVIATIONS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# 与旧仓库(SA-CUAharness,冻结于 main@bc290e3)的已知行为差异

> 命名说明:项目已由 Tandem 更名为 JevDuo。下文历史条目中的 `tandem` / `tandem_eval` / `src/tandem/`
> 即现在的 `jevduo` / `jevduo_eval` / `src/jevduo/`,条目正文保持原样。

人工验收与代码评审对照本清单。按计划 §3.1 有意排除的项标注【计划内】,接口移植中的取舍标注【重构】。

## M1(planner_led 主路径移植)
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Tandem
# JevDuo

**面向 Computer Use 场景的快慢双模 CUA harness。**

慢模型负责**理解与规划**——读长指令、读屏幕、读无障碍树,产出「下一步该做什么」的意图;
快模型(JEV / Kev 这类 System One 决策模型)负责**选择**——在一个已被裁剪过的候选动作空间里,
一次前向就挑出具体动作。Tandem 把这两级拼成一条可评测、可回放、可对比的决策闭环。
一次前向就挑出具体动作。JevDuo 把这两级拼成一条可评测、可回放、可对比的决策闭环。

```
观察 Observation
Expand Down Expand Up @@ -218,7 +218,7 @@ jevduo check # 确认入口可用
```

- 自己的脚本里 `import tandem` / `from tandem_eval ...` 需改为 `jevduo` / `jevduo_eval`。
- 环境变量 `TANDEM_*` 本次未改名,仍按原名使用。
- 环境变量 `TANDEM_EXPERIMENT_ROOT` / `TANDEM_CONFIG_FILE` 改名为 `JEVDUO_EXPERIMENT_ROOT` / `JEVDUO_CONFIG_FILE`;旧名作为过渡仍可用(stderr 提示弃用),后续版本移除。
- rjob 任务名前缀由 `tandem-<run-id>` 变为 `jevduo-<run-id>`;开发机 `infra/config.json` 里的 `eval.repo_root` / `eval.deploy_root` 需按新目录同步,旧的部署副本确认无在跑任务后再清理。

## 会话配置
Expand Down
2 changes: 1 addition & 1 deletion demo/build_site.py
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ def build_episodes(


def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description="构建 Tandem demo 单页工作台")
parser = argparse.ArgumentParser(description="构建 JevDuo demo 单页工作台")
parser.add_argument("--runs-root", type=Path, action="append", default=None,
help="结果目录根,可给多次(默认:合并基线清单里的批次)")
parser.add_argument("--out", type=Path, default=REPO_ROOT / "demo" / "site")
Expand Down
2 changes: 1 addition & 1 deletion demo/fetch_sai_frames.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@

CACHE_DIR = REPO_ROOT / "demo" / ".cache" / "sai_frames"
SAI_REPO = "https://huggingface.co/datasets/simular-ai/sai-osworld-v21-reference-runs/resolve/main"
USER_AGENT = "tandem-demo/0.1 (+frame fetch for trajectory comparison)"
USER_AGENT = "jevduo-demo/0.1 (+frame fetch for trajectory comparison)"

FRAME_WIDTH = 960
FRAME_QUALITY = 72
Expand Down
2 changes: 1 addition & 1 deletion demo/sync_official.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@
SAI_INDEX_URL = f"{SAI_REPO}/index.jsonl"
SAI_SCORECARD_URL = f"{SAI_REPO}/scorecard/ref-scorecard.json"

USER_AGENT = "tandem-demo-sync/0.1 (+offline snapshot refresh)"
USER_AGENT = "jevduo-demo-sync/0.1 (+offline snapshot refresh)"


def fetch(url: str, *, timeout: int = 120) -> bytes:
Expand Down
2 changes: 1 addition & 1 deletion demo/web/assets/app.css
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
/* Tandem demo 工作台 · 设计系统
/* JevDuo demo 工作台 · 设计系统
*
* 一条原则:**少边框、多留白、用字号和色阶分层**,别把信息堆成表格墙。
* 颜色只在这里定义(CSS 变量),组件内禁止硬编码色值;深浅两套主题共用变量名。
Expand Down
4 changes: 2 additions & 2 deletions demo/web/assets/app.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
/* Tandem demo 工作台 · 应用内核(无框架、无构建、离线可开)
/* JevDuo demo 工作台 · 应用内核(无框架、无构建、离线可开)
*
* 纪律:
* - 不引入任何外部资源(CDN 只作为可选增强,主链路零外链);
Expand Down Expand Up @@ -74,7 +74,7 @@ window.TD = (function () {
if (!view) {
// 未知路由:回退到首页,但留一条痕迹(别静默吞掉)
if (typeof console !== "undefined" && console.warn) {
console.warn("[tandem-demo] 未知视图:" + current.view + ",回退到首页");
console.warn("[jevduo-demo] 未知视图:" + current.view + ",回退到首页");
}
view = views.home;
}
Expand Down
6 changes: 3 additions & 3 deletions demo/web/assets/views/browser.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
/* 轨迹浏览:三栏工作台(页面骨架)
*
* 左:任务卡(首帧预览 + 域 + 得分 + 步数)
* 中:Tandem 与 Sai 两边独立回放,**花费与用时按步骤累计**
* 中:JevDuo 与 Sai 两边独立回放,**花费与用时按步骤累计**
* 右:Task Instruction、模型底座、评测(得分 / 步数 / 逐条要求)
*
* 播放器本身在 browser_player.js(TD.browserPlayer)——拆开是为了让这个文件
Expand Down Expand Up @@ -89,7 +89,7 @@
]));
var duo = TD.el("div", { class: "duo" }, [
TD.browserPlayer.player({
title: "Tandem",
title: "JevDuo",
who: "MiniMax-M3.1-Flash + JEV",
score: episode && episode.score !== null ? "得分 " + episode.score.toFixed(4) : "",
items: episode ? TD.browserPlayer.ourItems(episode) : []
Expand All @@ -116,7 +116,7 @@

var scoreBox = TD.el("div", { class: "score-box" }, [
TD.el("div", { class: "score-big", text: row.ours === null ? "—" : (row.ours * 100).toFixed(2) + "%" }),
TD.el("div", { class: "score-sub", text: "Tandem · " + ((episode && episode.steps) || 0) + " 步" }),
TD.el("div", { class: "score-sub", text: "JevDuo · " + ((episode && episode.steps) || 0) + " 步" }),
TD.el("div", { class: "score-sub", text: "Sai · " + (row.sai_best === null || row.sai_best === undefined ? "—" : (row.sai_best * 100).toFixed(2) + "%") })
]);
right.appendChild(scoreBox);
Expand Down
2 changes: 1 addition & 1 deletion demo/web/assets/views/home.js
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
/* ------------------------------------------------------------ 1. 简介 */
function intro() {
return TD.el("section", { class: "intro" }, [
TD.el("p", { class: "eyebrow", text: "TANDEM · 快慢双模 CUA HARNESS" }),
TD.el("p", { class: "eyebrow", text: "JEVDUO · 快慢双模 CUA HARNESS" }),
TD.el("h1", { class: "intro-title", text: "一个大脑,一个小脑,一起操作电脑" }),
TD.el("p", { class: "intro-lede", html:
"慢模型负责拆解、规划与纠错;快模型负责在候选空间里挑选动作。" +
Expand Down
2 changes: 1 addition & 1 deletion demo/web/assets/views/results.js
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@
}

var rows = [
row("Tandem", "MiniMax-M3.1-Flash + JEV", "v2.1", 500,
row("JevDuo", "MiniMax-M3.1-Flash + JEV", "v2.1", 500,
(ours.binary_rate_attempted || 0) * 100, (ours.partial_over_dataset || 0) * 100,
ours.usd_per_task, true)
];
Expand Down
4 changes: 2 additions & 2 deletions demo/web/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Tandem Harness · OSWorld 2.1 工作台</title>
<title>JevDuo Harness · OSWorld 2.1 工作台</title>
<link rel="stylesheet" href="assets/app.css">
<link rel="stylesheet" href="assets/views.css">
<link rel="stylesheet" href="assets/browser.css">
Expand All @@ -13,7 +13,7 @@
<header class="topbar">
<div class="brand">
<span class="logo" aria-hidden="true"></span>
<strong>Tandem</strong>
<strong>JevDuo</strong>
</div>
<nav class="nav" id="nav">
<a href="#/" data-view="home">首页</a>
Expand Down
8 changes: 6 additions & 2 deletions docs/CLUSTER_RUNTIME.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 内部集群运行手册

本文覆盖 **Tandem 在内部集群(adpc / evobox)上的运行方式**:配置、run 生命周期、
本文覆盖 **JevDuo 在内部集群(adpc / evobox)上的运行方式**:配置、run 生命周期、
worker 入口、共享资源与已知坑。

产品定位与架构说明见根目录 [README.md](../README.md);结果目录字段级定义见
Expand Down Expand Up @@ -233,7 +233,7 @@ OSW2.1 走**独立环境**(独立 runtime / VM / 资产 / venv / env.sh),
清单不等于环境验收报告;API、VM或评分异常仍记为 `failed_unscored`。
- **共享环境**:通用VM为 `vm/unified-20261022-complete.qcow2`,经 `prepared/runtime`
接入环境准备适配。依赖缓存、网站DNS/证书和控制器传输适配在共享环境侧维护,
不改变Tandem规划、候选、验证或评分逻辑。worker只读基盘与冻结venv,每题使用隔离VM。
不改变JevDuo规划、候选、验证或评分逻辑。worker只读基盘与冻结venv,每题使用隔离VM。
- **网站与权限**:从环境侧 `services/web/endpoint.json` 与 `services/gitlab/endpoint.json`
发现内网地址,不依赖开发机的localhost。使用者需有共享盘、内网和镜像仓库权限;
026、041通过配置模板中的 `gitlab.private_token_file` 读取共享测试管理员token。
Expand Down Expand Up @@ -332,3 +332,7 @@ worker 侧各段日志:
脚本只同步一次代码,然后按指定并发提交。worker 的 adpc 挂载只读时,结果先落
evobox 暂存,单题完成后立即 rsync 到实验目录。`--dry-run` 只生成 manifest 和 batch log,
不提交 rjob。

可选环境变量:`JEVDUO_EXPERIMENT_ROOT` 覆盖实验根目录(默认 `rjob_logs/`),
`JEVDUO_CONFIG_FILE` 覆盖配置文件(默认 `infra/config.json`)。改名前的 `TANDEM_EXPERIMENT_ROOT` /
`TANDEM_CONFIG_FILE` 作为过渡仍可用,会在 stderr 提示弃用,后续版本移除。
18 changes: 16 additions & 2 deletions scripts/run_experiment.sh
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ Usage:

Experiment names use lowercase letters, digits and hyphens only.
Artifacts: rjob_logs/NAME/{manifest.json,batch.log,drivers/,NAME-TASK/}.
Env: JEVDUO_EXPERIMENT_ROOT (default rjob_logs), JEVDUO_CONFIG_FILE (default infra/config.json).
EOF
}

Expand Down Expand Up @@ -60,7 +61,20 @@ if [[ ! "$EXECUTION_MODE" =~ ^(step|stage|goal)$ || "$STAGE_MAX_ACTIONS" -lt 1 ]
exit 2
fi

EXPERIMENT_ROOT="${TANDEM_EXPERIMENT_ROOT:-$REPO/rjob_logs}"
# JEVDUO_* is the current name; TANDEM_* is the pre-rename name, still honored with a warning.
env_or_legacy() {
local name="$1" legacy="$2" default="$3"
if [[ -n "${!name:-}" ]]; then
printf '%s' "${!name}"
elif [[ -n "${!legacy:-}" ]]; then
echo "warning: $legacy is deprecated; use $name instead" >&2
printf '%s' "${!legacy}"
else
printf '%s' "$default"
fi
}

EXPERIMENT_ROOT="$(env_or_legacy JEVDUO_EXPERIMENT_ROOT TANDEM_EXPERIMENT_ROOT "$REPO/rjob_logs")"
EXPERIMENT_DIR="$EXPERIMENT_ROOT/$EXPERIMENT"
DRIVER_DIR="$EXPERIMENT_DIR/drivers"
if [[ -e "$EXPERIMENT_DIR/manifest.json" ]]; then
Expand All @@ -73,7 +87,7 @@ if [[ ! -f infra/config.json && "$DRY_RUN" -ne 1 ]]; then
echo "infra/config.json is required for a real run" >&2
exit 4
fi
CONFIG_FILE="${TANDEM_CONFIG_FILE:-infra/config.json}"
CONFIG_FILE="$(env_or_legacy JEVDUO_CONFIG_FILE TANDEM_CONFIG_FILE infra/config.json)"
if [[ ! -f "$CONFIG_FILE" ]]; then
CONFIG_FILE=infra/config.example.json
fi
Expand Down
51 changes: 46 additions & 5 deletions tests/eval/test_experiment_script.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@


def test_dry_run_creates_manifest_and_batch_log(tmp_path):
env = {**os.environ, "TANDEM_EXPERIMENT_ROOT": str(tmp_path)}
env = {**os.environ, "JEVDUO_EXPERIMENT_ROOT": str(tmp_path)}
result = subprocess.run(
[
"bash", str(SCRIPT),
Expand Down Expand Up @@ -40,7 +40,7 @@ def test_dry_run_creates_manifest_and_batch_log(tmp_path):


def test_invalid_experiment_name_is_rejected(tmp_path):
env = {**os.environ, "TANDEM_EXPERIMENT_ROOT": str(tmp_path)}
env = {**os.environ, "JEVDUO_EXPERIMENT_ROOT": str(tmp_path)}
result = subprocess.run(
["bash", str(SCRIPT), "--experiment", "bad name", "--tasks", "024", "--dry-run"],
capture_output=True,
Expand All @@ -56,7 +56,7 @@ def test_dry_run_selects_goal_mode_without_changing_script_default(tmp_path):
result = subprocess.run([
"bash", str(SCRIPT), "--experiment", "goal-smoke", "--tasks", "008", "--execution-mode", "goal",
"--stage-max-actions", "2", "--dry-run"], capture_output=True, text=True,
env={**os.environ, "TANDEM_EXPERIMENT_ROOT": str(tmp_path)})
env={**os.environ, "JEVDUO_EXPERIMENT_ROOT": str(tmp_path)})
assert result.returncode == 0, result.stderr
manifest = json.loads((tmp_path / "goal-smoke/manifest.json").read_text())
assert manifest["execution_mode"] == "goal" and manifest["stage_max_actions"] == 2
Expand All @@ -65,8 +65,8 @@ def test_dry_run_selects_goal_mode_without_changing_script_default(tmp_path):
def test_dry_run_uses_example_when_private_config_is_missing(tmp_path):
env = {
**os.environ,
"TANDEM_EXPERIMENT_ROOT": str(tmp_path),
"TANDEM_CONFIG_FILE": str(tmp_path / "missing.json"),
"JEVDUO_EXPERIMENT_ROOT": str(tmp_path),
"JEVDUO_CONFIG_FILE": str(tmp_path / "missing.json"),
}
result = subprocess.run(
["bash", str(SCRIPT), "--experiment", "ci-dry-run", "--tasks", "024", "--dry-run"],
Expand All @@ -76,3 +76,44 @@ def test_dry_run_uses_example_when_private_config_is_missing(tmp_path):
)
assert result.returncode == 0, result.stderr
assert (tmp_path / "ci-dry-run" / "manifest.json").is_file()


def _env_without_overrides(**extra):
"""Drop any inherited JEVDUO_*/TANDEM_* so the precedence tests see only what they set."""
env = {k: v for k, v in os.environ.items() if not k.startswith(("JEVDUO_", "TANDEM_"))}
return {**env, **extra}


def test_legacy_tandem_env_vars_still_work_with_deprecation_warning(tmp_path):
env = _env_without_overrides(
TANDEM_EXPERIMENT_ROOT=str(tmp_path),
TANDEM_CONFIG_FILE=str(tmp_path / "missing.json"),
)
result = subprocess.run(
["bash", str(SCRIPT), "--experiment", "legacy-env", "--tasks", "024", "--dry-run"],
capture_output=True,
text=True,
env=env,
)
assert result.returncode == 0, result.stderr
assert (tmp_path / "legacy-env" / "manifest.json").is_file()
assert "TANDEM_EXPERIMENT_ROOT is deprecated; use JEVDUO_EXPERIMENT_ROOT" in result.stderr
assert "TANDEM_CONFIG_FILE is deprecated; use JEVDUO_CONFIG_FILE" in result.stderr


def test_jevduo_env_var_takes_precedence_over_legacy(tmp_path):
new_root, old_root = tmp_path / "new", tmp_path / "old"
env = _env_without_overrides(
JEVDUO_EXPERIMENT_ROOT=str(new_root),
TANDEM_EXPERIMENT_ROOT=str(old_root),
)
result = subprocess.run(
["bash", str(SCRIPT), "--experiment", "both-env", "--tasks", "024", "--dry-run"],
capture_output=True,
text=True,
env=env,
)
assert result.returncode == 0, result.stderr
assert (new_root / "both-env" / "manifest.json").is_file()
assert not old_root.exists()
assert "deprecated" not in result.stderr
4 changes: 2 additions & 2 deletions tools/render_summary.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
from pathlib import Path

_PAGE = """<!doctype html><html lang="zh"><meta charset="utf-8">
<title>Tandem run {run_id}</title>
<title>JevDuo run {run_id}</title>
<style>
body{{font-family:system-ui,sans-serif;margin:2rem;color:#1a1a1a}}
table{{border-collapse:collapse;width:100%}}
Expand All @@ -18,7 +18,7 @@
.scored{{color:#0a7d32}}.failed_unscored{{color:#c0392b}}
h1{{font-size:1.2rem}} code{{background:#f4f4f4;padding:0 .3rem}}
</style>
<h1>Tandem run <code>{run_id}</code></h1>
<h1>JevDuo run <code>{run_id}</code></h1>
<p>benchmark: <b>{benchmark}</b> · total {total} · scored {scored} ·
failed_unscored {failed_unscored} · average {average}</p>
<table><tr><th>task</th><th>score</th><th>status</th><th>stop_reason</th><th>steps</th></tr>
Expand Down
Loading