Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions docs/sessions/261008-local-upload/reviews/A-r1-verdict.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
<!-- delegate-outcome: succeeded -->
## 结论

failure-visibility: skipped

人工独立审查已完成;本次固定 diff 未发现可复现的事实性 finding。OCR 前置扫描为 `skipped`,不能把空 findings 当作 clean;因此机器行保留 `skipped`。无人工确认的 P1/P2 缺陷,代码层面不构成独立阻断;OCR 跳过本身按审查纪律记录,不伪报通过。

## 不变式与消费者证据

- 身份/幂等:`cache_manager.py:346-363,640-692` 建窄表并以 owner/key 唯一、metadata 指纹拒绝冲突;`test_local_upload_policy.py:77,146` 用真实 SQLite 覆盖跨 owner、重放、冲突及并发唯一。
- token 隔离:接收根写旧 `view_token=''`、独立 `upload_` token(`cache_manager.py:694-738`);Resolver 的 view 与 cache 两个公共入口均按前缀只查上传记录(`view_token_resolver.py:161-195,263-297`),旧查询拒绝 blank/`upload_`(`cache_manager.py:3383-3411`),LLM 配置与 owner fallback 也拒绝上传 token(`view_token_resolver.py:128-130,302-307`; `audit.py:660-665`)。测试:`test_local_upload_policy.py:175,259,312`、原 URL Resolver 测试。
- 资格/时间:资格读取由 `local_upload_share_is_active` 检查状态、撤销和 expiry(`cache_manager.py:779-793`);唯一终态入口 `update_task_status` 用 SQLite `CURRENT_TIMESTAMP` 写 `completed_at`,同事务以该值设 30d expiry,never 不设截止(`cache_manager.py:2734-2780`)。恢复/关闭收敛也调用该入口(`:2958-3125`)。测试:`test_local_upload_policy.py:221,240,304`。所有持久字符串比较以 UTC 秒级格式;部署/systemd 实际时钟未探测,不从造时测试推断部署结论。
- 读取与清理:`cleanup_old_cache` 在媒体锁内复查活动上传,且不看功能开关(`cache_manager.py:2101-2116`);`cleanup_task_status` 用 keyset 前进,并在 `BEGIN IMMEDIATE` 内复查活动 root 后才归档、撤销快照、删 task(`:2284-2337,2345-2369`)。实际维护入口将 `task_exists` 传给审计清理(`api/app.py:253-257`);`AuditLogger.cleanup_old_logs` 保留仍有 task root 的旧快照(`audit_logger.py:835-861`)。测试:`test_local_upload_policy.py:352,366,388,424,485,508`。
- 实际消费者核对:独立临时 cache.db/audit.db 实测活动 never 上传正文可由 Resolver 读到;人为回置旧时间后,cache/task 清理均删 0 行,审计快照保留。撤销后 Resolver 拒绝,cache/task 各清 1 行;root 删除后快照在自身审计期限内继续保留,回置其归档时间超过期限才由真实 `task_exists` 清理删除。所有文件和数据库均在 `/tmp`。
- URL 与媒体兼容:旧 token 生成 `view_`(`cache_manager.py:2520`),Resolver 对非 `upload_` 仍走原 task 查询;原 URL/history/清理套件包含在 H0 `make test` 中。临时媒体仍由 `TempFileManager` 清理,测试 `test_local_upload_policy.py:508`。
- 文档/目标:设计、进度和六个 goal 仅记录 A 现状及后续门槛;M1 仍为实现中,后续未开始;没有改写 `GOALS.md`,没有把生产/恢复标成已验证。

## 验证、限制与阻断

- 本地冻结依赖(仅 `/tmp/vta-upload-A-r1-venv`):`python -m pytest -q tests/unit/test_local_upload_policy.py tests/unit/test_view_token_resolver.py tests/cache/test_task_status_cleanup.py` 全部通过。
- H0 CI:run `37765345309`,`gate / quality` job `113271519803` 与 `Tests` 步骤成功;`make test`。PR #201 仍为 draft,`gate / primary` 为 skipped,不视为通过。派发主干基线不可用,继承红/新红无法判定。
- OCR:wrapper `skipped`,reason=`primary=primary_marker_active(primary_failed_unclassified); backup:deepseek=leg_timeout`。诊断为 MiniMax 429 rate-limited,DeepSeek 180s timeout;直调也未产生 findings。stdout JSON 在仓外 `/tmp/vta-upload-A-r1-ocr-stdout.json`;不是 clean。
- 未访问生产、systemd、备份恢复域或真实容量;这些不是 A 的本地 SQLite 证据,也未获本卡授权。独立 review 本身完成;没有发现需 lead 阻断本次代码的已证实问题。
149 changes: 149 additions & 0 deletions docs/sessions/261008-local-upload/reviews/A-runtime-probes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
<!-- delegate-outcome: succeeded -->
# PR201 三项运行探针

固定 head:`3a25dccc2cb92e22b4031715c37a584c041e582d`。只改本证据文件;所有 SQLite、缓存目录和 audit.db 均在 `TemporaryDirectory`,未读写 repo 配置。先以 mkdtemp 写入确认权限,再 `uv sync --frozen`(仅本 worktree `.venv`),`PYTHONPATH=src` 导入真实 `CacheManager`。无 worker/HTTP 端点;不外推到生产。

## 判定

1. **preterminal cache deletion:默认完整清理路径已证伪。** register/accept + `save_cache` 真写入正文;`local_upload_share_is_active=True`,而 `_has_active_local_upload_for_media=False`(`expires_at=NULL` 的 SQL 条件缺失)。但 `cleanup_old_cache` 先命中 queued/processing/calibrating task guard,三份文件与行均保留。撤销且 failed 上传、URL 对照由完整入口删除(2 行),不是 0 候选。仅用临时 SQL 把 `updated_at` 设为 31 天前以越过年龄筛选,不代表自然发生。覆盖 `update_task_status` 文档列出的全部非终态。后续:lead 决定是否统一两个 predicate 的语义;本次未证明实际清理丢失。
2. **task cleanup FK crash:默认路径已证伪;FK ON 条件反例复现。** 默认 `_get_connection()` 实测 `foreign_keys=0`,真实 `cleanup_task_status` 无异常并删 root;receipt 保留、root_task_id 成孤儿(1)。同流程显式 `PRAGMA foreign_keys=ON` 后实际清理抛 `IntegrityError`,事务回滚,root/receipt 保留。`rg -n "PRAGMA foreign_keys" src` 无命中;CacheManager、AuditLogger 与只读 SQL 构造均未开启它。条件反例不代表默认路径或 P1。后续:lead 决定是否约束外部启用 FK 的契约,以及默认路径孤儿 receipt 是否符合历史查询需要。
3. **accept after 24h:默认路径可复现。** 窗内 register,跨窗 24h+1s 调真实 accept 得 `ValueError`;DB 仍是 `receiving`、无 root。过窗重复 register 仍读出原 receipt。契约写明 server 验证 24h acceptance window、正式接受后才返回 202(design.md:35,42);按此字面拒绝符合约定。design 同时写“首次登记”窗口,若意为只校验 register,则与 accept 二次校验有歧义,需 lead 定义时点;本证据不改逻辑。

## 运行输出

探针1(旧年龄为受控条件;producer 文件及完整 consumer 路径):

```text
queued: share=True sql_guard=False root=queued expiry=none file_before=True file_after=True
processing: share=True sql_guard=False root=processing expiry=none file_before=True file_after=True
calibrating: share=True sql_guard=False root=calibrating expiry=none file_before=True file_after=True
inactive: share=False sql_guard=False root=failed expiry=set file_before=True file_after=False
url: file_before=True file_after=False
cleanup_deleted=2 cache_rows=5->3
```

探针2(临时 AuditLogger/DB;终态 root;now 控制为完成后 61 天以满足过期条件):

```text
foreign_keys=0 outcome=ok deleted=1 root=1->0 upload=1->1 orphan=1
foreign_keys=1 outcome=IntegrityError deleted=na root=1->1 upload=1->1 orphan=0
```

探针3:

```text
accept_exception=ValueError receipt_state=receiving root_task_id=null root_rows=0
```

## 可复制探针

从仓库根目录运行;仅输出状态、计数、异常类型与文件存在性,不打印路径。

### 1. cache 清理

```bash
PYTHONPATH=src .venv/bin/python - <<'PY'
from datetime import datetime, timedelta, timezone; from pathlib import Path; import tempfile, uuid
from loguru import logger; from video_transcript_api.cache.cache_manager import CacheManager
logger.remove(); now = datetime.now(timezone.utc)
old = (now - timedelta(days=31)).strftime('%Y-%m-%d %H:%M:%S')
with tempfile.TemporaryDirectory() as td:
base = Path(td); m = CacheManager(str(base/'cache'), str(base/'cache/cache.db')); cases = {}
def make_upload(label, status=None, revoked=False):
mid = 'probe-' + label; key = f"{int(now.timestamp()*1000)}-{uuid.uuid4()}"
u = m.register_local_upload(owner_user_id='owner', idempotency_key=key, retention='30d', now=now)
a = m.accept_local_upload(u['upload_id'], media_id=mid, task_id='task-'+label, now=now)
if status in ('processing','calibrating'): m.update_task_status(a['root_task_id'], status, suppress_terminal_notification=True)
saved = m.save_cache('local_upload', '', mid, False, 'producer-body', 'capswriter')
if not saved: raise RuntimeError('save_cache failed')
if status == 'failed': m.update_task_status(a['root_task_id'], status, suppress_terminal_notification=True)
if revoked: m.revoke_local_upload(u['upload_id'], now=now)
with m._get_cursor() as c: c.execute('UPDATE video_cache SET updated_at=? WHERE media_id=?', (old, mid))
with m._get_cursor() as c:
c.execute('SELECT u.*, t.status root_status FROM local_uploads u JOIN task_status t ON t.task_id=u.root_task_id WHERE u.media_id=?', (mid,))
row = dict(c.fetchone()); c.execute('SELECT 1 FROM video_cache WHERE media_id=?', (mid,)); before = c.fetchone() is not None
with m._get_cursor() as c: sql_active = m._has_active_local_upload_for_media(c, 'local_upload', mid, m._utc_sql_timestamp(now))
cases[label] = (row, CacheManager.local_upload_share_is_active(row, now), sql_active, base/'cache'/saved['files_loc'], before)
for s in ('queued','processing','calibrating'): make_upload(s, s)
make_upload('inactive', 'failed', True)
saved = m.save_cache('youtube', 'https://example.invalid/v', 'url-control', False, 'url-body', 'capswriter')
if not saved: raise RuntimeError('URL save_cache failed')
with m._get_cursor() as c: c.execute('UPDATE video_cache SET updated_at=? WHERE media_id=?', (old, 'url-control'))
url_path = base/'cache'/saved['files_loc']
with m._get_cursor() as c: before_rows = c.execute('SELECT COUNT(*) FROM video_cache').fetchone()[0]
deleted = m.cleanup_old_cache(days=30, now=now)
with m._get_cursor() as c: after_rows = c.execute('SELECT COUNT(*) FROM video_cache').fetchone()[0]
for label, (row, active, sql_active, path, before) in cases.items():
print(f"{label}: share={active} sql_guard={sql_active} root={row['root_status']} expiry={'none' if row['expires_at'] is None else 'set'} file_before={before} file_after={path.exists()}")
print(f"url: file_before=True file_after={url_path.exists()}"); print(f"cleanup_deleted={deleted} cache_rows={before_rows}->{after_rows}")
m._get_connection().close()
PY
```

### 2. task cleanup / FK

```bash
PYTHONPATH=src .venv/bin/python - <<'PY'
from datetime import datetime, timedelta, timezone; from pathlib import Path; import tempfile, uuid
from loguru import logger; from video_transcript_api.cache.cache_manager import CacheManager
from video_transcript_api.utils.logging.audit_logger import AuditLogger
logger.remove()
def run(force_fk):
with tempfile.TemporaryDirectory() as td:
base = Path(td); m = CacheManager(str(base/'cache'), str(base/'cache/cache.db'))
audit = AuditLogger(str(base/'audit.db')); m.audit_logger = audit; conn = m._get_connection()
if force_fk: conn.execute('PRAGMA foreign_keys=ON')
fk = conn.execute('PRAGMA foreign_keys').fetchone()[0]
now = datetime.now(timezone.utc); late = now + timedelta(days=61); key = f"{int(now.timestamp()*1000)}-{uuid.uuid4()}"
u = m.register_local_upload(owner_user_id='owner', idempotency_key=key, retention='30d', now=now)
a = m.accept_local_upload(u['upload_id'], media_id='fk-probe', task_id='task-fk', now=now)
m.update_task_status(a['root_task_id'], 'failed', suppress_terminal_notification=True)
with m._get_cursor() as c:
c.execute('SELECT COUNT(*) FROM task_status WHERE task_id=?', (a['root_task_id'],)); rb = c.fetchone()[0]
c.execute('SELECT COUNT(*) FROM local_uploads WHERE upload_id=?', (u['upload_id'],)); ub = c.fetchone()[0]
try: deleted = str(m.cleanup_task_status(0, now=late)); outcome = 'ok'
except Exception as e: deleted = 'na'; outcome = type(e).__name__
with m._get_cursor() as c:
c.execute('SELECT COUNT(*) FROM task_status WHERE task_id=?', (a['root_task_id'],)); ra = c.fetchone()[0]
c.execute('SELECT COUNT(*) FROM local_uploads WHERE upload_id=?', (u['upload_id'],)); ua = c.fetchone()[0]
c.execute('SELECT COUNT(*) FROM local_uploads u LEFT JOIN task_status t ON t.task_id=u.root_task_id WHERE u.upload_id=? AND t.task_id IS NULL', (u['upload_id'],)); orphan = c.fetchone()[0]
print(f"foreign_keys={fk} outcome={outcome} deleted={deleted} root={rb}->{ra} upload={ub}->{ua} orphan={orphan}")
conn.close(); audit.close()
run(False); run(True)
PY
```

### 3. 窗口过期后正式 accept

```bash
PYTHONPATH=src .venv/bin/python - <<'PY'
from datetime import datetime, timedelta, timezone; from pathlib import Path; import tempfile, uuid
from loguru import logger; from video_transcript_api.cache.cache_manager import CacheManager
logger.remove()
with tempfile.TemporaryDirectory() as td:
base = Path(td); m = CacheManager(str(base/'cache'), str(base/'cache/cache.db'))
start = datetime.now(timezone.utc); late = start + timedelta(hours=24, seconds=1)
key = f"{int(start.timestamp()*1000)}-{uuid.uuid4()}"
u = m.register_local_upload(owner_user_id='owner', idempotency_key=key, retention='30d', now=start)
try: m.accept_local_upload(u['upload_id'], media_id='late-probe', task_id='task-late', now=late); exc = 'none'
except Exception as e: exc = type(e).__name__
receipt = m.register_local_upload(owner_user_id='owner', idempotency_key=key, retention='30d', now=late)
with m._get_cursor() as c: c.execute('SELECT COUNT(*) FROM task_status WHERE task_id=?', ('task-late',)); roots = c.fetchone()[0]
print(f"accept_exception={exc} receipt_state={receipt['state']} root_task_id={'null' if receipt['root_task_id'] is None else 'set'} root_rows={roots}")
m._get_connection().close()
PY
```

## 约束与验收

固定 H0:`docs/sessions/261008-local-upload/design.md`。相关锁定测试名:`test_cleanup_protects_active_upload_cache_even_when_feature_is_disabled`、`test_cleanup_reclaims_revoked_or_expired_upload_cache`、`test_task_cleanup_keeps_effective_share_root_for_audit_and_url_rows_still_expire`、`test_expired_or_revoked_upload_task_rows_follow_existing_cleanup`、`test_real_sqlite_upload_identity_owner_key_and_intent_window`。只执行运行探针及卡面指定文件存在性判据,未跑 pytest/CI。

主干基线派发时不可用(`gh api request failed`),继承红未能判定;本卡没有新 CI 红。未改应用/测试/workflow。

## pickup 巡检

open issue 总数:3
summary: orphan 0 owned 0 unattributable 0 too-new 0 recent-7d 0 stale-over-7d 0 missing_ledger_repos 0
memory 巡检报告不可用:memory_dir_mismatch;未复用其他会话报告。

接手锚点:无交接单。当前分支 `card/vta-upload-A-probes-261008`,派发简报中 dispatch 即本任务本身;未将其视作他人占用。
Loading
Loading