Implement MCP protocol, resource identity, retrieval, ranking and prerequisite planning

.cursor/mcp.json

@@ -2,7 +2,7 @@
   "mcpServers": {
     "fields-study-flow": {
       "command": "python",
-      "args": ["-m", "fields_study_flow.mcp_server"]
+      "args": ["-m", "fields_study_flow.mcp_server", "--mcp"]
     }
   }
 }

.vscode/mcp.json

@@ -3,7 +3,7 @@
     "fields-study-flow": {
       "type": "stdio",
       "command": "python",
-      "args": ["-m", "fields_study_flow.mcp_server"]
+      "args": ["-m", "fields_study_flow.mcp_server", "--mcp"]
     }
   }
 }

README.md

@@ -27,7 +27,7 @@ Most learning-path tools stop at a plausible list of links. fields-study-flow ke
 | Area | What it does |
 | --- | --- |
 | Personalized interview | Captures goal type, known topics, skill levels, time budget, output language, and resource language preference. |
-| AI/CS taxonomy | Seeds routes with math, programming, ML/DL, LLM, RL, systems, and paper-reading prerequisites. |
+| Curated prerequisites | Covers math, Python, ML/DL, Transformer, diffusion, and LLM; YOLO/CNN map to DL. RL, systems, and general paper-reading prerequisite graphs are not implemented. |
 | Multi-source discovery | Models GitHub, arXiv, OpenAlex, Semantic Scholar, Unpaywall, Papers with Code, YouTube, Bilibili, Zhihu, Hugging Face, Kaggle, MIT OCW, fast.ai, Google MLCC, and more. |
 | Language-aware ranking | Supports `zh-first`, `en-first`, `balanced`, `zh-only`, and `en-only`. |
 | Paper deep reading | Makes the target paper first-class, then adds prerequisites, intuition resources, and reproduction checkpoints. |
@@ -177,7 +177,7 @@ fields_study_flow/
   ranking.py          # scoring, target-paper boost, canonical URL de-duplication
   roadmap.py          # roadmap schema and Markdown rendering
   mcp_tools.py        # tool functions for agents
-  mcp_server.py       # simple JSON-lines tool server
+  mcp_server.py       # SDK MCP stdio (--mcp), legacy JSONL default
   cli.py              # command-line interface
 ```
 
@@ -200,16 +200,35 @@ python -m pip install -e .
 pytest -q
 ```
 
-Current test coverage includes language preference parsing, bilingual query generation, registry policy filtering, GitHub-style ranking signals, URL de-duplication, MCP tool boundaries, safety validation, roadmap schema, and CLI smoke tests.
+For a reproducible Windows CPython 3.14.7 environment, use [the pinned constraints and build/install steps](docs/REPRODUCIBILITY.md). The constraints include transitive runtime, test, and build dependencies; unconstrained installs can resolve different versions.
+
+Current test coverage includes language preference parsing, bilingual query generation, registry policy filtering, GitHub-style ranking signals, URL de-duplication, MCP tool boundaries, safety validation, concurrent rate limiting and atomic cache writes, dependency ordering and unknown schedules, roadmap schema, and CLI smoke tests.
 
 ## Roadmap
 
-- Live official API adapters for GitHub, YouTube, OpenAlex, Semantic Scholar, Hugging Face, and Papers with Code.
+- Additional live API adapters for YouTube, OpenAlex, Semantic Scholar, Hugging Face, and Papers with Code.
 - A richer AI/CS prerequisite graph with versioned topic nodes.
 - Progress tracking and spaced-review exports.
 - Browser-friendly roadmap preview.
-- Full MCP SDK packaging.
+- Wider MCP host GUI compatibility checks.
 
 ## License
 
 MIT. See [LICENSE](LICENSE).
+
+
+## Current capabilities and limits
+
+`roadmap` remains offline by default. Add `--live --sources arxiv,github` to query public metadata APIs. MCP `searchResources` uses `filters={"offline": false}` for live queries; `type`, `language`, and `difficulty` filters apply to actual results. Offline catalog entries are suggestions, not live search matches. Other registered platforms are link/catalog only. Retrieval reports `offline`, `live`, `cache`, `mixed`, or `degraded` with reasons; network errors return no invented matches. API calls use a 5-second timeout, a 3-second minimum interval shared across service instances per API host in one process, and a one-hour disk cache with atomic replacement (`FIELDS_STUDY_FLOW_CACHE` overrides its directory). There are no paid APIs or authenticated fetches.
+
+`ingestUrl` is URL-only by default. `fetchMetadata=true` requests public GitHub repository or arXiv metadata through the same service, refusing credentials and redirects. Unsupported hosts remain URL-only with an explicit degraded reason. GitHub resource language is the interface/metadata language (English); programming language is recorded separately and is not a natural-language classifier.
+
+Standard MCP hosts must run `python -m fields_study_flow.mcp_server --mcp`. Both editor configurations use this command. The no-flag invocation and `--legacy-jsonl` retain the previous JSONL protocol, including the `git4study` alias. See [MCP transport](docs/MCP.md). The official SDK 2.x negotiates its supported protocol versions; local tests exercise initialize, eight tool schemas, calls, and errors. Editor GUI attachment remains an external check.
+
+Ranking exposes relevance, trust, structure, difficulty, outcome and language components plus unmatched goal terms. A small bilingual alias map and domain levels improve explanations; this is lexical ranking, not semantic search. Thirty fixed review cases live in `tests/fixtures/ranking_cases.json`. Low lexical relevance is labeled supplemental. Prerequisites use the small curated graph described above, seeded by the goal and supplied resource concepts (including the README's known Transformer paper), and skip explicitly known topics. Arbitrary paper URLs do not imply extracted paper concepts. Phases and sequential schedules share one resource dependency order; cycles and unknown prerequisite references raise explicit errors. Target papers start core study after their dependencies. Unknown durations leave that item and all later weeks undetermined; `known_estimated_hours` retains the known subtotal while total hours/weeks remain unknown. These curated estimates may overlap between prerequisites and resource study and are planning aids, not completion guarantees. Learner levels accept `beginner`, `familiar`, `advanced` and the existing Chinese aliases `初学`, `熟悉`, `老手`; invalid levels report allowed values.
+
+Safety checks use parsed URL hosts/schemes. When present, structured `metadata.access_policy` is authoritative for access actions: `metadata`, `link`, and `summarize` are allowed, while unsupported actions, login bypass, mirroring, or `authorized: false` are rejected. Legacy prose is not cross-checked against a present structured policy; URL and missing-note checks still apply independently.
+
+Without structured policy, English access notes receive **finite compatibility rules**, not arbitrary English semantic parsing. The rules recognize the listed download-video, bypass-login, copy-full-text/copyrighted-content, restricted-scraping actions and downloader tool names, including inflections, determiners, and limited passive auxiliaries (`must`, `need(s) to`, `has/have to`, `is/are required to`). Local prohibitions include `do not`, `never`, and trailing `is/are prohibited` or `not/never allowed/permitted`; coordinated action lists can share prohibition scope. Punctuation, `but/however/then/instead`, explicit new subjects, and recognized finite/imperative markers reset inherited scope, including markers before intervening ordinary actions. Thus `Videos need to be downloaded` and `Do not copy full text and instead watch and download video content` are rejected, while `Downloading videos is never permitted` is accepted.
+
+These lexical rules do not resolve arbitrary synonyms, indirect requests, quotation, conditionals, pronouns, nested clauses, all double negations, or other languages. Even some unlisted English constructions can be missed or misclassified; a legacy `valid` result is not proof that arbitrary prose is safe. Supply explicit structured policy for machine-generated access decisions. The implementation does not fetch, download, or execute instructions from these notes.

README.zh-CN.md

@@ -27,10 +27,10 @@ fields-study-flow 可以把“读懂这篇论文”、“学习 Transformer”
 | 能力 | 说明 |
 | --- | --- |
 | 个性化访谈 | 收集目标类型、已知知识、各领域水平、时间预算、路线语言和资料语言偏好。 |
-| AI/CS 技能树 | 内置数学、编程、ML/DL、LLM、RL、系统、论文阅读等前置知识结构。 |
+| 人工维护的先修图 | 覆盖数学、Python、ML/DL、Transformer、diffusion、LLM；YOLO/CNN 映射到 DL。尚未实现 RL、系统和通用论文阅读的先修图。 |
 | 多源资源发现 | 覆盖 GitHub、arXiv、OpenAlex、Semantic Scholar、Unpaywall、Papers with Code、YouTube、Bilibili、知乎、Hugging Face、Kaggle、MIT OCW、fast.ai、Google MLCC 等。 |
 | 语言感知排序 | 支持 `zh-first`、`en-first`、`balanced`、`zh-only`、`en-only`。 |
-| 论文深读路线 | 把目标论文作为第一资源，再补充前置知识、直觉视频、代码实现和复现检查点。 |
+| 论文深读路线 | 把目标论文放入依赖之后的核心阶段，使用已有概念标签补充先修，并提供复现检查点。 |
 | 安全边界 | 拒绝盗版来源、绕登录、下载视频、长篇复制版权内容等不合规路径。 |
 | Agent 友好输出 | 生成 Markdown 与 JSON，方便后续 agent 继续执行、复盘或扩展。 |
 
@@ -180,7 +180,7 @@ fields_study_flow/
   ranking.py          # 评分、目标论文 boost、URL 规范化去重
   roadmap.py          # roadmap schema 与 Markdown 渲染
   mcp_tools.py        # agent 可调用工具函数
-  mcp_server.py       # JSON-lines 工具服务器
+  mcp_server.py       # SDK MCP stdio（--mcp）与旧 JSONL
   cli.py              # 命令行入口
 ```
 
@@ -203,16 +203,35 @@ python -m pip install -e .
 pytest -q
 ```
 
-当前测试覆盖语言偏好解析、双语 query 生成、registry policy、GitHub 风格排序信号、URL 去重、MCP 工具边界、安全校验、roadmap schema 和 CLI smoke tests。
+Windows CPython 3.14.7 的可复现环境见[固定版本 constraints 与构建/安装说明](docs/REPRODUCIBILITY.md)。约束包含运行、测试和构建的传递依赖；无约束安装可能解析到其他版本。
+
+当前测试覆盖语言偏好解析、双语 query 生成、registry policy、GitHub 风格排序信号、URL 去重、MCP 工具边界、安全校验、并发限流与原子缓存写入、依赖顺序与未知排期、roadmap schema 和 CLI smoke tests。
 
 ## 路线图
 
-- 接入 GitHub、YouTube、OpenAlex、Semantic Scholar、Hugging Face、Papers with Code 的官方 API adapter；
+- 继续接入 YouTube、OpenAlex、Semantic Scholar、Hugging Face、Papers with Code adapter；
 - 扩展 AI/CS 前置知识图谱；
 - 增加进度追踪和间隔复习导出；
 - 增加浏览器版路线预览；
-- 打包正式 MCP SDK server。
+- 补充编辑器 GUI 的 MCP 连接验证。
 
 ## License
 
 MIT. See [LICENSE](LICENSE).
+
+
+## 当前能力与限制
+
+CLI `roadmap` 默认离线；`--live --sources arxiv,github` 显式调用公开元数据 API。MCP `searchResources` 传 `filters={"offline": false}` 启用实时检索，`type`、`language`、`difficulty` 会真正过滤资源。其他 registry 平台目前只有链接/目录能力。返回 `offline/live/cache/mixed/degraded` 和降级原因；失败不会伪造查询结果。默认超时 5 秒、同一进程内各服务实例按 API 主机共享至少 3 秒的请求间隔、磁盘缓存 1 小时并原子替换，缓存目录可由 `FIELDS_STUDY_FLOW_CACHE` 设置。不使用付费 API 或认证内容。
+
+`ingestUrl` 默认只录入 URL；`fetchMetadata=true` 从公开 GitHub/arXiv API 获取元数据，拒绝凭据和重定向，其他来源明确降级。GitHub 的 `language=en` 表示元数据界面语言；编程语言另存 metadata，不保证仓库正文语言分类。
+
+标准 MCP 启动命令为 `python -m fields_study_flow.mcp_server --mcp`，两份编辑器配置已同步。无参数启动和 `--legacy-jsonl` 保留旧 JSONL，`git4study` 入口兼容。协议版本由官方 SDK 2.x 协商，详见 [MCP 文档](docs/MCP.md)。本地 SDK 测试涵盖握手、8 工具 schema、调用和错误；编辑器 GUI 实际连接仍需外部验收。
+
+排序输出相关性、可信度、结构、难度、成果偏好、语言等分项及缺失词项。小型中英别名表和领域等级避免全局降级；仍是词汇匹配，不是语义模型。30 条固定样例位于 `tests/fixtures/ranking_cases.json`。低相关性标为补充资料，不声称匹配。先修图使用上文的小型人工图，由目标和已提供的资源概念（包括 README 的已知 Transformer 论文）触发，并跳过明确已知主题；任意论文 URL 不代表已提取论文概念。阶段和串行周计划共用资源依赖顺序，环和未知先修引用会明确报错。目标论文在依赖之后进入核心阶段。未知时长使该项及后续周次未定，`known_estimated_hours` 保留已知小计，总工时与总周数保持 unknown。先修与资源学习工时可能重叠，人工估算不保证实际完成时间。学习等级接受 `beginner/familiar/advanced` 和已有中文别名 `初学/熟悉/老手`，错误等级会提示允许值。
+
+合规校验使用解析后的 URL 主机/协议。存在 `metadata.access_policy` 时，以结构化 policy 为访问动作的权威：允许 `metadata/link/summarize`，拒绝其他动作、绕登录、镜像或 `authorized: false`；此时不再对旧文案做动作交叉检查。URL 与访问说明缺失检查仍独立执行。
+
+没有结构化 policy 时，英文自由文案仅使用**有限兼容规则**，不支持任意英文语义解析。规则识别既定的视频下载、绕登录、全文/长篇版权复制、受限页面抓取动作及下载工具名，覆盖词形、限定词和有限被动助动表达（`must`、`need(s) to`、`has/have to`、`is/are required to`）。局部明确禁止包括 `do not`、`never`、尾部 `is/are prohibited` 或 `not/never allowed/permitted`；并列动作列表可以共享禁止范围。标点、`but/however/then/instead`、显式新主语及识别到的限定谓语/祈使标记终止继承的否定，标记与敏感动作之间插入普通动作也如此。因此 `Videos need to be downloaded` 和 `Do not copy full text and instead watch and download video content` 被拒绝，`Downloading videos is never permitted` 被允许。
+
+这些词法规则不解析任意同义词、间接请求、引语、条件句、代词、嵌套从句、所有双重否定或其他语言。未列出的英文句式仍可能漏判或误判；legacy 返回 `valid` 不代表任意文案安全。机器生成的访问决策应提供明确的结构化 policy。校验器不会抓取、下载或执行文案中的指令。

constraints/windows-py314.txt

@@ -0,0 +1,43 @@
+# Resolved baseline: CPython 3.14.7, Windows x64. See docs/REPRODUCIBILITY.md.
+# Includes runtime, test, and build tools; the project itself is installed separately.
+annotated-types==0.8.0
+anyio==4.15.1
+attrs==26.1.0
+build==1.6.1
+cffi==2.1.1
+click==8.5.0
+colorama==0.4.6
+cryptography==50.0.2
+h11==0.16.0
+httpcore2==2.13.1
+httpx2==2.13.1
+idna==3.20
+iniconfig==2.3.1
+jsonschema==4.26.0
+jsonschema-specifications==2025.9.1
+mcp==2.3.0
+mcp-types==2.3.0
+opentelemetry-api==1.45.1
+packaging==26.3
+pip==26.2.1
+pluggy==1.7.0
+pycparser==3.11
+pydantic==2.14.1
+pydantic-core==2.50.1
+Pygments==2.21.0
+PyJWT==2.15.1
+pyproject-hooks==1.3.3
+pytest==9.1.1
+python-multipart==0.0.32
+pywin32==312 ; sys_platform == 'win32'
+PyYAML==6.0.3
+referencing==0.37.0
+rpds-py==2026.9.1
+setuptools==84.0.0
+sse-starlette==3.5.0
+starlette==1.8.0
+truststore==0.10.5
+typing-extensions==4.16.0
+typing-inspection==0.4.4
+uvicorn==0.54.0
+wheel==0.48.0

docs/MCP.md

@@ -1,17 +1,9 @@
-# MCP Integration
+# MCP transport
 
-`fields_study_flow.mcp_tools` contains pure Python functions that can be wrapped by an MCP host.
+Install the package, then configure your host to run `python -m fields_study_flow.mcp_server --mcp`. The Cursor and VS Code examples select this mode explicitly. Use the Python interpreter from the environment where the package and its MCP SDK dependency are installed.
 
-For local experiments, `python -m fields_study_flow.mcp_server` runs a JSON-lines tool process. Each line should be:
+The server uses the official MCP Python SDK 2.x `MCPServer` stdio transport. Protocol versions and capabilities are negotiated by that SDK; this project does not implement additional protocol versions. Eight tools expose schemas derived from their existing typed signatures: assessKnowledge, discoverSources, searchResources, ingestUrl, rankResources, buildRoadmap, validateSources, exportPlan. Unknown tools and invalid arguments return SDK tool errors. Logs use stderr; stdout is reserved for protocol messages.
 
-```json
-{"tool":"searchResources","arguments":{"query":"Transformer derivation","languagePreference":"en-first"}}
-```
+For compatibility, `python -m fields_study_flow.mcp_server --legacy-jsonl` accepts one `{"tool":"assessKnowledge","arguments":{"goal":"python"}}` per line and returns `{"ok":true,"result":...}`. Running without a flag retains this legacy behavior, including `python -m git4study.mcp_server`. JSONL is not MCP. Use `--mcp` for standard hosts.
 
-The process returns:
-
-```json
-{"ok":true,"result":{}}
-```
-
-The JSON-lines server is intentionally small and deterministic. Production MCP packaging can map the same functions onto a full MCP SDK server without changing the tool behavior.
+Run `python -m pytest tests/test_mcp_protocol.py -q` for an actual SDK stdio initialization, tools/list, buildRoadmap call, error cases, legacy subprocess checks and configuration checks. Editor GUI attachment is a separate external check.

docs/REPRODUCIBILITY.md

@@ -0,0 +1,32 @@
+# Reproducing the reviewed environment
+
+`constraints/windows-py314.txt` pins the resolved runtime dependencies, their transitive dependencies, pytest, and build tools for **CPython 3.14.7 on Windows x64**. `pyproject.toml` retains compatible dependency ranges for library consumers. A normal unconstrained `pip install .` does not reproduce this baseline.
+
+From a repository checkout in PowerShell:
+
+```powershell
+py -3.14 -m venv .venv
+$python = '.\.venv\Scripts\python.exe'
+& $python -m pip install --constraint constraints/windows-py314.txt pip
+& $python -m pip install --constraint constraints/windows-py314.txt build setuptools wheel
+& $python -m pip install --constraint constraints/windows-py314.txt --no-build-isolation -e '.[dev]'
+& $python -m pytest -q
+& $python -m pip check
+& $python -m build --wheel --no-isolation
+```
+
+`--no-isolation` / `--no-build-isolation` ensure the build uses the explicitly installed, pinned tools. Without these flags, a separate build environment can resolve different versions of setuptools.
+
+To verify the wheel outside the checkout, create another venv, use its Python, and install the wheel with an absolute path to the constraints file:
+
+```powershell
+& $wheelPython -m pip install --constraint $constraintsPath $wheelPath
+& $wheelPython -m pip check
+& $wheelCli discover-sources --goal transformer
+```
+
+`$wheelPython`, `$wheelCli`, `$constraintsPath`, and `$wheelPath` refer to that venv's Python/CLI and the absolute constraint/wheel paths. Run the CLI from a temporary directory so an import from the source checkout cannot hide a packaging omission.
+
+These are version constraints, not an artifact hash lock: package availability still depends on the configured index, and wheels are not claimed to be byte-identical. Other Python versions or operating systems require their own resolved baseline and test run. No cross-platform installation claim follows from the package's declared Python minimum.
+
+When updating dependencies, resolve in a fresh venv, update the pins including transitive dependencies, run the complete tests and MCP SDK stdio round trip, then build and install the wheel in a separate venv. Record the actual Python version, installed packages, and results before replacing this baseline.