[feature] 重写 llama.cpp Quick Start 并接入看护 - #144
Conversation
把 peft quick start 文档接入 doc-test 看护流程：PR 触发 + 调度运行，
并把下载路径从 modelscope 切到 HuggingFace Hub 以满足 120 min
job timeout 预算。
CI 基础设施
- .github/workflows/peft-quick-start.yml: PR 触发 + 调度三小时一次
- .github/workflows/quick-start-template.yml: 共享执行引擎
(peft 不再传 container_options,NFS 已持久化 /root/.cache,
bind mount 在缺 CAP_SYS_ADMIN 的 burstable K8s pod 上是 no-op)
- job timeout 提到 120 min,DEFAULT_COMMAND_TIMEOUT 同步到 7200s
诊断探针
- tests/doc_test/model_cache.py 新增 diagnose_mount_environment:
findmnt / stat / /proc/self/mountinfo / cgroup / CapBnd / df +
modelscope.cn 与 huggingface.co 原始 endpoint 吞吐对比,
辅助"是否切换 snapshot_download provider"决策
缓存校验(provider-split 架构)
- tests/doc_test/modelscope_cache.py → model_cache.py
- 按 provider 拆 modelscope / huggingface 两组函数
- HF 新增 resolve_huggingface_cache / report_huggingface_state /
purge_huggingface_corrupt(HF 标准 layout:
models--<org>--<model>/{blobs,snapshots})
- provider-agnostic 工具: ensure_safetensors /
safetensors_header_ok / _curl_throughput_probe
切换到 HuggingFace Hub
- 决策依据: modelscope.cn 0.39 MiB/s(补 3.70 GB 缺失 shard 需 2.7h,
已超 120 min job timeout);huggingface.co 16.71 MiB/s(42x 快);
HF 缓存 /root/.cache/huggingface/hub/ 已暖好 5.75 GB 完整模型
- sources/peft/quick_start.md: 组件表 / pip install /
snapshot 入口全部从 modelscope 切到 huggingface_hub
- tests/peft/test_quick_start_ascend.py: imports 与
prepare_environment 调用方改用 report_huggingface_state /
purge_huggingface_corrupt / resolve_huggingface_cache
替换过时的 RST 安装/上手页，按文档字面在昇腾上编译 CANN 后端并做单卡推理。
CLA Signature Guide@licy666 , thanks for your pull request. The following commit(s) are not associated with a signed Contributor License Agreement (CLA).
To sign CLA, click here. To check if your email is configured correctly, refer to the FAQs. Once you've signed the CLA or updating your email, please comment |
|
📖 文档预览：https://ascend.github.io/docs/pr-preview/pr-144/sources/llama_cpp/ 也可打开整站首页。 构建自 |
licy666
changed the title
真机日志同时出现 CANN0/CANN1 设备与 model buffer；触发器改为 2 卡 runner。
|
补充：5.2 双卡推理现在也看护。
|
CLA Signature Guide@licy666 , thanks for your pull request. The following commit(s) are not associated with a signed Contributor License Agreement (CLA).
To sign CLA, click here. To check if your email is configured correctly, refer to the FAQs. Once you've signed the CLA or updating your email, please comment |
用户可见路径只留编译、下模型和单卡/双卡推理；看护块与预期输出未改。
CLA Signature Guide@licy666 , thanks for your pull request. The following commit(s) are not associated with a signed Contributor License Agreement (CLA).
To sign CLA, click here. To check if your email is configured correctly, refer to the FAQs. Once you've signed the CLA or updating your email, please comment |
栏目首页不再套一层「快速开始」子页，改为收录同一份 quick_start.md。
CLA Signature Guide@licy666 , thanks for your pull request. The following commit(s) are not associated with a signed Contributor License Agreement (CLA).
To sign CLA, click here. To check if your email is configured correctly, refer to the FAQs. Once you've signed the CLA or updating your email, please comment |
Each runner pod starts with an empty workdir, so the visible curl would re-download ~400 MB every time. Hidden setup now restores and saves the file under the already-mounted /root/.cache/huggingface/llama.cpp/.
CLA Signature Guide@licy666 , thanks for your pull request. The following commit(s) are not associated with a signed Contributor License Agreement (CLA).
To sign CLA, click here. To check if your email is configured correctly, refer to the FAQs. Once you've signed the CLA or updating your email, please comment |
Keep llama.cpp's Sphinx exclude and tutorial landing page. Take main's peft index.md, shared doc-test engine, and peft Quick Start.
CLA Signature Guide@licy666 , thanks for your pull request. The following commit(s) are not associated with a signed Contributor License Agreement (CLA).
To sign CLA, click here. To check if your email is configured correctly, refer to the FAQs. Once you've signed the CLA or updating your email, please comment |
3 participants
Summary
sources/llama_cpp/quick_start.md：源码编译 CANN 后端、从 Hugging Face 下一份 Qwen2.5-0.5B Q4_0 GGUF、单卡 / 双卡llama-completion。llama_cpp-quick-start.yml+tests/llama_cpp子类。首页卡片官方链接改为ggml-org/llama.cpp，去掉会 404 的「安装指南」。