dev-stats

English | 中文

A CLI tool for observing metrics across GitHub repositories and content platforms (stars / clones / views / likes / comments), with rich terminal tables and CSV export. Metrics are layered by "API cost × value" — lightweight by default, opt-in for deeper metrics.

• L0 Public metrics (no token needed): repo list, Stars, Forks, Issues, last push, language
• L1 Traffic metrics (repo admin required): 14-day Clone / unique cloners / Views; --traffic-full adds top paths and referrers
• L2 Repo details (--detail, public): size, License, topics, archived status, default branch, created at, homepage, subscribers
• L3 Community health (--community, public): health score 0-100 + README/license/code-of-conduct file completeness
• L4 Activity (--activity, public): commits in last 4 weeks, latest release
• L5 CI status (--actions, public): latest GitHub Actions run conclusion per repo (✓ success / ✗ failure / ● running, - when no workflows); failed runs automatically fetch failure details (only failing repos cost +1~2 requests): which job/step failed and GitHub annotation error messages
• Account info: followers / following / public repos / account age (--no-user-info to disable)
• Auth: auto-reuses gh auth token, falls back to GITHUB_TOKEN / GH_TOKEN env vars, or --token flag; supports .env file (copy .env.example to .env, gitignored)
• Output: rich terminal tables + summary row; --csv exports details (sanitized by default, no non-public traffic data)
• HTML preview: every --csv export also writes a sortable <name>-preview.html beside it (--no-html skips it)
• CI-only mode (--ci-only): skips the traffic fetch and the wide repo table, printing just a short CI summary — suited to agents / pipes
• Private repos (--include-private): your own private repos are fetched but hidden by default; the flag reveals them (only effective when querying yourself with auth)
• Traffic scope (--traffic-since-days N, default 730): only fetch clone / view traffic for repos updated within N days; older repos are recorded as 0
• Repo exclusion (--exclude a,b): drop named repos from the report (legacy CI noise from unmaintained projects)

Usage

cd work/python

# Query all your repos (star/fork + clone traffic, requires gh auth login)
uv run dev-stats

# Query a specific user (others' repos only show public metrics + account info)
uv run dev-stats --user torvalds

# Sort by 14-day clones, top 10 non-fork repos
uv run dev-stats --sort clones --no-forks --limit 10

# Per-repo deep metrics: details + community health + activity (--limit also caps fetch volume)
uv run dev-stats --no-forks --limit 10 --detail --community --activity

# Identify CI status: whether the latest GitHub Actions run succeeded per repo
uv run dev-stats --no-forks --limit 10 --actions

# Exclude specific repos (legacy CI noise, e.g. Dependabot failures on unmaintained projects)
uv run dev-stats --no-forks --actions --exclude wildsKick,king-power

# Full traffic: top paths + referrers (requires own repos + admin access)
uv run dev-stats --no-forks --limit 10 --traffic-full

# Export CSV (sanitized by default, no traffic data; add --include-traffic to include)
uv run dev-stats --csv output/stats.csv

# CSV export also writes output/stats-preview.html (sortable); --no-html to skip it
uv run dev-stats --csv output/stats.csv --no-html

# Reveal your own private repos (hidden by default even when authenticated)
uv run dev-stats --include-private --sort updated

# Only collect traffic for repos updated in the last 90 days (default 730)
uv run dev-stats --no-forks --traffic-since-days 90

# CI-only patrol: no traffic fetch, no wide repo table, just the CI summary
uv run dev-stats --ci-only --no-forks --sort updated

# Skip traffic fetch, only public data (faster, saves API quota)
uv run dev-stats --no-traffic

# Juejin: query articles for JUEJIN_USER_ID in .env
uv run dev-stats juejin --sort diggs --limit 10
uv run dev-stats juejin --sort daily --limit 10     # sort by daily avg views (=views ÷ days since publish, reflects reach efficiency)
uv run dev-stats juejin --user-id 123456789012 --csv juejin.csv

# SegmentFault: scan sf_id from wordpress-tools articles (requires SEGMENTFAULT_ENABLED=true in .env)
uv run dev-stats segmentfault --sort views --limit 10
uv run dev-stats segmentfault --ids 1190000000000000 --csv sf.csv

# Aggregate Juejin + SegmentFault, merged by master article for cross-platform comparison
uv run dev-stats report --sort total --limit 10
# The "linked" column in Juejin/SegmentFault tables = master article (erishen.cn) + GitHub repos
# (data from data/repo_articles.json, generated by scanning erishen.cn links in each repo's README)

See all options with uv run dev-stats --help.

Structure

dev-stats/
├── pyproject.toml            # Package definition, entry point dev-stats = dev_stats:main
├── src/dev_stats/
│   ├── __init__.py           # Package entry, exports main
│   ├── api.py                # GitHubClient (requests.Session + pagination + rate-limit headers) / RepoStats / metric collectors
│   ├── cli.py                # argparse, sorting, rich table rendering, CSV export, juejin/segmentfault/report subcommand skeletons
│   └── link.py               # Repo ↔ master article reverse index (wp_id → repo list)
├── data/
│   └── repo_articles.json    # Repo → master article mapping (generated by scanning erishen.cn links in repo READMEs)
├── tests/
│   ├── test_dev_stats.py     # Offline unit tests: sorting / CSV / rendering / token detection / API field regression
│   ├── test_juejin.py        # Juejin module tests (auto-skip when local module is absent)
│   ├── test_segmentfault.py  # SegmentFault module tests (auto-skip when local module is absent)
│   ├── test_link.py          # Mapping tests
│   └── test_report.py        # Cross-platform aggregation tests
├── .env.example              # Env var template (copy to .env and fill in)
└── output/                   # Runtime CSV output directory (gitignored)

Local modules: src/dev_stats/juejin.py and src/dev_stats/segmentfault.py are local-only scraping modules for personal use — not tracked, not in git history. cli.py includes defensive imports; when modules are absent, subcommands gracefully degrade with a hint.

Known Limitations (determined by GitHub API)

• Clone data is only visible to repo admins: /traffic/clones only works for repos you have access to, with only 14-day data and no cumulative totals. When querying others' repos, the table automatically falls back to public metrics.
• Unauthenticated rate limit: 60 req/hour; authenticated with gh token (repo scope): 5000 req/hour.
• Deep metrics are per-repo requests: --detail / --community each add +1 request/repo, --activity adds +12 requests/repo, --actions adds +1 request/repo (failing repos cost +12 more for failure details). Large accounts should use --limit to cap fetch volume, or --no-traffic to skip traffic.
• Community health score returns 404 for fork repos (GitHub limitation), table shows - automatically.
• Commits in last 4 weeks is counted via Link header pagination (per_page=1), an exact GitHub-side value, not an estimate.
• Traffic fetch loop has a built-in 50ms delay to prevent secondary rate limiting; a warning is printed when remaining API quota drops below 500.

Privacy & Compliance

• CSV sanitized by default: --csv export excludes non-public traffic data (clones_14d / views_14d / top_paths / top_referrers) by default. Add --include-traffic for local use; do not share the exported file publicly.
• SegmentFault scraping disabled by default: dev-stats segmentfault requires SEGMENTFAULT_ENABLED=true in .env; do not enable in public deployments.
• Scraping modules not tracked: juejin.py / segmentfault.py are local modules, completely removed from git tracking and history — public repo contains no scraping implementation.
• Tokens not tracked: .env (including GITHUB_TOKEN / JUEJIN_USER_ID / local paths) is excluded by .gitignore; output/ directory is also not tracked.
• Commit identity: git config uses GitHub noreply email, no real email exposed.

Development

Common tasks are consolidated in the Makefile (make or make help to see all targets):

make check      # ruff lint + format check + offline unit tests, all in one
make run ARGS="--sort clones"   # Run CLI with pass-through args
make csv        # Export repo stats to output/stats.csv (no traffic, so no clone columns; most recently updated first)
make csv-traffic # Same export but collecting traffic, i.e. with the clone/view columns — use this for clone analysis
make ci         # CI patrol only: short plain-text summary, skips traffic fetch and the wide repo table
make lint       # ruff check only, no rewrites (make fmt reformats src + tests)
make juejin     # Query Juejin article metrics (default sort by daily avg views)
make segmentfault # Query SegmentFault article metrics (default sort by daily avg views, requires SEGMENTFAULT_ENABLED=true)
make report     # Aggregate Juejin + SegmentFault cross-platform comparison (default sort by total daily avg views)
make actions    # Patrol CI status across repos (--actions, excludes legacy CI noise by default, failures include failed step + error)

License

MIT — Copyright (c) 2026 Erishen