crawler-verification

Here are 2 public repositories matching this topic...

Quincunx33 / bot-guard-py

Dependency-free HTTP bot detection and application-layer flood mitigation for Python.

python http flask django rate-limiting wsgi web-security anti-bot ddos-mitigation bot-detection asgi ddos-protection fastapi bot-protection crawler-verification http-flood-protection

Updated Oct 30, 2026
Python

nimajafari / crawl-log-toolkit

Star

Turn raw server/CDN access logs into verified-crawler analytics: parse many log formats, verify Googlebot/Bingbot by IP (not user-agent), enrich with DuckDB, and analyze crawl budget, crawl waste, and crawl traps with SQL.

python log-analysis ci seo access-logs googlebot observability bingbot log-parsing server-logs technical-seo duckdb crawl-budget crawler-verification

Updated Jun 23, 2026
Python

Add this topic to your repo

To associate your repository with the crawler-verification topic, visit your repo's landing page and select "manage topics."

Learn more