Skip to content

Identify the R2 manifest validator to Cloudflare #33

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
AlexMikhalev merged 1 commit into from
Sep 25, 2026
Merged

Identify the R2 manifest validator to Cloudflare #33

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 12 additions & 6 deletions scripts/validate-r2-manifests.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,13 @@ def version_tuple(value: str) -> tuple[int, int, int]:


def fetch(base_url: str, path: str, limit: int) -> bytes:
with urllib.request.urlopen( # nosec B310: validate() allowlists the scheme
f"{base_url}/{path}", timeout=60
) as response:
# Cloudflare bot management on the public channel answers the default
# Python-urllib User-Agent with 403, so identify as the validator.
request = urllib.request.Request( # nosec B310: validate() allowlists the scheme
f"{base_url}/{path}",
headers={"User-Agent": "terraphim-r2-manifest-validator/1.0"},
)
with urllib.request.urlopen(request, timeout=60) as response:
data = response.read(limit + 1)
if len(data) > limit:
raise ValueError(f"{path}: response exceeds {limit} bytes")
Expand Down Expand Up @@ -144,9 +148,11 @@ def verify_asset(
raise ValueError("chunk size must be positive")
digest = hashlib.sha256()
total = 0
with opener( # nosec B310: validate() allowlists the scheme
f"{base_url}/{path}", timeout=60
) as response:
request = urllib.request.Request( # nosec B310: validate() allowlists the scheme
f"{base_url}/{path}",
headers={"User-Agent": "terraphim-r2-manifest-validator/1.0"},
)
with opener(request, timeout=60) as response:
while True:
chunk = response.read(min(chunk_size, declared_size - total + 1))
if not chunk:
Expand Down
Loading