Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
654 changes: 654 additions & 0 deletions .github/copilot-instructions.md

Large diffs are not rendered by default.

536 changes: 536 additions & 0 deletions .github/fork/PROCESS.md

Large diffs are not rendered by default.

1,550 changes: 1,550 additions & 0 deletions .github/fork/forkctl.py

Large diffs are not rendered by default.

1,255 changes: 1,255 additions & 0 deletions .github/fork/test_forkctl.py

Large diffs are not rendered by default.

85 changes: 85 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
<!-- Fork PR into integration/3.0. Delete nothing; fill everything in. -->

## What

<!-- One or two sentences. If you need "and" to describe unrelated changes, split the PR. -->

## Why

<!-- Problem, or the capability we need. Link the issue if there is one. -->

## How tested

<!-- Commands you actually ran in this environment, and their outcome. -->

- [ ] `pre-commit run --all-files`
- [ ] `make rebuild` (required if `.c`/`.h` changed — zero new warnings)
- [ ] `make test-unit` / `make test-eval`
- [ ] `make test-c` (C sim logic)
- [ ] `make test-notebooks` (notebooks touched)
- [ ] `make test-docker-smoke` (dynamics, rewards, observations, or training loop touched)
- [ ] `make -C tests/drive perf` (hot paths touched)
- [ ] `make test` before marking ready (unit + eval + c + notebooks — does **not** include
docker-smoke or perf; those are the two boxes above)

New behaviour is covered by a new test / the fix has a test that fails without it:

<!-- name the test(s) -->

## Stack hygiene

<!-- One command checks all four boxes:
python .github/fork/forkctl.py check-commit
On a chore/<slug> branch add --chore: the fork-only checks are exempt by design, the
one-commit and subject checks still apply in full. Paste the output below. -->

- [ ] `forkctl check-commit` passes (`--chore` on a fork-only branch)
- [ ] Conventional Commits subject, rebased on current `integration/3.0`

<!-- paste the forkctl check-commit output -->

## Merge gate

<!-- All three must be green before this is stacked. See copilot-instructions.md §4.8. -->

- [ ] **Gate 1 — CI green** on the final commit (no "green except a flake")
- [ ] **Gate 2 — adversarial review** posted by someone other than the author; findings fixed or
explicitly accepted in the thread
- [ ] **Gate 3 — upstream-readiness proven**: `forkctl mirror <slug> --sha HEAD` passes, i.e. the
commit cherry-picks onto pure `upstream/3.0` and its diff is upstream-clean
<!-- exempt for chore(fork): fork-only changes — tick the "Never upstream" box below -->

Gate 3 result:

<!-- paste the `forkctl mirror` output, or write "exempt: chore(fork)" -->

## Upstream disposition

- [ ] Mirror PR to `Emerge-Lab/PufferDrive` requested now — needs a human to verify the final
title, body and diff first, and the body must carry the authorship disclosure from
`.github/fork/PROCESS.md` → *Agent authorship*
- [ ] Contributable, but we hold it for now, because: <!-- reason -->
- [ ] **Never upstream** — fork-only `chore(fork):` change, Gate 3 exempt

<!--
Fork-only change? Two independent axes, BOTH must be marked:

do-not-mirror (never reaches upstream) — all four:
1. PR is a DRAFT
2. title contains [do-not-mirror]
3. the `do-not-mirror` label is applied
4. the "Never upstream" box above is ticked

do-not-merge (the merge button is never pressed) — both:
5. title contains [do-not-merge]
6. the `do-not-merge` label is applied

Merging it would put a SECOND chore(fork) commit on the stack. `forkctl check-stack` asserts there
is exactly one, so every future sync would abort. It is folded into the base chore(fork) commit by
fixup instead. See .github/fork/PROCESS.md -> Changing fork-only content.
-->


## Risk / notes

<!-- Behaviour changes, config or observation-layout changes, perf impact, follow-ups. -->
34 changes: 25 additions & 9 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: CI

on:
pull_request:
branches: [ main, '3.0' ]
branches: [ main, '3.0', 'integration/3.0' ]
push:
branches: [ main, '3.0' ]

Expand Down Expand Up @@ -33,8 +33,12 @@ jobs:
PIP_NO_CACHE_DIR: 1
run: |
sudo apt-get update && sudo apt-get install -y build-essential cmake
python -m pip install -U pip pytest
pip install -e . --no-cache-dir
python -m pip install -U pip "setuptools<81" wheel numpy pytest
# CPU-only torch: the CUDA stack no longer fits the runner's disk
# quota. --no-build-isolation then reuses it for the build instead
# of resolving a second copy. See tests/smoke_tests/Dockerfile.
pip install --index-url https://download.pytorch.org/whl/cpu torch
pip install -e . --no-build-isolation --no-cache-dir
python setup.py build_ext --inplace --force

- name: Unit tests
Expand Down Expand Up @@ -70,8 +74,12 @@ jobs:
PIP_NO_CACHE_DIR: 1
run: |
sudo apt-get update && sudo apt-get install -y build-essential cmake xvfb libgl1-mesa-dev libegl1-mesa-dev libx11-dev libxcursor-dev libxrandr-dev libxinerama-dev libxi-dev libxxf86vm-dev
python -m pip install -U pip
pip install -e . --no-cache-dir
python -m pip install -U pip "setuptools<81" wheel numpy
# CPU-only torch: the CUDA stack no longer fits the runner's disk
# quota. --no-build-isolation then reuses it for the build instead
# of resolving a second copy. See tests/smoke_tests/Dockerfile.
pip install --index-url https://download.pytorch.org/whl/cpu torch
pip install -e . --no-build-isolation --no-cache-dir
python setup.py build_ext --inplace --force

- name: C drive tests
Expand Down Expand Up @@ -103,8 +111,12 @@ jobs:
PIP_NO_CACHE_DIR: 1
run: |
sudo apt-get update && sudo apt-get install -y build-essential cmake
python -m pip install -U pip pytest jupytext nbclient ipykernel
pip install -e . --no-cache-dir
python -m pip install -U pip "setuptools<81" wheel numpy pytest jupytext nbclient ipykernel
# CPU-only torch: the CUDA stack no longer fits the runner's disk
# quota. --no-build-isolation then reuses it for the build instead
# of resolving a second copy. See tests/smoke_tests/Dockerfile.
pip install --index-url https://download.pytorch.org/whl/cpu torch
pip install -e . --no-build-isolation --no-cache-dir
python setup.py build_ext --inplace --force

- name: Run notebooks
Expand Down Expand Up @@ -151,8 +163,12 @@ jobs:
PIP_NO_CACHE_DIR: 1
run: |
sudo apt-get update && sudo apt-get install -y build-essential cmake libgl1-mesa-dev libegl1-mesa-dev libx11-dev libxcursor-dev libxrandr-dev libxinerama-dev libxi-dev libxxf86vm-dev
python -m pip install -U pip
pip install -e . --no-cache-dir
python -m pip install -U pip "setuptools<81" wheel numpy
# CPU-only torch: the CUDA stack no longer fits the runner's disk
# quota. --no-build-isolation then reuses it for the build instead
# of resolving a second copy. See tests/smoke_tests/Dockerfile.
pip install --index-url https://download.pytorch.org/whl/cpu torch
pip install -e . --no-build-isolation --no-cache-dir
python setup.py build_ext --inplace --force

- name: Test simulator performance
Expand Down
148 changes: 148 additions & 0 deletions .github/workflows/fork-only-view.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
name: Fork-only view

# Fork-only. Verifies that the `fork-only/3.0` view branch still matches the single
# `chore(fork):` commit on `integration/3.0`, and that its standing draft PR is intact.
#
# It deliberately does NOT publish the branch. The fork base is made almost entirely of
# files under `.github/workflows/`, and GitHub blocks `GITHUB_TOKEN` from pushing any
# commit that creates or updates a workflow file — a platform rule no `permissions:`
# block can lift. Publishing therefore happens by hand, in the two procedures that
# already force-push with human credentials (see .github/fork/PROCESS.md). This job is
# the detector that makes drift loud, and it needs no write access at all.
#
# The checks themselves live in .github/fork/forkctl.py so that CI, humans and agents
# run the same tested code. This file only decides *when* to run them.
#
# Residual trust, stated plainly. A workflow cannot be a tamper-proof monitor of the
# same repository that stores it: on `push`, GitHub runs the workflow *definition* from
# the pushed ref, so a tampered `fork-only/3.0` can neuter or delete this file before the
# trusted checkout above ever executes. The backstop is the `schedule:` trigger, which
# GitHub always runs from the **default branch** (`integration/3.0`) regardless of what
# any other branch contains -- so tampering is caught within a day even if the push-time
# run was suppressed. `integration/3.0` policing itself is the irreducible core: it is
# branch-protected, and section 3.3 / section 7 are the only procedures permitted to
# force-push it.
#
# Never cherry-pick this workflow upstream.

on:
push:
branches:
- 'integration/3.0'
- 'fork-only/3.0'
schedule:
- cron: '41 5 * * *'
workflow_dispatch:
inputs:
stack:
description: 'Stack ref to analyse'
required: false
default: 'origin/integration/3.0'
mirror:
description: 'Mirror ref to analyse'
required: false
default: 'origin/3.0'

permissions:
contents: read
pull-requests: read

concurrency:
group: fork-only-view
cancel-in-progress: false

jobs:
verify:
# The name itself carries the verdict's authority: a manual dispatch runs
# candidate detector code and must never read as the production check in
# the Actions list, where often only the job name and a green tick are seen.
name: >-
${{ github.event_name == 'workflow_dispatch'
&& 'ADVISORY -- candidate detector, not authoritative'
|| 'Check fork-only/3.0 is current' }}
runs-on: ubuntu-latest
steps:
# The detector must not come from the ref it polices. A push to
# `fork-only/3.0` is exactly the tampering this job exists to catch, so the
# code is taken from `integration/3.0` -- a protected branch -- and only a
# human-driven `workflow_dispatch` may test a candidate ref instead.
- name: Checkout the detector
uses: actions/checkout@v4
with:
ref: ${{ github.event_name == 'workflow_dispatch' && github.ref || 'integration/3.0' }}
fetch-depth: 0

- name: Fetch the refs under analysis
run: |
git fetch --no-tags origin \
'+refs/heads/3.0:refs/remotes/origin/3.0' \
'+refs/heads/integration/3.0:refs/remotes/origin/integration/3.0'

# On push and schedule the production refs are hardcoded, so a dispatch
# input can never make the authoritative verdict describe something else.
- name: Resolve the refs under analysis
id: refs
env:
EVENT: ${{ github.event_name }}
STACK_INPUT: ${{ inputs.stack }}
MIRROR_INPUT: ${{ inputs.mirror }}
run: |
if [ "$EVENT" = "workflow_dispatch" ]; then
echo "stack=${STACK_INPUT:-origin/integration/3.0}" >> "$GITHUB_OUTPUT"
echo "mirror=${MIRROR_INPUT:-origin/3.0}" >> "$GITHUB_OUTPUT"
else
echo "stack=origin/integration/3.0" >> "$GITHUB_OUTPUT"
echo "mirror=origin/3.0" >> "$GITHUB_OUTPUT"
fi

- name: Report what is being checked
env:
EVENT: ${{ github.event_name }}
STACK: ${{ steps.refs.outputs.stack }}
MIRROR: ${{ steps.refs.outputs.mirror }}
DETECTOR: ${{ github.event_name == 'workflow_dispatch' && github.ref || 'integration/3.0' }}
run: |
if [ "$EVENT" = "workflow_dispatch" ]; then
AUTHORITY="advisory (candidate detector)"
else
AUTHORITY="authoritative"
fi
{
echo "### Fork-only view check"
echo ""
echo "| what | ref |"
echo "| --- | --- |"
echo "| authority | $AUTHORITY |"
echo "| detector code from | \`$DETECTOR\` |"
echo "| stack analysed | \`$STACK\` |"
echo "| mirror analysed | \`$MIRROR\` |"
} >> "$GITHUB_STEP_SUMMARY"
if [ "$EVENT" = "workflow_dispatch" ]; then
{
echo ""
echo "> **This run is advisory.** A dispatch executes the detector from"
echo "> \`$DETECTOR\`, which is candidate code, and may analyse non-production"
echo "> refs. Only \`push\` and \`schedule\` runs are authoritative."
} >> "$GITHUB_STEP_SUMMARY"
fi

- name: Check the stack, the view branch and the standing PR
env:
GH_TOKEN: ${{ github.token }}
STACK: ${{ steps.refs.outputs.stack }}
MIRROR: ${{ steps.refs.outputs.mirror }}
run: |
python .github/fork/forkctl.py check-view \
--stack "$STACK" --mirror "$MIRROR" --format github

self-test:
name: forkctl self-test (candidate ref)
runs-on: ubuntu-latest
steps:
# Advisory only: this deliberately runs the *candidate* code, so it proves
# the tool is sound, never that the repository state is.
- name: Checkout the triggering ref
uses: actions/checkout@v4

- name: Run the forkctl unit suite
run: python .github/fork/test_forkctl.py
16 changes: 14 additions & 2 deletions .github/workflows/install.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ name: Installation Test

on:
pull_request:
branches: [ main, '3.0' ]
branches: [ main, '3.0', 'integration/3.0' ]
push:
branches: [ main, '3.0' ]

Expand Down Expand Up @@ -38,7 +38,19 @@ jobs:
sudo apt-get clean
sudo rm -rf ~/.cache/pip /tmp/* /var/tmp/*

- name: Install pufferlib
- name: Install pufferlib (Linux)
if: runner.os == 'Linux'
run: |
python -m pip install -U pip "setuptools<81" wheel numpy
# CPU-only torch. The CUDA stack is ~3 GB of wheels and no longer
# fits the runner's disk quota; see the Install steps in ci.yml.
pip install --index-url https://download.pytorch.org/whl/cpu torch
pip install -e . --no-build-isolation

- name: Install pufferlib (macOS)
if: runner.os != 'Linux'
run: |
python -m pip install -U pip
# Deliberately a plain install: macOS wheels are already CPU-only, so
# this keeps the default PEP 517 build-isolation path under test.
pip install -e .
Loading
Loading