Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions .github/workflows/resources.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ on:
- "resources/**"
- "tests/**"
- "scripts/check-resources.py"
- "scripts/verify-oee-export.py"
- "scripts/render-resource-guides.py"
- "scripts/new-course-workspace"
- ".github/workflows/resources.yml"
Expand All @@ -14,6 +15,7 @@ on:
- "resources/**"
- "tests/**"
- "scripts/check-resources.py"
- "scripts/verify-oee-export.py"
- "scripts/render-resource-guides.py"
- "scripts/new-course-workspace"
- ".github/workflows/resources.yml"
Expand All @@ -30,11 +32,12 @@ jobs:
repository: 0xbuffdev/open-engineering-education
path: _oee
persist-credentials: false
- name: Validate catalogue only (no active v1 maps)
run: python3 scripts/check-resources.py
fetch-depth: 0
- name: Validate all maps against exact source commits
run: python3 scripts/check-resources.py --oee _oee
- name: Reject stale generated reading guides
run: python3 scripts/render-resource-guides.py --check
- name: Strict v2 shape and pinned export regression suite
run: python3 scripts/render-resource-guides.py --oee _oee --check
- name: Strict v2 and pinned source export regression suite
run: python3 -m unittest discover -s tests -p 'test_*.py' -v
- name: Verify unmapped scaffolding and preserve learner files
shell: bash
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ A completion claim should be supported by inspectable evidence, not by a checked

The [public resource library](resources/README.md) contains optional book and online resources and a strict v2 mapping schema. Its historical SEF1 mapping and guide have been retired because they describe a different curriculum. No reviewed v2 course mapping or guide is currently published.

Courses currently initialize **without a reading-plan snapshot** while the new v2 map undergoes pinned-source and human editorial review. Learner-owned `resources.md` remains available, and existing learner directories are never overwritten. Public resources stay independent of private 0xNAND content.
Courses currently initialize **without a reading-plan snapshot** while the new v2 map undergoes pinned-source and human editorial review. Once a map exists, its validator resolves the exact OEE Git commit recorded in that map, not the checkout's moving HEAD. Learner-owned `resources.md` remains available, and existing learner directories are never overwritten. Public resources stay independent of private 0xNAND content.

## Evidence and reproducibility

Expand Down
8 changes: 4 additions & 4 deletions resources/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,11 @@ python3 -m unittest discover -s tests -p 'test_*.py' -v
python3 scripts/render-resource-guides.py --check
~~~

For a newly reviewed v2 map and an independently generated pinned OEE Toolkit export, use:
For a future reviewed v2 map, validate against its **exact local OEE Git commit**:

~~~sh
python3 scripts/check-resources.py --course SEF1 --export /path/to/pinned-SEF1-export.json
python3 scripts/render-resource-guides.py --course SEF1 --export /path/to/pinned-SEF1-export.json --stdout
python3 scripts/check-resources.py --course SEF1 --oee /path/to/open-engineering-education --json
python3 scripts/render-resource-guides.py --course SEF1 --oee /path/to/open-engineering-education --stdout
~~~

The export consumer checks source identity, module/lesson locators and outcome/assessment references. The caller must independently prove that the export was derived from the named Git commit; a JSON claim alone is not cryptographic provenance. Source lookup and end-to-end binding remain WS-I01; editorial resource review is WS-I02; safe final publishing and regeneration are WS-I04. A map presented without its pinned export **fails closed as source_missing**. No new course guide has yet been accepted.
The resource validator now invokes the public OEE Toolkit directly on the map's recorded full Git commit and checks the derived export's exact source identity, modules, lessons, outcomes and assessments. When the commit is missing or unsupported, it returns **incomplete** (exit `3`) rather than using `HEAD`. The selected local checkout must be trusted; Git object IDs do not independently authenticate repository origin. Human bibliography/edition review remains separate (WS-I02), as does safe mapped-guide publishing (WS-I04). No current SEF1 map or reading guide has been accepted.
37 changes: 19 additions & 18 deletions scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,15 +32,9 @@ version, wrong SHA, unknown assessment, missing outcome coverage or changed
module/lesson locator **fails explicitly**. There is no v1 reader,
conversion process or fallback to the checkout's moving HEAD.

This export consumer is a narrow reference-checking seam, not an editorial review. WS-I03 now provides one active v2 schema, and historical v1 SEF1 data is only a negative fixture. WS-I01 will wire exact source Git-object lookup into the validator, WS-I02 will author a new reviewed map, and WS-I04 will harden guide publishing. No active course mapping is currently approved.
This standalone export consumer is a narrow interface check, not proof of source provenance or an editorial review. WS-I01 now invokes the public OEE Toolkit on each v2 map's exact locally available Git commit, then checks the freshly derived export with this consumer. Old schema v1 is rejected. WS-I02 will author a reviewed map and WS-I04 will harden guide publishing. No active reviewed map has been published yet.

The source revision text in an exported file is **not cryptographic proof**
by itself. For trusted provenance, derive exports from the actual selected
OEE Git commit with the public Toolkit, and compare the generated bytes
against the pinned golden fixtures in that repository. This consumer does
not fetch commits or accept arbitrary source revision claims as verified.
Passing it establishes **interface consistency only**, not pedagogical
conformance, learner evidence or resource/editorial review.
Do not use a standalone, externally provided export as proof of source provenance. The Workspace resource validator runs the public OEE Toolkit against the selected local Git commit, then checks the export. Its SHA identifies local bytes but does not authenticate the origin of an untrusted checkout. These checks establish structural consistency, **not** educational conformance, resource verification or learner achievement.

## Prerequisites

Expand Down Expand Up @@ -125,19 +119,26 @@ Test generation in a disposable directory before relying on a changed script in

## Public reading plans

There is currently **no active reviewed map or public study guide**. The retired SEF1 v1 mapping must not be used for the redesigned OEE modules. Unmapped course scaffolding creates notes, labs, exercises, and an editable resources log without a reading plan.
There is currently no reviewed active v2 course mapping. Unmapped course scaffolding
creates independent learner notes and resources logs without a reading plan.
The old SEF1 v1 map is a negative regression fixture, never current study advice.

For mapped courses, validation now invokes the **single OEE Toolkit parser** on
the map's full Git SHA from a local OEE checkout. Neither the worktree nor HEAD
is a fallback. No v1 reader, automatic remapping or arbitrary supplied export
is accepted by the full resource validator.

```sh
python3 scripts/check-resources.py
python3 scripts/check-resources.py --oee /path/to/open-engineering-education
python3 scripts/check-resources.py --oee /path/to/open-engineering-education --course SEF1 --json
python3 scripts/render-resource-guides.py --oee /path/to/open-engineering-education --course SEF1 --stdout
python3 -m unittest discover -s tests -p 'test_*.py' -v
python3 scripts/render-resource-guides.py --check
```

For a future reviewed v2 map, validation requires the exact pinned Toolkit export:

```sh
python3 scripts/check-resources.py --course SEF1 --export /path/to/pinned-export.json
python3 scripts/render-resource-guides.py --course SEF1 --export /path/to/pinned-export.json --stdout
```
The resource-check exit codes are `0` for structurally passed, `1` for
invalid references and `3` for missing or unsupported pinned sources.
With no active maps, the command checks the public catalogue only. This
does not imply a reviewed curriculum mapping or learner certification.

Without a source export, mapped validation and guide generation fail closed; there is no fallback to the checkout's HEAD. Automatic commit resolution and review/publishing gates belong to WS-I01, WS-I02, and WS-I04. The generator continues to protect existing learner directories.
WS-I02 owns SEF1 human resource remapping, WS-I04 owns publication-safe
guide writes, and WS-I11 owns remaining upstream drift notifications.
132 changes: 108 additions & 24 deletions scripts/check-resources.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,10 @@
from __future__ import annotations

import argparse
import functools
import importlib.util
import os
import subprocess
import json
import re
import sys
Expand All @@ -29,13 +32,17 @@ class Check:
def __init__(self):
self.errors: list[str] = []
self.warnings: list[str] = []
self.incomplete: list[str] = []

def error(self, location: str, msg: str):
self.errors.append(f"{location}: {msg}")

def warn(self, location: str, msg: str):
self.warnings.append(f"{location}: {msg}")

def missing(self, location: str, msg: str):
self.incomplete.append(f"{location}: {msg}")


def text(value):
return isinstance(value, str) and bool(value.strip())
Expand Down Expand Up @@ -233,6 +240,7 @@ def validate_map_shape(mapping, catalog, check, loc):
return len(check.errors) == before


@functools.lru_cache(maxsize=1)
def load_consumer():
name = "oee_workspace_export_consumer"
spec = importlib.util.spec_from_file_location(name, ROOT / "scripts" / "verify-oee-export.py")
Expand All @@ -242,18 +250,74 @@ def load_consumer():
return module


def validate_maps(catalog, check, course_filter=None, export_path=None):
class PinnedSourceError(Exception):
"""Typed failure to obtain a supported immutable OEE course export."""

def __init__(self, category: str, detail: str):
super().__init__(detail)
self.category = category


def read_pinned_export(oee: Path, revision: str, course: str) -> dict:
"""Ask the single public Toolkit parser to read the map's local Git commit.

No HEAD, worktree, network or user-supplied JSON export is a substitute.
"""
if not isinstance(revision, str) or not SHA.fullmatch(revision):
raise PinnedSourceError("unsupported_format", "full lowercase commit SHA required")
if not isinstance(course, str) or not CODE.fullmatch(course):
raise PinnedSourceError("invalid_reference", "invalid course code")
repo = oee.expanduser().resolve()
if not repo.is_dir():
raise PinnedSourceError("source_missing", "local OEE Git checkout not found")
argv = [
sys.executable, "-m", "tooling.oee_toolkit", "export",
"--repo", str(repo), "--revision", revision,
"--course", course, "--output", "-",
]
try:
result = subprocess.run(argv, cwd=repo, capture_output=True,
timeout=90, check=False)
except (OSError, subprocess.TimeoutExpired) as exc:
raise PinnedSourceError("tool_error", f"Toolkit execution: {type(exc).__name__}") from exc
if result.returncode:
stderr = result.stderr.decode("utf-8", errors="replace")
detail = " ".join(stderr.strip().splitlines()[:3])[:500] or "Toolkit export failed"
if result.returncode == 3:
category = "unsupported_format" if "unsupported_format" in stderr else "source_missing"
elif result.returncode == 1:
category = "invalid_reference"
else:
category = "tool_error"
raise PinnedSourceError(category, detail)
try:
exported = json.loads(result.stdout.decode("utf-8"))
except (UnicodeError, ValueError) as exc:
raise PinnedSourceError("tool_error", "invalid Toolkit export JSON") from exc
if not isinstance(exported, dict):
raise PinnedSourceError("tool_error", "Toolkit export must be an object")
return exported


def check_pinned_map(mapping: dict, oee: Path) -> dict:
canon = mapping["curriculum"]
consumer = load_consumer()
exported = read_pinned_export(oee, canon["revision"], canon["course"])
checked = consumer.check_export(exported, canon["revision"])
consumer.check_map(mapping, checked)
return checked


def validate_maps(catalog, check, course_filter=None, oee=None):
paths = sorted(MAPPINGS.rglob("*.json"))
if course_filter:
paths = [path for path in paths if path.stem == course_filter]
paths = [p for p in paths if p.stem == course_filter]
if not paths:
if course_filter:
check.error("mappings", f"unmapped course {course_filter}; no active v2 map")
return
if export_path is not None and len(paths) != 1:
check.error("mappings", "--export requires exactly one selected map (--course CODE)")
return
consumer = load_consumer() if export_path is not None else None
source = oee or Path(os.environ.get(
"OEE_REPO", str(ROOT.parent / "open-engineering-education")))
for path in paths:
loc = str(path.relative_to(ROOT))
mapping = load_json(path, check)
Expand All @@ -265,32 +329,52 @@ def validate_maps(catalog, check, course_filter=None, export_path=None):
if path.parent.name != program or path.stem != course:
check.error(loc, f"map path must be mappings/{program}/{course}.json")
continue
if consumer is None:
check.error(loc, "source_missing: exact pinned OEE export required; automatic Git resolution pending WS-I01")
continue
try:
verified = consumer.check_export(consumer.load_json(export_path), canon["revision"])
consumer.check_map(mapping, verified)
except consumer.ContractError as exc:
check.error(loc, f"{exc.category}: {exc.code}: {exc}")
check_pinned_map(mapping, source)
except PinnedSourceError as exc:
if exc.category in ("source_missing", "unsupported_format"):
check.missing(loc, f"{exc.category}: {exc}")
else:
check.error(loc, f"{exc.category}: {exc}")
except load_consumer().ContractError as exc:
if exc.category in ("missing_input", "unsupported_format"):
check.missing(loc, f"{exc.category}: {exc.code}: {exc}")
else:
check.error(loc, f"{exc.category}: {exc.code}: {exc}")


def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--course", help="check one active v2 course map")
parser.add_argument("--export", type=Path, help="independently generated pinned OEE export-format-v1 JSON")
parser.add_argument("--oee", type=Path, default=Path(os.environ.get(
"OEE_REPO", str(ROOT.parent / "open-engineering-education"))),
help="trusted local OEE Git checkout; never fetches")
parser.add_argument("--course", help="check one active v2 map")
parser.add_argument("--json", action="store_true", help="machine-readable diagnostics")
args = parser.parse_args()
check = Check()
catalog = validate_catalog(check)
validate_maps(catalog, check, args.course, args.export)
for warning in check.warnings:
print(f"WARNING: {warning}", file=sys.stderr)
for error in check.errors:
print(f"ERROR: {error}", file=sys.stderr)
print(f"Resource check: {len(catalog)} catalogue IDs, {len(check.warnings)} warnings, {len(check.errors)} errors")
if not list(MAPPINGS.rglob("*.json")):
print("No active v2 resource maps; catalog validation only.")
return 1 if check.errors else 0
validate_maps(catalog, check, args.course, args.oee)
status = "failed" if check.errors else "incomplete" if check.incomplete else "passed"
if args.json:
print(json.dumps({
"outcome": status, "errors": check.errors,
"incomplete": check.incomplete, "warnings": check.warnings,
"cataloguedResources": len(catalog),
"manualReviewRequired": True
}, sort_keys=True, separators=(",", ":")))
else:
for warning in check.warnings:
print(f"WARNING: {warning}", file=sys.stderr)
for missing in check.incomplete:
print(f"INCOMPLETE: {missing}", file=sys.stderr)
for error in check.errors:
print(f"ERROR: {error}", file=sys.stderr)
print(f"Resource check: {len(catalog)} catalogue IDs, "
f"{len(check.warnings)} warnings, {len(check.incomplete)} incomplete, "
f"{len(check.errors)} errors")
if not list(MAPPINGS.rglob("*.json")):
print("No active v2 resource maps; catalog validation only.")
return {"passed": 0, "failed": 1, "incomplete": 3}[status]


if __name__ == "__main__":
Expand Down
4 changes: 2 additions & 2 deletions scripts/new-course-workspace
Original file line number Diff line number Diff line change
Expand Up @@ -133,8 +133,8 @@ if [[ -f "$mapping_file" ]]; then
echo "Python 3 is required for public resource mapping." >&2
exit 1
fi
python3 "$workspace_repo_root/scripts/check-resources.py" --course "$course"
python3 "$workspace_repo_root/scripts/render-resource-guides.py" --course "$course" --stdout >/dev/null
python3 "$workspace_repo_root/scripts/check-resources.py" --oee "$oee_repo" --course "$course"
python3 "$workspace_repo_root/scripts/render-resource-guides.py" --oee "$oee_repo" --course "$course" --stdout >/dev/null
fi

mkdir -p "$lab_dir" "$exercise_dir" "$notes_dir"
Expand Down
14 changes: 7 additions & 7 deletions scripts/render-resource-guides.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

import argparse
import json
import os
import re
import subprocess
import sys
Expand Down Expand Up @@ -110,7 +111,7 @@ def render(mapping, catalog):
return "\n".join(lines)


def outputs(course_filter, export_path):
def outputs(course_filter, oee_repo):
catalog = load_catalog()
found = False
for path in sorted(MAPPINGS.rglob("*.json")):
Expand All @@ -121,11 +122,9 @@ def outputs(course_filter, export_path):
if course_filter and code != course_filter:
continue
found = True
if export_path is None:
raise ValueError("source_missing: mapped guides require a reviewed, pinned --export")
checked = subprocess.run(
[sys.executable, str(ROOT / "scripts" / "check-resources.py"),
"--course", code, "--export", str(export_path)],
"--course", code, "--oee", str(oee_repo)],
capture_output=True, text=True, check=False)
if checked.returncode:
raise ValueError("resource map failed pinned-source validation: " + checked.stderr.strip())
Expand All @@ -142,8 +141,9 @@ def main():
modes.add_argument("--check", action="store_true", help="verify all guides match their mappings")
modes.add_argument("--stdout", action="store_true", help="print one course guide")
parser.add_argument("--course", help="course code (required with --stdout)")
parser.add_argument("--export", type=Path,
help="independently generated pinned OEE Toolkit export for one selected map")
parser.add_argument("--oee", type=Path, default=Path(os.environ.get(
"OEE_REPO", str(ROOT.parent / "open-engineering-education"))),
help="local OEE Git checkout supplying the map's exact pinned commit")
args = parser.parse_args()
if args.stdout and not args.course:
parser.error("--stdout requires --course")
Expand All @@ -152,7 +152,7 @@ def main():
if args.check and args.course:
parser.error("--check validates all guides; omit --course")
try:
results = list(outputs(args.course, args.export))
results = list(outputs(args.course, args.oee))
if args.stdout:
if len(results) != 1:
raise ValueError("Expected exactly one mapped course")
Expand Down
Loading
Loading