Skip to content

fix(drills): live testing refinements, CI/CD with Lula runtime gate, and docs alignment - #2

Merged
celttechie merged 5 commits into
mainfrom
fix/live-cluster-tunnel-and-health-checks
Nov 9, 2026
Merged

celttechie merged 5 commits into
mainfrom
fix/live-cluster-tunnel-and-health-checks

Conversation

@celttechie

@celttechie celttechie commented Nov 9, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • CI/CD Compliance & Quality Gates (.github/workflows/ci.yml):
    • Tier 1 (Shift-Left Schema & Linting): Runs lula tools lint for OSCAL 1.1.2 schema compliance, ShellCheck on scripts, Ruff & py_compile on Python modules, multi-document YAML validation, and Gitleaks secret scanning.
    • Tier 2 (Static Compliance): Enforces NIST SP 800-53 controls (AC-3, AC-4, SC-8, CM-8) statically on manifests in tests/test_drill_manager.py.
    • Tier 3 (KinD Ephemeral Runtime Gate): Deploys workloads to an ephemeral KinD cluster and executes live lula validate against DoD IL5 controls on every pull request.
  • Auto-Recovering K3s API Tunnel (scripts/common.py): Automatically establishes the SSH port-forward (127.0.0.1:16443 -> 10.160.0.34:6443 via t5600) when kubeconfig targets the local tunnel endpoint and the tunnel is not yet listening.
  • Workload-Aware Namespace Detection (scripts/common.py): Checks for the order-api Deployment rather than only namespace existence so empty legacy namespaces do not mask the active workload namespace.
  • Rollout-Safe Health Checks (scripts/drill_manager.py): Ignores Terminating pods from superseded ReplicaSets during check_cluster_health() and evaluates Lula against a /tmp copy to keep git status clean.
  • Workstation-to-Sandbox Air-Gap Verification (scripts/verify-airgap.sh): Automatically delegates make verify-airgap over SSH (-J t5600) into airgapped-sandbox-hypervisor when invoked from the workstation.
  • Documentation Alignment: Updated README.md, docs/design/platform-bundle-architecture.md, and docs/adr/0003-dod-il5-compliance-and-zero-trust-secrets.md to reflect CI/CD, the 3-Tier Lula compliance model, and the troubleshooting lab.

Test Plan

  • Booted airgapped-sandbox-hypervisor on t5600 (10.160.0.34)
  • Verified make verify-airgap passes all zero-egress isolation checks inside the VM
  • Verified make compliance and make drill-check pass all 5 NIST SP 800-53 Rev 5 controls (ac-3, ac-4, ia-2, sc-8, cm-8)
  • Live-tested scenario injection, verification, health check detection, and healing (net-service-endpoint, sec-lula-drift, zarf-webhook-bypass)
  • Verified make lint passes ShellCheck, Python compile, YAML schemas, and lula tools lint
  • Verified make test passes all automated unit, schema, and shift-left compliance tests
  • GitHub Actions CI workflow running on PR

…health checks

- Auto-establish SSH port-forward (127.0.0.1:16443 -> 10.160.0.34:6443 via t5600) in scripts/common.py
- Detect training workload namespace by checking for the order-api deployment
- Ignore Terminating pods from old ReplicaSets during check_cluster_health()
- Auto-delegate verify-airgap.sh over SSH to airgapped-sandbox-hypervisor when run from workstation
…lign documentation

- Add .github/workflows/ci.yml with static-checks (lula tools lint, ShellCheck, Ruff, YAML) and ephemeral KinD runtime compliance validation
- Add shift-left DoD IL5 compliance assertions in tests/test_drill_manager.py
- Add make lint target in Makefile
- Update README.md, ADR-0003, and design specs to reflect CI/CD, Lula compliance architecture, and CLI troubleshooting lab
@celttechie celttechie changed the title fix(drills): auto-establish t5600 SSH tunnel and refine live cluster health checks fix(drills): live testing refinements, CI/CD with Lula runtime gate, and docs alignment Nov 9, 2026
@celttechie
celttechie merged commit 3747389 into main Nov 9, 2026
4 checks passed
@celttechie
celttechie deleted the fix/live-cluster-tunnel-and-health-checks branch November 9, 2026 22:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant