Skip to content

feat: add golden output snapshots for supported languages (#1253) - #1286

Open
p3ris0n wants to merge 1 commit into
MDTechLabs:mainfrom
p3ris0n:feat/add-golden-output-snapshots-for-supported-languages-1253
Open

p3ris0n wants to merge 1 commit into
MDTechLabs:mainfrom
p3ris0n:feat/add-golden-output-snapshots-for-supported-languages-1253

Conversation

@p3ris0n

@p3ris0n p3ris0n commented Oct 19, 2026 •

Copy link
Copy Markdown

Summary

Closes #1253
Closes #1249
Closes #1248
Closes #1247

This PR extends GasGuard's repository analysis platform across both the Quality and Evaluation and API and Integrations workstreams. It introduces golden output snapshot verification across supported languages, a labeled benchmark corpus for analyzer rules, API deprecation and sunset policy enforcement with lifecycle headers, and end-to-end request correlation identifiers across API, worker, and RPC boundaries.


Key Implementations

1. Golden Output Snapshots for Supported Languages (Closes #1253)

  • Deterministic Evaluator: Canonical finding sorting (file -> startLine -> startColumn -> ruleId -> message) and deterministic 16-char hex fingerprinting across multi-language scans (Solidity, Soroban Rust, Vyper, Cairo, TypeScript).
  • Multi-Language Diff Engine: Computes structured diffs (added, removed, modified) across finding lists, files analyzed, and severity summaries.
  • Tenant Isolation & Scan Pinning: Storage partitioned strictly by tenantId; write-once scan pins preserve the comparison snapshot across retries.
  • Fixtures & Documentation: Added realistic multi-language repository fixture (realistic-repo-fixture.json), regression test fixture (regression-case-fixture.json), and documentation in docs/GOLDEN_OUTPUT_SNAPSHOTS.md.

2. Labeled Benchmark Corpus for Analyzer Rules (Closes #1249)

  • Corpus Schema & Catalog: Established canonical benchmark dataset schema with ground-truth labeled findings (true positives, true negatives, gas savings annotations, and CWE/SWC vulnerability mappings).
  • Benchmark Evaluator: Calculates precision, recall, F1 score, and false positive rates per rule and language under fixed rule-pack versions.
  • Deterministic Baselines: Regression evaluation compares rule revisions against corpus baselines to detect drop in detection quality or surge in false positives.
  • Fixtures & Documentation: Included representative rule corpus suites and complete runbook in docs/ANALYZER_BENCHMARK_CORPUS.md.

3. API Deprecation and Sunset Policy (Closes #1248)

  • Lifecycle Headers Middleware: Implements standard HTTP deprecation headers (Deprecation: @<timestamp>, Sunset: <date>, and Link: <replacement>; rel="successor-version").
  • Policy Registry & Governance: Structured, versioned deprecation schedules preventing sudden sunset transitions without mandatory notice periods.
  • Telemetry & Alerts: Emits bounded Prometheus metrics (gasguard_api_deprecated_route_requests_total) and structured warnings for client migration tracking.
  • Contract & Documentation: Published formal API deprecation and sunset policy in docs/API_DEPRECATION_POLICY.md.

4. API Request Correlation Identifiers (Closes #1247)

  • Correlation Middleware: Ingests incoming X-Correlation-ID / X-Request-ID headers or generates standardized UUIDv4 correlation IDs on ingress.
  • Propagation Boundary: Injects correlation identifiers into AsyncLocalStorage context, outgoing HTTP/RPC client calls, and background queue job metadata (scanId, correlationId).
  • Sanitized Structured Logging: Automatically attaches correlationId to all log records while strictly redacting auth headers, tokens, private keys, and passwords.
  • Egress Header: Mirrors X-Correlation-ID in all API responses to simplify client-side tracing and support debugging.

Architectural & Security Compliance

  • Tenant Isolation: All caches, storage paths (<baseDir>/<sanitized_tenantId>/...), and evaluation contexts are isolated by tenantId.
  • Zero Secrets in Logs: Context sanitization scrubs tokens (ghp_*), JWTs, private keys, and repository source payloads.
  • Determinism: Identical commits, configurations, and rule-pack versions yield byte-identical evaluation fingerprints.

Verification Evidence & Test Coverage

  • Unit Tests:
    • golden-snapshot-evaluator.spec.ts (canonicalization, hashing, diffing)
    • rule-benchmark-evaluator.spec.ts (precision/recall calculations, baseline checks)
    • api-deprecation.middleware.spec.ts (header injection, sunset expiration dates)
    • correlation-id.middleware.spec.ts (header validation, generation, propagation)
  • Integration Tests:
    • golden-snapshot.integration.spec.ts (file-store persistence, worker task execution)
    • correlation-id.integration.spec.ts (end-to-end trace propagation across API -> Queue -> Worker)
  • Fixtures: Verified against multi-language DEX repositories and regression suites.

@drips-wave

drips-wave Bot commented Oct 19, 2026

Copy link
Copy Markdown

@p3ris0n Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant