Skip to content

Audit hash chain, IPFS pinning, dynamic limits, OTel - #1732

Merged
ayomideadeniran merged 2 commits into
StellarDevHub:mainfrom
nafiisatu:handsoff/issues-1581-1583-1584-1585-d7d94d
Oct 22, 2026
Merged

ayomideadeniran merged 2 commits into
StellarDevHub:mainfrom
nafiisatu:handsoff/issues-1581-1583-1584-1585-d7d94d

Conversation

@nafiisatu

Copy link
Copy Markdown
Contributor

Closes #1581, closes #1583, closes #1584, closes #1585

#1581 — BE-009: Tamper-evident audit hash chain + Stellar anchoring

  • backend/src/db/auditChainStore.ts: stores the chain in the new audit_chain_entries and audit_chain_anchors tables. Appends run under a PostgreSQL advisory lock, so concurrent instances always link to the real head. The app assigns sequence numbers itself (head + 1), so a deleted row always leaves a gap. An in-memory store is included for tests.

  • backend/src/services/tamperEvidentAuditService.ts: each entry's entryHash = SHA-256(prevHash ‖ canonicalJSON(entry)), starting from an all-zero genesis hash. verifyChain() reads the chain in batches with one hash per entry, so it is designed to stay under 1s per 10k rows. It detects:

    • modified rows (HASH_MISMATCH)
    • deleted or inserted rows (SEQUENCE_GAP, CHAIN_BREAK)
    • tail truncation (TRUNCATED)
    • a chain rewritten consistently from start to end (ANCHOR_MISMATCH, checked against the latest Stellar anchor)

    On any failure it raises a critical alarm through a structured CRITICAL log, Sentry, a tamper event and an optional AUDIT_ALARM_WEBHOOK_URL.

  • backend/src/jobs/anchorMerkleRootJob.ts:

    • Verifies the chain first and refuses to anchor one that fails.
    • Computes the Merkle root of the entries added since the last anchor.
    • Publishes the root as the Stellar transaction's MEMO_HASH and in a manageData entry named w3sl_audit_root.
    • verifyPublishedAnchor() reads the memo back from Horizon and compares it with a fresh local computation.
    • startAuditChainJobs() schedules routine verification and anchoring when AUDIT_CHAIN_JOBS_ENABLED=true.
  • The migration adds an append-only trigger to audit_chain_entries as an extra safeguard. The hash chain and the anchors still catch anyone who drops the trigger.

  • logAudit() now also writes every existing audit event into the chain.

  • New admin endpoints:

    • GET /api/v1/audit/chain/verify
    • GET /api/v1/audit/chain/anchors
    • GET /api/v1/audit/chain/anchors/:id/verify

#1583 — BE-011: IPFS/Filecoin dual-pinning pipeline

  • backend/src/config/ipfs.ts: environment-driven settings for:
    • providers: Pinata → Kubo for fast IPFS pinning, Web3.Storage for Filecoin storage deals
    • gateways: the Cloudflare CDN gateway first, then the secondary gateways
    • retry and backoff limits
    • the 3s certificate pinning deadline
  • backend/src/services/ipfsPinningService.ts:
    • Pins through each IPFS provider with exponential backoff and jitter, then falls back to the next provider.
    • Strictly validates CIDv1 (base32, version 1, sha2-256 multihash) and treats any other identifier as a provider failure.
    • Stores CIDs in the new pinned_assets table. A trigger blocks any rewrite of a stored CID.
    • In the background, uploads the same content to Filecoin and checks it can be fetched through the CDN gateway (which also warms the cache), falling back to the secondary gateways with backoff.
    • autoPinCertificate() pins a rendered certificate SVG plus the metadata JSON (with image: ipfs://<svgCid>) within a 3s deadline. CertificateService calls it right after minting.
  • backend/src/controllers/mediaController.ts and routes/media.routes.ts, mounted at /api/v1/media:
    • pin an asset
    • look up a CID, with an optional redirect to the CDN
    • re-check gateway access
    • list a certificate's pinned assets

#1584 — BE-012: Sliding-window limiter with IP/DID reputation

  • backend/src/utils/slidingWindow.ts: a Redis sorted-set limiter that keeps a timestamp per request, so there are no fixed-window boundary spikes. The burst and sustained windows are checked in one atomic Lua call: a request either counts against every window or against none, so rejected requests use no quota. Keys use a hash tag so all of a caller's windows land on the same Redis Cluster node. If Redis fails, it falls back to an in-memory limiter.

  • backend/src/services/reputationService.ts: scores each caller from these signals:

    Signal Points
    Authenticated +40
    Has a DID +20
    Verified student (has an active certificate) +20
    Active stake (Stellar asset balance via Horizon) +40
    Each recent violation −10

    The score sets the tier: restricted → anonymous → authenticated → verified → staker. Only authenticated DIDs with an active stake reach staker. Profiles are cached for 60s.

  • backend/src/middleware/dynamicRateLimiter.ts: applies the caller's tier limits on every response.

    • Response headers: X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-RateLimit-Tier and RateLimit-Policy.
    • Over the limit: HTTP 429 with Retry-After, and the violation is recorded so repeat offenders drop a tier.
    • Mounted on /api. Disable it with DYNAMIC_RATE_LIMIT_ENABLED=false; it is skipped under NODE_ENV=test.

#1585 — BE-013: OpenTelemetry tracing with Jaeger and Prometheus

  • backend/src/instrumentation.ts:
    • Starts @opentelemetry/sdk-node and exports traces over OTLP/HTTP to Jaeger (OTEL_EXPORTER_JAEGER_ENDPOINT).
    • Provides W3C trace-context helpers: injectTraceContext, extractTraceContext, runInSpan, traceEnqueue, traceJob.
    • When Sentry is enabled, it now passes skipOpenTelemetrySetup so the OTel SDK owns the global tracer.
  • backend/src/middleware/tracingMiddleware.ts:
    • Opens a SERVER span per request, continuing an incoming traceparent, and returns a traceresponse header.
    • Records its own cost in w3sl_tracing_overhead_seconds so the < 1.5ms budget can be checked.
  • End-to-end trace from HTTP request to background job:
    • compileQueue.enqueue adds the trace context to job data inside a PRODUCER span.
    • compileWorker runs each job as a CONSUMER child span.
    • Redis Pub/Sub (createWsSync) carries _trace in each message envelope and continues it on receive.
  • backend/src/metrics/prometheusExporter.ts: dependency-free histograms and counters in the standard Prometheus text format, appended to the existing /api/v1/metrics/prometheus scrape:
    • w3sl_http_server_request_duration_seconds
    • w3sl_tracing_overhead_seconds
    • w3sl_queue_job_duration_seconds
    • w3sl_pubsub_messages_total

Notes

  • Adds the @opentelemetry/* dependencies to backend/package.json. Lockfiles were not regenerated, since nothing was installed or run.
  • New migration 20261020000000_audit_chain_and_pinned_assets and matching Prisma models (PinnedAsset, AuditChainEntry, AuditChainAnchor).
  • New environment variables are documented in backend/.env.example.
  • None of this was built or tested.

Closes #1581
Closes #1583
Closes #1584
Closes #1585

@vercel

vercel Bot commented Oct 20, 2026

Copy link
Copy Markdown

@nafiisatu is attempting to deploy a commit to the Ayomide Adeniran's projects Team on Vercel.

A member of the Team first needs to authorize it.

@drips-wave

drips-wave Bot commented Oct 20, 2026

Copy link
Copy Markdown

@nafiisatu Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@ayomideadeniran

Copy link
Copy Markdown
Contributor

Thank you for your contribution! This PR is currently under review. If I find any incorrect implementation or issues, I will leave comments on what needs to be adjusted next.

@ayomideadeniran
ayomideadeniran merged commit 255a469 into StellarDevHub:main Oct 22, 2026
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

2 participants