Repository navigation
Technical Design Specification
This TDS implements the Product & Engineering Manual. Product behavior is not redefined here. Product decisions PD-003 through PD-017 that were closed for MVP are treated as system contracts; remaining open decisions remain configuration or rollout blockers where relevant.
- Agents interpret uncertain external evidence; deterministic services control financial invariants and publication.
- Every price decision is traceable to observations, normalized offers, mappings, market-price snapshot, policy versions, approval, publication, and verification.
- Policy/guardrail failure fails closed for automatic publication.
- Publication is idempotent and verified against authoritative commerce state.
- Missing/stale evidence never becomes invented competitor data.
- Probabilistic mappings do not auto-accept on raw model score; auto-accept requires a calibrated, evaluated release threshold.
| Contract | MVP value | Manual trace |
|---|---|---|
| Competitive strategy | Match lowest eligible comparable landed price; no undercut | PD-003/004 |
| Landed price | Item + mandatory shipping − unconditional public promotion; pre-tax by default | PD-007/008/009 |
| Margin floor |
max(variable_cost, variable_cost/(1-required_margin), MAP, regulatory_floor); default required contribution margin 10% |
PD-005 |
| Freshness | Tier A 15m; Tier B 60m; eligibility TTL 2× cadence capped 120m | PD-012 |
| Mapping auto-accept | Exact trusted IDs deterministic; probabilistic only at evaluated threshold achieving ≥99.5% precision | PD-013 |
| Auto price movement | decrease ≤5%; increase ≤10%; otherwise review | PD-014 |
| Approval | risk conditions in PD-015 | PD-015 |
| Processing SLO | P95 ≤15m, P99 ≤30m usable observation → verified publication | PD-016 |
| Availability | 99.9% monthly core decision/publish path | PD-017 |
flowchart LR
SRC[Approved Sources] --> COL[Market Intelligence / Collectors]
COL --> RAW[(Raw Evidence Store)]
COL --> BUS[(Event Bus)]
BUS --> EXT[Offer Extraction]
EXT --> NORM[Normalization]
NORM --> MAP[Product Identity / Mapper]
MAP -->|accepted| MP[Market Price Service]
MAP -->|review| HR[Human Review]
MP --> PE[Pricing Decision Engine]
PE --> GE[Governance / Guardrail Engine]
GE -->|allow| DEC[(Decision Store)]
GE -->|review| HR
GE -->|block| AUD[(Audit Log)]
HR -->|approve| DEC
DEC --> PUB[Publisher]
PUB --> COM[Commerce Platform]
COM --> VER[Publication Verifier]
VER --> AUD
RAW --> EVAL[Evaluation Service]
MAP --> EVAL
PE --> EVAL
VER --> MON[Monitoring]
Module boundaries are logical. MVP deployment may combine compatible modules into fewer services.
COLLECTION_PENDING
→ COLLECTED
→ EXTRACTED
→ NORMALIZED
→ MAPPING_PENDING
→ {MAPPED | MAPPING_REVIEW | MAPPING_REJECTED}
→ MARKET_PRICE_READY
→ PRICE_COMPUTED
→ POLICY_EVALUATED
→ {BLOCKED | REVIEW_REQUIRED | APPROVED}
→ PUBLISH_PENDING
→ PUBLISHING
→ PUBLISHED
→ VERIFYING
→ {VERIFIED | VERIFICATION_FAILED}
Terminal/recovery states: COLLECTION_FAILED, EXTRACTION_FAILED, STALE, PUBLISH_FAILED, ROLLBACK_PENDING, ROLLED_BACK, QUARANTINED.
{
"event_id": "uuid",
"event_type": "offer.normalized",
"schema_version": "1.0",
"correlation_id": "uuid",
"causation_id": "uuid",
"occurred_at": "ISO-8601",
"producer": "service-version",
"payload": {}
}Required fields: offer_id, observation_id, source_id, external_product_id, item_price, currency, mandatory_shipping, promotion_amount, promotion_eligibility, quantity, condition, availability, observed_at, evidence_refs, extractor_version.
Derived fields: landed_price, comparison_eligible, exclusion_reasons.
Required fields: mapping_id, offer_id, internal_sku, method, confidence_score, calibration_version, mapper_version, status, evidence, reviewed_by, reviewed_at.
status ∈ {AUTO_ACCEPTED, HUMAN_ACCEPTED, REVIEW_REQUIRED, REJECTED}.
Required fields: sku, market_price, winning_offer_id, eligible_offer_ids, excluded_offers_with_reasons, freshness_status, computed_at, policy_version.
Required fields: decision_id, sku, current_price, market_price, competitive_target, variable_cost, required_margin_pct, margin_floor_price, map_price, regulatory_floor, allowed_floor, proposed_price, change_pct, decision_reason, risk_flags, policy_version, status.
Collectors are source adapters. Prefer official APIs/feeds when available; public-web collection is source-specific and subject to approved policy.
Each observation MUST preserve source, URL/external ID, collection timestamp, raw evidence reference, collector version, response metadata, and status.
Scheduling policy:
- Tier A target cadence: 15m.
- Tier B target cadence: 60m.
- Scheduler applies jitter and source-specific rate limits.
- Freshness eligibility TTL =
min(2 * configured_cadence, 120m).
Collection failure emits a failure event; it never emits a valid offer with zero/null price.
Deterministic parsers are preferred. Probabilistic extraction/promotion interpretation is allowed behind a versioned model gateway when deterministic parsing is insufficient.
MVP landed price:
eligible_public_promotion = unconditional promotion available to a normal customer
landed_price = item_price + mandatory_shipping - eligible_public_promotion
Exclude from automatic market-lowest establishment: member/loyalty-only, personalized, payment-method-specific, subscription-only, checkout-code requiring unknown eligibility, ambiguous bundle economics, and non-equivalent quantity.
Tax is excluded unless it is an unavoidable seller-specific charge that can be normalized consistently within the target market.
Matching cascade:
- trusted GTIN/UPC/EAN exact match;
- approved manufacturer + MPN exact match;
- structured attribute candidate filtering;
- retrieval + probabilistic ranker;
- calibrated confidence policy;
- human review.
Exact identifier auto-accept still validates variant/quantity where applicable.
Probabilistic auto-accept requires the deployed mapping release to demonstrate ≥99.5% precision on representative human-labeled evaluation data at the configured threshold. Precision must be inspected by important category/source slices. Threshold is stored in configuration with calibration_version; it is not a universal fixed model score.
First production use of a newly created probabilistic mapping is routed to review for MVP even if above threshold; once accepted, the mapping may be reused until invalidated or material product/source attributes change.
For SKU s:
EligibleOffers(s) = offers where:
mapping accepted
AND offer comparable
AND availability eligible
AND observation fresh
AND promotion eligible
AND quantity/condition policy satisfied
MarketPrice(s) = min(offer.landed_price for offer in EligibleOffers(s))
If EligibleOffers(s) is empty, emit NO_FRESH_COMPARABLE_MARKET_PRICE; do not automatically reprice competitively.
Every excluded offer stores reason codes.
MVP competitive strategy:
competitive_target = market_price
No automatic undercutting.
Cost/margin calculation:
variable_cost = COGS + variable_fulfillment_cost + variable_payment_cost
absolute_floor = variable_cost
margin_floor_price = variable_cost / (1 - required_contribution_margin_pct)
allowed_floor = max(absolute_floor, margin_floor_price, MAP, regulatory_floor)
proposed_price = max(competitive_target, allowed_floor)
Default required_contribution_margin_pct = 0.10; override by category/SKU configuration. Missing/stale mandatory cost or restriction input blocks automatic publication.
All intermediate values are persisted for audit.
Deterministic result: ALLOW | REVIEW | BLOCK.
- proposed price violates a hard constraint after calculation inconsistency;
- mandatory cost/MAP/regulatory input unavailable;
- policy service unavailable;
- no accepted mapping;
- no fresh comparable offer for a competitive reprice;
- active kill switch/hold;
- authorization failure.
- probabilistic mapping below auto-accept policy;
- first production use of a new probabilistic mapping;
- ambiguous/conditional promotion could affect market price;
- conflicting source evidence;
- proposed decrease >5% from current price;
- proposed increase >10% from current price;
- proposed price equals an active margin/MAP/regulatory floor;
- protected/high-impact SKU;
- configured anomaly rule fires.
All hard checks pass and no review condition applies.
Human override cannot authorize a hard BLOCK without an explicit higher-level policy exception workflow; normal reviewer actions operate on REVIEW decisions.
Publisher API accepts price_decision_id, not arbitrary SKU/price pairs.
POST /v1/publications
Idempotency-Key: <price_decision_id>Publisher retrieves authoritative approved decision, rechecks authorization and policy version validity, writes to commerce, then performs read-after-write verification.
Verification checks SKU, currency, expected price, active effective time, and commerce revision/version where available.
Uncertain outcomes reconcile before retry. Retry uses the same idempotency key. Verification mismatch raises incident/reconciliation workflow; rollback uses a previously verified price decision.
Minimum internal APIs:
POST /v1/observationsPOST /v1/mappings/evaluatePOST /v1/market-prices/computePOST /v1/price-decisions/computePOST /v1/price-decisions/{id}/evaluate-policyPOST /v1/reviews/{id}/approvePOST /v1/reviews/{id}/rejectPOST /v1/publicationsPOST /v1/publications/{id}/verifyPOST /v1/rollbacksGET /v1/skus/{sku}/decision-trace
External/event-driven implementations may use events instead of synchronous calls; contracts remain equivalent.
collection.requestedobservation.createdoffer.extractedoffer.normalizedmapping.evaluatedmapping.review_requestedmarket_price.computedpricing.requestedprice.computedpolicy.evaluatedapproval.requestedprice.approvedpublish.requestedprice.publishedpublish.verifiedrollback.requestedworkflow.failed
Consumers are at-least-once safe and idempotent. Poison messages enter a DLQ with correlation context.
MVP default:
- PostgreSQL: operational entities/configuration/decisions/approvals/publications.
- Object storage: immutable raw HTML/API payloads, evidence snapshots, agent inputs/outputs, eval datasets/reports.
- Managed event bus/queue: workflow events.
- Optional cache only after measured need.
Core tables: sku_policy, competitor, source_config, observation, normalized_offer, sku_mapping, market_price_snapshot, pricing_policy, price_decision, approval, publication, manual_override, experiment_assignment, audit_event.
All model calls use a single interface capturing:
- provider/model identifier;
- model version where exposed;
- prompt/template version;
- parameters;
- tool configuration;
- input schema version;
- output schema version;
- latency/token/cost telemetry;
- correlation ID.
Production code must support deterministic mocks for tests. Prompt/model changes are release artifacts and trigger evaluation.
100% pass required for hard financial invariants: margin/MAP/regulatory floors, arithmetic, rounding, state transitions, authorization, idempotency, rollback semantics.
Probabilistic auto-accept threshold must demonstrate ≥99.5% precision on representative labeled evaluation data and pass category/source slice checks. False-positive mappings are separately tracked as critical defects.
Measure price, shipping, availability, quantity, currency, and promotion correctness. Any error capable of creating a materially lower false landed price is a critical evaluation failure until quarantined or mitigated.
Golden scenarios include normal match, competitor below floor, ambiguous mapping, stale data, extreme movement, conditional promotion, out-of-stock lowest offer, wrong pack size, refurbished-vs-new, currency ambiguity, publication timeout, duplicate event, and rollback.
DEV → OFFLINE_EVAL → SHADOW → CANARY → PRODUCTION for probabilistic component changes that can influence price.
Core SLOs:
- monthly availability: 99.9% for decision/governance/publication/verification path;
- P95 usable observation → verified publication ≤15m;
- P99 ≤30m;
- human review waiting time excluded and measured separately.
Operational metrics: source freshness, collection success, eligible coverage, mapping auto-accept/review/reject, mapper precision from reviewed samples, queue depth, decision latency, policy outcomes, publish success, verification mismatch, rollback rate, and cost per observed SKU/source.
SLO burn alerts use multi-window alerting. External source failures are reported separately from core-service availability.
- Source unavailable: retain history, mark stale, retry with backoff; no invented offer.
- Parser/extractor failure: quarantine observation and alert on rate threshold.
- Mapper unavailable: queue/replay; no downstream pricing.
- Policy engine unavailable: fail closed.
- Human review unavailable: hold review-required decisions.
- Publisher timeout: reconcile before retry.
- Duplicate event: deduplicate/idempotent processing.
- DLQ: operator-visible with replay tooling.
- Model outage: deterministic fallback where explicitly implemented; otherwise review/exclude.
- Service identities with least privilege.
- SSO/RBAC for admin UI.
- Separate roles for policy author, pricing reviewer, publisher/operator, and auditor where organizationally feasible.
- Secrets in managed secret store.
- TLS in transit; encryption at rest.
- Immutable/auditable records for price-affecting administrative actions.
- Publisher is the only service principal permitted to invoke production price-write capability.
Vertical slice: ~100 SKUs, 2–3 approved sources, complete workflow, staging or controlled production publisher, human review UI, evaluation harness, audit, monitoring, rollback and kill switch.
Prefer managed containers/serverless + managed PostgreSQL/object storage/queue unless existing platform standards require Kubernetes. Do not introduce extra data stores without measured need.
- Zero hard financial-policy violations in deterministic test suite.
- Zero unauthorized publication paths in security/contract tests.
- 100% decision-trace completeness for published pilot decisions.
- Mapping auto-accept gate satisfied at ≥99.5% evaluated precision.
- Publication idempotency, reconciliation, rollback, and kill switch tested.
- Core SLO instrumentation active and pilot performance within agreed error budget.
- Human review workflow operational with auditable reasons.
- Remaining rollout-blocking product decisions (especially PD-006, PD-010, PD-011, PD-020) resolved for each pilot market/source.
The following are intentionally not invented by this TDS and remain Product/Legal decisions: authoritative MAP/contractual source of truth; condition-comparability rules; marketplace-seller eligibility; legal/source collection policy by market; exact priority SKU selection and approved competitor set. These must be configured/approved before the affected SKU/source is eligible for production auto-pricing.