SDE-2 Order Processing System

A Docker Compose application that coordinates order placement across Order, Inventory, and Shipping services. It implements parallel Order creation and Inventory reservation, compensating actions, fixed retries and timeouts, database-backed idempotency, persisted workflow history, restart recovery, NEEDS_ATTENTION with manual compensation retry, scheduled shipping, one dispatch business effect per order, streaming bulk CSV ingestion, duplicate-safe reloads, and a single-page Angular dashboard.

Prerequisites

• Git
• Docker Desktop with Docker Compose
• Node.js 18 or newer only when running the automated tests from the host

MySQL and Redis run in containers; local installations are not required.

Clone and run

git clone https://github.com/anonymous-cloud/Order_Processing_System.git
cd Order_Processing_System
docker compose up --build

Wait until MySQL and Redis are healthy and all application services report that they have started. Keep this terminal running.

Service URLs

Service	URL
Angular dashboard	http://localhost:8080
Coordinator	http://localhost:3010
Order Service	http://localhost:3001
Inventory Service	http://localhost:3002
Shipping Service	http://localhost:3003

Each Node.js service exposes /health and /ready endpoints.

High-level architecture

Angular / API client
        |
        | POST /internal/workflows
        v
   Coordinator
        |
        | persist IN_PROGRESS workflow and history
        |
        +--------------------------+
        |                          |
        v                          v
  Order Service             Inventory Service
  Create Order              Reserve Inventory
        |                          |
        +-------------+------------+
                      |
                      v
           PLACED / CANCELLED /
              NEEDS_ATTENTION
                      |
                      | scheduled Shipping job
                      v
                   SHIPPED

The project uses one MySQL server with service-owned logical schemas: coordinator_db, order_db, inventory_db, and shipping_db. Each service connects with its own database user. Redis supports rate limiting and cache-aside reads, but MySQL remains authoritative for workflow and idempotency correctness.

Order placement flow

Create a workflow with:

POST http://localhost:3010/internal/workflows
Content-Type: application/json

{
  "orderId": "ORDER_001",
  "sku": "WIDGET-A",
  "qty": 2,
  "amount": 100
}

The Coordinator persists an IN_PROGRESS workflow and starts Create Order and Reserve Inventory concurrently using Promise.all.

Order Create	Inventory Reserve	Coordinator action	Final placement status
Succeeds	Succeeds	No compensation	PLACED
Succeeds	Fails	Cancel Order	CANCELLED, or NEEDS_ATTENTION if cancellation fails
Fails	Succeeds	Release Inventory	CANCELLED, or NEEDS_ATTENTION if release fails
Fails	Fails	No compensation	CANCELLED

Only a forward operation that actually succeeded is compensated. The initial POST returns the persisted workflow identifier and IN_PROGRESS; current details are available from GET /internal/workflows/:idOrOrderId.

Retry and timeout behavior

The four placement operations—Order Create, Inventory Reserve, Order Cancel, and Inventory Release—use fixed assignment values:

Setting	Value
Maximum automatic attempts	3
Delay between failed attempts	1 second
Timeout per attempt	5 seconds

There is no exponential backoff. Every attempt is persisted with its number, status, timestamps, and error. Retries reuse deterministic idempotency keys, so a committed operation is not performed again when a response is lost or times out.

Shipping

Shipping is separate from placement. ShippingScheduler periodically requests PLACED workflows from the Coordinator, creates a dispatch in shipping_db, and asks the Coordinator to transition the workflow to SHIPPED.

• Default interval: 15 minutes (SHIPPING_INTERVAL_MS=900000)
• Docker Compose demo interval: 5 seconds (SHIPPING_INTERVAL_MS=5000)
• Dispatch key: DISPATCH:<orderId>
• Database guarantee: dispatches.order_id has a unique constraint

This provides one dispatch business record per order across repeated scheduler runs; it does not claim literal exactly-once distributed message delivery.

Bulk CSV

Endpoint:

POST http://localhost:3010/internal/workflows/bulk
Content-Type: text/csv

Headers:

order_id,sku,qty,amount,fail_at,comp_fail_at

Example:

order_id,sku,qty,amount,fail_at,comp_fail_at
BULK_001,WIDGET-A,1,10,,
BULK_002,WIDGET-A,1,10,inventory,order

fail_at and comp_fail_at support blank, order, or inventory. When comp_fail_at is populated, fail_at must identify the opposite service. These fields exist only for deterministic assignment demonstrations.

The backend reads the request line by line and limits active row processing to five workflows. order_workflows.order_id is unique, so reloading the same CSV reports duplicates without creating duplicate logical workflows or downstream business effects.

Testing Bulk CSV Upload

The repository contains two CSV files with different purposes:

File	Repository path	Purpose
sample_inventory.csv	apps/inventory-service/src/sample_inventory.csv	Initial Inventory Service seed data. It defines starting SKU quantities and is loaded with INSERT IGNORE, so service restarts do not reset existing stock. It is not an order-upload file.
test_bulk.csv	test_bulk.csv	Small bulk-order input containing three assignment demonstration rows.

The included test_bulk.csv contains:

Order	Demonstration
CURL_BULK_OK_001	Normal successful placement
CURL_BULK_CANCEL_001	Inventory failure followed by successful Order cancellation
CURL_BULK_ATTENTION_001	Inventory failure followed by simulated Order cancellation failure, producing NEEDS_ATTENTION

It uses the required headers:

order_id,sku,qty,amount,fail_at,comp_fail_at

To upload it through the UI:

1. Open http://localhost:8080.
2. Find Bulk CSV Order Import.
3. Select test_bulk.csv from the repository root.
4. Click Upload & Process CSV.
5. Review the displayed total, accepted, duplicate, and rejected counts.
6. Refresh or inspect the workflow table to observe the successful, compensated, and NEEDS_ATTENTION outcomes.
7. Upload the exact same file again. The second upload should report the existing order_id values as duplicates and must not create new logical workflows, reserve stock again, or create additional dispatch records.

The direct backend interface used by the UI is:

POST /internal/workflows/bulk
Content-Type: text/csv

The endpoint expects the CSV text as the raw request body; it does not require multipart form data.

Important limitations of the currently included artifact:

• test_bulk.csv has only three order rows. It does not demonstrate the assignment's approximately 2,500-order streaming and sustained bounded-concurrency scenario.
• It has Inventory failure and compensation-failure rows, but no fail_at=order row demonstrating Order Service creation failure and Inventory release.
• The file begins with a UTF-8 byte-order mark. The backend's raw CSV parser compares the first header literally and does not explicitly remove a BOM, so direct byte-for-byte API upload is not confirmed from the current code to recognize order_id. The Angular FileReader upload is the intended documented path for this file.

MISSING REQUIRED TEST ARTIFACT: orders_bulk.csv

No orders_bulk.csv or equivalent approximately 2,500-row order file exists in the repository. That artifact is needed for an evaluator to demonstrate the large streamed upload, bounded concurrency under sustained input, duplicate-safe reload at assignment scale, and complete deterministic forward-failure coverage without creating another local file.

Assignment tests

Leave Docker Compose running. In another terminal, from the repository root, run:

node apps/coordinator/src/test/run_all_tests.js

The required coverage is:

• T1: both forward steps succeed; placement reaches PLACED and each business effect occurs once
• T2: one forward step fails; the successful step is undone and the workflow becomes CANCELLED
• T3: a committed side effect followed by a lost/slow response is retried without executing the business effect twice
• T4: a PLACED order is shipped and repeated scheduler runs reuse the same dispatch

Additional coverage verifies compensation failure, transition to NEEDS_ATTENTION, and a targeted manual compensation retry.

Persistence and restart recovery

MySQL stores its data in the named Compose volume mysql_data:

volumes:
  mysql_data:

Safe restarts preserve workflows, workflow steps and attempts, orders, inventory state, idempotency records, and dispatch records:

docker compose restart

or:

docker compose down
docker compose up -d

On startup, the Coordinator scans stale IN_PROGRESS and COMPENSATING workflows. Persisted successful steps are reused rather than restarted, and unfinished work resumes from stored state.

The following command is an intentional full database reset and deletes the named volume and all application data:

docker compose down -v

Needs Attention and manual retry

When a required Order Cancel or Inventory Release cannot finish after its automatic attempts, the Coordinator records the failed compensation and sets the workflow to NEEDS_ATTENTION.

POST /internal/workflows/:idOrOrderId/retry

Manual retry is accepted only for NEEDS_ATTENTION. It retries only the recorded failed compensation; completed Order Create and Inventory Reserve steps are not rerun. Each manual retry gets a new batch of up to three attempts, which are appended to the existing history.

For a deterministic demo, the simulated compensation fault can be cleared during retry:

{
  "clearFailureSimulation": true
}

This flag is for assignment test recovery, not ordinary production input.

Idempotency design note

MySQL uniqueness is the correctness source. Order operations use unique keys in order_operations, Inventory operations use unique keys in inventory_operations, Coordinator workflows use a unique order_id, and Shipping dispatches use a unique order_id. A retry uses the same logical key and returns the previously committed response instead of repeating the business side effect. Redis is not trusted to authorize a second execution.

Scale design note

The never-do-a-step-twice check stays fast through unique, indexed idempotency keys in MySQL. A cache could be used only as a read optimization: the database remains authoritative, a cache miss falls back to the database, durable state is committed before cache invalidation/update, and stale cache data must never authorize another side effect.

Multiple Coordinator instances

The assignment demo runs one Coordinator instance. Service-level idempotency and database uniqueness already protect Order, Inventory, and Shipping business effects from duplicate requests. Multiple Coordinator instances would additionally require exclusive workflow claiming—such as an atomic compare-and-set, row lock, or lease—so only one Coordinator advances a workflow at a time. That distributed claim mechanism is not implemented by the current single-instance demo.

Stop the project

Stop containers while preserving MySQL data:

docker compose down

Delete containers and all persisted MySQL data only when an intentional clean reset is required:

docker compose down -v

Evaluator readiness check

NO — a new evaluator can clone the repository, start Docker Compose, use Angular for single orders, upload the small test_bulk.csv, repeat that file to inspect duplicate handling, and run T1–T4. However, the repository is missing orders_bulk.csv (or an equivalent approximately 2,500-row artifact) needed to demonstrate the assignment-scale streaming/concurrency requirement and an Order Service failure row. The BOM compatibility of raw direct upload for the existing test_bulk.csv is also not confirmed by the parser.