DDAS — Data Download Duplication Alert System

DDAS (Data Download Duplication Alert System) is a production-hardened, multi-tenant file storage and compliance platform engineered for duplicate file detection, near-duplicate content identification, and data loss prevention (DLP). Built with FastAPI, MongoDB, and React, DDAS eliminates redundant data accumulation across enterprise environments while enforcing strict multi-tenant isolation, cryptographic access controls, and comprehensive audit logging.

DDAS delivers multi-layered file deduplication by combining SHA-256 exact deduplication, MinHash and Locality-Sensitive Hashing (LSH) for document text similarity, and difference hashing (dHash) perceptual hashing for image near-duplicate detection. Every asset is protected at rest using authenticated AES-256-GCM encryption, defended against sensitive data exfiltration via deterministic DLP inspection with Luhn validation, and guarded by robust role-based access control (RBAC).

Deterministic Security Guarantee: DDAS operates strictly on deterministic algorithms (SHA-256 cryptographic hashing, MinHash LSH, dHash perceptual hashing, AES-256-GCM encryption, regular expressions with Luhn checksum verification). DDAS currently does not use ML, AI, LLMs, neural networks, embeddings, or predictive models. (ML/AI classification remains exclusively as a potential future roadmap item).

⚠️ Security Notice: If deploying to production, you MUST rotate ALL secrets in your .env file. Generate new values using: python -c "import secrets; print(secrets.token_urlsafe(48))" for JWT_SECRET and python -c "import secrets; print(secrets.token_hex(32))" for ENCRYPTION_MASTER_KEY.

---

💡 Why DDAS?

Enterprise workspaces, cloud drives, and collaborative portals inevitably accumulate massive volumes of redundant files. Employees download and re-upload identical attachments, modify a few paragraphs in contracts or reports, and save slightly edited or resized versions of brand assets. This sprawl causes:

1. Storage Inefficiency & Spiraling Costs: Identical and near-identical files waste significant disk capacity, backup bandwidth, and cloud storage budgets.
2. Data Governance & Compliance Risks: Sensitive corporate assets (PII, API keys, credentials, financial records) proliferate unchecked across unmonitored duplicate copies.
3. Cross-Tenant Security Challenges: Multi-tenant environments require strict logical isolation so that global storage optimization never leaks data or metadata across organizational boundaries.

DDAS solves these challenges with a deterministic, high-throughput pipeline that intercepts file uploads and downloads, calculates cryptographic and similarity fingerprints, screens for compliance violations, deduplicates physical blobs safely, and maintains an immutable audit trail.

---

✨ Key Features

• SHA-256 Exact File Deduplication: Global content-addressed physical storage eliminates redundant disk usage across tenants while maintaining isolated logical records.
• MinHash / LSH Text Near-Duplicate Detection: Tokenizes document text into 128-permutation MinHash signatures with sub-linear Locality-Sensitive Hashing (LSH) candidate discovery and exact Jaccard similarity verification.
• dHash Image Near-Duplicate Detection: 16×16 perceptual difference hashing with a 32-bucket Pigeonhole candidate index for fast, zero-false-negative visual match discovery.
• Deterministic Data Loss Prevention (DLP): Real-time regex scanning for credit cards (with Luhn checksum validation), AWS keys, GitHub tokens, API secrets, SSNs, IBANs, phone numbers, and emails with automatic quarantine.
• AES-256-GCM Encryption at Rest: Every physical blob is encrypted with an independent 256-bit master key and unique 96-bit nonces per write.
• Strict Multi-Tenant Isolation: Complete logical scoping across all collections; cross-tenant queries return strict 404 responses to eliminate Insecure Direct Object Reference (IDOR) vulnerabilities.
• Role-Based Access Control (RBAC): Granular roles (employee, admin, super_admin, auditor) enforcing administrative quarantine remediation and file ownership boundaries.
• Comprehensive Audit Logging: Immutable tenant-scoped audit stream tracking uploads, downloads, deletions, password resets, and policy violations.
• Safe File Lifecycle & Concurrency Controls: BlobLockManager mutex synchronization, atomic temporary writes (os.replace), and guarded MongoDB reference counting prevent race conditions and data loss.
• Memory-Conscious Ingestion & Streaming: Incremental chunked reading with early upload size aborts and large-download disk spooling for files over 10 MB.

---

🏗️ System Architecture

DDAS leverages a modular, production-hardened backend built with FastAPI, connected to a MongoDB document registry and an encrypted-at-rest storage vault. It services a modern, responsive React Web application.

graph TD
    subgraph ClientLayer["Client Layer"]
        WebUI["React Web UI"]
    end

    subgraph APILayer["API Layer"]
        Router["/api/v1/* and Root Aliases"]
        RateLimit["Rate Limiter (SlowAPI)"]
        AuthMW["Auth and RBAC (get_current_user, require_role)"]
    end

    subgraph ServiceLayer["Service Layer"]
        AuthService["Auth Service"]
        FileService["File Orchestration Service"]
        DedupService["Dedup Service"]
        DLPService["DLP Engine"]
        EncService["AES-256-GCM Encryption Service"]
        QuarantineService["Quarantine Remediation Service"]
        StorageService["Storage Abstraction Service"]
        WebhookService["Webhook Notification Service"]
    end

    subgraph AlgorithmLayer["Algorithm Layer"]
        SHA256["SHA-256 Digest"]
        MinHash["MinHash Signatures (128 Permutations)"]
        LSH["MinHashLSH Candidate Discovery"]
        Jaccard["Exact Jaccard Verifier"]
        dHash["dHash 16x16 Perceptual Hash"]
        dHashBuckets["32-Bucket Pigeonhole Candidate Index"]
        MIMECheck["libmagic and Extension Filter"]
        Luhn["Luhn Checksum Verifier"]
    end

    subgraph RepositoryLayer["Concurrency and Repository Layer"]
        LockMgr["BlobLockManager (Keyed Mutex per Hash)"]
        FileRepo["Tenant-Scoped File Repository"]
        BlobRepo["Atomic Ref-Counting Blob Repository"]
        UserRepo["Tenant-Scoped User Repository"]
        AuditRepo["Tenant-Scoped Audit Repository"]
        SettingsRepo["Tenant-Scoped Settings Repository"]
    end

    subgraph DataTier["Data Tier"]
        DB[("MongoDB (ddas_db)")]
        Vault[("Encrypted Storage Vault (AES-256-GCM)")]
    end

    WebUI --> Router
    Router --> RateLimit
    RateLimit --> AuthMW
    AuthMW --> AuthService
    AuthMW --> FileService
    AuthMW --> QuarantineService
    FileService --> DedupService
    FileService --> DLPService
    FileService --> EncService
    FileService --> StorageService
    FileService --> WebhookService
    DedupService --> SHA256
    DedupService --> MinHash
    MinHash --> LSH
    LSH --> Jaccard
    DedupService --> dHash
    dHash --> dHashBuckets
    DLPService --> Luhn
    FileService --> MIMECheck
    FileService --> LockMgr
    QuarantineService --> LockMgr
    LockMgr --> BlobRepo
    LockMgr --> FileRepo
    AuthService --> UserRepo
    StorageService --> Vault
    FileRepo --> DB
    BlobRepo --> DB
    UserRepo --> DB
    AuditRepo --> DB
    SettingsRepo --> DB

Loading

---

🔒 Security Architecture

DDAS enforces security at every layer of the ingestion and retrieval lifecycle:

1. Authentication:
   • Standard JWT tokens signed with HS256 and verified with UTC expiration.
   • Passwords hashed using bcrypt with cryptographically generated salts.
   • User credentials verified against tenant bounds; user existence confirmed on every authenticated request.
2. Authorization & RBAC:
   • Four distinct roles: employee, admin, super_admin, and auditor.
   • Dependency-injected authorization (require_role, get_admin_user) guards privileged routes.
   • Privileged operations (user deletion, password resets, quarantine remediation, audit inspections) require admin or super_admin credentials.
   • File deletion permits either file owners or tenant administrators (admin, super_admin).
3. Tenant Isolation & IDOR Protection:
   • Strict logical scoping: every document (files, users, logs, settings) carries a mandatory company field.
   • All repository queries and mutations explicitly filter by company.
   • Cross-tenant queries (/files/download/{id}, /files/text/{id}, /files/summary/{id}, /files/{id}, /admin/quarantine/remediate) return strict 404 Not Found to prevent direct object reference leakage.
4. Encryption at Rest:
   • AES-256-GCM authenticated encryption.
   • Cryptographically independent 256-bit master key (ENCRYPTION_MASTER_KEY), completely decoupled from JWT secrets.
   • Unique 96-bit random nonce (os.urandom(12)) generated per write operation.
   • Magic header DDAS_ENC\x01 enables zero-copy memoryview decrypt slicing and backward-compatible fallback for unencrypted legacy assets.
5. Defensive Ingestion:
   • Early Content-Length header validation rejecting oversized uploads before reading the request body.
   • Incremental chunked reading (read_upload_safely) terminating body transfer immediately upon exceeding the configured size limit (MAX_UPLOAD_SIZE_MB).
   • Filename sanitization neutralizing directory traversal (.., /, \), null bytes (\x00), and non-printable control characters, with a 255-byte cap.
   • Bounded MIME type validation using libmagic inspecting the first 64 KB of the file, alongside an executable/script extension blocklist (.exe, .sh, .bat, etc.).
   • RFC 6266 compliant filename*=UTF-8'' Content-Disposition header encoding preventing header injection.
6. Data Loss Prevention (DLP):
   • 8 deterministic patterns covering Credit Cards, AWS Access Keys, GitHub Tokens, API Keys, SSNs, IBANs, Phone Numbers, and Emails.
   • Full Luhn checksum verification for credit card numbers to minimize false positives.
   • Automated quarantine classification for documents containing HIGH or CRITICAL policy violations.
   • Administrative remediation pipeline supporting manual approval, secure purging, or automated plain-text redaction.
7. Rate Limiting:
   • SlowAPI rate limiting protecting sensitive endpoints (/login: 5/min, /register: 3/min, /upload: 10/min), automatically disabled during pytest runs.

(Note: DDAS provides robust defensive security controls; however, no software system can guarantee 100% security against all conceivable external attack vectors or hardware compromise).

---

🔍 Deduplication Architecture

DDAS implements a comprehensive, three-layer deduplication pipeline:

1. Exact Deduplication (SHA-256)

• Files are fingerprinted using cryptographically collision-resistant SHA-256 hashes.
• Physical storage is globally content-addressed (filename = sha256_hash), avoiding duplicate storage for identical content across tenants.
• Logical file documents remain strictly isolated per tenant.
• Exact duplicate detection within a tenant flags redundant uploads and links them to the existing document.

2. Text Near-Duplicate Detection (MinHash / LSH)

• Plain text is extracted from documents (PDF, DOCX, XLSX, XLS, PPTX, RTF, TXT, CSV, JSON, XML, HTML, MD, YAML, LOG).
• Word-level shingling produces a 128-permutation MinHash signature.
• A Locality-Sensitive Hashing (LSH) index performs sub-linear candidate discovery within the tenant.
• Discovered candidates undergo exact Jaccard similarity verification against the stored 64-bit hash values.
• Documents exceeding the threshold (default: 80% similarity) are classified as near-duplicates and linked to the base document.

3. Image Near-Duplicate Detection (Perceptual dHash)

• Supported formats: JPG, PNG, GIF, WebP, BMP, TIFF.
• A 16×16 difference hash (dHash) generates a 256-bit perceptual fingerprint based on pixel gradient transitions.
• Indexed Candidate Discovery: dHash is partitioned into 32 8-bit bucket tokens, indexed in MongoDB with ("company", ASCENDING), ("dhash_buckets", ASCENDING). By the Pigeonhole Principle, any image with $\ge 90.0%$ similarity ($r \le 25$ differing bits) is mathematically guaranteed to share at least 7 bucket tokens with its match (zero false negatives).
• Candidates retrieved from MongoDB undergo exact Hamming distance verification for $O(1)$ similarity calculation.

---

⚡ Concurrency & Blob Lifecycle

The physical blob lifecycle reconciles tenant-isolated logical records with globally content-addressed physical storage:

[Upload] ──> [Acquire Content Lock] ──> [Atomic Storage Save] ──> [Blob Ref Incremented (+1)] ──> [Logical Record Inserted]
                                                                                                            │ (on failure)
                                                                                                            └──> [Rollback: Ref Decrement (-1)]

[Delete] ──> [Acquire Content Lock] ──> [Logical Record Deleted] ──> [Blob Ref Decremented (-1)] ──> [If Ref == 0: Purge Physical File]

1. In-Process Mutual Exclusion (BlobLockManager):
   • Fine-grained, keyed mutex per content_hash coordinates concurrent uploads, deletes, and redactions targeting the same physical blob.
2. Atomic Storage Writes:
   • New blobs are written to a unique .tmp_* file in the same directory, flushed, fsynced to disk, and published via os.replace(). This guarantees that partially written blobs are never visible or accessible.
3. Atomic Reference Counting (blobs collection):
   • Registrations use atomic $inc: {"ref_count": 1} with upsert semantics.
   • Deletions use guarded atomic $inc: {"ref_count": -1} with ref_count > 0 preconditions, preventing negative reference counters under race conditions.
4. Physical Deletion Guard:
   • Physical blobs are unlinked from disk ONLY when the global reference count reaches 0 AND a verification query confirms that zero logical documents reference the hash.
5. Orphaned Temporary File Cleanup:
   • On application startup (lifespan), StorageService.cleanup_stale_temp_files() sweeps the storage directory and removes any .tmp_* or .dl_spool_* files older than 1 hour, safely reclaiming disk space after abnormal server terminations while leaving active writes and legitimate storage blobs untouched.

Concurrency Boundary & Limitation: BlobLockManager synchronizes lifecycle operations within a single application process (using thread-level mutexes). MongoDB atomic operations ($inc, find_one_and_update, unique indexes) remain safe across multiple processes. However, because local filesystem operations and MongoDB transactions are separate systems without a distributed lock manager (such as Redis or ZooKeeper), multi-container deployments running against a shared NFS/EFS mount should use single-process workers or introduce external distributed locking.

---

💾 Memory Pipeline Behavior

The DDAS file handling pipeline is carefully structured to avoid memory amplification while respecting cryptographic and format constraints:

• Upload Streaming & Early Abort: Requests with a valid Content-Length exceeding MAX_UPLOAD_SIZE_MB are rejected immediately. Requests without length headers are read in bounded 1 MB chunks and aborted if the accumulated size exceeds the limit.
• Payload Assembly: Because AES-256-GCM authentication tags verify the integrity of the complete ciphertext block as a single unit, and downstream document parsers (PDF, DOCX, XLSX, PIL) require complete binary buffers, uploaded files up to the configured limit are assembled into memory during active ingestion. Application memory usage scales with active concurrent uploads up to MAX_UPLOAD_SIZE_MB per worker.
• Duplicate Fast Path: If a physical blob already exists in storage, redundant AES-GCM encryption is completely bypassed, eliminating CPU cycles and duplicate memory allocation.
• Bounded MIME Inspection: Only the first 64 KB of the payload is passed to libmagic, preventing multi-hundred-megabyte buffer allocations in C-bindings.
• Large Download Spooling: Decrypted downloads exceeding 10 MB are spooled to a temporary file on disk, freeing the memory buffer immediately. The file is streamed to the HTTP client in bounded 64 KB chunks, reducing memory consumption by over 99.9% for large downloads. Small downloads ($\le 10$ MB) are streamed directly from a zero-copy memoryview.

---

📡 API Reference

The DDAS API exposes modern versioned /api/v1/* routes alongside backward-compatible root aliases:

Endpoint	Method	Description	Auth Required	Scope / RBAC
/api/v1/health	GET	Liveness probe (process status & UTC timestamp)	No	Public
/api/v1/ready	GET	Readiness probe (MongoDB ping connectivity)	No	Public
/api/v1/register	POST	Register user (requires invite code or admin secret)	No	Rate-limited (3/min)
/api/v1/login	POST	Authenticate credentials & issue JWT token	No	Rate-limited (5/min)
/api/v1/users/me	GET	Authenticated user profile and storage metrics	Yes	Tenant / User scoped
/api/v1/users/change-password	POST	Update authenticated user password	Yes	User scoped
/api/v1/files/upload	POST	Upload file with DLP, deduplication & encryption	Yes	Tenant scoped (10/min)
/api/v1/files	GET	List tenant files (supports pagination or flat list)	Yes	Tenant scoped
/api/v1/files/download/{id}	GET	Download and stream decrypted file	Yes	Tenant scoped
/api/v1/files/text/{id}	GET	Fetch extracted plain-text content for diffing	Yes	Tenant scoped
/api/v1/files/summary/{id}	GET	Generate deterministic compliance summary	Yes	Tenant scoped
/api/v1/files/{id}	DELETE	Concurrency-safe file delete & storage reclaim	Yes	Owner / Admin / Super Admin
/api/v1/files/bulk-delete	POST	Batch delete multiple files safely	Yes	Owner / Admin / Super Admin
/api/v1/dashboard/stats	GET	Aggregated tenant analytics (activity, hoarders, DLP)	Yes	Tenant scoped
/api/v1/admin/users	GET	List all users within the company tenant	Yes	Admin / Super Admin
/api/v1/admin/user-details/{username}	GET	View detailed storage metrics for a tenant user	Yes	Admin / Super Admin
/api/v1/admin/reset-password	POST	Reset password for a tenant employee	Yes	Admin / Super Admin
/api/v1/admin/users/{username}	DELETE	Remove employee account from company tenant	Yes	Admin / Super Admin
/api/v1/admin/invite-code	GET	Retrieve company onboarding invite code	Yes	Admin / Super Admin
/api/v1/admin/global-duplicates	GET	Tenant-scoped duplicate storage aggregation	Yes	Admin / Super Admin
/api/v1/admin/quarantined-files	GET	List quarantined files awaiting remediation	Yes	Admin / Super Admin
/api/v1/admin/quarantine/remediate	POST	Remediate file (approve, purge, redact)	Yes	Admin / Super Admin
/api/v1/admin/logs	GET	Paginated audit log stream	Yes	Admin / Super Admin
/api/v1/admin/settings	GET/POST	Fetch or update webhook notification settings	Yes	Admin / Super Admin
/api/v1/admin/settings/test	POST	Dispatch asynchronous test webhook	Yes	Admin / Super Admin

---

⚙️ Configuration

DDAS is configured via environment variables or a .env file loaded at application startup:

Variable	Type	Default	Description
MONGO_URI	string	mongodb://localhost:27017	MongoDB connection URI
DATABASE_NAME	string	ddas_db	MongoDB database name (alias: DB_NAME)
STORAGE_DIR	string	./storage	Directory for encrypted physical blob storage (alias: STORAGE_PATH)
JWT_SECRET	string	Required	Secret key for JWT signing (minimum 32 characters)
JWT_ALGORITHM	string	HS256	JWT signature algorithm (alias: JWT_ALGO)
ACCESS_TOKEN_EXPIRE_HOURS	int	24	JWT access token lifetime in hours
ACCESS_TOKEN_EXPIRE_MINUTES	int	Optional	Compatibility alias for token lifetime in minutes
ADMIN_SECRET	string	Optional	Secret passphrase required for registering tenant admin accounts
ENCRYPTION_MASTER_KEY	string	Required	64-hex-character (32 bytes) master key for AES-256-GCM
MAX_UPLOAD_SIZE_MB	int	500	Maximum file upload size limit in megabytes
CORS_ORIGINS	string/list	localhost:5173,127.0.0.1:5173	Allowed CORS origins (comma-separated or JSON list)
NEAR_DUP_THRESHOLD	float	0.8	Minimum Jaccard similarity for text near-duplicate detection
MINHASH_NUM_PERM	int	128	Number of permutation functions for MinHash computation
LOG_LEVEL	string	INFO	Logging level (DEBUG, INFO, WARNING, ERROR)

---

📁 Project Structure

DDAS_UI2/
├── backend/
│   ├── app/
│   │   ├── __init__.py                # Package re-exports and backward-compatibility aliases
│   │   ├── main.py                    # FastAPI application, CORS, lifespan, and global exception handlers
│   │   ├── algorithms/
│   │   │   ├── hashing.py             # SHA-256 digest, safe ObjectId, filename sanitization
│   │   │   ├── minhash.py             # MinHash signature generation
│   │   │   ├── lsh.py                 # MinHashLSH candidate indexing and verification
│   │   │   ├── similarity.py          # Exact Jaccard similarity from stored hash values
│   │   │   ├── perceptual_hash.py     # 16x16 dHash, Hamming distance, 32-bucket index tokens
│   │   │   ├── text_extraction.py     # Multi-format document text extraction
│   │   │   └── mime_validation.py     # libmagic bounded MIME inspection and extension filtering
│   │   ├── api/
│   │   │   └── v1/
│   │   │       ├── router.py          # Unified API v1 router
│   │   │       ├── auth.py            # Registration and login endpoints
│   │   │       ├── users.py           # Self-service user profile and password change
│   │   │       ├── files.py           # Upload, list, download, diff, summary, delete
│   │   │       ├── admin.py           # Team management, quarantine, logs, webhooks
│   │   │       ├── dashboard.py       # Aggregated tenant statistics
│   │   │       └── health.py          # Liveness (/health) and readiness (/ready) probes
│   │   ├── core/
│   │   │   ├── config.py              # Centralized Pydantic BaseSettings
│   │   │   ├── security.py            # Password hashing, JWT creation/decoding, UserRole enum
│   │   │   ├── dependencies.py        # FastAPI dependency injection (auth, RBAC, tenant context)
│   │   │   ├── limiter.py             # SlowAPI shared rate limiting instance
│   │   │   └── logging.py             # Structured logger with automatic PII/credential scrubbing
│   │   ├── db/
│   │   │   ├── database.py            # MongoDB connection singleton and collection handles
│   │   │   └── indexes.py             # Compound and unique index creation on application startup
│   │   ├── repositories/
│   │   │   ├── file_repository.py     # Tenant-scoped file CRUD and aggregation pipelines
│   │   │   ├── blob_repository.py     # Concurrency-safe physical blob reference counting & BlobLockManager
│   │   │   ├── user_repository.py     # Tenant-scoped user and organization queries
│   │   │   ├── audit_repository.py    # Immutable tenant audit logging
│   │   │   └── settings_repository.py # Tenant webhook notification configurations
│   │   ├── schemas/
│   │   │   ├── auth.py                # Token and registration Pydantic models
│   │   │   ├── files.py               # File metadata, upload response, bulk delete models
│   │   │   ├── dlp.py                 # DLP findings, severity levels, scan result schemas
│   │   │   ├── users.py               # User response schemas
│   │   │   ├── audit.py               # Audit log schemas
│   │   │   └── common.py              # Message and pagination schemas
│   │   └── services/
│   │       ├── auth_service.py        # Authentication and credential workflows
│   │       ├── file_service.py        # Upload orchestration, spooling downloads, atomic deletion
│   │       ├── dedup_service.py       # Exact, MinHash, and dHash deduplication coordinator
│   │       ├── dlp_service.py         # Regular expression scanning with Luhn verification
│   │       ├── encryption_service.py  # AES-256-GCM encryption/decryption at rest
│   │       ├── quarantine_service.py  # Admin approval, purge, and atomic redaction
│   │       ├── storage_service.py     # Atomic filesystem I/O abstraction and stale temp cleanup
│   │       └── webhook_service.py     # Asynchronous Slack, Discord, and Teams dispatcher
│   ├── tests/
│   │   ├── conftest.py                # Multi-tenant test fixtures and client setup
│   │   ├── test_integration/          # Authentication, file lifecycle, dedup, and dashboard tests
│   │   ├── test_security/             # RBAC, tenant isolation, JWT, memory upload, production readiness
│   │   └── test_unit/                 # Encryption, hashing, DLP, MinHash, dHash, storage atomicity, blob concurrency
│   ├── app.py                         # Re-export entrypoint for ASGI servers
│   ├── Dockerfile                     # Hardened multi-stage non-root container definition
│   ├── requirements.txt               # Pinned Python dependencies
│   ├── .env.example                   # Environment configuration template
│   └── seed_db.py                     # Initial database seeding script
├── frontend/
│   ├── src/
│   │   ├── App.jsx                    # Routing, top navigation, and layout coordinator
│   │   ├── api.js                     # Axios HTTP client with JWT interceptors
│   │   ├── pages/                     # React application views (Upload, Files, Dashboard, Admin, etc.)
│   │   ├── components/                # Reusable UI components
│   │   ├── context/                   # React context providers (ThemeContext)
│   │   └── stubs/                     # Web/desktop compatibility stubs
│   ├── package.json                   # Node.js dependencies
│   └── vite.config.js                 # Vite build configuration
├── docs/
│   └── ARCHITECTURE_AUDIT.md          # Comprehensive architectural and security audit report
└── README.md                          # Main project documentation

---

🐳 Docker Deployment

The backend includes a multi-stage, production-hardened Dockerfile executing under a dedicated non-root user (ddasuser, UID 1000):

cd backend

# Build Docker image
docker build -t ddas-backend .

# Run container with environment configuration
docker run -d \
  --name ddas-backend \
  -p 8000:8000 \
  --env-file .env \
  -v $(pwd)/storage:/app/storage \
  ddas-backend

• Builder Stage: Compiles build dependencies and installs Python packages with --prefix=/install.
• Runtime Stage: Copies packages to /usr/local and sets up ddasuser without root privileges.
• Healthcheck: Configured with 30s interval querying http://localhost:8000/health.

---

🧪 Testing & Validation

Run the complete automated test suite:

# Backend test suite
python -m pytest backend/tests/ -v

# Python syntax and bytecode compilation verification
python -m compileall backend/app backend/tests

# Frontend production build
cd frontend
npm run build

Verified Test Result:

• 120 passed, 0 failed in 7.53 seconds.
• Python compilation: 0 errors.
• Frontend production bundle: Built successfully.

---

🚀 Production Status & Readiness

• Current Status: 🟢 PRODUCTION READY WITH DOCUMENTED LIMITATIONS
• Verified Strengths:
  • Robust multi-tenant isolation with zero cross-tenant data leaks (tested across download, diff, text, summary, deletion, audit, and quarantine).
  • Deterministic algorithms guaranteeing predictable, explainable behavior without black-box ML/AI models.
  • Concurrency-safe physical blob reference counting and atomic filesystem writes.
  • Hardened non-root Docker container packaging with automated liveness probes.
  • High automated test coverage with 120 passing unit, integration, security, and concurrency tests.
• Known Limitations:
  • BlobLockManager is single-process: for horizontal scaling across multiple container instances, an external distributed lock coordinator (e.g. Redis) is required.
  • AES-256-GCM requires complete payload buffering during encryption/decryption, requiring RAM proportional to active concurrent uploads up to MAX_UPLOAD_SIZE_MB.