🪄 AI Background Remover (Images & Videos)

A high-performance web application and REST API for removing backgrounds from both images and videos using deep learning (rembg with the isnet-general-use model), OpenCV, and FFmpeg. Features automated GPU acceleration (CUDA) with CPU fallback, fine detail protection (hair, thin edges), and transparent WebM video export.

---

✨ Features

• 🖼️ Image Background Removal: Instant background removal for image formats (PNG, JPG, JPEG, WEBP) returning transparent RGBA PNGs.
• 🎬 Video Background Removal: Frame-by-frame AI segmentation on MP4 videos, compiled into transparent VP9 WebM (yuva420p) videos with alpha channel.
• 🚀 Hardware Acceleration: Automatic GPU acceleration via ONNX Runtime CUDA Execution Provider (CUDAExecutionProvider), falling back gracefully to CPU.
• ✂️ Foreground Protection & Edge Retention:
  • Fine edge protection for complex structures like hair, thin objects, and fabrics.
  • Configurable foreground morphological dilation to eliminate harsh edge clipping.
  • Mild alpha thresholding (ALPHA_CLEANUP_THRESHOLD = 1) to preserve soft transitions.
• 📦 Batch Processing: Upload and process up to 10 files simultaneously.
• 💻 Modern Web Interface:
  • Interactive upload box with multi-file drag-and-drop / selector support.
  • Dynamic transparency checkerboard preview for both images and video playback.
  • Direct one-click download buttons for transparent results.
• ⚡ RESTful API: Fast and modular FastAPI endpoints for headless integration into automated workflows.

---

🏗️ Architecture & Pipeline

Image Processing

flowchart LR
    A[Input Image] --> B[rembg / ISNet]
    B --> C[RGBA Conversion]
    C --> D[Foreground Protection & Dilation]
    D --> E[Encode PNG with Alpha]
    E --> F[Static Output URL]

Loading

Video Processing

flowchart LR
    A[Input Video MP4] --> B[OpenCV Frame Extraction]
    B --> C[Batch rembg Frame Segmentation]
    C --> D[RGBA Alpha Processing]
    D --> E[FFmpeg VP9 Alpha Encoding]
    E --> F[Transparent WebM Output]

Loading

---

📁 Project Structure

Background-Remover/
├── app.py                 # FastAPI application, AI segmentation pipeline, & FFmpeg encoder
├── requirements.txt       # Python package dependencies
├── README.md              # Project documentation
├── templates/
│   └── index.html         # Frontend web UI with checkerboard transparency preview
├── uploads/               # Temporary storage for uploaded media (auto-cleaned)
├── outputs/               # Generated transparent PNG images and WebM videos
└── temp/                  # Temporary frame buffer for video processing (auto-cleaned)

---

📋 Prerequisites

1. Python 3.10+ installed on your system.
2. FFmpeg installed and accessible in your system PATH (required for video background removal & WebM encoding).
   • Windows (Chocolatey): choco install ffmpeg
   • Windows (Scoop): scoop install ffmpeg
   • Linux (Ubuntu/Debian): sudo apt install ffmpeg
   • macOS (Homebrew): brew install ffmpeg
3. (Optional for GPU acceleration) NVIDIA GPU with compatible CUDA and cuDNN drivers for ONNX Runtime.

---

🚀 Getting Started

1. Clone the Repository

git clone https://github.com/Pushkar-Shahi/Background---Remover.git
cd Background---Remover

2. Create and Activate a Virtual Environment

Windows (PowerShell):

python -m venv venv
.\venv\Scripts\Activate.ps1

Linux / macOS:

python3 -m venv venv
source venv/bin/activate

3. Install Dependencies

pip install -r requirements.txt

Note on GPU Acceleration: If you want GPU acceleration with NVIDIA CUDA, ensure you have onnxruntime-gpu installed matching your CUDA version:

pip install onnxruntime-gpu

4. Run the Application

Start the FastAPI server with Uvicorn:

uvicorn app:app --host 127.0.0.1 --port 8000 --reload

Open your browser and visit: 👉 http://127.0.0.1:8000

---

🔌 API Reference

1. Web UI

• URL: GET /
• Description: Renders the background removal web dashboard.

---

2. Process Media

• URL: POST /process
• Content-Type: multipart/form-data
• Parameters:
  • media: List of files (UploadFile) — up to 10 files per request (images or MP4 videos).

Example curl Request:

curl -X POST "http://127.0.0.1:8000/process" \
  -F "media=@sample_photo.jpg" \
  -F "media=@sample_clip.mp4"

Example Response:

{
  "files": [
    {
      "type": "image",
      "url": "/outputs/9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d.png"
    },
    {
      "type": "video",
      "url": "/outputs/e6b23a07-88cf-48f1-8f5b-80dfa2e6f491.webm",
      "fps": 30.0,
      "frames": 120
    }
  ],
  "time": "4.12 seconds"
}

---

3. Health & Provider Status

• URL: GET /health
• Description: Check server status, model configuration, and active ONNX providers (CUDA / CPU).

Example Response:

{
  "status": "ok",
  "model": "isnet-general-use",
  "providers": [
    "CUDAExecutionProvider",
    "CPUExecutionProvider"
  ],
  "cuda": true
}

---

⚙️ Configuration & Fine-Tuning

Key parameters can be adjusted directly in app.py:

Setting	Default	Description
REMBG_MODEL	"isnet-general-use"	ONNX model loaded by rembg (e.g. u2net, isnet-general-use, birefnet-general).
FOREGROUND_DILATION	1	Alpha mask expansion radius (0 = none, 1 = small, 2 = stronger). Protects fine details.
ALPHA_CLEANUP_THRESHOLD	1	Cuts out near-zero alpha values while preserving soft edges.
IMAGE_ALPHA_MATTING	False	Enables/disables alpha matting refinement for images.
VIDEO_ALPHA_MATTING	False	Enables/disables alpha matting refinement for video frames.
POST_PROCESS_MASK	False	Enables/disables rembg post-processing mask cleanup.

---

🛠️ Tech Stack

• Backend Framework: FastAPI + Starlette
• ASGI Server: Uvicorn
• AI / Segmentation: rembg + ONNX Runtime
• Computer Vision & Image Processing: OpenCV (cv2) + NumPy
• Video Transcoding: FFmpeg (VP9 codec with yuva420p alpha channel)
• Frontend: HTML5, Vanilla JavaScript, CSS3 with Checkerboard transparency preview

---

📄 License

This project is licensed under the MIT License. Feel free to use and modify it.