DubMaster 🎬

AI-powered backend for automated multilingual video dubbing. Uploads a short video, transcribes and translates the speech, generates dubbed audio, and merges it back into the video — all through an async, pipeline-based Node.js backend.

Overview

DubMaster takes a short user-uploaded video and runs it through an automated pipeline:

1. Speech-to-text — extract and transcribe the original audio
2. Translation — translate the transcript into the target language
3. Text-to-speech — synthesize dubbed audio in the target language
4. Sync & merge — align the new audio with the original video and render the final output

Built to demonstrate backend system design and real AI-pipeline integration, not just API wrapping — including auth, rate limiting, media processing, and cloud storage lifecycle management.

Tech Stack

• Backend: Node.js, Express
• Auth: JWT, bcrypt, rate limiting (express-rate-limit)
• Media processing: FFmpeg (fluent-ffmpeg, ffmpeg-static, ffprobe-static)
• AI / ML: Hugging Face Inference & Transformers, Whisper (speech-to-text), Google GenAI, Gradio Client
• Database: MongoDB / Mongoose
• Storage: Cloudinary
• Subtitles: srt-parser-2, subtitle

Architecture

Upload API (Express)
      ↓
Cloud Storage (Cloudinary)
      ↓
Processing Pipeline
      ↓
Speech-to-Text (Whisper) → Translation → Text-to-Speech
      ↓
Audio/Video Sync (FFmpeg)
      ↓
Final Output (Cloudinary)

Core Features

• JWT-based authentication with rate-limited login endpoints
• Video upload and validation (multer)
• Automated STT → translation → TTS pipeline
• Subtitle generation (.srt)
• Audio-video synchronization and rendering via FFmpeg
• Cloud storage integration with automatic cleanup

Getting Started

git clone https://github.com/SHAHZIL14/DubMaster.git
cd DubMaster
npm install

Create a .env file with the required credentials (Cloudinary, MongoDB URI, Hugging Face API key, JWT secret, etc. — see configuration/).

npm start

Project Structure

DubMaster/
├── Sources/          # Core pipeline logic
├── Documents/        # Docs / reference material
├── configuration/     # Environment & pipeline config
├── Public/            # Public assets
├── index.js           # Entry point
└── input.txt

Status

This is a portfolio-stage backend project — the pipeline architecture and core dependencies (Whisper, FFmpeg, Hugging Face) are wired up, and it's intended to demonstrate async pipeline design and AI-model integration rather than serve production traffic.

Author

Mohd Shazil Raza GitHub · LinkedIn