Skip to content

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

1 Commit

Folders and files

Repository files navigation

CaptionInsight

An efficient subtitle-based learning workspace — automatically download subtitles from video links, or upload subtitle files / paste text, then use custom prompts to generate structured learning notes with any OpenAI-compatible LLM.

license

简体中文

What Problem Does It Solve?

Subtitles from online courses, lectures, and technical talks (SRT / VTT) are cluttered with timestamps and formatting tags. Feeding them directly to an LLM wastes tokens and pollutes the output. CaptionInsight addresses this with three capabilities:

  1. Restore: Convert subtitles into clean spoken text without rewriting or summarizing a single word.
  2. Measure: Use the official tokenizer to calculate the actual token count of the request and report it before submission.
  3. Preserve: Store results in a draggable folder hierarchy for easy archiving, retrieval, and long-term reading in a dedicated reader.

Core Features

Module Capabilities
Accounts & Permissions Registration automatically logs users in; sessions use HttpOnly cookies. Each user has an independent LLM configuration, with API keys stored separately and inaccessible to other users. The first registered user automatically becomes an administrator. Administrators can manage users and enable or disable self-registration.
Subtitle Upload Supports .srt, .vtt, .ass, .ssa, .txt, and .md; multi-select and drag-and-drop; automatically removes sequence numbers, timestamps, and styling tags.
Text Input Bypasses subtitle cleaning entirely; pasted text is used as-is for token counting and processing.
Batch Processing Process multiple files and text inputs in one batch. Each has its own task, status, and result, independent of the others. Existing video subtitles in the library can also be selected directly.
Video Subtitle Download Paste a Bilibili URL (BBDown) or a YouTube URL / 11-character video ID (yt-dlp) to automatically download subtitles and clean them into plain text. Each downloader is installed independently, so either can be used without the other. Multi-part videos produce separate transcripts for each part, grouped into a folder. Asynchronous tasks with visible progress, prioritized language selection, automatic duplicate URL detection, and cancellation of running tasks.
Speech Recognition Fallback If a video has no subtitles, automatically download its audio and transcribe it, labeling the source as "Speech Recognition." The recognition service URL and parameters can be changed in Settings and take effect immediately. Audio is deleted after transcription.
Token Counting Uses the official DeepSeek v4 tokenizer. Counts tokens after serialization with the actual chat template, rather than estimating from character counts. Output tokens are added from the upstream usage data after processing.
Prompt Management Includes five built-in prompts (course summaries, key concepts, study notes, technical deep dives, and Chinese-English bilingual notes). Separate prompt collections for "Video Subtitles" and "Files," each with its own default. Create, edit, delete, duplicate, save as new, drag to reorder, and set defaults. Order and default selections persist on the server.
LLM Provider OpenAI-compatible protocol abstraction; exponential backoff retries for timeouts and 429/5xx errors (4xx errors fail immediately); concurrency semaphore; API keys remain server-side.
Folders & Archiving Unlimited folder nesting, drag-and-drop movement, and cycle validation. Separate Workspace and Archive pages. Deleting a folder does not delete its data.
Reader Markdown and KaTeX equations, indented and navigable table of contents, scroll highlighting, in-document search, code block copying, and Markdown export.
Workspace Overview statistics, processing task progress, and recently processed items.
Real-Time Status Adaptive frontend polling automatically updates the interface when tasks finish, without requiring a page refresh.
Customizable Navigation Reorder top-level navigation items or move less frequently used items into a "More ▾" menu. Preferences are stored server-side and follow the user's account.
Security Extension allowlist, file size limits, filename sanitization to prevent path traversal, DOMPurify sanitization of Markdown output, and API keys never exposed outside the server.

Installation & Running

Requires Python 3.11+ and Node.js 20+.

git clone <your-repository-url> CaptionInsight
cd CaptionInsight

# ---- Backend ----
cd backend
python3 -m venv .venv
.venv/bin/pip install -e ".[dev]"
.venv/bin/uvicorn app.main:app --port 8000
# API documentation: http://127.0.0.1:8000/docs

# ---- Frontend (in a separate terminal) ----
cd frontend
npm install
npm run dev     # http://localhost:5173, with /api proxy to port 8000

For production deployment, build the frontend. The backend will serve the generated assets directly:

cd frontend && npm run build

Configuring the LLM

Both configuration methods can coexist. Priority: current user's web configuration > environment variables > legacy shared database configuration.

Web Configuration (Recommended)

Open the Settings page, enter the API Base URL, model, API key, temperature, and concurrency, then click "Save and Apply." Configuration is stored in the server-side SQLite database and takes effect immediately.

"Test Connectivity" sends an actual request to verify availability. Once saved, API keys are never returned in plaintext by any endpoint; only masked values are shown.

Environment Variables

Set the following in the project's root .env file (see .env.example):

LLM_API_KEY=sk-...
LLM_BASE_URL=https://api.deepseek.com/v1
LLM_MODEL=deepseek-chat

Any OpenAI-compatible service works. For Kimi, Qwen, or local vLLM, simply change BASE_URL.

Trade-off: To support web-based configuration, API keys are stored in plaintext in the backend database. Protection relies on keeping them server-side, excluding them from Git, and restricting file permissions. If this is unacceptable, use environment variables exclusively.

Video Subtitle Download (Optional)

The server needs BBDown (for Bilibili) and ffmpeg.

The application remains fully functional without them; the download interface will simply indicate that the feature is unavailable.

# 1. Place BBDown anywhere and make it executable
chmod +x /usr/local/bin/BBDown/BBDown
apt-get install -y ffmpeg
# Without ffmpeg, BBDown exits with an error message that misleadingly says
# "Video does not exist or is unavailable."

# 2. Confirm the paths in .env (defaults shown; usually no changes needed)
BBDOWN_PATH=/usr/local/bin/BBDown/BBDown
# Only required for subtitles that need an authenticated session
BBDOWN_COOKIE=

YouTube Subtitles (Optional)

YouTube is also supported. Paste a youtube.com/watch?v=… URL, a youtu.be/… short link, or an 11-character video ID. Downloads use yt-dlp, installed in an independent virtual environment:

python3 -m venv /opt/yt-dlp/venv
/opt/yt-dlp/venv/bin/pip install -U yt-dlp

# .env
YTDLP_PYTHON=/opt/yt-dlp/venv/bin/python
# Only required for age-restricted or login-required videos (Netscape format)
YTDLP_COOKIE=
# Required if the server cannot access YouTube directly
YTDLP_PROXY=

The proxy can also be changed in the web Settings page (administrators only). Changes take effect immediately.

The two downloaders are independent: installing only BBDown or only yt-dlp is perfectly valid. If subtitles are unavailable, speech recognition is used as a fallback; in this case, yt-dlp downloads the audio. Playlist and channel URLs are explicitly rejected; only individual videos are supported.

Videos Without Subtitles: Speech Recognition

If a video has neither official nor AI-generated subtitles, its audio is automatically downloaded and sent to a speech recognition service to generate a transcript. The service must provide two endpoints:

GET  {base_url}/api/read?filename=<name>.txt
     -> {"exists":true,"text":"..."}

POST {base_url}/api/transcribe
     multipart field: file
     -> {"success":true,...}

The service URL, polling interval, maximum wait time, and audio size limit can all be changed in the Settings page and take effect immediately. The values in .env serve only as defaults.

Data & Backups

The SQLite database and all uploaded/downloaded files are stored in DATA_DIR (default: ./data). Back up this directory:

sqlite3 data/captioninsight.db ".backup /tmp/backup.db"

License

MIT

About

No description, website, or topics provided.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages