VideoDB Node.js SDK

Video Database for your AI Applications
Explore the docs »

Report Bug · Request Feature

VideoDB Node.js SDK

The VideoDB Node.js SDK lets you upload, understand, search, and edit video from Node.js and TypeScript. Full guides live at docs.videodb.io.

Table of Contents

• Installation
• Connect
• Upload and stream a video
• Understand, index, and search
• Ask questions
• Legacy spoken-word search
• Edit with a timeline
• Handle errors
• Contributing
• License

Installation

npm install videodb

On macOS and Windows the install step also downloads the desktop Capture binary, which only the Capture SDK needs. Set VIDEODB_SKIP_CAPTURE_INSTALL=1 to skip it, for example on servers and in CI. If the download fails, the install still succeeds, and capture fetches the binary on first use.

Connect

Get an API key from the VideoDB console.

import { connect } from 'videodb';

// Reads VIDEO_DB_API_KEY from the environment.
const conn = connect();
// Or pass the key explicitly: connect({ apiKey: 'sk-...' })

const coll = await conn.getCollection(); // the default collection

Upload and stream a video

uploadURL() accepts YouTube links, public URLs, and S3 links. uploadFile() takes a local filePath. Both wait until the media is ready and return a Video, Audio, or Image.

import { Video } from 'videodb';

const video = await coll.uploadURL({
  url: 'https://www.youtube.com/watch?v=LPZh9BOjkQs',
});
if (!(video instanceof Video)) throw new Error('Expected a video');

console.log(video.id, video.name, video.length);
console.log(video.playerUrl); // open in a browser

// Stream only 0-10s and 120-140s
const clipUrl = await video.generateStream([
  [0, 10],
  [120, 140],
]);

Understand, index, and search

Search runs over indexes you build. First run an understanding pass, then index its output.

const understanding = await video.understand([
  { type: 'spoken_words', name: 'transcript' },
]);
await understanding.waitUntilComplete();

const transcript = await understanding.getAnalyzer('transcript');
const index = await video.index(transcript, { name: 'transcript' });
await index.waitUntilComplete();

const results = await video.semanticSearch('what is attention?', {
  indexIds: [index.indexId],
  topK: 5,
});
for (const shot of results.getShots()) {
  console.log(`${shot.start}s-${shot.end}s: ${shot.text}`);
}

// One stream that plays every matching moment
console.log(await results.compile());

Add a vlm analyzer, such as { type: 'vlm', name: 'scene', config: { prompt: 'Describe the visual content.' } }, to search what is on screen as well.

video.search(query) plans the search across all of the video's indexes. If the video has no index yet, the result is empty, and results.warnings explains why.

Ask questions

const answer = await video.ask('What does attention let a transformer do?');
console.log(answer.answer);

coll.search() and coll.ask() work the same way across every video in a collection.

Legacy spoken-word search

Older projects index transcripts with indexSpokenWords() and search them with legacySearch().

await video.indexSpokenWords();
const legacy = await video.legacySearch('attention');
console.log(legacy.getShots().length, await legacy.compile());

Edit with a timeline

import { Clip, EditorTextAsset, EditorTimeline, EditorVideoAsset, Track } from 'videodb';

const timeline = new EditorTimeline(conn);

const main = new Track();
main.addClip(0, new Clip({ asset: new EditorVideoAsset({ id: video.id, start: 0 }), duration: 5 }));
timeline.addTrack(main);

const titles = new Track();
titles.addClip(0, new Clip({ asset: new EditorTextAsset({ text: 'Hello from VideoDB' }), duration: 3 }));
timeline.addTrack(titles);

console.log(await timeline.generateStream());

// Burn in subtitles generated from the transcript
console.log(await video.addSubtitle());

Handle errors

Every error the SDK throws extends VdbBaseError. Errors from the API include the server's explanation in the message, and also carry the HTTP status, a machine-readable code when the server sends one, and the response body in data.

import { VdbBaseError } from 'videodb';

try {
  await coll.getVideo('m-does-not-exist');
} catch (err) {
  if (err instanceof VdbBaseError) {
    console.error(err.message); // "Error 400: Video not found"
    console.error(err.status, err.code); // 400 "400"
  }
}

---

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

---

License