From bc832277bd29548a5ac71bdfb7c347333daffd72 Mon Sep 17 00:00:00 2001 From: raz Date: Mon, 14 Sep 2026 17:39:12 +0300 Subject: [PATCH 1/3] Add the cloudinary-video skill MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Covers building video experiences on the Cloudinary Video Player — adaptive streaming, chapters, captions, transcripts, hotspots and AI analysis — plus migrating media in from Brightcove, Vimeo, YouTube and Wistia. The skill ships with the claim tests it was built against. They assert the load-bearing factual claims (transformation behaviour, async output shapes, player distribution, source-platform limits) against live endpoints, so a claim going stale fails loudly rather than misleading an agent. The default mode needs no credentials; --cloud additionally provisions a cloud and needs media fixtures via MEDIA_DIR, which are too large to vendor here. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 1 + skills/cloudinary-video/SKILL.md | 697 ++++++++++++++++++ .../references/build-patterns.md | 198 +++++ .../references/capabilities.md | 258 +++++++ .../references/field-guide.md | 122 +++ skills/cloudinary-video/references/gotchas.md | 250 +++++++ skills/cloudinary-video/references/player.md | 306 ++++++++ .../references/source-platforms.md | 125 ++++ skills/cloudinary-video/test/lib.sh | 56 ++ skills/cloudinary-video/test/skill-claims.sh | 283 +++++++ 10 files changed, 2296 insertions(+) create mode 100644 skills/cloudinary-video/SKILL.md create mode 100644 skills/cloudinary-video/references/build-patterns.md create mode 100644 skills/cloudinary-video/references/capabilities.md create mode 100644 skills/cloudinary-video/references/field-guide.md create mode 100644 skills/cloudinary-video/references/gotchas.md create mode 100644 skills/cloudinary-video/references/player.md create mode 100644 skills/cloudinary-video/references/source-platforms.md create mode 100644 skills/cloudinary-video/test/lib.sh create mode 100755 skills/cloudinary-video/test/skill-claims.sh diff --git a/README.md b/README.md index c9fe919..125f378 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,7 @@ The agent skills in this package help your AI coding assistant write correct Clo | `cloudinary-transformations` | Turns natural language image and video transformation requirements into valid URL transformation strings that follow Cloudinary best practices. Use when building delivery URLs, applying transformations, optimizing media, or debugging transformation syntax errors. | | `cloudinary-react` | Provides opinionated React SDK patterns for configuration, common integration scenarios, and troubleshooting for frequent errors and TypeScript pitfalls. Use when developing React apps with Cloudinary. | | `cloudinary-next` | Provides opinionated Next.js SDK patterns for Server and Client Component boundaries, server-side uploads and deletes, and troubleshooting for frequent errors and TypeScript pitfalls. Use when developing Next.js apps with Cloudinary. | +| `cloudinary-video` | Builds video experiences on the Cloudinary Video Player — adaptive streaming, chapters, captions, transcripts, hotspots, and AI analysis — and migrates media onto Cloudinary from Brightcove, Vimeo, YouTube, or Wistia. Use when building or improving a video site on Cloudinary. | ## Install diff --git a/skills/cloudinary-video/SKILL.md b/skills/cloudinary-video/SKILL.md new file mode 100644 index 0000000..4e0834a --- /dev/null +++ b/skills/cloudinary-video/SKILL.md @@ -0,0 +1,697 @@ +--- +name: cloudinary-video +description: Build websites with the Cloudinary Video Player — adaptive streaming, chapters, captions, transcripts, hotspots, AI analysis and metadata — on an existing Cloudinary cloud, or migrating the media on first. Use when building or improving a video experience on Cloudinary, moving video off Brightcove, Vimeo, YouTube or Wistia, or standing up a cloud for a demo. +license: MIT +metadata: + author: cloudinary + version: '2.1.0' +--- + +# Building video sites on Cloudinary + +Covers what the published `cloudinary-transformations` and `cloudinary-docs` +skills do not: **the video player, the async AI features, provisioning, and +getting existing media onto a cloud.** Those skills handle transformation URLs. +Use them for that; use this for everything else. + +## How to use this + +You are helping someone reach a goal for their video experience, not applying a +checklist. So: + +- **Ask when the answer changes the build.** The delivery branch, where the + media comes from, which cloud it lands on, whether an add-on exists — a wrong + guess here costs a whole run. One question beats an assumption. The questions + worth asking up front are in *Start here* below. +- **Ask when you are unsure.** That is legitimate and always preferable to + quietly picking a default and presenting it as the decision. Say what you + would pick and why, then let them redirect. +- **Recommend, don't enumerate — in conversation.** Where this skill gives + options, pick the one that fits and say so in a line. Spend their attention + on the thing you built, not on a tour of the choices behind it. +- **But build generously.** The restraint is in what you *say*, never in what + you ship. Build the full experience — chapters, transcript, captions, AI + poster, preview loops, structured data, responsive delivery — not a minimum + viable one. Deleting a feature they do not want takes a minute; never seeing + it means they never knew to ask. Err toward the wow — but the wow is the + experience working, never the page advertising how it works. Ship their site, + not an exhibit about Cloudinary; see *Ship the site, not the demo + scaffolding* in [references/build-patterns.md](references/build-patterns.md). +- **Say what is not there.** A missing add-on, an unreachable original, a + feature Cloudinary does not generate — name it plainly and move on. Silence + and approximation both cost more later. + +The work has two phases, and the first one is often skipped: + +- **Phase A — establish the assets.** Only needed if the media is not already + on a Cloudinary cloud. +- **Phase B — build the site.** The substance: optimization strategy, player + capabilities, analysis and metadata. + +## Start here: get the intent, then the logistics + +Do not begin until you know what they are building. Each answer below changes +the work substantially, so **ask for any you were not given** — one question +costs a message, a wrong guess costs a run. + +**1. What are you building, and what should the video do there?** + +The one question the rest depends on. A product-page loop, a long-form library, +a course, a news page and a landing hero want different delivery, different +player features and different page shapes. Get this and most later choices +follow; skip it and you will build a competent version of the wrong thing. + +**Do not interview them about features.** Do not ask whether they want +captions, chapters, transcripts or hover previews — build them. Someone +describing what they want in chat should get the magic without being asked to +specify it, and should see everything Cloudinary can do for them before +deciding what to keep. + +**2. Do you have an existing website or app — is this a migration or from +scratch?** + +A migration gives you three things at once: the media, the conventions to match +(layout, naming, structure), and human-authored metadata worth more than +anything regenerated — see A2. From scratch, the builder supplies the sources; +do not go looking. + +**3. Are the assets already on a Cloudinary cloud?** + +Yes → **skip phase A entirely.** Go to phase B. Confirm the cloud name, how you +reach it (MCP or `CLOUDINARY_URL` — see the routing table in +[references/capabilities.md](references/capabilities.md)), and which public IDs +to build against. Do not re-upload assets that are already there. + +No → migration is needed, so: + +**4. Where are the resources?** + +| Source | What it means | +| --- | --- | +| **An existing website** | Try to fetch it for them; otherwise ask them to provide or map it. The site is also your source of guidelines and mapping — see below. | +| **From scratch** | The builder provides the asset sources. Do not go looking; ask. | + +**A named real site is never a cue to reach for Cloudinary's `demo` cloud or +any other stock/sample assets.** If the user names a real site or brand +("build a page for gap.com"), you must actually open that site and pull its +real images/video before writing any page code — do not silently substitute +placeholder assets because fetching feels slower or the site has no obvious +video element. Only fall back to demo/stock assets if, after actually trying, +the real assets are genuinely unreachable (login-gated, blocked, no usable +media) — and even then, say so explicitly and ask the user before +substituting anything. Silently swapping in demo-cloud assets for a named +brand is the single failure mode this rule exists to prevent. + +**5. Where does it land — an existing cloud, or a new claimable one?** + +| Destination | Consequences | +| --- | --- | +| Their **existing cloud** | No provisioning. No claim URL. Respect existing folders, naming and presets. Re-runs cost real quota and can overwrite live assets — be idempotent, and never loop the catalogue to probe. | +| A **fresh claimable cloud** | Provision it (below). Expect no add-ons. Surface the claim URL at the end. Provision a new one per test run. | + +Local files into an existing cloud is a legitimate combination — source and +destination are independent choices. + +**Detect what you can, ask what you cannot.** Duration, whether there is an +audio track, resolution and which add-ons a cloud has are all measurable — +`ffprobe` the source, probe one asset. Do not spend a question on something you +can check, and do not guess at something you cannot. + +--- + +# Phase A — establish the assets + +## A1. Get the originals + +**Find the video in the raw HTML, not in a browser's DOM.** `curl` the page +and grep the source *first*: + +```sh +curl -s -A "" "" -o page.html +grep -oE '[A-Za-z0-9_-]+\.(mp4|m3u8|webm|mov)' page.html | sort -u +``` + +**Do not conclude "this site has no video" from `document.querySelectorAll`.** +An automated/headless browser's DOM is systematically *less complete* than +what a real user sees, and it fails silently — you get a clean empty array, +not an error: + +- On a React/Next.js site the `
`, not the +`` fails with `401 Unauthorized`. Cloudinary's fetcher is not one of the +allowed delivery IPs. Upload from a local file instead. + +Claiming the cloud lifts the restriction entirely. + +## Claimable clouds + +- Expire **24 hours** after provisioning unless claimed +- Media delivery is restricted to the IPs in `delivery_ips`; **uploads and API + calls are not restricted**, so a wrong IP looks like a working migration with + broken media +- The CLI's default (no `--ip`) locks delivery to the calling address, which is + usually correct — prefer it over guessing +- Surface the `claim_url` to the user; it is what makes the cloud permanent diff --git a/skills/cloudinary-video/references/player.md b/skills/cloudinary-video/references/player.md new file mode 100644 index 0000000..8bdbe1b --- /dev/null +++ b/skills/cloudinary-video/references/player.md @@ -0,0 +1,306 @@ +# Cloudinary Video Player option shapes + +Verified against `cloudinary-video-player@4.1.0`. Option names come from the +player API reference, not from memory. + +## Use `cloudinary.player()`, not `cloudinary.videoPlayer()` + +The 4.1.0 UMD bundle attaches five things to `window.cloudinary`: +`videoPlayer`, `videoPlayers`, `player`, `players`, `Cloudinary`. Two of those +construct a player and they are **not** the same function: + +| | `cloudinary.videoPlayer()` | `cloudinary.player()` | +| --- | --- | --- | +| Status | legacy constructor | **current** — the bundle tags its own analytics `newPlayerMethod: true` | +| Returns | the player, synchronously | a **Promise** resolving to the player | +| Saved configuration | none — options only | fetches saved config before constructing | + +```js +const player = await cloudinary.player('hero', { + cloudName: 'my-cloud', + publicId: 'my/public-id', // also loads the source + // …local options win over anything fetched +}); +``` + +Because it is async, anything that touches the player (seeking from a +transcript or a timecode chip) must tolerate being called before it resolves — +queue the request and replay it on resolve. + +### Saved configuration: assets and profiles + +Video Player Studio saves settings **onto the video asset** by default. Passing +`publicId` makes the player retrieve them automatically (requires player +3.6.0+), so a page can inherit the styling and behaviour someone configured in +the Studio without hard-coding it. + +`profile` loads a stored *profile* instead — either a built-in +(`cld-default`, `cld-looping`, `cld-adaptive-stream`, `cld-live-streaming`) or +one saved on the cloud, fetched from +`/_applet_/video_service/video_player_profiles/.json`. + +**Profiles are writable over HTTP — no console needed.** When the same +presentation repeats across many assets, that is the signal to create one +rather than repeat it in page code: + +```sh +# create or update; also GET /player/profiles to list, DELETE to remove +curl -X PUT https://api.cloudinary.com/v2/video//player/profiles/ \ + -u : -H "Content-Type: application/json" \ + -d '{"playerOptions":{…},"sourceOptions":{…}}' +``` + +Basic auth with the API key and secret; both `playerOptions` and +`sourceOptions` are required and take arbitrary player/source keys. Full +reference: + +Precedence, and the trap: **specifying a `profile` overrides the asset's own +saved settings.** If you want the per-video configuration to apply, do not pass +a profile. Local options passed at construction beat both. + +Profiles deliberately exclude per-embed values — the video's `publicId` and +`posterOptions` are never part of a profile, so keep passing those yourself. + +`cld-looping` is worth knowing: `{fluid, controls:false, muted:true, +autoplay:true, loop:true}` — the standard hero/background-loop configuration. + +**Precedence is two different mechanisms, not one ranked list.** Getting this +wrong leads to over-specifying the page defensively: + +- **A `profile` replaces the asset's video settings.** It is a substitution: + pass one and the per-asset saved configuration no longer supplies the base. + So a profile cannot be combined with per-video settings — if some setting + genuinely varies per asset, it cannot live behind a profile. +- **Construction options are additive.** They layer *on top* of whichever base + applies and do **not** suppress it. Settings you do not pass keep coming from + the asset (or profile), so a page that sets three options still inherits + everything else that was saved. + +(Cloudinary's own statement; the public profiles API reference does not +document precedence, so do not infer it from the endpoint docs.) + +Practically: save the presentation, pass only what is genuinely page-specific. +Not because inline options would otherwise clobber the saved config — they +won't — but because anything inline is a value nobody can change without a +deploy. + +### What the Studio's own embed looks like + +Worth calibrating against, because it is the shape Cloudinary steers users to. +The Studio's Share dialog offers two modes, and neither produces a wall of +options: + +| mode | generated code | +| --- | --- | +| **Video settings** (asset-saved) | `cloudinary.player('player', { cloudName, publicId })` | +| **Profile settings** | `cloudinary.player('player', { cloudName, publicId, profile: 'cld-default' })` | + +That is the whole call — `fluid` and the rest of the presentation all live in +the asset's video settings, not in the page. The profile picker lists real +reused ones (`pdp`, `abr-4k`, `translations`) beside the `cld-*` built-ins. + +**Configuration belongs on the asset or in a profile — not inline.** This is +the practice, not a judgement call: presentation saved on the asset can be +changed by anyone in Studio, while the same values in page code need a deploy. +Repeat the same presentation across several assets and it should be a profile. + +Inline options are not *broken* — they layer on top rather than suppressing +what was saved — but reach for them only for what genuinely cannot live on the +asset, and expect that to be close to nothing. `cloudName` and the `publicId` +are the call; everything else has a home. + +The Studio also has a raw-JS escape hatch — *Advanced settings editing* — and +scopes it explicitly to "player options that are not currently exposed in the +Studio UI", warning that default player options will be ignored. Treat +hand-written config the same way: last resort, for what the supported surface +does not cover. + +### Saving settings without the console — the Video Config API + +Video Player Studio saves settings through the console UI, but the same +configuration is readable and writable over HTTP, keyed by **`asset_id`** (not +public ID), with Basic auth: + +```sh +# read +curl https://api.cloudinary.com/v2/video//player/config/video/ \ + -u : + +# write — PUT REPLACES the whole document; GET, merge, then PUT to patch +curl -X PUT https://api.cloudinary.com/v2/video//player/config/video/ \ + -u : -H "Content-Type: application/json" \ + -d '{"playerOptions":{…},"sourceOptions":{…}}' +``` + +`playerOptions` takes constructor keys; `sourceOptions` takes `source()` keys +(`sourceTypes`, `transformation`, `posterOptions`, `textTracks`). Get the +`asset_id` from the Admin API — the resource endpoint returns it alongside the +public ID. + +The player reads the same config at delivery time from +`/_applet_/video_service/video_player_config/video//.json`, +which is handy for verifying what a page will actually receive. **An empty +`{"playerOptions":{},"sourceOptions":{}}` and HTTP 200 means nothing is saved** +— the fetch fails silently by design, so a page falls back to its local +options and looks fine while inheriting nothing. + +### Confirming saved config actually applied + +Two checks that avoid false alarms: + +- The player's analytics beacon to `analytics-api-s.cloudinary.com/video_player_source` + spells it out: `newPlayerMethod=true`, `videoConfig=true`, and + `fetchedConfig=` listing exactly which keys came from + the saved config. +- **Do not test captions with `videoEl.textTracks`** — it reads empty because + video.js manages them as *remote* tracks. Use + `videojs.getPlayer(id).remoteTextTracks()`. + +## Constructor options + +These apply to either constructor. + +```js +{ + cloudName: 'my-cloud', + sourceTypes: ['hls'], // ABR only. OMIT for progressive — setting + // ['mp4'] disables the f_auto:video default + fluid: true, // fills its container + controls: true, + showLogo: false, + seekThumbnails: true, // default true + chaptersButton: true, + aiHighlightsGraph: false, // interest curve on the scrub bar + colors: { + accent: '#88c0d0', + base: '#0b1015', + text: '#eceff4', // MUST be light — chrome sits over dark video + }, +} +``` + +`breakpoints` — a **boolean**, and a **progressive-only** option: it picks a +delivery width for the container. It has no meaning under +`sourceTypes: ['hls']`, where the streaming profile's ladder governs +resolution, and setting it there yields a 400 — including when both are set +purely as player options, since the player composes the rejected URL for you. +Both are constructor options, so a player must be **disposed and rebuilt** to +move between the two strategies. See the comparison in +[capabilities.md](capabilities.md). + +Never pass `transformation:{streaming_profile:…}` to `source()`: +`sourceTypes:['hls']` already emits `sp_auto`, and writing it yourself only +collapses it into another directive's component. + +`maxDpr` — number or `true`. Only meaningful **with `breakpoints` on** (Studio +reveals Max DPR only once breakpoints is enabled), so it is progressive-only by +inheritance. + +**You do not need to set it.** Enabling `breakpoints` turns on DPR handling +too: breakpoints chooses the *width* and the device's pixel density is matched +automatically — `maxDpr: true` (auto), which is what Studio's "Auto" writes and +what it defaults to. Setting it is redundant, in the same way writing `f_auto` +for a player-delivered source is. + +Pass a **number** only to deliberately cap below auto — trading sharpness on +dense screens for bytes (Studio offers 1.0, 1.5, 2.0). With a number, effective +DPR is the minimum of `maxDpr`, the device's actual DPR, and 2.0. Only relevant +on the MP4 path. + +## source() + +```js +player.source('my/public-id', { + chapters: true, // loads -chapters.vtt + title: true, // AI-generated title in the title bar + description: true, + + posterOptions: { + transformation: { start_offset: 'auto' }, // AI-selected keyframe + }, + + textTracks: { + captions: { + label: 'English', default: true, // omit url → uses .transcript + maxWords: 3, // also TRIGGERS the .transcript lookup + wordHighlight: true, // highlight each word as spoken + }, + subtitles: [ + { label: 'Español', language: 'es', url: '…/my/public-id.es.transcript' }, + { label: 'Français', language: 'fr', url: '…/my/public-id.fr.transcript' }, + ], + options: { theme: 'player-colors', wordHighlightStyle: { color: 'royalblue' } }, + }, + + interactionAreas: { + enable: true, + template: [ + { left: 8, top: 62, width: 34, height: 30, id: 'SKU-1' }, // percentages + ], + onClick: (event) => addToCart(event?.item?.id), + }, + interactionDisplay: { enable: true, template: 'pulsing' }, // or 'shadowed' +}); +``` + +### chapters + +- `true` — loads `-chapters.vtt` (what `auto_chaptering` produces) +- `{ url: '…' }` — an external VTT +- `{ 1: 'Chapter 1', 6: 'Chapter 2' }` — manual, keys are seconds + +### textTracks + +Omitting `url` on `captions` makes the player resolve the `.transcript` file for +that public ID. Setting `maxWords` also triggers that lookup — it is the +documented trigger, not merely a display cap. Translated transcripts live at +`..transcript` and must be given explicitly. + +`wordHighlight: true` uses the transcript's word-level timings to highlight +words as they are spoken; pair it with `maxWords` so the caption box does not +fill. Style it with `options.wordHighlightStyle`. + +There is **no** native searchable-transcript panel option — see +[build-patterns.md](build-patterns.md). + +`options.theme`: `'default'`, `'videojs-default'`, `'yellow-outlined'`, +`'player-colors'`, `'3d'`. + +### interactionAreas + +`template` is either a preset string — `'portrait'`, `'landscape'`, `'all'`, +`'center'` — or an array of areas. Area coordinates are **percentages** of the +video frame, not pixels. + +The `onClick` handler receives an event with `event.item.id` and an +`event.zoom()` method. + +## React and other frameworks + +Build the `