From b55424ae8d406178cc94a6a9acf736d74f33cb37 Mon Sep 17 00:00:00 2001 From: Aleksandr Shevchuk Date: Fri, 20 Nov 2026 11:43:00 +0200 Subject: [PATCH 1/5] feat(author): ask YouTube and Vimeo whether they have a hosted video (#218) --- SECURITY.md | 2 + design/author.md | 10 +++- packages/author/README.md | 8 ++- packages/author/lib/media.mjs | 74 +++++++++++++++++++++++-- packages/author/tests/build.test.ts | 39 +++++++++++++ packages/author/tests/helpers/oembed.ts | 8 +++ packages/author/vitest.config.ts | 1 + packages/mcp/README.md | 3 +- 8 files changed, 136 insertions(+), 9 deletions(-) create mode 100644 packages/author/tests/helpers/oembed.ts diff --git a/SECURITY.md b/SECURITY.md index abff0fa2..2dd4ed9a 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -79,6 +79,8 @@ model may have written: name and then asks, so it guards against a mistaken or planted address, not a hostile network. A video on YouTube, Vimeo, Panopto or Echo360 is not downloaded: its address is kept, and the player loads it wherever it points. + A YouTube or Vimeo address is sent to that site's oEmbed endpoint, which says + whether it has the video. - The MCP server always builds so, in the folders the person gave it, else the app's roots, as they are given. Given none, it uses Desktop, Documents and Downloads, and never the home folder whole. It never writes over a file, and diff --git a/design/author.md b/design/author.md index a6b0f4d8..4a495fcb 100644 --- a/design/author.md +++ b/design/author.md @@ -92,7 +92,15 @@ text to the content check's rules, and `apps/skills` plays every example untrust Echo360 address stays an address, as the editor embeds it, and only where H5P.Video's own pattern for that host takes it, since an address none matches goes to its HTML5 player, which cannot play a page: a Short or a live stream is written - as its video's watch address, and another form on those hosts is refused by name. + as its video's watch address, and another form on those hosts is refused by name. A YouTube or + Vimeo address is asked of the host's oEmbed, which answers anyone, a page too (#218): an address + holds to its form only, and a made-up id built and showed a dead player. 400 or 404 is a + problem; 401 or 403, a private video or one its owner lets only some sites embed, a note, since + it may play where it will be used; anything else, no network, or no answer in ten seconds says + nothing, and the build goes on. YouTube is asked by the video's watch address, since its oEmbed + answers 404 for the `/embed/` and `/v/` forms of a video it has (2026-11-20); one video named + twice is asked once. Panopto and Echo360 answer no one and are not asked. The tests never ask: + a setup file answers for both hosts (`tests/helpers/oembed.ts`). `build.test.ts` holds each accepted form to the patterns in H5P.Video's scripts. A package with media goes through the normalizer's `normalizeArchive`, stored and laid out to stream. An addon whose pattern the text matches, H5P.MathDisplay for LaTeX, is carried as a site with it diff --git a/packages/author/README.md b/packages/author/README.md index c763b2b0..14986153 100644 --- a/packages/author/README.md +++ b/packages/author/README.md @@ -86,7 +86,10 @@ What `params` takes is what H5P's editor would let an author save in the same fo and, where it gives one, its length, past which an Interactive Video's times are refused. The package is then laid out to stream, media stored and an mp4's index first. A YouTube, Vimeo, Panopto or Echo360 address stays where it is, if it is one H5P.Video plays: a Short's is taken - as its video's, and another address on those sites is refused. A download is given up once + as its video's, and another address on those sites is refused. A YouTube or Vimeo video is + asked of the site's oEmbed, which is sent the video's address: one the site has no video at is + refused, one it says is private or not for every site to embed is a note, and with no answer in + ten seconds the build goes on without one. A download is given up once nothing has arrived for a minute, however long it runs. - **Interface texts** (Check, Retry…): leave them out; they come from the libraries' translations, and the command lists what stayed English. A source may still set one by name. @@ -233,7 +236,8 @@ const { output, source: built, builtWith, manifest, content, libraries, install, and throws a `SourceError` whose `problems` are `{ path, message }`; with `check` it builds and writes no package and takes no output (`buildPackage(source, undefined, { check: true })`), though a media file named by web address is still downloaded, to a temporary folder removed after; a video on -YouTube, Vimeo, Panopto or Echo360 stays an address, as in a build. For a build run +YouTube, Vimeo, Panopto or Echo360 stays an address, as in a build, a YouTube or Vimeo one asked +of the site with the same `fetch`. For a build run on what a model wrote: `within` is a list of the folders the output and every local file the source names must be inside, on their real paths, and its problems then name no path but the one the source wrote. It does not cover a file named by web address, which is downloaded with the `fetch` diff --git a/packages/author/lib/media.mjs b/packages/author/lib/media.mjs index 9362b739..919d1ada 100644 --- a/packages/author/lib/media.mjs +++ b/packages/author/lib/media.mjs @@ -47,8 +47,9 @@ const under = (host, domains) => domains.some((domain) => host === domain || hos * an address by its own pattern for each host, and gives one no pattern matches to its HTML5 * player, which cannot play a page. So `plays` takes only addresses H5P.Video 1.6's pattern for * the host takes as well, and any other address on the host (`on`, told by its name, with no - * pattern to backtrack through its labels) is refused, naming the form. - * @type {{ host: string, mime: string, plays: RegExp, on: (url: URL) => boolean, form: string }[]} + * pattern to backtrack through its labels) is refused, naming the form. `oembed` is where the + * host says whether it has the video, for the two hosts that answer anyone (#218). + * @type {{ host: string, mime: string, plays: RegExp, on: (url: URL) => boolean, form: string, oembed?: (address: string) => string }[]} */ const HOSTED = [ { @@ -56,14 +57,22 @@ const HOSTED = [ mime: 'video/Vimeo', plays: /^https?:\/\/(?:[\w-]+\.)*vimeo\.com\/(?:channels\/[A-Za-z_]+\/|groups\/[A-Za-z_]+\/videos\/)?\d/, on: (url) => under(url.hostname, ['vimeo.com']), - form: 'https://vimeo.com/ and the video\'s number' + form: 'https://vimeo.com/ and the video\'s number', + // Every form it plays, a channel's and a group's too, and an unlisted video's with its hash. + oembed: (address) => `https://vimeo.com/api/oembed.json?url=${encodeURIComponent(address)}` }, { host: 'YouTube', mime: 'video/YouTube', plays: /^https?:\/\/(?:[\w-]+\.)*(?:youtube\.com\/(?:watch\?(?:[^#\s]*&)?v=|embed\/|v\/)|youtu\.be\/)[\w-]{11}/i, on: (url) => under(url.hostname, ['youtube.com', 'youtube-nocookie.com', 'youtu.be']), - form: 'https://www.youtube.com/watch?v= and the video\'s id' + form: 'https://www.youtube.com/watch?v= and the video\'s id', + // Asked by its watch address: YouTube's oEmbed answers 404 for /embed/ and /v/ addresses of a + // video it has (2026-11-20). + oembed: (address) => { + const id = YOUTUBE_ID.exec(address)?.[1] + return id ? `https://www.youtube.com/oembed?format=json&url=${encodeURIComponent(`https://www.youtube.com/watch?v=${id}`)}` : '' + } }, { host: 'Panopto', @@ -81,6 +90,12 @@ const HOSTED = [ } ] +/** A YouTube video's id, in an address H5P.Video plays. */ +const YOUTUBE_ID = /(?:[?&]v=|\/embed\/|\/v\/|youtu\.be\/)([\w-]{11})/i + +/** How long a host may take to say whether it has a video, before the build goes on without it. */ +const ASK_MS = 10_000 + /** A YouTube Short or live stream, by its video's id, which H5P.Video plays at the watch address. */ const YOUTUBE_SHORT = /^https?:\/\/(?:[\w-]+\.)*youtube\.com\/(?:shorts|live)\/([\w-]{11})(?![\w-])/i @@ -263,6 +278,8 @@ export class MediaStore { this.names = new Set() /** @type {Map} a video's or a recording's length in seconds, by its path in the package, where its index gave one */ this.lengths = new Map() + /** @type {Map>} what a host answered of a video, by the address asked, so one video named twice is asked of once */ + this.asked = new Map() } /** @@ -293,7 +310,10 @@ export class MediaStore { const short = YOUTUBE_SHORT.exec(source) const address = short ? `https://www.youtube.com/watch?v=${short[1]}` : source const hosted = HOSTED.find(({ plays }) => plays.test(address)) - if (hosted) return { path: address, mime: hosted.mime, copyright } + if (hosted) { + await this.#there(hosted, address, source, `${at}.path`, report, note) + return { path: address, mime: hosted.mime, copyright } + } const url = URL.canParse(address) ? new URL(address) : null const other = url ? HOSTED.find(({ on }) => on(url)) : undefined if (other) { @@ -432,6 +452,50 @@ export class MediaStore { return { from: found.file, extension, stem: name.slice(0, name.length - extension.length - 1) } } + /** + * Whether a host has a video, asked of its oEmbed endpoint where it has one (#218): an address + * holds to its form only, so a made-up id built and showed a dead player. An address the host + * has no video at is a problem, and one it will not say anything of to anyone, a private video + * or one its owner lets only some sites embed, a note. Any other answer, no network, or no + * answer in ten seconds says nothing either way, and the build goes on without it. + * @param {(typeof HOSTED)[number]} hosted + * @param {string} address as H5P.Video will play it + * @param {string} source as written + * @param {string} at + * @param {(path: string, message: string) => void} report + * @param {(path: string, message: string) => void} note + */ + async #there(hosted, address, source, at, report, note) { + const ask = hosted.oembed?.(address) + if (!ask) return + let answer = this.asked.get(ask) + if (!answer) { + answer = this.#ask(ask) + this.asked.set(ask, answer) + } + const status = await answer + if (status === 400 || status === 404) { + report(at, `${source}: ${hosted.host} has no video there (it answered ${status}): give ${hosted.form} of a public or unlisted video`) + } else if (status === 401 || status === 403) { + note(at, `${source}: ${hosted.host} would not say it may be played elsewhere (it answered ${status}): it may be private, or its owner may let only some sites embed it, so play it where it will be used`) + } + } + + /** + * The status a host's oEmbed endpoint answers, or 0 where it does not answer in time. + * @param {string} url + * @returns {Promise} + */ + async #ask(url) { + try { + const response = await this.fetch(url, { signal: AbortSignal.timeout(ASK_MS) }) + await response.body?.cancel().catch(() => {}) + return response.status + } catch { + return 0 + } + } + /** * A web address downloaded and kept, its type taken from its name or, failing that, from what * the server says it is. diff --git a/packages/author/tests/build.test.ts b/packages/author/tests/build.test.ts index 195d3dcd..5d7f604b 100644 --- a/packages/author/tests/build.test.ts +++ b/packages/author/tests/build.test.ts @@ -840,6 +840,45 @@ describe('media', () => { ]) }) + it('asks YouTube and Vimeo whether they have a hosted video, once each, and says what they answered (#218)', async () => { + const video = (path: string) => ({ content: { library: 'H5P.Video', params: { sources: { path } } } }) + const asked: string[] = [] + const answers: Record = { aaaaaaaaaaa: 400, '999999999': 404, bbbbbbbbbbb: 401, '888888888': 403, ccccccccccc: 500 } + const fetch = async (input: string | URL | Request) => { + const url = String(input) + asked.push(url) + const status = Object.entries(answers).find(([id]) => url.includes(id))?.[1] ?? 200 + return new Response(status === 200 ? '{}' : 'Not Found', { status }) + } + const paths = [ + 'https://www.youtube.com/embed/dQw4w9WgXcQ', 'https://youtu.be/dQw4w9WgXcQ', 'https://vimeo.com/channels/staffpicks/76979871', + 'https://video.example.edu/Panopto/Pages/Viewer.aspx?id=0b6d3a5e', 'https://www.youtube.com/watch?v=bbbbbbbbbbb', + 'https://vimeo.com/888888888', 'https://www.youtube.com/watch?v=ccccccccccc' + ] + const { notes } = await build({ ...META, type: 'H5P.Column', params: { content: paths.map(video) } }, { fetch }) + // A YouTube video by its watch address, which its oEmbed answers for any form; once for the two + // forms of one video; Vimeo's as written; Panopto's not, since it answers no one. + expect(asked).toEqual([ + `https://www.youtube.com/oembed?format=json&url=${encodeURIComponent('https://www.youtube.com/watch?v=dQw4w9WgXcQ')}`, + `https://vimeo.com/api/oembed.json?url=${encodeURIComponent('https://vimeo.com/channels/staffpicks/76979871')}`, + `https://www.youtube.com/oembed?format=json&url=${encodeURIComponent('https://www.youtube.com/watch?v=bbbbbbbbbbb')}`, + `https://vimeo.com/api/oembed.json?url=${encodeURIComponent('https://vimeo.com/888888888')}`, + `https://www.youtube.com/oembed?format=json&url=${encodeURIComponent('https://www.youtube.com/watch?v=ccccccccccc')}` + ]) + // One it will not say anything of is a note; an answer that says nothing either way, nothing. + expect(notes).toEqual([ + { path: 'params.content[4].content.params.sources.path', message: 'https://www.youtube.com/watch?v=bbbbbbbbbbb: YouTube would not say it may be played elsewhere (it answered 401): it may be private, or its owner may let only some sites embed it, so play it where it will be used' }, + { path: 'params.content[5].content.params.sources.path', message: 'https://vimeo.com/888888888: Vimeo would not say it may be played elsewhere (it answered 403): it may be private, or its owner may let only some sites embed it, so play it where it will be used' } + ]) + expect(await problems({ ...META, type: 'H5P.Column', params: { content: ['https://youtu.be/aaaaaaaaaaa', 'https://vimeo.com/999999999'].map(video) } }, { fetch })).toEqual([ + 'params.content[0].content.params.sources.path: https://youtu.be/aaaaaaaaaaa: YouTube has no video there (it answered 400): give https://www.youtube.com/watch?v= and the video\'s id of a public or unlisted video', + 'params.content[1].content.params.sources.path: https://vimeo.com/999999999: Vimeo has no video there (it answered 404): give https://vimeo.com/ and the video\'s number of a public or unlisted video' + ]) + // With no network the build goes on without asking. + const offline = async () => { throw new TypeError('fetch failed') } + expect((await build({ ...META, type: 'H5P.Column', params: { content: [video('https://youtu.be/aaaaaaaaaaa')] } }, { fetch: offline })).notes).toEqual([]) + }) + it('reads a video\'s and a recording\'s first bytes, and refuses one that is not what the field takes (#218)', async () => { await writeFile(join(dir, 'page.mp4'), 'clip') await writeFile(join(dir, 'song.mp3'), 'not a recording at all') diff --git a/packages/author/tests/helpers/oembed.ts b/packages/author/tests/helpers/oembed.ts new file mode 100644 index 00000000..913641a3 --- /dev/null +++ b/packages/author/tests/helpers/oembed.ts @@ -0,0 +1,8 @@ +// The tests never ask YouTube or Vimeo whether they have a video (#218): a build of a hosted +// video's address hears that they do, unless a test passes a `fetch` of its own to say otherwise. +const ask = /^https:\/\/(?:www\.youtube\.com\/oembed|vimeo\.com\/api\/oembed\.json)\?/ +const real = globalThis.fetch +globalThis.fetch = (input, init) => { + const url = input instanceof Request ? input.url : String(input) + return ask.test(url) ? Promise.resolve(Response.json({ type: 'video' })) : real(input, init) +} diff --git a/packages/author/vitest.config.ts b/packages/author/vitest.config.ts index 64c4d80a..e3be2b59 100644 --- a/packages/author/vitest.config.ts +++ b/packages/author/vitest.config.ts @@ -35,6 +35,7 @@ export default defineConfig({ environment: 'node', include: ['tests/**/*.test.ts'], exclude: [...configDefaults.exclude, 'tests/browser/**'], + setupFiles: ['tests/helpers/oembed.ts'], // The package's limit, and no test's own (design/testing.md, #255). These tests build // packages from the library pack and write every content type's bundle, which a CI // runner does two to four times slower than a laptop: on vitest's 5 s a test, three diff --git a/packages/mcp/README.md b/packages/mcp/README.md index 5c481fdc..95557185 100644 --- a/packages/mcp/README.md +++ b/packages/mcp/README.md @@ -58,7 +58,8 @@ The build downloads each picture, video, recording or other file your content na address, and refuses to download from an address on your local network: a guard against a mistaken or planted address, not against a hostile network. A video on YouTube, Vimeo, Panopto or Echo360 is not downloaded: it stays an address, which the player loads wherever it points, as H5P does -anywhere. Math is drawn by the package's own copy of MathJax, which loads nothing from elsewhere. +anywhere. A YouTube or Vimeo address is sent to that site's oEmbed, to ask whether it has the +video. Math is drawn by the package's own copy of MathJax, which loads nothing from elsewhere. The check and the preview play the package from your computer. Your app sends your conversation to its model, as it does without this server. From 8774e4e97783848ddb5f3f7d602b3a4e73c755ee Mon Sep 17 00:00:00 2001 From: Aleksandr Shevchuk Date: Fri, 20 Nov 2026 11:46:17 +0200 Subject: [PATCH 2/5] docs(author): a check asks YouTube and Vimeo too (#218) --- packages/author/lib/node.mjs | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/packages/author/lib/node.mjs b/packages/author/lib/node.mjs index 2d3f4d7a..0587ebc6 100644 --- a/packages/author/lib/node.mjs +++ b/packages/author/lib/node.mjs @@ -41,7 +41,8 @@ import { SourceError } from './source-error.mjs' * list even of one; `check`, to build and write no package: no output is needed or looked at, and * the result's `output` is empty, but a media file named by web address is still downloaded, to a * temporary folder removed after, as the build checks it; a video on YouTube, Vimeo, Panopto or - * Echo360 stays an address, as in a build + * Echo360 stays an address, as in a build, a YouTube or Vimeo one asked of the site's oEmbed with + * `fetch` as a build asks it * @typedef {{ output: string, source: import('./build.mjs').Source, files: string[], notes: Problem[] }} ReadResult * `output`, the source's path; `files`, the media written beside it, by the paths the source * names them by, from its folder; `notes`, what was left out or could not be read back as it is From 7440fa951e10adbdee7a722bf4ea0e02a220acb1 Mon Sep 17 00:00:00 2001 From: Aleksandr Shevchuk Date: Fri, 20 Nov 2026 12:02:10 +0200 Subject: [PATCH 3/5] fix(author): only a host's own answer of a video counts, by what each status was seen to mean (#218) --- design/author.md | 28 +++++++++--- packages/author/README.md | 5 ++- packages/author/lib/media.mjs | 70 +++++++++++++++++++++-------- packages/author/tests/build.test.ts | 60 ++++++++++++++++--------- 4 files changed, 114 insertions(+), 49 deletions(-) diff --git a/design/author.md b/design/author.md index 4a495fcb..11679bc5 100644 --- a/design/author.md +++ b/design/author.md @@ -94,13 +94,27 @@ text to the content check's rules, and `apps/skills` plays every example untrust matches goes to its HTML5 player, which cannot play a page: a Short or a live stream is written as its video's watch address, and another form on those hosts is refused by name. A YouTube or Vimeo address is asked of the host's oEmbed, which answers anyone, a page too (#218): an address - holds to its form only, and a made-up id built and showed a dead player. 400 or 404 is a - problem; 401 or 403, a private video or one its owner lets only some sites embed, a note, since - it may play where it will be used; anything else, no network, or no answer in ten seconds says - nothing, and the build goes on. YouTube is asked by the video's watch address, since its oEmbed - answers 404 for the `/embed/` and `/v/` forms of a video it has (2026-11-20); one video named - twice is asked once. Panopto and Echo360 answer no one and are not asked. The tests never ask: - a setup file answers for both hosts (`tests/helpers/oembed.ts`). + holds to its form only, and a made-up id built and showed a dead player. What a status means is + each host's own list (`answers` in `media.mjs`), as seen on real videos on 2026-11-20. YouTube: + 400 for an id it never had and 404 for a video taken down, both a problem; 403 for a private + video, a note, since its owner can open it; 200 for one that asks a viewer's age; 401 was not + seen, is said to be embedding turned off, and is a note. Vimeo: 404 for no such video, a + private one, and an unlisted one given without the rest of its link, a problem that says so; a + video behind a password or let to some sites only answers 200 (Vimeo's help centre), and its + 403 is Vimeo refusing the asker's network, so it says nothing. + - **Only the host's own answer counts.** Both hosts send a status that is no video with its + words alone ("Bad Request", "Not Found", "Forbidden", Vimeo's "404 Not Found"). A sandbox's + proxy or a school's filter answers for the host with a page of its own, and would otherwise + refuse a build with no way round, or put a note on every video. So a status that means + something counts only with those words (`OWN_WORDS`); with any other it says nothing, as does + any other status, no network, and no answer in ten seconds, and the build goes on. If a host + changes its words, the check goes quiet, not wrong. + - **A network that is not there costs ten seconds once.** Videos are asked one at a time, and + after the first with no answer none is asked again in that build. One video named twice is + asked once. YouTube is asked by the video's watch address, since its oEmbed answers 404 for + the `/embed/` and `/v/` forms of a video it has. + - Panopto and Echo360 answer no one and are not asked. A read asks nothing. The tests never + ask: a setup file answers for both hosts (`tests/helpers/oembed.ts`). `build.test.ts` holds each accepted form to the patterns in H5P.Video's scripts. A package with media goes through the normalizer's `normalizeArchive`, stored and laid out to stream. An addon whose pattern the text matches, H5P.MathDisplay for LaTeX, is carried as a site with it diff --git a/packages/author/README.md b/packages/author/README.md index 14986153..dae4049c 100644 --- a/packages/author/README.md +++ b/packages/author/README.md @@ -88,8 +88,9 @@ What `params` takes is what H5P's editor would let an author save in the same fo Panopto or Echo360 address stays where it is, if it is one H5P.Video plays: a Short's is taken as its video's, and another address on those sites is refused. A YouTube or Vimeo video is asked of the site's oEmbed, which is sent the video's address: one the site has no video at is - refused, one it says is private or not for every site to embed is a note, and with no answer in - ten seconds the build goes on without one. A download is given up once + refused, one YouTube says is private or not to be embedded is a note, and with no answer of the + site's own in ten seconds, behind a proxy that answers for it or with no network, the build goes + on without one. A download is given up once nothing has arrived for a minute, however long it runs. - **Interface texts** (Check, Retry…): leave them out; they come from the libraries' translations, and the command lists what stayed English. A source may still set one by name. diff --git a/packages/author/lib/media.mjs b/packages/author/lib/media.mjs index 919d1ada..ad9874ad 100644 --- a/packages/author/lib/media.mjs +++ b/packages/author/lib/media.mjs @@ -48,8 +48,10 @@ const under = (host, domains) => domains.some((domain) => host === domain || hos * player, which cannot play a page. So `plays` takes only addresses H5P.Video 1.6's pattern for * the host takes as well, and any other address on the host (`on`, told by its name, with no * pattern to backtrack through its labels) is refused, naming the form. `oembed` is where the - * host says whether it has the video, for the two hosts that answer anyone (#218). - * @type {{ host: string, mime: string, plays: RegExp, on: (url: URL) => boolean, form: string, oembed?: (address: string) => string }[]} + * host says whether it has the video, for the two hosts that answer anyone (#218), and `answers` + * what each status it answers there means, as seen on 2026-11-20: a problem (`gone`) or a note + * (`closed`), each with the words for it. A status not listed says nothing. + * @type {{ host: string, mime: string, plays: RegExp, on: (url: URL) => boolean, form: string, oembed?: (address: string) => string, answers?: Record }[]} */ const HOSTED = [ { @@ -59,7 +61,13 @@ const HOSTED = [ on: (url) => under(url.hostname, ['vimeo.com']), form: 'https://vimeo.com/ and the video\'s number', // Every form it plays, a channel's and a group's too, and an unlisted video's with its hash. - oembed: (address) => `https://vimeo.com/api/oembed.json?url=${encodeURIComponent(address)}` + oembed: (address) => `https://vimeo.com/api/oembed.json?url=${encodeURIComponent(address)}`, + // 404 for no such video, a private one, and an unlisted one asked of without the rest of its + // link (Vimeo's help centre). A video behind a password or let to some sites only answers + // 200, and a 403 is Vimeo refusing the asker's network, which says nothing of the video. + answers: { + 404: { gone: 'has no video there that others can play: give https://vimeo.com/ and the number of a public video, or all of an unlisted video\'s link, which has more after the number' } + } }, { host: 'YouTube', @@ -72,6 +80,15 @@ const HOSTED = [ oembed: (address) => { const id = YOUTUBE_ID.exec(address)?.[1] return id ? `https://www.youtube.com/oembed?format=json&url=${encodeURIComponent(`https://www.youtube.com/watch?v=${id}`)}` : '' + }, + // Seen: 400 for an id it never had, 404 for a video taken down, 403 for a private one, and + // 200 for one that asks a viewer's age. 401 was not seen: it is said to be a video whose + // owner has turned embedding off. + answers: { + 400: { gone: 'has no video there: give https://www.youtube.com/watch?v= and the id of a public or unlisted video' }, + 404: { gone: 'has no video there any more: give https://www.youtube.com/watch?v= and the id of a public or unlisted video' }, + 403: { closed: 'says the video is private: nobody but its owner can play it until it is made public or unlisted' }, + 401: { closed: 'would not say the video may be embedded: its owner may have turned embedding off, so play it where it will be used' } } }, { @@ -96,6 +113,14 @@ const YOUTUBE_ID = /(?:[?&]v=|\/embed\/|\/v\/|youtu\.be\/)([\w-]{11})/i /** How long a host may take to say whether it has a video, before the build goes on without it. */ const ASK_MS = 10_000 +/** + * What YouTube and Vimeo send with an answer that is no video: the status's own words and nothing + * else ("Bad Request", "Not Found", "Forbidden", Vimeo's "404 Not Found"). A box between the build + * and the host, a sandbox's proxy or a school's filter, answers with a page of its own, and so does + * Vimeo where it refuses the asker's network: none of those is the host speaking of the video. + */ +const OWN_WORDS = /^(?:\d{3} )?(?:Bad Request|Unauthorized|Forbidden|Not Found)$/i + /** A YouTube Short or live stream, by its video's id, which H5P.Video plays at the watch address. */ const YOUTUBE_SHORT = /^https?:\/\/(?:[\w-]+\.)*youtube\.com\/(?:shorts|live)\/([\w-]{11})(?![\w-])/i @@ -280,6 +305,8 @@ export class MediaStore { this.lengths = new Map() /** @type {Map>} what a host answered of a video, by the address asked, so one video named twice is asked of once */ this.asked = new Map() + /** Whether a host gave no answer at all: with no network, none is asked again in this build. */ + this.unanswered = false } /** @@ -454,10 +481,11 @@ export class MediaStore { /** * Whether a host has a video, asked of its oEmbed endpoint where it has one (#218): an address - * holds to its form only, so a made-up id built and showed a dead player. An address the host - * has no video at is a problem, and one it will not say anything of to anyone, a private video - * or one its owner lets only some sites embed, a note. Any other answer, no network, or no - * answer in ten seconds says nothing either way, and the build goes on without it. + * holds to its form only, so a made-up id built and showed a dead player. What an answer means + * is the host's own list (`answers`): a video that is not there is a problem, and one nobody + * else can play yet a note, since its owner can open it. Any other answer, one that is not the + * host's own (`OWN_WORDS`), no network, or no answer in ten seconds says nothing either way, + * and the build goes on without it. * @param {(typeof HOSTED)[number]} hosted * @param {string} address as H5P.Video will play it * @param {string} source as written @@ -467,30 +495,36 @@ export class MediaStore { */ async #there(hosted, address, source, at, report, note) { const ask = hosted.oembed?.(address) - if (!ask) return + if (!ask || !hosted.answers) return let answer = this.asked.get(ask) if (!answer) { - answer = this.#ask(ask) + // One at a time, and a network that is not there costs its ten seconds once, not a video. + if (this.unanswered) return + answer = this.#ask(ask, hosted.answers) this.asked.set(ask, answer) } const status = await answer - if (status === 400 || status === 404) { - report(at, `${source}: ${hosted.host} has no video there (it answered ${status}): give ${hosted.form} of a public or unlisted video`) - } else if (status === 401 || status === 403) { - note(at, `${source}: ${hosted.host} would not say it may be played elsewhere (it answered ${status}): it may be private, or its owner may let only some sites embed it, so play it where it will be used`) - } + if (status === 0) this.unanswered = true + const meaning = hosted.answers[status] + if (meaning?.gone) report(at, `${source}: ${hosted.host} ${meaning.gone} (it answered ${status})`) + else if (meaning?.closed) note(at, `${source}: ${hosted.host} ${meaning.closed} (it answered ${status})`) } /** - * The status a host's oEmbed endpoint answers, or 0 where it does not answer in time. + * The status a host's oEmbed endpoint answers; 0 where it does not answer in time, and -1 where + * a status that would say something of the video comes with other words than the host's own. * @param {string} url + * @param {Record} answers the statuses that mean something * @returns {Promise} */ - async #ask(url) { + async #ask(url, answers) { try { const response = await this.fetch(url, { signal: AbortSignal.timeout(ASK_MS) }) - await response.body?.cancel().catch(() => {}) - return response.status + if (!Object.hasOwn(answers, response.status)) { + await response.body?.cancel().catch(() => {}) + return response.status + } + return OWN_WORDS.test((await response.text()).trim()) ? response.status : -1 } catch { return 0 } diff --git a/packages/author/tests/build.test.ts b/packages/author/tests/build.test.ts index 5d7f604b..e80af211 100644 --- a/packages/author/tests/build.test.ts +++ b/packages/author/tests/build.test.ts @@ -842,41 +842,57 @@ describe('media', () => { it('asks YouTube and Vimeo whether they have a hosted video, once each, and says what they answered (#218)', async () => { const video = (path: string) => ({ content: { library: 'H5P.Video', params: { sources: { path } } } }) + const column = (paths: string[]) => ({ ...META, type: 'H5P.Column', params: { content: paths.map(video) } }) + const at = (i: number) => `params.content[${i}].content.params.sources.path` const asked: string[] = [] - const answers: Record = { aaaaaaaaaaa: 400, '999999999': 404, bbbbbbbbbbb: 401, '888888888': 403, ccccccccccc: 500 } + // Each host's own words for a status, as it sent them on 2026-11-20, by the video asked of. + const answers: Record = { + aaaaaaaaaaa: [400, 'Bad Request'], bbbbbbbbbbb: [404, 'Not Found'], ccccccccccc: [403, 'Forbidden'], ddddddddddd: [401, 'Unauthorized'], + eeeeeeeeeee: [500, 'Internal Server Error'], '999999999': [404, '404 Not Found'], '888888888': [403, 'Forbidden'], + // A box between the build and the host, answering for it with a page of its own. + fffffffffff: [403, 'BlockedThis site is not allowed on this network.'], '777777777': [404, 'Not Found on this proxy'] + } const fetch = async (input: string | URL | Request) => { const url = String(input) asked.push(url) - const status = Object.entries(answers).find(([id]) => url.includes(id))?.[1] ?? 200 - return new Response(status === 200 ? '{}' : 'Not Found', { status }) + const [status, body] = Object.entries(answers).find(([id]) => url.includes(id))?.[1] ?? [200, '{}'] + return new Response(body, { status }) } - const paths = [ + const youtube = (id: string) => `https://www.youtube.com/oembed?format=json&url=${encodeURIComponent(`https://www.youtube.com/watch?v=${id}`)}` + const vimeo = (address: string) => `https://vimeo.com/api/oembed.json?url=${encodeURIComponent(address)}` + + const { notes } = await build(column([ 'https://www.youtube.com/embed/dQw4w9WgXcQ', 'https://youtu.be/dQw4w9WgXcQ', 'https://vimeo.com/channels/staffpicks/76979871', - 'https://video.example.edu/Panopto/Pages/Viewer.aspx?id=0b6d3a5e', 'https://www.youtube.com/watch?v=bbbbbbbbbbb', - 'https://vimeo.com/888888888', 'https://www.youtube.com/watch?v=ccccccccccc' - ] - const { notes } = await build({ ...META, type: 'H5P.Column', params: { content: paths.map(video) } }, { fetch }) + 'https://video.example.edu/Panopto/Pages/Viewer.aspx?id=0b6d3a5e', 'https://www.youtube.com/watch?v=ccccccccccc', 'https://www.youtube.com/watch?v=ddddddddddd', + 'https://www.youtube.com/watch?v=eeeeeeeeeee', 'https://www.youtube.com/watch?v=fffffffffff', 'https://vimeo.com/888888888', 'https://vimeo.com/777777777' + ]), { fetch }) // A YouTube video by its watch address, which its oEmbed answers for any form; once for the two // forms of one video; Vimeo's as written; Panopto's not, since it answers no one. expect(asked).toEqual([ - `https://www.youtube.com/oembed?format=json&url=${encodeURIComponent('https://www.youtube.com/watch?v=dQw4w9WgXcQ')}`, - `https://vimeo.com/api/oembed.json?url=${encodeURIComponent('https://vimeo.com/channels/staffpicks/76979871')}`, - `https://www.youtube.com/oembed?format=json&url=${encodeURIComponent('https://www.youtube.com/watch?v=bbbbbbbbbbb')}`, - `https://vimeo.com/api/oembed.json?url=${encodeURIComponent('https://vimeo.com/888888888')}`, - `https://www.youtube.com/oembed?format=json&url=${encodeURIComponent('https://www.youtube.com/watch?v=ccccccccccc')}` + youtube('dQw4w9WgXcQ'), vimeo('https://vimeo.com/channels/staffpicks/76979871'), youtube('ccccccccccc'), youtube('ddddddddddd'), + youtube('eeeeeeeeeee'), youtube('fffffffffff'), vimeo('https://vimeo.com/888888888'), vimeo('https://vimeo.com/777777777') ]) - // One it will not say anything of is a note; an answer that says nothing either way, nothing. + // A video nobody else can play yet is a note. Nothing is said of a status that means nothing + // for the host (a 500; Vimeo's 403, which refuses the asker's network), nor of one that comes + // with other words than the host's own. expect(notes).toEqual([ - { path: 'params.content[4].content.params.sources.path', message: 'https://www.youtube.com/watch?v=bbbbbbbbbbb: YouTube would not say it may be played elsewhere (it answered 401): it may be private, or its owner may let only some sites embed it, so play it where it will be used' }, - { path: 'params.content[5].content.params.sources.path', message: 'https://vimeo.com/888888888: Vimeo would not say it may be played elsewhere (it answered 403): it may be private, or its owner may let only some sites embed it, so play it where it will be used' } + { path: at(4), message: 'https://www.youtube.com/watch?v=ccccccccccc: YouTube says the video is private: nobody but its owner can play it until it is made public or unlisted (it answered 403)' }, + { path: at(5), message: 'https://www.youtube.com/watch?v=ddddddddddd: YouTube would not say the video may be embedded: its owner may have turned embedding off, so play it where it will be used (it answered 401)' } ]) - expect(await problems({ ...META, type: 'H5P.Column', params: { content: ['https://youtu.be/aaaaaaaaaaa', 'https://vimeo.com/999999999'].map(video) } }, { fetch })).toEqual([ - 'params.content[0].content.params.sources.path: https://youtu.be/aaaaaaaaaaa: YouTube has no video there (it answered 400): give https://www.youtube.com/watch?v= and the video\'s id of a public or unlisted video', - 'params.content[1].content.params.sources.path: https://vimeo.com/999999999: Vimeo has no video there (it answered 404): give https://vimeo.com/ and the video\'s number of a public or unlisted video' + expect(await problems(column(['https://youtu.be/aaaaaaaaaaa', 'https://www.youtube.com/watch?v=bbbbbbbbbbb', 'https://vimeo.com/999999999']), { fetch })).toEqual([ + `${at(0)}: https://youtu.be/aaaaaaaaaaa: YouTube has no video there: give https://www.youtube.com/watch?v= and the id of a public or unlisted video (it answered 400)`, + `${at(1)}: https://www.youtube.com/watch?v=bbbbbbbbbbb: YouTube has no video there any more: give https://www.youtube.com/watch?v= and the id of a public or unlisted video (it answered 404)`, + `${at(2)}: https://vimeo.com/999999999: Vimeo has no video there that others can play: give https://vimeo.com/ and the number of a public video, or all of an unlisted video's link, which has more after the number (it answered 404)` ]) - // With no network the build goes on without asking. - const offline = async () => { throw new TypeError('fetch failed') } - expect((await build({ ...META, type: 'H5P.Column', params: { content: [video('https://youtu.be/aaaaaaaaaaa')] } }, { fetch: offline })).notes).toEqual([]) + + // With no network the build goes on, and asks once, not once a video. + let tries = 0 + const offline = async () => { + tries++ + throw new TypeError('fetch failed') + } + expect((await build(column(['https://youtu.be/aaaaaaaaaaa', 'https://vimeo.com/999999999', 'https://youtu.be/bbbbbbbbbbb']), { fetch: offline })).notes).toEqual([]) + expect(tries).toBe(1) }) it('reads a video\'s and a recording\'s first bytes, and refuses one that is not what the field takes (#218)', async () => { From 6e9ef9ecc22480398cf4ded1eeb7e97413a2e707 Mon Sep 17 00:00:00 2001 From: Aleksandr Shevchuk Date: Fri, 20 Nov 2026 12:11:13 +0200 Subject: [PATCH 4/5] fix(author): a host's answer is told by its words for the status and its type, and a silent host leaves the other asked (#218) --- design/author.md | 23 ++++++------ packages/author/README.md | 6 ++-- packages/author/lib/media.mjs | 56 +++++++++++++++++------------ packages/author/tests/build.test.ts | 54 +++++++++++++++++----------- 4 files changed, 84 insertions(+), 55 deletions(-) diff --git a/design/author.md b/design/author.md index 11679bc5..964f8107 100644 --- a/design/author.md +++ b/design/author.md @@ -102,16 +102,19 @@ text to the content check's rules, and `apps/skills` plays every example untrust private one, and an unlisted one given without the rest of its link, a problem that says so; a video behind a password or let to some sites only answers 200 (Vimeo's help centre), and its 403 is Vimeo refusing the asker's network, so it says nothing. - - **Only the host's own answer counts.** Both hosts send a status that is no video with its - words alone ("Bad Request", "Not Found", "Forbidden", Vimeo's "404 Not Found"). A sandbox's - proxy or a school's filter answers for the host with a page of its own, and would otherwise - refuse a build with no way round, or put a note on every video. So a status that means - something counts only with those words (`OWN_WORDS`); with any other it says nothing, as does - any other status, no network, and no answer in ten seconds, and the build goes on. If a host - changes its words, the check goes quiet, not wrong. - - **A network that is not there costs ten seconds once.** Videos are asked one at a time, and - after the first with no answer none is asked again in that build. One video named twice is - asked once. YouTube is asked by the video's watch address, since its oEmbed answers 404 for + - **Only the host's own answer counts.** A sandbox's proxy or a school's filter answers for the + host, and would otherwise refuse a build with no way round, or put a note on every video. + Both hosts send a status that is no video in a way of their own (`sends`): YouTube that + status's words alone ("Bad Request", "Not Found", "Forbidden") under its JSON type, Vimeo a + page of them ("404 Not Found", `text/html`). A status that means something counts only when + it comes so, its own words and the host's type; a page of the box's own, the words under + another type as a small proxy sends them, or another status's words say nothing, as do any + other status, no network, and no answer in ten seconds, and the build goes on. A box that + copies both the words and the type would still be taken for the host: none was seen. If a + host changes how it answers, the check goes quiet, not wrong. + - **A host that does not answer costs ten seconds once.** Videos are asked one at a time, and + after the first with no answer that host is not asked again in that build; the other still + is. One video named twice is asked once. YouTube is asked by the video's watch address, since its oEmbed answers 404 for the `/embed/` and `/v/` forms of a video it has. - Panopto and Echo360 answer no one and are not asked. A read asks nothing. The tests never ask: a setup file answers for both hosts (`tests/helpers/oembed.ts`). diff --git a/packages/author/README.md b/packages/author/README.md index dae4049c..d5db4a30 100644 --- a/packages/author/README.md +++ b/packages/author/README.md @@ -88,9 +88,9 @@ What `params` takes is what H5P's editor would let an author save in the same fo Panopto or Echo360 address stays where it is, if it is one H5P.Video plays: a Short's is taken as its video's, and another address on those sites is refused. A YouTube or Vimeo video is asked of the site's oEmbed, which is sent the video's address: one the site has no video at is - refused, one YouTube says is private or not to be embedded is a note, and with no answer of the - site's own in ten seconds, behind a proxy that answers for it or with no network, the build goes - on without one. A download is given up once + refused, one YouTube says is private is a note, as is its 401, taken to mean that embedding is + turned off (not seen yet), and with no answer of the site's own in ten seconds, behind a proxy + that answers for it or with no network, the build goes on without one. A download is given up once nothing has arrived for a minute, however long it runs. - **Interface texts** (Check, Retry…): leave them out; they come from the libraries' translations, and the command lists what stayed English. A source may still set one by name. diff --git a/packages/author/lib/media.mjs b/packages/author/lib/media.mjs index ad9874ad..a39b0226 100644 --- a/packages/author/lib/media.mjs +++ b/packages/author/lib/media.mjs @@ -50,8 +50,10 @@ const under = (host, domains) => domains.some((domain) => host === domain || hos * pattern to backtrack through its labels) is refused, naming the form. `oembed` is where the * host says whether it has the video, for the two hosts that answer anyone (#218), and `answers` * what each status it answers there means, as seen on 2026-11-20: a problem (`gone`) or a note - * (`closed`), each with the words for it. A status not listed says nothing. - * @type {{ host: string, mime: string, plays: RegExp, on: (url: URL) => boolean, form: string, oembed?: (address: string) => string, answers?: Record }[]} + * (`closed`), each with the words for it. A status not listed says nothing. `sends` is how the + * host itself sends such a status, its body and the type it gives it, by which its own answer is + * told from one a box in between gives for it (`#ask`). + * @type {{ host: string, mime: string, plays: RegExp, on: (url: URL) => boolean, form: string, oembed?: (address: string) => string, sends?: { type: string, body: (status: number) => string }, answers?: Record }[]} */ const HOSTED = [ { @@ -62,6 +64,8 @@ const HOSTED = [ form: 'https://vimeo.com/ and the video\'s number', // Every form it plays, a channel's and a group's too, and an unlisted video's with its hash. oembed: (address) => `https://vimeo.com/api/oembed.json?url=${encodeURIComponent(address)}`, + // Its 404 is a page of the words alone. + sends: { type: 'text/html', body: (status) => `${status} ${STATUS_WORDS[status]}` }, // 404 for no such video, a private one, and an unlisted one asked of without the rest of its // link (Vimeo's help centre). A video behind a password or let to some sites only answers // 200, and a 403 is Vimeo refusing the asker's network, which says nothing of the video. @@ -81,6 +85,8 @@ const HOSTED = [ const id = YOUTUBE_ID.exec(address)?.[1] return id ? `https://www.youtube.com/oembed?format=json&url=${encodeURIComponent(`https://www.youtube.com/watch?v=${id}`)}` : '' }, + // The status's words alone, under the type of the answer it would have given. + sends: { type: 'application/json', body: (status) => STATUS_WORDS[status] }, // Seen: 400 for an id it never had, 404 for a video taken down, 403 for a private one, and // 200 for one that asks a viewer's age. 401 was not seen: it is said to be a video whose // owner has turned embedding off. @@ -88,7 +94,7 @@ const HOSTED = [ 400: { gone: 'has no video there: give https://www.youtube.com/watch?v= and the id of a public or unlisted video' }, 404: { gone: 'has no video there any more: give https://www.youtube.com/watch?v= and the id of a public or unlisted video' }, 403: { closed: 'says the video is private: nobody but its owner can play it until it is made public or unlisted' }, - 401: { closed: 'would not say the video may be embedded: its owner may have turned embedding off, so play it where it will be used' } + 401: { closed: 'would not say the video may be embedded, which is taken to mean its owner has turned embedding off: play it where it will be used' } } }, { @@ -114,12 +120,13 @@ const YOUTUBE_ID = /(?:[?&]v=|\/embed\/|\/v\/|youtu\.be\/)([\w-]{11})/i const ASK_MS = 10_000 /** - * What YouTube and Vimeo send with an answer that is no video: the status's own words and nothing - * else ("Bad Request", "Not Found", "Forbidden", Vimeo's "404 Not Found"). A box between the build - * and the host, a sandbox's proxy or a school's filter, answers with a page of its own, and so does - * Vimeo where it refuses the asker's network: none of those is the host speaking of the video. + * A status's words, which YouTube and Vimeo send alone with an answer that is no video ("Bad + * Request" under YouTube's JSON type, "404 Not Found" as Vimeo's page): each host's `sends`. A box + * between the build and the host, a sandbox's proxy or a school's filter, answers with a page of + * its own or under a type of its own, and so does Vimeo where it refuses the asker's network: none + * of those is the host speaking of the video. */ -const OWN_WORDS = /^(?:\d{3} )?(?:Bad Request|Unauthorized|Forbidden|Not Found)$/i +const STATUS_WORDS = /** @type {Record} */ ({ 400: 'Bad Request', 401: 'Unauthorized', 403: 'Forbidden', 404: 'Not Found' }) /** A YouTube Short or live stream, by its video's id, which H5P.Video plays at the watch address. */ const YOUTUBE_SHORT = /^https?:\/\/(?:[\w-]+\.)*youtube\.com\/(?:shorts|live)\/([\w-]{11})(?![\w-])/i @@ -305,8 +312,8 @@ export class MediaStore { this.lengths = new Map() /** @type {Map>} what a host answered of a video, by the address asked, so one video named twice is asked of once */ this.asked = new Map() - /** Whether a host gave no answer at all: with no network, none is asked again in this build. */ - this.unanswered = false + /** @type {Set} the hosts that gave no answer at all, none of which is asked again in this build */ + this.unanswered = new Set() } /** @@ -484,8 +491,8 @@ export class MediaStore { * holds to its form only, so a made-up id built and showed a dead player. What an answer means * is the host's own list (`answers`): a video that is not there is a problem, and one nobody * else can play yet a note, since its owner can open it. Any other answer, one that is not the - * host's own (`OWN_WORDS`), no network, or no answer in ten seconds says nothing either way, - * and the build goes on without it. + * host's own (`#ask`), no network, or no answer in ten seconds says nothing either way, and the + * build goes on without it. * @param {(typeof HOSTED)[number]} hosted * @param {string} address as H5P.Video will play it * @param {string} source as written @@ -498,13 +505,13 @@ export class MediaStore { if (!ask || !hosted.answers) return let answer = this.asked.get(ask) if (!answer) { - // One at a time, and a network that is not there costs its ten seconds once, not a video. - if (this.unanswered) return - answer = this.#ask(ask, hosted.answers) + // One at a time, and a host that does not answer costs its ten seconds once, not a video. + if (this.unanswered.has(hosted.host)) return + answer = this.#ask(ask, hosted) this.asked.set(ask, answer) } const status = await answer - if (status === 0) this.unanswered = true + if (status === 0) this.unanswered.add(hosted.host) const meaning = hosted.answers[status] if (meaning?.gone) report(at, `${source}: ${hosted.host} ${meaning.gone} (it answered ${status})`) else if (meaning?.closed) note(at, `${source}: ${hosted.host} ${meaning.closed} (it answered ${status})`) @@ -512,19 +519,24 @@ export class MediaStore { /** * The status a host's oEmbed endpoint answers; 0 where it does not answer in time, and -1 where - * a status that would say something of the video comes with other words than the host's own. + * a status that would say something of the video is not sent as the host sends it (`sends`): + * that status's own words and nothing else, under the host's type. * @param {string} url - * @param {Record} answers the statuses that mean something + * @param {(typeof HOSTED)[number]} hosted * @returns {Promise} */ - async #ask(url, answers) { + async #ask(url, hosted) { try { const response = await this.fetch(url, { signal: AbortSignal.timeout(ASK_MS) }) - if (!Object.hasOwn(answers, response.status)) { + const { status } = response + if (!hosted.answers || !hosted.sends || !Object.hasOwn(hosted.answers, status)) { await response.body?.cancel().catch(() => {}) - return response.status + return status } - return OWN_WORDS.test((await response.text()).trim()) ? response.status : -1 + const type = (response.headers.get('content-type') ?? '').split(';')[0].trim().toLowerCase() + const own = type === hosted.sends.type && (await response.text()).trim() === hosted.sends.body(status) + if (!own) await response.body?.cancel().catch(() => {}) + return own ? status : -1 } catch { return 0 } diff --git a/packages/author/tests/build.test.ts b/packages/author/tests/build.test.ts index e80af211..16153190 100644 --- a/packages/author/tests/build.test.ts +++ b/packages/author/tests/build.test.ts @@ -845,18 +845,23 @@ describe('media', () => { const column = (paths: string[]) => ({ ...META, type: 'H5P.Column', params: { content: paths.map(video) } }) const at = (i: number) => `params.content[${i}].content.params.sources.path` const asked: string[] = [] - // Each host's own words for a status, as it sent them on 2026-11-20, by the video asked of. - const answers: Record = { - aaaaaaaaaaa: [400, 'Bad Request'], bbbbbbbbbbb: [404, 'Not Found'], ccccccccccc: [403, 'Forbidden'], ddddddddddd: [401, 'Unauthorized'], - eeeeeeeeeee: [500, 'Internal Server Error'], '999999999': [404, '404 Not Found'], '888888888': [403, 'Forbidden'], - // A box between the build and the host, answering for it with a page of its own. - fffffffffff: [403, 'BlockedThis site is not allowed on this network.'], '777777777': [404, 'Not Found on this proxy'] + const json = 'application/json; charset=UTF-8' + const page = 'text/html; charset=UTF-8' + // Each host's own answer for a status, as it sent it on 2026-11-20, by the video asked of: + // YouTube the status's words under its JSON type, Vimeo a page of them. + const answers: Record = { + aaaaaaaaaaa: [400, 'Bad Request', json], bbbbbbbbbbb: [404, 'Not Found', json], ccccccccccc: [403, 'Forbidden', json], ddddddddddd: [401, 'Unauthorized', json], + eeeeeeeeeee: [500, 'Internal Server Error', json], '999999999': [404, '404 Not Found', page], '888888888': [403, '403 Forbidden', page], + // A box between the build and the host, answering for it: with a page of its own, with the + // status's words alone under its own type, as a small proxy does, and with another status's. + fffffffffff: [403, 'BlockedThis site is not allowed on this network.', page], ggggggggggg: [403, 'Forbidden\n', 'text/plain; charset=utf-8'], + hhhhhhhhhhh: [404, 'Forbidden', json], '777777777': [404, '404 page not found', 'text/plain; charset=utf-8'], '666666666': [404, 'Not Found', page] } const fetch = async (input: string | URL | Request) => { const url = String(input) asked.push(url) - const [status, body] = Object.entries(answers).find(([id]) => url.includes(id))?.[1] ?? [200, '{}'] - return new Response(body, { status }) + const [status, body, type] = Object.entries(answers).find(([id]) => url.includes(id))?.[1] ?? [200, '{}', json] + return new Response(body, { status, headers: { 'content-type': type } }) } const youtube = (id: string) => `https://www.youtube.com/oembed?format=json&url=${encodeURIComponent(`https://www.youtube.com/watch?v=${id}`)}` const vimeo = (address: string) => `https://vimeo.com/api/oembed.json?url=${encodeURIComponent(address)}` @@ -864,20 +869,21 @@ describe('media', () => { const { notes } = await build(column([ 'https://www.youtube.com/embed/dQw4w9WgXcQ', 'https://youtu.be/dQw4w9WgXcQ', 'https://vimeo.com/channels/staffpicks/76979871', 'https://video.example.edu/Panopto/Pages/Viewer.aspx?id=0b6d3a5e', 'https://www.youtube.com/watch?v=ccccccccccc', 'https://www.youtube.com/watch?v=ddddddddddd', - 'https://www.youtube.com/watch?v=eeeeeeeeeee', 'https://www.youtube.com/watch?v=fffffffffff', 'https://vimeo.com/888888888', 'https://vimeo.com/777777777' + 'https://www.youtube.com/watch?v=eeeeeeeeeee', 'https://www.youtube.com/watch?v=fffffffffff', 'https://www.youtube.com/watch?v=ggggggggggg', + 'https://www.youtube.com/watch?v=hhhhhhhhhhh', 'https://vimeo.com/888888888', 'https://vimeo.com/777777777', 'https://vimeo.com/666666666' ]), { fetch }) // A YouTube video by its watch address, which its oEmbed answers for any form; once for the two // forms of one video; Vimeo's as written; Panopto's not, since it answers no one. expect(asked).toEqual([ - youtube('dQw4w9WgXcQ'), vimeo('https://vimeo.com/channels/staffpicks/76979871'), youtube('ccccccccccc'), youtube('ddddddddddd'), - youtube('eeeeeeeeeee'), youtube('fffffffffff'), vimeo('https://vimeo.com/888888888'), vimeo('https://vimeo.com/777777777') + youtube('dQw4w9WgXcQ'), vimeo('https://vimeo.com/channels/staffpicks/76979871'), youtube('ccccccccccc'), youtube('ddddddddddd'), youtube('eeeeeeeeeee'), + youtube('fffffffffff'), youtube('ggggggggggg'), youtube('hhhhhhhhhhh'), vimeo('https://vimeo.com/888888888'), vimeo('https://vimeo.com/777777777'), vimeo('https://vimeo.com/666666666') ]) // A video nobody else can play yet is a note. Nothing is said of a status that means nothing - // for the host (a 500; Vimeo's 403, which refuses the asker's network), nor of one that comes - // with other words than the host's own. + // for the host (a 500; Vimeo's 403, which refuses the asker's network), nor of one that is not + // sent as the host sends it: other words, another status's, or under another type. expect(notes).toEqual([ { path: at(4), message: 'https://www.youtube.com/watch?v=ccccccccccc: YouTube says the video is private: nobody but its owner can play it until it is made public or unlisted (it answered 403)' }, - { path: at(5), message: 'https://www.youtube.com/watch?v=ddddddddddd: YouTube would not say the video may be embedded: its owner may have turned embedding off, so play it where it will be used (it answered 401)' } + { path: at(5), message: 'https://www.youtube.com/watch?v=ddddddddddd: YouTube would not say the video may be embedded, which is taken to mean its owner has turned embedding off: play it where it will be used (it answered 401)' } ]) expect(await problems(column(['https://youtu.be/aaaaaaaaaaa', 'https://www.youtube.com/watch?v=bbbbbbbbbbb', 'https://vimeo.com/999999999']), { fetch })).toEqual([ `${at(0)}: https://youtu.be/aaaaaaaaaaa: YouTube has no video there: give https://www.youtube.com/watch?v= and the id of a public or unlisted video (it answered 400)`, @@ -885,14 +891,22 @@ describe('media', () => { `${at(2)}: https://vimeo.com/999999999: Vimeo has no video there that others can play: give https://vimeo.com/ and the number of a public video, or all of an unlisted video's link, which has more after the number (it answered 404)` ]) - // With no network the build goes on, and asks once, not once a video. - let tries = 0 - const offline = async () => { - tries++ + // With no network the build goes on, and asks each host once, not once a video. + const tried: string[] = [] + const offline = async (input: string | URL | Request) => { + tried.push(new URL(String(input)).hostname) throw new TypeError('fetch failed') } - expect((await build(column(['https://youtu.be/aaaaaaaaaaa', 'https://vimeo.com/999999999', 'https://youtu.be/bbbbbbbbbbb']), { fetch: offline })).notes).toEqual([]) - expect(tries).toBe(1) + expect((await build(column(['https://youtu.be/aaaaaaaaaaa', 'https://vimeo.com/999999999', 'https://youtu.be/bbbbbbbbbbb', 'https://vimeo.com/888888888']), { fetch: offline })).notes).toEqual([]) + expect(tried).toEqual(['www.youtube.com', 'vimeo.com']) + // And one host that does not answer leaves the other asked: its video that is not there is still refused. + const vimeoDown = async (input: string | URL | Request) => { + if (String(input).startsWith('https://vimeo.com/')) throw new TypeError('fetch failed') + return fetch(input) + } + expect(await problems(column(['https://vimeo.com/999999999', 'https://youtu.be/aaaaaaaaaaa']), { fetch: vimeoDown })).toEqual([ + `${at(1)}: https://youtu.be/aaaaaaaaaaa: YouTube has no video there: give https://www.youtube.com/watch?v= and the id of a public or unlisted video (it answered 400)` + ]) }) it('reads a video\'s and a recording\'s first bytes, and refuses one that is not what the field takes (#218)', async () => { From 0be552956bc940f773f2e54d897ffae053c15e4a Mon Sep 17 00:00:00 2001 From: Aleksandr Shevchuk Date: Fri, 20 Nov 2026 12:16:32 +0200 Subject: [PATCH 5/5] docs(author): the words expected of YouTube's 401, with the others (#218) --- design/author.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/design/author.md b/design/author.md index 964f8107..504d25b9 100644 --- a/design/author.md +++ b/design/author.md @@ -105,8 +105,9 @@ text to the content check's rules, and `apps/skills` plays every example untrust - **Only the host's own answer counts.** A sandbox's proxy or a school's filter answers for the host, and would otherwise refuse a build with no way round, or put a note on every video. Both hosts send a status that is no video in a way of their own (`sends`): YouTube that - status's words alone ("Bad Request", "Not Found", "Forbidden") under its JSON type, Vimeo a - page of them ("404 Not Found", `text/html`). A status that means something counts only when + status's words alone ("Bad Request", "Forbidden", "Not Found", and "Unauthorized" expected + of the 401 that was not seen) under its JSON type, Vimeo a page of them ("404 Not Found", + `text/html`). A status that means something counts only when it comes so, its own words and the host's type; a page of the box's own, the words under another type as a small proxy sends them, or another status's words say nothing, as do any other status, no network, and no answer in ten seconds, and the build goes on. A box that