From db8870add6bd6994b87528dcbf510d73175ab5cc Mon Sep 17 00:00:00 2001 From: Aleksandr Shevchuk Date: Wed, 18 Nov 2026 15:54:19 +0200 Subject: [PATCH 1/5] feat(author): read a package back into its source, h5p-author read (#236) readPackage and h5p-author read write the source a build would take to make a package again, its media beside it: its own fields from h5p.json, its content less what a build fills in, by the generator's own rules walked back (ContentGenerator.written). The source is built again in memory and held to the content as checked. Content for other versions of its libraries than the pack's is refused, with both named. --- design/author.md | 39 +++- packages/author/README.md | 43 +++- packages/author/bin/h5p-author.mjs | 17 +- packages/author/lib/author.mjs | 7 +- packages/author/lib/generator.mjs | 208 ++++++++++++++++++- packages/author/lib/media.mjs | 19 +- packages/author/lib/node.mjs | 135 +++++++++++- packages/author/lib/read.mjs | 316 +++++++++++++++++++++++++++++ packages/author/lib/text.mjs | 33 ++- packages/author/lib/types.mjs | 19 ++ packages/author/tests/read.test.ts | 231 +++++++++++++++++++++ 11 files changed, 1045 insertions(+), 22 deletions(-) create mode 100644 packages/author/lib/read.mjs create mode 100644 packages/author/tests/read.test.ts diff --git a/design/author.md b/design/author.md index b9549b6..d98000f 100644 --- a/design/author.md +++ b/design/author.md @@ -175,6 +175,38 @@ text to the content check's rules, and `apps/skills` plays every example untrust too, as the editor's `initSubContent` gives one, since Summary names a statement's statements by it. Of the owner's 88 real packages, 61 report nothing; the others hold what their editor let an author save, such as an empty alt text or styles pasted from Word. +- **A package reads back into a source, by the generator's own rules walked back** (#236, step 3 + of #219). `readPackage` and `h5p-author read` write the source a build would take to make the + package again, its media beside it at the paths the source names them by. + - Its own fields are `h5p.json`'s; a package with no author reads as "Unknown". `params` are + `content.json` as `checkContent` returns it, less what a build fills in, worked out by + `ContentGenerator.written`: of each field, what the generator writes there with nothing + written, at the field's own level with the saved values beside it taken as written, so a + `showWhen` reads them as it reads a source. The libraries are loaded in the content's + language, so an interface text equal to its translation drops out as any default does. + - Left out besides: a part's id, and its metadata but its title; a title the build gives, its + content type's or Question Set's own (the layout's `read`); and what H5P's editor saves for an + author who never touched it, as `checkContent` takes it: an empty text or choice, a list it + started whose items are only an item's defaults (Interactive Video's text track with no file), + and a default's content nobody opened (its summary task). Markup that is only the blocks a + build wraps typing in is written as the typing (`unwrapText`, held by `wrapText`); Game Map's + neighbours go back to numbers. Positions and sizes are kept as written. + - The source is then built in memory, the package's files standing for themselves, and held to + the content as checked: a text, a number, a flag or a title that comes back otherwise is + written as stored, anything else is a note, and so is what a build of it would refuse. + - Only content for the pack's major.minor of every library it names is read; any other is + refused with both versions named, since moving it is the library's own upgrade, the runtime's + (GPL). Of the owner's 88 real packages, 58 were refused so, mostly Interactive Video 1.27 and + Course Presentation 1.25 and 1.26. Of the other 30, 26 build as read; 4 need what their notes + name: pictures missing from the package, styles pasted from Word, a question with one answer, + and H5P.Row's column widths with decimals, which its editor sets and the number rule refuses. + - What a source has no field for is said and left out: the author's comments, the change log, + the licence's extra terms, and a part's own licence or authors, a build giving every part the + source's. + - An archive is untrusted: only the media the content names are written, under the folder; one + whose entry leads out of its folders is not read. Nothing is written over, and nothing at all + where a name is taken. Every type file's example, built, read and built again, is the same + content but for the ids, with no note (`read.test.ts`). - **A page builds too** (#108). The package has three entries: `.` for Node, `./core` for what needs neither Node nor a page, and `./browser`, the core and a page's `openPack` and `buildPackage`. They are subpaths and not the `browser` condition, which TypeScript ignores @@ -416,6 +448,7 @@ text to the content check's rules, and `apps/skills` plays every example untrust errors, a sandbox that downloads only what a browsing tool has opened, a code tool that can be unavailable for hours. -Not built: editing an existing package, which a tool beside it does with `checkContent` as its -check; layout for Drag and Drop and Crossword; and any judgement of whether content is right, which -stays the person's, in the verifier's preview. +Not built: changing a package in place, which a tool beside it does with `checkContent` as its +check (a package is changed by reading it into a source and building that); reading content made +for another version of a library than the pack's; layout for Drag and Drop and Crossword; and any +judgement of whether content is right, which stays the person's, in the verifier's preview. diff --git a/packages/author/README.md b/packages/author/README.md index b0cd981..5795da5 100644 --- a/packages/author/README.md +++ b/packages/author/README.md @@ -5,8 +5,9 @@ type, and only the fields that differ from its defaults. The rest comes from the `semantics.json` and their translations for the content's language; the libraries come from [@missing-elements/h5p-libraries](https://www.npmjs.com/package/@missing-elements/h5p-libraries), the pack this release pins, each checked against the pack's SHA-256 index. No H5P server, no -editor, no network but for media named by web address. Needs Node 22 or later; write `@latest`, -since without it npm on an older Node quietly installs an older release. +editor, no network but for media named by web address. `read` turns a package back into its +source, to change it and build it again. Needs Node 22 or later; write `@latest`, since without it +npm on an older Node quietly installs an older release. ```bash npx @missing-elements/h5p-author@latest build rivers.json rivers.h5p @@ -147,6 +148,32 @@ packs them as the zips chat apps install, and lists the build tool and each bund skill's route 3, from two files, fetches at [missing-elements.github.io/h5p/skill/](https://missing-elements.github.io/h5p/skill/). +## A package back to its source + +```bash +npx @missing-elements/h5p-author@latest read rivers.h5p rivers +``` + +``` +Read /work/rivers.h5p: H5P.Blanks. Wrote its source to /work/rivers/rivers.source.json. +Change it, then build it again: h5p-author build rivers/rivers.source.json +``` + +The source a build would take to make the package again, to change it and build it again: the +content type and the metadata from its `h5p.json`, and of its content only what differs from the +defaults, interface texts in the content's language included. Its pictures, videos and other files +go beside the source, at the paths it names them by. Positions and sizes are kept, so a slide or a +video keeps its layout. Each part gets a new id when it is built again. + +It reads content made for the pack's versions of its libraries, and refuses any other, naming both +versions: H5P moves content to a newer version as it plays it, and saved again on a site that has +the pack's versions, it reads. What a source has no field for is said and left out: the author's comments, +the change log, a part's own licence or authors (a build gives every part the source's). The source +is built again in memory before it is written, and what would not come back as it is, or what a +build would refuse, such as styles pasted from Word, is said, to fix before building. It never +writes over a file. An archive is untrusted: only the files its content names are written, and +only under the folder. + ## A content type's bundle ```bash @@ -212,6 +239,12 @@ const source = JSON.parse(await readFile(join(folder, 'rivers.json'), 'utf8')) const { outline } = await buildPackage(source, join(folder, 'rivers.h5p'), { base: folder, within: [folder], fetch: publicFetch }) ``` +`readPackage(file, folder, { libraries, within })` reads a package back into its source, as +`h5p-author read` does, and returns `{ output, source, files, notes }`: the source's path, the +source, the media written beside it by the paths it names them by, and what was left out or would +not come back as it is, each `{ path, message }`. With `within`, the package and the folder must be +inside those folders, on their real paths. + A result's `libraries` are the folders the package plays with, `install` those carried for an H5P site's upload check. `missingForInstall(libraries)` is that check, over a package's `library.json` files: what a site with no library installed would @@ -343,8 +376,8 @@ the package's main export has them. A tool beside a build in either imports it f It checks structure, not pedagogy: whether the answers are right, the questions fair, or the pictures placed well is for a person to judge, in -[h5p-verify](https://www.npmjs.com/package/@missing-elements/h5p-verify)'s `--preview`. It does not -edit an existing package itself: `checkContent` is the check for a tool that does. The licence of -each library is in the pack's `libraries.txt`. +[h5p-verify](https://www.npmjs.com/package/@missing-elements/h5p-verify)'s `--preview`. It changes an +existing package only by way of its source, read and built again; `checkContent` is the check for a +tool that changes one in place. The licence of each library is in the pack's `libraries.txt`. MIT. diff --git a/packages/author/bin/h5p-author.mjs b/packages/author/bin/h5p-author.mjs index ba25762..e0fd498 100755 --- a/packages/author/bin/h5p-author.mjs +++ b/packages/author/bin/h5p-author.mjs @@ -1,13 +1,14 @@ #!/usr/bin/env node import { readFile } from 'node:fs/promises' -import { dirname, resolve } from 'node:path' +import { dirname, relative, resolve } from 'node:path' import { parseArgs } from 'node:util' -import { buildPackage, schemaOf, SourceError, writeBundle, writeBundles } from '../lib/author.mjs' +import { buildPackage, readPackage, schemaOf, SourceError, writeBundle, writeBundles } from '../lib/author.mjs' import { packVersion } from '../lib/node.mjs' const USAGE = `Usage: h5p-author build [--libraries ] h5p-author build --check [--libraries ] + h5p-author read [--libraries ] h5p-author schema [--markdown] [--all] [--libraries ] h5p-author bundle [--libraries ] h5p-author bundle --all [--libraries ] @@ -16,6 +17,9 @@ build Makes a playable .h5p from a source: { "type", "title", "language", "lic "authors", "params" }, or a quiz in the h5p-author skill's quiz format. Writes only when the whole source is right, and never over an existing file; --check lists the problems and notes and writes no package. +read Writes the source a build would take to make a package again into a folder, + with the package's media beside it: to change it and build it again. Reads + content made for the pack's versions of its libraries; never writes over a file. schema Prints a content type's fields as a source writes them: --markdown as a skill's type file carries them, --all with the interface texts. bundle Writes the libraries a build of one content type reads, as a zip of a parts/ @@ -112,6 +116,15 @@ try { } for (const { path, message } of result.notes) console.warn(`Note: ${path} ${message}`) if (!check) console.log('Not verified: run npx @missing-elements/h5p-verify@latest on it and look at its screenshot before handing it over.') + } else if (command === 'read') { + if (rest.length !== 2) throw new Error('read takes a package and a folder to write its source in: h5p-author read quiz.h5p quiz') + for (const flag of Object.keys(flags)) if (flag !== 'libraries') throw new Error(`read does not take --${flag}`) + const { output, source, files, notes } = await readPackage(rest[0], rest[1], { libraries: /** @type {string | undefined} */ (flags.libraries) }) + const media = files.length ? `, and ${files.length} media file${files.length === 1 ? '' : 's'} beside it` : '' + console.log(`Read ${resolve(rest[0])}: ${source.type}. Wrote its source to ${output}${media}.`) + for (const { path, message } of notes) console.warn(`Note: ${path} ${message}`) + const near = relative(process.cwd(), output) + console.log(`Change it, then build it again: h5p-author build ${near.startsWith('..') ? output : near} `) } else if (command === 'bundle') { for (const flag of Object.keys(flags)) if (flag !== 'all' && flag !== 'libraries') throw new Error(`bundle does not take --${flag}`) const libraries = /** @type {string | undefined} */ (flags.libraries) diff --git a/packages/author/lib/author.mjs b/packages/author/lib/author.mjs index 98bc5d2..2ca75e0 100644 --- a/packages/author/lib/author.mjs +++ b/packages/author/lib/author.mjs @@ -5,7 +5,8 @@ * package pins, checked against its index. A tool beside the build takes the libraries as the build * loads them (`openPack`, `loadLibraries`, and `packageLibraries` for a package's own), its rules * for text (`markupText`, `plainText`, `allowedTags`), and the check it runs last on what it writes, - * for saved content (`checkContent`). A content type's bundle (`writeBundle`) carries the parts a + * for saved content (`checkContent`). `readPackage` reads a package back into its source, to change + * it and build it again. A content type's bundle (`writeBundle`) carries the parts a * build of it reads, for a build where the pack cannot be installed. This is the entry for Node: * `./core` is what of it needs no Node, and `./browser` builds in a page. */ @@ -16,12 +17,14 @@ import { schemaLines, schemaMarkdown } from './schema.mjs' import { LAYOUTS } from './types.mjs' export * from './core.mjs' -export { buildPackage, inside, openPack, publicFetch, writeBundle, writeBundles } from './node.mjs' +export { buildPackage, inside, openPack, publicFetch, readPackage, writeBundle, writeBundles } from './node.mjs' /** * @typedef {import('./node.mjs').BuildResult} BuildResult * @typedef {import('./node.mjs').BuildOptions} BuildOptions * @typedef {import('./node.mjs').BundleResult} BundleResult + * @typedef {import('./node.mjs').ReadResult} ReadResult + * @typedef {import('./node.mjs').ReadOptions} ReadOptions */ /** diff --git a/packages/author/lib/generator.mjs b/packages/author/lib/generator.mjs index 75b483b..2815986 100644 --- a/packages/author/lib/generator.mjs +++ b/packages/author/lib/generator.mjs @@ -1,7 +1,8 @@ +import { writtenCopyright } from './media.mjs' import { checkMetadata } from './metadata.mjs' import { machineNameOf, namingKey } from './names.mjs' import { isInterfaceText } from './schema.mjs' -import { allowedTags, markupText, plainText, stripTags, takesMarkup } from './text.mjs' +import { allowedTags, blockTag, escapeText, markupText, plainText, stripTags, takesMarkup, unwrapText } from './text.mjs' /** * Content for a content type, made from its own `semantics.json` as H5P's editor saves a form: @@ -140,6 +141,10 @@ export class ContentGenerator { * where the source has it, and where the content does, outermost first */ this.moved = [] + /** @type {Record[]} the files saved content names, as saved, collected as `written` writes them */ + this.stored = [] + /** @type {string[]} what `written` left out as H5P's editor started it for nobody: a list (`#started`), a default's content (`#unopened`) */ + this.unfilled = [] } /** @@ -265,6 +270,177 @@ export class ContentGenerator { } } + /** + * Saved parameters written as a source writes them: the generation walked back (#236). Of each + * field, what this generator would write there with nothing written is left out. That is worked + * out at the field's own level, with the saved values beside it taken as written, so that a + * `showWhen` reads them as it reads a source; and with the libraries loaded in the content's + * language, so that an interface text equal to its translation drops out as any default does. A + * group, a list and sub-content are written as what differs within them; sub-content as its + * library's machine name, its parameters and its title, unless that is its content type's, which a + * build gives a part with none; a file as its path, and its copyright where it gives one. Every + * other value is written as it is stored: text, numbers, positions, a layout's ids. A part's id and + * the rest of its metadata are left out, since a build gives each part a new id and every part the + * source's licence and authors. The files are kept, as saved, in `stored`. + * @param {PackEntry} entry the content type's, as loaded for the content + * @param {unknown} saved its parameters, as `checkContent` returned them + * @returns {Record} + */ + written(entry, saved) { + return this.#writtenFields(entry.semantics ?? [], saved, [entry.library.machineName], 'params') + } + + /** + * A level's saved values, as a source writes them: each that is not what its field holds when + * nothing is written for it, as `written` says. + * @param {any[]} fields + * @param {unknown} saved + * @param {string[]} stack + * @param {string} path + * @returns {Record} + */ + #writtenFields(fields, saved, stack, path) { + /** @type {Record} */ + const out = {} + if (!isObject(saved)) return out + this.#level(fields, saved, () => { + for (const field of fields) { + const value = saved[field.name] + if (value === undefined) continue + const shown = this.#shown(field) + const at = `${path}.${field.name}` + // In a written level, as a build generates it: nothing in it is taken as left closed. + const unwritten = this.#quietly(() => this.field(shown, undefined, stack, false, at)) + // An empty text or choice where nothing is written is what H5P's editor saves for a field + // nobody filled, as `checkContent` takes it. + if (sameContent(value, unwritten) || (value === '' && unwritten === undefined)) continue + if (this.#started(shown, value, stack, at) || this.#unopened(shown, value, stack, at)) { + this.unfilled.push(at) + continue + } + out[field.name] = this.#writtenValue(shown, value, stack, at) + } + }) + return out + } + + /** + * Whether a saved list is one H5P's editor started for an author who never filled it: optional, + * and every item what an item is with nothing written, as the text track an Interactive Video's + * editor starts with, which has no file. `checkContent` asks nothing of such an item; a build + * asks it for what it lacks, so a source leaves the list out. + * @param {any} field + * @param {unknown} value + * @param {string[]} stack + * @param {string} path + */ + #started(field, value, stack, path) { + if (field.type !== 'list' || !field.optional || !Array.isArray(value) || !value.length) return false + const item = this.#copy(field.field, { optional: false }) + return this.#level([], {}, () => value.every((each, index) => sameContent(each, this.#quietly(() => this.field(item, undefined, stack, false, `${path}[${index}]`))))) + } + + /** + * Whether saved sub-content is the content a field gives as its default, as the editor started it + * for an author who never opened it: Interactive Video's summary task, whose parameters, written + * as a source writes them, are nothing but empty items. `checkContent` asks nothing of it, a build + * leaves it as the editor does where nothing is written, and a source leaves it out. + * @param {any} field + * @param {unknown} value + * @param {string[]} stack + * @param {string} path + */ + #unopened(field, value, stack, path) { + if (field.type !== 'library' || !isObject(field.default) || typeof field.default.library !== 'string' || !isObject(value) || typeof value.library !== 'string') return false + if (value.library.trim().split(/\s+/)[0] !== field.default.library.trim().split(/\s+/)[0]) return false + const counts = [this.stored.length, this.unfilled.length] + try { + const written = this.#writtenPart(field, value, stack, path) + return isObject(written) && vacant(written.params) + } finally { + this.stored.length = counts[0] + this.unfilled.length = counts[1] + } + } + + /** + * A saved value as a source writes it for its field: a group, a list and sub-content as what + * differs within them, a file as its path and copyright, anything else as it is. + * @param {any} field + * @param {unknown} value + * @param {string[]} stack + * @param {string} path + * @returns {unknown} + */ + #writtenValue(field, value, stack, path) { + switch (field.type) { + case 'group': { + const children = field.fields ?? [] + // A group of one field is saved as that field's value, and still a level for `showWhen`. + if (children.length === 1 && !field.isSubContent) { + const [only] = children + return this.#level(children, { [only.name]: value }, () => this.#writtenValue(this.#shown(only), value, stack, path)) + } + if (!isObject(value)) return structuredClone(value) + if (field.widget && field.widget !== 'showWhen' && field.widget !== 'none') this.#draw(children) + return this.#writtenFields(children, value, stack, path) + } + case 'list': { + if (!Array.isArray(value)) return structuredClone(value) + const item = this.#copy(field.field, { optional: false }) + // Every item is written, an item with its defaults alone as what makes it: `{}` for a group. + return this.#level([], {}, () => value.map((each, index) => this.#writtenValue(item, each, stack, `${path}[${index}]`))) + } + case 'library': + return this.#writtenPart(field, value, stack, path) + case 'image': + case 'file': + case 'video': + case 'audio': + return Array.isArray(value) ? value.map((each) => this.#writtenFile(each)) : this.#writtenFile(value) + case 'text': + // Markup that is only the blocks a build wraps typing in is written as the typing. + return typeof value === 'string' && takesMarkup(field) ? unwrapText(value, blockTag(field)) ?? value : value + default: + return structuredClone(value) + } + } + + /** + * Saved sub-content as a source writes it: its library by machine name, with its version only + * where the field takes more than one major of it; its parameters as what differs from their + * defaults; and its title, unless that is its content type's, which a build gives a part with + * none. + * @param {any} field + * @param {unknown} value + * @param {string[]} stack + * @param {string} path + */ + #writtenPart(field, value, stack, path) { + if (!isObject(value) || typeof value.library !== 'string') return structuredClone(value) + const match = /^\s*(\S+)\s+(\d+)\.(\d+)\s*$/.exec(value.library) + const entry = match ? this.find(match[1], Number(match[2])) : null + if (!match || !entry) return { library: value.library, params: structuredClone(value.params ?? {}) } + const majors = new Set(libraryOptions(field).filter((option) => option.machineName === match[1]).map((option) => option.major)) + const library = majors.size > 1 ? `${match[1]} ${match[2]}.${match[3]}` : match[1] + const params = this.#writtenFields(entry.semantics ?? [], value.params, [...stack, match[1]], `${path}.params`) + const title = isObject(value.metadata) ? value.metadata.title : undefined + const own = typeof title === 'string' && title.trim() !== '' && title !== escapeText(String(entry.library.title ?? match[1])) + return { library, params, ...(own ? { metadata: { title } } : {}) } + } + + /** + * A saved file as a source writes it: its path, and its copyright where it gives one + * (`writtenCopyright`). Kept as saved in `stored`. + * @param {unknown} value + */ + #writtenFile(value) { + if (!isObject(value) || typeof value.path !== 'string') return structuredClone(value) + this.stored.push(value) + const copyright = writtenCopyright(value.copyright) + return { path: value.path, ...(copyright ? { copyright } : {}) } + } + /** A sub-content id: random, or the caller's, numbered, where it wants the same bytes every run. */ nextId() { this.ids += 1 @@ -1088,6 +1264,36 @@ function holds(field, value) { !child.optional && child.default === undefined && child.type !== 'boolean' && holds(child, value[child.name])) } +/** + * Whether written content holds nothing but empty objects and lists: items an editor started and + * nobody filled. + * @param {unknown} value + * @returns {boolean} + */ +function vacant(value) { + if (Array.isArray(value)) return value.every(vacant) + return isObject(value) && Object.values(value).every(vacant) +} + +/** + * Whether two values are the same content: equal, but for what a build gives each part anew, its + * id and its metadata. A key whose value is `undefined` is no key, and nor is one whose value is + * empty text, which H5P's editor saves for a field nobody filled. + * @param {unknown} a + * @param {unknown} b + * @returns {boolean} + */ +function sameContent(a, b) { + if (Object.is(a, b)) return true + if (Array.isArray(a) || Array.isArray(b)) return Array.isArray(a) && Array.isArray(b) && a.length === b.length && a.every((each, i) => sameContent(each, b[i])) + if (!isObject(a) || !isObject(b)) return false + const part = typeof a.library === 'string' || typeof b.library === 'string' + /** @param {Record} object */ + const keys = (object) => Object.keys(object).filter((key) => object[key] !== undefined && object[key] !== '' && key !== 'subContentId' && !(part && key === 'metadata')) + const own = keys(a) + return own.length === keys(b).length && own.every((key) => sameContent(a[key], b[key])) +} + /** * Why saved content may hold a field the semantics it is checked against do not name. * @param {{ library: any, minor: number }} other diff --git a/packages/author/lib/media.mjs b/packages/author/lib/media.mjs index 95f3ee8..080a356 100644 --- a/packages/author/lib/media.mjs +++ b/packages/author/lib/media.mjs @@ -87,6 +87,23 @@ const YOUTUBE_SHORT = /^https?:\/\/(?:[\w-]+\.)*youtube\.com\/(?:shorts|live)\/( /** The licences a file's copyright may name, as H5P lists them. */ const COPYRIGHT_LICENSES = ['U', 'CC BY', 'CC BY-SA', 'CC BY-ND', 'CC BY-NC', 'CC BY-NC-SA', 'CC BY-NC-ND', 'GNU GPL', 'PD', 'C'] +/** The parts of a file's copyright a source writes, as H5P records them. */ +const COPYRIGHT_FIELDS = ['license', 'author', 'title', 'year', 'source', 'version'] + +/** + * A saved file's copyright as a source writes it (#236): its parts that hold text, and nothing + * where that is the licence "U" alone, which a build gives a file with none. H5P's editor saves + * every part, empty where nobody wrote it, and a build refuses an empty one. + * @param {unknown} copyright + * @returns {Record | undefined} + * @internal + */ +export function writtenCopyright(copyright) { + if (!copyright || typeof copyright !== 'object' || Array.isArray(copyright)) return undefined + const kept = Object.fromEntries(Object.entries(copyright).filter(([key, value]) => COPYRIGHT_FIELDS.includes(key) && typeof value === 'string' && value.trim() !== '')) + return Object.entries(kept).some(([key, value]) => key !== 'license' || value !== 'U') ? kept : undefined +} + /** The most a download may bring: a package of this size is already past what a learner waits for. */ const MAX_DOWNLOAD = 1024 ** 3 @@ -327,7 +344,7 @@ export class MediaStore { /** @type {Record} */ const out = { license: 'U' } for (const [key, entry] of Object.entries(value)) { - if (!['license', 'author', 'title', 'year', 'source', 'version'].includes(key)) { + if (!COPYRIGHT_FIELDS.includes(key)) { report(`${at}.${key}`, 'is not part of a copyright; write license, author, title, year, source or version') continue } diff --git a/packages/author/lib/node.mjs b/packages/author/lib/node.mjs index 46de1c6..22f3762 100644 --- a/packages/author/lib/node.mjs +++ b/packages/author/lib/node.mjs @@ -1,4 +1,4 @@ -import { constants, createReadStream, createWriteStream } from 'node:fs' +import { constants, createReadStream, createWriteStream, openAsBlob } from 'node:fs' import { copyFile, lstat, mkdir, mkdtemp, open, readFile, realpath, rm, stat } from 'node:fs/promises' import { lookup } from 'node:dns/promises' import { tmpdir } from 'node:os' @@ -6,12 +6,13 @@ import { BlockList, isIP } from 'node:net' import { basename, dirname, isAbsolute, join, relative, resolve, sep } from 'node:path' import { Readable, Writable } from 'node:stream' import { fileURLToPath, pathToFileURL } from 'node:url' -import { Uint8ArrayReader, ZipWriter } from '@zip.js/zip.js' +import { BlobReader, ERR_UNSAFE_FILENAME, TextWriter, Uint8ArrayReader, ZipReader, ZipWriter } from '@zip.js/zip.js' import { normalizeArchive } from '@missing-elements/h5p-normalize' import { composePackage, sourceOf, typeNames, writePackage } from './build.mjs' import { bundleLibraries, contentTypes } from './bundle.mjs' import { MediaStore } from './media.mjs' import { Pack } from './pack.mjs' +import { readContent } from './read.mjs' import { SourceError } from './source-error.mjs' /** @@ -38,6 +39,12 @@ import { SourceError } from './source-error.mjs' * the result's `output` is empty, but a media file named by web address is still downloaded, to a * temporary folder removed after, as the build checks it; a video on YouTube, Vimeo, Panopto or * Echo360 stays an address, as in a build + * @typedef {{ output: string, source: import('./build.mjs').Source, files: string[], notes: Problem[] }} ReadResult + * `output`, the source's path; `files`, the media written beside it, by the paths the source + * names them by, from its folder; `notes`, what was left out or could not be read back as it is + * @typedef {{ libraries?: string, within?: string[] }} ReadOptions + * `libraries`, a library pack's parts folder in place of the pinned pack; `within`, the folders + * the package and the folder written in must be inside, on their real paths, a list even of one * @typedef {{ output: string, type: string, parts: string[], size: number }} BundleResult * `type`, the content type with the version bundled, `H5P.QuestionSet 1.21`; `parts`, the * folders of the parts it holds; `size`, the zip's in bytes @@ -425,6 +432,130 @@ export async function buildPackage(input, output, options = {}) { } } +/** + * Whether a media path a package's content names is one a read writes a file at, under its folder: + * names joined by `/`, none of them empty, `.` or `..`, and none with a character a file name + * cannot have on Windows, macOS or Linux. An archive is untrusted, and its content names paths. + * @param {string} path + */ +function writable(path) { + return path.split('/').every((name) => name !== '' && name !== '.' && name !== '..' && !/[\\:*?"<>|\u0000-\u001f]/.test(name)) +} + +/** + * A package read back into the source a build would take to make it again (#236), to change it and + * build it again: written into `folder`, made where it is not there, as `.source.json`, with + * the package's media beside it at the paths the source names them by. Only the media the content + * names are written, and only under the folder, since an archive is untrusted. Nothing is written + * over: where a name is taken, nothing is written, and what was written is taken away again where + * a later write fails. + * @param {string} file the package, a .h5p + * @param {string} folder + * @param {ReadOptions} [options] + * @returns {Promise} + */ +export async function readPackage(file, folder, options = {}) { + const within = foldersOf(options.within) + const input = resolve(file) + const target = resolve(folder) + if (within && !(await inside(input, within))) throw new Error(`${input} is outside the folders this read may use`) + if (within && !(await inside(target, within))) throw new Error(`${target} is outside the folders this read may write in`) + const blob = await openAsBlob(input).catch((error) => { + throw new Error(`${input} cannot be read: ${error?.code === 'ENOENT' ? 'there is no such file' : error instanceof Error ? error.message : error}`) + }) + const reader = new ZipReader(new BlobReader(blob)) + try { + /** @type {Map} */ + const entries = new Map() + try { + for (const entry of await reader.getEntries()) if (!entry.directory) entries.set(entry.filename, entry) + } catch (error) { + // zip.js refuses a name that leads out of the archive's folders, as `../` does. + if (error instanceof Error && error.message === ERR_UNSAFE_FILENAME) { + throw new Error(`${input} has an entry named ${JSON.stringify(/** @type {{ filename?: unknown }} */ (error).filename)}, which leads out of its folders: an archive like that is not read`) + } + throw new Error(`${input} is not a zip, as a .h5p is`, { cause: error }) + } + const json = async (/** @type {string} */ name) => { + const entry = entries.get(name) + if (!entry) throw new Error(`${input} has no ${name}: it is not an H5P package`) + try { + return JSON.parse(await entry.getData(new TextWriter())) + } catch (error) { + throw new Error(`${name} in ${input} is not JSON`, { cause: error }) + } + } + const saved = { manifest: await json('h5p.json'), content: await json('content/content.json') } + const pack = await openPack(options.libraries) + let read + try { + read = await readContent(pack, saved) + } finally { + await pack.close() + } + const notes = [...read.notes] + const output = join(target, `${basename(input).replace(/\.h5p$/i, '') || 'package'}.source.json`) + /** @type {{ entry: import('@zip.js/zip.js').FileEntry, to: string, path: string }[]} */ + const media = [] + for (const path of read.files) { + if (!writable(path)) { + notes.push({ path: 'params', message: `names the file ${JSON.stringify(path)}, which is no path a read writes a file at: not written` }) + continue + } + let decoded = path + try { + decoded = decodeURIComponent(path) + } catch { + // Not encoded: the name as it is. + } + const entry = entries.get(`content/${path}`) ?? entries.get(`content/${decoded}`) + if (!entry) notes.push({ path: 'params', message: `names the file ${path}, which is not in the package: a build of this source asks for it` }) + else media.push({ entry, to: join(target, ...path.split('/')), path }) + } + // Every name looked at before anything is written, so a read that cannot write all of it + // writes nothing. + for (const to of [output, ...media.map((each) => each.to)]) { + if (await lstat(to).then(() => true, () => false)) throw exists(to, 'a read') + } + await mkdir(target, { recursive: true }) + /** @type {{ path: string, made: Made }[]} */ + const made = [] + /** @param {string} to @param {(file: import('node:fs/promises').FileHandle) => Promise} write */ + const create = async (to, write) => { + await mkdir(dirname(to), { recursive: true }) + // A folder on the way may be a link out, which a confined read does not write through. + if (within && !(await inside(to, within))) throw new Error(`${to} is outside the folders this read may write in`) + const handle = await open(to, 'wx').catch((error) => { + throw error?.code === 'EEXIST' ? exists(to, 'a read') : error + }) + try { + const { dev, ino } = await handle.stat({ bigint: true }) + made.push({ path: to, made: { dev, ino } }) + await write(handle) + } finally { + await handle.close() + } + } + try { + for (const { entry, to } of media) { + // Through the handle itself: a stream made from it would hold it open past the last byte. + await create(to, (handle) => entry.getData(new WritableStream({ + async write(chunk) { + for (let done = 0; done < chunk.length;) done += (await handle.write(chunk, done, chunk.length - done)).bytesWritten + } + })).then(() => undefined)) + } + await create(output, (handle) => handle.writeFile(`${JSON.stringify(read.source, null, 2)}\n`)) + } catch (error) { + await Promise.all(made.map(({ path, made: file }) => removeIfMade(path, file))) + throw error + } + return { output, source: read.source, files: media.map((each) => each.path), notes } + } finally { + await reader.close() + } +} + /** The date on every entry of a bundle, as on the pack's parts: a bundle of the same pack is the same bytes. */ const BUNDLE_DATE = new Date(2000, 0, 1, 2) diff --git a/packages/author/lib/read.mjs b/packages/author/lib/read.mjs new file mode 100644 index 0000000..37f4da6 --- /dev/null +++ b/packages/author/lib/read.mjs @@ -0,0 +1,316 @@ +import { composePackage, sourceOf, typeNames } from './build.mjs' +import { checkContent, layoutContext, valueAt } from './check.mjs' +import { ContentGenerator, librariesIn } from './generator.mjs' +import { loadLibraries } from './libraries.mjs' +import { MediaStore, writtenCopyright } from './media.mjs' +import { CC_VERSIONS, LICENSES, ROLES } from './metadata.mjs' +import { SourceError } from './source-error.mjs' +import { sourceSchema } from './source-schema.mjs' +import { hasMarkup, wrapText } from './text.mjs' +import { LAYOUTS, placesOf } from './types.mjs' + +/** + * A package read back into a source (#236): the source a build would take to make the package + * again, to change it and build it again. Its own fields come from `h5p.json`; its parameters from + * `content.json`, less what a build fills in, by the generator's own rule walked back + * (`ContentGenerator.written`) and the layouts' `read`. The source is then built in memory, and a + * value that comes back otherwise is written as stored. No Node import of its own: the package's + * files are read by the caller and the pack is handed in, as a build's are. + */ + +/** + * @typedef {import('./build.mjs').Source} Source + * @typedef {import('./build.mjs').Problem} Problem + * @typedef {{ source: Source, files: string[], notes: Problem[] }} ReadContent + * `files`, the package's media the source names, by their paths in its `content/` folder, as + * the source names them; `notes`, what was left out or could not be read back as it is + */ + +/** @param {unknown} value @returns {value is Record} */ +const isObject = (value) => value !== null && typeof value === 'object' && !Array.isArray(value) + +/** What `h5p.json` may hold that a source has no field for. */ +const UNWRITTEN = /** @type {const} */ ({ + authorComments: 'the author\'s comments', + changes: 'the change log', + licenseExtras: 'the licence\'s extra terms', + a11yTitle: 'the title for screen readers' +}) + +/** What a part's metadata may say of its own, where a build gives every part the source's. */ +const OWN_METADATA = ['license', 'licenseVersion', 'authors', 'source', 'yearFrom', 'yearTo', 'changes', 'authorComments', 'licenseExtras'] + +/** Whether a value says something: not empty text, an empty list or nothing. @param {unknown} value */ +const says = (value) => value !== undefined && value !== null && !(typeof value === 'string' && !value.trim()) && !(Array.isArray(value) && !value.length) + +/** + * The media a build of the read source is given: each file as the package stores it, found by the + * path the source names it by, its copyright as a build makes one. Nothing is read: the check + * compares content, and the files' bytes are the package's. + * @extends {MediaStore} + */ +class SavedMedia extends MediaStore { + /** @param {Map>} saved the files the content names, by path */ + constructor(saved) { + super({ + what: 'a file in the package', + name: (source) => source, + local: async () => ({ missing: 'is not read here' }), + keep: async () => { throw new Error('nothing is downloaded here') }, + open: async () => { throw new Error('nothing is opened here') } + }) + this.saved = saved + } + + /** + * @param {any} field + * @param {{ path: string, copyright?: any }} value as the source writes it + * @param {string} at + * @param {(path: string, message: string) => void} report + * @returns {Promise>} + */ + async resolve(field, value, at, report) { + const stored = { ...(this.saved.get(value.path) ?? { path: value.path }) } + delete stored.copyright + return { ...stored, copyright: this.copyright(value.copyright, `${at}.copyright`, report) } + } +} + +/** + * A package's content and metadata read back into a source. + * @param {import('./pack.mjs').Pack} pack the pack a build of the source would use + * @param {{ manifest: unknown, content: unknown }} saved the package's `h5p.json` and + * `content/content.json`, parsed + * @returns {Promise} + * @throws {Error} where the package names no main library, or its content is for other versions + * of its libraries than the pack's + */ +export async function readContent(pack, { manifest, content }) { + if (!isObject(manifest)) throw new Error('h5p.json is not an object') + if (!isObject(content)) throw new Error('content/content.json is not an object') + const main = typeof manifest.mainLibrary === 'string' ? manifest.mainLibrary : '' + const named = (Array.isArray(manifest.preloadedDependencies) ? manifest.preloadedDependencies : []) + .find((/** @type {unknown} */ each) => isObject(each) && each.machineName === main) + if (!main || !named || !/^\d+$/.test(String(named.majorVersion)) || !/^\d+$/.test(String(named.minorVersion))) { + throw new Error('h5p.json names no main library with its version: it is not an H5P package') + } + const type = `${main} ${Number(named.majorVersion)}.${Number(named.minorVersion)}` + + // The pack's versions only. A build makes content for those, and content for another minor is + // moved to the pack's by the library's own upgrade, which H5P runs as it plays or opens it. + /** @type {string[]} */ + const other = [] + for (const library of new Set([type, ...librariesIn(content)])) { + const [machineName, version] = library.split(' ') + const [major, minor] = version.split('.').map(Number) + const entry = pack.newest(machineName, major) + if (!entry) other.push(`${library}, of which the pack has no ${major}.x`) + else if (entry.minorVersion !== minor) other.push(`${library}, where the pack has ${major}.${entry.minorVersion}`) + } + if (other.length) { + throw new Error(`This package's content is made for other versions of its libraries than the library pack has: ${other.join('; ')}. A read takes content made for the pack's versions: H5P moves content to a newer version as it plays it, and saved again on a site that has the pack's versions, it reads.`) + } + + /** @type {Problem[]} */ + const notes = [] + const note = (/** @type {string} */ path, /** @type {string} */ message) => notes.push({ path, message }) + + // The content's language: H5P's strings follow `defaultLanguage`, and the metadata's is `language`. + const tags = [manifest.defaultLanguage, manifest.language].filter((each) => typeof each === 'string' && each.trim()) + const language = tags.find((each) => sourceSchema.shape.language.safeParse(each).success) ?? 'en' + if (!tags.includes(language)) note('language', tags.length ? `${JSON.stringify(tags[0])} is no language tag a source takes: written as "en"` : 'is not in h5p.json: written as "en"') + else if (tags.length === 2 && tags[0] !== tags[1]) note('language', `is the content's, ${tags[0]}; h5p.json gives the metadata ${tags[1]}, and a source has one language for both`) + + const libraries = await loadLibraries(pack, { type, language, params: content }) + const checked = checkContent(libraries, { type, params: content }) + // What the content holds beside its fields is not read. Any other problem of the content as + // saved, a build of the source asks about, below. + for (const { path, message } of checked.problems) { + if (/^is not (?:a field here|part of)/.test(message)) note(path, 'is no field of its content type: left out') + } + + const generator = new ContentGenerator(libraries.entries) + const params = generator.written(libraries.main, checked.params) + const layout = layoutContext(libraries.entries, () => {}, () => {}) + for (const place of placesOf(main, params)) LAYOUTS[place.library]?.read?.(place.params, place.path, layout(place.library)) + + const source = /** @type {Source} */ ({ type: main, ...ownFields(manifest, language, note), params }) + partsMetadata(checked.params, source, note) + + // Built again in memory, the package's files standing for themselves: a value that comes back + // otherwise is written as stored, where it is a value a source writes as stored. + const saved = new Map(generator.stored.map((file) => [String(file.path), file])) + const read = sourceOf(source, typeNames(pack)) + for (const { path, message } of read.problems) note(path, message) + for (let round = 0; round < 2; round++) { + let built + try { + built = await composePackage(read.source, pack, new SavedMedia(saved), []) + } catch (error) { + if (!(error instanceof SourceError)) throw error + for (const { path, message } of error.problems) note(path, `${message}: a build of this source asks for it`) + break + } + // A list H5P's editor started for nobody is left out of the source: it is what a build makes. + for (const path of generator.unfilled) setAt(checked.params, path, valueAt(built.content, path)) + const differ = differences(built.content, checked.params, new Set(generator.stored)) + if (!differ.length) break + const left = [] + for (const path of differ) { + const value = valueAt(checked.params, path) + // A text, a number, a flag or a title: written as stored. Anything else is said. + if (round === 0 && value !== undefined && (value === null || typeof value !== 'object')) setAt(params, path, value) + else left.push(path) + } + if (left.length || round === 1) { + for (const path of round === 1 ? differ : left) note(path, 'reads back otherwise: a build of this source writes it differently from the package') + break + } + } + + const files = [...new Set(generator.stored.map((file) => String(file.path)))].filter((path) => !/^[a-z][a-z0-9+.-]*:/i.test(path)) + return { source, files, notes } +} + +/** + * A source's own fields from `h5p.json`, as a build takes them, and a note of what a source has no + * field for, or what is not in a form it takes. + * @param {Record} manifest + * @param {string} language + * @param {(path: string, message: string) => void} note + */ +function ownFields(manifest, language, note) { + const title = typeof manifest.title === 'string' && manifest.title.trim() ? manifest.title : 'Untitled' + if (title === 'Untitled' && manifest.title !== 'Untitled') note('title', 'is not in h5p.json: written as "Untitled"') + const license = LICENSES.includes(manifest.license) ? manifest.license : 'U' + if (says(manifest.license) && license !== manifest.license) note('license', `${JSON.stringify(manifest.license)} is no licence H5P names: written as "U"`) + const versioned = license.startsWith('CC BY') && CC_VERSIONS.includes(manifest.licenseVersion) + /** @type {{ name: string, role: string }[]} */ + const authors = (Array.isArray(manifest.authors) ? manifest.authors : []) + .filter((/** @type {unknown} */ author) => isObject(author) && typeof author.name === 'string' && author.name.trim()) + .map((/** @type {Record} */ author) => ({ name: author.name, role: ROLES.includes(author.role) ? author.role : 'Author' })) + if (!authors.length) { + authors.push({ name: 'Unknown', role: 'Author' }) + note('authors', 'are not in h5p.json: written as "Unknown", since a source names one') + } + /** @type {Record} */ + const own = { title, language, license, ...(versioned ? { licenseVersion: manifest.licenseVersion } : {}), authors } + if (typeof manifest.source === 'string' && sourceSchema.shape.source.safeParse(manifest.source).success) own.source = manifest.source + else if (says(manifest.source)) note('source', `${JSON.stringify(manifest.source)} is no web address: left out`) + for (const key of /** @type {const} */ (['yearFrom', 'yearTo'])) { + const year = typeof manifest[key] === 'string' && /^\s*\d+\s*$/.test(manifest[key]) ? Number(manifest[key]) : manifest[key] + if (Number.isInteger(year)) own[key] = year + } + for (const [key, what] of Object.entries(UNWRITTEN)) { + if (says(manifest[key])) note(key, `is ${what}, which a source has no field for: left out`) + } + return own +} + +/** + * A note of the parts whose metadata says something of its own, a licence or authors, which a + * build replaces with the source's. One that says nothing, the licence "U" and no authors as H5P's + * editor saves a part, is not named. + * @param {Record} params as checked + * @param {Source} source + * @param {(path: string, message: string) => void} note + */ +function partsMetadata(params, source, note) { + /** @type {string[]} */ + const parts = [] + /** @param {unknown} value @param {string} path */ + const walk = (value, path) => { + if (Array.isArray(value)) value.forEach((each, i) => walk(each, `${path}[${i}]`)) + else if (isObject(value)) { + if (typeof value.library === 'string' && isObject(value.metadata) && ownMetadata(value.metadata, source)) parts.push(`${path}.metadata`) + for (const [key, each] of Object.entries(value)) walk(each, `${path}.${key}`) + } + } + walk(params, 'params') + if (!parts.length) return + const shown = parts.slice(0, 3).join(', ') + note(parts[0], `${parts.length === 1 ? 'is a part\'s metadata' : `and ${parts.length - 1} more are parts' metadata`} with a licence, authors or more of their own, which a source has no field for: a build gives every part the source's${parts.length > 1 ? ` (${shown}${parts.length > 3 ? ', …' : ''})` : ''}`) +} + +/** + * Whether a part's metadata says something of its own that the source does not. + * @param {Record} metadata + * @param {Source} source + */ +function ownMetadata(metadata, source) { + return OWN_METADATA.some((key) => { + const value = metadata[key] + if (!says(value)) return false + if (key === 'license') return value !== 'U' && value !== source.license + if (key === 'licenseVersion') return value !== source.licenseVersion + if (key === 'authors') return JSON.stringify(value.map((/** @type {any} */ author) => [author?.name, author?.role])) !== JSON.stringify(source.authors.map((author) => [author.name, author.role])) + return true + }) +} + +/** + * The paths at which content built from the read source is not the package's: equal but for each + * part's id and the metadata a build gives it, its title apart, and a file's bytes, which are the + * package's own. + * @param {unknown} built + * @param {unknown} saved + * @param {Set} files the saved content's files, as the generator collected them + * @param {string} [path] + * @param {string[]} [found] + * @returns {string[]} + */ +function differences(built, saved, files, path = 'params', found = []) { + if (Array.isArray(built) && Array.isArray(saved) && built.length === saved.length) { + built.forEach((each, i) => differences(each, saved[i], files, `${path}[${i}]`, found)) + } else if (isObject(built) && isObject(saved)) { + if (files.has(saved)) { + if (built.path !== saved.path || JSON.stringify(writtenCopyright(built.copyright) ?? null) !== JSON.stringify(writtenCopyright(saved.copyright) ?? null)) found.push(path) + return found + } + const part = typeof saved.library === 'string' + for (const key of new Set([...Object.keys(built), ...Object.keys(saved)])) { + if (key === 'subContentId') continue + if (part && key === 'metadata') { + // A part saved without a title, as older content is, gets its content type's from a build. + if (isObject(saved.metadata) && saved.metadata.title !== undefined && built.metadata?.title !== saved.metadata.title) found.push(`${path}.metadata.title`) + continue + } + differences(built[key], saved[key], files, `${path}.${key}`, found) + } + } else if (!Object.is(built, saved) && !sameText(built, saved)) found.push(path) + return found +} + +/** + * Whether a built text is the saved one as an author would write it: an empty text where nothing + * is built, as H5P's editor saves a field nobody typed in; and text without markup in a markup + * field, which a build wraps as the editor wraps typing, where a package made elsewhere kept it + * bare. A learner sees the same. + * @param {unknown} built + * @param {unknown} saved + */ +function sameText(built, saved) { + if (built === undefined) return saved === '' + if (typeof built !== 'string' || typeof saved !== 'string' || hasMarkup(saved)) return false + return built === wrapText(saved, 'p') || built === wrapText(saved, 'div') +} + +/** + * Sets a value at a content path in a source's parameters, which are in the same shape, making + * the objects on the way that a source left out. + * @param {Record} params + * @param {string} path `params.a[0].b` + * @param {unknown} value + */ +function setAt(params, path, value) { + const keys = path.replace(/\[(\d+)\]/g, '.$1').split('.').slice(1) + /** @type {any} */ + let at = params + keys.forEach((key, i) => { + if (i === keys.length - 1) at[key] = value + else { + if (at[key] === undefined || at[key] === null) at[key] = /^\d+$/.test(keys[i + 1]) ? [] : {} + at = at[key] + } + }) +} diff --git a/packages/author/lib/text.mjs b/packages/author/lib/text.mjs index 0dcea35..e4969a4 100644 --- a/packages/author/lib/text.mjs +++ b/packages/author/lib/text.mjs @@ -181,6 +181,32 @@ export function hasMarkup(value) { return /<[a-zA-Z!/]/.test(value) } +/** + * Plain text as a markup field's editor stores what an author types: escaped, in a block per + * paragraph, a blank line between two, and a `
` for each other line break. + * @param {string} value + * @param {string} block `p` or `div`, as `blockTag` says for the field + */ +export function wrapText(value, block) { + const paragraphs = value.trim().split(/\n[ \t]*\n+/).filter((paragraph) => paragraph.trim()) + return paragraphs.map((paragraph) => `<${block}>${escapeText(paragraph.trim()).replace(/\r?\n/g, '
')}`).join('') +} + +/** + * Stored markup as the plain text an author would type for it, where `wrapText` makes exactly that + * markup of it again: blocks and line breaks alone, entities kept, which a build keeps. Otherwise + * `undefined`: the markup is what a source writes. + * @param {string} value + * @param {string} block the field's, as `blockTag` says + * @returns {string | undefined} + */ +export function unwrapText(value, block) { + const blocks = value.match(new RegExp(`^(?:<${block}>[^<]*(?:
[^<]*)*)+$`)) && value.split(``).slice(0, -1) + if (!blocks) return undefined + const plain = blocks.map((each) => each.slice(block.length + 2).replaceAll('
', '\n')).join('\n\n') + return wrapText(plain, block) === value ? plain : undefined +} + /** * A markup field's value as it is stored. Plain text is escaped and wrapped in the field's block, * a paragraph per blank line and a `
` per other line break. Markup is checked: a tag the field @@ -194,12 +220,7 @@ export function hasMarkup(value) { */ export function markupText(value, field, report, { saved = false } = {}) { const tags = allowedTags(field) ?? new Set() - if (!hasMarkup(value) && !saved) { - const block = blockTag(field) - const paragraphs = value.trim().split(/\n[ \t]*\n+/).filter((paragraph) => paragraph.trim()) - if (!paragraphs.length) return '' - return paragraphs.map((paragraph) => `<${block}>${escapeText(paragraph.trim()).replace(/\r?\n/g, '
')}`).join('') - } + if (!hasMarkup(value) && !saved) return wrapText(value, blockTag(field)) const styles = stylePatterns(field) // Each thing said once, however often the value repeats it: an opening tag and its closing one. const said = new Set() diff --git a/packages/author/lib/types.mjs b/packages/author/lib/types.mjs index 68d3720..43b5158 100644 --- a/packages/author/lib/types.mjs +++ b/packages/author/lib/types.mjs @@ -10,6 +10,8 @@ import { stripTags } from './text.mjs' * - `before` adjusts the written parameters before generation; * - `after` sets the hidden fields on the generated ones, once the media are resolved, and checks * what the editor's widget would not have let an author save; + * - `read` turns what the two wrote back into what a source writes, for a package read back + * into a source (#236), run on that source's parameters; * - `sets` names the hidden fields the layout sets, which the schema leaves out; * - `optional` names fields the semantics call required that the content plays without, and the * editor never asks for. @@ -38,6 +40,7 @@ import { stripTags } from './text.mjs' * @typedef {{ * before?: (written: any, path: string, context: LayoutContext) => void, * after?: (params: any, path: string, context: LayoutContext) => void, + * read?: (written: any, path: string, context: LayoutContext) => void, * sets?: string[], * optional?: string[] * }} Layout @@ -90,6 +93,14 @@ const questionSet = { ;(params.questions ?? []).forEach((/** @type {any} */ question, /** @type {number} */ i) => { if (isObject(question?.metadata) && question.metadata.title === undefined) question.metadata.title = questionTitle(context.semantics, i) }) + }, + // Such a title is the build's, and a source read back leaves it out. + read(written, path, context) { + ;(written?.questions ?? []).forEach((/** @type {any} */ question, /** @type {number} */ i) => { + if (!isObject(question?.metadata) || question.metadata.title !== questionTitle(context.semantics, i)) return + delete question.metadata.title + if (!Object.keys(question.metadata).length) delete question.metadata + }) } } @@ -653,6 +664,14 @@ const gameMap = { if (!element.neighbors.length && elements.length > 1) context.note(`${path}.gamemapSteps.gamemap.elements[${i}]`, 'is joined to no other stage') }) if (gamemap.paths === undefined) gamemap.paths = [] + }, + // A neighbour is stored as its index in text, which `before` would look for among the labels: a + // source read back writes the number. + read(written) { + for (const element of written?.gamemapSteps?.gamemap?.elements ?? []) { + if (!Array.isArray(element?.neighbors)) continue + element.neighbors = element.neighbors.map((/** @type {unknown} */ neighbor) => (typeof neighbor === 'string' && /^\d+$/.test(neighbor) ? Number(neighbor) : neighbor)) + } } } diff --git a/packages/author/tests/read.test.ts b/packages/author/tests/read.test.ts new file mode 100644 index 0000000..6dca9dc --- /dev/null +++ b/packages/author/tests/read.test.ts @@ -0,0 +1,231 @@ +import { execFile } from 'node:child_process' +import { lstat, mkdir, mkdtemp, readdir, readFile, realpath, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join, resolve } from 'node:path' +import { promisify } from 'node:util' +import { BlobReader, BlobWriter, TextReader, TextWriter, Uint8ArrayReader, ZipReader, ZipWriter, type FileEntry } from '@zip.js/zip.js' +import { afterAll, beforeAll, describe, expect, it } from 'vitest' +import { buildPackage, readPackage } from '../lib/author.mjs' +import { mp4, png } from './helpers/media' + +// The skill's type files, whose examples a read is held to (#236). +const types = resolve(import.meta.dirname, '../../../skills/h5p-author/references/types') +const bin = resolve(import.meta.dirname, '../bin/h5p-author.mjs') +const exec = promisify(execFile) +const META = { title: 'Example', language: 'en', license: 'CC BY', authors: [{ name: 'Ann Author', role: 'Author' }] } +const files = (await readdir(types)).filter((name) => name.endsWith('.md')).sort() +let dir: string +let count = 0 + +beforeAll(async () => { dir = await mkdtemp(join(tmpdir(), 'h5p-author-read-')) }) +afterAll(async () => { await rm(dir, { recursive: true, force: true }) }) + +/** A new folder of the test's own. */ +async function folder() { + const path = join(dir, String(++count)) + await mkdir(path) + return path +} + +/** The source a type file gives as its example, the metadata every source carries added. */ +async function exampleOf(file: string) { + const block = /## Example\s+```json\n([\s\S]*?)\n```/.exec(await readFile(join(types, file), 'utf8')) + if (!block) throw new Error(`${file} has no example`) + return { ...META, ...JSON.parse(block[1]) } +} + +/** Every file a source names, by its `path`. */ +function paths(value: unknown, found: string[] = []): string[] { + if (Array.isArray(value)) value.forEach((item) => paths(item, found)) + else if (value && typeof value === 'object') { + const object = value as Record + if (typeof object.path === 'string') found.push(object.path) + Object.values(object).forEach((item) => paths(item, found)) + } + return found +} + +/** A second of silence as MPEG-1 Layer III, 128 kbit/s at 44.1 kHz. */ +function mp3() { + const frame = Buffer.alloc(417) + frame.set([0xff, 0xfb, 0x90, 0x64]) + return Buffer.concat(Array.from({ length: 38 }, () => frame)) +} + +/** A file for each name an example gives, by its type. */ +function sample(name: string) { + const extension = name.split('.').pop() + if (extension === 'png') return png(480, 270) + if (extension === 'mp4') return mp4(true, { seconds: 60, tracks: [['vide', 'avc1'], ['soun', 'mp4a']] }) + if (extension === 'mp3') return mp3() + if (extension === 'vtt') return Buffer.from('WEBVTT\n\n00:00.000 --> 00:02.000\nA river.\n') + throw new Error(`No sample for ${name}`) +} + +/** Content but for each part's id, which a build gives anew. */ +function withoutIds(value: unknown): unknown { + if (Array.isArray(value)) return value.map(withoutIds) + if (value && typeof value === 'object') return Object.fromEntries(Object.entries(value).filter(([key]) => key !== 'subContentId').map(([key, each]) => [key, withoutIds(each)])) + return value +} + +/** A package's `h5p.json` and content, as a build wrote them. */ +async function opened(path: string) { + const reader = new ZipReader(new BlobReader(new Blob([await readFile(path)]))) + try { + const entries = new Map((await reader.getEntries()).map((entry) => [entry.filename, entry as FileEntry])) + const json = async (name: string) => JSON.parse(await entries.get(name)!.getData(new TextWriter())) + return { manifest: await json('h5p.json'), content: await json('content/content.json') } + } finally { + await reader.close() + } +} + +/** + * A package as another tool exports it: its `h5p.json`, its content and its media, and no library, + * as h5p.com exports one without them. + */ +async function exported(path: string, manifest: unknown, content: unknown, media: Record = {}) { + const writer = new ZipWriter(new BlobWriter()) + await writer.add('h5p.json', new TextReader(JSON.stringify(manifest))) + await writer.add('content/content.json', new TextReader(JSON.stringify(content))) + for (const [name, bytes] of Object.entries(media)) await writer.add(`content/${name}`, new Uint8ArrayReader(bytes)) + await writeFile(path, new Uint8Array(await (await writer.close()).arrayBuffer())) +} + +const exists = (path: string) => lstat(path).then(() => true, () => false) + +describe.concurrent('every type file', () => { + it.each(files)('%s: its example, built, read and built again, is the same content but for the ids', async (file) => { + const source = await exampleOf(file) + const at = await folder() + for (const path of paths(source.params)) if (!/^https?:/.test(path)) await writeFile(join(at, path), sample(path)) + const first = await buildPackage(source, join(at, 'example.h5p'), { base: at }) + const read = await readPackage(join(at, 'example.h5p'), join(at, 'read')) + expect(read.notes).toEqual([]) + expect(read.output).toBe(join(at, 'read', 'example.source.json')) + expect(JSON.parse(await readFile(read.output, 'utf8'))).toEqual(read.source) + const { type, title, language, license, authors } = read.source + expect({ type, title, language, license, authors }).toEqual({ type: first.manifest.mainLibrary, ...META }) + for (const path of read.files) expect(await exists(join(at, 'read', path)), path).toBe(true) + const again = await buildPackage(read.source, undefined, { base: join(at, 'read'), check: true }) + expect(withoutIds(again.content)).toEqual(withoutIds(first.content)) + }, 60_000) +}) + +describe('a package read back', () => { + /** + * A Question Set as H5P's editor saves one and another tool exports it, without its libraries: + * no author, a language for the metadata apart from the content's, the comments and change log, + * a part with a licence and authors of its own, a field its library does not have, an empty tip + * nobody typed in, the copyright of a picture saved with every part empty, and a question + * without a title, which the set names itself. + */ + async function elsewhere(at: string) { + const built = join(at, 'built.h5p') + await writeFile(join(at, 'river.png'), png(320, 180)) + await buildPackage({ + ...META, + type: 'H5P.QuestionSet', + params: { + questions: [ + { library: 'H5P.MultiChoice', params: { question: 'Which river is the longest in Europe?', media: { type: { library: 'H5P.Image', params: { file: { path: 'river.png' }, alt: 'A river' } } }, answers: [{ text: 'Volga', correct: true }, { text: 'Danube' }] } }, + { library: 'H5P.TrueFalse', params: { question: 'The Rhine flows into the North Sea.' } } + ] + } + }, built, { base: at }) + const { manifest, content } = await opened(built) + const { authors, ...rest } = manifest + const saved = { ...rest, language: 'und', defaultLanguage: 'en', authorComments: 'Check the dates with Ann.', changes: [{ date: '01-11-26 10:00:00', author: 'Ann', log: 'First draft' }] } + const [choice, statement] = content.questions + choice.metadata = { ...choice.metadata, license: 'CC BY-SA', licenseVersion: '4.0', authors: [{ name: 'Bo Brown', role: 'Author' }], title: 'Untitled Multiple Choice' } + choice.params.oldField = true + choice.params.answers[1].tipsAndFeedback = { ...choice.params.answers[1].tipsAndFeedback, tip: '' } + choice.params.media.type.params.file.copyright = { license: 'U', author: '', title: '', year: '', source: '', version: '' } + delete statement.metadata.title + const image = choice.params.media.type.params.file.path + const bytes = new Uint8Array(png(320, 180)) + const output = join(at, 'elsewhere.h5p') + await exported(output, saved, content, { [image]: bytes }) + return { output, image } + } + + it('from another tool: its own fields and what a build fills in left out, and what a source cannot hold said', async () => { + const at = await folder() + const { output, image } = await elsewhere(at) + const read = await readPackage(output, join(at, 'read')) + expect(read.source).toMatchObject({ type: 'H5P.QuestionSet', title: 'Example', language: 'en', license: 'CC BY', authors: [{ name: 'Unknown', role: 'Author' }] }) + expect(read.source.params).toEqual({ + questions: [ + { + library: 'H5P.MultiChoice', + params: { question: 'Which river is the longest in Europe?', media: { type: { library: 'H5P.Image', params: { file: { path: image }, alt: 'A river' } } }, answers: [{ text: 'Volga', correct: true }, { text: 'Danube' }] }, + metadata: { title: 'Untitled Multiple Choice' } + }, + { library: 'H5P.TrueFalse', params: { question: 'The Rhine flows into the North Sea.' } } + ] + }) + expect(read.notes.map(({ path }) => path)).toEqual(['language', 'params.questions[0].params.oldField', 'authors', 'authorComments', 'changes', 'params.questions[0].metadata']) + expect(read.notes.find(({ path }) => path === 'params.questions[0].metadata')?.message).toMatch(/licence, authors or more of their own.*a build gives every part the source's/) + expect(read.files).toEqual([image]) + expect(await exists(join(at, 'read', image))).toBe(true) + const again = await buildPackage(read.source, undefined, { base: join(at, 'read'), check: true }) + expect(again.notes).toEqual([]) + }) + + it('refuses content made for another version of a library than the pack has, naming both', async () => { + const at = await folder() + const { output } = await elsewhere(at) + const { manifest, content } = await opened(output) + const main = manifest.preloadedDependencies.find((each: { machineName: string }) => each.machineName === 'H5P.QuestionSet') + const minor = Number(main.minorVersion) + main.minorVersion = String(minor - 1) + const older = join(at, 'older.h5p') + await exported(older, manifest, content) + await expect(readPackage(older, join(at, 'read'))).rejects.toThrow(`H5P.QuestionSet ${main.majorVersion}.${minor - 1}, where the pack has ${main.majorVersion}.${minor}`) + expect(await exists(join(at, 'read'))).toBe(false) + }) + + it('never writes over a file, and writes nothing where a name is taken', async () => { + const at = await folder() + const { output, image } = await elsewhere(at) + await mkdir(join(at, 'read')) + await writeFile(join(at, 'read', 'elsewhere.source.json'), 'mine') + await expect(readPackage(output, join(at, 'read'))).rejects.toThrow(/exists, and a read never writes over a file/) + expect(await readFile(join(at, 'read', 'elsewhere.source.json'), 'utf8')).toBe('mine') + expect(await exists(join(at, 'read', image))).toBe(false) + }) + + it('writes only inside its folder, and only where it may', async () => { + const at = await folder() + const { output } = await elsewhere(at) + const { manifest, content } = await opened(output) + content.questions[0].params.media.type.params.file.path = '../../outside.png' + const hostile = join(at, 'hostile.h5p') + await exported(hostile, manifest, content) + const read = await readPackage(hostile, join(at, 'read')) + expect(read.notes).toContainEqual({ path: 'params', message: 'names the file "../../outside.png", which is no path a read writes a file at: not written' }) + expect(read.files).toEqual([]) + expect(await exists(join(at, 'outside.png'))).toBe(false) + // An archive whose entry leads out of its folders is not read at all. + const leading = join(at, 'leading.h5p') + await exported(leading, manifest, content, { '../../outside.png': new Uint8Array(png(8, 8)) }) + await expect(readPackage(leading, join(at, 'read-1'))).rejects.toThrow('has an entry named "content/../../outside.png", which leads out of its folders: an archive like that is not read') + expect(await exists(join(at, 'outside.png'))).toBe(false) + const elsewhereFolder = await folder() + await expect(readPackage(output, join(at, 'read-2'), { within: [elsewhereFolder] })).rejects.toThrow(/is outside the folders this read may use/) + await expect(readPackage(output, join(elsewhereFolder, '..', 'read-3'), { within: [at] })).rejects.toThrow(/is outside the folders this read may write in/) + }) + + it('is a command: h5p-author read writes the source and its media, and says how to build it again', async () => { + const at = await folder() + const { image } = await elsewhere(at) + // As the command sees its folder, a temporary folder's link followed. + const real = await realpath(at) + const { stdout, stderr } = await exec(process.execPath, [bin, 'read', 'elsewhere.h5p', 'read'], { cwd: at }) + expect(stdout).toContain(`Read ${join(real, 'elsewhere.h5p')}: H5P.QuestionSet. Wrote its source to ${join(real, 'read', 'elsewhere.source.json')}, and 1 media file beside it.`) + expect(stdout).toContain(`h5p-author build ${join('read', 'elsewhere.source.json')} `) + expect(stderr).toContain('Note: authorComments is the author\'s comments, which a source has no field for: left out') + expect(await exists(join(at, 'read', image))).toBe(true) + }) +}) From ad811c5ce7c92749cebc4f58d02f0a8884ce6f29 Mon Sep 17 00:00:00 2001 From: Aleksandr Shevchuk Date: Wed, 18 Nov 2026 15:54:19 +0200 Subject: [PATCH 2/5] feat(mcp): h5p_read, a package back into its source to change and build again (#236) Reads a package in the server's folders into a new folder beside it, and answers with the source, each file named where it was written, for h5p_build. The instructions gain one line for changing a package the person has. --- AGENTS.md | 4 +- README.md | 4 +- design/layout.md | 5 ++- design/mcp.md | 39 ++++++++++------- packages/mcp/README.md | 6 ++- packages/mcp/lib/server.mjs | 69 +++++++++++++++++++++++++++++-- packages/mcp/tests/server.test.ts | 45 ++++++++++++++++++-- 7 files changed, 143 insertions(+), 29 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index dca5f5f..5954d29 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -50,8 +50,8 @@ folders; in `design/`, `src/…` and `scripts/…` mean the player's unless a fi | `packages/libraries/` | the library pack, one archive per library | [libraries](design/libraries.md) | | `packages/normalize/` | rewrites a package so it streams | [normalize](design/normalize.md) | | `packages/verify/` | plays a package headless and reports | [verify](design/verify.md) | -| `packages/author/`, `skills/h5p-author/` | a playable package of any content type from a short source | [author](design/author.md) | -| `packages/mcp/` | the local MCP server: builds, checks and previews in an app with no shell | [mcp](design/mcp.md) | +| `packages/author/`, `skills/h5p-author/` | a playable package of any content type from a short source, and a package back to one | [author](design/author.md) | +| `packages/mcp/` | the local MCP server: builds, reads, checks and previews in an app with no shell | [mcp](design/mcp.md) | | `packages/cmi5/` | the element as a cmi5 assignable unit | [cmi5](design/cmi5.md) | | `packages/embed/`, `skills/h5p-to-scorm/` | the embed site for a player domain, and a package as a SCORM zip | [embed](design/embed.md), [scorm](design/scorm.md) | | `packages/vscode/` | H5P Preview, the VS Code extension | [vscode](design/vscode.md) | diff --git a/README.md b/README.md index b7de100..1f3777d 100644 --- a/README.md +++ b/README.md @@ -102,8 +102,8 @@ npx @missing-elements/h5p-verify@latest course.h5p | [`@missing-elements/h5p-embed`](https://github.com/missing-elements/h5p-toolkit/tree/main/packages/embed#readme) | The player page as a static site, and a package as a SCORM zip | | [`@missing-elements/h5p-verify`](https://github.com/missing-elements/h5p-toolkit/tree/main/packages/verify#readme) | Plays a package headless and reports whether it works, or opens it in your browser | | [`@missing-elements/h5p-normalize`](https://github.com/missing-elements/h5p-toolkit/tree/main/packages/normalize#readme) | Rewrites a package so its video streams | -| [`@missing-elements/h5p-author`](https://github.com/missing-elements/h5p-toolkit/tree/main/packages/author#readme) | Builds a package of any content type from a short JSON source | -| [`@missing-elements/h5p-mcp`](https://github.com/missing-elements/h5p-toolkit/tree/main/packages/mcp#readme) | A local MCP server: an app such as Claude Desktop builds a package onto your disk, checks it and opens it in your browser | +| [`@missing-elements/h5p-author`](https://github.com/missing-elements/h5p-toolkit/tree/main/packages/author#readme) | Builds a package of any content type from a short JSON source, and reads one back into its source | +| [`@missing-elements/h5p-mcp`](https://github.com/missing-elements/h5p-toolkit/tree/main/packages/mcp#readme) | A local MCP server: an app such as Claude Desktop builds a package onto your disk, checks it and opens it in your browser, or reads one to change it | | [`@missing-elements/h5p-cmi5`](https://github.com/missing-elements/h5p-toolkit/tree/main/packages/cmi5#readme) | Makes a page holding the element a cmi5 assignable unit | | [H5P Preview](https://marketplace.visualstudio.com/items?itemName=MissingElements.h5p-preview) | Plays `.h5p` files in VS Code, and from [Open VSX](https://open-vsx.org/extension/MissingElements/h5p-preview) in Cursor and VSCodium | diff --git a/design/layout.md b/design/layout.md index 537f34c..b455791 100644 --- a/design/layout.md +++ b/design/layout.md @@ -29,8 +29,9 @@ packages/verify/ a package played headless (lib/verify.mjs) and in a brows (lib/preview.mjs); lib/serve.mjs what both share, which the extension bundles packages/author/ a playable .h5p from a short source: lib/build.mjs both builds, node.mjs Node's, generator.mjs content from semantics, check.mjs, outline.mjs, - types.mjs the editor widgets' rules; scripts/ the skill's sync and copy -packages/mcp/ the local MCP server: bin/h5p-mcp.mjs the command, lib/server.mjs its three + types.mjs the editor widgets' rules, read.mjs a package back to a source; + scripts/ the skill's sync and copy +packages/mcp/ the local MCP server: bin/h5p-mcp.mjs the command, lib/server.mjs its four tools, lib/folders.mjs the folders it may use packages/cmi5/ the element as a cmi5 AU: src/index.ts startCmi5, src/client.ts the protocol packages/embed/ site/ the embed page and the SCORM page; lib/build.mjs the site, diff --git a/design/mcp.md b/design/mcp.md index 98c4d0c..d442550 100644 --- a/design/mcp.md +++ b/design/mcp.md @@ -2,22 +2,24 @@ `packages/mcp` is `@missing-elements/h5p-mcp`, shown as **H5P Author MCP by Missing Elements**: a local MCP server over standard input and output that builds a package onto the person's disk, -checks it, and opens it in their browser. It is for apps with no shell, Claude Desktop first; in an -app with a shell, the skill and `npx` build already. A remote server, for ChatGPT or Claude on the -web, is #210 and not this. +checks it, and opens it in their browser, or reads one back to change it. It is for apps with no +shell, Claude Desktop first; in an app with a shell, the skill and `npx` build already. A remote +server, for ChatGPT or Claude on the web, is #210 and not this. -- **It holds no rules of its own.** The build, its checks, the outline and the confinement to - folders (`buildPackage`, `within`, `publicFetch`, `inside`) are h5p-author's; the check, the - preview and opening the browser (`verifyPackage`, `summarize`, `previewPackage`, `openBrowser`) - are h5p-verify's. A rule a tool needs goes into those packages first, exported, so the author - package and the verifier are published before a server that uses something new of theirs. -- **Three tools, and the skill's flow as the instructions.** `h5p_types` lists the pack's content +- **It holds no rules of its own.** The build, the read, their checks, the outline and the + confinement to folders (`buildPackage`, `readPackage`, `within`, `publicFetch`, `inside`) are + h5p-author's; the check, the preview and opening the browser (`verifyPackage`, `summarize`, + `previewPackage`, `openBrowser`) are h5p-verify's. A rule a tool needs goes into those packages + first, exported, so the author package and the verifier are published before a server that uses + something new of theirs. +- **Four tools, and the skill's flow as the instructions.** `h5p_types` lists the pack's content types, those with a type file marked, and the skill's brief; given a type, read as a build reads one (`sourceType`, so "true false" works), it answers with the type's file, or with `schemaOf` for - a type with none. `h5p_build` builds a source and `h5p_preview` opens a package. The instructions - are the flow in a few lines: the type is the one question, its answer is read before the source is - written, then the build, where the file is and what the check found, and the preview offered. - Not `SKILL.md`, whose build routes do not apply here. An app sends the descriptions and the + a type with none. `h5p_build` builds a source, `h5p_preview` opens a package, and `h5p_read` + reads one back into its source. The instructions are the flow in a few lines: the type is the one + question, its answer is read before the source is written, then the build, where the file is and + what the check found, and the preview offered; a package the person has is read, its source + changed, and built. Not `SKILL.md`, whose build routes do not apply here. An app sends the descriptions and the instructions with every message, so they stay under about 1,500 tokens; `server.test.ts` holds them under 6,000 characters. - **`h5p_build` shows a source's own fields; the build checks them.** Its `source` carries the @@ -49,6 +51,14 @@ web, is #210 and not this. in a temporary folder, with its screenshot as an image. Where neither Chrome, Edge nor Playwright's Chromium is found, the answer says "Not verified" and why. A source with problems is refused, every problem by its path, and nothing is written. +- **A read writes beside the package, and names its files where it wrote them** (#236). + `h5p_read` takes a package in the folders, a name alone being the first folder's, and writes its + source and media into a new folder beside it, named after it and numbered where the name is + taken, as a package is. It answers with the source, each file it wrote named by its full path, + so `h5p_build` reads them wherever its package goes. Asked to add two harder questions to the + demo's quiz, Haiku read it and built at the first build in 6 of 6 runs. With the package + described as a path in the server's folders, it first guessed one from its working folder in 3 + of 3; described "by its file name alone", in none of 3. - **The SDK is v2, and the package ships its source.** `@modelcontextprotocol/server` brings only its core and zod, none of the HTTP frameworks v1 installed, so `bin/` and `lib/` are published as written, with no bundle. `McpServer` on `StdioServerTransport` serves the 2025 protocol, which @@ -62,5 +72,4 @@ web, is #210 and not this. one refused. Each test builds what it needs, so each runs alone. The test script first copies the skill's type files into the author package, as the author's `prepack` does. -Not built yet: reading a package back to a source (step 3 of #219), the MCP registry's -`server.json` and Claude Desktop's one-click bundle. +Not built yet: the MCP registry's `server.json` and Claude Desktop's one-click bundle. diff --git a/packages/mcp/README.md b/packages/mcp/README.md index 34c0675..30a07bc 100644 --- a/packages/mcp/README.md +++ b/packages/mcp/README.md @@ -3,7 +3,8 @@ `@missing-elements/h5p-mcp` lets an AI app make H5P content on your computer: quizzes, fill in the blanks, flash cards, presentations, interactive books, branching scenarios and the rest of the 53 content types the H5P hub serves. It writes a playable `.h5p` that carries its libraries, checks -that it plays in a real H5P player, and opens it in your browser. No H5P site, server or account. +that it plays in a real H5P player, and opens it in your browser. It reads a `.h5p` you have back +into its source too, to change it and build it again. No H5P site, server or account. It is a local [MCP](https://modelcontextprotocol.io) server: your app starts it on your computer, and it sends nothing to a server of ours. It needs Node 22 or newer, and for the check Chrome or @@ -46,6 +47,9 @@ gets a number, and it says which. problems is refused, each problem named. It checks the package and says what it holds. - `h5p_preview`: opens a package in your browser, in the real player, served from your computer until its tab is closed. +- `h5p_read`: a package back into its source, with its pictures and other files, in a new folder + beside it, to change and build again. It reads content made for the library pack's versions of + its libraries, and says what a source cannot hold, such as the author's comments. ## What leaves your computer diff --git a/packages/mcp/lib/server.mjs b/packages/mcp/lib/server.mjs index 7bdab70..3c141f2 100644 --- a/packages/mcp/lib/server.mjs +++ b/packages/mcp/lib/server.mjs @@ -4,7 +4,7 @@ import { basename, dirname, join, resolve } from 'node:path' import { fileURLToPath } from 'node:url' import { McpServer } from '@modelcontextprotocol/server' import { z } from 'zod' -import { SourceError, buildPackage, contentTypes, inside, openPack, publicFetch, schemaOf, sourceSchema, sourceType } from '@missing-elements/h5p-author' +import { SourceError, buildPackage, contentTypes, inside, openPack, publicFetch, readPackage, schemaOf, sourceSchema, sourceType } from '@missing-elements/h5p-author' import { VerifyError, openBrowser, previewPackage, summarize, verifyPackage } from '@missing-elements/h5p-verify' import { defaultFolders, fileStem, rootFolders } from './folders.mjs' @@ -23,7 +23,10 @@ const INSTRUCTIONS = `Builds H5P content as a .h5p file from a short JSON source 2. Call h5p_types with the type and write the source from its notes, example and fields: only what differs from the defaults. 3. Call h5p_build. Where it lists problems, fix each at its path and build again. 4. Tell the person where the file is and what the check found, and offer h5p_preview. -The author and the licence are what the person gave; otherwise the name you know them by, or "Unknown", and the licence "U". Say which you used.` +The author and the licence are what the person gave; otherwise the name you know them by, or "Unknown", and the licence "U". Say which you used. +To change a .h5p the person has, h5p_read it, change its source as asked, and h5p_build that.` + +const READ = `Reads a .h5p package back into its source, to change it and build it again with h5p_build: its type, title, language, licence, authors, and the params that differ from the defaults. Writes the source and the package's media into a new folder beside it, and answers with the source, each file named where it was written. Reads content made for the library pack's versions of its libraries.` const TYPES = `The H5P content types and how to write a source for one. With no type: the list, and how to write good content. With a type: its notes, a tested example and its fields. Read the type's answer before writing its source.` @@ -65,8 +68,23 @@ const OUTLINE_LINES = 80 const sourceBeside = (path) => path.replace(/\.h5p$/i, '.source.json') /** - * The server and its three tools. It holds no rules of its own: the build, its checks and the - * confinement to folders are h5p-author's, the check and the preview h5p-verify's. + * A read source's files named where they were written, so that a build reads them wherever its + * package goes: each file the read wrote, by its path from the folder, made that folder's. + * @param {unknown} value + * @param {string} folder + * @param {Set} files + * @returns {unknown} + */ +function located(value, folder, files) { + if (Array.isArray(value)) return value.map((each) => located(each, folder, files)) + if (!value || typeof value !== 'object') return value + const entries = Object.entries(value).map(([key, each]) => [key, key === 'path' && typeof each === 'string' && files.has(each) ? join(folder, ...each.split('/')) : located(each, folder, files)]) + return Object.fromEntries(entries) +} + +/** + * The server and its four tools. It holds no rules of its own: the build, the read, their checks + * and the confinement to folders are h5p-author's, the check and the preview h5p-verify's. * @param {ServerOptions} [options] */ export async function createServer(options = {}) { @@ -134,6 +152,22 @@ export async function createServer(options = {}) { } } + /** + * A new folder beside a package for what a read writes, named after it, with a number added where + * the name is taken, as a build's package takes one; held while the read runs. + * @param {string} parent + * @param {string} stem + */ + async function freeFolder(parent, stem) { + for (let n = 1; ; n++) { + const path = join(parent, n === 1 ? stem : `${stem}-${n}`) + const taken = await lstat(path).then(() => true, () => false) + if (taken || building.has(path.toLowerCase())) continue + building.add(path.toLowerCase()) + return path + } + } + server.registerTool('h5p_types', { title: 'H5P content types', description: TYPES, @@ -219,6 +253,33 @@ export async function createServer(options = {}) { return { content: [{ type: /** @type {const} */ ('text'), text: lines.join('\n') }, ...images] } }) + server.registerTool('h5p_read', { + title: 'Read an H5P package', + description: READ, + inputSchema: z.object({ package: z.string().describe('The .h5p by its file name alone, as the person names it, or its path in a folder this server may use') }), + annotations: { readOnlyHint: false, destructiveHint: false, openWorldHint: false } + }, async ({ package: file }) => { + const allowed = await folders() + // A name alone is the first folder's, as a source's file is. + const path = resolve(allowed[0] ?? '.', file) + if (!allowed.length || !(await inside(path, allowed))) return refusal(`${file} is not in a folder this server may use.${allowed.length ? ` ${whereFiles(allowed)}` : ''}`) + const folder = await freeFolder(dirname(path), fileStem(basename(path).replace(/\.h5p$/i, ''))) + let result + try { + result = await readPackage(path, folder, { within: allowed, libraries: options.libraries }) + } catch (error) { + return refusal(error instanceof Error ? error.message : String(error)) + } finally { + building.delete(folder.toLowerCase()) + } + const media = result.files.length ? `, with its ${result.files.length} media file${result.files.length === 1 ? '' : 's'} beside it` : '' + const lines = [`Read ${path}: ${result.source.type}. Its source is in ${result.output}${media}.`] + for (const { path: at, message } of result.notes) lines.push(`Note: ${at} ${message}`) + const source = { ...result.source, params: located(result.source.params, folder, new Set(result.files)) } + lines.push('', 'The source, each file named where it was written: change it as the person asks and pass it to h5p_build.', JSON.stringify(source)) + return answer(lines.join('\n')) + }) + server.registerTool('h5p_preview', { title: 'Preview an H5P package', description: PREVIEW, diff --git a/packages/mcp/tests/server.test.ts b/packages/mcp/tests/server.test.ts index aac5fd2..1aa35ae 100644 --- a/packages/mcp/tests/server.test.ts +++ b/packages/mcp/tests/server.test.ts @@ -50,10 +50,10 @@ afterAll(async () => { }) describe('the server', () => { - it('offers three tools and the flow, in few words', async () => { + it('offers four tools and the flow, in few words', async () => { const { tools } = await client.listTools() - expect(tools.map((tool) => tool.name).sort()).toEqual(['h5p_build', 'h5p_preview', 'h5p_types']) - expect(client.getInstructions()).toMatch(/h5p_types[\s\S]*h5p_build[\s\S]*h5p_preview/) + expect(tools.map((tool) => tool.name).sort()).toEqual(['h5p_build', 'h5p_preview', 'h5p_read', 'h5p_types']) + expect(client.getInstructions()).toMatch(/h5p_types[\s\S]*h5p_build[\s\S]*h5p_preview[\s\S]*h5p_read/) // A source's own fields as the author package declares them, before one is written (#230), // with nothing required or forbidden, so no app refuses a source the build would read (#174). const source = tools.find((tool) => tool.name === 'h5p_build')?.inputSchema.properties?.source as Record @@ -151,6 +151,45 @@ describe('h5p_build', () => { }) }) +describe('h5p_read', () => { + /** A picture of one pixel, which a build reads the size of. */ + const PIXEL = Buffer.from('iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mP8z8BQDwAEhQGAhKmMIQAAAABJRU5ErkJggg==', 'base64') + + it('reads a package back into its source in a new folder beside it, its files named where they were written, to build again', async () => { + await writeFile(join(folder, 'pixel.png'), PIXEL) + const picture = { library: 'H5P.Image', params: { file: { path: 'pixel.png' }, alt: 'A pixel' } } + const source = { ...SOURCE, type: 'H5P.Column', title: 'One pixel', params: { content: [{ content: picture }] } } + const built = builtOf(await call('h5p_build', { source, verify: false })) + expect(built).toBe(join(folder, 'one-pixel.h5p')) + const answer = await call('h5p_read', { package: 'one-pixel.h5p' }) + const text = textOf(answer) + expect(answer.isError).toBeFalsy() + const beside = join(folder, 'one-pixel') + expect(text).toContain(`Read ${built}: H5P.Column. Its source is in ${join(beside, 'one-pixel.source.json')}, with its 1 media file beside it.`) + const read = JSON.parse(text.split('\n').at(-1)!) + expect(read).toMatchObject({ type: 'H5P.Column', title: 'One pixel', params: { content: [{ content: { library: 'H5P.Image', params: { alt: 'A pixel' } } }] } }) + // Where it was written, read by a build wherever its package goes. + const path = read.params.content[0].content.params.file.path + expect(path.startsWith(`${beside}/`)).toBe(true) + await access(path) + const again = builtOf(await call('h5p_build', { source: { ...read, title: 'One pixel, again' }, verify: false })) + expect(again).toBe(join(folder, 'one-pixel-again.h5p')) + // Read again, it takes a folder of its own: nothing is written over. + expect(textOf(await call('h5p_read', { package: 'one-pixel.h5p' }))).toContain(`Its source is in ${join(folder, 'one-pixel-2', 'one-pixel.source.json')}`) + }) + + it('refuses a package outside its folders, and says why one is refused', async () => { + await writeFile(join(elsewhere, 'outside.h5p'), 'not a package') + const outside = await call('h5p_read', { package: join(elsewhere, 'outside.h5p') }) + expect(outside.isError).toBe(true) + expect(textOf(outside)).toContain('is not in a folder this server may use.') + await writeFile(join(folder, 'broken.h5p'), 'not a package') + const broken = await call('h5p_read', { package: 'broken.h5p' }) + expect(broken.isError).toBe(true) + expect(textOf(broken)).toContain('is not a zip, as a .h5p is') + }) +}) + describe('h5p_preview', () => { it('serves a package in its folders, and answers with the address', async () => { const file = await build('Preview') From 96c49c09bc35e99bd2bab4332d391f4941b4856d Mon Sep 17 00:00:00 2001 From: Aleksandr Shevchuk Date: Wed, 18 Nov 2026 20:11:46 +0200 Subject: [PATCH 3/5] fix(author): what cubic found in the read (#236) - Every path is checked against the folder before anything is made, with or without `within`, so a link in the folder that leads out of it stops the read; `inside` resolves a folder not there yet as it does a path. - A file named by a path a read cannot write is left out of the source, which so names no file outside its folder. - A read that fails takes away the folders it made, where they are empty. - A file's licence H5P does not list, or a copyright source that is no web address, is left out with a note; copyrights compare in one order. - Authors left out or given the role "Author", a licence version a source cannot take, and a year that is none are noted; a part's authors that are no list no longer stop the read. - wrapText splits paragraphs at a blank line in Windows line endings. - The round trip compares the media written with the package's bytes. - The README's two unclear sentences, and the refusal's, reworded. --- design/author.md | 11 ++-- packages/author/README.md | 25 +++++---- packages/author/lib/generator.mjs | 14 +++-- packages/author/lib/media.mjs | 20 +++++-- packages/author/lib/node.mjs | 83 ++++++++++++++++++++------- packages/author/lib/read.mjs | 37 +++++++++--- packages/author/lib/text.mjs | 2 +- packages/author/tests/read.test.ts | 90 ++++++++++++++++++++++++++++-- packages/author/tests/text.test.ts | 4 ++ 9 files changed, 229 insertions(+), 57 deletions(-) diff --git a/design/author.md b/design/author.md index d98000f..1cde0e9 100644 --- a/design/author.md +++ b/design/author.md @@ -203,10 +203,13 @@ text to the content check's rules, and `apps/skills` plays every example untrust - What a source has no field for is said and left out: the author's comments, the change log, the licence's extra terms, and a part's own licence or authors, a build giving every part the source's. - - An archive is untrusted: only the media the content names are written, under the folder; one - whose entry leads out of its folders is not read. Nothing is written over, and nothing at all - where a name is taken. Every type file's example, built, read and built again, is the same - content but for the ids, with no note (`read.test.ts`). + - An archive is untrusted: only the media the content names are written, under the folder, each + path checked before anything is made, so that a link in the folder leading out of it stops the + read; a file named by a path that could lead out is left out of the source, which so names no + file outside its folder; one whose entry leads out of its folders is not read. Nothing is + written over, and nothing at all where a name is taken; a read that fails takes away what it + made. Every type file's example, built, read and built again, is the same content and media + but for the ids, with no note (`read.test.ts`). - **A page builds too** (#108). The package has three entries: `.` for Node, `./core` for what needs neither Node nor a page, and `./browser`, the core and a page's `openPack` and `buildPackage`. They are subpaths and not the `browser` condition, which TypeScript ignores diff --git a/packages/author/README.md b/packages/author/README.md index 5795da5..edc2ef9 100644 --- a/packages/author/README.md +++ b/packages/author/README.md @@ -159,20 +159,21 @@ Read /work/rivers.h5p: H5P.Blanks. Wrote its source to /work/rivers/rivers.sourc Change it, then build it again: h5p-author build rivers/rivers.source.json ``` -The source a build would take to make the package again, to change it and build it again: the -content type and the metadata from its `h5p.json`, and of its content only what differs from the -defaults, interface texts in the content's language included. Its pictures, videos and other files -go beside the source, at the paths it names them by. Positions and sizes are kept, so a slide or a -video keeps its layout. Each part gets a new id when it is built again. +The source holds what a build takes to make the package again: the content type and the metadata +from its `h5p.json`, and of its content only what differs from the defaults, interface texts in the +content's language included. Its pictures, videos and other files go beside the source, at the +paths it names them by. Positions and sizes are kept, so a slide or a video keeps its layout. Each +part gets a new id when it is built again. It reads content made for the pack's versions of its libraries, and refuses any other, naming both -versions: H5P moves content to a newer version as it plays it, and saved again on a site that has -the pack's versions, it reads. What a source has no field for is said and left out: the author's comments, -the change log, a part's own licence or authors (a build gives every part the source's). The source -is built again in memory before it is written, and what would not come back as it is, or what a -build would refuse, such as styles pasted from Word, is said, to fix before building. It never -writes over a file. An archive is untrusted: only the files its content names are written, and -only under the folder. +versions. H5P moves content to a newer version as it plays it, so a package saved again on a site +that has the pack's versions reads. What a source has no field for, or no form of, is said and left +out: the author's comments, the change log, a part's own licence or authors (a build gives every +part the source's), a file's licence that H5P does not list. The source is built again in memory +before it is written, and what would not come back as it is, or what a build would refuse, such as +styles pasted from Word, is said, to fix before building. It never writes over a file. An archive +is untrusted: only the files its content names are written, only under the folder, and a file +named by a path that could lead out of it is left out of the source. ## A content type's bundle diff --git a/packages/author/lib/generator.mjs b/packages/author/lib/generator.mjs index 2815986..fe9e125 100644 --- a/packages/author/lib/generator.mjs +++ b/packages/author/lib/generator.mjs @@ -353,13 +353,14 @@ export class ContentGenerator { #unopened(field, value, stack, path) { if (field.type !== 'library' || !isObject(field.default) || typeof field.default.library !== 'string' || !isObject(value) || typeof value.library !== 'string') return false if (value.library.trim().split(/\s+/)[0] !== field.default.library.trim().split(/\s+/)[0]) return false - const counts = [this.stored.length, this.unfilled.length] + const counts = [this.stored.length, this.unfilled.length, this.notes.length] try { const written = this.#writtenPart(field, value, stack, path) return isObject(written) && vacant(written.params) } finally { this.stored.length = counts[0] this.unfilled.length = counts[1] + this.notes.length = counts[2] } } @@ -397,7 +398,7 @@ export class ContentGenerator { case 'file': case 'video': case 'audio': - return Array.isArray(value) ? value.map((each) => this.#writtenFile(each)) : this.#writtenFile(value) + return Array.isArray(value) ? value.map((each, index) => this.#writtenFile(each, `${path}[${index}]`)) : this.#writtenFile(value, path) case 'text': // Markup that is only the blocks a build wraps typing in is written as the typing. return typeof value === 'string' && takesMarkup(field) ? unwrapText(value, blockTag(field)) ?? value : value @@ -431,13 +432,16 @@ export class ContentGenerator { /** * A saved file as a source writes it: its path, and its copyright where it gives one - * (`writtenCopyright`). Kept as saved in `stored`. + * (`writtenCopyright`), less a part a build refuses, which is noted. Kept as saved in `stored`. * @param {unknown} value + * @param {string} path */ - #writtenFile(value) { + #writtenFile(value, path) { if (!isObject(value) || typeof value.path !== 'string') return structuredClone(value) this.stored.push(value) - const copyright = writtenCopyright(value.copyright) + const copyright = writtenCopyright(value.copyright, (key, kept) => this.note(`${path}.copyright.${key}`, key === 'license' + ? `${JSON.stringify(kept)} is no licence H5P lists for a file: left out, with any version, so a build gives the file "U"` + : `${JSON.stringify(kept)} is no web address: left out`)) return { path: value.path, ...(copyright ? { copyright } : {}) } } diff --git a/packages/author/lib/media.mjs b/packages/author/lib/media.mjs index 080a356..9362b73 100644 --- a/packages/author/lib/media.mjs +++ b/packages/author/lib/media.mjs @@ -90,17 +90,29 @@ const COPYRIGHT_LICENSES = ['U', 'CC BY', 'CC BY-SA', 'CC BY-ND', 'CC BY-NC', 'C /** The parts of a file's copyright a source writes, as H5P records them. */ const COPYRIGHT_FIELDS = ['license', 'author', 'title', 'year', 'source', 'version'] +/** A file's copyright source as H5P's editor takes it, and a build: a web address. */ +const COPYRIGHT_SOURCE = /^https?:\/\/.+/i + /** * A saved file's copyright as a source writes it (#236): its parts that hold text, and nothing * where that is the licence "U" alone, which a build gives a file with none. H5P's editor saves - * every part, empty where nobody wrote it, and a build refuses an empty one. + * every part, empty where nobody wrote it, and a build refuses an empty one. A part a build + * refuses, a licence H5P does not list for a file or a source that is no web address, is left out, + * and `left` is told; a licence goes with its version. * @param {unknown} copyright + * @param {(key: 'license' | 'source', value: string) => void} [left] * @returns {Record | undefined} * @internal */ -export function writtenCopyright(copyright) { +export function writtenCopyright(copyright, left = () => {}) { if (!copyright || typeof copyright !== 'object' || Array.isArray(copyright)) return undefined - const kept = Object.fromEntries(Object.entries(copyright).filter(([key, value]) => COPYRIGHT_FIELDS.includes(key) && typeof value === 'string' && value.trim() !== '')) + const parts = Object.entries(copyright).filter(([key, value]) => COPYRIGHT_FIELDS.includes(key) && typeof value === 'string' && value.trim() !== '') + const license = parts.find(([key]) => key === 'license')?.[1] + const listed = license === undefined || COPYRIGHT_LICENSES.includes(license) + if (!listed) left('license', license) + const source = parts.find(([key]) => key === 'source')?.[1] + if (source !== undefined && !COPYRIGHT_SOURCE.test(source)) left('source', source) + const kept = Object.fromEntries(parts.filter(([key, value]) => key === 'source' ? COPYRIGHT_SOURCE.test(value) : listed || (key !== 'license' && key !== 'version'))) return Object.entries(kept).some(([key, value]) => key !== 'license' || value !== 'U') ? kept : undefined } @@ -353,7 +365,7 @@ export class MediaStore { continue } if (key === 'license' && !COPYRIGHT_LICENSES.includes(entry)) report(`${at}.license`, `use one of ${COPYRIGHT_LICENSES.join(', ')}`) - if (key === 'source' && !/^https?:\/\/.+/i.test(entry)) report(`${at}.source`, 'expected an http(s) address') + if (key === 'source' && !COPYRIGHT_SOURCE.test(entry)) report(`${at}.source`, 'expected an http(s) address') out[key] = key === 'license' || key === 'version' ? entry : escapeText(entry) } return out diff --git a/packages/author/lib/node.mjs b/packages/author/lib/node.mjs index 22f3762..677093a 100644 --- a/packages/author/lib/node.mjs +++ b/packages/author/lib/node.mjs @@ -1,5 +1,5 @@ import { constants, createReadStream, createWriteStream, openAsBlob } from 'node:fs' -import { copyFile, lstat, mkdir, mkdtemp, open, readFile, realpath, rm, stat } from 'node:fs/promises' +import { copyFile, lstat, mkdir, mkdtemp, open, readFile, realpath, rm, rmdir, stat } from 'node:fs/promises' import { lookup } from 'node:dns/promises' import { tmpdir } from 'node:os' import { BlockList, isIP } from 'node:net' @@ -161,27 +161,36 @@ export function nodeMedia({ base, temporary, within }) { } /** - * Whether a path, as the file system resolves it, a link followed, is in one of the folders. - * A path that does not exist yet is judged by its nearest folder that does. + * A path as the file system resolves it, links followed; of a path that does not exist yet, its + * nearest folder that does, with the rest of the path after it. Undefined where nothing on it exists. * @param {string} path - * @param {string[]} folders */ -export async function inside(path, folders) { +async function resolved(path) { let real = resolve(path) const tail = [] for (;;) { try { - real = join(await realpath(real), ...tail.reverse()) - break + return join(await realpath(real), ...tail.reverse()) } catch { const parent = dirname(real) - if (parent === real) return false + if (parent === real) return undefined tail.push(basename(real)) real = parent } } +} + +/** + * Whether a path, as the file system resolves it, a link followed, is in one of the folders. + * A path or a folder that does not exist yet is judged by its nearest folder that does. + * @param {string} path + * @param {string[]} folders + */ +export async function inside(path, folders) { + const real = await resolved(path) + if (real === undefined) return false for (const folder of folders) { - const root = await realpath(resolve(folder)).catch(() => resolve(folder)) + const root = (await resolved(folder)) ?? resolve(folder) const from = relative(root, real) // `..notes.png` is a name in the folder; only `..` itself, or a path through it, leaves it. if (!isAbsolute(from) && from !== '..' && !from.startsWith(`..${sep}`)) return true @@ -446,7 +455,9 @@ function writable(path) { * A package read back into the source a build would take to make it again (#236), to change it and * build it again: written into `folder`, made where it is not there, as `.source.json`, with * the package's media beside it at the paths the source names them by. Only the media the content - * names are written, and only under the folder, since an archive is untrusted. Nothing is written + * names are written, and only under the folder, since an archive is untrusted: a file whose path + * is no name a read writes is left out of the source, which so names no file outside the folder, + * and a read that a link in the folder would take outside it writes nothing. Nothing is written * over: where a name is taken, nothing is written, and what was written is taken away again where * a later write fails. * @param {string} file the package, a .h5p @@ -497,9 +508,13 @@ export async function readPackage(file, folder, options = {}) { const output = join(target, `${basename(input).replace(/\.h5p$/i, '') || 'package'}.source.json`) /** @type {{ entry: import('@zip.js/zip.js').FileEntry, to: string, path: string }[]} */ const media = [] + /** @type {Set} */ + const unwritable = new Set() for (const path of read.files) { if (!writable(path)) { - notes.push({ path: 'params', message: `names the file ${JSON.stringify(path)}, which is no path a read writes a file at: not written` }) + // Such a path may lead anywhere, which a build of the source would read from. + unwritable.add(path) + notes.push({ path: 'params', message: `names the file ${JSON.stringify(path)}, which is no path a read writes a file at: left out of the source` }) continue } let decoded = path @@ -512,19 +527,26 @@ export async function readPackage(file, folder, options = {}) { if (!entry) notes.push({ path: 'params', message: `names the file ${path}, which is not in the package: a build of this source asks for it` }) else media.push({ entry, to: join(target, ...path.split('/')), path }) } - // Every name looked at before anything is written, so a read that cannot write all of it - // writes nothing. + const source = unwritable.size ? /** @type {import('./build.mjs').Source} */ (withoutFiles(read.source, unwritable)) : read.source + // Every name looked at before anything is made, so a read that cannot write all of it writes + // nothing: none taken, and none that a link on the way, which the folder may hold, takes out + // of it. The archive names the paths. for (const to of [output, ...media.map((each) => each.to)]) { if (await lstat(to).then(() => true, () => false)) throw exists(to, 'a read') + if (!(await inside(to, [target]))) throw new Error(`${to} leads out of ${target} through a link on its way: a read writes only in its folder`) } - await mkdir(target, { recursive: true }) /** @type {{ path: string, made: Made }[]} */ const made = [] + /** @type {{ first: string, last: string }[]} the folders made, each `mkdir`'s from the first it made to the last */ + const folders = [] + /** @param {string} path */ + const makeFolder = async (path) => { + const first = await mkdir(path, { recursive: true }) + if (first) folders.push({ first, last: path }) + } /** @param {string} to @param {(file: import('node:fs/promises').FileHandle) => Promise} write */ const create = async (to, write) => { - await mkdir(dirname(to), { recursive: true }) - // A folder on the way may be a link out, which a confined read does not write through. - if (within && !(await inside(to, within))) throw new Error(`${to} is outside the folders this read may write in`) + await makeFolder(dirname(to)) const handle = await open(to, 'wx').catch((error) => { throw error?.code === 'EEXIST' ? exists(to, 'a read') : error }) @@ -537,6 +559,7 @@ export async function readPackage(file, folder, options = {}) { } } try { + await makeFolder(target) for (const { entry, to } of media) { // Through the handle itself: a stream made from it would hold it open past the last byte. await create(to, (handle) => entry.getData(new WritableStream({ @@ -545,17 +568,39 @@ export async function readPackage(file, folder, options = {}) { } })).then(() => undefined)) } - await create(output, (handle) => handle.writeFile(`${JSON.stringify(read.source, null, 2)}\n`)) + await create(output, (handle) => handle.writeFile(`${JSON.stringify(source, null, 2)}\n`)) } catch (error) { await Promise.all(made.map(({ path, made: file }) => removeIfMade(path, file))) + // Then the folders it made, the deepest first, where they are empty: what another process has + // put in one since keeps it. + for (const { first, last } of folders.reverse()) { + for (let path = last; ; path = dirname(path)) { + await rmdir(path).catch(() => {}) + if (path === first || dirname(path) === path) break + } + } throw error } - return { output, source: read.source, files: media.map((each) => each.path), notes } + return { output, source, files: media.map((each) => each.path), notes } } finally { await reader.close() } } +/** + * A source without the files at `paths`: one in a list is taken out of it, any other left unwritten. + * @param {unknown} value + * @param {Set} paths + * @returns {unknown} + */ +function withoutFiles(value, paths) { + /** @param {unknown} each */ + const named = (each) => each !== null && typeof each === 'object' && typeof (/** @type {any} */ (each).path) === 'string' && paths.has(/** @type {any} */ (each).path) + if (Array.isArray(value)) return value.filter((each) => !named(each)).map((each) => withoutFiles(each, paths)) + if (value === null || typeof value !== 'object') return value + return Object.fromEntries(Object.entries(value).filter(([, each]) => !named(each)).map(([key, each]) => [key, withoutFiles(each, paths)])) +} + /** The date on every entry of a bundle, as on the pack's parts: a bundle of the same pack is the same bytes. */ const BUNDLE_DATE = new Date(2000, 0, 1, 2) diff --git a/packages/author/lib/read.mjs b/packages/author/lib/read.mjs index 37f4da6..802814e 100644 --- a/packages/author/lib/read.mjs +++ b/packages/author/lib/read.mjs @@ -108,7 +108,7 @@ export async function readContent(pack, { manifest, content }) { else if (entry.minorVersion !== minor) other.push(`${library}, where the pack has ${major}.${entry.minorVersion}`) } if (other.length) { - throw new Error(`This package's content is made for other versions of its libraries than the library pack has: ${other.join('; ')}. A read takes content made for the pack's versions: H5P moves content to a newer version as it plays it, and saved again on a site that has the pack's versions, it reads.`) + throw new Error(`This package's content is made for other versions of its libraries than the library pack has: ${other.join('; ')}. A read takes content made for the pack's versions. H5P moves content to a newer version as it plays it, so the package, saved again on a site that has the pack's versions, reads.`) } /** @type {Problem[]} */ @@ -131,6 +131,7 @@ export async function readContent(pack, { manifest, content }) { const generator = new ContentGenerator(libraries.entries) const params = generator.written(libraries.main, checked.params) + for (const { path, message } of generator.notes) note(path, message) const layout = layoutContext(libraries.entries, () => {}, () => {}) for (const place of placesOf(main, params)) LAYOUTS[place.library]?.read?.(place.params, place.path, layout(place.library)) @@ -185,13 +186,23 @@ function ownFields(manifest, language, note) { const license = LICENSES.includes(manifest.license) ? manifest.license : 'U' if (says(manifest.license) && license !== manifest.license) note('license', `${JSON.stringify(manifest.license)} is no licence H5P names: written as "U"`) const versioned = license.startsWith('CC BY') && CC_VERSIONS.includes(manifest.licenseVersion) + if (says(manifest.licenseVersion) && !versioned) note('licenseVersion', `${JSON.stringify(manifest.licenseVersion)} is no version a source takes, which writes one for a CC BY licence only, as ${CC_VERSIONS.join(', ')}: left out`) /** @type {{ name: string, role: string }[]} */ - const authors = (Array.isArray(manifest.authors) ? manifest.authors : []) - .filter((/** @type {unknown} */ author) => isObject(author) && typeof author.name === 'string' && author.name.trim()) - .map((/** @type {Record} */ author) => ({ name: author.name, role: ROLES.includes(author.role) ? author.role : 'Author' })) + const authors = [] + const listed = Array.isArray(manifest.authors) ? manifest.authors : [] + for (const [i, author] of listed.entries()) { + if (!isObject(author) || typeof author.name !== 'string' || !author.name.trim()) { + note(`authors[${i}]`, 'is no author with a name: left out') + continue + } + const role = ROLES.includes(author.role) ? author.role : 'Author' + if (says(author.role) && role !== author.role) note(`authors[${i}].role`, `${JSON.stringify(author.role)} is no role H5P names: written as "Author"`) + authors.push({ name: author.name, role }) + } if (!authors.length) { authors.push({ name: 'Unknown', role: 'Author' }) - note('authors', 'are not in h5p.json: written as "Unknown", since a source names one') + const why = listed.length ? 'have no name among them' : says(manifest.authors) ? 'are not a list' : 'are not in h5p.json' + note('authors', `${why}: written as "Unknown", since a source names one`) } /** @type {Record} */ const own = { title, language, license, ...(versioned ? { licenseVersion: manifest.licenseVersion } : {}), authors } @@ -200,6 +211,7 @@ function ownFields(manifest, language, note) { for (const key of /** @type {const} */ (['yearFrom', 'yearTo'])) { const year = typeof manifest[key] === 'string' && /^\s*\d+\s*$/.test(manifest[key]) ? Number(manifest[key]) : manifest[key] if (Number.isInteger(year)) own[key] = year + else if (says(manifest[key])) note(key, `${JSON.stringify(manifest[key])} is no year: left out`) } for (const [key, what] of Object.entries(UNWRITTEN)) { if (says(manifest[key])) note(key, `is ${what}, which a source has no field for: left out`) @@ -243,7 +255,8 @@ function ownMetadata(metadata, source) { if (!says(value)) return false if (key === 'license') return value !== 'U' && value !== source.license if (key === 'licenseVersion') return value !== source.licenseVersion - if (key === 'authors') return JSON.stringify(value.map((/** @type {any} */ author) => [author?.name, author?.role])) !== JSON.stringify(source.authors.map((author) => [author.name, author.role])) + // Anything but a list of authors says something of its own too, if not in a form a build takes. + if (key === 'authors') return !Array.isArray(value) || JSON.stringify(value.map((/** @type {any} */ author) => [author?.name, author?.role])) !== JSON.stringify(source.authors.map((author) => [author.name, author.role])) return true }) } @@ -264,7 +277,7 @@ function differences(built, saved, files, path = 'params', found = []) { built.forEach((each, i) => differences(each, saved[i], files, `${path}[${i}]`, found)) } else if (isObject(built) && isObject(saved)) { if (files.has(saved)) { - if (built.path !== saved.path || JSON.stringify(writtenCopyright(built.copyright) ?? null) !== JSON.stringify(writtenCopyright(saved.copyright) ?? null)) found.push(path) + if (built.path !== saved.path || copyrightOf(built.copyright) !== copyrightOf(saved.copyright)) found.push(path) return found } const part = typeof saved.library === 'string' @@ -281,6 +294,16 @@ function differences(built, saved, files, path = 'params', found = []) { return found } +/** + * A file's copyright as a source writes it, for comparing: its parts in one order, H5P's editor + * saving them in its own, and a licence left out being the "U" a build gives. + * @param {unknown} copyright + */ +function copyrightOf(copyright) { + const written = writtenCopyright(copyright) + return written ? JSON.stringify(Object.entries({ license: 'U', ...written }).sort()) : '' +} + /** * Whether a built text is the saved one as an author would write it: an empty text where nothing * is built, as H5P's editor saves a field nobody typed in; and text without markup in a markup diff --git a/packages/author/lib/text.mjs b/packages/author/lib/text.mjs index e4969a4..cf3ad66 100644 --- a/packages/author/lib/text.mjs +++ b/packages/author/lib/text.mjs @@ -188,7 +188,7 @@ export function hasMarkup(value) { * @param {string} block `p` or `div`, as `blockTag` says for the field */ export function wrapText(value, block) { - const paragraphs = value.trim().split(/\n[ \t]*\n+/).filter((paragraph) => paragraph.trim()) + const paragraphs = value.trim().split(/\r?\n[ \t]*(?:\r?\n)+/).filter((paragraph) => paragraph.trim()) return paragraphs.map((paragraph) => `<${block}>${escapeText(paragraph.trim()).replace(/\r?\n/g, '
')}`).join('') } diff --git a/packages/author/tests/read.test.ts b/packages/author/tests/read.test.ts index 6dca9dc..775036a 100644 --- a/packages/author/tests/read.test.ts +++ b/packages/author/tests/read.test.ts @@ -1,9 +1,9 @@ import { execFile } from 'node:child_process' -import { lstat, mkdir, mkdtemp, readdir, readFile, realpath, rm, writeFile } from 'node:fs/promises' +import { lstat, mkdir, mkdtemp, readdir, readFile, realpath, rm, symlink, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join, resolve } from 'node:path' import { promisify } from 'node:util' -import { BlobReader, BlobWriter, TextReader, TextWriter, Uint8ArrayReader, ZipReader, ZipWriter, type FileEntry } from '@zip.js/zip.js' +import { BlobReader, BlobWriter, TextReader, TextWriter, Uint8ArrayReader, Uint8ArrayWriter, ZipReader, ZipWriter, type FileEntry } from '@zip.js/zip.js' import { afterAll, beforeAll, describe, expect, it } from 'vitest' import { buildPackage, readPackage } from '../lib/author.mjs' import { mp4, png } from './helpers/media' @@ -81,6 +81,17 @@ async function opened(path: string) { } } +/** The bytes of a package's entry. */ +async function entryOf(path: string, name: string) { + const reader = new ZipReader(new BlobReader(new Blob([await readFile(path)]))) + try { + const entry = (await reader.getEntries()).find((each) => each.filename === name) as FileEntry | undefined + return entry && await entry.getData(new Uint8ArrayWriter()) + } finally { + await reader.close() + } +} + /** * A package as another tool exports it: its `h5p.json`, its content and its media, and no library, * as h5p.com exports one without them. @@ -107,7 +118,8 @@ describe.concurrent('every type file', () => { expect(JSON.parse(await readFile(read.output, 'utf8'))).toEqual(read.source) const { type, title, language, license, authors } = read.source expect({ type, title, language, license, authors }).toEqual({ type: first.manifest.mainLibrary, ...META }) - for (const path of read.files) expect(await exists(join(at, 'read', path)), path).toBe(true) + // Each file as the package holds it, at the path the source names it by. + for (const path of read.files) expect(new Uint8Array(await readFile(join(at, 'read', path))), path).toEqual(await entryOf(join(at, 'example.h5p'), `content/${path}`)) const again = await buildPackage(read.source, undefined, { base: join(at, 'read'), check: true }) expect(withoutIds(again.content)).toEqual(withoutIds(first.content)) }, 60_000) @@ -198,15 +210,27 @@ describe('a package read back', () => { it('writes only inside its folder, and only where it may', async () => { const at = await folder() - const { output } = await elsewhere(at) + const { output, image } = await elsewhere(at) const { manifest, content } = await opened(output) content.questions[0].params.media.type.params.file.path = '../../outside.png' const hostile = join(at, 'hostile.h5p') await exported(hostile, manifest, content) const read = await readPackage(hostile, join(at, 'read')) - expect(read.notes).toContainEqual({ path: 'params', message: 'names the file "../../outside.png", which is no path a read writes a file at: not written' }) + expect(read.notes).toContainEqual({ path: 'params', message: 'names the file "../../outside.png", which is no path a read writes a file at: left out of the source' }) + // Nor does the source name it, for a build to read it from there. + expect(paths(read.source.params)).toEqual([]) + expect(paths(JSON.parse(await readFile(read.output, 'utf8')).params)).toEqual([]) expect(read.files).toEqual([]) expect(await exists(join(at, 'outside.png'))).toBe(false) + // A link in the folder that the package's names lead through stops the read before anything is made. + const outside = await folder() + const [first] = image.split('/') + expect(image).toContain('/') + await mkdir(join(at, 'linked')) + await symlink(outside, join(at, 'linked', first), 'junction') + await expect(readPackage(output, join(at, 'linked'))).rejects.toThrow(/leads out of .+ through a link on its way: a read writes only in its folder/) + expect(await readdir(outside)).toEqual([]) + expect(await readdir(join(at, 'linked'))).toEqual([first]) // An archive whose entry leads out of its folders is not read at all. const leading = join(at, 'leading.h5p') await exported(leading, manifest, content, { '../../outside.png': new Uint8Array(png(8, 8)) }) @@ -217,6 +241,62 @@ describe('a package read back', () => { await expect(readPackage(output, join(elsewhereFolder, '..', 'read-3'), { within: [at] })).rejects.toThrow(/is outside the folders this read may write in/) }) + it('takes away what it made where a write fails, and keeps what was there', async () => { + const at = await folder() + const { output, image } = await elsewhere(at) + // The picture's compressed bytes made to start a block of a type deflate has none of. + const bytes = new Uint8Array(await readFile(output)) + const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength) + const name = new TextEncoder().encode(`content/${image}`) + const start = bytes.findIndex((_, i) => name.every((byte, j) => bytes[i + j] === byte)) + const header = start - 30 + expect(view.getUint32(header, true)).toBe(0x04034b50) + expect(view.getUint16(header + 8, true)).toBe(8) + bytes[start + view.getUint16(header + 26, true) + view.getUint16(header + 28, true)] = 0xff + const broken = join(at, 'broken.h5p') + await writeFile(broken, bytes) + // Found broken as it is written, once the folders for it are made. + await expect(readPackage(broken, join(at, 'read', 'deeper'))).rejects.toThrow('Invalid compressed data') + expect(await exists(join(at, 'read'))).toBe(false) + // A folder that was there stays, as does what is in it. + await mkdir(join(at, 'mine')) + await writeFile(join(at, 'mine', 'notes.txt'), 'mine') + await expect(readPackage(broken, join(at, 'mine'))).rejects.toThrow('Invalid compressed data') + expect(await readdir(join(at, 'mine'))).toEqual(['notes.txt']) + }) + + it('says what of its metadata a source has no form of, and leaves it out', async () => { + const at = await folder() + const { output, image } = await elsewhere(at) + const { manifest, content } = await opened(output) + Object.assign(manifest, { license: 'GNU GPL', licenseVersion: 'v3', yearFrom: 'last year', authors: [{ name: 'Ann Author', role: 'Writer' }, { role: 'Editor' }, 'Bo Brown'] }) + content.questions[1].metadata = { ...content.questions[1].metadata, authors: 'Cy' } + content.questions[0].params.media.type.params.file.copyright = { title: 'River', license: 'CC0 1.0', version: '1.0', source: 'Wikipedia' } + const odd = join(at, 'odd.h5p') + await exported(odd, manifest, content, { [image]: new Uint8Array(png(320, 180)) }) + const read = await readPackage(odd, join(at, 'read')) + expect(read.source).toMatchObject({ license: 'GNU GPL', authors: [{ name: 'Ann Author', role: 'Author' }] }) + expect(read.source).not.toHaveProperty('licenseVersion') + expect(read.source).not.toHaveProperty('yearFrom') + const copyright = 'params.questions[0].params.media.type.params.file.copyright' + expect(read.notes).toEqual(expect.arrayContaining([ + { path: 'licenseVersion', message: '"v3" is no version a source takes, which writes one for a CC BY licence only, as 4.0, 3.0, 2.5, 2.0, 1.0: left out' }, + { path: 'authors[0].role', message: '"Writer" is no role H5P names: written as "Author"' }, + { path: 'authors[1]', message: 'is no author with a name: left out' }, + { path: 'authors[2]', message: 'is no author with a name: left out' }, + { path: 'yearFrom', message: '"last year" is no year: left out' }, + { path: `${copyright}.license`, message: '"CC0 1.0" is no licence H5P lists for a file: left out, with any version, so a build gives the file "U"' }, + { path: `${copyright}.source`, message: '"Wikipedia" is no web address: left out' } + ])) + // A part's authors that are no list are its own too. + expect(read.notes.find(({ path }) => path === 'params.questions[0].metadata')?.message).toContain('params.questions[1].metadata') + expect(read.notes.filter(({ message }) => message.includes('reads back otherwise'))).toEqual([]) + const params = read.source.params as { questions: { params: { media: { type: { params: { file: unknown } } } } }[] } + expect(params.questions[0].params.media.type.params.file).toEqual({ path: image, copyright: { title: 'River' } }) + const again = await buildPackage(read.source, undefined, { base: join(at, 'read'), check: true }) + expect(again.notes).toEqual([]) + }) + it('is a command: h5p-author read writes the source and its media, and says how to build it again', async () => { const at = await folder() const { image } = await elsewhere(at) diff --git a/packages/author/tests/text.test.ts b/packages/author/tests/text.test.ts index 4f0faee..2ede29b 100644 --- a/packages/author/tests/text.test.ts +++ b/packages/author/tests/text.test.ts @@ -63,6 +63,10 @@ describe('text as H5P stores it', () => { '

One line
and the next

A second: x < 5 & more

' ) expect(markupText('Plain', { tags: ['em'] }, report)).toBe('
Plain
') + // Windows line endings, a blank line among them, as text copied from a Windows file has. + expect(markupText('One line\r\nand the next\r\n \r\n\r\nA second', { tags: [], enterMode: 'p' }, report)).toBe( + '

One line
and the next

A second

' + ) }) it('keeps markup the field takes, escapes a stray < or >, and reports what H5P would remove', () => { From a8f429192fbd29469d2d18f4cdf57c241a17f5b4 Mon Sep 17 00:00:00 2001 From: Aleksandr Shevchuk Date: Wed, 18 Nov 2026 20:11:46 +0200 Subject: [PATCH 4/5] fix(mcp): h5p_read names every file in the folder it read into (#236) A file the package lacks was passed back by its path in the package, which h5p_build looked for in the first folder. Every local path is now named in the read's folder, so a build looks for it there and nowhere else. The tests check the source file written, compare paths portably, and read a package that lacks its picture. --- design/mcp.md | 5 +++-- packages/mcp/lib/server.mjs | 16 +++++++++------- packages/mcp/tests/server.test.ts | 28 +++++++++++++++++++++++++--- 3 files changed, 37 insertions(+), 12 deletions(-) diff --git a/design/mcp.md b/design/mcp.md index d442550..b444b5e 100644 --- a/design/mcp.md +++ b/design/mcp.md @@ -54,8 +54,9 @@ server, for ChatGPT or Claude on the web, is #210 and not this. - **A read writes beside the package, and names its files where it wrote them** (#236). `h5p_read` takes a package in the folders, a name alone being the first folder's, and writes its source and media into a new folder beside it, named after it and numbered where the name is - taken, as a package is. It answers with the source, each file it wrote named by its full path, - so `h5p_build` reads them wherever its package goes. Asked to add two harder questions to the + taken, as a package is. It answers with the source, each file named by its full path in that + folder, so `h5p_build` reads them wherever its package goes, and looks for one the package lacks + there and nowhere else. Asked to add two harder questions to the demo's quiz, Haiku read it and built at the first build in 6 of 6 runs. With the package described as a path in the server's folders, it first guessed one from its working folder in 3 of 3; described "by its file name alone", in none of 3. diff --git a/packages/mcp/lib/server.mjs b/packages/mcp/lib/server.mjs index 3c141f2..d25c0c1 100644 --- a/packages/mcp/lib/server.mjs +++ b/packages/mcp/lib/server.mjs @@ -68,17 +68,19 @@ const OUTLINE_LINES = 80 const sourceBeside = (path) => path.replace(/\.h5p$/i, '.source.json') /** - * A read source's files named where they were written, so that a build reads them wherever its - * package goes: each file the read wrote, by its path from the folder, made that folder's. + * A read source's files named in the folder it was read into, so that a build looks for each there + * and nowhere else: one the read wrote, where it is, and one the package lacks, where the person can + * put it. Every file a read source names, but by a web address, is under its folder by its path + * (`readPackage`); a path is a field of a file alone in every content type. * @param {unknown} value * @param {string} folder - * @param {Set} files * @returns {unknown} */ -function located(value, folder, files) { - if (Array.isArray(value)) return value.map((each) => located(each, folder, files)) +function located(value, folder) { + if (Array.isArray(value)) return value.map((each) => located(each, folder)) if (!value || typeof value !== 'object') return value - const entries = Object.entries(value).map(([key, each]) => [key, key === 'path' && typeof each === 'string' && files.has(each) ? join(folder, ...each.split('/')) : located(each, folder, files)]) + const local = (/** @type {unknown} */ each) => typeof each === 'string' && !/^[a-z][a-z0-9+.-]*:/i.test(each) + const entries = Object.entries(value).map(([key, each]) => [key, key === 'path' && local(each) ? join(folder, ...String(each).split('/')) : located(each, folder)]) return Object.fromEntries(entries) } @@ -275,7 +277,7 @@ export async function createServer(options = {}) { const media = result.files.length ? `, with its ${result.files.length} media file${result.files.length === 1 ? '' : 's'} beside it` : '' const lines = [`Read ${path}: ${result.source.type}. Its source is in ${result.output}${media}.`] for (const { path: at, message } of result.notes) lines.push(`Note: ${at} ${message}`) - const source = { ...result.source, params: located(result.source.params, folder, new Set(result.files)) } + const source = { ...result.source, params: located(result.source.params, folder) } lines.push('', 'The source, each file named where it was written: change it as the person asks and pass it to h5p_build.', JSON.stringify(source)) return answer(lines.join('\n')) }) diff --git a/packages/mcp/tests/server.test.ts b/packages/mcp/tests/server.test.ts index 1aa35ae..0f7f546 100644 --- a/packages/mcp/tests/server.test.ts +++ b/packages/mcp/tests/server.test.ts @@ -1,4 +1,4 @@ -import { access, mkdtemp, readFile, readdir, rm, symlink, writeFile } from 'node:fs/promises' +import { access, mkdir, mkdtemp, readFile, readdir, rm, symlink, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { basename, join, resolve } from 'node:path' import { Client } from '@modelcontextprotocol/client' @@ -168,9 +168,11 @@ describe('h5p_read', () => { expect(text).toContain(`Read ${built}: H5P.Column. Its source is in ${join(beside, 'one-pixel.source.json')}, with its 1 media file beside it.`) const read = JSON.parse(text.split('\n').at(-1)!) expect(read).toMatchObject({ type: 'H5P.Column', title: 'One pixel', params: { content: [{ content: { library: 'H5P.Image', params: { alt: 'A pixel' } } }] } }) - // Where it was written, read by a build wherever its package goes. + // Where it was written, read by a build wherever its package goes: the source file, written + // beside it, names it from there. + const written = JSON.parse(await readFile(join(beside, 'one-pixel.source.json'), 'utf8')) const path = read.params.content[0].content.params.file.path - expect(path.startsWith(`${beside}/`)).toBe(true) + expect(path).toBe(join(beside, ...written.params.content[0].content.params.file.path.split('/'))) await access(path) const again = builtOf(await call('h5p_build', { source: { ...read, title: 'One pixel, again' }, verify: false })) expect(again).toBe(join(folder, 'one-pixel-again.h5p')) @@ -178,6 +180,26 @@ describe('h5p_read', () => { expect(textOf(await call('h5p_read', { package: 'one-pixel.h5p' }))).toContain(`Its source is in ${join(folder, 'one-pixel-2', 'one-pixel.source.json')}`) }) + it('names a file the package lacks in the folder it read into, so a build looks for it there and nowhere else', async () => { + await writeFile(join(folder, 'pixel.png'), PIXEL) + const picture = { library: 'H5P.Image', params: { file: { path: 'pixel.png' }, alt: 'A pixel' } } + const built = builtOf(await call('h5p_build', { source: { ...SOURCE, type: 'H5P.Column', title: 'Lost pixel', params: { content: [{ content: picture }] } }, verify: false }))! + // The picture's entry renamed, as a package made elsewhere may lack a file its content names. + const bytes = (await readFile(built)).toString('latin1') + const entry = /content\/(images\/[^/\0]+?)\.png/.exec(bytes)![1] + await writeFile(join(folder, 'lost.h5p'), Buffer.from(bytes.replaceAll(`content/${entry}.png`, `content/${entry}.pnx`), 'latin1')) + // Where a build given the package's path alone would look. + await mkdir(join(folder, 'images'), { recursive: true }) + await writeFile(join(folder, `${entry}.png`), PIXEL) + const text = textOf(await call('h5p_read', { package: 'lost.h5p' })) + expect(text).toContain(`Note: params names the file ${entry}.png, which is not in the package: a build of this source asks for it`) + const read = JSON.parse(text.split('\n').at(-1)!) + expect(read.params.content[0].content.params.file.path).toBe(join(folder, 'lost', ...`${entry}.png`.split('/'))) + const again = await call('h5p_build', { source: { ...read, title: 'Lost pixel, again' }, verify: false }) + expect(builtOf(again)).toBeUndefined() + expect(textOf(again)).toContain('no such file in the folders this build may read') + }) + it('refuses a package outside its folders, and says why one is refused', async () => { await writeFile(join(elsewhere, 'outside.h5p'), 'not a package') const outside = await call('h5p_read', { package: join(elsewhere, 'outside.h5p') }) From f47a2c54d1fd2e6f94d0c0dbaa7e3a0e38e3e638 Mon Sep 17 00:00:00 2001 From: Aleksandr Shevchuk Date: Wed, 18 Nov 2026 20:19:34 +0200 Subject: [PATCH 5/5] fix(author): authors that are an empty list are said as such (#236) --- packages/author/lib/read.mjs | 4 +++- packages/author/tests/read.test.ts | 5 +++++ 2 files changed, 8 insertions(+), 1 deletion(-) diff --git a/packages/author/lib/read.mjs b/packages/author/lib/read.mjs index 802814e..943a00b 100644 --- a/packages/author/lib/read.mjs +++ b/packages/author/lib/read.mjs @@ -201,7 +201,9 @@ function ownFields(manifest, language, note) { } if (!authors.length) { authors.push({ name: 'Unknown', role: 'Author' }) - const why = listed.length ? 'have no name among them' : says(manifest.authors) ? 'are not a list' : 'are not in h5p.json' + const why = manifest.authors === undefined || manifest.authors === null ? 'are not in h5p.json' + : !Array.isArray(manifest.authors) ? 'are not a list' + : listed.length ? 'have no name among them' : 'are an empty list' note('authors', `${why}: written as "Unknown", since a source names one`) } /** @type {Record} */ diff --git a/packages/author/tests/read.test.ts b/packages/author/tests/read.test.ts index 775036a..b2f2a0e 100644 --- a/packages/author/tests/read.test.ts +++ b/packages/author/tests/read.test.ts @@ -178,6 +178,7 @@ describe('a package read back', () => { ] }) expect(read.notes.map(({ path }) => path)).toEqual(['language', 'params.questions[0].params.oldField', 'authors', 'authorComments', 'changes', 'params.questions[0].metadata']) + expect(read.notes.find(({ path }) => path === 'authors')?.message).toBe('are not in h5p.json: written as "Unknown", since a source names one') expect(read.notes.find(({ path }) => path === 'params.questions[0].metadata')?.message).toMatch(/licence, authors or more of their own.*a build gives every part the source's/) expect(read.files).toEqual([image]) expect(await exists(join(at, 'read', image))).toBe(true) @@ -295,6 +296,10 @@ describe('a package read back', () => { expect(params.questions[0].params.media.type.params.file).toEqual({ path: image, copyright: { title: 'River' } }) const again = await buildPackage(read.source, undefined, { base: join(at, 'read'), check: true }) expect(again.notes).toEqual([]) + // Authors as a list with no one in it, which H5P never exports, are said as they are. + const empty = join(at, 'empty.h5p') + await exported(empty, { ...manifest, authors: [] }, content, { [image]: new Uint8Array(png(320, 180)) }) + expect((await readPackage(empty, join(at, 'read-empty'))).notes).toContainEqual({ path: 'authors', message: 'are an empty list: written as "Unknown", since a source names one' }) }) it('is a command: h5p-author read writes the source and its media, and says how to build it again', async () => {