Skip to content

feat(core): import a file's fields from a JSON object - #754

Open
azfoo wants to merge 1 commit into
v1/275-build-documentfrom
v1/276-import-metadata
Open

azfoo wants to merge 1 commit into
v1/275-build-documentfrom
v1/276-import-metadata

Conversation

@azfoo

@azfoo azfoo commented Oct 19, 2026

Copy link
Copy Markdown
Collaborator

🤖 Claude

Adds tilia_core.importers.import_metadata(document, path), TiLiA's metadata import in the core. It changes the document in memory and returns an ImportResult.

  • Fields. Each field of the JSON object is set where the document has it, or added after the others. The JSON's value wins, and the document's other fields stay. Names and values are stored in NFC, by build.set_field.

  • Values become text as the migration makes an old file's fields text (migrate/to_1.py's _field, in Read files from older TiLiA versions through the migration #745 (read old files through the migration)):

    • text stays as it is, and null becomes empty text;
    • a list of plain values becomes a list of texts;
    • anything else becomes its JSON: 1795 gives "1795", true gives "true", {"a": 1} gives '{"a": 1}'.
  • media length, in any letter case, since metadata set refuses it as a field in any letter case:

    • While no media is attached, it sets the length through build.set_media_length, with all its checks. A positive number written as text ("312.5") counts, as the migration reads an old file's length. A length it refuses fails the import with the refusal's text, and nothing is set, not even the other fields.
    • While media is attached, it is ignored, with a note in result.notes: media length ignored: Can't change media length when a media file is loaded.
    • A JSON without one leaves the length as it was. TiLiA resets it to 0.
  • A file it can't import raises ImportFailed and changes nothing:

    • a file that doesn't exist: TiLiA's File {path} not found.;
    • a file that isn't JSON: TiLiA's Error when parsing file {path}: + line break + the error's repr, as TiLiA shows it. This includes nesting too deep to read and a number too long to read, where TiLiA would crash;
    • JSON that isn't an object: the same text, with expected an object of fields;
    • a file that isn't UTF-8: '{path}' is not a valid UTF-8 encoded JSON file., TiLiA's UTF8_DECODE_FAILED filled with JSON.

    A byte-order mark is accepted. A path that can't be opened for another reason, such as a folder, raises OSError, as _csv.read_csv does.

  • tilia_core.importers exports import_metadata, ImportResult, RowProblem and ImportFailed.

TimeLineAnnotator/project-management#326 (metadata import: a refused media length and three texts) is open on four of these points. It covers a refused length failing the import, a length written as text, the note's text, and the texts for JSON that isn't an object and a file that isn't UTF-8. An answer there changes only that behaviour and its tests.

Tests

  • tests/importers/test_importers_metadata.py (77 tests), written first:

    • the spec's User Story 6, scenarios 7 and 8;
    • each kind of value as text, NFC, a repeated key, a byte-order mark;
    • media length with and without media, in other letter cases, written as text, and each refusal;
    • each file it can't import, with TiLiA's texts word for word.

    Every failure is checked to leave the document as it was.

  • uv run --package tilia-core pytest packages/tilia-core -q: 978 passed, 21 skipped, 8 xfailed. The importers' tests also pass on Python 3.10 and 3.14.

  • python scripts/check_packages.py passes. black (22.12.0) and ruff are clean, and the files have LF line endings.

  • This branch starts from feat(core): build a script's document: fields, media, its length and timelines #748's 7e935a2. feat(core): build a script's document: fields, media, its length and timelines #748 has since gained b032e08. This commit cherry-picks onto that cleanly, and the suite passes there too (989 passed, 21 skipped, 8 xfailed). The branch still needs a rebase onto it, which means a force-push.

Guarantees. I neutralised the code behind each one, ran test_importers_metadata.py (77 tests), then restored it:

Guarantee Code neutralised Tests failing
the JSON's value wins over a field the document has set only fields it lacks 1
a value that isn't text becomes its JSON str() in place of json.dumps 11
… non-ASCII kept as it is ensure_ascii=False 2
null becomes empty text the None case 2
a list of plain values stays a list of texts the list case 3
a list holding lists or objects becomes one text the check on its items 2
media length in any letter case lower() 5
ignored with a note while media is attached the media_path check 7
the note names the key as the JSON spells it the name 1
the length goes through build.set_media_length set media_length directly 19
a refused length fails the import return in place of raise 19
… and no field is set before it fields set before the length 17
… and an earlier key's length is undone the rollback 1
a length written as text is read as a number the text case 4
text that isn't a positive number is refused as written the isfinite and > 0 check 3
a missing file: TiLiA's text the FileNotFoundError case 3
a file that isn't UTF-8: its own text the UnicodeDecodeError case 1
a file that isn't JSON fails the import ValueError in the except 5
nesting too deep to read fails the import RecursionError in the except 1
TiLiA's form, with the error's repr str() in place of repr() 6
JSON that isn't an object fails the import the dict check 6
a byte-order mark is accepted utf-8-sig 1
a value too deep to write as text fails the import its except 1

Two guarantees have no code of their own here, so none could be neutralised:

Stacked on #748.

Part of TimeLineAnnotator/project-management#276

🤖 Generated with Claude Code

importers.import_metadata adds a JSON object's fields to the document's:
each is set where it is, or added after the others, the JSON's value
winning. Values become text as the migration makes an old file's fields
text: a list of plain values stays a list of texts, null is empty text,
anything else is its JSON.

A `media length` key, in any letter case, sets the length through
build.set_media_length while no media is attached; a positive number
written as text counts. While media is attached it is ignored, with a
note. A length it refuses fails the import.

A file that doesn't exist, isn't UTF-8, isn't JSON or isn't an object
raises ImportFailed with TiLiA's texts, and the import changes nothing.
The importers package exports import_metadata and its result types.

Part of TimeLineAnnotator/project-management#276

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant