Create scans in an archive, and drive one from a parsed PDF

CLAUDE.md

@@ -458,6 +458,36 @@ before/after pair is the only way to learn where a printed value is stored --
   files then differ from what we wrote in exactly one respect: the console
   recompresses (its DEFLATE is tighter -- 770 KB against our 934 KB) and
   otherwise the ASCCONV text is identical field for field, churn included.
+- **A scan can be added to an archive, and the scanner accepts it.** Four
+  duplicated scans loaded, kept their running order, and one carrying an edit
+  kept its own protocol -- 19 distinct protocols in and 19 out. So generation
+  is not blocked by the format. Adding one scan means three new instances
+  (step, protocol, label) with fresh ids in all three GUID spaces, an `Element`
+  row and an `InstanceChangeSet` row for each, three pairs appended to the head
+  changeset's `ElementToInstanceMap`, and the step's element appended to the
+  program's `Children`.
+- **`EdfProgramContent` has five parallel maps keyed by step id, not one.**
+  `LinksFrom` (outgoing), `LinksTo` (incoming, as `$ref` back-pointers into the
+  link objects `LinksFrom` defines), `Ranks` (`{Rank: 0..N, StepId}`, the
+  running order), and `RelationsFrom`/`RelationsTo` (an empty list each). A step
+  present in only some of them leaves the console unable to build the program,
+  which it reports by showing the folder tree with no protocols in it -- the
+  same symptom as an archive exported from an empty folder node, and the first
+  duplication attempt was rejected exactly that way.
+- **A protocol and a label each carry `ParentElementId` pointing at their own
+  step**, and only the step points at the program. The console resolves a
+  step's protocol through that reverse link rather than through the step's
+  `Children` blob, so a copy that keeps the source's pointer is served the
+  source's protocol. That is invisible while the copy is identical and silently
+  discards every edit once it is not: the second attempt loaded all 22 scans
+  and returned the edited copy with its source's TR. Any generated scan needs
+  one deliberately edited parameter or the test cannot fail.
+- **`Instance.Tags` on a protocol carries `#ContentHash|<sha1 of the Data
+  string>`**, which is neither the `ContentHash` column nor the stored blob's
+  hash. It matches all 18 protocols in the reference archive.
+  `replace_content` recomputes it. A stale one is tolerated on load -- two
+  scans in the NAV option scan shared one and both kept their values -- but it
+  is derivable and describes the content.
 - **Import and re-export is not the same as editing.** `tCheckUUID`, the GUID
   leading `sWipMemBlock.tFree` and the date hidden in `sSpecPara.lFinalMatrixSize*`
   are regenerated when a protocol is *edited on the console*, and left alone
@@ -495,6 +525,26 @@ before/after pair is the only way to learn where a printed value is stored --
   the corpus, and the sweeps that need scans take `protocol_archive_path`
   while the structural ones (envelope, hashing, GUID layout) still take
   `archive_path` and are exercised by it.
+- **`siemens-protocol-tool exar <archive> <pdf>` is the driver**, and its
+  manifest is as much the point as its output. Roughly a tenth of what a
+  protocol prints has a verified mapping, so a built archive is mostly the
+  template it started from; the report states that fraction, counts inherited
+  values and names the unmapped parameters by frequency, which is what says
+  where the next mapping is worth deriving. Driving an archive from its *own*
+  PDF must write nothing -- that one check exercises units, scales, the derived
+  basis, sparse arrays and change detection at once, and it caught two spurious
+  writes where a printed `0.00` met an assignment a sparse array omits.
+- **Scans are matched to the template by name, and an unmatched one is
+  reported rather than guessed at.** The PDF names a sequence by kernel
+  (`epfid`) and the archive by sequence file (`cmrr_mbep2d_bold`), so there is
+  no reliable way to pick a donor scan to copy. `generate.duplicate_step` is
+  available to a caller who knows which scan to copy; the driver will not
+  choose one.
+- **`patch.resolve` falls back to the label `Preview` prints.** A multi-echo
+  scan prints `TE 1` where a single-echo one prints `TE`, and the preview entry
+  is labelled the same way, so resolving through it follows the printout
+  instead of duplicating every spelling in the table. Without that the driver
+  silently skipped TE on exactly the scans that print it differently.
 - Everything so far is XA60 (`VA60A`). No XA30 archive has been seen, so the
   claim that the model is release-independent is untested -- that is the first
   thing to check when one arrives.

src/siemens_protocol/cli.py

@@ -341,6 +341,27 @@ def build_parser() -> argparse.ArgumentParser:
     )
     vocab_suggest.add_argument("--vocabulary", metavar="DIR", help="an overlay directory")
 
+    exar_cmd = sub.add_parser(
+        "exar",
+        help="write a protocol PDF's parameters into an XA .exar1 archive",
+        description=(
+            "Take a template .exar1 archive and a protocol PDF, write every "
+            "parameter that has a verified mapping, and report what could not "
+            "be written. Only a fraction of what a protocol prints is mapped, "
+            "so the result is mostly the template it started from -- the "
+            "manifest says how much, and names the parameters no mapping "
+            "covers so the gap is visible rather than implied. Nothing is "
+            "written without --out."
+        ),
+    )
+    exar_cmd.add_argument("archive", help="the template .exar1 archive")
+    exar_cmd.add_argument("input", help="a protocol PDF, or a previously parsed JSON file")
+    add_release_option(exar_cmd, "force a Siemens release profile for a PDF input (default: auto)")
+    exar_cmd.add_argument("--out", help="write the resulting archive here")
+    exar_cmd.add_argument(
+        "--show", type=int, default=12, metavar="N", help="how many entries to list (default: 12)"
+    )
+
     versions_cmd = sub.add_parser("versions", help="list the known version profiles")
     versions_cmd.set_defaults(command="versions")
 
@@ -1268,6 +1289,48 @@ def use_utf8_output() -> None:
             pass
 
 
+def _run_exar(args: argparse.Namespace) -> int:
+    """Write a PDF's mapped parameters into a template archive.
+
+    Parameters
+    ----------
+    args : argparse.Namespace
+        Parsed arguments carrying ``archive``, ``input``, ``out`` and ``show``.
+
+    Returns
+    -------
+    int
+        Process exit status.
+    """
+    from .exar import build as exar_build
+    from .exar import read as read_exar
+    from .exar import validate as exar_validate
+
+    try:
+        archive = read_exar(args.archive)
+        protocol = _load_protocol(args.input, getattr(args, "release", "auto"))
+    except (OSError, ValueError) as exc:
+        print(f"{exc}", file=sys.stderr)
+        return 1
+
+    report = exar_build.apply_protocol(archive, protocol)
+    print(report.report(limit=args.show))
+
+    problems = exar_validate.problems(archive)
+    if problems:
+        print("\nthe result is not structurally sound:", file=sys.stderr)
+        for line in problems:
+            print(f"  {line}", file=sys.stderr)
+        return 1
+
+    if args.out:
+        archive.write(args.out)
+        print(f"\nwrote {args.out}")
+    else:
+        print("\n(no --out given, so nothing was written)")
+    return 0
+
+
 def main(argv: list[str] | None = None) -> int:
     """Run the command line interface.
 
@@ -1300,6 +1363,9 @@ def main(argv: list[str] | None = None) -> int:
     if args.command == "list":
         return _run_list(args)
 
+    if args.command == "exar":
+        return _run_exar(args)
+
     if args.command == "sequences":
         return _run_sequences(args)
 

src/siemens_protocol/exar/__init__.py

@@ -14,6 +14,7 @@
 
 from __future__ import annotations
 
+from . import generate, patch, validate
 from .archive import (
     Archive,
     Instance,
@@ -28,6 +29,9 @@
 
 __all__ = [
     "Archive",
+    "generate",
+    "patch",
+    "validate",
     "Envelope",
     "Instance",
     "PreviewEntry",

src/siemens_protocol/exar/archive.py

@@ -28,6 +28,8 @@
 
 from __future__ import annotations
 
+import hashlib
+import re
 import uuid
 from dataclasses import dataclass, field
 from typing import Any, Iterator
@@ -493,6 +495,10 @@ def replace_content(self, instance: Instance, document: dict[str, Any]) -> str:
         rows = self.container.tables["Instance"]
         for position in rows.find("Id", instance.id):
             rows.set(position, "ContentHash", digest)
+            tags = rows.rows[position][rows.index_of("Tags")]
+            refreshed = _refresh_content_tag(tags, document)
+            if refreshed != tags:
+                rows.set(position, "Tags", refreshed)
         instance.content_hash = digest
         return digest
 
@@ -511,6 +517,34 @@ def write(self, path: str) -> None:
         store.write(self.container, path)
 
 
+def _refresh_content_tag(tags: Any, document: dict[str, Any]) -> Any:
+    """Keep a protocol's ``#ContentHash`` tag in step with its XProtocol text.
+
+    ``Instance.Tags`` carries a fingerprint of the protocol -- the SHA-1 of the
+    ``Data`` string, which is not the ``ContentHash`` column and not the hash of
+    the stored blob. The console recomputes it on save, and a stale one was
+    tolerated on load, but it is derivable and describes the content, so it is
+    kept correct rather than left to drift.
+
+    Parameters
+    ----------
+    tags : Any
+        The instance's current ``Tags`` value, which may be ``None``.
+    document : dict
+        The replacement document. Only protocol content carries ``Data``.
+
+    Returns
+    -------
+    Any
+        The updated tags, or the original when there is nothing to update.
+    """
+    data = document.get("Data") if isinstance(document, dict) else None
+    if not isinstance(tags, str) or not isinstance(data, str):
+        return tags
+    digest = hashlib.sha1(data.encode("utf-8")).hexdigest()
+    return re.sub(r"(#ContentHash\|)[0-9a-f]+", rf"\g<1>{digest}", tags)
+
+
 def _head_branch(container: store.Container) -> tuple[str, str]:
     """Choose which branch head to read the archive at.