Create scans in an archive, and drive one from a parsed PDF - #7
Merged
Conversation
A scanner accepts archives built this way: four added scans loaded, kept their running order, and the one carrying an edit kept its own protocol -- 19 distinct protocols in and 19 out. Generation is not blocked by the format. Getting there cost two defects, and both are the reason validate.py exists rather than a comment. The first was rejected outright. EdfProgramContent describes the running order in five maps keyed by step id -- LinksFrom, LinksTo, Ranks, RelationsFrom, RelationsTo -- and the first attempt updated one. A step present in only some leaves the console unable to build the program, which it reports by showing the folder tree with no protocols in it. That is indistinguishable from an archive exported off an empty folder node, which is how it was first misread. The second loaded and was silently wrong. A protocol and a label each carry ParentElementId pointing at their own step, and the console resolves a step's protocol through that reverse link rather than through the step's Children blob. Copies that kept the source's pointer were served the source's protocol. Three of the four duplicates looked perfect, because they duplicate content identical to their source by design; only the copy carrying a deliberate edit could show it, returning its source's TR. A generated object must differ from its template in some observable way or the test cannot tell "created correctly" from "aliased to the original". Both archives were internally coherent throughout: every reference resolved, every id was unique. So the checks in validate.py are stated as relationships that hold in every console-authored archive and re-derived from the archive under test, not as self-consistency. Four negative tests reintroduce each defect and require it to be reported, because a checker that never fires carries the appearance of assurance without the fact. Writing those found a flaw in the checker itself: "does the chain cover every step" compared the chain against archive.steps, which is derived by walking the chain. A chain stopping early agreed with itself. It now counts EdfMeasurementStep instances independently. A validator built from the same model as the thing it validates inherits its blind spots, which is how the original defects got past. Also: Instance.Tags on a protocol carries #ContentHash|<sha1 of the Data string>, which is neither the ContentHash column nor the stored blob's hash, and matches all 18 protocols in the reference archive. replace_content recomputes it. A stale one is tolerated on load -- two scans in the NAV option scan shared one and both kept their values -- but it is derivable and describes the content. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds `siemens-protocol-tool exar <archive> <pdf>`: take a template .exar1, write every parameter with a verified mapping, and report the rest. The manifest is as much the point as the output. Roughly a tenth of what a protocol prints is mapped, so a built archive is mostly the template it started from. A tool reporting only its successes would describe a small part of the result and imply the whole, so the report states the fraction, counts inherited values, and names the unmapped parameters by frequency -- which is what says where the next mapping is worth deriving rather than guessing at one. Driving an archive from its *own* PDF must write nothing, and that single check exercises units, scales, the derived basis, sparse arrays and change detection together. It caught two defects. Change detection compared the displayed value, which the Special card does not have -- every ASCCONV-only mapping therefore counted as a write and rewrote content that had not moved. And a printed "0.00" against an assignment a sparse array omits was reported as a change to "0.0" when writing zero into a sparse array in fact removes the line; records now say what is actually stored, for which "(absent)" is a value rather than a gap. patch.resolve now falls back to the label Preview prints. A multi-echo scan prints "TE 1" where a single-echo one prints "TE", and the preview entry is labelled the same way, so resolving through it follows the printout instead of duplicating every spelling in the table. Without it the driver silently skipped TE on exactly the scans that print it differently -- 2 fields out of 1353 wrong, which is the kind of miss that reads as a rounding difference. With it, driving the unmodified archive from the changed protocol's PDF lands on the console's own values in all 1353 mapped fields. Scans are matched to the template by name, and an unmatched one is reported rather than guessed at: the PDF names a sequence by kernel and the archive by sequence file, so there is no reliable way to choose a donor to copy. generate.duplicate_step is there for a caller who knows which scan to copy; the driver will not choose one. The GUI exposes the command through the one description in gui/commands.py, as its own test requires. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two commits: creating scans in an archive, and the driver that writes a parsed
PDF into one.
Creating scans
A scanner accepts archives built this way -- four added scans loaded, kept
their running order, and the one carrying an edit kept its own protocol, 19
distinct protocols in and 19 out. Generation is not blocked by the format.
Getting there cost two defects, and both are why
validate.pyexists.Rejected outright.
EdfProgramContentdescribes the running order infive maps keyed by step id --
LinksFrom,LinksTo,Ranks,RelationsFrom,RelationsTo-- and the first attempt updated one. A steppresent in only some leaves the console unable to build the program, which it
reports by showing the folder tree with no protocols in it. That is
indistinguishable from an archive exported off an empty folder node, which is
how it was first misread.
Loaded, and silently wrong. A protocol and a label each carry
ParentElementIdpointing at their own step, and the console resolves astep's protocol through that reverse link rather than through the step's
Childrenblob. Copies keeping the source's pointer were served the source'sprotocol. Three of four duplicates looked perfect, because they duplicate
content identical to their source by design; only the copy carrying a
deliberate edit could show it, returning its source's TR. A generated object
must differ from its template in some observable way or the test cannot tell
"created correctly" from "aliased to the original".
Both archives were internally coherent throughout -- every reference resolved,
every id unique. So
validate.problems()states rules as relationships thathold in every console-authored archive and re-derives them from the archive
under test. Four negative tests reintroduce each defect and require it to be
reported, because a checker that never fires carries the appearance of
assurance without the fact.
Writing those found a flaw in the checker itself: "does the chain cover every
step" compared the chain against
archive.steps, which is derived by walkingthe chain, so a chain stopping early agreed with itself. It now counts
EdfMeasurementStepinstances independently.The driver
The manifest is as much the point as the output. Roughly a tenth of what a
protocol prints has a verified mapping, so a built archive is mostly the
template it started from. The report states that fraction, counts inherited
values, and names the unmapped parameters by frequency -- which is what says
where the next mapping is worth deriving.
Driving the unmodified archive from the changed protocol's PDF lands on the
console's own values in all 1353 mapped fields.
Driving an archive from its own PDF must write nothing, and that one check
exercises units, scales, the derived basis, sparse arrays and change detection
together. It caught two defects: change detection compared the displayed
value, which the Special card does not have, so every ASCCONV-only mapping
counted as a write and rewrote content that had not moved; and a printed
0.00against an assignment a sparse array omits was reported as changing to0.0, when writing zero into a sparse array in fact removes the line.patch.resolvenow falls back to the labelPreviewprints. A multi-echo scanprints
TE 1where a single-echo one printsTE, so the driver had beensilently skipping TE on exactly the scans that print it differently -- 2 fields
out of 1353, the kind of miss that reads as a rounding difference.
Scans are matched to the template by name and an unmatched one is reported
rather than guessed at: the PDF names a sequence by kernel (
epfid) and thearchive by sequence file (
cmrr_mbep2d_bold), so a donor to copy cannot bechosen reliably.
generate.duplicate_stepis there for a caller who knowswhich scan to copy; the driver will not choose one.
The GUI exposes the command through the single description in
gui/commands.py, as its own test requires.Verification
Not yet done
No archive built end to end by
exarhas been through a loader. The parts areindividually validated -- patched values on five archives, created scans on
one -- but the combination has not been. That is the next cheap test.
Coverage is 10%: 218 of the 2276 parameters an 18-scan protocol prints. The
unmapped list is dominated by console UI settings rather than acquisition
parameters, which is reassuring and worth confirming before deriving more.
🤖 Generated with Claude Code