Skip to content

Add Sanger AB1 support, rename to Raw Data Reader - #5

Merged
mizraelson merged 3 commits into
mainfrom
feat/ab1-raw-data-reader
Oct 8, 2026
Merged

mizraelson merged 3 commits into
mainfrom
feat/ab1-raw-data-reader

Conversation

@mizraelson

@mizraelson mizraelson commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

What

Samples & Data now imports Sanger AB1 datasets (TaggedAb1, platforma-open/samples-and-data#136). This PR lets the block open them and renames it to Raw Data Reader.

  • Dataset picker accepts pl7.app/sequencing/data columns with fileExtension = ab1, alongside fastq/fasta.
  • Trace picker: a sample can hold several traces (e.g. forward and reverse primer). They are listed from the pre-run's fileIndex and told apart by their tag values. The picker appears in the viewer when a sample has more than one trace.
  • Viewer (no Run needed):
    • Chromatogram: the four analyzed dye traces, base calls, and per-base quality bars, at three zoom levels.
    • Sequence: base calls with bases below Q20 dimmed.
    • Summary: length, mean Q, bases ≥ Q20, and run metadata (sample name, instrument, well, basecaller, run start/end).
  • Downloads: the selected trace as the original .ab1 or as FASTQ (base calls + qualities). The existing sample and whole-dataset archive buttons download every original file, as before.

How

  • The pre-run already wraps every dataset file with file.exportFile. The model resolves only the selected trace with getFileHandle(), and the UI fetches its bytes with ReactiveFileContent.getContentBytes. This is the same route mixcr-amplicon-alignment uses for its prerun FASTA.
  • ui/src/ab1.ts is a small ABIF parser for the tags a viewer needs: PBAS2/1, PCON2/1, PLOC2/1, DATA9–12 (raw DATA1–4 as fallback), FWO_1, and the metadata tags. It was checked against Biopython on 177 real 3730 traces: sequence, qualities, peak positions, and all four trace channels match exactly.
  • There is nothing for the main workflow to compute on AB1, so args refuse AB1 datasets. Args can read only data, so the dataset kind is snapshotted into data.inputIsAb1 when the user picks a dataset (ab1Inputs output). main.tpl.tengo also asserts against AB1. The prerun runs regardless of args validity.
  • Software is unchanged (no new package version, no Docker image).

Other changes

  • Title, block meta, description and tags say "Raw Data Reader". Package IDs stay milaboratories.fastq-reader.*, so this stays the same block in the registry.
  • Raw-file archive names drop the synthetic R1 for datasets without a read-index axis: S1_Fwd.ab1, and S1.fasta where it used to be S1_R1.fasta.
  • The raw-download bar and the client-side download/zip helpers moved out of ReadViewer.vue into RawDownloads.vue and download.ts, so both viewers share them.
  • block-tools 2.16.13 and tengo-builder 4.1.11, for the require-latest preflight.

Testing

  • pnpm check: type-check, lint and format pass in all packages.
  • Built locally with build:dev-binary-existing.
  • Not yet run against a backend. The workflow integration test is still a stub (needs an AB1 dataset fixture).

RetriggerConfidence Score: 5/5

The PR appears safe to merge, with all three previous findings addressed and no new actionable issues found.

What we checked:

  • Small files cannot request huge arrays: elements rejects counts that exceed the tag’s checked payload before readShorts allocates the array.
  • Downloads no longer invent qualities: The download writes FASTA and returns when qualities are missing. Only traces with stored qualities produce FASTQ.

Summary

Adds Sanger AB1 viewing and renames the block to Raw Data Reader while keeping its package IDs.

  • Users can pick a sample trace, view its chromatogram or sequence, and download original files without running the block.
  • The latest changes check parser bounds, export FASTA when qualities are missing, and add automated parser tests.
  • All three previous findings are addressed. No new actionable issues were found.
  • Checks were based on code inspection; tests and backend flows were not run.

Important touched types:

  • BlockData: Saved user choices. Adds AB1 dataset detection, the selected trace, and view and zoom settings.
  • Ab1Trace: Parsed base calls, qualities, peak positions, four dye traces, and run details. New in this PR.
  • TraceOption: A trace’s picker value and label. New in this PR.
  • RawFileIndexEntry: A reference to an original file from the pre-run. Its existing extra values now also distinguish AB1 traces.
  • DirEntry: An ABIF tag’s location, count, and size. Adds tag so parser errors identify the bad entry.
Diagram
%%{init: {'theme': 'neutral'}}%%
flowchart TD
  Dataset[Pick dataset] --> PreRun[Pre-run exposes original files]
  PreRun --> Archive[Download sample or dataset]
  PreRun --> Kind{Dataset format}
  Kind -->|AB1| Pick[Pick sample and trace]
  Pick --> Bytes[Load selected file]
  Bytes --> Parse[Parse ABIF]
  Parse --> View[Show chromatogram and sequence]
  Parse --> Quality{Qualities available?}
  Quality -->|Yes| FASTQ[Download FASTQ]
  Quality -->|No| FASTA[Download FASTA]
  Kind -->|FASTQ or FASTA| Run[Run read extraction]
  Run --> Reads[Show selected reads]
Loading

Reviews (2) · Last reviewed commit: "Address review: bound AB1 counts, no inv..." · Reviewed by Greptile

AB1 datasets from Samples & Data (TaggedAb1) are now selectable. A sample's
traces are listed from the pre-run's file index (told apart by their tag
values); the selected trace's bytes are fetched from the pre-run's exported
file and parsed in the UI, so no Run is needed. The viewer shows the four
dye traces with base calls and quality bars, or the base calls as text,
with run metadata and .ab1 / FASTQ downloads of the trace.

Args refuse AB1 datasets (snapshot of the dataset kind taken on pick) and
the main workflow asserts the same. Raw-file downloads drop the synthetic
"R1" from file names of datasets without a read-index axis.

Bump block-tools to 2.16.13 and tengo-builder to 4.1.11 for the
require-latest CI preflight.
Comment thread ui/src/ab1.ts Outdated
Comment thread ui/src/pages/Ab1Viewer.vue Outdated
Comment thread ui/src/ab1.ts
- Check each tag's element count and element size against its payload
  before allocating or reading, so a malformed count can neither request
  a huge array nor read other tags' bytes; reject negative data sizes and
  too-short dates/times/longs. A malformed metadata tag is now skipped
  instead of failing the whole trace.
- Save a trace without per-base qualities as FASTA instead of FASTQ with
  made-up Q40 scores; the button label and sequence view say which.
- Add vitest to the ui package with 20 tests over synthetic ABIF files:
  channel order, edited/original tag fallback, raw-trace fallback, inline
  payloads, offset views, missing or mismatched qualities, metadata, and
  malformed files (bad signature, truncated directory, oversized and
  overlapping counts, wrong element size, out-of-file offsets).
@mizraelson

Copy link
Copy Markdown
Collaborator Author

@greptile review

@mizraelson
mizraelson merged commit 2406add into main Oct 8, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant