Skip to content

merge-backend: zipped output ends as the archive, other rows are an error; version 0.9.1 - #38

Merged
YPARK merged 1 commit into
mainfrom
merge-backend-fixes
Oct 20, 2026
Merged

YPARK merged 1 commit into
mainfrom
merge-backend-fixes

Conversation

@YPARK

@YPARK YPARK commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

Bug fixes to merge-backend, split out of the MCP work so they can ship now.

  • Zipped output. -o m (zip is on by default) left the working m.zarr and never made the m.zarr.zip the help promises: finalize took the bare output instead of the resolved one.
  • Output that is an input. -o pbmc beside pbmc.zarr.zip, or -o a --no-zip beside a.zarr, is now refused before anything is cleared. The output is deleted before the inputs are read and zipped over afterwards, so the first fix would otherwise overwrite the input.
  • Mismatched rows. Backends whose row names differ hit an assert_eq! panic that printed every name. They now get an error naming the two files, raised before anything is written. The first file is the reference even when it has no rows.
  • Batch labels. These go to {output}.batch.gz with the backend suffix stripped; -o m.zarr.zip used to write m.zarr.zip.batch.gz. This applies the same way in merge-backend, merge-mtx and squeeze's merge, which now share the end-of-merge step in one place. A column with no known batch takes the output's whole name (sample.v2).
  • merge-mtx backend. merge-mtx writes the backend the output's name resolves to (an .h5 name, or the zarr fallback when hdf5 isn't built), not the one the flag names.

Behaviour changes to note:

  • Batch file name: the new name above. Pipelines that pass the old {output}.zarr.zip.batch.gz name explicitly will need to update.
  • Explicit .zarr output: -o m.zarr without --no-zip now ends as m.zarr.zip, like every other command.

Tests: new regression tests for the archive, the batch labels, the refusals and mismatched rows. cargo fmt, cargo clippy --all-targets and cargo test are clean.

Left for follow-ups, all pre-existing:

  • union-mode squeeze rejects a bare -o prefix, and only after loading every input;
  • find_aligned_rows counts duplicate rows within one input as common;
  • merge-backend with a single input exits 0 and writes nothing;
  • from_h5ad panics (assert_eq!) on short name arrays;
  • move the merges onto zarr_io::prepare_output/finalize_output.

…rror; version 0.9.1

- `-o m` (zip on by default) left the working m.zarr and never made the
  m.zarr.zip it promised: finalize took the bare output, not the resolved
  one. It now finalizes to the .zarr.zip, as merge-mtx does.
- An output that is one of the inputs (`-o pbmc` beside pbmc.zarr.zip, or
  `-o a --no-zip` beside a.zarr) is refused before anything is cleared:
  the output is deleted before the inputs are read and zipped over its
  name after.
- Backends whose row names differ hit an `assert_eq!` panic that printed
  every name; they are now refused with an error naming the two files,
  before anything is written. The first file is the reference even when
  it has no rows.
- The batch labels go to `{output}.batch.gz` with the output's backend
  suffix stripped (`-o m.zarr.zip` wrote m.zarr.zip.batch.gz), in
  merge-backend, merge-mtx and squeeze's merge alike: the three share
  the end of a merge, labelling the columns and finalizing, in one place,
  from the resolved output only. A column with no known batch takes the
  output's whole name (`sample.v2`), no longer cut at its last dot.
- merge-mtx writes the backend the output's name resolves to (an .h5
  name, or the zarr fallback when hdf5 is not built), not the flag's.
@YPARK
YPARK merged commit 1fda45a into main Oct 20, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant