[VibhuAg]

Summary

This PR adds a new command-line tool, zarr-to-b2nd, for converting Zarr arrays to Blosc2 .b2nd files. I created a file for utility functions shared between the b2nd-to-zarr and zarr-to-b2nd tools. The new tool works by determining the shape of the input in terms of the chunks and shards (if they exist) used in the Zarr dataset and its compression settings. Then data is copied in batches, sized by --buffer-size. When shards are present in the Zarr dataset, shards become b2nd chunks and inner chunks become b2nd blocks. When shards are not present, Zarr chunks become b2nd chunks and the blocks are chosen by Blosc2.

Closes #721. I plan to write docs in a separate PR for #714.

Changes

• Shared helpers module in conversion_utils.py
• b2nd_to_zarr.py now uses the helper module with no behaviour change
• New zarr_to_b2nd command registered in pyproject.toml
• New tests for zarr_to_b2nd.py
• All options are validated before an existing output is deleted with --force

Compression handling

• Zarr compressed with Blosc -> same codec, level, and shuffle
• Zarr compressed with Zstd -> Level 0 -> clevel 5 and levels above 9 -> 9 since zstd goes up to level 22 and Blosc2 stops at 9
• Zarr compressed with Gzip -> Output gets zlib since Blosc2 has no gzip and zlib is the closest equivalent
• --codec / --clevel / --shuffle override all of the above
• If a Zarr Blosc array uses a codec Blosc2 lacks, the tool falls back to zstd. But if the user types an unknown codec it's an error.

Testing

• Added a new test file at tests/test_zarr_to_b2nd.py. 26 tests total. Converter tests pass, and so does the full suite.

I tested the utility on the 2-metre air temperature dataset from NOAA published as Zarr on AWS. It is a Zarr v2 dataset compressed with Blosc (lz4, level 9, shuffle) and carries attributes such as units: "K" and long_name.

I downloaded the dataset with the following commands:

mkdir -p TMP.zarr
B=https://hrrrzarr.s3.amazonaws.com/sfc/20210601/20210601_00z_anl.zarr/2m_above_ground/TMP/2m_above_ground/TMP
curl -sf -o TMP.zarr/.zarray "$B/.zarray"
curl -sf -o TMP.zarr/.zattrs "$B/.zattrs"
for i in $(seq 0 7); do for j in $(seq 0 11); do curl -sf -o "TMP.zarr/$i.$j" "$B/$i.$j"; done; done

I tested the conversion utility with the following command and got these results:

$ zarr-to-b2nd TMP.zarr --full-verify
Converting zarr -> b2nd... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% ? 0:00:00 0:00:00
Performing full verification across all slices...
✓ Full element verification passed!

                       Conversion Summary
 Property            Value
 Source              /Users/vagrawal118/hrrr/TMP.zarr (1.16 MB)
 Destination         /Users/vagrawal118/hrrr/TMP.b2nd (1.15 MB)
 Shape / Dtype       (1059, 1799) / float16
 Chunks              (150, 150)
 Uncompressed Size   3.63 MB
 Source Compression  3.14x
 Blosc2 Compression  3.15x
 Conversion Speed    51.6 MB/s (0.07 s)
 Total Time          0.10 s

Not included

• Remote urls are not supported as input yet.

AI tool usage

I used AI to make edits of the code and tests after I wrote an initial version. I have read through the code and understand how it works.

[FrancescAlted]

I like this PR. It is well-designed, effectively eliminates duplicate code between b2nd_to_zarr.py and zarr_to_b2nd.py, and maps Zarr shards →
b2nd chunks and Zarr inner chunks → b2nd blocks cleanly (great!). The test coverage is comprehensive too.

Some requests for improvements:

1. Add zarr-to-blosc2 Alias in pyproject.toml
2. Include Blocks in print_summary() (both in rich and text outputs)
3. Defensive Attribute Access on Zarr Arrays. In Zarr v2 array objects (e.g. if opened with older Zarr versions or custom store wrappers), .shards does not exist and raises AttributeError. Similarly, Zarr v2 uses .compressor (singular) rather than .compressors.
4. Please use type annotation on output_path in zarr_to_b2nd (like output_path: str | Path | None = None).
5. Warn on dropped attributes in verbose mode. Silently suppressing exceptions can lead to unexpected metadata loss (e.g. unstorable types). Suggestion: pass verbose: bool = False to copy_attrs() and emit a notification if an attribute cannot be stored.

Keep the good work!