LoginSign Up
← Back to all sparks
Z

zarr-python

DEVOPS
Velocity6.3

Chunked compressed N-dimensional arrays

github.com/zarr-developers/zarr-python ↗

Zarr is splitting into packages — and just gave its arrays an HTTP front door.

package splithttp servingchunked arraysperformancezarr v3
◆Current state
Zarr-python is mid-decomposition: the v3 monolith is spawning independently versioned siblings — zarr-metadata, zarr-indexing, and now zarr-http-server — each cut on its own tag. The 3.2 line carries the performance work in parallel: a full-shard write fast path, an oindex optimization, and experimental rectilinear chunks. Because the feed mixes package tags with core releases, the version string alone tells you almost nothing about what shipped.
◆Where it's heading
The split points toward Zarr as a set of composable pieces rather than one library, with metadata parsing, index transforms, and network serving each usable on their own. The HTTP server is the most consequential of the three: it makes a store addressable over the wire instead of requiring every client to mount object storage itself. Expect the core package to keep shedding responsibilities to these satellites as each reaches a usable version.
◆Prediction
The next tags are likely follow-on releases of the satellite packages, with zarr-http-server moving past 0.1.0 as range-request and access-control behavior get exercised, and the 3.2 line converting its release candidate into a final.

◆Recent moves

  1. 4h ago

    zarr_http_server-v0.1.0: HTTP server that exposes stores, arrays, groups (#3732)

    ⚡ SPARK

    The first release of a standalone zarr-http-server package, exposing stores, arrays, and groups over HTTP — the point where the package-split arc stops being a refactor and produces a surface Zarr did not have. The same tag corrects byte-order handling for structured dtypes in the bytes codec, which had been silently corrupting data whose field byte order differed from the stored layout.

    View source ↗
  2. 12d ago

    zarr-indexing 0.1.0: TensorStore-style index transforms as a standalone package

    A new zarr-indexing package lands with TensorStore-style index transforms, extracting indexing logic into something usable without pulling in the whole of zarr-python. It is the second of the satellite packages and reinforces the decomposition the HTTP server later completes.

    View source ↗
  3. 13d ago

    zarr-metadata 0.4.0: model-layer changes and a standalone docs site

    The 0.4.0 cut of zarr-metadata folds in the model layer's feature and removal notes plus a standalone documentation site, so the metadata package now has its own docs alongside its own version. Small in content, but it is what an independently maintained package looks like once the split is real.

    View source ↗
  4. 2mo ago

    zarr_metadata-v0.2.0: Widen ChunksLike type alias (#3990)

    Widens the ChunksLike type alias and swaps an if/else expression for a statement — type-surface housekeeping in the metadata package with no behavior change for callers.

    View source ↗
  5. 3mo ago

    3.2.0rc1: experimental rectilinear chunks and a full-shard write fast path

    The 3.2 release candidate is where the core library's performance work sits: a fast path for full-shard writes, an oindex optimization, chunk-transform work, and experimental support for rectilinear (variable-sized) chunks, alongside a bump to Python 3.12 as the minimum. Variable-sized chunks are the notable one — they loosen a constraint the format has carried since v2 — but this is still a candidate, not a final.

    View source ↗