# Release notes

## Changes from 4.10.1 to 4.11.0

Nullability in `CTable` is rebuilt on Arrow's own model. A nullable column now
keeps its nulls in a per-column validity sidecar instead of reserving a value
from its own range, which is what makes it lossless — an `int8` column can hold
`-128`, a `utf8` one can hold `""`, and a `float64` one can tell `NaN` from
missing. That is the new default for columns created from now on; nothing on
disk changes, and sentinel storage remains supported indefinitely, one keyword
away. Built on top of it: predicates follow three-valued (Kleene) logic, so
`~(t.price > 10)` no longer returns the null rows, and column indexes are
null-aware, which makes `min`/`max` over a nullable column answer from the
index instead of scanning.

On the packaging side, wheels are now a single Stable ABI (abi3) build per
platform covering CPython 3.11+, with free-threaded 3.14 and 3.15 shipping
alongside.

### New features

#### Mask-based nullable columns for CTable, and they are now the default

- **A nullable column keeps its nulls in a per-column validity sidecar** —
  Arrow's own model — instead of reserving a value from its own range. This is
  what a bare `nullable=True` resolves to, and what every nullable column
  inferred from Arrow, Parquet or CSV gets: `blosc2.bool(nullable=True)` no
  longer reserves `255` and keeps `np.bool_`, `blosc2.int8(nullable=True)` has
  all 256 values usable, `blosc2.utf8(nullable=True)` accepts any string
  including `""` and `"\x00"`, and `blosc2.complex128(nullable=True)` is
  nullable at all for the first time.
- **This is what makes nullability lossless.** A sentinel steals a value from
  the dtype, so a nullable `int8` could not hold `-128`, a free-text `utf8`
  column had no safe sentinel at all, and Arrow columns whose type had no value
  to spare could not be imported. `to_arrow(from_arrow(x))` now returns `x` for
  nullable `bool`, full-range `int8`/`uint8`, `float64` containing
  `nan`/`±inf`/`-0.0` as values, `utf8` containing `""` and
  `"__BLOSC2_NULL__"`, and `timestamp` with `int64.min` as a value — none of
  which round-trip through a sentinel.
- **`None` is how you write a null** under mask storage (`t.append((None,))`,
  `t["price"][3] = None`), which a fixed-width sentinel column cannot accept at
  all. `is_null()` is unchanged and remains the uniform API across every kind.
- **Nothing on disk changes.** The new default governs *creation* only: opening
  a stored table never re-resolves anything, so every existing table keeps the
  storage, dtype and sentinel it was written with, and every rewrite rule for
  the reserved `255` stays permanently in place.
- **Sentinel storage is supported indefinitely and is one keyword away**, per
  column (`null_storage="sentinel"`, or any explicit `null_value=`) or globally
  through `NullPolicy`. Setting a type-wide `NullPolicy` sentinel field still
  implies sentinel storage for the kinds it covers, so existing
  `NullPolicy(float_value=...)` code is unaffected — with one unavoidable
  exception: `255` is the only value a nullable bool may reserve, so it is also
  `bool_value`'s default, and `NullPolicy(bool_value=255)` carries no
  information to act on. A bool column that wants a sentinel has to say so with
  `null_storage` or `column_null_values`.
- **A table containing a mask column records schema version 3.** Only such
  tables do: a table with no nullable column still records version 1, exactly
  as before, and a sentinel one does too. Readers older than 4.11.0 refuse a
  version-3 table rather than misreading it, but their message is a bare
  `ValueError: Unsupported schema version 3` — the hint naming
  `convert_nulls(to='sentinel')` ships in 4.11.0, so only readers that can
  already open the file will print it.
- **If some of your data has to stay readable by an earlier release, pin the
  storage** rather than discovering this downstream. Per column as above, or
  process-wide — including for schemas inferred from Arrow, Parquet and CSV —
  with `blosc2.null_policy(blosc2.NullPolicy(null_storage="sentinel"))`. That
  reinstates the sentinel's lossiness (a float column's nulls become `NaN`
  again, and a type with no value to spare still cannot be imported), which is
  the trade being made.
- **`Column.null_storage` reports where a column keeps its nulls** and `info`
  tags each column (`int64 nullable[mask]`), so `CTable.convert_nulls()` can
  move columns between the two in either direction — never implicitly, and
  refusing rather than silently relabelling data when a sentinel is
  unavailable.
- **One deliberate semantic difference:** in a mask column `NaN` is a *value*,
  following Arrow, and only the sidecar marks a null. Sentinel float columns
  keep NaN-as-null. See "Where nulls are stored" in the CTable reference.

#### Column indexes are null-aware

- **Per-segment `min`/`max` are taken over the rows that carry a value.** A
  column's nulls are read from its validity channel and left out, and a segment
  with no value at all is flagged rather than summarised. This applies to both
  storages — an `INT64_MIN` sentinel is exactly as invisible to a summary as a
  mask column's fill.
- **`Column.min`/`Column.max` answer from the index for a nullable column**
  (**236x** on a 20M-row `int64`, measured), where before every nullable column
  but a NaN-sentinel float had to scan.
- **`where()` with an `OR` over a nullable indexed column uses the index**
  instead of falling back to a full scan (**1.6x** on a 20M-row two-column
  probe). The fallback existed because the only null filtering available was
  global, and a global filter drops a row that is null in one branch but
  matches the other; the segment path never needed it, because it *evaluates*
  the predicate, which has been null-aware per leaf since the string-predicate
  fix below.
- **Indexes written by an earlier release are read as not null-aware** and keep
  the old fallback, so nothing silently changes meaning; `rebuild_index()`
  promotes them.
- **Building an index over a nullable column that holds nulls costs one
  decompression pass** (33 ms for a 20M-row `int64` column), because the
  incremental per-block summaries folded during writes carry no validity; a
  nullable column with no nulls keeps that fast path untouched.

#### Predicates over nulls follow three-valued (Kleene) logic

- **A comparison against a null is now unknown**, the third value SQL and Arrow
  both use, and `&`, `|`, `^` and `~` combine it by Kleene's rules instead of
  collapsing it to `False` at the leaf. `t.where(t.price > 10)` gives the rows
  definitely above 10, `t.where(~(t.price > 10))` the rows definitely *not*
  above 10 — nulls in neither.
- **`where()` keeps what a predicate is true for**, so the rows it returns for
  a plain comparison are unchanged. What this fixes is everything built on top
  of one: `~(t.price > 10)` used to invert a null that had already been
  collapsed to `False` and so returned every null row — the exact opposite of
  the intent — and `~((a > 10) & (b == 999))` dropped rows that qualify,
  because `unknown & false` is *false*, not unknown, and only a real third
  value can express that. Both query forms are covered and both now agree with
  SQL: the string form carries the second channel through an AST rewrite under
  negation.
- **A predicate can be asked about its unknown rows** rather than only filtered
  with: `p.is_null()` gives the rows it cannot answer for, `p.null_count()`
  counts them, and `t.where(p.fillna(True))` keeps what cannot be ruled out.
  `fillna(False)` is the other reading, and is what `where()` applies
  implicitly.
- **Predicates over non-nullable columns are untouched and cost nothing
  extra**; the result of a nullable comparison is still a `blosc2.LazyExpr`, so
  it computes, indexes and plans exactly as before. Measured cost of the exact
  answer: a negated two-column conjunction over a 20M-row nullable table runs
  1.15x slower than the wrong answer it replaces; every other predicate shape
  is unchanged.
- **Two consequences worth knowing.** `t.where(dict_col != "x")` no longer
  returns the rows where `dict_col` is null (its reserved code differs from
  every value's, so it used to match); `dict_col == None` remains how to ask
  for them. And `Column.isin()` stays deliberately two-valued — it returns a
  materialized array and has its own spelling for nulls (`None` among the
  values).

#### Packaging and other changes

- **Wheels are a single Stable ABI (abi3) build per platform.** Instead of one
  wheel per CPython minor version, each platform ships one `cp311-abi3` wheel
  that serves CPython 3.11 and every later version, including ones released
  after this one. Nothing changes for `pip install blosc2`; what changes is
  that a new CPython no longer has to wait for a blosc2 release to be
  installable from a wheel. CI installs that one wheel on 3.11 through 3.15 and
  runs a slice of the suite on each, since cibuildwheel only tests a wheel on
  the interpreter that built it.
- **Free-threaded builds are shipped too**, as version-specific `cp314t` and
  `cp315t` wheels — the free-threaded stable ABI (abi3t, PEP 803) starts at
  3.15 and Cython cannot emit it yet. The limited API costs nothing measurable
  here: across the Linux and Windows cells of a throwaway benchmark matrix the
  worst ratio over every benchmark was 1.075x and 1.054x respectively,
  including the call-heavy ones where an ABI cost would show up first.
- **CSV reads and writes take an `encoding=`.** `from_csv()` defaults to
  `utf-8-sig` (plain UTF-8, absorbing a byte-order mark if present) and
  `to_csv()` to `utf-8`, but either can be given another codec, which is what
  an existing file written in a platform codec needs.
- **Three new [optimization tips](https://www.blosc.org/python-blosc2/guides/optimization_tips.html)**:
  `utf8` versus fixed-width string columns, when a `dictionary` column pays
  off, and what the `@jit` decorator actually buys you (with a worked
  Mandelbrot benchmark, including the compilation cost).

### Bug fixes

- **`asarray()` corrupted arrays whose chunks overhang the shape.** Above 16 MB
  the fill goes chunk by chunk through `SChunk.update_data()`, and its guard
  only rejected partitions *smaller* than their container — so a chunk sticking
  out of the shape (e.g. shape `(146, 23802)` with chunks `(147, 23802)`) passed
  and got a slice one row short, after which reading the array back failed with
  "Error while getting the buffer". Thanks to @Zentrik.
- **`group_by` returned the wrong `min` for a `bool` value column.** The
  per-group accumulator was seeded from the dtype's opposite identity, and `bool`
  had none, so an all-`True` group reduced to `False`. Reachable with any plain
  non-nullable bool column on the generic aggregation path.
- **Descending `sort_by` on a `bool` column raised**, and on a signed-integer
  column holding its dtype's minimum (`-128` for `int8`) that row sorted as if it
  were the largest. The descending key negated in the column's own dtype, where
  `bool` has no unary minus and a narrow signed type wraps.
- **`add_column()` after `copy()` backfilled one row short**, and raised for a
  variable-length column: the copy recorded its write watermark one below the
  convention every other writer follows.
- **A string predicate over a nullable column returned its nulls as matches.**
  `t.where("a > 10")` compared the stored sentinel, so any sentinel satisfying
  the predicate (`null_value=999` against `> 10`) came back as a match. The
  operator form (`t.where(t.a > 10)`) was always correct. Fixed for both storages.
- **A nullable `uint8` ndarray column came back as `bool`.** The `bool → uint8`
  widening that sentinel storage needs was undone by dtype rather than by
  whether it had been applied, so a column declared `uint8` was truncated to
  flags.
- **`~` on a nullable bool column selected its nulls.** SQL `WHERE` semantics
  say a null satisfies neither a predicate nor its negation; the mask path
  inverted the stored `False` fill instead. (The sentinel path was already
  correct, via its `== 0` rewrite.)
- **CSV import and export ignored a validity sidecar.** `to_csv` compared
  against the sentinel to find nulls, so a mask column wrote its fill as if it
  were data, and `from_csv` had nothing to put in an empty field and raised.
  Both go through the sidecar now: an empty CSV field is a null in either
  direction. Sentinel columns keep writing their sentinel, unchanged.
- **Reductions on a sorted view of a mask column read the wrong rows.** The
  null flags were gathered in the view's order and the values in physical
  order, so `sum()` on a sorted view could return `NaN` from a column whose
  nulls are not `NaN`, and `unique()` could report the fill as data while
  dropping a real value. Sentinel columns were unaffected.
- **`convert_nulls()` flattened a nested column**, even when it had nothing to
  convert: the schema copy it makes dropped both the table metadata and the
  logical parent of a nested group, so a struct column came back as its leaves.
  An in-place conversion on a persistent table wrote that flattened schema to
  disk. Saving and reopening a nested table dropped the same parent, which is
  fixed alongside it.
- **Descending `sort_by` mis-ordered the widest integers.** The key was built
  by negating, and negation has a fixed point: `int64`'s minimum sorted as if
  it were the largest, and a `uint64` above 2**63 wrapped negative and sorted
  below small values.
- **CSV was written and read in the platform's locale encoding**, so text a
  column can hold but cp1252 cannot encode — anything outside Latin-1 — raised
  `UnicodeEncodeError` on Windows. Both directions are UTF-8 now, and reading
  absorbs a byte-order mark if one is present.
- **A timestamp column could not be written through `col[key] = value`.** A
  `datetime` was never encoded to the stored `int64`, so every key form failed;
  `extend()` was unaffected. ISO strings and `datetime64` are accepted too.
- **Assigning one value to many rows raised on a mask-storage column.**
  `col[0:2] = 7` — and the same write through a boolean mask or an index list —
  failed with `TypeError: iteration over a 0-d array`, because the write path
  looked for nulls *inside* a value that was a single cell rather than a batch.
  Scalar broadcast works again, and `col[0:2] = None` now makes every selected
  row null.
- **`extend()` from another table lost nulls between storages.** Copying rows
  from a mask-backed column into a sentinel-backed one (or the reverse, or
  between two sentinels reserving different values) wrote whatever stood in for
  the null as real data. Nullity is translated now.
- **A row read showed a mask column's fill instead of `None`.** `t[i]`,
  iteration and `repr` surfaced the placeholder that occupies a null slot, which
  is not part of the format contract — and disagreed with a `vlstring` column in
  the same row, which already read `None`. Sentinel columns still show their
  sentinel, which is the value you chose.
- **`to_numpy(masked=True)` and `dropna()` raised on a nullable dictionary
  column**, which reported its nulls per physical slot rather than per live row.
- **A null in a complex column reached pandas as `nan+0j`** rather than as
  missing.
- **A nullable `ndarray` of `bool` forgot it had been widened when reopened**, so
  converting it to mask storage left it `uint8`, and the guard against a
  dtype-changing in-place conversion on a persistent table stopped firing.
- **`convert_nulls(to="sentinel")` refused over a value in a deleted row.** The
  collision check scanned physical slots, so a proposed sentinel present only in
  a row already deleted — unreadable, and dropped by the next `compact()` —
  blocked the conversion. Only live rows are consulted now, and the row named in
  the refusal is the logical one the caller can index rather than a physical
  slot.


## Changes from 4.10.0 to 4.10.1

A correctness release: lazy indexing and reductions now follow NumPy in a
batch of cases where they quietly did not, the stores close several
cross-process read races, and wheels finally ship usable C-Blosc2 development
files.  Bundled C-Blosc2 moves to 3.3.2.

### Bug fixes

#### Lazy expressions and indexing

- **Indexing a lazy expression with an integer squeezed too much.**
  `expr[0]` dropped *every* length-1 axis of the result, including ones the
  index kept, so a `(1, 4)` expression indexed at `[0]` came back as `(4,)`
  where NumPy gives `(4,)` only for the consumed axis and keeps the rest.
  Only the dimensions the integer indices actually consumed are dropped now.
  `where()` results, whose length is data-dependent, are left alone.
  Closes #319.
- **`LazyUDF` and broadcast operands mis-indexed on `None`.** A `None` in the
  key inserts an axis in the result but consumes none in the operand;
  aligning it as if it did shifted every axis to its left by one, so
  `expr[None, 2]` read the wrong operand region. Closes #403, #688.
- **Indexing a full reduction materialized the whole operand.** `(a + b).sum()
  [key]` evaluated the reduction over everything and then indexed; the
  operands are sliced first now. Closes #457.
- **Datetime comparisons in lazy expressions raised.** numexpr has no datetime
  type, so `t1 < t2` on `datetime64`/`timedelta64` died with `unknown type
  datetime64[s]` and the error was re-raised rather than letting the NumPy
  fallback try. They are compared as their underlying int64 counts, which is
  exact, keeping the fast path. Mixed units and `NaT` decline that route and
  fall back to NumPy, since raw counts would silently lie about both.
  Closes #409.
- **`NDArray.nbytes` reported the padded size.** It now returns the logical
  `size * itemsize`, matching NumPy, whenever the shape does not fill the
  chunk grid exactly. `cratio` still measures the stored (padded) data, so
  `nbytes / cbytes` need not equal `cratio`; use `.schunk.nbytes` for the
  padded figure. Closes #544.

#### Tables and stores

- **`CTable.where()` applied a short boolean mask to the wrong rows.** A mask
  shorter than the live-row count was padded out to the *physical* length,
  which aligned it with the underlying column and selected rows outside the
  view. A mask no longer than the live-row count is now treated as logical —
  entry *i* selects the *i*-th live row — with a short one simply leaving the
  trailing rows unselected. Closes #607.
- **Cross-process read races in `EmbedStore` and `DictStore`.** Both resolved
  a key under the store lock but read the data after releasing it, so a
  concurrent writer could be caught mid-mutation: `EmbedStore.__getitem__`
  returned bytes from a stale offset, and `DictStore.__getitem__` opened an
  external leaf that a concurrent overwrite had just removed or half-rewritten
  (`KeyError`, or `RuntimeError: Error while getting the buffer`). The
  resolve, the existence check and the open now share one lock; non-shared
  stores skip it entirely. Fixes #691, #692.
- **Overwriting an external `DictStore` leaf is now atomic.** `__setitem__`
  removed the old leaf and rebuilt it at the same path, and the handle
  `__getitem__` returns holds no file descriptor — the C layer re-opens the
  leaf by path for every chunk it decompresses — so a read already in flight
  could open a truncated file. The new leaf is built beside its final name and
  moved in with a single `os.replace()`, so every such re-open sees one
  complete cframe or the other. A crash mid-write now leaves a stray `.tmp`
  staging file rather than a partial leaf.
- **`TreeStore.close()` swallowed inline handle failures.** A `CTable` that
  failed to flush left an archive whose row count disagreed with a varlen
  column, reported only on read, long after `close()` said it succeeded. Every
  handle still gets a chance to close and the store is still packed; the
  failure is then re-raised.

#### Other

- **`unpack_tensor()` turned padding into a phantom column.** `np.dtype()`
  renames the empty-named padding fields of a structured descr (`''` -> `f2`),
  so a packed tensor with padding came back with an extra field. Closes #287.

### Packaging

- **Wheels ship usable C-Blosc2 development files.** Install paths are now
  relative to `CMAKE_INSTALL_PREFIX`; the absolute ones made C-Blosc2 generate
  a `blosc2.pc` and exported targets pointing into the build tempdir, so
  `pkg-config` and `find_package(Blosc2)` both failed against an installed
  wheel. Verified by building and linking a C program against a wheel three
  ways: `pkg-config`, `Blosc2::blosc2_shared` and `Blosc2::blosc2_static`.
  miniexpr's license texts are mirrored into `.dist-info/licenses`, where PEP
  639 tooling looks. Closes #627.
- **Wheels carry two copies of `libblosc2` instead of three**, ~1.5 MB
  smaller. C-Blosc2 set both `VERSION` and `SOVERSION`, and scikit-build-core
  follows symlinks, so the fully versioned file — referenced by nothing but
  the symlinks pointing at it — was shipped as a third full copy.

### Development

- The test suite runs in parallel by default (`-n auto --dist loadfile` in
  `pytest.ini`), 130s -> 35s locally, falling back to a serial run when
  pytest-xdist is absent. The `heavy` tests, 58% of everything collected and
  excluded from every push-time job, now run in a nightly workflow. The
  network tests run once per push on a single Linux job instead of five times.
- The ruff rule set is spelled out with `select` rather than `extend-select`,
  so a ruff release widening its defaults no longer redefines what CI
  enforces.

## Changes from 4.9.1 to 4.10.0

This release is the string-support milestone: string expressions and DSL
kernels now run on miniexpr, `utf8()` and `dictionary()` columns gain full
indexing, comparisons, and a documented conversion pair, and NumPy's
`StringDType` is understood by the array constructors. Alongside, slicing
with plain keys is up to 1.7x faster, a new `blosc2.random` module provides
chunk-parallel NumPy-quality random constructors, and the optimization-tips
guide gained two new tips and refreshed figures.

### New features

- **String-valued expressions and DSL kernels** over fixed-width `<Un`
  arrays now run on miniexpr instead of falling back to NumPy.
  Concatenation (`arr + "suffix"`, `"prefix=" + arr`) plus `lower`,
  `upper`, `strip`/`lstrip`/`rstrip`, `removeprefix`, `removesuffix`,
  `replace`, `substr` and `split_part` all produce string results, and
  `@blosc2.dsl_kernel` accepts method syntax (`name.lower()`) and tuple
  unpacking (`before, after = desc.split(sep, 1)`), which are rewritten to
  the DSL grammar. The output width is inferred by miniexpr and the
  container is allocated from it, so nothing truncates — `.dtype` may be
  wider than NumPy's exact answer, never narrower.
- **Bytes (`S`) arrays** go through the same engine, with NumPy's `S`
  semantics rather than `<U`'s: ASCII-only case mapping (so `upper`/`lower`
  keep the width instead of growing) and ASCII-only stripping. `S` and `<U`
  operands do not mix in one expression, which is what NumPy does too.
  Variable-width `utf8()` columns still use the NumPy path.
- **String expressions now work on `utf8()` columns.** `t.where("name ==
  'x'")`, `startswith`/`endswith`/`contains` and mixed predicates such as
  `t.where("(name == 'b') | (x > 2)")` used to raise `NotImplementedError`;
  only the operator form `t[t.name == "x"]` was available. A variable-length
  column cannot be an expression operand (its offsets and data have
  independent chunk grids, so the prefilter contract does not apply), so
  these are evaluated span by span, each span materialized to a fixed-width
  array whose width is rounded up to a power of two and handed to miniexpr.
  Nulls are materialized to `""` before any kernel sees them and re-masked
  afterwards, so a null never satisfies a predicate — the same answer the
  operator form gives.
- **Scalar comparisons on `utf8()` columns are 5-6x faster in expression
  form.** `t.where("name == 'x'")` (and `!=`, `<`, `<=`, `>`, `>=`, either
  operand order) is now answered by the same raw-byte scan the operator form
  `t[t.name == "x"]` uses, instead of decoding the column to fixed-width
  first: 156 -> 28 ms over 1M short values, 268 -> 56 ms over 1M ~31-byte
  values. Mixed expressions get whatever they can -- in
  `startswith(name, 'x') | (name == 'zz')` the comparison takes the fast
  path and `startswith` still decodes.
- **New `blosc2.utf8_array(seq, spec=None)`** builds a `UTF8Array` from an
  iterable of strings; `UTF8Array` is exported too. Previously the only
  construction path was `UTF8Array(spec)` + `.extend()` + `.flush()`, which
  was not exported at all.
- **`df.apply(f, axis=1, engine=blosc2.jit)` now runs `row["colname"]`
  kernels that contain an `if`.** Neither dispatch route could before:
  tracing evaluated the branch over a whole column (`truth value ... is
  ambiguous`) and the DSL parser rejected the subscript. Such references are
  now rewritten into named parameters, so the function is compiled and every
  branch runs. This is not string-specific — numeric row kernels with a
  branch were equally blocked. String columns reach this route too, which
  makes the pandas-3 "format room info" kernel run unmodified. Nulls in a
  string column are rejected rather than substituted, since a row-wise kernel
  over a null raises in pandas as well.
- New `blosc2.random` module: seedable, NumPy-quality random `NDArray`
  constructors. Each chunk gets its own independent `SeedSequence`-spawned
  stream and is generated concurrently in a thread pool, giving full `PCG64`
  quality with genuinely parallel generation (measured ~3x faster than
  `asarray(np.random.default_rng(...).random(...))` on a 100M-element array).
  Covers 42 of `numpy.random.Generator`'s 43 public methods (full
  compatibility table in `doc/reference/random.rst`):
    - Core: `random`, `integers`, `normal`, `uniform`, `choice`
      (`replace=True` only).
    - 30 scalar distributions: `beta`, `binomial`, `chisquare`,
      `exponential`, `f`, `gamma`, `geometric`, `gumbel`, `hypergeometric`,
      `laplace`, `logistic`, `lognormal`, `logseries`, `negative_binomial`,
      `noncentral_chisquare`, `noncentral_f`, `pareto`, `poisson`, `power`,
      `rayleigh`, `standard_cauchy`, `standard_exponential`,
      `standard_gamma`, `standard_normal`, `standard_t`, `triangular`,
      `vonmises`, `wald`, `weibull`, `zipf`.
    - 4 vector-valued distributions (output shape `shape + (k,)`, one draw
      per trailing vector): `dirichlet`, `multinomial`,
      `multivariate_hypergeometric`, `multivariate_normal`.
    - `permutation`, `permuted`, `shuffle`: unlike the rest of the module,
      these are not chunk-parallel — whole-array shuffling is inherently
      sequential, so they materialize the full array and shuffle it
      single-threaded. `shuffle` additionally requires its argument to
      already be an `NDArray`, since it mutates in place and returns `None`,
      matching numpy.
    - Not implemented: `bytes` (returns raw `bytes`, not an `NDArray`).

- **`create_index()` now works on `utf8()` columns**, the last string flavour
  without one. Both `utf8` and `dictionary` are indexed by the *alphabetical
  rank* of each value: sorting by rank is sorting by the decoded string, so an
  `int32` rank column drives the same machinery a numeric column uses. At 1M
  rows / cardinality 20k: `sort_by` 424 ms -> 7.2 ms, `sorted_slice` 458 ms ->
  43 ms, and the index is the cheapest of the three flavours to build (277 ms
  against 867 ms for `<U`). Scalar comparisons are served from it too — utf8
  `==` 29.0 ms -> 5.5 ms, `<` 34.6 ms -> 5.5 ms, and the dictionary operator
  form `t[t.c == v]` 329.6 ms -> 8.4 ms. `startswith`/substring searches are
  not accelerated (no index covers them), and ranks are frozen at build time,
  so a value inserted ahead of existing ones sends the index stale until it is
  rebuilt.

- **`CTable.add_column()` accepts `values=`**, a sequence with one entry per
  live row, as an alternative to backfilling from a declared default. This is
  the supported way to land a result computed outside the table back into it,
  which matters most for `utf8()` columns: string-returning expressions are
  evaluated on fixed-width arrays, and the result previously had to be written
  through the private `t._cols[name].set_all(...)`. A declared default is still
  honoured for rows appended later, so the two can be combined. `values=` is
  checked against the constraints declared on the spec, like the constructor
  and `extend()` are: without that, coercion to a fixed-width dtype would
  truncate an over-long string to `max_length` instead of complaining.
- **`blosc2.from_utf8()` / `blosc2.to_utf8()` and `UTF8Array.astype()`** make
  the conversion between variable-length and fixed-width text an explicit,
  documented pair. utf8 columns store and filter text compactly, but
  string-*returning* expressions need miniexpr's compile-time output width, so
  they run on fixed-width arrays; the rule is now written down (see "Computing
  strings on a utf8 column" in the CTable reference) rather than left for
  callers to discover. `from_utf8()` sizes the result to the longest value in
  **codepoints**, counted from the raw bytes without decoding a row, so nothing
  truncates and non-ASCII text does not over-allocate the 3-4x a byte-length
  bound would.
- **The array constructors dispatch on NumPy's `StringDType`.**
  `blosc2.asarray(np.array([...], dtype=StringDType()))` used to raise
  `TypeError: data type 'StringDType()' not understood`, and
  `blosc2.zeros(n, dtype=StringDType())` a `malformed node` `ValueError`; both
  now return a `UTF8Array`, as do `empty`, `ones` and `full`, with the same
  fill values NumPy uses (`''`, `''`, `'1'`, `str(fill_value)`). The dispatch
  is on the *target* dtype, so `asarray(utf8_source, dtype="<U8")` still gives
  a fixed-width NDArray. `StringDType` still cannot back an NDArray — it keeps
  each row's payload outside the array buffer and offers no buffer protocol,
  so compressing that buffer would persist pointers — which is why the
  variable-length container is what comes back.
- **`UTF8Array` gained `.shape`, `.ndim`, `.size` and `__array__`**, so it now
  satisfies the `blosc2.Array` protocol (`.shape` was the only member it
  lacked) and `np.asarray(arr)` returns `StringDType` instead of silently
  widening to a fixed-width `<Un` — for 200-character values that was 1600
  bytes where the payload is 203, and a different dtype than `arr[:]` reported
  for the same object.
- **`Column.assign()` works on utf8, vlstring, vlbytes, struct and object
  columns.** It previously raised `TypeError: UTF8Array assignment index must
  be int`, leaving no public way to overwrite a variable-length column's
  values. These are now rewritten whole (one write per backing batch) rather
  than row by row, which for the batched varlen columns would have rewritten a
  whole batch per row.
- **`@blosc2.jit` dispatches control flow to the DSL, and the DSL engine
  widened its operand and function coverage.** miniexpr's prefilter now
  gathers blocks directly from raw NumPy buffers instead of converting
  operands with `asarray()`, and `Series` are accepted as DSL operands.
  `np.foo(...)` calls inside a jitted function are rewritten to the bare
  names miniexpr recognizes, `np.sign` is supported, and `np.square`,
  `np.negative`, `np.positive` and `np.reciprocal` are dispatched. A
  configured `blosc2.jit(...)` is accepted as a pandas engine.
- C-Blosc2 bumped to 3.3.1 (bundled; min version 3.3.0).

### Improvements

- **Dictionary columns decode once per read, not once per row.** Each
  `dict_store[code]` decompresses a whole msgpack batch, so reads and
  lexsort-based `sort_by` cost O(N) decompressions. At 1M rows an unindexed
  `sort_by` drops from 236 s to 713 ms, and a full column read from 44 s (at
  200k rows) to 193 ms.
- **`sort_by` on a dictionary column sorts int32 ranks, not decoded
  strings.** A row's alphabetical rank orders exactly as its value does — the
  trick the FULL index already used — so the sort key needs neither the decode
  nor lexsort's string comparisons. Key construction drops from 106 ms to 21 ms
  per 200k rows (cardinality 5000) and `sort_by(view=True)` from 247 ms to
  157 ms. The filtered small-copy path, which had its own copy of the key
  builder, now shares this one and picks up the same speedup.
- **`kind=BUCKET` indexes no longer cost more than the scan they replace.**
  Scattered matches were read one bucket run at a time, re-decompressing the
  same blocks many times, and the planner measured selectivity in buckets while
  the cost is paid in blocks — a mask selecting 21% of buckets could touch 96%
  of them. Affected every indexable dtype; the relative penalty was worst on
  numerics (`float64` 6.3 ms -> 77.9 ms before, 6.6 ms after).
- **The utf8 compute refusals now route instead of merely refusing.** Every
  path that cannot take a utf8 column — `add_computed_column`,
  `add_generated_column`, `assign`, `apply`, `lazyudf`, with a string
  expression or a DSL kernel — raises `NotImplementedError` naming the column
  and printing the three-line conversion, echoing the user's own expression
  where there is one. Two of those paths previously failed with a raw NumPy
  `DTypePromotionError` and a `ValueError: malformed node or string ...
  StringDType()`, neither of which named the column or the fix.
- **Slicing with plain slice/int keys is up to 1.7x faster.** `process_key()`
  used to route every key through ndindex's general index machinery, ~50x
  slower than needed for the common `arr[2:7, :50]`-style tuple: in the
  scattered-read benchmark it accounted for 43% of the loop. Plain tuples (and
  bare scalars) of slices and ints are now normalized directly — padding short
  tuples, converting ints to `slice(k, k+1)` with negative-index and
  out-of-bounds handling identical to ndindex's — and everything else
  (strided or negative steps, ellipsis, newaxis, fancy arrays) still falls
  through to ndindex. The optimization-tips guide's scattered-read tip went
  from 0.627 s to 0.363 s on the plain-open variant.

### Bug fixes

- **`@blosc2.jit` raised when a storage kwarg and an execution-tuning kwarg
  were combined** and the decorated function returned a NumPy array —
  `@blosc2.jit(jit=False, cparams=...)` ended in
  `blosc2.asarray(retval, jit=False, ...)`, which rejects the tuning kwargs.
  Only storage kwargs reach `asarray()` now; the function has already run, so
  there is nothing left to tune.
- **A DSL kernel over a utf8 column registered as a computed column, then
  broke the table.** `add_computed_column(name, kernel, inputs=["utf8_col"])`
  was accepted, after which every read of that column *and* `str(table)`
  raised `ValueError: malformed node or string`. The kernel is now refused at
  registration, where the table is still untouched.
- **`min()`/`max()` read from a column index returned the wrong value.** Two
  independent causes, both affecting every indexable dtype. The block summaries
  cover the column's *physical* extent, so the capacity padding (zeros, empty
  strings) was reduced along with the data and `min()` reported it — wrong on
  any table whose row count is not exactly its slot capacity. And `delete()`
  bumps a visibility epoch that nothing recorded, so deleted rows kept
  contributing their values to the block they sat in. Whole blocks below the
  live row count are still read from the sidecar; the block straddling the
  boundary is now rescanned, and a deletion since the index was built makes the
  shortcut stand down.
- **`create_index` on `utf8()` and `dictionary()` columns accepted any index
  kind** and built one over the alphabetical ranks that no query would ever
  consult — only `IndexKind.FULL` reaches a rank index. `kind` now defaults to
  `FULL` for these two column kinds (`BUCKET` elsewhere, unchanged) and raises
  `ValueError` when another kind is requested explicitly. Previously
  `create_index("category")` on a dictionary column built an unused BUCKET
  index by default.
- **Comparison operators on `UTF8Array`, dictionary and varlen scalar columns**
  returned a plain `False`: none defined them, so `column == "value"` fell
  through to object identity. Silently wrong rather than an error. All now
  return boolean masks; `UTF8Array` and `DictionaryColumn` answer a scalar
  without decoding any row.
- **`dictcol != value` raised `IndexError`** on any table with capacity
  padding: the negation was applied after the live-row intersection, turning
  every dead slot `True`.
- **Expressions over a bare `UTF8Array`** (`blosc2.lazyexpr("'x=' + a", {"a":
  arr})`) produced correct values down the wrong path — widened to fixed-width
  `<Un` and evaluated by the NumPy fallback, never reaching miniexpr, ignoring
  the span budget and losing the utf8 container. They now run through the span
  driver and return a `UTF8Array`.
- **`add_column()` on a varlen column left it short on tables with deleted
  rows.** vlstring/vlbytes/utf8/struct/object columns are indexed by physical
  position but the new column was filled with only as many entries as there
  were *live* rows, so the first read after a `delete()` raised `IndexError`.
  The dead slots are now filled too. `add_column()` on a `dictionary()` column
  raised `AttributeError` from deep inside the fixed-width path; it now raises
  `TypeError` naming the limitation.
- **`blosc2.utf8(null_value="\x00")` is rejected.** NumPy does not match a lone
  NUL against a `StringDType` array (`"\x00x"` and `"a\x00b"` compare fine), so
  every null mask would silently stop marking nulls. The default sentinel
  `'__BLOSC2_NULL__'` was never affected.
- **Nested (dotted) `utf8()` leaves can be filtered.** `t.where("trip.name ==
  'x'")` raised `NotImplementedError` on a utf8 leaf, while the same query on a
  `<Un`, `bytes()` or `dictionary()` leaf worked — utf8 was the only flavour
  where a dotted name could not be queried at all. Dotted names are aliased to
  safe identifiers before evaluation, but utf8 columns are outside the operand
  namespace, so they never reached that rewrite; they are now aliased by the
  utf8 driver itself. Covers scalar comparisons, `startswith`/`upper` and
  friends, mixed numeric predicates and `sum(where=)`.
- **`SChunk` slices were broken for typesizes above 255 bytes.** c-blosc2's
  `blosc2_schunk_get_slice_buffer()` derives the `getitem` for a partially
  covered chunk by dividing byte offsets by `schunk->typesize`, which the
  chunk header contradicts once the typesize is capped: the unit changes
  silently with the data, so blocks past the first could come back as
  uninitialised memory. Reachable from ordinary data — an `<U64` NDArray is a
  256-byte typesize, so `arr.schunk[1:4]` hit it. Partial reads now count in
  bytes via `blosc2_getitem_bytes_ctx()` (upstream c-blosc2 fix for
  Blosc/c-blosc2#796, included in the bundled 3.3.1), which is unambiguous at
  any typesize.
- **`create_index()` on a string column made every query on it return zero
  rows.** Silently — adding an index, an optimization, changed the answer. A
  segment summary is a `(min, max, flags)` record, so a `<Un` column makes it
  `8n + 1` bytes: 257 for `max_length=32`, which is the **default** width for
  a plain `str` annotation. Past 255 bytes c-blosc2 records the chunk typesize
  as 1, and the sidecar reader asked for spans in element units, so summaries
  decoded to garbage and pruned every candidate away. The boundary is exactly
  `8*max_length + 1 > 255` (31 works, 32 does not); `summary`, `bucket`,
  `partial` and `full` indexes were affected, `opsi` was not. A short span
  read now raises instead of leaving the destination partly uninitialised.
- **Scalar bools inside tuple keys now match NumPy.** `a[(True, :)]` raised
  `ValueError` from the fancy-index path, which cannot handle the 0-d bool
  array ndindex expands tuple bools to. NumPy treats them as a single
  `np.newaxis`-like dimension of length 1 (all True) or 0 (any False), placed
  at the first bool's position with multiple bools collapsing; `__getitem__`
  now rewrites the first bool to `None` and drops the rest, covering `bool`,
  `np.bool_` and 0-d bool arrays. Bare `a[True]` was already correct.
- **Slices with `start > stop` crashed with `ValueError: negative dimensions
  are not allowed`.** The slicing fast path normalized slices with
  `slice.indices()`, which leaves `a[100:50]` as `(100, 50)`; ndindex clamps
  any empty positive-step slice to `(0, 0)`, and without that clamp the
  result shape went negative. The fast path now clamps empty slices exactly
  like ndindex.
- **`blosc2.pack_tensor()` failed on 0-dim arrays** (e.g. a scalar
  `np.array(17)`); 0-dim inputs are now packed correctly.
- **`SChunk.meta.get(key, default)` recursed forever** when the key was
  absent; it now returns the default.
- **Pinned miniexpr fixes:** complex division with a scalar operand returned
  wrong results, expressions whose evaluation block was exactly 4096 elements
  could be mis-evaluated, and prefilter reads for operands wider than 255
  bytes read the wrong data. String expression results also keep the code-unit
  shuffle width (the UCS4 shuffle for `<Un`, instead of silently falling back
  to byte shuffle), `upper`/`lower` preserve the width, and a string result
  containing the utf8 null sentinel is refused rather than corrupting null
  masks.
- **`blosc2.random` constructors with NumPy integer scalar arguments** (e.g.
  `rng.integers(np.int64(3))`) no longer produce wrong shapes.
- **`engine=blosc2.jit` with `Series` operands no longer crashes on
  `axis=1`**, and an already-jitted function is not jitted a second time.

### Documentation

- **Two new optimization tips** joined `doc/guides/optimization_tips.md`:
  generating arrays with DSL kernels, and broadcasting small operands into
  large on-disk arrays; the guide's figures were regenerated against the
  reference machine and the benchmark harness now checks c-blosc2's
  pread/handle-cache work.
- The pandas engine guide was rewritten around the readable-UDF pitch (with a
  Kepler plot), the pandas guide around one question, and `blosc2.argsort` is
  now documented.

## Changes from 4.9.0 to 4.9.1

A small hot-fix release for the Arrow interop work in 4.9.0: a real
performance regression in dictionary-column export, and a clearer error
message when opening a nonexistent `CTable` in append mode.

### Improvements

- `CTable.iter_arrow_batches()` (and therefore `to_arrow()` and the Arrow
  PyCapsule interchange, `__arrow_c_stream__`) no longer recomputes the
  full live-row-position array from scratch on every batch, for every
  dictionary column — an `O(n_rows)` scan that was repeated
  `O(n_rows / batch_size)` times. The position array is now computed once
  per export call instead. Measured 6-14x faster export for
  dictionary-encoded string columns (e.g. `company`) on a 1M-row table.
- Reminder for anyone consuming a `CTable` through the Arrow PyCapsule
  protocol (DuckDB, pyarrow, Polars, pandas): the raw Arrow C Stream
  interface has no column-projection pushdown, so a consumer that only
  needs a few columns still triggers export of every column in the table.
  Use `CTable.select([...])` to project down to the columns you actually
  need before handing the table to the consumer, particularly if any
  column is an expensive nested/list type.

### Bug fixes

- **`SChunk` slices were broken for typesizes above 255 bytes.** c-blosc2's
  `blosc2_schunk_get_slice_buffer()` derives the `getitem` for a partially
  covered chunk by dividing byte offsets by `schunk->typesize`, which the
  chunk header contradicts once the typesize is capped. Across 153 slice
  shapes at typesize 256, 150 raised `"Error while getting the slice"` and
  the 3 single-element ones returned the wrong bytes with no error at all.
  Reachable from ordinary data -- an `<U64` NDArray is a 256-byte typesize,
  so `arr.schunk[1:4]` hit it. Fixed upstream in Blosc/c-blosc2#796, so this
  release requires a c-blosc2 that carries that fix.
- **`create_index()` on a string column made every query on it return zero
  rows.** Silently -- adding an index, an optimization, changed the answer.
  A segment summary is a `(min, max, flags)` record, so a `<Un` column makes
  it `8n + 1` bytes: 257 for `max_length=32`, which is the **default** width
  for a plain `str` annotation. Past 255 bytes c-blosc2 records the chunk
  typesize as 1, and the sidecar reader asked for spans in element units, so
  summaries decoded to garbage and pruned every candidate away. The boundary
  is exactly `8*max_length + 1 > 255` (31 works, 32 does not); `summary`,
  `bucket`, `partial` and `full` indexes were affected, `opsi` was not.
  A short span read now raises instead of leaving the destination partly
  uninitialised.
- **Expressions over operands wider than 255 bytes returned wrong results.**
  c-blosc2 caps a typesize above 255 to 1 in the chunk header so its split
  machinery keeps working, and the miniexpr prefilter was asking
  `blosc2_getitem_ctx()` for operand blocks in element units, which the chunk
  then read as a byte range: every block past the first was uninitialised
  memory. `arr == "hello"` over 1200 rows of `<U64` (256 bytes) matched 1 row
  instead of 400, non-deterministically and with no error raised. Affects any
  dtype whose itemsize exceeds 255 bytes; `<U64` is the first fixed-width
  string that hits it.
- Opening a `CTable` with `mode="a"` at a path that doesn't exist yet now
  raises a clear `FileNotFoundError` ("mode='a' opens an existing table;
  use mode='w' to create a new one") instead of silently falling through
  and creating a new, empty table.

## Changes from 4.8.1 to 4.9.0

This release is about cooperation: `CTable` now speaks the tabular
ecosystem's own protocols instead of asking it to speak blosc2's. Arrow
tools (pyarrow, DuckDB, Polars, and pandas >= 3.0 via `DataFrame.from_arrow()`)
can consume or produce a `CTable` directly through the Arrow PyCapsule
interface; a new `utf8()` string column stores text in Arrow's own
offsets+bytes layout and reads back as NumPy `StringDType`; and
`engine=blosc2.jit` now runs correctly inside pandas 3 itself. Alongside
that, `CTable` gained a proper missing-data story (`fillna`/`dropna`,
null-safe arithmetic and comparisons) and a pandas-3-style chaining API
(`assign()`/`col()`, UDF aggregations, `CTable.apply()`).

### New features

#### Ecosystem interop

- **Arrow PyCapsule interchange**: `CTable.__arrow_c_stream__` lets
  pyarrow, DuckDB, and Polars consume a `CTable` directly as a stream of
  record batches, with bounded memory — no `to_arrow()`/copy step
  required. pandas >= 3.0 can do the same via the new
  `pandas.DataFrame.from_arrow()` classmethod (the plain `pd.DataFrame(t)`
  constructor does not use this protocol). `CTable.from_arrow()` now
  accepts any object implementing the same protocol on ingest
  (single-argument form), in addition to the existing `(schema, batches)`
  form — including Arrow's `string_view` layout (Polars' default string
  export type), which raised `TypeError` before this release.
- **`blosc2.utf8()`**: a new column type for high-cardinality/free-text
  strings, storing each column as two companion NDArrays — int64 row
  offsets plus a UTF-8 byte blob — the same layout Arrow uses for
  `large_string`. A row costs exactly its encoded byte length (7-13x
  smaller uncompressed than fixed-width `string()` on high-cardinality
  text), and reads materialize as NumPy `StringDType` arrays (NumPy >=
  2.0 required; older NumPy falls back to `vlstring` on Arrow/Parquet
  import with a clear message). Full query surface: comparisons,
  `where()`, `sort_by`, `group_by` keys, `fillna`, and Arrow export
  (`large_string`, sentinel-null mask) / import (Arrow/Parquet string
  columns now default to `utf8` instead of `vlstring`). Measured on the
  1e7-row NYC-taxi `company` column: ingest 3622 ms -> 598.6 ms
  (**6.05x** faster, only 1.22x slower than `string()` and 2.63x faster
  than `vlstring()`), full-column read 2472.6 ms -> 165.3 ms (**14.96x**
  faster), equality filter ~1900 ms -> 162 ms (**~11.7x** faster), and
  groupby-key factorization 3304 ms -> 558 ms (**2.83x**, down from a
  16.9x-slower initial fallback, now faster than fixed-width `string()`
  keys). See the new "Choosing a string column type" guide in the
  `CTable` reference and `examples/ctable/utf8_strings.py`. Known gaps:
  string-expression filters (`t.where("name == 'x'")`) and
  `create_index` on `utf8` columns still raise a clear
  `NotImplementedError`; use fixed-width `string()` if you need those.
- **pandas engine, pandas 3 compatible**: `engine=blosc2.jit` for
  `DataFrame.apply` now returns a properly indexed `DataFrame`/`Series`
  under pandas 3.0.3's default `raw=False` (previously a raw NumPy
  array, so results only matched by value, never by type).
  `Series.map(func, engine=blosc2.jit)` is now implemented (it
  previously always raised `NotImplementedError`). Non-numeric columns
  now raise a clear `ValueError` instead of a deep `numexpr` error. See
  the guide "Using Blosc2 as a pandas engine" and
  `bench/bench_pandas_engine.py`.

#### pandas-style CTable API

- `CTable.assign(**named_exprs)`: return a view with additional computed
  columns, without mutating the table or copying column data. Pairs with
  the new `blosc2.col(name)` — an unbound column expression that defers
  operator replay until it's bound to a table (`assign()`, `t[...]`,
  `where()`) — to write pandas-3-style chains:
  `t.assign(profit=col("revenue") - col("cost"))[col("profit") > 0].sort_by("profit", ascending=False).head(10)`.
- **Missing data**: `Column.fillna(value)` replaces sentinel/`None`
  values for scalar, dictionary, and varlen-scalar columns;
  `CTable.dropna(subset=None)` returns a view excluding rows where any
  nullable column (or a chosen subset) is null. Column arithmetic
  (`+ - * / // % **`) and comparisons (`< <= > >= == !=`) on nullable
  int/timestamp/bool columns now propagate nulls instead of operating on
  the raw sentinel: arithmetic promotes to `float64`/NaN, comparisons
  follow SQL `WHERE` semantics (a null operand never satisfies any
  comparison) — fixing filters like `t[t.x < 0]` wrongly matching null
  rows. Also fixes null detection for timestamp columns
  (`is_null()`/`null_count()`/`dropna()` previously missed every
  `NaT`).
- **UDF aggregations**: `group_by().agg()` accepts a custom callable as
  the op via the named form (`output_name=(column, callable[, dtype])`);
  it receives each group's live, non-null values as a 1-D NumPy array.
  `CTable.apply(func, columns=None, dtype=None, engine="auto")` applies
  a UDF across the table's live rows, sugar over `blosc2.lazyudf()`.
  `group_by(engine=...)` now accepts `"auto"`/`"numpy"` explicitly
  alongside the existing default.
- `CTable` views are now read-only for value writes: `Column.__setitem__`
  and `Column.assign()` raise `ValueError` on a view, pointing at
  `take()`/`copy()` as the escape hatch (structural mutations already
  raised; this closes the one unguarded cell-write path).
- `NDArray.iter_sorted()`/`argsort()` on a `FULL`-indexed array now reads
  the sidecar range directly instead of building the full permutation —
  ~52x faster and ~193x less memory on a 20M-element array for
  `iter_sorted(start=-k)`-style tail queries. See the new optimization
  tip.

### Improvements

- String-key `group_by()` (fixed-width `string()` keys) now factorizes
  via an exact hash of each row's raw bytes instead of a NumPy
  UTF-32 argsort, with a vectorized collision-checked verify pass
  keeping the result bit-identical: 1157 ms -> 737 ms on a 1e7-row
  benchmark. Every caller benefits automatically; no `engine=` switch
  involved.
- An example showing ctables and ndarrays bundled together in the same
  `TreeStore`.
- C-Blosc2 bumped to 3.2.3.

### Bug fixes

- Fixed a `@blosc2.dsl_kernel`-decorated function crashing unconditionally
  when passed as a groupby UDF aggregation (`g.agg(name=(col,
  dsl_kernel_fn))`): it now runs like the equivalent undecorated callable.
- Fixed `CTable.head()`/`tail()` silently discarding row order when called
  on a lazily-sorted view (e.g. `t.sort_by("col", ascending=False)` on a
  view, or any `.sort_by()` result chained off a prior filter): they
  ignored `_cached_live_positions` and built a plain physical-order mask
  instead, so `t.where(...).sort_by("x", ascending=False).head(10)` came
  back in the wrong order.
- Fixed a `NameError` when `nan`/`inf` scalars appeared in lazy
  expressions; `ShapeInferencer` no longer ignores user-provided shapes
  for `nan`/`inf`.
- Fixed `CTable.from_arrow()` raising `TypeError: No blosc2 spec for Arrow
  type DataType(string_view)` on any Arrow `string_view`/`binary_view`
  column — the layout Polars exports by default through the PyCapsule
  protocol. `string_view`/`binary_view` now import exactly like
  `string`/`large_string`/`binary`/`large_binary` everywhere a column's
  Arrow type is inspected (schema inference, null-sentinel selection,
  list/struct/dictionary value types).

## Changes from 4.8.0 to 4.8.1

### Improvements

- Read-only memory mapping for `CTable` stores: `CTable.open()` (and
  `FileTableStorage`) gain an `mmap_mode="r"` parameter, mirroring
  `blosc2.open()`. All members of a read-only store — scalar, list, varlen
  and dictionary columns alike — are then read from mapped pages; for `.b2z`
  archives, in place at their offsets inside the single mapped container
  file. With several concurrent readers on one file this pays off quickly:
  2.5x/4.4x/4.5x faster wall time for 1/4/8 readers in our benchmark
  (`bench/optim_tips/tip_10_mmap_many_readers.py`).
- Reduced memory consumption in `CTable.extend()` when passed an `NDArray`.
- C-Blosc2 bumped to 3.2.2.

### Bug fixes

- Fixed lifetime/use-after-free hazards around zero-copy cframes and
  `vlmeta`: `schunk_from_cframe()`/`ndarray_from_cframe()` with `copy=False`
  (the default) returned objects pointing into the caller's bytes buffer
  without keeping it alive, so a temporary cframe (e.g.
  `ndarray_from_cframe(response.content)`) could be reclaimed under the live
  object, corrupting reads. The buffer is now pinned on the returned object.
  Also, `vlmeta` read paths (`__getitem__`/`__len__`/`__iter__`) now raise
  `ReferenceError` on an orphaned owner instead of segfaulting, matching the
  write paths.
- `BatchArray.delete()` / `ObjectArray.delete()`: negative-step slices
  (e.g. `del arr[3:0:-1]`) deleted chunks in ascending order, shifting the
  indices of chunks still to be deleted and removing the wrong ones (or
  raising `RuntimeError`).
- `ListArray.extend_arrow()`: Arrow chunks were appended to the backend
  without flushing pending cells first, reordering unflushed rows after the
  new ones.
- Shape inference: `stack()` with a negative axis inserted the new dimension
  one position too early, so a lazyexpr's reported `.shape` disagreed with
  its computed result; `vecdot()` also normalized positive axes as if they
  were negative.
- Chunked `matmul()`: broadcast (size-1) operand batch dims were sliced with
  the result-chunk coordinates, producing empty slices when the broadcast
  dim spans several result chunks.
- `DictStore.__setitem__()`: overwrite semantics depended on value size —
  embedded keys refused overwrite ("already exists"), while an
  embedded-to-external overwrite double-stored the key and resurrected the
  stale embedded value after a delete. Assignments now behave uniformly
  dict-like, dropping any previous value.
- Explicitly passing `cparams=None` or `dparams=None` to NDArray
  constructors crashed; both now mean "defaults".
- Fixed a builtin-shadowing bug that made store detection in
  `blosc2.open()` recurse ~250 times (silently swallowed) on every
  `.b2z`/`.b2d` open; opening a `.b2z` is now ~10x faster under allocation
  tracing.

### Documentation

- Restructured docs (#674), with a new
  [Optimization tips](https://www.blosc.org/python-blosc2/guides/optimization_tips.html)
  section, including new tips on grouping related data into a single
  memory-mapped `.b2z` file and on using `mmap_mode="r"` with many
  concurrent readers.

## Changes from 4.7.0 to 4.8.0

### Sharing containers across processes

- New `locking` storage parameter (and the `BLOSC_LOCKING` environment
  variable to enable it fleet-wide) serializes accesses to an on-disk
  `SChunk`/`NDArray`/`EmbedStore`/`DictStore` against other handles and other
  processes, via a small sidecar lock file (`.b2lock`). Advisory: every
  handle touching the container must opt in.
- `SChunk.holding_lock()` / `NDArray.holding_lock()`: a context manager to
  hold the exclusive lock across several operations, making a multi-step
  mutation atomic to other locked handles.
- New `SChunk.refresh()`, mirroring the existing `NDArray.refresh()`.
- Fixed a data-loss bug in `NDArray.append()`: it read the cached,
  unrefreshed shape before computing the resize target, so under
  concurrent growth/shrink — even inside `holding_lock()` — another writer's
  just-appended data could be silently deleted.
- `EmbedStore` and `DictStore` (`.b2d`) now support cross-process writers
  under locking: transactional writes plus key-map re-sync, so readers
  follow keys added or removed by another process.
- `DictStore.to_b2z()` (and `TreeStore`, which inherits from it) now replaces
  the target file atomically, so concurrent readers always see either the old
  or the new archive, never a torn one.
- Growth-SWMR (single writer, multiple readers): a reader `NDArray` handle
  opened before a `resize()` made through another handle follows the new
  shape on its next data access, or via the new explicit `NDArray.refresh()`.
- New user guide page,
  [Sharing containers across processes](https://www.blosc.org/python-blosc2/getting_started/sharing_across_processes.html),
  covering all of the above plus the caveats (NFS, `mmap_mode`, Windows
  in-use-file rename).

### Bug fixes

- Fixed `detect_aligned_chunks()` (used internally to fast-path aligned
  slice reads/writes): a floor-division undercounted the chunk grid for
  arrays whose shape isn't a multiple of the chunk shape, which could
  silently return the wrong chunk's data for an otherwise-aligned slice
  with a nonzero start in an earlier dimension.

### Others

- Raised the manylinux wheel baseline from `manylinux2014` (CentOS 7, glibc
  2.17, GCC 10.2) to `manylinux_2_28` (AlmaLinux 8, glibc 2.28, GCC 12),
  fixing a build failure with NumPy >=2.5 which requires GCC >=10.3.

## Changes from 4.6.0 to 4.7.0

### DSL → JavaScript backend for WebAssembly (`jit_backend="js"`)

- Under WebAssembly/Pyodide, `@blosc2.dsl_kernel` kernels can now be transpiled
  to JavaScript and run via the browser's JIT.  It is the **default** there for
  transpilable floating-point kernels (silently falling back to miniexpr for
  anything it can't handle), and beats the WASM TinyCC JIT on compute-heavy
  kernels (e.g. ~2.8x on a Newton-fractal kernel).  Request it explicitly with
  `compute(jit_backend="js")`; outside WebAssembly that raises.
- Supports index/shape symbols (`_i0`/`_n0`/`_ndim`/`_flat_idx`) and integer inputs
  with a floating-point output.  Integer/complex *output*, reductions, and
  unsupported constructs stay on miniexpr.  Native builds are unaffected.

### New `blosc2.validate_dsl_jit()`

- A new introspection helper reports whether a DSL kernel actually JIT-compiles
  (vs. silently falling back to the interpreter) for a given set of operand and
  output dtypes, without running it on real data::

      status = blosc2.validate_dsl_jit(kernel, [np.float64, np.float64], np.float64)
      status["jit"]  # True if a runtime JIT kernel was produced

### miniexpr fixes

- `DSLValidator` now rejects `;`-joined sibling statements with a clear
  "one statement per line" error, and assigning to an input parameter raises a
  targeted error naming the param.
- Fixed (in miniexpr) a name collision where DSL variables named `out`, `idx`,
  `nitems`, `inputs` or `output` clashed with codegen-internal identifiers,
  causing the generated C to fail to compile and silently fall back to the
  interpreter.  Codegen identifiers are now namespaced under `__me`.

## Changes from 4.5.1 to 4.6.0

### `CTable.sort_by(view=True)`: zero-copy sorted views

- `CTable.sort_by()` now accepts **`view=True`**, returning a lightweight
  **sorted view** that shares the parent's column data and gathers rows on
  demand in sorted order — no whole-table copy.  This is ideal for reading a
  sorted slice of a large (possibly on-disk) table::

      t.sort_by("col", view=True)[:10]      # top-10 without materialising

  Sorting on a **fully indexed** column streams directly from the index, so the
  table is never materialised.  Multi-column sorts and dotted (nested) leaf
  names are supported (e.g. `t.sort_by(["trip.begin.lon", "payment.fare"],
  ascending=[True, False])`).

### `where` on dictionary (string) columns

- `where` expressions now work over **dictionary-encoded (string) columns**,
  including membership tests such as `'"Acme" in company'`, so categorical
  text columns can be filtered without decoding the whole column.

### `b2view` is now an opt-in extra

- The `b2view` terminal browser and its TUI stack (`textual`,
  `textual-plotext`) are no longer core dependencies: a plain
  `pip install blosc2` no longer pulls them, keeping the compression library
  lean (and dropping deps that are unusable under wasm32, which has no TTY).
  Install the viewer with `pip install "blosc2[tui]"`, or
  `pip install "blosc2[hires]"` to also get the high-res `h` view.  The
  `b2view` command prints this hint if the dependencies are missing.

### `group_by`: flexible aggregation naming

- `CTable.group_by(...).agg()` now accepts a **list of `(column, ops)` pairs**
  and **explicit output names** (pandas-style keyword arguments), alongside the
  existing auto-suffixed mapping; the forms can be combined::

      g.agg({"sales": ["sum", "mean"]})              # auto: sales_sum, sales_mean
      g.agg([(t.sales, ["sum", "mean"])])            # auto, but accepts Column objects
      g.agg(revenue=("sales", "sum"))                # explicit: revenue
      g.agg({"sales": "sum"}, n=("*", "size"))       # combined, with a named row count

  The list-of-pairs and named forms accept `Column` objects (`t.sales`), which
  the mapping form cannot because `Column` is unhashable and so cannot be a dict
  key.
- Aggregation ops may also be given as the matching blosc2 reduction *functions*
  (`blosc2.sum`, `mean`, `min`, `max`, `argmin`, `argmax`), matched **by
  identity** -- e.g. `g.agg([(t.sales, [blosc2.sum, "mean"])])`.  This is a
  naming shorthand only; arbitrary/UDF callables (and look-alikes such as
  `np.sum` or a user function named `sum`) are rejected rather than silently
  misinterpreted.

### `group_by` / `group_reduce`: tri-state `sort=`

- **Vectorized dictionary group ordering**: `group_by()` result building now
  batch-decodes dictionary (string) keys in one pass (`decode_batch`) instead of
  one `decode()` per group, making high-cardinality string group-bys dramatically
  faster (end-to-end `group_by().size()` dropped from seconds to milliseconds on
  ~100k-group workloads).
- **`sort=` is now a tri-state** (`None` / `True` / `False`) on both
  `CTable.group_by()` and `blosc2.group_reduce()`:
  - `True` — always return groups sorted by key.
  - `False` — never sort; deterministic but unspecified order.
  - `None` (the **new default**) — *auto*: sort only when cheap. Integer and
    dictionary keys are sorted (free / vectorized); float and multi-key results,
    whose only ordering is an O(G log G) Python sort over every distinct group,
    are left unsorted to avoid a cost that can rival the grouping itself on
    high-cardinality data.
- **Behavior changes** (the two APIs had different prior defaults, so they move
  in opposite directions):
  - `CTable.group_by()` previously returned results *always* sorted. Under the
    new `None` default, **float-key and multi-key group-bys are no longer
    key-sorted by default** — pass `sort=True` to restore sorted output. This is
    a deliberate divergence from pandas (which defaults to `sort=True`), suited
    to blosc2's large / on-disk datasets.
  - `blosc2.group_reduce()` previously defaulted to `sort=False` (unsorted).
    Under the new `None` default its **cheap kernels now sort by default** —
    most visibly float keys, which previously came out in hash order. Integer
    keys were already ascending; the generic Python fallback stays unsorted.
    Pass `sort=False` to opt out.

### Accelerated reductions from index summaries

- `min`/`max` on indexed `Column`s, and `argmin`/`argmax` inside `group_by`, are
  now **accelerated using the index's per-block min/max summaries**: when an
  index is available these reductions run from the precomputed summaries instead
  of decompressing the underlying data, which is dramatically faster on large
  columns.  A fast path also builds **min/max envelope plots** from any index.
- The **last `group_by` operation is memoized** and reused when the same
  grouping is requested again, avoiding recomputation in interactive / repeated
  workflows (e.g. `b2view`).

### b2view: group-by, sort, and richer plots

- **Interactive group-by (`G`)**: group a `CTable` by a column (integer, string,
  or now **float** keys) directly in the viewer, with a three-list / two-column
  menu; while grouped, `S`/`R` operate on the grouped result and the data
  panel's subtitle shows a `G(roup)` chip.  The last grouping is memoized for
  instant reuse.
- **Sort by column (`S`)**: sort a `CTable` by a **fully indexed** column via a
  dropdown (`R` toggles reverse) as a zero-copy `sort_by(view=True)` that streams
  from the index — the table is never materialised, `Esc` restores the original
  order, and a `SORTED` chip shows in the status bar.  Non-indexed columns can
  now be sorted too.  Sort and filter are mutually exclusive; a row window
  composes over a sort, and an `f`ilter is preserved across `S`ort / `G`roup.
- **Better plots of grouped/sorted views**: a grouped view plots **bars for a
  categorical key** and **lines for a numeric key**; numeric-key group plots
  render as **stem/impulse charts** rather than misleading connected lines.  Bar
  plots gain an `h`i-res counterpart mirroring the line/scatter plots, and `+`/`-`
  zoom about the view's left edge.
- **`--max`** maximizes the current panel, and `escape` is now the single,
  consistent way to back out of every modal.

### Other / bug fixes

- **C-Blosc2 upgraded to 3.1.5.**
- **Open-file cache correctness**: cached open handles are now validated against
  the file's fingerprint (`st_mtime_ns`, `st_size`) and cached index handles are
  released when a table closes, so a file changed underneath an open handle is no
  longer served stale.
- **NumPy 2.5 compatibility**: adjusted for deprecations in NumPy 2.5.
- Substantially **reduced test-suite runtime**, and emscripten builds no longer
  attempt to spawn subprocesses (unsupported there).

## Changes from 4.5.0 to 4.5.1

This follow-up release builds the `b2view` terminal viewer into a richer
data-exploration tool — a **scatter plot**, a **searchable column picker**, a
**one-shot demo download**, refreshed chrome, and several interaction fixes — and
upgrades the bundled **C-Blosc2 to 3.1.4**.  WASM/Pyodide is now a **fully
supported platform**, and `CTable.info` reports **per-column compressed sizes**.

### b2view: richer exploration

- **Scatter plots**: from a column plot, press `s` to scatter the current column
  (X) against another column (Y) chosen from a list, over the current (zoomed)
  row range; `h` then opens a high-resolution `matplotlib` scatter.
- **High-res for 1-D series is now an envelope plot** (matching the in-terminal
  view), and a new `r` key toggles between the min/max envelope and the raw
  values (strided-sampled when the range is wide).
- **Searchable column picker**: the `c` go-to-column key now opens a searchable,
  selectable list (type to filter, ↑/↓, Enter) for CTables, instead of a text
  field; N-D arrays still go by numeric index.
- **Show/hide columns**: `/` opens a searchable multi-select to pick which CTable
  columns are displayed.
- **Demo download**: `b2view --download` fetches a demo bundle
  (`chicago-taxi-flat.b2z` by default) into the current directory if it is not
  already there, then opens it.
- **Refreshed chrome**: a branded header, a left-docked filename label in the
  title, and clearer status chips.

### b2view: interaction fixes

- **Go-to-row/column pre-fill is now pre-selected**, so the first keystroke
  replaces the current index instead of appending to it (typing a column name no
  longer produced e.g. `0payment.fare`).
- **Escape keeps its layered exit while a panel is maximized**: with the data
  panel maximized, escape now unlocks a plot's locked row window (and clears
  filters) as documented, instead of being hijacked into restoring the panel —
  use `r` to restore (`ESCAPE_TO_MINIMIZE = False`).
- Test-suite robustness fixes (a timing flake and a Windows rendering glitch).

### Other

- **C-Blosc2 upgraded to 3.1.4.**
- **WASM/Pyodide is now a fully supported platform**, with more frequent CI runs.
- **`CTable.info` shows per-column compressed sizes** (`cbytes` and `cratio`),
  and `print_versions()` uses clearer `Python-Blosc2` / `C-Blosc2` labels.

## Changes from 4.4.5 to 4.5.0

This release teaches the `b2view` terminal viewer to **plot** — peak-preserving
envelope line plots of any series, with zoom, a row-window lock, and an optional
high-resolution matplotlib view — and gives `CTable` a **pandas-like display and
CSV** experience.  It also publishes **WASM/Pyodide wheels to PyPI** and adds
faster strided reads for `NDArray` and `Column`.

### b2view: plotting and data inspection

- **In-terminal plots**: press `p` on a numeric series (a CTable column or an
  array row) to draw a braille line plot.  Plots are **peak-preserving min/max
  envelopes by default**, so no spike or trough is hidden however large the
  series is; large local series stream their envelope *exactly* in bounded
  spans (only remote c2arrays fall back to a labeled strided sample).
- **Zoom and row-window lock**: zoom the plot into a row range and pan it; press
  `v` to **lock the data grid to the plotted range** so paging stays inside it
  (escape unlocks).  The plot and high-res views honor the locked window.
- **High-resolution view**: `h` opens a high-res `matplotlib` image of the
  plotted range (new optional `hires` extra: `matplotlib` + `textual-image`).
- **On-demand cell decode**: `enter` decodes a single skipped/expensive CTable
  cell, and SChunk nodes now preview as a paged hex dump.
- **Fixes and polish**: row paging re-aligns to the page grid after dim-mode
  single-row scrolls; the data panel now focuses correctly with
  `--path ... --panel data`; status chips are branded yellow.

### CTable display

- **`CTable.to_string()` now renders the whole table by default** (every row and
  every column), like `pandas`' `DataFrame.to_string()`.  New `max_rows` and
  `max_width` parameters truncate on demand.  *Behaviour change*: previously
  `to_string()` returned the truncated view; code that relied on that should
  pass `max_rows=`/`max_width=` (or use `str()`).
- **The `[N rows x M columns]` dimensions footer now follows pandas**: omitted by
  `to_string()` (pass `show_dimensions=True` to force it), and shown by
  `str`/`repr`/`print` only when the view is actually truncated.  Previously it
  was always appended.
- **`repr(ctable)` now shows the same truncated table as `str(ctable)`**
  (pandas/polars convention), instead of the one-line `CTable<…>` summary.  The
  compact summary remains available via `ctable.info`.
- **New display options** in `set_printoptions`: `display_width` controls the
  column-fitting width budget (`None` = auto-detect terminal, `-1` = show all
  columns, positive int = fixed budget), and `display_rows` now accepts `-1` to
  show all rows (`0` still shows none).
- **New `blosc2.printoptions(...)` context manager** temporarily sets the display
  options and restores them on exit, e.g.
  `with blosc2.printoptions(display_rows=-1, display_width=-1): print(t)`.

### CTable I/O

- **`CTable.to_csv()` now accepts no path**, returning the CSV as a string like
  `pandas`' `DataFrame.to_csv()`.  Passing a path still writes the file (and
  returns `None`); the returned string is byte-for-byte the same as the file.

### Performance

- **Faster strided reads**: `NDArray.__getitem__` gains a sparse-gather fast
  path for large strides, and `Column.__getitem__` short-circuits when the
  logical positions equal the physical ones.
- **Fix**: a negative `step` in `Column` getitem could return `[]`; it now
  returns the reversed selection.

### Indexing

- **Fix**: a sidecar-handle cache collision could return the wrong SUMMARY
  index for a compact-store column.
- **Cross-column index pruning** is now enabled for compact CTable queries, so
  more predicates prune blocks before any data is materialized.  The docs also
  note when summary indexes are *not* created automatically.

### Packaging

- **WASM/Pyodide wheels on PyPI**: the main wheel build now also produces
  `pyemscripten` wheels for CPython 3.13 (2025 ABI) and 3.14 (2026 ABI) and
  uploads them to PyPI, so `blosc2` is `micropip`-installable in Pyodide, and
  `b2view` prints a clear message instead of crashing when run under WASM.
  *Known limitation*: slicing an in-memory `SChunk` loaded from a frame fails on
  the Pyodide 0.29.x Emscripten toolchain (cp313); it works on Pyodide 314
  (cp314) and natively.  See issue #664.
- **cibuildwheel updated to 4.1.**

## Changes from 4.4.3 to 4.4.5

Note: 4.4.4 was skipped due to a failure during the release process.

This release promotes the `b2view` terminal viewer to a core feature —
installed by default, with new interactive row and column filtering — and
makes BatchArray block layouts (and hence compression ratios) reproducible
across CPUs.

### b2view, the terminal data viewer

- **Installed by default**: `textual` and `rich` are now regular
  dependencies, so the `b2view` CLI works out of the box (the `[tui]` extra
  is gone).  A getting-started walkthrough was added to the docs, and the
  README now lists the CLI tools.
- **Row filtering**: pressing `f` on a CTable node opens a modal that takes
  the same string expressions as `CTable.where()` (dotted nested names,
  `and`/`or`) and pages through the matching view.  Filters are remembered
  per node for the session, the data header shows the active filter plus
  the unfiltered total, and escape (or an empty expression) clears it.
- **Column filtering**: `/` narrows the visible columns by case-insensitive
  substring; column paging and the `c` goto-column modal then operate on
  that subset.  Combines freely with the row filter; escape clears one
  layer per press (rows first, then columns).
- **Mouse handling**: the terminal owns the mouse by default, so native
  text selection/copy works like in any CLI program; `--mouse` lets b2view
  capture it instead (click-to-focus, wheel scrolling by half a page,
  paging at the edges).
- **Navigation**: `?` opens a help screen listing all keys; `c` jumps to a
  column by index, exact name or unique name prefix; `s`/`e` jump to the
  first/last column window; row paging and jumps keep the cursor on its
  column; dim-mode index/viewport movements clamp at the boundaries instead
  of wrapping around.
- **Rendering**: column windows are fitted from measured rendered widths
  (and re-fitted on terminal resize and panel maximize/restore), and float
  columns use a uniform number of decimals so decimal points align down
  the column.
- **Test suite**: first automated tests for the TUI — Pilot-driven keyboard
  journeys against a deterministic generated store (marker `tui`), plus
  render unit tests.  Skipped on wasm, where Textual apps cannot start
  (no termios).

### BatchArray

- **Reproducible block layouts**: automatic variable-length block sizing
  now uses fixed byte budgets (1 MiB for clevel 1-3, 8 MiB for 4-6, 16 MiB
  for 7-8) instead of the CPU cache sizes, so the layout — and hence the
  compression ratio — no longer depends on the machine that created the
  array.

### Build and docs

- **Installing test dependencies**: the docs now use
  `pip install . --group test` (a PEP 735 dependency group); the stale
  `[test]` extra syntax was removed.
- **cibuildwheel updated to 4.0.**

## Changes from 4.4.2 to 4.4.3

This is a maintenance release focused on faster CTable cold-start, printing
and groupby performance, a lighter `import blosc2`, new raw-storage access
for columns, and support for the new J2K/HTJ2K codec plugins.

### CTable performance

- **Lazy column opening in views**: `select()` (and other view-producing
  operations) no longer open every projected column up front.  A column is
  only opened from storage when the view actually reads it, so selecting and
  then touching a subset of columns — or aggregating a single one — skips
  the cold-start cost of the rest.
- **Lazy index opening in queries**: query planning no longer opens every
  SUMMARY-indexed column on a wide persistent table; only indexes for
  columns actually referenced by the predicate are loaded.
- **Faster table printing**: `repr()`/`to_string()` now memoise per-column
  sparse gathers for the duration of a render and combine the head and tail
  rows into a single sparse read per column.  Each column is read from
  storage once instead of ~6 times (precision detection, width sizing and
  row rendering all hit the cache).
- **Groupby with integral float keys**: float key columns whose values are
  integral and fit a compact non-negative range (e.g. float32 id/second
  columns) now take the dense single-key fast path instead of the markedly
  slower generic float-hash path.  Fractional or non-finite keys fall back
  automatically.
- **No tempdir in read mode**: opening a `.b2z`/`.b2d` store in `'r'` mode
  no longer creates a temporary working directory, since nothing is ever
  written.

### Lighter imports and prefetcher rework

- **asyncio dependency dropped**: the on-disk chunk prefetcher used by the
  UDF and numexpr fallback engines now uses plain `concurrent.futures`
  instead of an asyncio event loop.  `import blosc2` no longer pulls in
  ~30 asyncio modules, saving ~3 MB of memory footprint at import time.
- **Prefetcher deadlock fixed**: an exception during evaluation could leave
  the generator finalizer blocked forever in `thread.join()` while the
  reader thread was stuck on a full prefetch queue.  A stop event now makes
  the producer bail out when its consumer goes away.

### New features

- **`Column.raw` accessor**: returns the underlying storage container of a
  column (`NDArray`, `ListArray`, `DictionaryColumn`, …) directly.  Unlike
  `Column.__getitem__`, which always materializes NumPy arrays, this is the
  column as a blosc2-native compressed object — usable as a lazy-expression
  operand without decompressing, and exposing storage details like `schunk`,
  `chunks` or `cparams`.  Note that this is a *physical* view: fixed-width
  containers are over-allocated to chunk capacity, so slice to `len(table)`
  to get just the live rows, and no validity-mask or null-sentinel
  processing is applied.  Raises `AttributeError` for computed columns,
  which have no backing storage.
- **J2K and HTJ2K codec IDs**: `blosc2.Codec.J2K` and `blosc2.Codec.HTJ2K`
  expose the IDs for the new JPEG 2000 codec plugins (installable with
  `pip install blosc2-j2k` and `pip install blosc2-htj2k`).

### Fixes

- **`--float-trunc-prec` and nested columns**: the precision-truncation
  filter of the `parquet_to_blosc2` CLI now propagates to float fields
  inside nested (struct/list) columns too.
- **Guard for unsupported computed-column expressions**: expressions that
  would serialize to an empty or non-round-trippable string are now rejected
  with an early, actionable `ValueError` at `add_computed_column()` time,
  instead of silently breaking on reload.

### Build

- **C-Blosc2 updated to 3.1.3.**

## Changes from 4.4.1 to 4.4.2

This is a feature and maintenance release that promotes DSL kernels to
first-class CTable computed columns, adds a new `CTable.__setitem__`
assignment idiom, optimises bulk NDArray writes, and fixes several
correctness issues.

### DSL kernels as first-class CTable columns

- **`add_computed_column()` accepts DSL kernels**: `@blosc2.dsl_kernel`-decorated
  functions can now back virtual computed columns directly, in addition to
  the existing string-expression form.  The column survives save/open
  round-trips via persisted `dsl_source`.
- **`add_generated_column()` accepts DSL kernels**: stored generated columns
  (written during `append`/`extend` and on `refresh_generated_column()`)
  now support DSL kernels as their transformer.
- **`CTable.where()` accepts UDF/DSL kernels**: filter predicates are no
  longer limited to expression strings — any DSL kernel can be passed directly.
- **`dtype` inference for DSL kernels**: when `dtype` is omitted,
  `lazyudf()` infers the output dtype via NumPy type promotion of the input
  column dtypes.  Pass `dtype` explicitly for type-changing kernels
  (comparisons, casts).
- **`kernel_from_source()` utility**: new `dsl_kernel.kernel_from_source()`
  reconstructs a `DSLKernel` from its stored source text, shared by the
  CTable DSL-column loaders and the persisted `LazyUDF` decoder.
- **Security note**: `.b2d` files from untrusted sources that contain DSL
  computed columns execute stored Python source on open.  A warning is now
  included in the documentation.

### New `CTable.__setitem__` column-assignment API

- **`t["col"] = arr`**: new shorthand equivalent to `t["col"][:] = arr`.
  Accepts any array-like including `blosc2.NDArray`.  Raises `KeyError` for
  unknown columns and `ValueError` for views or read-only tables.

### Chunked NDArray writes in `extend()` and `Column.__setitem__`

- **`extend({"col": ndarray})` decompresses chunk-by-chunk**: when a
  `blosc2.NDArray` is passed as a column value to `extend()`, it is now
  written in chunks instead of being fully decompressed upfront.  Pass
  `validate=False` to avoid a transient full decompression during constraint
  checking.
- **`col[:] = blosc2_ndarray` fast path**: a new no-holes fast path in
  `Column.__setitem__` skips the O(n) validity-mask gather and writes the
  NDArray one chunk at a time using contiguous slice writes.  Works for both
  scalar and fixed-shape ndarray columns.  Falls back to a chunked fancy-index
  path when deleted rows are present.

### `BLOSC_ME_JIT` environment variable override

- **Full CLI override**: `BLOSC_ME_JIT` now takes unconditional priority over
  both the `jit=` and `jit_backend=` keyword arguments, making it easy to
  switch JIT backends from the command line without modifying code.

### Correctness fixes

- **View corruption in `Column.__setitem__`**: a `None == None` guard
  evaluation on view-backed columns could fire the NDArray fast path,
  bypassing physical-position remapping and silently corrupting rows.  Fixed
  by explicitly checking `base is None` before activating the fast path.
- **`CTable.__setitem__` view guard**: the new `t["col"] = arr` API now
  raises `ValueError` on views, matching the contract of all other mutating
  CTable methods.
- **Fast path enabled for disk-opened tables**: the fast path previously
  remained dormant for tables opened from disk because `_last_pos` starts as
  `None`.  The guard now calls `_resolve_last_pos()` to lazily initialise it.
- **DSL column `jit_backend` preserved in `_empty_copy`**: the `jit_backend`
  setting was silently dropped during internal table copies; it is now
  retained.
- **`lazyexpr` Column unwrapping**: `convert_inputs()` now automatically
  unwraps `CTable.Column` objects to their backing NDArray so that shape and
  identity checks work correctly.

### Documentation and examples

- **Parquet-to-blosc2 walkthrough**: new step-by-step tutorial added to the
  getting-started section. Thanks to @SyedIshmumAhnaf.
- **CTable performance tips**: new section in the overview covering when to
  prefer computed vs. generated columns, chunk sizing, and query optimisation.
- **Simplified docstring examples**: examples throughout `ndarray.py` and
  `ctable.py` now use `blosc2.array()`, `blosc2.arange()`, and
  `blosc2.linspace()` directly instead of two-step numpy-then-`asarray`
  patterns.
- **`udf-computed-col.py` example**: new end-to-end example demonstrating DSL
  kernel computed and generated columns.

## Changes from 4.3.3 to 4.4.1

This is a feature release focused on a new interactive data viewer, automatic
SUMMARY indexes for fast WHERE queries, chunk-aligned Arrow/Parquet imports,
expanded `where()` acceleration via miniexpr, and a range of CTable ergonomics
and performance improvements.  Python 3.10 support has been dropped; Python
3.11 is now the minimum.

### b2view: interactive Text User Interface data viewer

- **New `b2view` command**: a terminal-based interactive viewer for all
  blosc2 containers — `NDArray`, `CTable`, `SChunk`, `BatchArray`, and more.
  Launch it with `b2view <file>` or as `blosc2.b2view()` from Python.
- **Full 1-D and 2-D browsing**: arrays with more than two dimensions are
  sliceable along any axis; 1-D arrays are shown as a single-column table.
- **CTable navigation**: scroll through rows with keyboard shortcuts; `t`/`b`
  jump to the top/bottom; `--panel` jumps straight to a named panel on launch.
- **`CTable.vlmeta` panel**: variable-length metadata is exposed in a dedicated
  panel.
- **New dim mode**: navigate along all dimensions freely for N-D arrays.

### SUMMARY indexes for fast WHERE queries

- **Automatic SUMMARY index creation**: when a `CTable` is closed after a
  write session, SUMMARY indexes (per-block min/max) are built by default for
  all eligible scalar columns with no extra configuration needed.
- **Incremental build during writes**: indexes are accumulated block-by-block
  during `extend()` and Arrow import, so closing the table costs almost nothing
  beyond the write already done.
- **Block-skip prefilter**: the miniexpr prefilter uses SUMMARY bitmaps to skip
  entire blocks whose min/max range cannot satisfy the WHERE predicate, reducing
  decompression work for selective queries.
- **Conjunction support**: per-column SUMMARY block masks are combined with
  bitwise AND so multi-column conjunctions prune blocks efficiently.
- **Cost gate**: a cost model guards index use; the SUMMARY path is skipped when
  block skipping is unlikely to help (e.g. very low selectivity).
- **`--no-summary-index`**: new CLI flag for `parquet-to-blosc2` to disable
  automatic index creation on import.

### CTable column grid alignment

- **Shared chunk/block grid for scalar columns**: fixed-size columns are now
  written on a shared chunk/block grid derived from the numeric column widths,
  so all columns have identical chunk boundaries.  This makes multi-column
  SUMMARY scans and chunk-parallel reads significantly faster.
- **Chunk-aligned Arrow import**: incoming Arrow/Parquet batches are buffered
  and flushed in exact chunk-sized blocks, so each chunk is compressed exactly
  once instead of being split across batch boundaries.
- **Vectorized dictionary-column import**: dictionary codes are now written in
  bulk at full chunk capacity rather than element by element.
- **Small fixed strings on the grid**: fixed-length string columns narrow enough
  to share the numeric grid are admitted to it, reducing the number of distinct
  chunk sizes.
- **`--reduce-mem`**: new CLI option for `parquet-to-blosc2` to cap the Arrow
  read-batch size on nested `list<struct>` imports, keeping peak RSS low at a
  modest speed cost.

### CTable.copy() enhancements

- **C-level bulk copy for `ListArray` and `BatchArray`**: a new `chunk_copy()`
  method transfers pre-compressed chunks directly at the C level, bypassing
  Python-level serialization and recompression.  `CTable.copy()` uses this path
  automatically.
- **`chunks=` / `blocks=` overrides in `CTable.copy()`**: callers can now
  specify target chunk and block sizes for the output copy.
- **`cparams` and `blocks` overrides**: `CTable.copy()` accepts `cparams` and
  `blocks` to recompress the copy with different settings.
- **`--chunks` / `--blocks`** added to the `parquet-to-blosc2` CLI.

### Take/gather APIs

- Added `NDArray.take()` following Array API `take` shape semantics, including
  `axis=None` flattening and N-dimensional integer indices.  One-dimensional
  gathers use a new sparse C-level path (`b2nd_get_sparse_cbuffer`) internally.
- Extended top-level `blosc2.take()` to dispatch to `NDArray.take()`,
  `CTable.take()`, and `Column.take()` while preserving the input container
  type.
- Added `CTable.take()` and `Column.take()` for logical row/value gathers that
  preserve order and duplicate indices, unlike mask-based views.
- For `ndim > 1` axis-based take, orthogonal selection is used internally for
  better performance.

### where() and miniexpr acceleration

- **`where(cond, x)` via miniexpr**: the single-argument `where` (fill-with-zero
  variant) is now handled directly by the miniexpr engine when the condition is
  a boolean array, avoiding a numexpr round-trip.
- **`where(cond, x, y)` via miniexpr**: the two-argument flavor is likewise
  dispatched to miniexpr for element-wise conditional selection.
- **Sparse boolean mask fast path**: when a boolean indexing result is very
  sparse (high selectivity), auto-detection switches to a fast gather path
  instead of a full-array scan.
- **Early boolean key check**: `NDArray.__getitem__` with a boolean array key
  now detects it before the general `process_key` / `nonzero` path, avoiding
  wasted work.
- **Compressed transient masks**: temporary boolean masks created during
  queries are now stored as LZ4-compressed blosc2 arrays, reducing memory
  pressure without measurable speed regression.
- **`BLOSC_ME_JIT` / `BLOSC_ME_JIT_TRACE`**: new environment variables to
  control and trace the miniexpr JIT backend at runtime.

### CTable views and lazy sorting

- **`sort_by()` on a view is now lazy**: calling `sort_by()` on a filtered view
  returns a position-reordered view without materializing data; the sort
  positions are cached and used directly on column access.
- **Lazy column materialization in filtered views**: `select()` on a view no
  longer materializes unneeded columns eagerly; columns are resolved only when
  accessed.

### NestedColumn and .info improvements

- **`NestedColumn` public class**: the previously internal
  `_NestedColumnNamespace` has been renamed and promoted to `NestedColumn`,
  providing aggregate metadata (`col_names`, `nrows`, `nbytes`, `cbytes`,
  `cratio`) and a structured `.info` report over a group of dotted columns.
- **Uniform `.info` across containers**: `Column.info`, `CTable.info`,
  `NestedColumn.info`, and related classes now follow a consistent field order
  (identity → shape/grid → sizes → content → compression params).

### Context manager support for blosc2.open()

- All objects returned by `blosc2.open()` — `NDArray`, `SChunk`, `CTable`,
  `BatchArray`, `ListArray`, and stores — now support the `with` statement.
  The `__exit__` method flushes and closes the underlying storage.

### Performance improvements and fixes

- **CTable.nrows stored persistently**: row counts are written to metadata on
  close and read back on open, avoiding a full column scan at startup.
- **Index sidecar loading from .b2z**: SUMMARY/BUCKET sidecars inside `.b2z`
  archives are now read in-place rather than extracted to a temporary directory,
  cutting open latency for indexed tables.
- **Compressed query cache**: the hot query-result cache is now stored
  LZ4-compressed, reducing its memory footprint with negligible overhead.
- **Query cache consistency fixes**: on-disk query cache side effects and a
  miniexpr chunk-cache race condition on Apple Silicon have been resolved.
- **macOS L2 floor for chunk sizing**: on macOS the full L2 cache is used as a
  floor for automatic chunk sizing, giving better compression/speed trade-offs.
- **Better Apple Silicon L3 handling**: missing L3 cache on Apple Silicon is
  handled more gracefully in the cache-size heuristic.
- **Table capacity management**: large CTables grow more conservatively, and
  capacity is trimmed on close and after Arrow import to reclaim over-allocated
  space.
- **Faster iteration with `iterchunks_info()`**: several hot loops switched to
  `iterchunks_info()` for lower overhead per chunk.
- **Cost-model index refinement threshold**: the previously hardcoded threshold
  for switching between index and scan has been replaced with a data-driven cost
  model.
- **Index prefetch reuse**: data already prefetched during an index lookup is
  reused in the refinement phase, avoiding redundant I/O.
- **Simplify index sidecar filenames** in _indexes/{col}/ directories.
- **`DictStore` embed disabled by default**: embedding a store inside a dict
  store is now opt-in (it was error-prone as the default).
- **Fixed wasm32 issue**: a 32-bit platform arithmetic fix for reduce operations.
- **Chunks never exceed array dimension**: `compute_chunks_blocks` now
  guarantees chunk dimensions are capped at the array shape dimension.
- **`max_rows` robust to older PyArrow**: truncation logic no longer depends on
  PyArrow APIs that are absent in older releases.
- **`cratio` display**: compression ratio is now shown with an explicit `x`
  suffix (e.g. `2.47x`) throughout `.info` output.
- Updated bundled C-Blosc2 to the latest release.

### Dropped Python 3.10 support

- Python 3.11 is now the minimum supported version.

## Changes from 4.3.1 to 4.3.3

note: 4.3.2 was an internal pre-release that was not published to PyPI.

This is a maintenance release focused on CTable display ergonomics, indexed-query
correctness, and query-planner performance.

### CTable display and print options

- **Pandas-like CTable display by default**: `str(table)` / `print(table)` now use
  a compact, pandas/DuckDB-style table representation, including a displayed
  logical row index, numeric alignment, compact spacing, and a trailing footer
  such as `[726017 rows x 5 columns]`.
- **Configurable display options**: added `blosc2.set_printoptions()` and
  `blosc2.get_printoptions()` for CTable rendering.  The supported options are
  `display_index`, `display_rows`, `display_precision`, and `fancy`.
- **`CTable.to_string()`**: added a one-off formatting API for producing CTable
  string representations without changing global print options.
- **Compact truncation for large tables**: when a table exceeds the configured
  `display_rows` threshold, only the first five and last five rows are shown,
  with an ellipsis row in between.
- **Float display refinements**: compact mode uses pandas-like fixed precision
  for floating-point columns, and integer-valued float columns are displayed
  with a single decimal place.
- **Fancy display preserved**: `set_printoptions(fancy=True)` restores the more
  decorated display with dtype rows, separator rules, and hidden row/column
  counts.

### Indexed queries and sorting

- **Cross-column exact index refinement**: multi-column conjunctions can now use
  exact positions from a selective indexed column (`FULL`, `PARTIAL`, or `OPSI`)
  as a compact pre-filter, then refine the remaining predicates on those
  positions instead of scanning the full table.
- **NaN-safe index boundary navigation**: fixed sorted-boundary navigation for
  floating-point indexes containing `NaN` values, so indexed results match scan
  results for bucket/full index lookups.
- **Better index-planner heuristics**: the planner now avoids low-value indexed
  paths when segment pruning is unlikely to help, and avoids expensive scalar
  specialization for non-scalar arrays.
- **Faster filtered sorting**: small filtered views can be materialized and
  sorted directly, avoiding an extra gather of sort keys.

### Performance and fixes

- Avoid full materialization of `valid_rows` in several CTable code paths.
- Keep row counts lazy for views and avoid unnecessary `nrows` calls in the
  query planner.
- Reduced overhead in root-column iteration and small query-planner operations.
- Fixed dictionary-column capacity handling during Arrow import and a regression
  affecting dictionary columns.
- Marked additional long-running tests as `heavy` to reduce default test-suite
  runtime.

### Documentation

- Updated the containers tutorial with dedicated `ListArray` and `CTable`
  sections, including CTable's columnar storage model and support for columns
  backed by `NDArray`, `BatchArray`, `ObjectArray`, `ListArray`, and related
  containers.

## Changes from 4.3.0 to 4.3.1

This is a maintenance release focused on CTable nested-column ergonomics,
grouped reductions, and API/documentation polish.

### CTable nested columns and grouped reductions

- **Nested column names in `group_by()` results**: grouped output columns can now
  preserve dotted/nested names such as `trip.sec` instead of requiring valid
  Python identifiers.
- **Column-object selectors**: `CTable.group_by()` and `CTable.sort_by()` now
  accept `Column` objects as well as string names, enabling idioms such as
  `t.group_by(t.trip.sec)` and `t.sort_by(t.trip.sec)`.
- **Grouped arg reductions**: `CTableGroupBy` now supports `argmin()` and
  `argmax()`, plus `agg({"col": "argmin"})` / `agg({"col": "argmax"})`.
  Results are logical row positions in the grouped table or view; groups with no
  non-null values return `-1`.

### NDArray constructor ergonomics

- **`blosc2.array()`**: added a NumPy-like constructor for NDArrays.  It mirrors
  `blosc2.asarray()` but defaults to `copy=True`, so passing an existing
  `NDArray` creates a copy unless `copy=False` or `copy=None` is requested.

### Documentation

- Expanded the CTable reference with `RowTransformer`, `Column.row_transformer`,
  and `CTableGroupBy.argmin` / `argmax` documentation.
- Added `blosc2.ndarray()`, `blosc2.dictionary()`, and related public schema
  factory functions to the Schema Specs reference.
- Moved `blosc2.group_reduce()` into the Reduction Functions reference and
  updated its example to use Blosc2 NDArrays.

## Changes from 4.2.0 to 4.3.0

### CTable: N-dimensional (ndarray) columns

- **Multidimensional columns**: CTable columns can now hold NDArray-backed cells, allowing
  each row of a column to contain a full n-dimensional compressed array.  This enables
  use cases such as embedding vectors, image patches, time-series windows, or any other
  multidimensional per-row payload.
- **CSV and DataFrame import/export**: Multidimensional column data can be imported and
  exported via CSV and pandas DataFrames, with automatic detection of array-valued cells.
- **Nullable ndarray columns**: Multidimensional columns fully support the nullable
  semantics (`null_count`, sentinel handling, `null_policy`) already available for scalar
  columns.
- **`from_pandas()` improvements**: `CTable.from_pandas()` now creates the correct
  specialized backing storage for `DictionarySpec`, `ListSpec`, `VLStringSpec`,
  `VLBytesSpec`, and other variable-length scalar specifications.
- **Improved schema coverage**: New CTable timestamp schema type and extended
  `Column.info` output with `shape`, `chunks`, and `blocks` descriptors.
- **Arg reductions**: Added `argmin()` and `argmax()` for scalar and ndarray
  CTable columns, plus row-transformer support for generated columns such as
  per-row peak-hour or dominant-embedding-dimension features.

### CTable: Group-by and filtered aggregation

- **`CTable.group_by()`**: The primary group-by interface.  Call
  ``t.group_by("city", sort=True).agg({"qty": "mean"})`` to produce a new
  :class:`CTable` with aggregated results.  Single-key and multi-key groupings are
  supported, along with convenience methods such as ``.size()``, ``.count()``,
  ``.sum()``, ``.mean()``, ``.min()`` and ``.max()``:

  .. code-block:: python

      by_city = t.group_by("city", sort=True)
      by_city.size()  # COUNT(*)
      by_city.sum("sales")  # SUM(sales) per city
      by_city.agg({"sales": ["sum", "mean"]})  # SUM(sales), AVG(sales) per city

- **Performance accelerators**: Dedicated Cython fast paths deliver significant speedups:
  ~25× for float32/64 group-by keys, ~8× for integer and dictionary-code keys, and a
  general-purpose hash table for arbitrary float keys.
- **Filtered aggregate pushdown**: The `where=` parameter is now accepted in aggregation
  methods, pushing the filter into the compute engine so that only matching rows are
  read and reduced.
- **Persistent grouped output**: Group-by results can be saved directly to persistent
  storage via the `urlpath=` parameter.
- **`blosc2.group_reduce()`**: New public function that performs group-by reduction over
  NDArray instances and CTable columns, with Cython-accelerated backends for common
  key/reduction combinations.

### CTable: Dictionary / categorical columns

- **`DictionarySpec` column type**: Introduced a new dictionary-encoded (categorical)
  column type that stores string or integer codes mapped to a shared dictionary, providing
  compact storage and accelerated equality and membership queries.
- **Dictionary types in `where` clauses**: Dictionary columns can be queried with the same
  `where=` expression syntax as other column types, including nested dotted-name access.
- **Improved display**: `CTable` printing now adapts to the terminal width, and dictionary
  values are shown in their decoded form.  `Column.info` has been extended with type
  details, shape, chunks, and blocks.

### CTable: Nested columns and field-name escaping

- **Dotted nested column access**: Columns whose names contain literal `.`
  (e.g., `"root.nested"`) are now fully addressable via the dotted accessor syntax in
  `where` expressions, `__getitem__`, and the public API.
- **Hierarchical `_cols` storage paths**: The internal column storage layout now preserves
  a hierarchical structure that mirrors the logical nesting, improving introspection
  and interop.
- **Nested-field pipeline**: A new flattened-storage pipeline with logical mapping
  preserves nested schema structure (field names, types, and hierarchy) through
  Arrow and Parquet import/export.  For unnamed top-level `list<struct<...>>` Parquet
  files, the logical schema round-trips faithfully, though the original physical row
  grouping is intentionally not preserved.
- **Field-name escaping**: Special characters (`.` and `/`) in column names are
  automatically escaped during schema construction and metadata round-trips.

### Parquet import/export improvements

- **Arrow serializer by default**: `CTable.from_parquet()` now defaults to the Arrow
  serializer, providing better schema fidelity and nested-type support.
- **Progress reporting**: A `--progress` flag and an ETA estimator have been added to
  the `parquet-to-blosc2` CLI for long-running imports.
- **`--max-rows` parameter**: `CTable.from_parquet()` and the CLI now accept `max_rows`
  to limit the number of imported rows.
- **`--timestamp-unit`**: New CLI option to control timestamp unit conversion on import.
- **`--float-trunc-prec`**: New CLI option to truncate floating-point precision on import.
- **Separated nested columns enabled by default**: The `separate_nested_cols` flag is now
  `True` by default for both the Python API and the CLI, ensuring nested Arrow structs
  are always expanded into flat columns.
- **`list_serializer` parameter**: New option to control how list-type columns are
  serialized, with sensible defaults for different list layouts.
- **Validation optimizations**: Arrow datetime values are validated only during import,
  reducing runtime overhead on subsequent operations.

### TreeStore: Inline CTable support

- **CTables inside TreeStore**: `CTable` objects can now be stored inline as items
  inside a `TreeStore`, enabling hierarchical storage that mixes arrays and tables in a
  single persistent container.
- **Cache hardening**: TreeStore cache assignments now use defensive copies and cache
  effective object roots to avoid aliasing and stale-cache errors.
- **Examples and tutorials**: New tutorials and docstring examples demonstrate how to
  store, retrieve, and query CTables within a TreeStore.

### Performance and usability enhancements

- **Faster open and import**: `blosc2.open()` and store constructors now assume valid
  file extensions and defer column metainfo loading, making `CTable.open()` and
  package import noticeably faster.
- **`CTable.nrows` is now lazy**: The row count is computed on demand rather than eagerly,
  speeding up open and schema-inspection workflows.
- **Accelerated scalar and small-slice access**: The batch/list path for reading scalar
  values or small column slices has been overhauled, eliminating internal placeholder
  materialization and yielding lower latency.
- **Late-import optimizations**: Heavy optional dependencies are imported lazily at the
  blosc2 package level, reducing the baseline `import blosc2` overhead.
- **`iter_arrow_batches()` optimization**: Avoids full Python object materialization of
  batches during iteration, reducing memory pressure.
- **`NDArray`-to-list conversion**: Small optimization when converting NDArray objects
  to Python lists.
- **`_last_pos` invalidation skipped**: Mid-table deletes no longer eagerly invalidate
  cached positional state, improving delete latency.

### Documentation, examples and benchmarks

- **API reference expanded**: `blosc2.group_reduce()` has been added to the Sphinx
  reference, along with updated CTable, Column, and TreeStore pages.
- **New tutorials and examples**: Added sections on CTable–TreeStore integration,
  nested fields, dictionary columns, aggregates, grouping and querying with `where=`.
- **New benchmarks**: Graph benchmarks for CTable insert time, column count, memory usage,
  and `where=` queries, plus dedicated group-by, nested-filter, and Parquet round-trip
  benchmarks.

### Fixes and compatibility

- **Null and NaN handling**: NumPy scalar null sentinels are now normalized to plain Python
  scalars, and floating-point NaN sentinels are treated consistently with Python
  `float('nan')`.
- **Empty aggregate results**: Filtered aggregations that produce no rows now handle the
  empty result gracefully.
- **Generated column safety**: Accessing a stalled (unfillable) generated column now raises
  a clear exception instead of producing undefined results.
- **Miniexpr bundling**: Miniexpr’s bundled `libtcc` and related runtime files are now
  kept inside the `blosc2` package, avoiding conflicts with other TCC installations.
- **Test improvements**: Torch-dependent tests are marked as `heavy`, PyArrow-optional
  tests are skipped when the library is absent, and parametrization matrices have been
  trimmed to reduce CI time.
- **Missing Cython validation**: Added validation guards for several Cython extension
  functions that previously lacked explicit error checking.
- **C-Blosc2 update**: Bundled C-Blosc2 has been updated to the latest version (3.0.3).
- **``blosc2.open()`` default mode changed from 'a' to 'r'**: Removed the FutureWarning that
  was added to prepare for this transition.

## Changes from 4.1.2 to 4.2.0

### CTable: columnar compressed tables

- Introduced `blosc2.CTable`, a new columnar table container for compressed, typed columns.  CTables support dataclass- and schema-based construction, row iteration, column access, table views, `head()` / `tail()` / `sample()`, sorting, selection and compact `where` expressions.
- Added persistent CTables backed by `TreeStore`, with support for `blosc2.open()`, `CTable.open()`, `CTable.load()`, `CTable.save()`, `CTable.to_b2d()` and `CTable.to_b2z()`.  CTable views can be saved too, and `.b2z`/`.b2d` path handling has been tightened.
- Added mutation operations for CTables, including `append()`, `extend()`, `delete()`, `compact()`, `add_column()`, `drop_column()`, `rename_column()` and related schema validation.
- Added computed columns, including virtual computed columns backed by lazy expressions, materialized computed columns and automatic filling of materialized computed columns during inserts.
- Added CTable indexing support, including persistent indexes, direct expression indexes, ordered index reuse, boolean `LazyExpr`/`NDArray` masks in `CTable.__getitem__`, `iter_sorted()` and indexing support for `.b2z` tables.
- Added nullable schema support and null policies for CTable scalar columns, preserving nullable scalar Parquet round-trips.
- Added variable-length CTable column support via `ListArray` / `ObjectArray`, including `vlstring` and `vlbytes` schema specs, fixed-length string/bytes import support and list/struct Arrow/Parquet round-trips.
- Added Arrow, Parquet and CSV interoperability for CTables, including batch-wise Arrow/Parquet import/export, Arrow schema metadata preservation, `CTable.from_arrow_batches()` improvements and a new `parquet-to-blosc2` CLI utility.
- Added CTable documentation, tutorials, examples and benchmarks covering schema definition, persistence, querying, indexing, mutations, nullable columns, computed columns and variable-length columns.

### Indexing and ordering

- Added a new indexing subsystem for NDArrays and CTables, including full, partial/bucket, light/medium and OPSI-style index kinds, out-of-core index builders and sidecar storage.
- Added `blosc2.Index` as the unified public index handle, plus APIs such as `create_index()`, `compact_index()`, `iter_sorted()`, `will_use_index()` and related query explanation support.
- Added materialized expression indexes for NDArrays and direct expression indexes for CTables.
- Added persistent query-result caching for indexed lookups, with FIFO pruning and cache accounting.
- Added `blosc2.argsort()` and refactored indexing APIs around explicit index enums and sorting helpers.
- Improved indexed query performance with Cython accelerators, threaded chunk batching, zero-copy/cached mmap reads, chunk-aware and reduced-order layouts and faster scattered row gathering.
- Reduced memory usage during index creation and lookup by avoiding full sidecar materialization, replacing memmap staging with Blosc2 scratch arrays and adding `tmpdir` support for full out-of-core indexes.

### Persistence, stores and serialization

- Added structured Blosc2 serialization based on b2object carriers, including persisted `C2Array`, `LazyExpr` and DSL `LazyUDF` objects.
- Added `blosc2.Ref` for serializing external references, plus examples for b2object bundles and persisted expressions/UDFs.
- Added `blosc2.load()` as a convenience loader.
- Added `vlmeta` support to `LazyArray` objects.
- Improved store handling by preserving lazy b2object carriers in `DictStore`, allowing reopened proxies to refill caches after read-only opens, relaxing `DictStore`/`TreeStore` suffix requirements and adding `DictStore.to_b2d()`.
- Accelerated `blosc2.open()` by trying standard opens first and warning on implicit append mode.

### Arrays, computation and containers

- Added `ObjectArray` for fully general object data and renamed the earlier `VLArray` work accordingly; added `ListArray` docstrings and Arrow integration improvements.
- Added schema helpers including numeric specs, `blosc2.struct()` and `blosc2.object()` for nested/fully general column declarations.
- Improved `fromiter()` with direct chunked construction and substantially lower peak memory use.
- Improved `asarray()` behavior for NDArray inputs when copy-inducing keyword arguments are supplied.
- Added `SChunk.reorder_offsets()`.
- Improved `BatchArray` defaults and documentation; the default compression level is now tuned for faster lookup/scan behavior.
- Continued matmul/linalg optimization work and shared-thread-pool integration.

### CLI, docs and examples

- Added the `parquet-to-blosc2` command with options such as `--max-rows`, `--parquet-batch-size`, `--blosc2-items-per-block` and `--use-dict`.
- Added new CTable, ObjectArray, BatchArray, containers, indexing and serialization tutorials and examples.
- Reorganized and expanded the API reference for CTable, Column, schema specs, Index, save/load helpers and miscellaneous APIs.
- Updated benchmark suites for CTables, indexing, Parquet import/export, BatchArray and NDArray construction/indexing.

### Fixes and compatibility

- Updated bundled C-Blosc2 to v3.0.2 and require C-Blosc2 >= 3.0.0 when building against a system library.
- Updated bundled C-Blosc2 and miniexpr sources multiple times.
- Restored compatibility with NumPy < 2.
- Fixed Windows and mmap/file-locking issues in index creation, rebuilds and temporary file cleanup.
- Fixed full-index query failures for large CTable columns and full out-of-core merge failures on systems with small `/tmp`.
- Fixed stale sidecar/cache reuse and targeted cache invalidation when persistent sidecars are replaced.
- Fixed `.b2z` double-open corruption caused by GC-triggered repacking and made temporary `.b2z` unpacking default to the source file directory.
- Fixed a regression when reopening persisted proxies in read-only mode.
- Fixed GC-induced thread hangs on macOS with Python 3.14 and hardened async chunk reading/cache cleanup paths.
- Fixed lazy-chunk source-size handling in decode/getitem callers.
- Fixed nullable validation, dictionary extend validation, CTable close propagation, print alignment and NumPy mask support.
- Fixed `arange()` regressions and several pre-existing `set_slice` error-handling issues.
- Clamped indexing/thread defaults for wasm32.

## Changes from 4.1.1 to 4.1.2

- A new fast path for src/blosc2/linalg.py that uses the matmul prefilter machinery in src/blosc2/blosc2_ext.pyx.
  - The fast path is only used for supported cases:
      - blosc2.NDArray inputs
      - 2-D only
      - floating-point only
      - matching dtypes
      - aligned chunk/block layouts that satisfy the current kernel assumptions
  - All other valid cases fall back to the existing chunk-by-chunk implementation in src/blosc2/linalg.py.
  - Some benchmarks for the supported cases show significant speedups over the chunked implementation:
    - aligned 400x400 float32: about 3.7x faster over chunked
    - aligned 400x400 float64: about 3.0x
    - aligned 800x800 float32: about 1.5x
    - misaligned case: auto correctly stays on chunked

## Changes from 4.1.0 to 4.1.1

- Update ``miniexpr`` to fix bug on ubuntu with ARM64.

## Changes from 4.0.0 to 4.1.0

- Add DSL kernel functionality for faster, compiled, user-defined functions which broadly respect python syntax and implement the `LazyArray` interface. See the introductory tutorial at: https://blosc.org/python-blosc2/getting_started/tutorials/03.lazyarray-udf-kernels.html
- Add read-only mmap support for store containers:
  `DictStore`, `TreeStore`, and `EmbedStore` now accept `mmap_mode="r"`
  when opened with `mode="r"` (including via `blosc2.open` for `.b2d`,
  `.b2z`, and `.b2e`).
- New .meta entry for store containers, allowing better store recognition at `blosc2.open()` time.  Fixes #546.
- Add `cumulative_sum` and `cumulative_prod` functions for Array API compliance.
- Add Unicode string arrays, support comparison operations with them, and optimised compression path.
- Add ``endswith`` and ``startswith`` and extend ``contains`` to support strings and offer `miniexpr` multithreaded computation when possible.
- Use DSL kernels to accelerate `arange`/`linspace` constructors by 6-10x.
- Improve documentation for `filters` and `filters_meta`.
- Fix edge case issues with `resize` and `constructors` so that `chunks` may be set independently of shape, and arrays may be extended from empty consistently.
- Continued work on `miniexpr` integration, interface, and support.
- Ruff fixes and implementation of PEP recommendations.

## Changes from 4.0.0-b1 to 4.0.0

- On Windows, miniexpr is temporarily disabled for integral outputs and mixed-dtype expressions.
  Set `BLOSC2_ENABLE_MINIEXPR_WINDOWS=1` to override this for testing.
- Handle thread workers for computation to ensure never exceeds NUMEXPR_MAX_THREADS. Thanks @skmendez!

## Changes from 3.12.2 to 4.0.0-b1

- PEP 427 compatibility changes to ensure C-blosc2 files and binaries are stored under blosc2/ subdirectories in shipped Python wheels
- Introduce miniexpr for hyper-fast multithreaded element-wise computations and reductions (on macOS and Linux). This justifies the major version number bump.
- Indexing with None for LazyExpr now matches Numpy behaviour (i.e. newaxis)
- Improvements to open and generally handle Treestore objects and b2z, .b2d, .b2e files. Thanks @bossbeagle1509!
- Minor changes to support new blosc2-openzl plugin

## Changes from 3.12.1 to 3.12.2

* Hotfix to change WASM wheel hosting to separate repo

## Changes from 3.12.0 to 3.12.1

* Hotfix for security - disallow ``import`` in (saved) ``LazyUDF`` objects
* Automate WASM wheel upload via YAML file

## Changes from 3.11.1 to 3.12.0

* `LazyUDF` objects can now be saved to disk
* Calls to ``__matmul__`` NumPy ufunc now passed to ``blosc2.matmul``
* Streamlined ``LazyUDF.compute`` is now much more robust and functional
* The ``get_chunk`` method for ``LazyExpr`` is more efficient and enabled for general ``LazyArray`` objects
* ``LazyExpr`` calculation can now be done even with expressions with pure scalar operands, e.g ``10 * 3 +1.``.

## Changes from 3.11.0 to 3.11.1

* Change the `NDArray.size` to return the number of elements in array,
  instead of the size of the array in bytes. This follows the array
  API, so it is considered a fix, and takes precedence over a possible
  backward incompatibility.
* Tweak automatic chunk sizing of results for certain (e.g. linalg) operations
  to enhance performance
* Bug fixes for lazy expressions to allow a wider range of functionality
* Small bug fix for slice indexing with step larger than chunksize
* Various cosmetic fixes and streamlining (thanks to the indefatigable @DimitriPapadopoulos)

## Changes from 3.10.2 to 3.11.0

* Small optimisation for chunking in lazy expressions
* Extend Blosc2 computation machinery to accept general array inputs (PR #510)
* Refactoring and streamlining of get/setitem for non-unit steps (PR #513)
* Remote array testing now performed with `cat2cloud` (PR #511)
* Added argmax/argmin functions (PR #514)
* Change `squeeze` to return view (rather than modify array in-place) (PR #518)
* Modify `setitem` to load general array inputs into NDArrays (PR #517)

## Changes from 3.10.1 to 3.10.2

* LazyExpr.compute() now honors the `out` parameter for regular expressions (and not only for reductions).  See PR #506.

## Changes from 3.10.0 to 3.10.1

* Bumped to numexpr 2.14.1 to improve overflow behaviour for complex arguments for ``tanh`` and ``tanh``
* Bug fixes for lazy expression calculation
* Optimised computation for non-blosc2 chunked array arguments (e.g. Zarr, HDF5)
* Various cleanups and most importantly shipping of python 3.14 wheels due to @DimitriPapadopoulos!
* Now able to use blosc2 in AWS Lambda

## Changes from 3.9.1 to 3.10.0

* Improved documentation on thread management (thanks to [@orena1](@orena1) in PR #495)
* Enabled direct ingestion of Zarr arrays, and added examples for xarray ingestion
* Extended string-based lazy expression computation using a shape parser and modified lazy expression machinery so that expressions like "matmul(a, b) + c" can now be handled (PR #496).
* Streamlined inheritance from ``Operand`` to ensure access to basic methods like ``__add__`` for all computable objects (``NDArray``, ``LazyExpr``, ``LazyArray`` etc.) (PR ##500).

## Changes from 3.9.0 to 3.9.1

* Bumped to numexpr 2.13.1 to incorporate new maximum/minimum NaN handling and +/* for booleans
  which matches NumPy behaviour.
* Refactoring in order to ensure Blosc2 functions with NumPy 1.26.
* Streamlined documentation by introducing Array Protocol

## Changes from 3.8.0 to 3.9.0
Most changes come from PR #467 relating to array-api compliance.

* C-Blosc2 internal library updated to latest 2.21.3, increasing MAX_DIMS from 8 to 16

* numexpr version requirement pushed to 2.13.0 to incorporate
``round``, ``sign``, ``signbit``, ``copysign``, ``nextafter``, ``hypot``,
``maximum``, ``minimum``, ``trunc``, ``log2`` functions, as well as allow
integer outputs for certain functions when integr arguments are passed.
We also add floor division (``//``) and full dual bitwise (logical) AND, OR, XOR, NOT
support for integer (bool) arrays.

* Extended linear algebra functionality, offering generalised matrix multiplication
for arrays of arbitrary dimension via ``tensordot`` and an improved ``matmul``. In addition,
introduced ``vecdot``, ``diagonal`` and ``outer``, as well as useful indexing and associated functions such as ``take``, ``take_along_axis``, ``meshgrid`` and ``broadcast_to``.

* Added many ufuncs and methods (around 60) to ``NDArray`` to bring the library into further alignment with the array-api. Introduced a chunkwise lazyudf paradigm which is very powerful in order to implement ``clip`` and ``logaddexp``.

* Fixed a subtle but important bug for ``expand_dims`` (PR #479, PR #483) relating to reference counting for views.

## Changes from 3.7.2 to 3.8.0

* C-Blosc2 internal library updated to latest 2.21.2.

* numexpr version requirement pushed to 2.12.1 to incorporate
``isnan``, ``isfinite``, ``isinf`` functions.

* Indexing is now supported extensively and reasonably optimally for slices
with negative steps and general boolean arrays, with both get/setitem having
equal functionality. In PR #459 we extended the 1D fast path to general N-D,
with consequent speedups. In PR # we allowed fancy indexing and general slicing
with negative steps for set and getitem, with a memory-optimised path for setitem.

* Various attributes and methods for the ``NDArray`` class, as well as functions, have
been added to increase compliance with the array-api standard. In addition,
linspace and arange functions have been made more numerically stable and now strictly
comply even with difficult floating-point edge cases.

## Changes from 3.7.1 to 3.7.2

* C-Blosc2 internal library updated to latest 2.21.1.

* Revert signature of `TreeStore.__init__` for making benchmarks to get back
  to normal performance.

## Changes from 3.7.0 to 3.7.1

* Added `C2Array.slice()` method and `C2Array.nbytes`, `C2Array.cbytes`, `C2Array.cratio`, `C2Array.vlmeta` and `C2Array.info` properties (PR #455).

* Many usability improvements to the `TreeStore` class and friends.

* New section about `TreeStore` in basics NDArray tutorial.

* New blog post about `TreeStore` usage and performance at: https://www.blosc.org/posts/new-treestore-blosc2

* C-Blosc2 internal library updated to latest 2.21.0.

## Changes from 3.6.1 to 3.7.0

* Overhaul of documentation (API reference and Tutorials)

* Improvements to lazy expression indexing and in particular much more efficient memory usage when applying non-unit steps (PR #446).

* Extended functionality of ``expand_dims`` to match that of NumPy (note that this breaks the previous API) (PR #453).

* The biggest change is in the form of three new data storage classes (``EmbedStore``, ``DictStore`` and ``TreeStore``) which allow for the efficient storage of heterogeneous array data (PR #451). ``EmbedStore`` is essentially an ``SChunk`` wrapper which can be stored on-disk or in-memory; ``DictStore`` allows for mixed storage across memory, disk or indeed remote; and ``TreeStore`` is a hieracrhically-formatted version of ``DictStore`` which mimics the HDF5 file format. Write, access and storage performance are all very competitive with other packages - see [plots here](https://github.com/Blosc/python-blosc2/pull/451#issuecomment-3178828765).

## Changes from 3.6.0 to 3.6.1

* C-Blosc2 internal library updated to latest 2.19.1.

## Changes from 3.5.1 to 3.6.0

* Expose the `oindex` C-level functionality in Blosc2 for `NDArray`.

* Implement fancy indexing which closely matches NumPy functionality, using
`ndindex` library. Includes a fast path for 1D arrays, based on Zarr's implementation.

* A major refactoring of slicing for lazy expressions using `ndindex`. We have also
added support for slices with non-unit steps for reduction expressions, which has introduced
improvements that could be incorporated into other lazy expression machinery in the future.
More complex slicing is now supported.

* Minor bug fixes to ensure that Blosc2 indexing does not introduce dummy dimensions when NumPy does not,
and a more comprehensive `squeeze` function which squeezes specified dimensions.

## Changes from 3.5.0 to 3.5.1

* Reduced memory usage when computing slices of lazy expressions.
  This is a significant improvement for large arrays (up to 20x less).
  Also, we have added a fast path for slices that are small and fit in
  memory, which can be up to 20x faster than the previous implementation.
  See PR #430.

* `blosc2.concatenate()` has been renamed to `blosc2.concat()`.
  This is in line with the [Array API](https://data-apis.org/array-api).
  The old name is still available for backward compatibility, but it will
  be removed in a future release.

* Improve mode handling for concatenating to disk. See PR #428.
  Useful for concatenating arrays that are stored in disk, and allows
  specifying the mode to use when concatenating.

## Changes from 3.4.0 to 3.5.0

* New `blosc2.stack()` function for stacking multiple arrays along a new axis.
  Useful for creating multi-dimensional arrays from multiple 1D arrays.
  See PR #427. Thanks to [Luke Shaw](@lshaw8317) for the implementation!
  Blog: https://www.blosc.org/posts/blosc2-new-concatenate/#stacking-arrays

* New `blosc2.expand_dims()` function for expanding the dimensions of an array.
  This is useful for adding a new axis to an array, similar to NumPy's `np.expand_dims()`.
  See PR #427. Thanks to [Luke Shaw](@lshaw8317) for the implementation!

## Changes from 3.3.4 to 3.4.0

* Added C-level ``concatenate`` function in response to community request. When possible, uses an optimised path which avoids decompression and recompression, giving a significant performance boost. See PR #423.

* Slicing has been added to string-based lazyexprs, so that one may use
  expressions like `expr[1:3] +1` to compute a slice of the expression. This is useful
  for getting a sub-expression of a larger expression, and it works with both
  string-based and lazy expressions. See PR #417.

* Relatedly, the behaviour of the `slice` parameter in the `compute()` method of `LazyExpr` has been made more consistent and is now better documented, so that results are as expected. See PR #419.

* UDF support for pandas has been added to allow for the use of ``blosc2.jit``. See PR #418. Thanks to [@datapythonista](https://github.com/datapythonista) for the implementation!

## Changes from 3.3.3 to 3.3.4

* Expand possibilities for chaining string-based lazy expressions to incorporate
  data types which do not have shape attribute, e.g. int, float etc.
  See #406 and PR #411.

* Enable slicing within string-based lazy expressions. See PR #414.

* Improved casting for string-based lazy expressions.

* Documentation improvements, see PR #410.

* Compatibility fixes for working with `h5py` files.

## Changes from 3.3.2 to 3.3.3

* Expand possibilities for chaining string-based lazy expressions to include
  main operand types (LazyExpr and NDArray). Still have to incorporate other
  data types (which do not have shape attribute, e.g. int, float etc.).
  See #406.

* Fix indexing for lazy expressions, and allow use of None in getitem.
  See PR #402.

* Fix incorrect appending of dim to computed reductions. See PR #404.

* Fix `blosc2.linspace()` for incompatible num/shape.  See PR #408.

* Add support for NumPy dtypes that are n-dimensional (e.g.
  `np.dtype(("<i4,>f4", (10,))),`).

* New MAX_DIM constant for the maximum number of dimensions supported.
  This is useful for checking if a given array is too large to be handled.

* More refinements on guessing cache sizes for Linux.

* Update to C-Blosc2 2.17.2.dev.  Now, we are forcing the flush of modified
  pages only in write mode for mmap files. This fixes mmap issues on Windows.
  Thanks to @JanSellner for the implementation.

## Changes from 3.3.1 to 3.3.2

* Fixed a bug in the determination of chunk shape for the `NDArray` constructor.
  This was causing problems when creating `NDArray` instances with a CPU that
  was reporting a L3 cache size close (or exceeding) 2 GB.  See PR #392.

* Fixed a bug preventing the correct chaining of *string* lazy expressions for
  logical operators (`&`, `|`, `^`...).  See PR #391.

* More performance optimization for `blosc2.permute_dims`.  Thanks to
  Ricardo Sales Piquer (@ricardosp4) for the implementation.

* Now, storage defaults (`blosc2.storage_dflts`) are honored, even if no
  `storage=` param is used in constructors.

* We are distributing Python 3.10 wheels now.

## Changes from 3.3.0 to 3.3.1

* In our effort to better adapt to better adapt to the array API
  (https://data-apis.org/array-api/latest/), we have introduced
  permute_dims() and matrix_transpose() functions, and the .T property.
  This replaces to previous transpose() function, which is now deprecated.
  See PR #384.  Thanks to Ricardo Sales Piquer (@ricardosp4).

* Constructors like `arange()`, `linspace()` and `fromiter()` now
  use far less memory when creating large arrays. As an example, a 5 TB
  array of 8-byte floats now uses less than 200 MB of memory instead of
  170 GB previously.  See PR #387.

* Now, when opening a lazy expression with `blosc2.open()`, and there is
  a missing operand, the open still works, but the dtype and shape
  attributes are None.  This is useful for lazy expressions that have
  lost some operands, but you still want to open them for inspection.
  See PR #385.

* Added an example of getting a slice out of a C2Array.

## Changes from 3.2.1 to 3.3.0

* New `blosc2.transpose()` function for transposing 2D NDArray instances
  natively. See PR #375 and docs at
  https://www.blosc.org/python-blosc2/reference/autofiles/operations_with_arrays/blosc2.transpose.html#blosc2.transpose
  Thanks to Ricardo Sales Piquer (@ricardosp4) for the implementation.

* New fast path for `NDArray.slice()` for getting slices that are aligned with
  underlying chunks. This is a common operation when working with NDArray
  instances, and now it is up to 40x faster in our benchmarks (see PR #380).

* Returned `NDArray` object in `NDarray.slice()` now defaults to original
  codec/clevel/filters. The previous behavior was to use the default
  codec/clevel/filters.  See PR #378.  Thanks to Luke Shaw (@lshaw8317).

* Several English edits in the documentation.  Thanks to Luke Shaw (@lshaw8317)
  for his help in this area.

## Changes from 3.2.0 to 3.2.1

* The array containers are now using the `__array_interface__` protocol to
  expose the data in the array.  This allows for better interoperability with
  other libraries that support the `__array_interface__` protocol, like NumPy,
  CuPy, etc.  Now, the range of functions that can be used within the `blosc2.jit`
  decorator is way larger, and essentially all NumPy functions should work now.

  See examples at: https://github.com/Blosc/python-blosc2/blob/main/examples/ndarray/jit-numpy-funcs.py
  See benchmarks at: https://github.com/Blosc/python-blosc2/blob/main/bench/ndarray/jit-numpy-funcs.py

* The performance of constructors like `arange()`, `linspace()` and `fromiter()`
  has been improved.  Now, they can be up to 3x faster, specially with large
  arrays.

* C-Blosc2 updated to 2.17.1.  This fixes various UB as well as compiler warnings.

## Changes from 3.1.1 to 3.2.0

* Structured arrays can be larger than 255 bytes now.  This was a limitation
  in the previous versions, but now it is gone (the new limit is ~512 MB,
  which I hope will be enough for some time).

* New `blosc2.matmul()` function for computing matrix multiplication on NDArray
  instances.  This allows for efficient computations on compressed data that
  can be in-memory, on-disk and in the network.  See
  [here](https://www.blosc.org/python-blosc2/reference/autofiles/operations_with_arrays/blosc2.matmul.html)
  for more information.

* Support for building WASM32 wheels.  This is a new feature that allows to
  build wheels for WebAssembly 32-bit platforms.  This is useful for running
  Python code in the browser.

* Tested support for NumPy<2 (at least 1.26 series).  Now, the library should
  work with NumPy 1.26 and up.

* C-Blosc2 updated to 2.17.0.

* httpx has replaced by requests library for the remote proxy.  This has been
  done to avoid the need of the `httpx` library, which is not supported by
  Pyodide.

## Changes from 3.1.0 to 3.1.1

* Quick release to fix an issue with version number in the package (was reporting 3.0.0
  instead of 3.1.0).


## Changes from 3.0.0 to 3.1.0

### Improvements

* Optimizations for the compute engine. Now, it is faster and uses less memory.
  In particular, careful attention has been paid to the memory handling, as
  this is the main bottleneck for the compute engine in many instances.

* Improved detection of CPU cache sizes for Linux and macOS.  In particular,
  support for multi-CCX (AMD EPYC) and multi-socket systems has been implemented.
  Now, the library should be able to detect the cache sizes for most of the
  CPUs out there (specially on Linux).

* Optimization on NDArray slicing when the slice is a single chunk.  This is a
  common operation when working with NDArray instances, and now it is faster.

### New API functions and decorators

* New `blosc2.evaluate()` function for evaluating expressions on NDArray/NumPy
  instances.  This a drop-in replacement of `numexpr.evaluate()`, but with the
  next improvements:
  - More functionality than numexpr (e.g. reductions).
  - Follow casting rules of NumPy more closely.
  - Use both NumPy arrays and Blosc2 NDArrays in the same expression.

  See [here](https://www.blosc.org/python-blosc2/reference/autofiles/utilities/blosc2.evaluate.html)
  for more information.

* New `blosc2.jit` decorator for allowing NumPy expressions to be computed
  using the Blosc2 compute engine.  This is a powerful feature that allows for
  efficient computations on compressed data, and supports advanced features like
  reductions, filters and broadcasting.  See
  [here](https://www.blosc.org/python-blosc2/reference/autofiles/utilities/blosc2.jit.html)
  for more information.

* Support `out=` in `blosc2.mean()`, `blosc2.std()` and `blosc2.var()` reductions
  (besides `blosc2.sum()` and `blosc2.prod()`).


### Others

* Bumped to use latest C-Blosc2 sources (2.16.0).

* The cache for cpuinfo is now stored in `${HOME}/.cache/python-blosc2/cpuinfo.json`
  instead of `${HOME}/.blosc2-cpuinfo.json`; you can get rid of the latter, as
  the former is more standard (see PR #360). Thanks to Jonas Lundholm Bertelsen
  (@jonaslb).

## Changes from 3.0.0-rc.3 to 3.0.0

* A persistent cache for cpuinfo (stored in `$HOME/.blosc2-cpuinfo.json`) is
  now used to avoid repeated calls to the cpuinfo library.  This accelerates
  the startup time of the library considerably (up to 5x on my box).

* We should be creating conda packages now.  Thanks to @hmaarrfk for his
  assistance in this area.


## Changes from 3.0.0-rc.2 to 3.0.0-rc.3

* Now you can get and set the whole values of VLMeta instances with the `vlmeta[:]` syntax.
  The get part is syntactic sugar for `vlmeta.getall()` actually.

* `blosc2.copy()` now honors `cparams=` parameter.

* Now, compiling the package with `USE_SYSTEM_BLOSC2` envar set to `1` will use the
  system-wide Blosc2 library.  This is useful for creating packages that do not want
  to bundle the Blosc2 library (e.g. conda).

* Several changes in the build process to enable conda-forge packaging.

* Now, `blosc2.pack_tensor()` can pack empty tensors/arrays.  Fixes #290.


## Changes from 3.0.0-rc.1 to 3.0.0-rc.2

* Improved docs, tutorials and examples.  Have a look at our new docs at: https://www.blosc.org/python-blosc2.

* `blosc2.save()` is using `contiguous=True` by default now.

* `vlmeta[:]` is syntactic sugar for vlmeta.getall() now.

* Add `NDArray.meta` property as a proxy to `NDArray.shunk.vlmeta`.

* Reductions over single fields in structured NDArrays are now supported.  For example, given an array `sarr` with fields 'a', 'b' and 'c', `sarr["a"]["b >= c"].std()` returns the standard deviation of the values in field 'a' for the rows that fulfills that values in fields in 'b' are larger than values in 'c' (`b >= c` above).

* As per discussion #337, the default of cparams.splitmode is now AUTO_SPLIT. See #338 though.


## Changes from 3.0.0-beta.4 to 3.0.0-rc.1

### General improvements

* New ufunc support for NDArray instances. Now, you can use NumPy ufuncs on NDArray instances, and mix them with other NumPy arrays. This is a powerful feature that allows for more interoperability with NumPy.

* Enhanced dtype inference, so that it mimics now more NumPy than the numexpr one. Although perfect adherence to NumPy casting conventions is not there yet, it is a big step forward towards better compatibility with NumPy.

* Fix dtype for sum and prod reductions. Now, the dtype of the result of a sum or prod reduction is the same as the input array, unless the dtype is not supported by the reduction, in which case the dtype is promoted to a supported one. It is more NumPy-like now.

* Many improvements on the computation of UDFs (User Defined Functions). Now, the lazy UDF computation is way more robust and efficient.

* Support reductions inside queries in structured NDArrays. For example, given an array `sarr` with fields 'a', 'b' and 'c', the next `farr = sarr["b >= c"].sum("a").compute()` puts in `farr` the sum of the values in field 'a' for the rows that fulfills that values in fields in 'b' are larger than values in 'c' (b >= c above).

* Implemented combining data filtering, as well as sorting, in structured NDArrays. For example, given an array `sarr` with fields 'a', 'b' and 'c', the next `farr = sarr["b >= c"].indices(order="c").compute()` puts in farr the indices of the rows that fulfills that values in fields in 'b' are larger than values in 'c' (`b >= c` above), ordered by column 'c'.

* Reductions can be stored in persistent lazy expressions. Now, if you have a lazy expression that contains a reduction, the result of the reduction is preserved in the expression, so that you can reuse it later on. See https://www.blosc.org/posts/persistent-reductions/ for more information.

* Many improvements in ruff linting and code style. Thanks to @DimitriPapadopoulos for the excellent work in this area.

### API changes

* `LazyArray.eval()` has been renamed to `LazyArray.compute()`. This avoids confusion with the `eval()` function in Python, and it is more in line with the Dask API.

This is the main change in the API that is not backward compatible with previous beta. If you have code that still uses `LazyArray.eval()`, you should change it to `LazyArray.compute()`.  Starting from this release, the API will be stable and backward compatibility will be maintained.

### New API calls

* New `reshape()` function and `NDArray.reshape()` method allow to do efficient reshaping between NDArrays that follows C order. Only 1-dim -> n-dim is currently supported though.

* `New NDArray.__iter__()` iterator following NumPy conventions.

* Now, `NDArray.__getitem__()` supports (n-dim) bool arrays or sequences of integers as indices (only 1-dim for now). This follows NumPy conventions.

* A new `NDField.__setitem__()` has been added to allow for setting values in a structured NDArray.

* `struct_ndarr['field']` now works as in NumPy, that is, it returns an array with the values in 'field' in the structured NDArray.

* Several new constructors are available for creating NDArray instances, like `arange()`, `linspace()` and `fromiter()`. These constructors leverage the internal `lazyudf()` function and make it easier to create NDArray instances from scratch. See e.g. https://github.com/Blosc/python-blosc2/blob/main/examples/ndarray/arange-constructor.py for an example.

* Structured LazyArrays received a new `.indices()` method that returns the indices of the elements that fulfill a condition. When combined with the new support of list of indices as key for `NDArray.__getitem__()`, this is useful for creating indexes for data.  See https://github.com/Blosc/python-blosc2/blob/main/examples/ndarray/filter_sort_fields.py for an example.

* LazyArrays received a new `.sort()` method that sorts the elements in the array.  For example, given an array `sarr` with fields 'a', 'b' and 'c', the next `farr = sarr["b >= c"].sort("c").compute()` puts in `farr` the rows that fulfills that values in fields in 'b' are larger than values in 'c' (`b >= c` above), ordered by column 'c'.

* New `expr_operands()` function for extracting operands from a string expression.

* New `validate_expr()` function for validating a string expression.

* New `CParams`, `DParams` and `Storage` dataclasses for better handling of parameters in the library. Now, you can use these dataclasses to pass parameters to the library, and get a better error handling. Thanks to @martaiborra for the excellent implementation and @omaech for revamping docs and examples to use them.  See e.g. https://www.blosc.org/python-blosc2/getting_started/tutorials/02.lazyarray-expressions.html.

### Documentation improvements

* Much improved documentation on how to efficiently compute with compressed NDArray data. Documentation updates highlight these features and improve usability for new users. Thanks to @omaech and @martaiborra for their excellent work on the documentation and examples, and to @NumFOCUS for their support in making this possible!  See https://www.blosc.org/python-blosc2/getting_started/tutorials/04.reductions.html for an example.

* New remote proxy tutorial. This tutorial shows how to use the Proxy class to access remote arrays, while providing caching. https://www.blosc.org/python-blosc2/getting_started/tutorials/06.remote_proxy.html . Thanks to @omaech for her work on this tutorial.

* New tutorial on "Mastering Persistent, Dynamic Reductions and Lazy Expressions". See https://www.blosc.org/posts/persistent-reductions/


## Changes from 3.0.0-beta.3 to 3.0.0-beta.4

* Many new examples in the documentation.  Now, the documentation is more complete and has a better structure.
 Have a look at our new docs at: https://www.blosc.org/python-blosc2/
 For a guide on using UDFs, check out: https://www.blosc.org/python-blosc2/reference/autofiles/lazyarray/blosc2.lazyudf.html
 If interested in asynchronously fetching parts of an array, take a look at: https://www.blosc.org/python-blosc2/reference/autofiles/proxy/blosc2.Proxy.afetch.html
 Finally, there is a new tutorial on optimizing reductions in large NDArray objects: https://www.blosc.org/python-blosc2/getting_started/tutorials/04.reductions.html
 Special thanks @omaech and @martaiborrar for the excellent work on the documentation and examples, and to @NumFOCUS for their support in making this possible!

* New CParams, DParams and Storage dataclasses for better handling of parameters in the library.  Now, you can use these dataclasses to pass parameters to the library, and get a better error handling.  See [here](https://www.blosc.org/python-blosc2/reference/storage.html).  Thanks to @martaiborra for the excellent implementation.

* Better support for CParams in Proxy and C2Array instances.  This allows to better propagate compression parameters from Caterva2 datasets to the Proxy and C2Array instances, improving the perception of codecs and filters used originally in datasets.  Thanks to @FrancescAlted for the implementation.

* Many improvements in ruff linting and code style.  Thanks to @DimitriPapadopoulos for the excellent work in this area.


## Changes from 3.0.0-beta.1 to 3.0.0-beta.3

* Revamped documentation.  Now, the documentation is more complete and has a better structure. See [here](https://www.blosc.org/python-blosc2/).  Thanks to Oumaima Ech Chdig (@omaech), our newcomer to the Blosc team.  Also, thanks to NumFOCUS for the support in this task.

* New `Proxy` class to access other arrays, while providing caching. This is useful for example when you have a big array, and you want to access a small part of it, but you want to cache the accessed data for later use.  See [its doc](https://www.blosc.org/python-blosc2/reference/proxy.html).

* Lazy expressions can accept proxies as operands.

* Read-ahead support for reading super-chunks from disk.  This allows for overlapping reads and computations, which can be a big performance boost for some workloads.

* New BLOSC_LOW_MEM envar for keeping memory under a minimum while evaluating expressions.  This makes it possible to evaluate expressions on very large arrays, even if the memory is limited (at the expense of performance).

* Fine tune block sizes for the internal compute engine.

* Better CPU cache size guessing for linux and macOS.

* Build tooling has been modernized and now uses `pyproject.toml` and `scikit-build-core` for managing dependencies and building the package.  Thanks to @LecrisUT for the excellent guidance in this area.

* Many code cleanup and syntax improvements in code.  Thanks to @DimitriPapadopoulos.


## Changes from 2.6.2 to 3.0.0-beta.1

* New evaluation engine (based on numexpr) for NDArray instances.  Now, you can evaluate expressions like `a + b + 1` where `a` and `b` are NDArray instances.  This is a powerful feature that allows for efficient computations on compressed data, and supports advanced features like reductions, filters, user-defined functions and broadcasting (still in beta).  See this [example](https://github.com/Blosc/python-blosc2/blob/main/examples/ndarray/eval_expr.py).

* As a consequence of the above, there are many new functions to operate with, and evaluate NDArray instances.  See the [function section docs](https://www.blosc.org/python-blosc2/reference/operations_with_arrays.html#functions) for more information.

* Support for NumPy 2.0.0 is here!  Now, the wheels are built with NumPy 2.0.0. If you want to use NumPy 1.x, you can still use it by installing NumPy 1.23 and up.

* Support for memory mapping in `SChunk` and `NDArray` instances.  This allows to map super-chunks stored in disk and access them as if they were in memory.  If curious, see  [some benchmarks here](https://github.com/Blosc/python-blosc2/blob/main/examples/ndarray/eval_expr.py).  Thanks to @JanSellner for the excellent implementation, both in the C and the Python libraries.

* Internal C-Blosc2 updated to 2.15.0.

* 32-bit platforms are officially unsupported now.  If you need support for 32-bit platforms, please use python-blosc 1.x series.

## Changes for 2.x series

* See the [release notes](https://github.com/Blosc/python-blosc2/blob/v2.x/RELEASE_NOTES.md) for the 2.x series.
