LoginSign Up
← Back to all sparks
S

sparsevctrs

ANALYTICS
Velocity0.0

Sparse vector class using ALTREP. Contribute to r-lib/sparsevctrs development by creating an account on GitHub.

github.com/r-lib/sparsevctrs ↗

Sparse vectors stopped being a storage trick and became something you can do arithmetic on

sparse-datatidymodelsaltrepnumerical-computingr-packages
◆Current state
sparsevctrs supplies sparse vectors that live inside ordinary data frames and tibbles, which is what lets tidymodels carry wide, mostly-zero feature matrices without densifying them. Through 0.2.0 and 0.3.0 the package built out a computation layer on top of that storage — first summary statistics, then scalar and element-wise arithmetic — and everything since has been correctness work at the C level.
◆Where it's heading
The release pattern splits cleanly at 0.3.0. Before it, new functions arrive in batches; after it, five consecutive releases are bug fixes, and the bugs are the kind that come with hand-written sparse kernels: a stack imbalance when sparse_multiplication() returns all zeros, undefined behaviour in multiplication, type errors in sparse_is_na(), coercion failures on NA input. That is the expected cost of an ALTREP-backed numerical layer, and the fixes are landing steadily.
◆Prediction
With the arithmetic surface in place and the recent releases all narrow fixes, the next one is more likely another correctness patch than a new function family. The R devel fix in 0.3.5 suggests upcoming R releases are the current source of breakage.

◆Recent moves

  1. 8mo ago

    Sparse character vector fix for R devel

    Fixes a sparse character vector bug that appeared only under R devel. Forward-compatibility work ahead of the next R release, with no effect on current behaviour.

    View source ↗
  2. 1y ago

    Stack imbalance in sparse multiplication fixed

    Fixes a stack imbalance in sparse_multiplication() when the result is all zeros, and a type error where sparse_is_na(type = "integer") failed on character vectors. The stack imbalance is a C-level defect in a function added only two releases earlier — the arithmetic layer is still settling.

    View source ↗
  3. 1y ago

    Sparse matrix coercion no longer errors on NA input

    Fixes coerce_to_sparse_matrix() erroring on input containing NA values. Narrow, but this is the entry point into the whole package and NA-bearing input is the normal case for real data.

    View source ↗
  4. 1y ago

    sparsity() fixed for classed numeric vectors

    Fixes sparsity() erroring on numeric vectors that carry a class attribute. A single-case guard on a utility function.

    View source ↗
  5. 1y ago

    Undefined behaviour in sparse multiplication fixed

    Fixes undefined behaviour in sparse multiplication, one week after the function shipped. Undefined behaviour in a numerical kernel means results that are wrong unpredictably rather than wrong consistently, so this is a correctness fix rather than a cosmetic one.

    View source ↗
  6. 1y ago

    Scalar and element-wise arithmetic for sparse vectors

    ⚡ SPARK

    Adds the arithmetic layer: scalar division, multiplication, addition and subtraction, element-wise sparse_multiplication(), plus sparse_lag(), sparse_sqrt(), sparse_replace_na(), sparse_is_na() and sparse_which_na(). Together with the statistics added in 0.2.0, this is what turns a sparse storage format into something a preprocessing pipeline can compute against directly. Every bug fix in the five releases that follow lands in this new code.

    View source ↗