LoginSign Up
← Back to all sparks
K

kernelshap

ANALYTICS
Velocity0.0

Different SHAP algorithms

github.com/ModelOriented/kernelshap ↗

kernelshap makes permutation SHAP practical past eight features, then fixes the kernel weights it had wrong.

shapmodel explainabilitysampling algorithmsnumerical correctnessr package
◆Current state
kernelshap computes model-agnostic SHAP values in R through Kernel SHAP, permutation SHAP and an exact additive explainer. Version 0.8.0 added a sampling permutation-SHAP algorithm with standard errors and early stopping, lifting the practical feature ceiling past what the exact method allows. Version 0.9.0 then corrected a bug in how kernel weights were computed — exact Kernel SHAP now agrees with exact permutation SHAP — and moved parallelism from foreach to doFuture.
◆Where it's heading
Two concerns drive this package: making exact methods reach further, and being demonstrably right. The first shows in the additive explainer, the optional background dataset and the sampling permutation algorithm; the second in unit tests written against Python's shap, credited fixes from outside contributors, and a willingness to ship a correctness fix that changes numbers people have already published. Speed work runs continuously underneath — direct solves replacing the Moore-Penrose pseudo-inverse, roughly 10% less memory.
◆Prediction
The 0.6.0 and 0.7.0 notes each promised a stable 1.0.0 that has not arrived; with the weighting bug fixed and parallelism reworked, a 1.0 release is the most plausible next step.

◆Recent moves

  1. 1y ago

    Kernel weight bug fixed; parallelism moves to doFuture

    A correctness fix in kernel weighting means exact Kernel SHAP finally agrees with exact permutation SHAP; the notes describe the differences as typically small, with models up to second-order interactions unaffected. Parallelism moves to doFuture, changing how packages and globals are declared.

    View source ↗
  2. 1y ago

    Sampling permutation SHAP with standard errors

    ⚡ SPARK

    Permutation SHAP stops being an exact-only method limited to a handful of features. The antithetic sampling scheme, early stopping and reported standard errors are what push the package past the ceiling its exact algorithms imposed.

    View source ↗
  3. 1y ago

    Background data now optional; ranger survival support

    Passing a background dataset becomes optional, sampled from the explanation data when it is large enough — one less required argument in the common case. ranger survival models work without a hand-written prediction function, matching the same addition in hstats.

    View source ↗
  4. 2y ago

    Factor-valued predictions dropped

    Support for factor-valued predictions is removed, the same narrowing hstats applied in its own 1.2.0. A CRAN note is cleared and MASS added for the pseudo-inverse that 0.9.0 later removed again.

    View source ↗
  5. 2y ago

    additive_shap() explains additive models exactly

    A new explainer for lm, glm, gam, coxph and survreg models uses prediction terms to produce the same values as the general algorithms but far faster. permshap() output folds into the kernelshap class, with an algorithm element recording which method ran.

    View source ↗
  6. 2y ago

    Faster on plain data.frames

    Performance work: a significant speed-up for pure data.frames, with caching for the all-on and all-off permutations. Incremental, and typical of the steady optimisation between algorithmic releases.

    View source ↗