LoginSign Up
← Back to all sparks
V

vimp

INFRA · APIS
Velocity0.0

Perform Inference on Algorithm-Agnostic Variable Importance

github.com/bdwilliamson/vimp ↗

Algorithm-agnostic variable importance, settled since 2022 and now answering CRAN checks

variable-importancecausal-inferencemachine-learningsample-splittingmaintenance
◆Current state
vimp performs inference on variable importance measures that do not depend on the fitting algorithm, using sample-splitting so tests stay valid under the zero-importance null. The statistical design settled in 2022: predictiveness measures gained their own S3 class, point estimation was decoupled from inference through the final_point_estimate argument, and method and family moved to the outer functions so binary outcomes work. The two 2025 releases are a CRAN documentation fix and an edge-case correction for cutoff-based measures when every prediction is the same value.
◆Where it's heading
Substantive development ended in 2022, followed by one narrow addition — a cluster bootstrap for correlated data in 2023 — and two housekeeping releases. What remains visible are careful decisions about the boundary between estimation and inference, including the warning that a Wald interval will not be centred on the point estimate when the full-data or averaged option is used. The maintainer is keeping the package correct and installable rather than extending it.
◆Prediction
Expect maintenance releases keyed to CRAN check changes; new predictiveness measures are the plausible extension, since the S3 class added in 2.3.0 was introduced specifically to make adding them simpler.

◆Recent moves

  1. 11mo ago

    Update PPV, NPV, Sensitivity, Specificity

    Corrects the cutoff-based measures — positive and negative predictive value, sensitivity and specificity — for the degenerate case where every prediction is identical, as happens when the mean is used as the predictor. An edge-case fix rather than a change in method.

    View source ↗
  2. 1y ago

    Fix documentation NOTEs

    Clears CRAN check notes caused by new handling of documentation links and by differences between the itemize and describe markup. Packaging work with no effect on results.

    View source ↗
  3. 2y ago

    Add cluster bootstrap

    Adds a cluster bootstrap so importance inference can be run on correlated data. A single-line release, but it extends the package to a data structure it previously could not handle honestly.

    View source ↗
  4. 3y ago

    Enhanced VIM point estimation

    Separates point estimation from inference: the final estimate can now use the full dataset or the average of the two split-specific values, rather than being forced to follow the sample-splitting used for valid testing. The notes are careful about the cost — confidence intervals are still built on sample-splitting, so they will not be centred on the reported estimate.

    View source ↗
  5. 3y ago

    S3 class for predictiveness measures

    Gives predictiveness measures their own S3 class, described in the notes as making new measures simpler to add — the extensibility groundwork the package has not yet built on. Also changes extract_sampled_split_predictions() to return a vector rather than a list.

    View source ↗
  6. 4y ago

    Specify 'method' and 'family' in outer functions

    Moves the method and family arguments for weighted estimation from the measure functions up to vim(), cv_vim() and sp_vim(), which is what makes binary outcomes work. Also adds a vignette for coarsened-data settings.

    View source ↗