LoginSign Up
← Back to all sparks
L

loo

ANALYTICS
Velocity2.5

Efficient leave-one-out cross-validation and model comparison for Bayesian models

github.com/stan-dev/loo ↗

loo keeps rewriting the diagnostics Bayesian modellers read off model comparison

bayesiancross-validationstanr-statsdiagnosticsmodel-comparison
◆Current state
loo computes leave-one-out cross-validation and model comparison for Bayesian models in the Stan ecosystem. Two releases in this window changed what users actually read: 2.7.0 replaced the fixed Pareto-k thresholds with sample-size-dependent ones and dropped the middle category, and 2.10.0 reshaped loo_compare's output into a data.frame with new uncertainty columns. The releases between are diagnostic robustness fixes and moment-matching corrections.
◆Where it's heading
The package is being brought in line with the current PSIS literature rather than extended with new features, and the practical effect is that the numbers practitioners quote in papers keep changing meaning. Work is increasingly delegated to posterior for shared computations, and the project has added contributor process, benchmarks and a published AI contribution policy.
◆Prediction
Expect further work on comparison diagnostics — the p_worse and diag_* columns are new enough that their defaults and documentation will likely be revised next.

◆Recent moves

  1. 20d ago

    psis_smooth_tail revert and simplify arg restored

    Reverts an internal tail-smoothing change that broke downstream tests, and restores the simplify argument to the comparison printer. Cleanup after the 2.10.0 output rework.

    View source ↗
  2. 1mo ago

    loo_compare returns a data.frame with new uncertainty columns

    ⚡ SPARK

    Reworks the output of the package's most-used function, adding diagnostics about the comparison itself. Continues the pattern set by 2.7.0: changing what the headline numbers mean rather than adding features.

    View source ↗
  3. 7mo ago

    Stacking overflow fixes and posterior-based ESS

    Fixes numerical under- and overflow in stacking, corrects psis_r_eff() NAs and stratified k-fold splitting, and moves effective sample size onto posterior. Consolidation work rather than new capability.

    View source ↗
  4. 2y ago

    More robust Pareto-k diagnostics and moment matching

    Hardens the E_loo Pareto-k diagnostic and fixes several moment-matching failures including Stan log_prob exceptions. Follows directly from the threshold redefinition in 2.7.0.

    View source ↗
  5. 2y ago

    Pareto-k thresholds now depend on sample size

    ⚡ SPARK

    Redefines the diagnostic every loo user checks first, following the 2022 revision of the PSIS paper. The origin of the recalibration arc that runs through the later releases.

    View source ↗
  6. 3y ago

    LOO predictive metrics and CRPS scoring functions

    Adds loo_predictive_metric() for out-of-sample error and accuracy measures plus the crps()/scrps() family, broadening the package from elpd toward general predictive scoring.

    View source ↗