tidytext
Finished, widely taught, and shipping roxygen fixes.
Comparison · Analytics
loo vs probably
A side-by-side editorial comparison of loo and probably — release velocity, themes, recent moves, and the top alternatives to consider.
loo vs probably: at a glance
What is loo?
loo keeps rewriting the diagnostics Bayesian modellers read off model comparison
loo computes leave-one-out cross-validation and model comparison for Bayesian models in the Stan ecosystem. Two releases in this window changed what users actually read: 2.7.0 replaced the fixed Pareto-k thresholds with sample-size-dependent ones and dropped the middle category, and 2.10.0 reshaped loo_compare's output into a data.frame with new uncertainty columns. The releases between are diagnostic robustness fixes and moment-matching corrections.
What is probably?
The package that made calibration a step instead of an afterthought.
probably started as a small utility for class predictions and equivocal zones, and version 1.0.0 turned it into tidymodels' calibration and uncertainty package: cal_plot_*, cal_estimate_*, cal_validate_* and cal_apply across binary, multiclass and regression problems, plus conformal prediction intervals. Since then the work has been consolidation — a large internal refactor with no API change, split conformal and conformal quantile regression, bound_prediction(), and required_pkgs() and butcher methods so conformal objects can be deployed and stripped.
loo vs probably: editorial side-by-side
loo
2.5ANALYTICS
loo keeps rewriting the diagnostics Bayesian modellers read off model comparison
◆ Current state
loo computes leave-one-out cross-validation and model comparison for Bayesian models in the Stan ecosystem. Two releases in this window changed what users actually read: 2.7.0 replaced the fixed Pareto-k thresholds with sample-size-dependent ones and dropped the middle category, and 2.10.0 reshaped loo_compare's output into a data.frame with new uncertainty columns. The releases between are diagnostic robustness fixes and moment-matching corrections.
◆ Where it's heading
The package is being brought in line with the current PSIS literature rather than extended with new features, and the practical effect is that the numbers practitioners quote in papers keep changing meaning. Work is increasingly delegated to posterior for shared computations, and the project has added contributor process, benchmarks and a published AI contribution policy.
◆ Prediction
Expect further work on comparison diagnostics — the p_worse and diag_* columns are new enough that their defaults and documentation will likely be revised next.
probably
0.0ANALYTICS
The package that made calibration a step instead of an afterthought.
◆ Current state
probably started as a small utility for class predictions and equivocal zones, and version 1.0.0 turned it into tidymodels' calibration and uncertainty package: cal_plot_*, cal_estimate_*, cal_validate_* and cal_apply across binary, multiclass and regression problems, plus conformal prediction intervals. Since then the work has been consolidation — a large internal refactor with no API change, split conformal and conformal quantile regression, bound_prediction(), and required_pkgs() and butcher methods so conformal objects can be deployed and stripped.
◆ Where it's heading
The recent releases are about making these objects survive leaving the session. butcher and required_pkgs() methods are what a model needs to be pinned, containerised and served, and their arrival alongside workflows adding a tailor postprocessing stage and vetiver adding probably support points the same way: calibration is being moved out of analysis scripts and into the deployed pipeline. The cal_*_none() reference implementations are the tell that calibration is now something people tune rather than apply once.
◆ Prediction
Expect the calibration functions to be reachable directly from a tuned workflow's postprocessing stage rather than applied to predictions afterwards, following the tailor integration that workflows just shipped.
Alternatives to loo and probably
Other Analytics products tracked by Sparkpulse, ranked by recent ship velocity. Each card links to a full editorial trajectory and lets you pivot into a head-to-head comparison with either loo or probably.
textrecipes
Text features finally stay sparse all the way to the model.
workflowsets
workflowsets keeps widening what counts as a model worth comparing.
workflows
The tidymodels pipeline grew a third stage, and it happens after the model runs.
vetiver
Posit's MLOps package went quiet for two years, then came back to keep up with recipes.
patchwork
patchwork stopped being a ggplot composer and became a page composer.
See all loo alternatives → · See all probably alternatives →
Recent activity from loo and probably
Latest ship moves from both products, interleaved chronologically. ⚡ = editorial spark.
- 20d agoloopsis_smooth_tail revert and simplify arg restored
- 1mo agolooloo_compare returns a data.frame with new uncertainty columns ⚡
- 7mo agolooStacking overflow fixes and posterior-based ESS
- 10mo agoprobablyConformal objects gain required_pkgs() and butcher methods
- 1y agoprobablyggplot2 test updates and a clearer validation-set error
- 1y agoprobablyCalibration internals refactored; isotonic bootstrap bug fixed
- 2y agolooMore robust Pareto-k diagnostics and moment matching
- 2y agolooPareto-k thresholds now depend on sample size ⚡
- 2y agoprobablyFix grouping sensitivity to variable type
- 3y agoprobablySplit conformal and conformal quantile regression added
- 3y agoprobablyCalibration and conformal inference arrive in tidymodels ⚡
- 3y agolooLOO predictive metrics and CRPS scoring functions
Frequently asked questions
What is the difference between loo and probably?
They serve adjacent needs but don't currently overlap on shipped themes. loo is currently shipping more aggressively (velocity 2.5 vs 0.0), with 0 editorial sparks in the last 30 days against 0. See the at-a-glance table above for a side-by-side breakdown of velocity, recent sparks, and editorial themes.
Is loo better than probably?
Sparkpulse doesn't pick a winner — we score release velocity, not feature parity. loo is currently shipping more aggressively (velocity 2.5 vs 0.0), with 0 editorial sparks in the last 30 days against 0. For your specific use case, the alternatives sections above list other Analytics products to evaluate alongside.
What are the best alternatives to loo?
Top loo alternatives in Analytics are ranked by recent ship velocity. Browse the "loo alternatives" section above for the current picks, or visit /alternatives/loo for the full list with editorial commentary on each.
What are the best alternatives to probably?
Top probably alternatives in Analytics are ranked by recent ship velocity. Browse the "probably alternatives" section above for the current picks, or visit /alternatives/probably for the full list with editorial commentary on each.