tidytext
Finished, widely taught, and shipping roxygen fixes.
Comparison · Analytics
loo vs mlr3mbo
A side-by-side editorial comparison of loo and mlr3mbo — release velocity, themes, recent moves, and the top alternatives to consider.
Shared themes:r-stats
loo vs mlr3mbo: at a glance
What is loo?
loo keeps rewriting the diagnostics Bayesian modellers read off model comparison
loo computes leave-one-out cross-validation and model comparison for Bayesian models in the Stan ecosystem. Two releases in this window changed what users actually read: 2.7.0 replaced the fixed Pareto-k thresholds with sample-size-dependent ones and dropped the middle category, and 2.10.0 reshaped loo_compare's output into a data.frame with new uncertainty columns. The releases between are diagnostic robustness fixes and moment-matching corrections.
What is mlr3mbo?
mlr3mbo picked its defaults from a benchmark study, not from taste
mlr3mbo does model-based and Bayesian optimisation for mlr3. Its 1.0.0 release added a dictionary of pre-built acquisition-function optimisers and, more consequentially, replaced the default surrogate, acquisition function and optimiser settings with values derived from a large-scale benchmark study. The releases since are corrections to the acquisition-optimiser path exposed by that new default configuration.
loo vs mlr3mbo: editorial side-by-side
loo
2.5ANALYTICS
loo keeps rewriting the diagnostics Bayesian modellers read off model comparison
◆ Current state
loo computes leave-one-out cross-validation and model comparison for Bayesian models in the Stan ecosystem. Two releases in this window changed what users actually read: 2.7.0 replaced the fixed Pareto-k thresholds with sample-size-dependent ones and dropped the middle category, and 2.10.0 reshaped loo_compare's output into a data.frame with new uncertainty columns. The releases between are diagnostic robustness fixes and moment-matching corrections.
◆ Where it's heading
The package is being brought in line with the current PSIS literature rather than extended with new features, and the practical effect is that the numbers practitioners quote in papers keep changing meaning. Work is increasingly delegated to posterior for shared computations, and the project has added contributor process, benchmarks and a published AI contribution policy.
◆ Prediction
Expect further work on comparison diagnostics — the p_worse and diag_* columns are new enough that their defaults and documentation will likely be revised next.
mlr3mbo
2.5ANALYTICS
mlr3mbo picked its defaults from a benchmark study, not from taste
◆ Current state
mlr3mbo does model-based and Bayesian optimisation for mlr3. Its 1.0.0 release added a dictionary of pre-built acquisition-function optimisers and, more consequentially, replaced the default surrogate, acquisition function and optimiser settings with values derived from a large-scale benchmark study. The releases since are corrections to the acquisition-optimiser path exposed by that new default configuration.
◆ Where it's heading
The package has moved from a toolkit that expected users to assemble a Bayesian optimisation loop into one with a defensible default loop, and the recent fixes — warm-start sizing on multi-objective archives, silently discarded terminators, stale x_domain values — are the consequences of more people running the default path.
◆ Prediction
Expect continued hardening of the acquisition-optimiser classes rather than new acquisition functions.
Alternatives to loo and mlr3mbo
Other Analytics products tracked by Sparkpulse, ranked by recent ship velocity. Each card links to a full editorial trajectory and lets you pivot into a head-to-head comparison with either loo or mlr3mbo.
textrecipes
Text features finally stay sparse all the way to the model.
probably
The package that made calibration a step instead of an afterthought.
workflowsets
workflowsets keeps widening what counts as a model worth comparing.
workflows
The tidymodels pipeline grew a third stage, and it happens after the model runs.
vetiver
Posit's MLOps package went quiet for two years, then came back to keep up with recipes.
Recent activity from loo and mlr3mbo
Latest ship moves from both products, interleaved chronologically. ⚡ = editorial spark.
- 20d agoloopsis_smooth_tail revert and simplify arg restored
- 23d agomlr3mboAcquisition optimiser fixes for warm starts and archives
- 1mo agolooloo_compare returns a data.frame with new uncertainty columns ⚡
- 3mo agomlr3mboDictionary lookup and restart-limit fixes
- 4mo agomlr3mborush 1.0.0 compatibility and Surrogate$check()
- 5mo agomlr3mbomlr3mbo 1.0.0 ships benchmark-derived default settings ⚡
- 7mo agolooStacking overflow fixes and posterior-based ESS
- 10mo agomlr3mbomlr3learners 0.13.0 compatibility
- 11mo agomlr3mboMaintainer change and mlr3pipelines 0.9.0 upkeep
- 2y agolooMore robust Pareto-k diagnostics and moment matching
- 2y agolooPareto-k thresholds now depend on sample size ⚡
- 3y agolooLOO predictive metrics and CRPS scoring functions
Frequently asked questions
What is the difference between loo and mlr3mbo?
Both compete on the same themes — r-stats — within Analytics. loo and mlr3mbo are shipping at a similar cadence (velocity 2.5 vs 2.5, both within Sparkpulse's "active" band). See the at-a-glance table above for a side-by-side breakdown of velocity, recent sparks, and editorial themes.
Is loo better than mlr3mbo?
Sparkpulse doesn't pick a winner — we score release velocity, not feature parity. loo and mlr3mbo are shipping at a similar cadence (velocity 2.5 vs 2.5, both within Sparkpulse's "active" band). For your specific use case, the alternatives sections above list other Analytics products to evaluate alongside.
What are the best alternatives to loo?
Top loo alternatives in Analytics are ranked by recent ship velocity. Browse the "loo alternatives" section above for the current picks, or visit /alternatives/loo for the full list with editorial commentary on each.
What are the best alternatives to mlr3mbo?
Top mlr3mbo alternatives in Analytics are ranked by recent ship velocity. Browse the "mlr3mbo alternatives" section above for the current picks, or visit /alternatives/mlr3mbo for the full list with editorial commentary on each.