LoginSign Up
← Back to all sparks
M

mlr3measures

ANALYTICS
Velocity0.0

Performance measures for supervised learning used by the mlr3 framework

github.com/mlr-org/mlr3measures ↗

mlr3measures is systematically retrofitting sample weights across every metric

metricsmlr3machine-learningr-statsevaluationsample-weights
◆Current state
mlr3measures is the metric library behind mlr3. Recent releases follow two threads: adding measures — linex, pinball, Mu AUC, gmean, gpr, mcc — and retrofitting sample_weights support across the existing ones, reaching AUC and the confusion-matrix family in 1.3.0. Along the way 1.1.0 deprecated four regression measures and corrected the bias definitions.
◆Where it's heading
The library is maturing rather than growing: weighted evaluation and observation-wise loss functions are being brought to metrics that already existed, which is what downstream weighted-resampling and per-observation analysis need. The deprecations suggest the maintainers are willing to remove measures they consider ill-defined rather than keep them for compatibility.
◆Prediction
Expect sample_weights and observation-wise variants to reach the remaining measures that lack them.

◆Recent moves

  1. 3mo ago

    Weighted AUC and weighted confusion-matrix measures

    Extends sample_weights to auc(), the confusion-matrix family, mbrier() and smape(). The widest single step in the weighting retrofit that has been running since 1.1.0.

    View source ↗
  2. 8mo ago

    Observation-wise loss for bbrier and logloss

    Adds per-observation variants of two probabilistic classification losses, continuing the pattern begun with the 0.5.0 observation-wise family.

    View source ↗
  3. 11mo ago

    rse, rsq, rrse and rae deprecated; bias measures corrected

    Deprecates four widely used regression measures and fixes the bias definitions, while extending sample_weights to sae and sse. A breaking cleanup that will surface in any script quoting R-squared from mlr3.

    View source ↗
  4. 1y ago

    linex, pinball and Mu AUC measures added

    Adds asymmetric and quantile losses plus a multiclass AUC variant, filling gaps for quantile regression and imbalanced multiclass evaluation.

    View source ↗
  5. 2y ago

    gmean, gpr and multiclass MCC added

    Adds two binary classification measures and multiclass Matthews correlation, both common asks for imbalanced problems.

    View source ↗
  6. 4y ago

    Observation-wise loss functions introduced

    Introduces ae(), ape(), se(), sle() and zero_one() as per-observation losses — the start of the observation-wise thread that 1.2.0 later extends.

    View source ↗