LoginSign Up
← Back to all sparks
Y

yardstick

ANALYTICS
Velocity0.0

Consistent measures of model performance and accuracy for R models

github.com/tidymodels/yardstick ↗

yardstick made fairness metrics a first-class part of tidymodels evaluation

metricstidymodelsfairnesssurvival-analysisdeprecation
◆Current state
yardstick supplies the metrics tidymodels evaluates models with. Its recent history is metric expansion into areas the package did not originally cover - survival analysis, model fairness, and in 1.4.0 a batch of regression and classification metrics filling remaining gaps - alongside a long deprecation cycle that finally turned errors on in 1.4.0.
◆Where it's heading
The direction is coverage plus extensibility. Rather than adding fairness metrics one at a time, 1.3.0 shipped new_groupwise_metric() so group-aware metrics can be defined for the problem at hand, which is the more durable contribution. The parallel thread is removing hidden state: the event_first global option, deprecated in 0.0.7, took until 1.4.0 to become an error.
◆Prediction
Expect the groupwise constructor to attract more fairness definitions than the three shipped, and the developer-facing metric creation helpers deprecated in 1.2.0 to be removed next; core metric coverage now looks close to complete.

◆Recent moves

  1. 4mo ago

    Adds Gini, MSE and rate metrics; old deprecations now error

    Fills remaining gaps in the metric catalogue - normalized Gini coefficient, mean squared error, relative RMSE, false positive and false negative rates, markedness - while turning the long-running deprecations hard: the event_first option and the pROC options argument now error rather than warn. Note the release notes are truncated in this feed.

    View source ↗
  2. 1y ago

    All messages translated to cli

    A single-line release moving every message, warning and error to the cli package. Consistency work matching what the rest of the tidyverse did years earlier.

    View source ↗
  3. 2y ago

    Fixes wrong weights in roc_curve_survival()

    Corrects the weights used in survival ROC curves and reorders its output columns to match roc_curve(). Follow-up to the survival metrics introduced two months earlier.

    View source ↗
  4. 2y ago

    Fairness metrics and a groupwise metric constructor

    ⚡ SPARK

    Two coverage gaps close at once - survival analysis gets Brier scores, concordance and time-dependent ROC, and fairness gets demographic parity, equalized odds and equal opportunity. The constructor underneath the fairness metrics is the part that outlasts the specific three.

    View source ↗
  5. 3y ago

    Brier score for classification; tidyselect interface throughout

    Adds brier_class() and moves metric functions to a pure tidyselect interface for their column arguments, aligning them with the rest of the tidyverse. The event_first global option is hard deprecated here, three releases before it finally errors.

    View source ↗
  6. 3y ago

    New maintainer; clearer metric_set() errors

    A maintainer handover to Emil Hvitfeldt plus an improvement to the chained error thrown when one metric in a set fails. Nothing else in the release.

    View source ↗