LoginSign Up
← Back to all sparks
S

SLmetrics

INFRA · APIS
Velocity0.0

Machine Learning Performance Evaluation on Steroids

github.com/serkor1/SLmetrics ↗

A young ML metrics package rewrote its own backend twice in six months chasing speed.

machine-learningmodel-evaluationperformancecpp-backendopenmpr-package
◆Current state
SLmetrics provides supervised learning evaluation metrics for R — confusion matrices, classification and regression measures, ROC and precision-recall curves — with the computation pushed into C++. After a series of pre-releases it now runs on an Armadillo backend, supports OpenMP parallelism and LAPACK/BLAS, and reports 5-20x speedups over its earlier implementations. The API has been reshaped for extensibility, with an estimator argument replacing the fixed aggregation options.
◆Where it's heading
Every release in this timeline is about making the same metrics compute faster or compose better. The backend moved from Rcpp to plain C++, gained OpenMP, then was ported wholesale from Eigen to Armadillo with heavy templating. In parallel the author has been widening the API's joints: generic S3 signatures, an extensible estimator argument, and function signatures loose enough that wrapping packages can rename arguments. Bundled datasets and embedded formulas in the docs point at teaching and benchmarking use. The package still labels itself pre-release, which is consistent with how freely it has broken argument names along the way.
◆Prediction
A stable non-pre-release version is the natural next step now that the backend has settled on Armadillo, though the repeated willingness to rename arguments suggests more API churn may come first.

◆Recent moves

  1. 1y ago

    Armadillo backend brings 5-20x speedups and an extensible metrics API

    ⚡ SPARK

    The backend is ported wholesale from Eigen to Armadillo with heavy templating, reported at 5-20x faster, and the API is loosened at the same time so aggregation methods and wrapper packages are no longer boxed in. It is the release where the performance arc and the extensibility arc converge.

    View source ↗
  2. 1y ago

    Consistent S3 signatures and three bundled datasets

    Gives every S3 method a generic signature, making the argument surface consistent across the package, and bundles the Wine Quality, Obesity and Banknote Authentication datasets with features and targets stored separately. The datasets point at benchmarking and teaching rather than production use.

    View source ↗
  3. 1y ago

    Regression metrics 2-10x faster with reworked OpenMP controls

    Regression metrics get the same optimisation treatment the classification side received, running 2-10x faster, with LAPACK/BLAS support added for matrix operations. The OpenMP controls move to the R side with openmp.on/off and openmp.threads, a breaking but more predictable interface that respects suppressMessages().

    View source ↗
  4. 1y ago

    OpenMP parallelism and a soft-label entropy family

    Introduces OpenMP parallelisation and an entropy family — entropy(), cross.entropy() and relative.entropy() — for soft labels, modelled on scipy. The logloss response argument is renamed to qk for consistency with that family, one of several breaking renames the pre-release status is being used to absorb.

    View source ↗
  5. 1y ago

    Cross-entropy loss and relative RMSE with three normalisations

    Adds weighted and unweighted cross-entropy loss and a relative RMSE offering mean, range and IQR normalisation, acknowledging that no single normalisation convention is standard. Coverage-widening work in the period before the performance push took over.

    View source ↗
  6. 1y ago

    Sample weights flow through the confusion matrix

    Sample weights arrive via a w argument on cmatrix(), so weighted metrics fall out of the weighted confusion matrix rather than needing separate code paths. Documentation gains embedded formulas, an early sign of the reference-material ambition the bundled datasets later reinforce.

    View source ↗