LoginSign Up
← Back to all sparks
K

KRLS

INFRA · APIS
Velocity0.0

Kernel-Based Regularized Least Squares

github.com/j-hai/krls ↗

A 2014 kernel regression method getting the scalability and tooling it never had, in a three-release afternoon.

kernel-methodsmachine-learningcausal-inferencescalabilitybroomr-package
◆Current state
KRLS fits kernel regularized least squares, a method whose exact form requires an n-by-n kernel matrix and therefore stops being usable well before modern sample sizes. Three releases shipped within 33 minutes of each other addressed exactly that: a Nystrom approximation mode with conditional approximate inference, kmeans landmark selection with an accessor for reusing landmarks across fits, and GCV as an alternative to leave-one-out for choosing lambda. The default path remains the exact one, and existing calls are unchanged.
◆Where it's heading
The package is being modernized on two tracks that reinforce each other. The interface track — a formula method, broom extractors, autoplot, summary and glance diagnostics — makes the estimator fit contemporary R workflows without touching the algorithm, and the notes are explicit that existing matrix-interface calls remain bit-identical. The performance track removes the reasons it could not be run at all: the Nystrom mode for the kernel matrix, and an average-marginal-effects variance computation rewritten via a row-sum identity to quadratic per-predictor cost. Everything is added as opt-in, which suggests the goal is reaching new users without disturbing replication of published results.
◆Prediction
With approximation, landmark reuse, and a second lambda criterion now in place, the remaining gap is guidance on when to trust them; the scaling vignette shipped alongside GCV points to more empirical validation rather than new estimation machinery.

◆Recent moves

  1. 3mo ago

    GCV added as an alternative lambda selection criterion

    Generalized cross-validation joins leave-one-out for regularization selection, with the existing default preserved. A second criterion for an existing task rather than a new capability, though it pairs naturally with the approximation work — GCV is cheaper on the large problems the Nystrom mode just made reachable.

    View source ↗
  2. 3mo ago

    Kmeans landmark selection and landmark reuse across fits

    The polish pass on the approximation shipped ten minutes earlier: kmeans-chosen landmarks instead of arbitrary ones, an accessor that lets the same landmarks be carried between fits, and diagnostics through summary and glance. Landmark round-tripping is the detail that makes approximated fits comparable to each other rather than each one its own draw.

    View source ↗
  3. 3mo ago

    Nystrom approximation mode lifts the sample-size ceiling

    ⚡ SPARK

    The release that changes which problems this estimator can be pointed at. An explicit approximation mode with approximate inference removes the exact kernel matrix requirement, and the marginal-effects variance rewrite cuts a second cost that scaled badly. The two follow-up releases the same afternoon exist to support this one.

    View source ↗
  4. 3mo ago

    Formula interface plus broom and autoplot support

    Interface modernization with the algorithm untouched: a formula method that expands factors and interactions through model.matrix(), and tidy, glance, and augment registered so broom picks them up. The tidy output carries pointwise quartiles alongside the average marginal effect, which surfaces the heterogeneity that makes this estimator worth using in the first place.

    View source ↗
  5. 3mo ago

    v1.1-0: Update Chad Hazlett affiliation MIT -> UCLA in 9 .Rd files

    An author affiliation corrected across nine help pages, unchanged since the 2014 paper. No functional content, but it dates the start of the maintenance push that produced everything after it.

    View source ↗