LoginSign Up
← Back to all sparks
D

dtplyr

ANALYTICS
Velocity0.0

data.table backend for dplyr giving fast performance with familiar syntax

github.com/tidyverse/dtplyr ↗

dtplyr stopped hijacking data.table objects and became an opt-in translator

data.tabledplyrtranslationperformancetidyverse
◆Current state
dtplyr converts dplyr and tidyr code into data.table syntax, and 1.3.0 redrew its boundary: verbs no longer dispatch to dtplyr translations just because dtplyr is loaded, so lazy_dt() has to be called explicitly. Since then the work has been translation coverage — reframe(), case_match(), consecutive_id() — plus a long tail of correctness fixes in grouping and .by.
◆Where it's heading
The package is trailing dplyr's own feature releases rather than leading them, adding each new verb once it settles upstream. Performance work is targeted at specific verbs where data.table has a faster primitive: setorder() for arrange(), reference drops for select(), rleid() for consecutive_id(). Release cadence has thinned considerably since 2023.
◆Prediction
Expect further one-for-one translations as dplyr adds verbs, and continued fixes around .by and non-standard column names; the entries show no sign of a broader redesign.

◆Recent moves

  1. 11mo ago

    reframe(), case_match() and consecutive_id() gain translations

    Catches dtplyr up to three dplyr verbs and fixes a batch of grouping problems, including .by leaking into prior steps and grouping on non-standard column names. Continuation of the trail-dplyr pattern rather than new direction.

    View source ↗
  2. 3y ago

    crayon dependency dropped

    A check fix and one dependency removed. Nothing changes for users.

    View source ↗
  3. 3y ago

    Verbs stop auto-dispatching; lazy_dt() now required

    ⚡ SPARK

    The release that defines dtplyr's current shape. Loading the package no longer changes what dplyr verbs do to a data.table, so translation becomes something you ask for. Shipped alongside .by support, several new translations, and targeted performance work.

    View source ↗
  4. 3y ago

    dtplyr 1.2.2

    A hot patch for CRAN check failures with no functional change.

    View source ↗
  5. 4y ago

    dtplyr 1.2.1

    A compatibility fix staged ahead of an rlang release. No user-facing change.

    View source ↗
  6. 4y ago

    Eight tidyr verbs gain data.table translations

    The largest coverage expansion in this window — pivot_longer(), nest(), complete(), fill(), separate() and more move from dplyr-only into tidyr territory, with three contributors promoted to authors. Sets up the boundary question 1.3.0 then answers.

    View source ↗