LoginSign Up
← Back to all sparks
D

datefixR

ANALYTICS
Velocity0.0

🗓 Standardize messy date data into consistent, machine-readable, formats

github.com/ropensci/datefixR ↗

The messy-date parser rewrote its core in Rust and came out 300x faster.

date-parsingrustdata-cleaninglocalizationr-packageperformance
◆Current state
datefixR standardizes inconsistently formatted dates — the kind that arrive from spreadsheets and hand-entered clinical or survey data, with mixed separators, ambiguous orders, missing components, and month names in whatever language the source used. Version 2.0.0 rewrote the parsing core in Rust, reporting over 300x throughput against previous versions through fastpath handling of common formats and parallel column processing via a cores argument. Version 2.0.1 then spent itself cleaning up after that rewrite, restoring ordinal indicator support, stopping malformed dates from being silently cast to NA, and reinstating error messages that had gone missing.
◆Where it's heading
Two long arcs meet here. The first is localization: Russian, Indonesian, German, Spanish month abbreviations, and experimental Roman numeral months accumulated release by release, with full translation of user-facing messages treated as a goal rather than a bonus. The second is the migration off R for the parsing hot path — internals began moving to C++ around 1.3.1 before the Rust rewrite replaced that work entirely. The 2.0.1 regressions show the cost of that move, since behavior that was implicit in the R implementation had to be re-specified.
◆Prediction
The Rust core is one release into stabilization and 2.0.1 was entirely regression repair, so expect further correctness fixes against pre-2.0.0 behavior before any new format support lands.

◆Recent moves

  1. 3mo ago

    Rust rewrite regressions repaired, silent NA casting stopped

    Cleanup after the 2.0.0 rewrite: º and ª ordinal indicators work again, informative errors are raised for fix_date_df() once more, and malformed dates no longer cast silently to NA — the most consequential fix, since a silent NA in date cleaning is a data loss that survives review. Dates with more than three components now error, and webR support improves.

    View source ↗
  2. 11mo ago

    Parsing core rewritten in Rust for a 300x speedup

    ⚡ SPARK

    The core date parsing logic is replaced with a Rust implementation reporting over 300x the throughput of previous versions, combining fastpath handling of common formats with parallel column processing. Changes what size of dataset this package is a plausible tool for, rather than what it can parse.

    View source ↗
  3. 1y ago

    Indonesian month names and translations added

    Adds Indonesian translations and month names, continuing the contributor-driven localization pattern that has carried Russian, German, and Spanish variants into the parser. Each addition widens the set of raw data the package can read without preprocessing.

    View source ↗
  4. 2y ago

    'ene' and 'ener' recognized as January

    Adds two Spanish abbreviations for January. A single-line contribution, and representative of how the localization surface grows here — one real-world variant at a time.

    View source ↗
  5. 3y ago

    Russian localization, Roman numeral months, Windows freeze fix

    Adds Russian translations and month names, experimental support for months written as Roman numerals, and reaches 100% translation of user-facing messages. Also fixes R freezing on Windows when the package was built with Rtools 43 — a hang rather than an error, which is the worst way for a build issue to present.

    View source ↗
  6. 3y ago

    Excel leap-year offset and single-digit day fixes

    Corrects numeric dates converted from Excel, which were slightly off because Excel treats 1900 as a leap year, and fixes parsing of single-digit days paired with two-digit years. Narrow, but the Excel offset silently shifted every converted date.

    View source ↗