LoginSign Up
← Back to all sparks
L

ltertools

INFRA · APIS
Velocity0.0

Tools Developed by the Long Term Ecological Research Community

github.com/lter/ltertools ↗

A column-key toolkit for stitching decades of ecological field data into one table.

data-harmonizationecologylterresearch-datar-packagereproducibility
◆Current state
ltertools serves the Long Term Ecological Research network, where the same measurement carries a different column name at every site and in every era. Its core is a column key: begin_key drafts one, harmonize applies it, and the 2.0.0 release added check_key to validate a key and standardize to apply one to a single dataset. Harmonization of files above 5 MB now runs in roughly half the time.
◆Where it's heading
Development has moved from breadth to depth. The first year added assorted utilities — temperature conversion, solar day length, a site timeline — while the last two releases have concentrated on the key workflow itself: incremental key expansion, validation, per-dataset application, and speed. A dependency archival forced the removal of the JSON helper, trimming the package back toward that core.
◆Prediction
The key workflow now has draft, expand, check and apply steps, so the remaining gap is diagnostics on the harmonized output; the entries show no other thread in progress.

◆Recent moves

  1. 11mo ago

    Warning wording and a ggplot2 4.0.0 test update

    Tightens the warning standardize emits for columns present in the key but absent from the data, and updates a unit test for ggplot2 4.0.0. Upkeep on the functions added six months earlier.

    View source ↗
  2. 1y ago

    Key validation and single-dataset standardizing; JSON helper dropped

    check_key validates a column key and strips rows with no tidy_name, standardize applies a key to one dataset rather than a whole list, and harmonize runs about twice as fast on large files. make_json is removed because its main dependency is being archived — a forced deletion rather than a design decision, but the major version number is earned by it.

    View source ↗
  3. 1y ago

    Incremental key expansion for newly added files

    expand_key generates key rows only for raw files not already covered by an existing key or harmonized table. It makes the column key a maintainable artifact across a growing data collection instead of something rebuilt whenever new files arrive.

    View source ↗
  4. 1y ago

    Solar day calculations, JSON export, defensive checks in harmonize

    Adds solar_day_info for sunrise, sunset and day length at given coordinates, a JSON name/value writer, and informative warnings for the common ways harmonize fails. Full unit-test coverage lands here too — the breadth phase, before later releases narrowed back to the key workflow.

    View source ↗
  5. 2y ago

    First CRAN release: the column-key harmonization workflow

    Establishes the package's core — read a folder of files, draft a column key, harmonize against it — alongside utilities for coefficient of variation, temperature conversion, and a dataset of LTER site habitats and coordinates. Everything since has sharpened this workflow rather than replaced it.

    View source ↗