LoginSign Up
← Back to all sparks
R

Rmonize

INFRA · APIS
Velocity0.0

Tools for Data Harmonization

github.com/maelstrom-research/rmonize ↗

Collapsed a pile of parameters into one object and renamed every report column

data-harmonizationepidemiologybreaking-changesreportingapi-simplification
◆Current state
Rmonize supports data harmonization: taking heterogeneous input datasets, applying processing rules against a DataSchema, and producing a harmonized dossier with assessment, summary and visual reports. Version 2.0.0 reshaped how that is driven — the evaluate, summarize and visualize functions now take the dossier alone rather than six or seven parallel arguments — and renamed every column in the assessment and summary outputs into plain language. The package is closely coupled to madshapR, whose changes the notes warn may require updates to existing user code.
◆Where it's heading
The arc runs from correctness toward interface. Version 1.0.1 was bug fixes found on real data, 1.1.0 added a debug parameter so harmonization could be tested with incomplete inputs, and 2.0.0 is a deliberate simplification that breaks existing code in exchange for a smaller surface. Renaming outputs from expressions like 'Categories::missing' and 'Nb. non-valid values' to 'Non-valid categories' and 'Number of non-valid values' points at reports being read by people who are not the person who wrote the harmonization rules.
◆Prediction
Expect the superseded parameters and the renamed demo object to be removed outright rather than left superseded, and continued work on the visual reports, which carry the largest volume of referenced issues across all three versions.

◆Recent moves

  1. 1y ago

    Report functions take a single dossier; output columns renamed

    ⚡ SPARK

    The release that turns a working harmonization engine into a usable one: three report functions lose most of their parameters in favour of the dossier object that already carries the information, and the assessment and summary outputs are relabelled in plain language. It completes the move from the correctness work of 1.0.1 and the testing support of 1.1.0 toward an interface someone other than the author can drive.

    View source ↗
  2. 2y ago

    Debug parameter for testing partial harmonizations

    Adds a .debug parameter so harmonization can be exercised when the DataSchema, processing elements or input datasets are not all available, and lets show_harmo_error() suppress warnings. Practical support for the iterative way these projects are actually built.

    View source ↗
  3. 2y ago

    Pooled-data handling and report fixes after real-world testing

    A wide set of corrections found by running the package on real data, the structural one being a shared harmonized_col_dataset parameter used consistently across the pipeline as a grouping variable. Most of the rest is report presentation — colour schemes, date histograms, percentage handling — which is where this package's issue traffic concentrates.

    View source ↗