LoginSign Up
← Back to all sparks
D

Deequ

ANALYTICS
Velocity0.0

Library built on Spark for defining and verifying data quality constraints

github.com/awslabs/deequ ↗

Deequ ships GitHub tags whose release notes are one commit message long

data-qualitysparkdqdljvm-librarymulti-branch
◆Current state
Deequ is a data quality library for Spark, and it releases per Spark version — tags carry a -spark-3.5 or -spark-4.0 suffix, so the same code line ships more than once. The release notes are whatever the last commit message said, which here means each entry is a single line. Four releases landed in a month between March and April 2026, and only two of them contain any product change: a Range analyzer with DQDL rule support, and a processRowsTyped API for typed outcome access.
◆Where it's heading
The visible work points in one direction — making check results programmatically consumable rather than just readable. A typed outcome API and a rule language binding are what you build when Deequ is being called from a pipeline that reacts to the result, not from a notebook where a human reads it. The column-pruning override added alongside the Range analyzer suggests the same attention on the cost side, keeping analyzers from scanning columns they do not reference.
◆Prediction
The entries are too thin to support a confident read of what comes next; the only clear pattern is that each change will ship separately against Spark 3.5 and Spark 4.0, so the version skew between those branches will keep widening.

◆Recent moves

  1. 3mo ago

    Deequ adds a processRowsTyped API for typed outcome access

    A typed API for reading check outcomes, which makes results usable from code without parsing an untyped structure. It fits the pattern of Deequ being wired into automated pipelines rather than read by hand.

    View source ↗
  2. 3mo ago

    Deequ 3.0.1 fixes the publish workflow branch

    A release cut solely to correct the publish workflow's branch handling and bump the version, shipped a day after 3.0.0. Release plumbing with no library change.

    View source ↗
  3. 3mo ago

    Deequ 3.0.0 adds a Range analyzer with DQDL rule support

    The major version number sits on a modest change: a Range analyzer wired into DQDL rules, plus a columnsReferenced override on Variance so V2 data sources can prune columns the analyzer never touches. The rule-language binding is the more consequential half, since it is how the analyzer becomes reachable declaratively.

    View source ↗
  4. 4mo ago

    Deequ 2.0.15 tag carries only a pom version bump

    The entire release note is a pom.xml version bump, and the version it bumps to does not match the tag's Spark suffix. An artifact of tagging every commit that touches a version rather than a deliberate release.

    View source ↗