LoginSign Up
← Back to all sparks
C

ClearML

AI-ASSISTANTS
Velocity2.5

End-to-end MLOps platform for experiment tracking and orchestration

clear.ml ↗

ClearML 2.2 turns operational knobs into config: compression, retries, logging, container-aware CPU.

experiment trackinghyperdatasetsstorage managersdk configurationartifact security
◆Current state
After a summer of hyperdataset lifecycle work and SDK input hardening, 2.2.0 is an operations release. It adds a configurable zip compression algorithm for dataset chunks and artifacts, environment and vault overrides for API retry settings, switches to disable stdout/stderr capture and resource monitoring, and reports CPU against the container's cgroup limit. Fixes close a shell-escaping injection path in the autoscaler driver and stop the cache from evicting files another worker is still reading.
◆Where it's heading
The SDK is shifting from adding hyperdataset verbs to making existing pieces behave correctly in containerised, multi-worker production setups. Security fixes continue to land as a steady background stream rather than as headline features, and Python 2 compatibility is being removed.
◆Prediction
Expect more runtime configuration surfaced through environment variables and vault, plus further cleanup now that Python 2 support is being dropped.

◆Recent moves

  1. 2d ago

    Zip compression choice, container-aware CPU, API retry overrides

    2.2.0 moves the SDK toward production operability: selectable zip compression for datasets and artifacts, runtime overrides for API retries, and options to turn off log capture and resource monitoring. CPU reporting now respects container limits, and an autoscaler shell-injection path and a cache-eviction race between DataLoader workers are fixed.

    View source ↗
  2. 2mo ago

    Hyperdataset entry deletion, metadata management and mapping rules

    The hyperdataset API gains the operations that were conspicuously missing: deleting data entries, getting and setting metadata, managing mapping rules, labelling dataviews and iterating a hyperdataset. Race-condition fixes in dataviews land alongside. This is the unglamorous half of an API becoming usable rather than demonstrable.

    View source ↗
  3. 3mo ago

    In-memory streaming in the storage manager, DataView retrieval

    In-memory data streaming lands in the storage manager and DataView gets a retrieval method against the API server, with a 100 MB cap placed on registration request payloads. The cap is the detail worth noting — it implies people were pushing registration payloads large enough to need one.

    View source ↗
  4. 3mo ago

    HPO trial pruning and hashlib usedforsecurity fixes

    Hyperparameter optimisation gains trial pruning for resilience, and hashlib calls are marked usedforsecurity=False throughout — a small change that matters on FIPS-restricted hosts. A parent_ids argument is deprecated in favour of parent_id without breaking existing callers.

    View source ↗
  5. 4mo ago

    Hyperdataset version snapshots and a static route validator

    Version snapshots give hyperdatasets a point-in-time handle, which is the prerequisite for reproducible references to a dataset that keeps changing. The f-string migration and dead-code removal continue in the background.

    View source ↗
  6. 5mo ago

    Hyperdataset tagging and publishing, plus Azure default credentials

    Tagging and single-call publishing extend the hyperdataset lifecycle, DefaultAzureCredential support removes a credential-plumbing step on Azure, and a path-traversal check lands in dataset merging — the same defensive thread running through the 2.1.6 and 2.1.7 releases.

    View source ↗