LoginSign Up
← Back to all sparks
W

Windmill

INFRA · APIS
Velocity6.3

Open-source developer platform to build internal tools and workflows from scripts

www.windmill.dev ↗

Windmill is building a DuckLake data-engineering stack on top of its workflow engine

ducklakedata-engineeringopen-coregitopsai-agentsworkflow-automation
◆Current state
Windmill has shifted much of its recent output into a DuckLake lakehouse layer — schema contracts, fork-isolated data environments, partition backfills, scheduled maintenance, and freshness watchdogs — while also ungating warehouse runtimes and adding reasoning controls to its AI agent steps. It's a high-cadence open-core product where the substance now sits in data engineering.
◆Where it's heading
The direction is a full data-engineering stack inside Windmill: reproducible pipelines, environment isolation, and observability around DuckLake, with the open-core line steadily loosening as BigQuery and Snowflake move into Community Edition. Expect continued DuckLake maturation and more Community/Enterprise runtime parity, with the heavier automation kept Enterprise.
◆Prediction
Next likely moves are more DuckLake operational tooling — lineage, alerting, richer backfills — and additional AI-agent-step controls, keeping heavy automation like the freshness watchdog Enterprise-gated.

◆Recent moves

  1. 5d ago

    BigQuery and Snowflake available in the community edition

    BigQuery and Snowflake script languages move out of the Enterprise license into every edition including self-hosted Community; Oracle and MS SQL stay Enterprise. A packaging shift that widens the open-core surface without adding new capability.

    View source ↗
  2. 8d ago

    AI sessions (beta)

    View source ↗
  3. 9d ago

    Attach text files to AI chat messages

    View source ↗
  4. 10d ago

    Web search sources in AI chat

    View source ↗
  5. 14d ago

    Automatic git-to-Windmill sync

    Git-to-Windmill sync now runs fully in-app via webhooks and polling, making the documented GitHub Actions optional and adding PR promotion and diff checks — completing the two-way GitOps loop natively.

    View source ↗
  6. 25d ago

    Schema contracts for pipeline consumers

    Consumer scripts against materialized DuckLake tables are validated against producer schemas at save time, surfacing missing columns and broken lineage as non-blocking warnings — part of hardening DuckLake for real pipelines.

    View source ↗