authentik
The first patch on 2026.8 is twenty-odd fixes and no new surface.
Comparison · Infra & APIs
TestingBot vs Windmill
A side-by-side editorial comparison of TestingBot and Windmill — release velocity, themes, recent moves, and the top alternatives to consider.
Shared themes:ai-agents
TestingBot vs Windmill: at a glance
What is TestingBot?
A browser grid that spent the summer becoming a team product, and is now opening itself to agents.
TestingBot runs a cross-browser and real-device cloud, and the last three months have gone almost entirely into what surrounds a test run rather than the run itself: custom roles enforced identically in dashboard and API, service accounts so pipelines stop borrowing a person's credentials, enforced two-factor authentication, comments with @mentions on failed runs, one-click filing into GitHub, Jira, Trello, Asana or Azure DevOps, and a notification inbox tying it together. The grid work continues underneath — WebRTC streaming for iOS simulators, a rebuilt web inspector, new devices and browser versions on the usual cadence.
What is Windmill?
Windmill gives its AI agents a test harness, the piece that lets them into production flows
Windmill is a developer platform for scripts, flows and internal apps, shipping weekly across three lanes. The AI lane is the busiest: sessions that build scripts and flows as reviewable drafts, a read-only plan mode that refuses writes until you approve, version history on session artifacts, and now evaluation for reusable agents. The runtime lane keeps absorbing data tooling — dbt projects run natively, BigQuery and Snowflake came out from behind the Enterprise licence. The workspace lane is consolidating deployment semantics around fork lineage.
TestingBot vs Windmill: editorial side-by-side
TestingBot
5.0INFRA · APIS
A browser grid that spent the summer becoming a team product, and is now opening itself to agents.
◆ Current state
TestingBot runs a cross-browser and real-device cloud, and the last three months have gone almost entirely into what surrounds a test run rather than the run itself: custom roles enforced identically in dashboard and API, service accounts so pipelines stop borrowing a person's credentials, enforced two-factor authentication, comments with @mentions on failed runs, one-click filing into GitHub, Jira, Trello, Asana or Azure DevOps, and a notification inbox tying it together. The grid work continues underneath — WebRTC streaming for iOS simulators, a rebuilt web inspector, new devices and browser versions on the usual cadence.
◆ Where it's heading
Two directions are visible. The platform is being made safe for organisations rather than individuals, which is what roles, service accounts, 2FA and audit-friendly reporting add up to. Alongside it, the grid is being made reachable by software instead of people: an Insights API exposing pass rates and flakiness, embeddable build reports for CI dashboards, GitHub checks on every pull request, and now Vibium's CLI and MCP server pointed straight at the grid over WebDriver BiDi. The AI work so far reads results — failure analysis, documentation questions — while Vibium is the first path where something other than a human-written script drives the browser.
◆ Prediction
Expect the agent-facing path to be developed further, since supporting Vibium's MCP server without extra configuration is a small step from offering a first-party one, and expect the device and browser catalogue to keep rotating on its established cadence.
Windmill
6.3INFRA · APIS
Windmill gives its AI agents a test harness, the piece that lets them into production flows
◆ Current state
Windmill is a developer platform for scripts, flows and internal apps, shipping weekly across three lanes. The AI lane is the busiest: sessions that build scripts and flows as reviewable drafts, a read-only plan mode that refuses writes until you approve, version history on session artifacts, and now evaluation for reusable agents. The runtime lane keeps absorbing data tooling — dbt projects run natively, BigQuery and Snowflake came out from behind the Enterprise licence. The workspace lane is consolidating deployment semantics around fork lineage.
◆ Where it's heading
The AI work has moved through a recognisable sequence: first let the assistant build things, then constrain what it may do without approval, then make its output reviewable and revertible, and now make agents measurable against a fixed set of cases. That is the arc of taking AI from a demo surface to something you are willing to put in a scheduled flow. In parallel, Windmill keeps pulling data-stack runtimes inside its own execution model rather than integrating with them, and keeps giving away runtimes that were previously licensed.
◆ Prediction
Evals are the obvious hook for regression gating, so expect them to be callable from a flow or CI rather than only from the agent card, and expect the eval run history to feed the same version-comparison surface the session artifacts already use.
Alternatives to TestingBot and Windmill
Other Infra & APIs products tracked by Sparkpulse, ranked by recent ship velocity. Each card links to a full editorial trajectory and lets you pivot into a head-to-head comparison with either TestingBot or Windmill.
incident.io
Alert rate limits stop being per-source and start being per-tenant, by query.
Warp
The essay series keeps running ahead of the products it describes.
Node-RED
Both lines patch on the same day, and the reverted JSONata upgrade finally sticks.
werf
The 3.x line finally ships function: authenticated secret writes and render patches.
Infisical
Folder-level RBAC lands, and approvals finally get a webhook to talk to.
See all TestingBot alternatives → · See all Windmill alternatives →
Recent activity from TestingBot and Windmill
Latest ship moves from both products, interleaved chronologically. ⚡ = editorial spark.
- 9d agoWindmillAI agent evals (beta) ⚡
- 19d agoWindmillPlan mode for AI sessions
- 22d agoTestingBotVibium browser automation
- 22d agoWindmillNested filter groups and dotted paths in trigger filters
- 23d agoWindmillVersion history for AI session artifacts
- 28d agoTestingBotSamsung Galaxy Tab S11 in the real device cloud
- 1mo agoWindmillRun dbt projects as a first-class Windmill runtime ⚡
- 1mo agoWindmillOne deployment target, derived from the workspace lineage
- 1mo agoTestingBotEmbeddable build reports for CI
- 1mo agoTestingBotNotifications in the dashboard
- 1mo agoTestingBotChrome 150 and Edge 150
- 1mo agoTestingBotGitHub pull request checks
Frequently asked questions
What is the difference between TestingBot and Windmill?
Both compete on the same themes — ai-agents — within Infra & APIs. Windmill is currently shipping more aggressively (velocity 6.3 vs 5.0), with 1 editorial sparks in the last 30 days against 0. See the at-a-glance table above for a side-by-side breakdown of velocity, recent sparks, and editorial themes.
Is TestingBot better than Windmill?
Sparkpulse doesn't pick a winner — we score release velocity, not feature parity. Windmill is currently shipping more aggressively (velocity 6.3 vs 5.0), with 1 editorial sparks in the last 30 days against 0. For your specific use case, the alternatives sections above list other Infra & APIs products to evaluate alongside.
What are the best alternatives to TestingBot?
Top TestingBot alternatives in Infra & APIs are ranked by recent ship velocity. Browse the "TestingBot alternatives" section above for the current picks, or visit /alternatives/testingbot for the full list with editorial commentary on each.
What are the best alternatives to Windmill?
Top Windmill alternatives in Infra & APIs are ranked by recent ship velocity. Browse the "Windmill alternatives" section above for the current picks, or visit /alternatives/windmill for the full list with editorial commentary on each.