Buildkite vs Langfuse

Langfuse's recent work is concentrated almost entirely on the evaluation surface. Experiments were rebuilt as a top-level feature that runs with or without a dataset attached, and can be compared across runs over time. The LLM-as-a-Judge evaluator gained categorical scores in late March and boolean true/false scores a week later, filling out the score types beyond plain numerics. Everything else in the window is documentation or scrape artifacts.

The direction is evaluation as the product's centre of gravity rather than an appendage to tracing. Decoupling Experiments from Datasets removes the setup cost of running an eval, and the widening score types let judges express verdicts rather than only magnitudes — both point at teams running evals continuously against live traces instead of curated fixtures. Regional expansion shows up in the feed as Langfuse Cloud Japan. Cadence is the open question: nothing has published since April 21, so this arc is described from a three-month-old window.

The score-type buildout and the run-comparison view are converging on scheduled or triggered evaluations against production traces, but the feed has been silent long enough that the next move cannot be called with confidence from these entries alone.