LoginSign Up
← Back to all sparks
M

mirai

ANALYTICS
Velocity2.5

Minimalist Async Evaluation Framework for R. Contribute to r-lib/mirai development by creating an account on GitHub.

github.com/r-lib/mirai ↗

mirai removed its dispatcher process and added memory backpressure to the queue.

parallel-computingasyncbackpressureshinydaemons
◆Current state
The async evaluation framework releases roughly monthly and moves fast at the architecture level. In 2.7.0 the dispatcher stopped being a separate process and became a thread, after its loop had already been rewritten in C inside nanonext one release earlier. The same release added an opt-in memory budget for queued task payloads and try_mirai(), which returns NULL immediately rather than blocking when that budget is exhausted.
◆Where it's heading
Two threads of work run together: cutting overhead out of the task path — thread-based dispatcher, in-process transport for synchronous daemons, lower per-element dispatch cost in mirai_map() — and making the framework safe to embed in an event loop, where blocking the host R thread is not acceptable. Deployment reach is growing too, with http_config() launching remote daemons over HTTP APIs and auto-configuring for Posit Workbench. Each release pins a minimum nanonext version, so the two packages advance as one unit.
◆Prediction
With backpressure in place but opt-in, the open question these notes leave is whether a default memory budget arrives; continued overhead reduction and Shiny-facing non-blocking paths are the safer bet.

◆Recent moves

  1. 25d ago

    Map collection and daemon lifecycle fixes

    A fix-heavy release concentrated on mirai_map() collection and daemon lifecycle: non-mirai errorValues now produce informative errors, .flat no longer ignores an error in the first element, and daemons restore options modified during evaluation. Dispatch overhead drops again for mirai_map() and everywhere().

    View source ↗
  2. 2mo ago

    Agent skill ships in-package; HTTP headers take over auth

    Ships mirai's agent skill inside the installed package where btw and packages following the same convention pick it up automatically, and makes headers the primary way to supply HTTP authentication in http_config(). Also fixes load-balanced parallel functions hanging when tasks outnumber nodes.

    View source ↗
  3. 3mo ago

    Dispatcher becomes a thread, and the queue gains a memory budget

    ⚡ SPARK

    The dispatcher becomes a thread rather than a separate process, and queued task payloads gain an opt-in memory budget with try_mirai() as the non-blocking path when it is exhausted. It changes both the process model and what happens when a producer outruns its workers.

    View source ↗
  4. 5mo ago

    Parallel RNG seeding leaves experimental status

    Takes the reproducible parallel RNG seed argument out of experimental status and works around Posit Workbench authentication cases. No change to the task path.

    View source ↗
  5. 6mo ago

    Remote daemons over HTTP, and a C dispatcher loop

    Adds http_config() for launching remote daemons through HTTP APIs, auto-configured for Posit Workbench, and moves the dispatcher loop entirely into C inside nanonext for roughly half the previous overhead. race_mirai() changes to return an index rather than the list, a breaking but small change.

    View source ↗
  6. 8mo ago

    Telemetry span timing and daemon-switch fix

    Corrects when OpenTelemetry daemon-connect spans are recorded, fixes an unresolved-mirai bug when switching daemon types mid-session, and makes mirai_map() return invisibly.

    View source ↗