LoginSign Up
← Back to all sparks
R

r-owidapi

ANALYTICS
Velocity2.5

R package r-owidapi by tidy-intelligence — release notes from GitHub.

github.com/tidy-intelligence/r-owidapi ↗

The R client for Our World in Data found its search had been reading a tenth of the catalog.

open-dataour-world-in-datar-packageapi-wrapperdata-qualitytesting
◆Current state
owidapi is a small R client for Our World in Data, covering chart data retrieval, metadata, the full chart catalog, and search over it, with experimental Shiny output helpers. It is three releases old and the most recent one is almost entirely repair: the catalog function was silently truncating at 1000 rows because of a Datasette row cap, which meant search had been operating on a fraction of what exists.
◆Where it's heading
Development is about making a thin wrapper trustworthy against an upstream that moves without notice. The truncation fix pages through the catalog properly; a separate fix stops the function breaking when Our World in Data dropped a column, by parsing typed columns only when present. Tests moved to mocked responses, with a small live suite retained purely to detect schema drift and skipped on CRAN — a sensible design for a package whose main risk is that the API changes shape rather than that the code is wrong. The user-facing surface has not grown since the initial release; the work is in defending it.
◆Prediction
On this pattern the next release is likelier to be another upstream-compatibility fix than new functionality, with the schema-drift tests the mechanism that surfaces it.

◆Recent moves

  1. 20d ago

    Catalog was silently capped at 1000 charts; now paged in full

    A silent-truncation bug with real consequences: Datasette caps the size parameter, so the catalog returned only its first 1000 charts and search covered only that slice — with no error to indicate anything was missing. The catalog is now paged through in full, and a second fix makes column parsing conditional so an upstream schema change no longer breaks the function outright. Tests move to mocked responses, keeping a small live suite to detect exactly this class of drift.

    View source ↗
  2. 1y ago

    Request logic consolidated, with graceful catalog errors

    Internal consolidation of request handling plus softer failure behaviour in the catalog function. Invisible to callers, but it is the refactor that made the following release's fixes tractable.

    View source ↗
  3. 1y ago

    First release: data, metadata, catalog and search

    The initial surface — retrieval, metadata, catalog listing, and search — plus experimental Shiny output and server helpers. That surface has not changed since; everything after has been about making it return what it claims to.

    View source ↗