LoginSign Up
← Back to all sparks
R

rerddap

DEVOPS
Velocity0.0

R client for working with ERDDAP servers

github.com/ropensci/rerddap ↗

An ERDDAP client steadily lowering the cost of pulling large oceanographic extracts

oceanographic-datar-packageapi-clientparquetdata-retrieval
◆Current state
rerddap fetches gridded and tabular data from ERDDAP servers into R. The recent releases follow one practical thread — making large extracts cheaper to move — alongside routine robustness work on error messages and failure modes. 1.3.0 brings parquet output to `griddap()` and adds a function that estimates a download's size before the request is made.
◆Where it's heading
Parquet support has propagated across the package one function at a time: introduced for `tabledap()` in 1.2.0 where the maintainer reports up to 90% smaller files for large extracts, patched in 1.2.3, and extended to `griddap()` in 1.3.0. The size-estimation function added alongside it addresses the same problem from the other end, letting a user find out what a request will cost before committing to it. Around this, the package has absorbed the usual CRAN-driven hardening, notably making all external resource requests fail gracefully.
◆Prediction
Expect the remaining data-access paths to converge on the same parquet-plus-pre-flight-estimate pattern, since that is how the last three feature releases have each extended it. The entries do not indicate work beyond data retrieval.

◆Recent moves

  1. 1mo ago

    Parquet output for griddap and pre-request size estimates

    `griddap()` gains parquet output, extending the format support `tabledap()` received in 1.2.0, and a new function estimates a `griddap()` download's size before the request is made. Together they address the two things that make large gridded extracts painful — transfer size, and not knowing it in advance.

    View source ↗
  2. 5mo ago

    Fix for parquet output in tabledap

    A narrow fix for a minor issue when using `fmt = 'parquet'` with `tabledap()`, tidying the format support added in 1.2.0.

    View source ↗
  3. 6mo ago

    Interactive plotting docs and clearer error messages

    Documentation on producing interactive plots plus improved error messages. No change to data retrieval behaviour.

    View source ↗
  4. 1y ago

    Graceful failure when ERDDAP servers are unavailable

    All requests to external resources now fail gracefully when a resource is unavailable, with vignette examples updated. This is the standard CRAN requirement for packages that depend on remote services rather than a change users request.

    View source ↗
  5. 1y ago

    Parquet downloads and units attributes for tabledap

    The release that started the parquet thread: against ERDDAP 2.25 or later, `tabledap()` can download parquet rather than csv, which the maintainer reports as up to 90% smaller for large extracts. It also attaches units as attributes to the returned dataframe, fixes `griddap` failing on large coordinate values in projected metre-based data, and makes `browse()` return a URL in non-interactive settings as documented.

    View source ↗
  6. 2y ago

    tabledap returns types matching the server dds

    `tabledap` now returns the same data type as declared in the dataset's dds, aligning returned types with the server's own description.

    View source ↗