LoginSign Up
← Back to all sparks
B

bcdata

INFRA · APIS
Velocity0.0

Search and Retrieve Data from the BC Data Catalogue

github.com/bcgov/bcdata ↗

The R gateway to B.C.'s data catalogue, mostly busy keeping pace with the servers behind it.

open-dataspatial-dataapi-clientcql-queriesr-package
◆Current state
bcdata searches the British Columbia Data Catalogue and pulls records from it, including spatial layers retrieved through WMS and WFS with dplyr-style filtering translated into CQL queries sent to the server. The recent releases are small and corrective: 0.5.3 fixes malformed CQL generated by geometry predicates such as INTERSECTS, which the server had been rejecting outright, and 0.5.2 drops a leaflet extension dependency in favour of a base leaflet call. Cadence is a few releases a year.
◆Where it's heading
Almost everything in this window is a response to something changing on the other side of the connection: a CKAN 2.9 upgrade that stopped search and listing functions returning complete results, WMS and WFS capability requests that cannot be relied on, dbplyr tightening the rules for local evaluation, and general catalogue-side churn. The package's own interface has been stable since the 0.4.0 change that required local() around locally-evaluated filter calls, and the work since has gone into keeping that stable interface working rather than extending it.
◆Prediction
Expect the same pattern: targeted fixes as the catalogue and its CKAN, WMS and WFS layers change, with occasional dependency pruning of the kind 0.5.2 did. Nothing in these entries points toward a 0.6 or a change in what the package covers.

◆Recent moves

  1. 1mo ago

    Geometry predicate filters no longer rejected by the server

    Filter calls using CQL geometry predicates such as INTERSECTS were producing malformed queries that the server refused with an HTTP 400, which made spatial filtering unusable rather than merely wrong. A single-item release, but it restores the capability that distinguishes this package from a plain catalogue client.

    View source ↗
  2. 6mo ago

    leaflet.extras dependency dropped for WMS legends

    Swaps a leaflet.extras legend call for the equivalent built on leaflet's own addControl, removing a dependency without changing what users see. Routine pruning of the sort this package does periodically.

    View source ↗
  3. 1y ago

    Search and listing functions return complete results again

    Repairs facet search and the group and organization listing functions, which had been silently returning partial results since the catalogue moved to CKAN 2.9, and adds package-specific options to control those queries. Silent truncation is the worst failure mode for a search client, so this matters more than its one-paragraph note suggests.

    View source ↗
  4. 1y ago

    Hardened against failing WMS and WFS capability requests

    Makes the package tolerate WMS and WFS GetCapabilities requests that do not come back cleanly, adapts to dbplyr 2.5.0 tightening the rules around local evaluation, and deprecates a download-limit option made redundant by the server-set chunk limit. Defensive work against an upstream that cannot be relied on.

    View source ↗
  5. 3y ago

    JSON resources readable; organization listing helpers added

    Adds JSON as a supported resource format so those records can be downloaded and read directly, plus helpers for listing organizations and their records, and fixes facet values that needed manual quoting to produce a valid query. Modest reach extension into parts of the catalogue that previously required manual handling.

    View source ↗
  6. 3y ago

    filter() now requires local() for locally-evaluated calls

    Stops auto-detecting which function calls inside filter() need evaluating locally before the CQL query is built, requiring an explicit local() wrapper instead. A breaking change, but one that brings bcdata into line with how other dbplyr backends behave, and the package's interface has been stable since.

    View source ↗