LoginSign Up
← Back to all sparks
B

bigrquery

ANALYTICS
Velocity0.0

R interface to Google BigQuery for querying and uploading large datasets

github.com/r-dbi/bigrquery ↗

bigrquery went MIT, then handed its slowest path to the BigQuery Storage API

bigquerydbidbplyrlicensingperformance
◆Current state
bigrquery is the R client for Google BigQuery. Version 1.5.0 was the structural release - MIT relicensing, removal of the long-deprecated non-bq_ API, and a move to the second edition of the dbplyr interface with a much fuller DBI implementation. Since then the work has been about the two things that hurt in practice: download throughput and cost visibility.
◆Where it's heading
The package is settling into being a well-behaved DBI and dbplyr backend rather than a bespoke API wrapper, and offloading its hard parts to specialist packages - clock for date parsing, bigrquerystorage for bulk downloads, gargle for auth. The recent additions read like responses to production use: job labels for cost allocation, microsecond timestamp precision, a configurable quiet option.
◆Prediction
Expect bigrquerystorage to move from optional to expected for large reads, and further work on upload fidelity, where digits and timezone handling have needed repeated correction.

◆Recent moves

  1. 3mo ago

    Job labels for cost allocation; microsecond upload precision

    Operational polish aimed at people running BigQuery at cost: queries can attach BigQuery labels for cost allocation, set per-connection, and uploads gain configurable numeric precision plus microsecond POSIXt handling with the timezone set correctly. The cli parse failure on server messages containing braces was hiding real errors.

    View source ↗
  2. 11mo ago

    Uses bigrquerystorage automatically for large downloads

    If bigrquerystorage is installed, every download path - collect(), dbGetQuery(), dbFetch() - routes through it, which the maintainers describe as a drastic speedup on large datasets. It fits the offloading pattern: the hard part is solved in a specialist package and picked up here when present.

    View source ↗
  3. 11mo ago

    Development snapshot advancing 1.4.0 deprecations

    A development-version tag whose only content is advancing deprecations from 1.4.0 along their schedule. It appears in the feed because the repository tags in-progress versions, not because anything shipped.

    View source ↗
  4. 2y ago

    Forward compatibility with an upcoming dbplyr release

    A single-line release keeping the dbplyr backend working against an unreleased dbplyr. The recurring tax of being a dbplyr backend, paid here in one commit.

    View source ↗
  5. 2y ago

    MIT relicensing, dbplyr second edition, full DBI support

    ⚡ SPARK

    The release that set the shape of everything after it: a license change, the removal of the pre-bq_ API, a move to the second edition dbplyr interface, and DBI support filled in to the point that BigQuery behaves like any other database connection. Later releases refine this design rather than change it.

    View source ↗
  6. 3y ago

    Syncs with gargle's OAuth client rename

    Auth changes arrive from gargle rather than from here: bq_oauth_app() gives way to bq_oauth_client() and the preferred path becomes supplying the JSON from the Google Developers Console. A clean example of the auth layer being owned elsewhere and propagating downward.

    View source ↗