LoginSign Up
← Back to all sparks
R

ralger

DEVOPS
Velocity0.0

Easy Web Scraping

github.com/feddelegrand7/ralger ↗

ralger stopped scraping only web pages and started scraping the files on them.

web-scrapingdata-extractionfile-formatsrvestr-package
◆Current state
ralger is a scraping package that wraps rvest behind task-named functions — titles_scrap(), table_scrap(), images_scrap() and so on — aimed at users who want data out of a page without writing selector logic. After four years of quiet it returned in 2.3.0 with a different kind of function: readers for PDF, XLS, XLSX and CSV files, plus comment extraction.
◆Where it's heading
The earlier arc was about making HTML scraping survive contact with the real web — wrapping every function in tryCatch so a dead link or missing connection returns NA with a message rather than an error, adding case-sensitivity control, widening heading extraction to h3. The 2.3.0 additions change the target rather than the robustness: the unit of interest becomes the document a page links to, not the page itself.
◆Prediction
If the file readers follow the pattern the image functions set, expect preview-and-batch companions next — images_scrap() arrived alongside images_preview() for exactly that reason. The four-year gap before this release makes timing unpredictable.

◆Recent moves

  1. 1y ago

    ralger 2.3.0 adds PDF, Excel and CSV scraping

    ⚡ SPARK

    Five new functions extend the package past HTML for the first time — pdf_scrap(), xls_scrap(), xlsx_scrap(), csv_scrap() and comments_scrap(). After four years of robustness work on page scraping, this release changes what ralger can read rather than how reliably it reads it.

    View source ↗
  2. 5y ago

    ralger 2.2.2 adds attribute and missing-alt image scraping

    attribute_scrap() generalizes extraction to arbitrary HTML attributes, and images_noalt_scrap() targets images lacking alt text — a small accessibility-auditing use case. Network-dependent tests are also skipped on CRAN.

    View source ↗
  3. 5y ago

    ralger 2.2.1 adds image download and preview

    images_scrap() downloads images from a page and images_preview() lists their URLs first — the preview-then-fetch pairing that makes bulk downloading reviewable. paragraphs_scrap() also gains error handling and empty elements are stripped from scrap().

    View source ↗
  4. 5y ago

    ralger 2.2.0 returns NA instead of erroring on dead links

    Every function is wrapped in tryCatch so missing connectivity or an invalid link produces a message and NA rather than an exception. For scraping loops over many URLs this is the difference between a partial result and no result.

    View source ↗
  5. 5y ago

    ralger 2.1.0 adds h3 headings and case-sensitive matching

    titles_scrap() picks up h3 elements alongside h1 and h2, and swapping stringr's str_detect for base grepl gives users control over case sensitivity. Incremental widening of what counts as a title.

    View source ↗
  6. 6y ago

    ralger 2.0.1 adds fill for ragged tables

    A contributed fill argument passed through to rvest lets table_scrap() handle tables with inconsistent row and column counts — the single most common reason HTML table extraction fails outright.

    View source ↗