LoginSign Up
← Back to all sparks
T

textreuse

ANALYTICS
Velocity2.5

Detect text reuse and document similarity. Contribute to ropensci/textreuse development by creating an account on GitHub.

github.com/ropensci/textreuse ↗

A dormant text-matching package revived, shipped as 1.0.0, and kept current with the tidyverse.

text-reuseminhashlshr-packagemaintenancecran
◆Current state
textreuse detects reused and quoted passages across document collections using minhash and locality-sensitive hashing, with local alignment for inspecting the matches it finds. After years of inactivity, the package reached a 1.0.0 CRAN release in May 2026 that folded accumulated feature work into one version — encoding control on corpus construction, deterministic skipped-document bookkeeping, and an align_local() that returns an empty alignment instead of erroring on non-matching texts. The 1.0.2 release since then is pure compatibility maintenance.
◆Where it's heading
The arc here is restoration rather than expansion. The work has gone into making the package survivable — silencing deprecated dplyr and tidyr selection and many-to-many join warnings, moving from dead Travis and AppVeyor configs to GitHub Actions, and validating across five R platform and version combinations. Release notes now lead with verification evidence rather than features, which is the signature of a maintainer stabilizing an inherited codebase.
◆Prediction
Expect continued compatibility releases tracking tidyverse deprecations; nothing in these entries indicates new hashing or alignment capability is planned.

◆Recent moves

  1. 20d ago

    Compatibility pass for current dplyr and tidyr

    Locality-sensitive hashing helpers updated to stop emitting deprecated-selection and many-to-many join warnings under current dplyr and tidyr, with Travis and AppVeyor replaced by GitHub Actions. No behavioral change users would observe beyond quieter output.

    View source ↗
  2. 3mo ago

    CRAN resubmission fixing a moved README URL

    A same-day resubmission of 1.0.0 correcting a relocated devtools URL in the README. Procedural, and only notable as evidence of how tight the CRAN return was.

    View source ↗
  3. 3mo ago

    1.0.0 consolidates years of accumulated feature work

    The release that brought the package back to CRAN, bundling encoding control on TextReuseTextDocument() and TextReuseCorpus(), deterministic skip bookkeeping, more predictable handling of very short documents, and an align_local() that returns an empty alignment rather than throwing when two texts share no words. The alignment change is the one that matters operationally, since it lets batch pipelines run without per-pair error handling.

    View source ↗
  4. 10y ago

    Minhashes split out from hashes in document objects

    An early development build separating x$minhashes from hashes on TextReuseTextDocument, with accessors of its own, so lsh() consumes minhashes directly. The point was to let callers discard tokens after hashing rather than retain them for possible rehashing, which is the expensive part of the pipeline in both memory and compute.

    View source ↗