n2kanalysis
n2kanalysis has spent eight years wiring INLA models to an S3 bucket.
Comparison · Analytics
filearray vs spanishoddata
A side-by-side editorial comparison of filearray and spanishoddata — release velocity, themes, recent moves, and the top alternatives to consider.
filearray vs spanishoddata: at a glance
What is filearray?
The on-disk array layer under RAVE spends its releases hunting segfaults.
filearray stores large arrays on disk and reads them back with little memory overhead, serving as the storage substrate for the RAVE intracranial EEG stack. The 0.2.2 release fixes out-of-bound indexing that caused segfaults along certain margins and an ASAN-flagged signed integer overflow in the load path. The user-facing API has been stable since 0.1.6.
What is spanishoddata?
spanishoddata spent a year finding out its 2020-2021 data was quietly incomplete.
spanishoddata provides access to Spain's open mobility origin-destination datasets from the Ministry of Transport, converting them into DuckDB and parquet for analysis at scale. Nearly every release in this window is a data-fidelity fix rather than a feature: district-to-municipal reaggregation was wrong for the 2020-2021 vintage, literal 'NA' strings in the source CSVs broke DuckDB enum casting, and the Amazon S3 metadata bucket turned out to be truncated at March 2021, silently hiding data.
filearray vs spanishoddata: editorial side-by-side
filearray
0.0ANALYTICS
The on-disk array layer under RAVE spends its releases hunting segfaults.
◆ Current state
filearray stores large arrays on disk and reads them back with little memory overhead, serving as the storage substrate for the RAVE intracranial EEG stack. The 0.2.2 release fixes out-of-bound indexing that caused segfaults along certain margins and an ASAN-flagged signed integer overflow in the load path. The user-facing API has been stable since 0.1.6.
◆ Where it's heading
This is infrastructure whose release history reads as a memory-safety log: unprotected C++ variables, buffer sizes exceeding array length, allocations one byte short, endianness on big-endian platforms, and now out-of-bound margins caught by sanitizers. The one sustained feature direction is reducing the cost of operating on arrays too large for memory — lazy operator evaluation through a proxy class, fmap-style application, and marginal collapse. Portability work has steadily removed hard requirements, dropping the C++11 declaration and swapping OpenMP for TinyThreads to get parallelism on macOS.
◆ Prediction
Expect continued sanitizer-driven patches rather than new interfaces; the three-year gap before 0.2.2 suggests releases now arrive only when a crash or a CRAN check demands one.
spanishoddata
0.0ANALYTICS
spanishoddata spent a year finding out its 2020-2021 data was quietly incomplete.
◆ Current state
spanishoddata provides access to Spain's open mobility origin-destination datasets from the Ministry of Transport, converting them into DuckDB and parquet for analysis at scale. Nearly every release in this window is a data-fidelity fix rather than a feature: district-to-municipal reaggregation was wrong for the 2020-2021 vintage, literal 'NA' strings in the source CSVs broke DuckDB enum casting, and the Amazon S3 metadata bucket turned out to be truncated at March 2021, silently hiding data.
◆ Where it's heading
The package is in a trust-building phase. The pattern across 0.2.1 through 0.2.6 is the maintainers repeatedly discovering that upstream metadata and the package's own aggregation were misrepresenting what data existed, then fixing it and adding a check so it surfaces next time. That is now backed by infrastructure: comprehensive unit tests plus weekly live-data runs on GitHub workers that alert maintainers when the upstream ministry changes something. The last feature release sits outside the six-entry window, which is itself the story.
◆ Prediction
Expect continued upstream-tracking fixes as the ministry's API and S3 layout shift, with the experimental quick-access and checksum functions the most likely candidates for promotion to stable.
Alternatives to filearray and spanishoddata
Other Analytics products tracked by Sparkpulse, ranked by recent ship velocity. Each card links to a full editorial trajectory and lets you pivot into a head-to-head comparison with either filearray or spanishoddata.
n1qn1c
A Fortran-descended optimizer got thread-safe, then found two flags that never worked.
ggstatsplot
ggstatsplot reached 1.0 by adding tests, having outsourced its statistics years ago.
collapse
collapse got a JSS paper and a 7x fmean speedup in the same release.
gtsummary
gtsummary is quietly rebuilding itself around analysis results data, one table verb at a time.
broadcast
broadcast is filling in NumPy-style array broadcasting for R, operator by operator.
See all filearray alternatives → · See all spanishoddata alternatives →
Recent activity from filearray and spanishoddata
Latest ship moves from both products, interleaved chronologically. ⚡ = editorial spark.
- 2mo agospanishoddataRedirected district files identified in v1 metadata
- 2mo agofilearraySegfault on out-of-bound margins fixed
- 2mo agospanishoddataS3 metadata truncation at March 2021 bypassed via XML feed
- 4mo agospanishoddataLiteral NA strings no longer break DuckDB casting
- 4mo agospanishoddataLarge urban area zones can be reloaded again
- 5mo agospanishoddatatime_slot column removed; test coverage goes live-weekly
- 1y agospanishoddataDistrict-to-municipal reaggregation corrected for 2020-2021
- 3y agofilearrayLazy operator evaluation and macOS parallelism
- 3y agofilearraySequential read bug in fmap corrected
- 4y agofilearrayPartition limit removed by opening files on demand
- 4y agofilearrayHeader signatures and symbolic link detection
- 4y agofilearrayFlush timing left to the operating system
Frequently asked questions
What is the difference between filearray and spanishoddata?
They serve adjacent needs but don't currently overlap on shipped themes. filearray and spanishoddata are shipping at a similar cadence (velocity 0.0 vs 0.0, both within Sparkpulse's "active" band). See the at-a-glance table above for a side-by-side breakdown of velocity, recent sparks, and editorial themes.
Is filearray better than spanishoddata?
Sparkpulse doesn't pick a winner — we score release velocity, not feature parity. filearray and spanishoddata are shipping at a similar cadence (velocity 0.0 vs 0.0, both within Sparkpulse's "active" band). For your specific use case, the alternatives sections above list other Analytics products to evaluate alongside.
What are the best alternatives to filearray?
Top filearray alternatives in Analytics are ranked by recent ship velocity. Browse the "filearray alternatives" section above for the current picks, or visit /alternatives/filearray-r for the full list with editorial commentary on each.
What are the best alternatives to spanishoddata?
Top spanishoddata alternatives in Analytics are ranked by recent ship velocity. Browse the "spanishoddata alternatives" section above for the current picks, or visit /alternatives/spanishoddata for the full list with editorial commentary on each.