LoginSign Up
← Back to all sparks
A

Apache TsFile

ANALYTICS
Velocity2.5

Columnar storage file format designed for time-series data in IoT scenarios

tsfile.apache.org ↗

TsFile 2.5 adds Go to its Java, C++ and Python implementations.

time seriescolumnar formatapache arrowmulti-languageiotdb
◆Current state
Apache TsFile is the columnar time-series file format underneath IoTDB. The 2.x line has concentrated on the native and Python ends: Arrow-compatible results into Python DataFrames in 2.3.0, CSV, Parquet and Arrow conversion scripts in 2.3.1, SIMD and parallel reads in 2.4.0, and in October 2.5.0 with table-model read and write APIs for Go plus persistent dataset indexes. The old 1.1 line still gets occasional backport releases, most recently 1.1.4 in September.
◆Where it's heading
TsFile is being shaped into an interchange format that non-JVM tools can read and write directly, with each release widening language coverage or speeding up the native path. The 1.1 backports show IoTDB users on older versions are still being supported.
◆Prediction
Expect a 2.5.x patch release fixing the new Go and index features, following the pattern of 2.3.1 after 2.3.0.

◆Recent moves

  1. 26d ago

    Apache TsFile 2.5.0

    TsFile 2.5.0 adds table-model read and write APIs for Go, persistent dataset indexes and a native query runtime for the Python DataFrame, and binary format support across Java, C++ and Python.

    View source ↗
  2. 1mo ago

    TsFile 1.1.4 backports metadata and filter fixes

    A backport patch on the 1.1 line: chunk metadata mask, TimeFilter infinite-loop and schema deserialization fixes. Maintenance for older IoTDB deployments while new work happens on 2.x.

    View source ↗
  3. 3mo ago

    SIMD and parallel read paths in Apache TsFile 2.4.0

    The heaviest C++ performance release in the window: batch codec APIs, single-pass TsBlock decoding, AVX2 and NEON SIMD paths, chunk-level parallel reads and column-parallel writes, plus faster metadata and tag lookups. On the Python side TsFileDataFrame extends to tree-model files, continuing the DataFrame reach that 2.3.0 established.

    View source ↗
  4. 5mo ago

    CSV, Parquet and Arrow conversion scripts for TsFile 2.3.1

    Adds tooling to convert CSV, Parquet and Arrow data into TsFile, with follow-up fixes for date and timestamp types, uppercase column names and non-ASCII table names. Together with the Arrow read path in 2.3.0, this closes the loop on moving data both directions between TsFile and the mainstream columnar ecosystem.

    View source ↗
  5. 6mo ago

    Arrow-backed DataFrames and paginated reads in TsFile 2.3.0

    The release that makes the Python path fast rather than merely present: the C++ layer returns Arrow-compatible results so the Python binding can hand back Arrow-backed DataFrames directly. Alongside it come row-number queries with limit and offset, nested-directory traversal, and C++ interfaces for series-level timestamp ranges and for salvaging data from corrupted files.

    View source ↗
  6. 6mo ago

    Write-time schema changes and read/write encryption in TsFile 2.2.1

    A Java-weighted release: schema can be modified mid-write, memory usage can be estimated for Tablets, I/O sizes are recorded during reads, and encryption becomes configurable for both directions. Python gains DataFrame-to-TsFile conversion and the newer date and timestamp types, the write-side counterpart to the read work in 2.3.0.

    View source ↗