LoginSign Up
← Back to all sparks
D

Deep Lake

AI-ASSISTANTS
Velocity0.0

Database and data lake purpose-built for AI workloads

www.activeloop.ai ↗

Deep Lake is rebuilding itself as a Postgres extension.

vector-storagepostgres-extensiondataset-versioningquery-enginemultimodal-data
◆Current state
The visible release history is thin — three entries spanning a version 3 patch and two version 4 releases. The 4.x work splits between the core dataset format and pg_deeplake, a Postgres extension that has been gaining SQL type support, automatic table reload and library preloading. The 4.4.1 release added a storage directory listing API, mesh type support, PLY visualisation, a simple visualiser, and a 30% improvement in LRU cache insertion time.
◆Where it's heading
Two things stand out. The query engine was separated from the execution module and group-by execution was pulled out on its own, which is architecture work done ahead of features rather than after them. And the pg_deeplake investment points at meeting users inside the database they already query rather than asking them to adopt a separate dataset API. Version-locked read-only views fit the same picture — reproducible reads for teams treating datasets as versioned artefacts.
◆Prediction
The query core separation and group-by refactor were both described as groundwork, so query execution features are the likely next visible step in pg_deeplake.

◆Recent moves

  1. 8mo ago

    Mesh type support, dataset visualisers and faster cache insertion

    Capability spread across three fronts: mesh and PLY support widen the data types Deep Lake holds, the visualiser and directory listing API make stored data inspectable, and version-locked read-only views serve teams that need reproducible reads. The 30% LRU cache insertion gain is the measurable part.

    View source ↗
  2. 10mo ago

    pg_deeplake gains CHAR types, auto table reload and a split query core

    Almost all of this release is the Postgres extension, and the interesting part is structural: the query core was separated from execution and group-by execution pulled out on its own, described as preparation for later work. Automatic table reload removes a failure mode when tables are missing from storage.

    View source ↗
  3. 11mo ago

    3.x line allows numpy v2

    A single dependency constraint relaxed on the legacy 3.x line. Compatibility maintenance while the work happens in 4.x.

    View source ↗