LoginSign Up
← Back to all sparks
O

osm2pgsql

DEVOPS
Velocity0.0

Importer for OpenStreetMap data into PostgreSQL

osm2pgsql.org ↗

osm2pgsql is rebuilding tile expiry so a small edit stops invalidating a whole lake.

openstreetmappostgistile-expirylua-scriptingdata-importgeospatial
◆Current state
osm2pgsql ships two or three releases a year and the last two carry substantial new machinery. 2.3.0 reworked tile expiry: polygons are now expired by shape rather than bounding box, and an opt-in diff expire mode computes the symmetric difference between an object's old and new geometry so a small edit to a large feature only invalidates the tiles it actually touched. It also added configurable limits on how many tiles one geometry or one run can expire. Before that, 2.2.0 introduced Locators for fast region lookup during import and process_deleted_* callbacks for handling removed objects in Lua.
◆Where it's heading
The project is adding primitives rather than features, and saying so explicitly — 2.2.0 describes Locators and deleted callbacks as building blocks for things users have requested for years. The pattern is to move work that previously happened in the database after import into the import pipeline itself: region tagging via Locators instead of a post-import spatial join, deletion handling via callbacks instead of database triggers. The expiry work follows the same logic, pushing precision upstream so downstream re-rendering does less. The new expiry limits are a defensive addition, added specifically because vandalism or misconfiguration can otherwise generate billions of tiles and exhaust memory.
◆Prediction
Expect diff expire to move from opt-in toward default once it has been exercised, following the pattern of the middle table format that became default in 1.11.0. The experimental osm2pgsql-expire command introduced in 2.2.0 is the other loose thread — it is explicitly marked as subject to change and has not stabilised.

◆Recent moves

  1. 1mo ago

    2.3.1 fixes a diff-expire segfault with multiple expire outputs

    A bugfix release for a segfault in the new diff expire path when a table has multiple expire outputs, plus a man page installation fix. The first fallout from 2.3.0's expiry rework, arriving less than a month later.

    View source ↗
  2. 1mo ago

    Diff expire invalidates only the tiles a change actually touched

    ⚡ SPARK

    The expiry rework this release line was building toward: shape-aware polygon expiry, opt-in diff expire based on the symmetric difference between old and new geometry, and hard limits on how many tiles a single run can invalidate. Together with the standalone style tester, it is the most consequential release in this window.

    View source ↗
  3. 10mo ago

    Locators and deleted-object callbacks land as new primitives

    ⚡ SPARK

    Introduces two building blocks the project expects to unlock long-requested features: Locators for testing geometries against named regions during import, and process_deleted_* Lua callbacks for handling removed objects. Sets up the pattern the 2.3.0 expiry work follows, moving work out of the database and into the import pipeline.

    View source ↗
  4. 1y ago

    2.1.1 refuses to update when the flatnode file is missing

    Two fixes: refusing to update a database when the flatnode file is absent, which prevents silent corruption, and a regression writing to tables without a managed id column. Small, but the flatnode guard is the kind of check that saves an import.

    View source ↗
  5. 1y ago

    2.1.0 makes prepared statements work with connection poolers

    Switches to protocol-level prepared statements instead of PREPARE SQL commands so osm2pgsql works behind connection poolers and database middleware, and adds an option to create a primary key on flex tables. A deployment-compatibility release rather than a feature one.

    View source ↗
  6. 1y ago

    2.0.1 fixes schema forwarding and missing binary installation

    Three small packaging and installation fixes following 2.0.0: the --schema parameter not being forwarded from osm2pgsql-replication, a function installed into the wrong schema, and the osm2pgsql-gen binary not being installed by default.

    View source ↗