OpenStreetMap logo OpenStreetMap

Serving Vector Tiles

Posted by pnorman on 20 November 2016 in English.

This is a repost from my blog because there’s been a fair amount of interest from OSM people on what I wrote.


If you want to serve vector tiles, there are a few server options that have developed, each with different strengths and weaknesses.

node-mapnik based

Language: nodejs
Layer definitions: Mapnik layer definitions in XML, typically preprocessed from YAML
Vector tile formats: Mapbox Vector Tiles
Data source support: PostGIS

Kartotherian, tessera, and other servers based on tilelive all rely on Node bindings to Mapnik to produce vector tiles. They all work with Mapnik layer definitions. This is a reasonably well understood language and consists primarily of a SQL statement for each layer. This is reasonably flexable and it’s possible to do proper code review, git conflict resolution, and other processes you need with an open style.

Some servers can turn the Mapbox Vector Tiles into GeoJSON, but not all do. There are other minor differences, but they all have the same major advantages and disadvantages.

The biggest problem with these options is that you have to either use the exact same versions of everything as the Mapbox developers while hoping their changes work with your code, or lock down your versions to a set of known good versions and periodically update when you need new features, retesting all your code. Neither of these is practical for an open-source style which wants to involve others.

If you don’t do this, you’ll find parts of your server failing with different combinations of Mapnik and node-mapnik.

Tilezen tileserver

Language: Python
Layer definitions: SQL in jinja2 templates, YAML
Vector tile formats: Mapbox Vector Tiles, TopoJSON, and GeoJSON
Data source support: PostGIS

Tilezen tileserver was written by Mapzen to replace their TileStache-based vector tile generation. Having been written by developers who wrote previous vector tile servers, it combines ideas and functionality other options don’t have.

The datasource definitions are written in SQL + YAML, a common choice, but unlike other options, the SQL is in its own files which are preprocessed by the jinja2 templating engine. This adds some complexity, but a great deal of power. Selecting different features by zoom level normally requires repetative SQL and lengthy UNION ALL queries, but the preprocessing allows queries to be written more naturally.

Tileserver’s unique feature is the post-processing capabilities it offers. This allows vector tiles to be operated on after the database, altering geometries, changing attributes, and combining geometries. Post-processing to reduce size is a necessary feature if targeting mobile devices on slower connections. Mapbox had been working on this in the open, but now that they no longer use node-mapnik it’s not clear how they do so. MapQuest had developed Avecado to specifically target this, but it became abandoned when they stopped doing their own map serving.

You don’t need any AWS services for a basic Tilezen tileserver deployment, but there might be some dependencies in the more advanced features needed to set up a full production environment.

Tegola

Language: Go
Layer definitions: SQL in TOML
Vector tile formats: Mapbox Vector Tiles
Data source support: PostGIS

Tegola is a new server written in Go. It operates with multiple providers which supply layers to maps, allowing them to be assembled different ways. It looks like it has most of the features needed for vector tiles for a basemap, but might be missing a few needed for changing data as zoom changes.

SQL in TOML is similar to SQL in YAML for layer definitions, and like this it is reasonably flexable and makes it possible to do proper code review, git conflict resolution, and other processes you need with an open style.

I haven’t had a chance to deploy it yet, so I’m not sure what difficulties there are.

t-rex

Language: Rust
Layer definitions: SQL in TOML
Vector tile formats: Mapbox Vector Tiles
Data source support: PostGIS

t-rex is a new server written in Rust. It’s unique feature it that it can auto-configure layers from PostGIS tables. It does have all the required features for selecting appropriate data in a basemap.

It’s layer definitions are different than Tegola’s, but they are both SQL in TOML, and share the same strengths.

Like Tegola, I haven’t had a chance to deploy it.

TileStache

Language: Python
Layer definitions: SQL in JSON Vector tile formats: Mapbox Vector Tiles, TopoJSON, GeoJSON, and Arc GeoServices JSON Data source support: PostGIS

TileStache is a general-purpose tile server which Mapzen used to use a fork of to serve their Tilezen schema. They’ve switched to Tilezen tileserver, but the functionality they added has been merged back into TileStache. Unfortunately, the documentation hasn’t caught up yet, so there’s not too much information about all of its functionality.

Deploying TileStache tends to be reasonable - particularly compared to node-mapnik - but the language of SQL in JSON is one that’s a problem for open projects with multiple authors and prevents proper code review and git conflict resolution.

Tilemaker

Language: C++
Layer definitions: Lua
Vector tile formats: Mapbox Vector Tiles
Data source support: OSM PBF and shapefiles

Tilemaker is built around the idea of vector tiles without a serving stack. It does this by doing an in-memory conversion directly from OSM PBF data to pre-generated vector tiles, which can then be served using Apache, a S3 bucket, or any means of serving files from disk. This vastly simplifies deployment and reduces sources of downtime.

For serving a city or most countries this can be the ideal method, but the same strengths that make it good for this are a problem for processing the planet. It takes large amounts of RAM, can’t consume minutely changes, and has to create vector tiles for the entire PBF at once.

Tilemaker is also the only server to support directly using shapefiles for low zoom data and OSM for high zoom. Other options require loading into PostGIS and using SQL that selects the appropriate data based on zoom.

VectorTileCreator

Language: Python
Layer definitions: osmfilter options
Vector tile formats: o5m
Data source support: OSM PBF and other raw OSM data

VectorTileCreator is part of KDE Marble and takes the unique approach of creating tiles of raw OSM data. It uses osmfilter’s language for filtering OSM data, but lacks the means to use other data sources, something most maps will need. The support of o5m vector tiles is also limited. Like tilemaker it runs from the command line and produces a set of vector tiles.

Which should I use?

What you should use depends on your needs. First figure out what support you need for the full planet, updates, data sources, and output formats. If you need diff update support, then you need something that can create a single vector tile and Tilemaker won’t work. If you need TopoJSON support, node-mapnik won’t work.

For a table showing features supported, see the original blog post

Location: Uptown, Moody Park, New Westminster, Metro Vancouver Regional District, British Columbia, V3M 6T9, Canada

Discussion

Log in to leave a comment