OpenStreetMap

OpenStreetMap Standard Layer: Introduction

Posted by pnorman on 18 July 2021 in English (English).

This blog post is a version of my recent SOTM 2021 presentation on the OpenStreetMap Standard Layer and who’s using it.

The OpenStreetMap Standard Layer is the default layer on openstreetmap.org, using most of the front page. It’s run by the OpenStreetMap Foundation, and the Operations Working Group is responsible for the planning, organisation and budgeting of OSMF-run services like this one and servers running it. There are other map layers on the front page like Cycle Map and Transport Map, and I encourage you to try them, but they’re not hosted or planned by us.

Technology

At the high level, this is the overview of the technology the OWG is responsible for. The standard layer is divided into million of parts, each of which is called a tile, and we serve tiles.

Flowchart of rendering

OSM updates flow into a tile server, where they go into a database. When a tile is needed, a program called renderd makes and store the tile, and something called mod_tile serves it over the web. We have multiple render servers for redundancy and capacity. We’re completely responsible for these, although some of them run on donated hardware.

In front of the tile server we have a content delivery network. This is a commercial service that caches files closer to the users, serving 90% of user requests. It is much faster and closer to the users, but knows nothing about maps. We’re only responsible for the configuration.

The difference between the tile store and tile cache is how they operate, and size. The tile store is much larger and stores more tiles.

Only the cache misses from the CDN impose a load on our servers. When looking at improving performance of the standard layer, I tend to look at cache misses and how to reduce them.

Policy

The OWG has a tile usage policy that sets out what you can and cannot do with our tile layer. We are in principle happy for our map tiles to be used by external users for creative and unexpected uses, but our priority is providing a quickly updating map to improve the editing cycle. This is a big difference between the standard layer and most other commercially available map layers, which might update weekly or monthly.

We prohibit some acitivities like bulk-downloading tiles for a large area (“scraping”) because it puts an excessive load on our servers. This is because we render tiles on-demand and someone scraping all the tiles in an area is downloading tiles they will never view.

Comment from bryceco on 18 July 2021 at 15:39

Very interesting. I had to go read https://github.com/openstreetmap/mod_tile#tile-serving to get an understanding of the role of mod_tile but it’s clear now.

My question is whether Render DB uses the normal OSM database schema or does it use one specialized for rendering?

Comment from pnorman on 25 July 2021 at 05:55

OpenStreetMap Carto has its own schema which is currently fairly similar to the osm2pgsql defaults. These are completely different than the API DB schema.


Login to leave a comment