OpenStreetMap logo OpenStreetMap

mmd's Diary

Recent diary entries

Testing planet import

Posted by mmd on 10 October 2022 in English.

For the first time in many years, I decided to repeat a series of planet imports using the Overpass API. As I didn’t want to spend too much money on hardware, I rented a commodity Intel machine for about 40€/month, with 2 Data center SSDs and 64GB ECC RAM.

Much to my surprise, the import on release took about 33.5 hours, which is at least 10 hours longer than I expected. I was able to improve runtime a bit by tweaking a few configuration settings, such as enabling lz4 compression everywhere and increasing a chunk size parameter, but still ended up with 26.5 hours.

I continued testing with my own experimental Overpass fork, that includes support for PBF, multithreading, and many other changes under the hood. Initial measurements looked quite promising with 10.5 hours total runtime. After some further analysis and improving some data structures, the import took 7 hours and 23 minutes. Peak memory consumption was still quite ok at 22G. I tried different settings to achieve lower memory consumption, at the cost of longer processing time (e.g. 8 hours and 13G peak memory).

Depending on compression settings, the final planet database was in a range of 230-265GB.

Detailed results are available on this wiki page:

That’s all for today.

Counting housenumber nodes

Posted by mmd on 14 August 2021 in English.

In this blog post we want to count addr:housenumber nodes across all EU member states, broken down by country. That’s a bit above 34 million nodes in total.

Queries are tested on different Overpass instances: two rather busy public ones, one idle private instance and a dev instance. With the exception of the latter, queries run on offical Overpass releases. Database contains a full planet (including metadata and history).


[out:csv(name, count)];
map_to_area -> .areas;
foreach.areas->.area (
  make data count = count(nodes),
            name = area.set(t["name"]);


Runtimes     gateway timeout after 1h  gateway timeout after 2h
private server:   timeout after >2h
dev branch:       30s

NB: dev branch is exploring alternative implementation options and is currently not released.

Download along my own virtual way in Overpass API

Posted by mmd on 12 August 2017 in English. Last updated on 2 May 2018.


Again a bit of prototyping today.

I always wondered what the best way to download all objects along some GPX track (more generally a list of lat/lon pairs) in Overpass API might be.

Usually, you don’t want to create some complex poly-filter for this, or even worse upload your gpx track as OSM way just for the sake of finding out what’s close by.

You might have come across the around filter, which comes in handy to find objects around a center point. I have extended this idea a bit to also handle linestrings.

Here’s a first result: a turquoise circle in the middle is our great made up GPX track, consisting of about 20 points. All the yellow stuff in the background is what Overpass API returned as highways up to 500m away from our circle. Obviously, that circle doesn’t exist in OSM, so we’re really looking at what’s close to our virtual way.



Live demo:

Road trip

Imagine you want to create a map with only minimum features (motorways, some important nodes), and some place names along your own lat/lon pairs. Here’s an example for such a trip from Frankfurt to Basel, downloaded as 10MB PBF file, processed via osm2pgsql and rendered by Kosmtik:

Image 2

Live demo:

Github issue:

[demo] Emojis in overpass turbo!! 😍😈😃

Posted by mmd on 7 January 2017 in English. Last updated on 10 January 2017.

Emojis are only a fingertip away on your iPhone… it was only a matter of time until they show up on OSM as well. The following query finds all those emojis - worldwide:

Thanks to tyr_asd for the MapCSS enhanced version below

A bit more serious: how about searching for mixed language names like name=Vinh-Loi Asien-Supermarkt 榮利市場 : check out the following query!

Do you find regular expressions with full Unicode support useful and want to see them in one of the next official Overpass API releases? It’s your turn now, vote for it on Github or leave a comment here!