mmd's Diary

Recent diary entries

Counting housenumber nodes

Posted by mmd on 14 August 2021 in English (English).

In this blog post we want to count addr:housenumber nodes across all EU member states, broken down by country. That’s a bit above 34 million nodes in total.

Queries are tested on different Overpass instances: two rather busy public ones, one idle private instance and a dev instance. With the exception of the latter, queries run on offical Overpass releases. Database contains a full planet (including metadata and history).


[out:csv(name, count)];
map_to_area -> .areas;
foreach.areas->.area (
  make data count = count(nodes),
            name = area.set(t["name"]);


Runtimes     gateway timeout after 1h  gateway timeout after 2h
private server:   timeout after >2h
dev branch:       30s

NB: dev branch is exploring alternative implementation options and is currently not released.

CGImap 0.8.0 released

Posted by mmd on 16 February 2020 in English (English).

A new version of CGImap, the rails port acceleration layer written in C++, has been released!

With the addition of endpoints for changeset create/update/close, it is now possible to run (most part of) an OSM editing session on CGImap only. We will probably not use this feature in production, still this may be interesting if you’re running your own local OSM instance.

In previous versions, CGImap supported four different backends (Read only, Write, Pgsnapshot and StaticXML). For end users, this had very little significance. However, it contributed quite a bit to the overall complexity and technical debt of the solution. For this reason, two out of those four backends have been removed in this version:

  • Write backend: replaced by Read only backend. Use of temporary tables to store intermediate results offered no real performance benefits. This backend wasn’t in productive use for a few years anymore.
  • pgsnapshot: removed due to lack of demand, also not used in production.

Besides, a few remaining OSM API endpoints for element calls have been added:

  • node/nnn/relations
  • way/nnn/relations
  • relation/nnn/relations
  • node/nnn/ways

Deployment to production is still TBD.

Full release notes:

A little experiment... 🧙✨

Posted by mmd on 15 December 2019 in English (English).

The year is almost over, and I just noticed that I didn’t write a single blog post this year. So here’s a quick one.

What if we take the top 8 “most frequently used keys on ways” according to taginfo, and squeeze them all in one global Overpass query. Will this work without getting some out of memory error?

way[building]                    // 366'209'402 objects
   [source]                      // 153'587'268 objects
   [highway]                     // 146'009'103 objects
   [name]                        //  51'050'056 objects
   ["addr:housenumber"]          //  46'319'742 objects
   ["addr:street"]               //  44'955'803 objects
   ["addr:city"]                 //  35'031'726 objects
   ["addr:postcode"];            //  30'249'900 objects
out center meta;

Well, surprisingly, it does work, assuming you bump up the maxsize parameter to 4GB, and wait a few minutes. However, that’s not ideal from a resource consumption point of view, as the query uses up quite a bit of the available capacity on the public instances. If you’re planning to run this sort of query on a regular basis, you should definitely consider setting up your own Overpass instance.

Some ideas to reduce both runtime and memory consumption are currently in discussion / prototyping stage. Results look pretty good so far (less than 20s and 0.5GB for the example above).

That’s all for now. 😉

New demo for changeset upload available

Posted by mmd on 4 July 2018 in English (English). Last updated on 20 June 2019.

Changeset upload is in production since May 2019, as part of CGImap 0.7.5

the changeset upload is known to take quite some time for larger changesets, sometimes resulting in timeouts and unclear overall status. Luckily, JOSM already covers some of those issues via its chunked upload, still there’s always room for improvement.

Test period ended

Thank you for your interest. The test period on a dedicated server has come to an end. There will be a separate announcement once this feature is available again for testing. Thanks everyone for participating in the tests!


(Image by zerebubuth, original source(defunct), mirror)

Download along my own virtual way in Overpass API

Posted by mmd on 12 August 2017 in English (English). Last updated on 2 May 2018.


Again a bit of prototyping today.

I always wondered what the best way to download all objects along some GPX track (more generally a list of lat/lon pairs) in Overpass API might be.

Usually, you don’t want to create some complex poly-filter for this, or even worse upload your gpx track as OSM way just for the sake of finding out what’s close by.

You might have come across the around filter, which comes in handy to find objects around a center point. I have extended this idea a bit to also handle linestrings.

Here’s a first result: a turquoise circle in the middle is our great made up GPX track, consisting of about 20 points. All the yellow stuff in the background is what Overpass API returned as highways up to 500m away from our circle. Obviously, that circle doesn’t exist in OSM, so we’re really looking at what’s close to our virtual way.



Live demo:

Road trip

Imagine you want to create a map with only minimum features (motorways, some important nodes), and some place names along your own lat/lon pairs. Here’s an example for such a trip from Frankfurt to Basel, downloaded as 10MB PBF file, processed via osm2pgsql and rendered by Kosmtik:

Image 2

Live demo:

Github issue:

Clickable roads demo

Posted by mmd on 7 April 2017 in English (English). Last updated on 2 May 2018.


I put up a small proof of concept to demonstrate @jotpe’s proposal on Github:

Link to Clickable Roads Demo

Just click on some street with a name and see all parts of the street highlighted, regardless of how fragmented the underlying OSM data is. Small gaps are also ok.


[demo] Emojis in overpass turbo!! 😍😈😃

Posted by mmd on 7 January 2017 in English (English). Last updated on 10 January 2017.

Emojis are only a fingertip away on your iPhone… it was only a matter of time until they show up on OSM as well. The following query finds all those emojis - worldwide:

Thanks to tyr_asd for the MapCSS enhanced version below

A bit more serious: how about searching for mixed language names like name=Vinh-Loi Asien-Supermarkt 榮利市場 : check out the following query!

Do you find regular expressions with full Unicode support useful and want to see them in one of the next official Overpass API releases? It’s your turn now, vote for it on Github or leave a comment here!

Overpass API - counting addr:housenumber nodes in EU member states

Posted by mmd on 30 September 2016 in English (English). Last updated on 7 August 2021.


[out:csv(name, count)];


map_to_area -> .areas;

foreach.areas->.area (
  make data count = count(nodes),
            name = area.set(t["name"]);


name	count
Nederland	9307881
France	5746147
Portugal	125216
España	818926
Éire / Ireland	15707
Hrvatska	87719
Italia	3795595
Бългaрия	13497
Ελλάς	58460
Malta	762
Κύπρος - Kıbrıs	1918
Österreich	413787
Danmark	2543375
Deutschland	3577852
Česko	2856166
Sverige	226873
België / Belgique / Belgien	136592
Lëtzebuerg	12261
Slovenija	54326
Magyarország	143186
Polska	3569364
Latvija	12956
România	221791
Slovensko	212013
Lietuva	462917
Suomi / Finland	108568
Eesti	6968

Query runtimes

Instance runtime Gateway timeout after 1h Gateway timeout after 2h
0.7.59_mmd 30s