OpenStreetMap

mmd's diary

Recent diary entries

CGImap 0.8.0 released

Posted by mmd on 16 February 2020 in English (English).

A new version of CGImap, the rails port acceleration layer written in C++, has been released!

With the addition of endpoints for changeset create/update/close, it is now possible to run (most part of) an OSM editing session on CGImap only. We will probably not use this feature in production, still this may be interesting if you’re running your own local OSM instance.

In previous versions, CGImap supported four different backends (Read only, Write, Pgsnapshot and StaticXML). For end users, this had very little significance. However, it contributed quite a bit to the overall complexity and technical debt of the solution. For this reason, two out of those four backends have been removed in this version:

  • Write backend: replaced by Read only backend. Use of temporary tables to store intermediate results offered no real performance benefits. This backend wasn’t in productive use for a few years anymore.
  • pgsnapshot: removed due to lack of demand, also not used in production.

Besides, a few remaining OSM API endpoints for element calls have been added:

  • node/nnn/relations
  • way/nnn/relations
  • relation/nnn/relations
  • node/nnn/ways

Deployment to production is still TBD.

Full release notes: https://github.com/zerebubuth/openstreetmap-cgimap/releases/tag/v0.8.0

A little experiment... 🧙✨

Posted by mmd on 15 December 2019 in English (English).

The year is almost over, and I just noticed that I didn’t write a single blog post this year. So here’s a quick one.

What if we take the top 8 “most frequently used keys on ways” according to taginfo, and squeeze them all in one global Overpass query. Will this work without getting some out of memory error?

way[building]                    // 366'209'402 objects
   [source]                      // 153'587'268 objects
   [highway]                     // 146'009'103 objects
   [name]                        //  51'050'056 objects
   ["addr:housenumber"]          //  46'319'742 objects
   ["addr:street"]               //  44'955'803 objects
   ["addr:city"]                 //  35'031'726 objects
   ["addr:postcode"];            //  30'249'900 objects
out center meta;

Well, surprisingly, it does work, assuming you bump up the maxsize parameter to 4GB, and wait a few minutes. However, that’s not ideal from a resource consumption point of view, as the query uses up quite a bit of the available capacity on the public instances. If you’re planning to run this sort of query on a regular basis, you should definitely consider setting up your own Overpass instance.

Some ideas to reduce both runtime and memory consumption are currently in discussion / prototyping stage. Results look pretty good so far (less than 20s and 0.5GB for the example above).

That’s all for now. 😉

New demo for changeset upload available

Posted by mmd on 4 July 2018 in English (English). Last updated on 20 June 2019.

Changeset upload is in production since May 2019, as part of CGImap 0.7.5


the changeset upload is known to take quite some time for larger changesets, sometimes resulting in timeouts and unclear overall status. Luckily, JOSM already covers some of those issues via its chunked upload, still there’s always room for improvement.

Test period ended

Thank you for your interest. The test period on a dedicated server has come to an end. There will be a separate announcement once this feature is available again for testing. Thanks everyone for participating in the tests!

Upload

(Image by zerebubuth, original source(defunct), mirror)

Download along my own virtual way in Overpass API

Posted by mmd on 12 August 2017 in English (English). Last updated on 2 May 2018.

THIS FEATURE IS NOW OFFICIALLY AVAILABLE IN OVERPASS API 0.7.55

Again a bit of prototyping today.

I always wondered what the best way to download all objects along some GPX track (more generally a list of lat/lon pairs) in Overpass API might be.

Usually, you don’t want to create some complex poly-filter for this, or even worse upload your gpx track as OSM way just for the sake of finding out what’s close by.

You might have come across the around filter, which comes in handy to find objects around a center point. I have extended this idea a bit to also handle linestrings.

Here’s a first result: a turquoise circle in the middle is our great made up GPX track, consisting of about 20 points. All the yellow stuff in the background is what Overpass API returned as highways up to 500m away from our circle. Obviously, that circle doesn’t exist in OSM, so we’re really looking at what’s close to our virtual way.

Circle

Image

Live demo: http://overpass-turbo.eu/s/yt2

Road trip

Imagine you want to create a map with only minimum features (motorways, some important nodes), and some place names along your own lat/lon pairs. Here’s an example for such a trip from Frankfurt to Basel, downloaded as 10MB PBF file, processed via osm2pgsql and rendered by Kosmtik:

Image 2

Live demo: http://overpass-turbo.eu/s/qXI

Github issue: https://github.com/drolbr/Overpass-API/issues/418

Clickable roads demo

Posted by mmd on 7 April 2017 in English (English). Last updated on 2 May 2018.

THIS FEATURE IS NOW OFFICIALLY AVAILABLE IN OVERPASS API 0.7.55

I put up a small proof of concept to demonstrate @jotpe’s proposal on Github:

Link to Clickable Roads Demo

Just click on some street with a name and see all parts of the street highlighted, regardless of how fragmented the underlying OSM data is. Small gaps are also ok.

image

[demo] Emojis in overpass turbo!! 😍😈😃

Posted by mmd on 7 January 2017 in English (English). Last updated on 10 January 2017.

Emojis are only a fingertip away on your iPhone… it was only a matter of time until they show up on OSM as well. The following query finds all those emojis - worldwide: http://overpass-turbo.eu/s/l6r

Thanks to tyr_asd for the MapCSS enhanced version below

A bit more serious: how about searching for mixed language names like name=Vinh-Loi Asien-Supermarkt 榮利市場 : check out the following query! http://overpass-turbo.eu/s/l6s

Do you find regular expressions with full Unicode support useful and want to see them in one of the next official Overpass API releases? It’s your turn now, vote for it on Github or leave a comment here!

Overpass API - pre-release test drive (no longer available)

Posted by mmd on 30 September 2016 in English (English). Last updated on 15 February 2021.

Thank you for your interest. Unfortunately, the test drive period has come to an end.