My Talk at State of the Map 2024

(How the iD Editor still can to do better in the long term)

This September, I presented some ideas for the future of OSM’s iD editor at the State of the Map conference in Nairobi. Now, the video recording of the talk is online. To summarize the talk’s content: I identified two core challenges faced by the iD editor which need to be addressed in the upcoming years:

1. Data Complexity

Naturally, OSM’s data got more complex over time – as an example, a simple school might have typically been mapped as a single POI node in OSM a couple of years ago, while nowadays the state of the art is to model it using multiple map features that are nested within each other. While this allows to represent the reality in a high level of detail, the added complexity needs to be accommodated for by the tooling, especially in an editor tailored for beginner mappers like iD. For a deep dive into this topic, see also Sarah Hoffmann’s talk at the same conference.

… See full entry

Attending the recent hack weekend in Berlin, I got inspired to work on a neat feature in overpass turbo. Honestly speaking, in the past few years, the tool did not the amount of attention from the developer’s (my) side that it deserves. What came clear to me at the hacking event is that a lot of people still use it on a very regular basis and that it is an invaluable part of the OSM bag of tools, which was really nice to hear.

Auto Styler (beta)

So, what someone proposed was to extend the current query wizard feature to also support the styling of the results using mapcss. What I took from that suggestion was that it should not be hard to implement a dedicated helper dialog that auto-generates MapCSS to style the data according to the values of a selectable tag and color palette. And I’m glad to say that after a quite productive two days, there’s now already a beta version already available to test out on overpass-turbo.eu.

… See full entry

It’s been a while since I last wrote about iD’s development and thought it would be a nice idea to make this a more regular thing in addition to the regular online community chats.

Recently, there have been some new features implemented that are worth showcasing, including several great ways that community members are stepping up to contribute to iD.

iD v2.30 release

In his Google Summer of Code project, Mattia Pezzotti has implemented support for street level imagery from Panoramax. Panoramax is a relatively new completely free and open-source alternative to other street level imagery providers such as Mapillary, KartaView or Bing Streetside. It even includes features like a federated system architecture and the possibility to self-host the server software if you need to. Check the project out on panoramax.fr to read more and how to contribute your own pictures.

… See full entry

About a year ago, I started working as the maintainer of the iD editor for the OSMF. Here’s a short recap of the last year or so and a bit of outlook of what’s lying ahead according to me.

A look back

The year was characterized by the reactivation of processes which had been dormant for a little while before I started:

The tagging schema repository has received regular releases (about one major update every second month on average) after been mostly dormant in the year before. Many of the improvements to the tagging schema came in by various contributors from the community: Thanks to everyone who took part in this, including everyone actively translating on Transifex!

For iD itself, I soon learned that it is better to perform small iterative improvements, rather than tackling big reworks all at once. Therefore, my focus in the last year was primarily on stability, bug fixes and incremental improvements. There were still a few improvements to point out from last year:

• version 2.21 switches the development build system to a more modern system based on esbuild and droped support for Internet Explorer 11
• version 2.22 contains a first step towards making iD understand lifecycle prefixes of tags
• version 2.23 significantly improves startup times, improved input fields in various ways and introduced a few improvements under the hood to manage presets
• version 2.24 adds a remaining input length indicator for fields which are constrained to OSM tags’ maximum length of 255 characters and introduces a new field type for directional tags

A glimpse ahead

Another highlights of the year 2022 was the State of the Map conference in Firenze, where I spoke about the history of the iD editor (in summer 2023, it will celebrate 10 years of being the default editor on osm.org!), and tried to explain what the mid to long term high level goals are which I would like to tackle for iD. I identified five big topics to work on:

… See full entry

Normally, I blog here more about technical topics, but today I would like to take the opportunity to honor three small parts of the OSM community, who I happened to have the chance to get to know over the years that I’ve been active in mapping. All of these groups deserve a shout-out for their great work they do on OpenStreetMap month after month.

• The OSM community in Graz (Austria) managed to map their city in great detail and keeps it complete and up to date. It was a pleasure to be a part of the monthly meetups back when I studied there. I remember many fruitful discussions over good food an a nice beer or two. Keep up the good work, guys!
• Geofabrik is a company from Karlsruhe (Germany) which provides OSM related services since many years, some of which are available for free to the OSM community. Thank you for maintaining your OSM data extract download service, and for hosting your iconic hack weekends. You’re awesome!
• The disastermappers heidelberg are a group of students of Heidelberg University who regularly organize mapathons and workshops, and try to raise awareness of the benefits OpenStreetMap data towards a wide audience. Don’t stop educating the mappers of tomorrow!

Do you also know groups or individuals that do great work in OSM? Let them know! :)

Want to get an idea about what contributions are happening in OSM in your region? Maybe you’re even looking for a way to better review map changes (hint, hint)? A good starting point is probably my latest-edits page tool. It displays all objects that have been modified during the last week alongside with their respective changesets.

One major drawbacks of the tool was that until recently, deleted map objects (and changesets consisting solely of deletions) were not displayed on the page. Now, deleted objects are displayed as faint “ghostly” outlines on the map. In the same way, you can now also see how modified objects looked like before the respective contributions happened. This can result in a nice looking “shadow” effect when for example buildings have been realigned to better aerial imagery.

Also, you can now select between the latest changes from the last day, week, or month, and directly zoom to the location of a particular changeset:

… See full entry

Tomorrow is the 10 year anniversary of OSM’s API version 0.5. This is the version of the OSM-API that first exposed (among other things) the version number on all OSM objects, making it possible to access the full history of every object modification from this point onward.

This means that very soon, the full history planet file published on planet.osm.org will contain more than 10 years of editing history which can be investigated, evaluated and analyzed (using tools like the OSM history database oshdb that’s currently under development at HeiGIT on the University of Heidelberg, which I presented earlier this year at the State of the Map).

Of course, OpenStreetMap as a project exists for a bit longer than that (about 13 years now) and there was already quite some data mapped before the OSM API 0.5 was introduced 10 years ago.

… See full entry

The latest installation of my yearly osm node density visualization is now online: https://tyrasd.github.io/osm-node-density shows the freshest data from mid 2017 (while the results from previous years starting with 2014 are also available on the site’s layer selection menu).

Can you guess where the following example screenshot was taken?

Click here to find out.

Brand new on overpass-turbo.eu: In addition to storing queries locally on your browser, you can now also save your important queries on your OSM account and access them from everywhere. Here’s the new option visible on the save dialog:

OpenStreetMap’s standard map layer is used by many people each day. OSM even provides a dump to the access logs from which one can see which parts of the world is viewed how many times for each day. Inspired by Lukas Martinelli’s work on Parsing and Visualizing OSM Access Logs, I’ve worked on a web app to visualize OSM’s tile access logs. Contrary on Lukas’ approach, I wanted to focus on an accurate representation of the data and wanted to make something works for the whole (Mercator-projected) world.

I’ve ended up with a viewer that takes an (uncompressed) log file from planet.osm.org and produces a two-dimensional histogram for each zoom level: For example, at zoom level 6 in the viewer each pixel on the viewer represents the number of accesses of the corresponding osm.org-tile at zoom level 14. That’s 8 zoom levels further in – or, put another way, each 256x256px² osm.org-tile is represented by a single pixel in the visualization.

The number of accesses of each tile is represented by a logarithmic color palette:

… See full entry

TL;DR: head over to http://taghistory.raifer.tech/ for usage graphs of arbitrary OSM tags over time (by number of OSM objects).

In OpenStreetMap, tags define what an object is. Whether it is a mountain, a river, a house, or a postbox: Every map feature has it’s own tag (or set of tags).

OSM doesn’t have a fixed set of object categories. Over time, a more and more faceted and diverse set of features got mapped in OSM, thus the amount of different tags grew. At the same time, sometimes, tagging of a specific thing changes: Features that used to be mapped with one tag, get newer, better and more refined tags. That’s OpenStreetMap evolving.

Of course, OpenStreetMap is also still growing, but not all the tags are getting more widely used at the same pace: For example, while it’s quite possible that most of the world’s railway stations are already mapped in OSM, there are still many juicy pastures left to be mapped out there.

… See full entry

My OSM node density visualization has just been updated to show the freshest OSM data!

A significant difference from the 2014 edition is that the density is now calculated as OSM-nodes per m² on the ground (as opposed to nodes per projected pixel in the previous version). If you want to learn more about why this change makes a difference, I’d strongly recommend the following article by Christoph Hormann (imagico).

The updated slippy map lets you chose between the layers of different years (always created around end of June of each year) as well as a display of what changed during each 12 month interval. For example you can see that in North America, there were some imports as well as some major import-cleanups going on during the last year:

… See full entry

overpass turbo has been around for a little over two years now. In this time, it arguably changed how developers and mappers interact with OSM data. Let us take this opportunity to look back and take a glimpse at some statistics:

users

The user-base has more than quintupled from the initial group of early adopters as can be seen in the following Piwik graph:

Note that the actual absolute number of visitors is likely significantly higher than what is reported here, because surely many of you have the do-not-track flag activated or are using tracker blocking software in your browsers. Speaking of it – as of today you can opt-out from any tracking on overpass-turbo.eu also by simply switching it off in the settings dialog under the privacy tab.

shared queries

Shortly after its release, overpass turbo got the ability to share queries in the form of short URLs (e.g. http://overpass-turbo.eu/s/4). Here is some insight into what queries people have been sharing since then:

… See full entry

This summer brought us some nice new features to the Overpass API universe, that are now finally available in overpass-turbo (and osmtogeojson). Here are some of the highlights:

geometry options

The Overpass API now has three new geometry types, which give you even more flexibility in your queries:

• out center; – this additionally prints the center coordinate for every OSM object
• out bb; – this additionally prints the bounding box coordinates for every OSM object
• out geom; – this additionally prints the full coordinates of every OSM object

The first two options are particularly usefull if one is only interested in the approximate location of some features, rather than their exact outline. For example, one finds that POIs are often mapped on building outlines. By requesting only the center coordinates (out center;), one saves transfer bandwidth and gets an overall quicker query.

Here is an example how this looks like (note that out of the 7 displayed POIs, 6 are mapped on ways in OSM): (try it on overpass turbo)

… See full entry

Exactly one year ago I published my first visualization of the global OpenStreetMap data density. This is the updated 2014 edition.

(click image for slippy map or here for high-res images)

Each pixel shows the number of nodes in its corresponding area¹. But this year every point that has data in it is shown (i.e. there is at least one node at that location - last year only locations with more than 1000 nodes were included). Also, the slippy map has two more zoom levels which reveal even more impressive details like on this crop of the central Alps:

… See full entry

In the last weeks a few new features got into overpass turbo. Here is a quick heads-up:

Text Markers in MapCSS

You can now finally use MapCSS to display text markers on the map. This is very useful to show the names of displayed features. The following MapCSS statement adds text markers to each node, way or relation on the map and displays whatever is in the tag name:

{{style:
  node, way relation {
    text: name;
  }
}}

Or take a look at the following example:

Export to Level0

Overpass turbo can now export data into the Level0 editor. Level0 is a very lightweight low-level textual online OSM editor. It can be very handy for example when inspecting tags of a collection of similar objects (such as when checking the names of all restaurants in town).

<todo: gif?>

Additional Stats

… See full entry

Imports sind für OpenStreetMap ein sehr zweischneidiges Schwert: Natürlich führen sie kurzfristig zu besseren Daten, aber sie führen zu keiner nachhaltigen Stärkung der Community. Vor allem von Außen kommende Imports sind für OSM nicht nachhaltig, weil sie keine neuen Mapper anlocken und nicht einmal bestehende Mapper einbindet. Bei OSM kommt es nicht so sehr auf die Quantität der Daten an. Viel wichtiger sind Qualitätsmerkmale wie Aktualität, Ausgewogenheit, Nützlichkeit, relative Genauigkeit, usw.

Was ich meine ist: Wenn eine Gegend/Dorf/… in OSM noch schlecht erfasst ist, ist auch wenig geholfen, wenn auf einmal alle Gebäude da sind, wenn die eigentlich wichtigeren Dinge wie Straßen, Straßennamen, POIs, usw. fehlen. Vielmehr würde das eher eine falsche Vollständigkeit vortäuschen („Da sind ja sogar schon alle Gebäude gezeichnet, da muss ich nichts mehr mappen“). Wenn aber eine Gegend in OSM schon recht gut erfasst ist, werden die lokalen Mapper von alleine irgendwann entscheiden, dass es an der Zeit ist, Gebäude nach-und-nach einzutragen. Dabei werden sie sich dann automatisch der jeweils besten verfügbaren Datenquellen bedienen: Entweder vom Luftbild abzeichnen (zur Zeit die einzige Option), oder ein Import von etwaigen offenen Datensätzen. Wie man sieht, ein von Außen “aufgezwungener” Import ist so oder so nicht notwendig.

… See full entry

overpass turbo’s query wizard just got a little bit more useful! It understands not only OSM tags, but even interprets object classes: Just use simple terms such as Restaurant, Museum, Hotel, Hospital or any other thing, the iD editor already has a preset for.

This even works in your own language, as you can see in the following screenshot (Trinkbrunnen is German for “drinking fountain”):

Btw: if you can read German, you can find some more information about this in my recent guest blog article on blog.openstreetmap.de.

overpass turbo is a helpful tool for hundreds of mappers every week. But it could be an even better tool for even more people if only there was a way to assist with the creation of actual overpass queries. You know that it’s quite tedious to type all those queries that are mostly the same all the time. Not to mention that one must remember all those overpass statements and their parameters all the time.

But here comes a way that makes overpass turbo both easier to use for beginners as well as quicker to use for experienced data-miners:

The Query Wizard

Designed to transform simple, human-readable search terms into functional Overpass queries.

Here is an example: To get restaurants, now all you have to do is fire up the Wizard from the tool-bar (or using the keyboard-shortcut Ctrl+I) and type in the appropriate tag amenity=restaurant:

… See full entry

A while ago, the SASA (a local public bus service operator in South Tyrol) gave us the permission to import their bus stops from opensasa.info into OSM.

Here is the result:

Btw: In the meantime SASA is working on getting their real time bus information out to the public. Take a look at their public beta! And of course they are using a beautiful OpenStreetMap background map. :)