Following various discussions and requests over the previous years, I’ve adapted parsing of the machine-readable osmc:symbol tag in waymarkedtrails for route symbols to cover more use cases. The new implementation comes a bit closer to what is specified on the wiki website, most notably now supporting a second foreground, but also extending the list of supported backgrounds. This posts gives a bit of background on the changes.

Supported syntax

The official specification of the osmc:symbol tag always had the issue that it largely relies on guessing the function of its parts from their content. A part called red_bar? Must be a foreground description because there is no “bar” background and the content looks to be too long to be a text description, too. This is a viable way of describing something but not very robust against typos and it makes it very hard to extend the value set. waymarkedtrails has therefore implemented a subset of the specification that could be parsed in a way that the function is determined from the position of the parts. This subset was essentially: waycolor[:background[:foreground[:text[:textcolor]]]] Unfortunately this approach made it impossible to support the second foreground when it popped up on the wiki page and that has left users confused ever since.

User mueschel has recently re-analised the format and came to the conclusion that we can still keep the positional approach when sticking to the following combinations of part descriptions:

waycolor:background waycolor:background:foreground waycolor:background:foreground:foreground2 waycolor:background:foreground:text:textcolor waycolor:background:foreground:foreground2:text:textcolor

The one officially allowed variant that isn’t covered is waycolor:background:text:textcolor. Luckily this variant isn’t really used in practice with very minor exceptions. And it can easily be made compliant by adding an empty foreground: waycolor:background::text:textcolor.

… See full entry

I’ve just rolled out some updates for waymarkedtrails, the website for browsing route relations. The most notable improvement is that waymarkedtrails is finally able to understand member roles in route relations. This post gives a quick overview of what works and what doesn’t and what has changed with in the route views.

Forward/backward roles

The first group of roles handled specially are forward and backward roles which indicate that the route follows the given way only in the direction of the way or in counterflow. These oneway sections are now shown when details of a route are displayed:

Waymarkedtrails will internally merge forward and backward sections into split sections and count them only once when computing the length of the route. However, this only works, when all ways of such a spilt section appear together in the relation and follow the order of the route. (See also the notes on ordering below.)

Roundabouts

… See full entry

I’d like to announce the release of the new major version 4 of pyosmium.

pyosmium was originally created as a thin Python wrapper around the osmium library, a fast and flexible C++ library for reading, writing and processing OSM data. With the new version 4, pyosmium adds a convenience layer on top which gives the library a more pythonic feel and speeds up processing considerably.

The most important new features are:

• Iterative processing. OSM files can now be iterated through with an simple for loop. (Processing a file using handler functions is still possible.)
• Filter functions allow to quickly skip over uninteresting parts of the file, making it now possible to use pyosmium scripts with large OSM files without pre-filtering.
• Writers with automatic reference completion. One of the challenges of writing OSM files is that for every way and relation written, you usually also need to write the nodes and members. pyosmium now implements writers which help with that.

Here is an example how to quickly find the most frequent tags used together with amenity=school using the new iterative syntax:

import osmium
from collections import Counter

tag_counter = Counter()
total = 0

for o in osmium.FileProcessor('planet.osm.pbf')\
               .with_filter(osmium.filter.TagFilter(('amenity', 'school'))):
    tag_counter.update([tag.k for tag in o.tags if tag.k != 'amenity'])
    total += 1

for tag, cnt in tag_counter.most_common(10):
    print(f"{cnt:6d} ({cnt*100/total:5.2f}%) {tag}")

Running this on a full OSM planet file takes less than 5 minutes on a 12-core machine with 128GB RAM.

Or do you want to create an thematic extract of schools:

import osmium

with osmium.BackReferenceWriter('schools.pbf', 'planet.osm.pbf',
                                overwrite=True) as writer:
    for o in osmium.FileProcessor('planet.osm.pbf')\
                   .with_filter(osmium.filter.TagFilter(('amenity', 'school'))):
        writer.add(o)

… See full entry

I’ve given a talk about the state of mapping of hiking routes in Europe at this month’s SotM-EU. A few people have asked for the slides, so here they are:

Hiking Routes - Past, Present and Future - SotM-EU 2023

I’m sorry if the slides lack some context for those who have not seen the talk. I hope there will be video recordings of the talk at some point.

Nominatim (the search engine that powers the search box on the OpenStreetMap website) has recently changed significantly its way how postcodes are handled. This post tries to give a bit of background on what has changed and why.

When you search for a place on osm.org, Nominatim not only presents the name of the place in the result but a complete address. This address not only helps distinguish the different places but is also used to narrow down your search. This address is not a postal address as you would put on a postcard. It is more a textual description where the place is located, in which suburb, city, state, country etc. This information is fairly easy to compute from OSM data. There are areas for all these administrative areas. So Nominatim just needs to check in which areas a place is inside, order all appropriately and there is the address.

Postcodes, however, are different. In most countries there is no such thing as postcode areas. Postcodes are simply assigned to a some place (a house or POI) in a fashion that is deemed most practical for the local postal service. Often the post codes follow delivery routes. It might be possible to draw an area around houses with the same postcode but this would be an artificial distinction and there is no guarantee that the resulting areas don’t overlap.

For that reason, there are very few boundaries in OSM that describe postcode areas. Mostly postcodes can be found on house numbers and POIs in the addr:postcode tag. But even here coverage is rather sparse. So when computing the address of a place, Nominatim has to go a different way to determine the most likely postcode for a place where no addr:postcode tag exists.

With the new version, Nominatim tries two different methods to infer the postcode of the place: an address lookup and an area-based lookup.

… See full entry

waymarkedtrails.org, the site to show all things route related, has always taken care to try to put route relation members in a sensible order before displaying them. A couple of weeks ago this has changed. The site now assumes that the members of each relation are already in the correct order. The reason for that is simple: sorting route relations is hard.

Before explaining why sorting is hard, let me explain why order even matters. As long as you simply want to display the route on a map, the order is not important. Simply color each way in the relation to your liking and the route is nicely visible for a human reader. However, waymarkedtrails.org does a bit more than that. It displays an elevation profile and allows to download a GPX of the route, so you can put it onto your favourite mobile device. When the route is not sorted in the expected order, then the GPX is unusable for many applications, for example in Garmin Basecamp.

So why not sort the route automatically? On first sight, this seems to be a simple task. After all, a route should be a simple linear connection from A to B. Unfortunately, the real world is much more messy.

Even the most simple case of a linear route from A to B already has two solutions to the problem. The route may go from A to B or from B to A. In most cases the direction doesn’t really matter but there are exceptions. Take a downhill mountain bike route, for example, or a nature trail with information panels that should be visited in the correct order.

Then there are loop routes which end at the point where they started. When sorting automatically there is no way to determine the starting point of the route. In both cases, we could add extra tagging to mark the start and end points. But why add the extra work when sorted route give the start and endpoints clearly at the beginning and end of the list?

… See full entry

Waymarked Trails is the map for all things route - hiking, cycling, skating and horse riding. It has been a long standing wish of many users to have hillshading and contours on the map. So last week the site has received a new feature that allows you to choose a different base map. The first addition to the available base layers is OpenTopoMap, a beautiful map crafted in spirit of German topological maps. To try it out, go to the settings menu (the little cockwheel on the bottom of the page) and choose the new base map from the drop down menu.

Many thanks to the friendly folks from OpenTopoMap for all their work creating the map and for allowing Waymarked Trails to offer it as a base layer.