OpenStreetMap

Polyglot's diary

Recent diary entries

JOSM's PT_Assistant learned a few new tricks

Posted by Polyglot on 13 August 2018 in English (English)

This summer Biswesh Mohapatra continued the work started by Darya Golovko and continued by Giacomo Servadei. The first year the main focus was on creating validator rules, that needed more advanced processing than what was possible using MapCSS. A layer for visualising the stops and itineraries of the currently selected route relation, was also a big step forward to get visual feedback about what happened when editing itineraries.

The second year Giacomo added the roundabout splitter, which takes care of all the route relations crossing it, so only those ways that are traversed remain. He also added a map mode that adds a node to terminate the itineraries, splits the ways automatically and only keeps the relevant part in all the route relations affected. Depending on the settings, this node also gets the tag public_transport=stop_position and tags for the mode of transport. Giacomo also added the ability to sort the stops in the route relations based on the order of the ways. He also added functionality to visualise bicycle and foot route relations. Especially for bicycle route relations, this helps a lot to get the ways in the right order and with the correct roles, where the path forks.

PT_Assistant visualisation layer

Biswesh started this summer by adding a map mode for ‘double splitting’ ways, which comes in handy, when you want to add bridges, tunnels, bus bays or traffic calming tables. This even works on 2 ways sharing a common node. You should try it! That functionality alone is worth installing the PT_Assistant plugin.

Then he added a wizard to help you set up the JOSM environment to make mapping of public transport more convenient. I’m afraid we got distracted by other things before it was completely finished though. Noémie asked me where those ‘specialised’ overpass queries went to. We’ll have to look into that. They will probably end up in the history pane of the download window.

He improved the roundabout splitter, which had some quirks and he added support for bicycle route relations to it. We all know those are a challenge to get right.

For the cherry on the pie, some changes were needed in JOSM core. We needed the ability to add some extra buttons to the relation editor. Thank you Michael Zangl for making the necessary changes in core to make this possible!

One of those buttons now enables you to sort stops from within the relation editor, instead of on the selected route relations.

You can find links to screencast videos on the documentation page on the wiki

Routing helper Turn-by-turn mode

The button that may completely change how we map bus lines, is what many of you will appreciate the most. We started calling it the ‘Routing Helper’, for lack of a better name. You could also call it ‘Add ways up to next intersection and then ask user where to turn to’. It will however, also look at other route relations and if it finds one with the same 2 ways in it, one were it is at that point in the itinerary and the next one in the route relation, it will present this as option 0. This will usually present suggestions covering a longer distance. It won't suggest sequences that are not continuous, that travel against oneway traffic, or that don't heed turn restrictions correctly.

Routing helper Fast Forward mode

Since our route relations tend to be extremely repetitious, this will mean that if a route between 2 stops is mapped once, it will then be proposed for ‘reuse’ in other route relations.

This will be a big step forward and away from:

  • select ways
  • select a node
  • split
  • deselect one part
  • add to relation editor
  • sort
  • rinse
  • repeat.

Pressing those numbers may become a bit tedious at times, but it’s at least 10 times more efficient than the list of actions above.

Cartographie du réseau de bus à Dakar

Posted by Polyglot on 1 August 2018 in French (Français)

En 2014 Odette Mendy a réalisé un travail de diplôme BTS Géomatique concernant le transport en commun à Dakar. A l'époque elle a utilisé ODK-collect pour répérer les arrêts et ARCGIS pour cartographier les lignes.

Par la suite elle a voulu partager ce travail avec le grand public. OpenStreetMap était le choix logique. Elle a trouvé la communauté OpenStreetMap Sénégal très accueillante et ils se sont organisé pour entâmer ce travail de grande envergure. L'été de 2018 le réseau du département de Dakar est maintenant complètement cartographié. La communauté souhaite élargir ce projet pour les autres localités de Dakar et pour l’ensemble du pays.

Ceux qui ont participé :

Odette MENDY avec l'aide de Ibrahim SANA et Inocent DIBLONI de la communauté OSM Burkina et les membre de la communauté OSM-Sénégal

Lamine NDIAYE Jean Marie MANCABOU Amadou NDONG Ismaïla SEYE Rokhaya SECK Alassane CISS Bougouma Lamine NDAO

Odette Mendy

Location: 14.664, -17.437

public_transport=stop_position nodes

Posted by Polyglot on 23 July 2018 in English (English)

It's no secret that I'm trying to figure out how to simplify mapping of public transport to bring it within reach of all mappers. All without losing the ability to map stops, lines and itineraries in full detail without any limitations. My vision is that a stop with no discernible objects in the middle of nowhere and a stop with all possible equipment should be mapped exactly the same: as a NODE next to the highway/railway.

There are many stop_position nodes I added over time that don't serve much of a purpose, but then there are some that PT_Assistant's validator actually started to depend on. It's those that 'terminate' itinerary variations.

Maybe we should get terminology defined first.

What I call a line, is represented as route_master relations in OSM (and confusingly found in routes.txt in GTFS). What I call an itinerary or itinerary variation is represented as route relations in OSM. And stops, well I'd say highway=bus_stop, railway=tram_stop, railway=halt to begin with and public_transport=platform + mode of transport, if you like.

Anyway, stop_position nodes as indication of where an itinerary starts/ends (and splitting the way on those nodes) does make sense.

I still wouldn't add them for each and every stop though, I still wouldn't add any details like name on them and I definitely wouldn't add them to the route relations though. That's what the nodes next to the highway/railway are used for.

public_transport tags don't add any information

Posted by Polyglot on 19 July 2018 in English (English)

In 2012, the 'new' public_transport scheme was voted. What seemed like a good idea at first, has not led to unification though. I was not smart enough at the time to see how this would develop, but today with the benefit of 20/20 hindsight it it's apparent that we are on the wrong track.

The way the wiki formulates it at present, even though I have tried over the years to keep it in check, we should supposedly map 2 objects for each and every stop and then add both of these to the route relations.

I'm sorry, but that is unacceptable! For one thing, I need a SINGLE object, preferably a node where all the details of the stop go. That has always been the way we do things in OpenStreetMap. The node that gets the stop_position tag is NOT suitable for that purpose, as it is not immediately apparent on what side of the street the stop is.

So, over the years I have tried to work with public_transport=platform and stop_position tags on the stop nodes I've been mapping. The idea was that, in time, highway=bus_stop would be replaced by those tags. Only that is not possible, for one thing, those nodes we're mapping to represent the stops, were not supposed to have a mode of transport on them. So it is utterly impossible to drop highway=bus_stop from those nodes.

OK, let's step back for a moment: highway=bus_stop is here to stay, wait for it... it's coming... the conclusion: public_transport=platform tags are NOT ADDING any information. highway=bus_stop says it all. Sorry that it took me a few years to figure that out!

Of course, now the epiphany dawned on me, I decided to revisit all 70000 bus stops around Belgium. So since yesterday I started to remove public_transport=platform/bus=yes tags from all those stops, in packages of about 100-150. Obviously that's not all I'm doing, some stops have moved, we have better imagery now than we did 5 years ago, there is a new bus_bay tag since last summer. It will take 6 to 12 months to complete this mission, but soon those tags that don't add actual information will disappear from sight.

It's time we came up with a scheme to map public transport that is: * easy to work with * easy to maintain * easy to visualise and that: * does not duplicate information * does not force mappers to add 2 objects to route relations

I have a clear vision of how we can accomplish that.

I changed my vote for the PT "v2" scheme to no

Posted by Polyglot on 8 July 2018 in English (English)

It seemed like a good idea on the face of it. Stops on the highway became public_transport=stop_position, stops next to it public_transport=platform.

But then it was suddenly necessary to add both these objects to the route relations. Already the fact that more than one object became necessary to describe a single bus stop, should have raised alarm bells.

There are, of course, some positive things to say about to say about the new scheme. A route_master relation for a whole line. route relations to describe the various itineraries and stops.

But the stops, that's where it goes seriously wrong. Why on earth is it necessary to add 2 objects to those route relations? As they describe all the variations, there are a lot of them. Lots of work to maintain them. And they break easily.

After the scheme was voted, I was trying my best to map stops according to it. I had my highway=bus_stop nodes which were all nicely positioned next to the highways, so it's clear on which side of the road they are located.

So I asked on the mailing list: hey what public_transport to use for them? The answer was public_transport=platform. So far, so good.

Since I had 40000 stops to add for half of a small country, I was not going to bother with adding stop_position nodes as well. So I trudged along for a few years.

In the mean time in Germany and France they had a different idea, let's convert those nodes next to the highway to platform ways (regardless of whether there was an actual platform present in some cases), oh and those stop_position nodes, let's duplicate all the details for the stops.

So, for the sake of simplicity, I would like us all to come back to our senses. Map each bus stop, exactly once on a node next to the highways and only add these nodes to the route relations. That's how I will continue to do it in Belgium, although unfortunately in Brussels somebody already started adding stop_position nodes everywhere and adding them to the route relations. Oh well.

So I'm not sure if it will ever happen that we all move to a more sensible scheme, but in case it doesn't, I promise to continue moaning and bitching about this overly complex way of mapping public transport.

Polyglot

QGIS 3 on Linux Mint

Posted by Polyglot on 10 March 2018 in English (English)

I finally managed to install QGIS 3 on my favourite linux distro. The magic incantation was found here:

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key 073D307A618E5811

echo "deb http://qgis.org/ubuntugis xenial main" > /etc/apt/sources.list.d/qgis.list

echo "deb-src http://qgis.org/ubuntugis xenial main" >> /etc/apt/sources.list.d/qgis.list

echo "deb http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu xenial main" > /etc/apt/sources.list.d/ubuntugis-unstable.list

echo "deb-src http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu xenial main" >> /etc/apt/sources.list.d/ubuntugis-unstable.list

sudo apt-get update

sudo apt-get install qgis python-qgis qgis-plugin-grass

#for some plugins QtWebKit is needed:

sudo apt-get install libqtwebkit-dev python3-pip python3-setuptools

pip install jupyter==1.0.0 qtconsole

PT v3 some thoughts

Posted by Polyglot on 21 December 2017 in English (English)

There is some talk about a new scheme for mapping public transport on the German forum.

I'll try to list some requirements such a scheme should have:

STOPS

  • It should be easier to map than the mess v2 has become.
  • Simple for simple cases
  • Scalable to more complex cases, without the need to convert nodes to ways or relations.
  • Details like ref, route_ref, operator, network, zone should only appear on a single object.
  • By looking at this object it is immediately clear on which side of the road the stop is located.

ROUTES

  • The object that represents a stop for a single direction of travel only appears once in the route relations.
  • easy to understand, for anybody who bothers to look inside the relation
  • straightforward to process for map rendering
    • This means that a scheme with 'hints' about the routes, using reference points along the way, is not suitable.
  • one route relation per variation of the itinerary

It doesn't really matter that highway=bus_stop, railway=*, highway/railway=platform are used in combination with public_transport tags. Rendering doesn't seem to be able to go without the 'deprecated' tags. Mapping (using JOSM) is more convenient when using public_transport style tags (roles are set automatically for example).

From the above requirements the only logical solution is to use a node to represent the bus stops. highway=bus_stop / railway=tram_stop bus=yes / tram=yes public_transport=platform

If an actual platform is present where passengers can wait, it can be mapped as a way/area using highway=platform and/or railway=platform. No need for public_transport=platform on these areas, lest someone might say we're mapping platforms twice. And no need for adding these objects to the route relations. They can be added to stop_area relations. My personal preference would be one such stop_area relation per direction of travel, or per platform in bus stations.

I'm preparing a proposal here: https://wiki.openstreetmap.org/wiki/User:Polyglot/Bus_stops_and_routes

Frustration about iD editor's inability to easily draw rectangular buildings

Posted by Polyglot on 15 December 2017 in English (English)

This tweet inspired me to write a new diary entry.

BHousel does not like the criticism he's been getting for several years now.

Drawing rectangular buildings using the iD editor is a lot harder than it ought to be.

My opinion is that if you create a tool that is going to be used by thousands of mappers, you owe it to your users to make sure that they can produce quality work with ease. Especially if the effort to do so would outweigh by an enormous ratio the frustration shoddy work creates for HOT's validators (and (re)mappers using that other mapping tool), who need to put more time into fixing the mess, than they would have actually mapping those buildings themselves in the first place. It's not like the science of creating a rectangle based on adding 3 points using 3 clicks still needs to be invented.

This is an ongoing problem for many years. So either it gets fixed one day, or frustrated messages will happen to escape, every once in a while. Or they will get muffled and HOT validators will continue to silently give up on trying to fix the deluge of poorly mapped buildings. Or you can simply unsubscribe from HOT's Twitter feed. Whatever works for you.

A more constructive solution would be to invite other programmers into the project. I can't imagine there wouldn't be candidates.

Here's hoping that something good can come out of this.

Polyglot

diary spam

Posted by Polyglot on 25 September 2017 in English (English)

I'm looking at the diary entries once a day now. All too often there is spam content. What can we do about this? Maybe a vote button and if 3 people mark an entry as spam, it is removed from view and verified by an admin?

Jo

GSoC 2017 PT_Assistant plugin for JOSM

Posted by Polyglot on 17 September 2017 in English (English)

Giacomo Servadei continued to work on the PT_Assistant plugin for JOSM

Hiking, bicycle and equestrian routes

The plugin is not just about public transport anymore, it can also highlight hiking and bicycle routes and report problems with their continuity. It now also visualizes forward/backward roles, which makes editing them a lot more convenient.

One of the problems I noticed over the past year, was that while fixing public transport route relations, sometimes foot and bicycle routes were broken and the validator didn't warn about this. The other problem is that when bicycle routes fork, forward and backward roles are needed, but they depend on the direction of the way and those little arrows are very hard to see.

Now one leg is coloured in blue, the other in red, which makes it immediately obvious whether the route is mapped correctly and it's easier to know if forward needs to become backward or vice versa.

forked bicycle route relation

Public transport improvements

Tools

  • The plugin can now sort stops according to the sequence of the ways in the route relations.

  • It can help with splitting roundabouts, while keeping the route relations that pass over them correct

  • There is a new map mode to add stop_position nodes. If such nodes are added to the first or last ways of the itinerary, the way is split and only the pertinent part is kept.

  • And there is a new map mode to help with ways selection. It will select all ways in between 2 forks, which are suitable for the mode of transport worked on. At the moment this defaults to bus.

Validation

There are also some new categories of problems the plugin reports and proposes fixes for.

  • gaps of a single (suitable) way + automatic fix

  • a bug with the detection of relations that could be fixed by simply sorting the ways was fixed.

  • a new category will tell the mapper about relations that end up with less gaps when the ways are sorted. This can help detect routes tagged with public_transport:version=2, which aren't composed of a simple sorted sequence of ways for each variation, but instead still have ways for both/all directions of travel in them.

  • There is a new category for routes that don't start or end neatly on a stop_position node near to a corresponding platform node.

  • The names of the first and last stops are compared to from and to tags in the route relations.

  • A warning is given if the first or last way don't correspond to the first or last stop in the route relation.

GSoC 2017 JOSM

Posted by Polyglot on 13 September 2017 in English (English)

Bogdans Afonins made several improvements to JOSM's search and to the download dialog:

Search dialog

JOSM search dialog now includes search based on presets

Search is incredibly powerful in JOSM. It's possible to add buttons to the toolbar for searches you need over and over again. Some examples:

Select all ways of a roundabout or a closed oneway way, which is in view plus all the nodes of that roundabout that have more than 1 one way connecting to it:

R inview ((junction=roundabout OR closed  oneway=yes) OR ways:2- child (junction=roundabout OR closed oneway=yes))

I use this to make them round and distribute the nodes or to conveniently select all the ways of a roundabout after it was split.

Find all routes that are already public_transport:version=2, which have members in view and which were modified:

R "public_transport:version"=2 route=bus inview modified

Download dialog

JOSM new download dialog with Overpass API wizard

The wizard works like in Overpass Turbo, but of course the output will be set to meta, so JOSM can digest it.

Some examples:

[out:xml][timeout:325][bbox:{{bbox}}];
(
  (
    node["highway"="bus_stop"];
    node["public_transport"="platform"]["train"!="yes"]["ferry"!="yes"];
    node["public_transport"="stop_position"]["train"!="yes"]["ferry"!="yes"];
  );
  ._;<;
  way["highway"="platform"];
  way["amenity"="shelter"];
  node["amenity"="shelter"];
  relation["route"="bus"];
);
(._;>;);
out meta;

This will download a skeleton of all bus routes in a the selected region. It's a bit complicated, but sometimes it's easier to go from complex to simple, by removing what is not needed for your use case.

Another one (now that I'm on a roll):

[out:xml][timeout:325][bbox:{{bbox}}];
    node["highway"="bus_stop"]["public_transport"!="platform"]->.allbusstopnodes;
    way["highway"];>;._->.allhighwaynodes;
    (.allbusstopnodes - .allhighwaynodes;);
out meta;

This stores all highway=bus_stop nodes in the selected bbox, which don't have public_transport tags yet in a named container. Then it puts all highway nodes in another. Then it subtracts these, resulting in all highway=bus_stop nodes, which are not part of highway ways.

bus stops

Posted by Polyglot on 17 August 2017 in English (English)

Over the past few years, I've been mapping a lot of bus stops and routes. Over the past few months I've been looking around how it's done in other places around the world.

What I find (this may of course be influenced by what I wanted to find) is that most bus stops start out as a node next to the way, tagged highway=bus_stop, usually with a name on it.

Then somebody comes along, who glues it to the way. Then somebody else comes along who maps a platform next to the way, using a way, copying all the tags over. They do this regardless of the whether a platform actually exists. They think they need this to add 2 objects per stop to the route relations. Somehow the wiki convinces them of the need for this.

Mapping platforms as ways where there aren't actual platforms feels wrong to me. Adding a set of details to more than one object also doesn't feel like the most sensible thing to do. Adding 2 objects to each route relation for 1 real life object makes working with these route relations harder to do.

I'd like to see that a casual mapper can start out mapping a bus stop as a node next to the way, as highway=bus_stop.

Possibly they add public_transport=platform to this node. It seems odd to do so, even if no platform is there, but that's how things evolved. Tagging it as public_transport=platform means that a role platform will be assigned to it in the route relation. (if the route relation has public_transpot:version=2). It's the answer I got when I asked the question a few years back on the mailing list. I usually put that node more or less where the pole is. But the exact position is not all that important, as long as it's more or less where passengers will be waiting.

As far as I'm concerned this node is what represents the bus stop at this side of the street, so it seems obvious that this is the node that should be added to the route relations. The nice thing about doing it that way, is that there is no need to transfer details or relation membership from one object to another. This node is there for the lifetime of the stop. One object to work with, containing all the details, including directly available coordinates.

Now the stop_position nodes. First off, I don't see a need to map all of them and since where I map bus lines they are not all present, I don't see why we would add any of them to the route relations. No need to repeat the stop's details on them either.

Then the platform ways. If there are actual platforms, I would tag them highway=platform/public_transport=platform. Nothing more, nothing less. No need to add them to the route relations.

Then the stop_area relations. This is where we get the chance to indicate which platform node(s) belongs to which stop_position node(s) (if mapped) and which platform way (if present). Doing it this way means to have such relations for each group, where a group is the ones on one side of the road or the ones belonging to 1 platform in a bus station. They can then be grouped in a hierarchy of stop_area_group relations, although this is only needed for the more complex ones and for bus stations.

Sometimes I wonder if I should create a proposal for this. Usually I give up before even getting started, as I know how hard it is to convince anyone in the OpenStreetMap universe, once they are set in their ways. Little by little I'm starting to build the courage to go ahead with it after all. Writing these diary entries is a small first step, also to see if there would be support for such a simpler way of mapping public transport.

Also don't get me wrong, it's still possible to fill in all the detail of the accomodation surrounding the bus stops, but in a way that builds further on what was mapped previously, instead of converting from one object type to another or needing to add details more than once.

Public Transport Mapping, why do we add the stop details several times over?

Posted by Polyglot on 27 July 2017 in English (English)

When we started mapping public transport stops, some people insisted on mapping them on nodes next to the way, others thought the right way to do it, was to add them as nodes being part of the highway, thereby losing the information on which side of the road the bus stop was.

Then somebody came by with the idea to unite both ways of mapping. In itself that sounds great. But where do we add the details then? On both? That doesn't really make sense. It's a maintenance nightmare.

So we still have some people adding the stops as stop_position nodes on the highways and others mapping them as isolated nodes next the ways as public_transport=platform. But of course a node is not a platform, so others map those as ways and areas. Nothing wrong with that, but why do we need to add all the details to these ways?

For some reason it was decided that both these stop_postion nodes and the platform ways/nodes need to be added over and over again to the route relations. These route relations represent each and every variation of the public transport lines, so there are thousands of them. Another maintenance nightmare.

Why can't we have a node next to the way, with all the relevant details and add those nodes to the route relations, then followed by a continuous string of ways? The node gets tagged public_transport=platform/highway=bus_stop.

The node isn't always representing an actual platform. If there is a platform, nothing wrong with representing it as a way or an area. But there is no real need to duplicate all the details like name/ref/route_ref/zone to these ways. And there isn't really a need to add them to the route relations.

For the simplest bus stops a node next to the way public_transport=platform/highway=bus_stop is all that is needed. It contains all the relevant information and it has coordinates, which makes it convenient to compare it to data from operators.

For more completely mapped bus stops, benches and waste_baskets can be added.

If you want to make explcit where the vehicle stops, a public_transport=stop_position can be added on the highway. For the first and last stops, the way should be split there, as it's the beginning or end of the route.

But these stop_position nodes are not all that important, so no real need to map them for every stop. Also no reason why the stop details should be repeated on them and no real need to add them to the route relations. It's enough to add them to stop_area relations.

Public Transport Plugin

Posted by Polyglot on 1 July 2016 in English (English)

Mapping public transport routes on OpenStreetMap can be a challenge, keeping them correct and up-to-date even more so. Last year I mentored the Mapillary plugin for GSoC. Before that, I had been working on public transport a lot, and to add the stops precisely where the poles are, horizontal imagery is a big help, when it's available. The idea to propose a GSoC project for public transport this year was a logical consequence of some Python scripting I had started on. It's possible to run Python in JOSM, but it involves too much setup for it to be practical for general use by other mappers.

Still, automation is key. Without it, it's next to impossible to even find all the problems with the route relations, let alone fix them.

Also the way we're mapping public transport, at the moment (who knows one day route segments become the thing), is very repetitive and fixing all those routes for all variations of the lines gets tedious fast. This in turn results in very few people interested in doing it. And the ones that do burn out sooner or later.

I'm sure the new plugin will be a big help to solve that problem.

At the moment the plugin is very good at detecting problems with the routes:

  • going against oneway traffic
  • gaps that are caused by wrongly ordered ways (automatic fix available)
  • routes that don't start/end on a stop_position node
  • ways that still need to be split on those terminal nodes
  • buses or trams that travel over ways unsuitable for that mode of transport

The latest addition is a dedicated extra layer. This layer highlights the route for which a relation editor window is open. You can have several relation editor windows open, but only the last one that had focus, will be shown highlighted. One thing that I was never able to achieve with plain MapCSS is to show sequence numbers for the stops and the platforms. The other was showing the refs of all the route relations serving those stops. All that becomes trivial (compared to MapCSS) if one can code it in JAVA, and Darya is doing a great job there.

So, what's next?

The initial idea was to be able to start from a route relation containing just stops in the correct order and have the code figure out which ways to add to the relations. That will still happen, but fixing existing route relations and helping all mappers to keep the route relations 'coherent', when they make changes to the underlying highway/railway infrastructure is a higher priority, so we started by integrating the plugin in the validator. Thus making it very accessible for anybody who installs the plugin. It's amazing how many problems have already been able to solved - both in the routes and in the ways they followm, as in the platforms and stop_positions -during beta testing of the code.

Want to try it yourself?

You can easily install the PT_assistant plugin in JOSM and help with testing.

User documentation can be found here

What I usually do, is File / Download from Overpass API (expert mode needs to be on for it to show that option in the menu, don't ask me why) Then use the following query:

[out:xml][timeout:125];
(
 relation["public_transport:version"="2"]["route"="bus"];
);
(._;>;);
out meta;

You can select routes for other modes of transport, of course. After downloading, I make sure nothing is selected, then press the validate button. Screenshot of dialog I usually use the last option, as I don't want all the fixes applied automatically for the time being. Implementing automatic fixes with user intervention where there is doubt, is in the pipeline. If you have the plugin installed, you'll find several categories that start with PT_ Screenshot of validator pane

The easiest ones to tackle are: * Route contains a gap that can be fixed by sorting

For those you can simply press the fix button. Don't select more than, say 5, at a time.

If your stops where not at the beginning of the route, now they will be. The order should be preserved. If they don't have roles, select all of them, make them selected in the editor window, remove them from the Relation Editor and readd them. Now go visit the ones that still don't have roles and fix their tagging.

Now press the upload button, which will result in a new validation Usually you'll find there are more problems to solve at this point. For the routes that have ways going against oneway traffic, pressing the fix button will result in removal of that way. It's better to press right mouse button Zoom to problem, select the way(s) that have a yellow casing around them, then press right mouse button edit. The relation editor will open with those ways selected in blue. Now it's possible to remove them, or to add oneway:bus=no, if that is the proper solution. If you remove them, you probably want to readd ways from the other side of the dual carriageway or roundabout. If you notice that there are several variations, which are likely to need the same fix, keep the 'problem ways' selected and open all of them in relation editor windows. After all are openedm select the ways to replace them with, and do this in each editor window.

Be careful not to split ways for which you have relation editor windows open.
You'll get 'conflicted with yourself', which is something to avoid at all cost :-)
Have a look in the comments to learn how to avoid conflicts while keeping RE windows opened.

All this may seem like it's still a lot of manual work and you're right, it is. The intention is to automate all this further. I'll probably write a new diary entry in a few weeks about that.

If you're interested in a hangout about the mapping of public transport and the use of the plugin, get in touch and we'll set one up.

Setting up JOSM

Posted by Polyglot on 13 January 2016 in English (English)

I recently configured OBS (Open Broadcast Software) to conveniently make screencasts toward Twitch.tv.

Usually I use this to show HOT contributors what I did while validating their tasks/tiles. I get some nice response to this. People like the feedback and the tips. I think it works as a motivator to keep going as well.

For the past few weeks I had this plan to create more to the point video material to show how to map with JOSM and why I think it's so much more convenient and efficient to do so.

I had a problem with Bing imagery that was solved, by dumping my profile, so I seized the opportunity to show to get started with JOSM. While doing this I noticed there was imagery for Svalbard available and WOW, that is great imagery! So went on to improve my mapping of the Global Seed Vault. The video shows how to improve way accuracy, how to make nice bends in the roads, how to draw buildings very accurately, how to link to Wikipedia and Wikidata. How to jump to these articles in your web browser from withing JOSM and so much more. As usual I went a bit overboard, sorry about that.

I did it once more, this time with sound: Setting up JOSM for HOT mapping

These have no sound:

Setting up JOSM (part 1)

Setting up JOSM (part 2)

Setting up JOSM (part 3)

Setting up JOSM (part 4)

Location: 78.235, 15.490

Cooperation between free content projects: how to link from Wikipedia to OSM

Posted by Polyglot on 14 August 2015 in English (English)

This blog entry describes how and why the following module was developed:

https://en.wikipedia.org/wiki/Module:OSM

In the mean time I added extensive documentation to it, which might be more interesting than what I wrote here... so go and have a look!

https://en.wikipedia.org/wiki/Module:OSM/doc

With the advent of Wikidata my interest in Wikipedia flared uponce more. I'm a mapper for Openstreetmap in the first place, but last year I already made a little sidestep to Wikivoyage. That's a project which is closer in many ways to Openstreetmap than Wikipedia.

Recently I started adding wikidata tags to streets and other objects in Openstreetmap. Which in itself doesn't do all that much. So a way is needed to make this available to the rest of the world. Wikipedia seems like the way to accomplish this.

I had already experimented a bit with links to saved queries on Overpass Turbo. But creating these queries from scratch over and over again and creating such links to them, isn't the most straightforward way to get things done.

When it became clear to me that sometimes an additional query of wikidata may be necessary, before invoking Overpass, I knew I had to tackle the problem in another way.

This made me look into Lua, the scripting language which was chosen to automate the plethora of MediaWiki projects. After one day I came up with a working script (in the mean time it's been developed quite a bit further):

On the Dutch Wikipedia reception was luke warm. I'm afraid, I hadn't been able to carry across what I was trying to achieve. On the English Wikipedia a Lua developer saw the potential and he helped me to improve the code and explained me how to go about adding test cases. This helps a lot to not introduce bugs while attempting to add new features. For the case of querying several Q-numbers at once using a regex query, help came from a specialist in Overpass.

So this saved query:

[http://overpass-turbo.eu/s/aFt]

now looks like this in the article:

{{#invoke:OSM|etym|display=Streets named after Leuven|timeout=50|query=[highway]|coord=50.879;4.701;9}}

which results in the following query:

[timeout:50][out:json]; ( node["name:etymology:wikidata"="Q118958"][highway]({{bbox}}); // remove the ({{bbox}}) if you want the query to be executed globally way["name:etymology:wikidata"="Q118958"][highway]({{bbox}}); relation["name:etymology:wikidata"="Q118958"][highway]({{bbox}}); ); out geom;

The difference with the stored query is that only streets closer to Leuven are fetched. If the user wishes to do so they can either remove the ({{bbox}}) statements or they can move the map to another region and press Run. In case the user also wants to see other objects named after Leuven, they can remove the [highway] part.

Now, let's hope the Wikipedia contributors at large see value in this. I deployed it on the French Wikipedia as well and will probably continue with Spanish, German and Esperanto. Feel free to deploy it on other language versions as well. I will keep all the modules in sync. I'm only going to develop test cases on the English Wikipedia, so that will be the primary place to continue development. One test is failing because of lack of arbitrary access, but that should be resolved shortly.

Actually there is a whole plethora of possibilities:

  • objects named after the subject of the Wikipedia article (on condition they got name:etymology:wikidata tags on the OSM side)
  • the object of the article itself (on condition they got wikidata tags on the OSM side)
  • other objects related to the article (this is probably not OK with Wikipedia policies)
  • objects of which the subject of the article is the architect or the artist who created them (architect and artist.wikidata)
  • objects related directly to the subject (subject.wikidata), this can be tombstones as well
  • objects carrying brand:wikidata or operator:wikidata tags
  • relations representing public transport lines. The names of the stops are emphasized with help of MapCSS styles
Location: 50.878, 4.700

Samenwerking tussen open content-projecten

Posted by Polyglot on 4 August 2015 in Dutch (Nederlands)

Met de komst van Wikidata is mijn interesse in Wikipedia weer wat opgeflakkerd. Ik ben in de eerste plaats een mapper voor Openstreetmap, maar vorig jaar heb 'k ook al 's een zijsprongetje gemaakt naar Wikivoyage. Een project dat vrij goed aansluit bij Openstreetmap.

Nu ben ik recent begonnen met het voorzien van straten en andere objecten in Openstreetmap van wikidatatags. En aan welk project voor vrije content denkt een mens meteen, om ervoor te zorgen dat dat werk ook voor de rest van de wereld ter beschikking komt? Inderdaad: Wikipedia.

Ik had al wat geëxperimenteerd met links naar opgeslagen queries op Overpass Turbo. Maar heel handig is dat toch niet. Een query moeten aanmaken voor elk wikidata-item en elke nieuwe soort bevraging.

Daar komt dan nog bij, dat een Kerkstraat vernoemd naar een bepaalde kerk naar die specifieke kerk verwijst, maar als je alle Kerkstraten wilt terugvinden tesamen met alle 'Rue de l'Église', moet je eigenlijk eerst wikidata gaan bevragen. Maar dat zal nog niet voor direct zijn.

Het heeft me er wel toe aangezet om me 's in Lua te gaan verdiepen, de scriptingtaal gekozen voor de MediaWikiprojecten. Een dag later hebben we nu dus een script dat Overpass queries genereert:

https://nl.wikipedia.org/wiki/Module:OSM

Nu komen de Wikipediamedewerkers en ikzelf niet altijd zo goed overeen. Als ik buslijnen toevoeg, krijg 'k te horen dat ze geen busboekje willen zijn. 't Kost dus allemaal nogal wat moeite, want dan moet 'k op zoek naar bronnen en referenties over het ontstaan en de geschiedenis van zo'n route.

M'n test case werd alweer ongedaan gemaakt. Gelukkig is het mogelijk om naar een specifieke versie van een pagina te verwijzen:

https://nl.wikipedia.org/w/index.php?title=Fontenoille&diff=prev&oldid=44648493

Dus wel verloren voor het nageslacht, maar niet voor wie mijn hersenspinsels hier leest.

Ook op de pagina over Leuven had 'k al eerder een query gezet die alle straten die naar Leuven vernoemd zijn ophaalt.

Oorspronkelijk dus deze opgeslagen query:

[http://overpass-turbo.eu/s/aFt]

Maar nu ziet het er als volgt uit (in het artikel):

{{#invoke:OSM|etym|linktext=Straten wereldwijd vernoemd naar Leuven, inzoombaar|query=[highway]|coord=50.879;4.701;9}}

wat resulteert in het volgende:

http://overpass-turbo.eu/?Q=( node["name:etymology:wikidata"="Q118958"][highway]({{bbox}}); // remove the ({{bbox}})if you want the query to be executed globally way["name:etymology:wikidata"="Q118958"][highway]({{bbox}}); relation["name:etymology:wikidata"="Q118958"][highway]({{bbox}}); ); out;

; out meta qt; &C=50.879;4.701;9&R

Het verschil is dat de query nu enkel straten ophaalt, die dichter in de nabijheid van Leuven liggen. Als de gebruiker dat wenst, kan hij/zij zelf de ({{bbox}})-statements weghalen. In het geval dat de gebruiker ook andere objecten wenst te zien, die naar Leuven werden vernoemd, dan volstaat het om [highway] te verwijderen.

Goed. De deur staat open om links te leggen van Wikipedia-artikelen naar OSM-objecten, iets wat we al langer in de andere richting doen. Nu is het echter de vraag of Wikipedia daar klaar voor is, of zelfs of het dat ooit zal (willen) zijn.

Veel meer dan de technische hulpmiddelen bieden, kan ik niet meer voor hen doen.

Jo

Location: 49.865, 4.932

Mapillary 1.0 for Android

Posted by Polyglot on 9 March 2015 in English (English)

Mapillary for Android was updated and I've been meaning to write about it here.

Blog entry by developer of the app

The app now has a big green hot-button, which is very practical to go into picture making mode quickly.

The app now allows automatic shooting of pictures every 2 seconds even when standing still (or walking/cycling, the intended usage). The way to make it pause is to point it downwards. It's not perfect at detecting this though, so I end up with quite a few pictures of my feet... It also stops shooting when you shake the phone too much. Maybe I should try that instead to pause it :-)

Oh, it's possible to pause manually as well, of course, but that has the side effect of starting a new sequence. Sometimes that's what you want, but not always.

It's possible to let the app upload as soon as it's in range of Wifi. That's practical, but I disabled that option in the mean time. I prefer to browse through my pictures on the larger screen of the portable, as it allows to weed out the bad ones more easily. The way to review pictures on the phone was improved, but it still feels like a lot a lot of finger movement and my phone doesn't seem to understand the 'long click' very well. I also lose track of which sequences I already reviewed and how far into them I had gotten, before having to context switch to irl.

Using the PC also has the advantage of the keyboard. On top of that, I can work with the pictures in the usual way of working with images in JOSM. Just load whatever GPX file, then use right mouse button on its name in the layer list to load pictures from the Mapillary folder on your phone's memory card. This works, because they are already geotagged. I do that while it's connected over the USB cable.

The advantage is that it's possible to review the pictures once more and you don't have to wait for the pictures to be processed.

After all that reviewing, I then release the pictures to Mapillary by enabling wifi and going into review mode.

Since I'm "multimodal", walking quite often, but also on the bicycle, I needed a way to secure the smartphone. I'm glad I finally found a use for 2 of those lanyards and a rubberband bracelet I've been receiving at conferences. I really should make a picture of that 'construct'.

The rubberband around the middle gets in the way when typing, so the next phone I get should come with a proper silicone protection jacket around it.

My findings are that pictures taken while cycling, while holding the phone in one hand, can be OK, but only with enough sunlight coming from behind. With the sun in front the pictures become dark and with insufficient light the shutter remains open longer, resulting in unsharp pictures.

The same is true to a lesser extent while walking. The sharpest pictures are those taken when standing still.

I'm not sure it they're very happy at Mapillary with my pictures, as I'm turning around a lot and focusing on those details that matter for adding details to Openstreetmap. I don't like switching back and forth to Panorama mode though. That new autoshooting walking mode is so convenient!

I'm sure glad Mapillary is around now, as it becomes possible to add context to the pictures of those details I'm interested in, by also making pictures of the surroundings and how one got there. At the end of the day, that's a lot of pictures though... but it opens possibilities we didn't have before. Like enabling other people to add the details to OSM that happen to be important to them.

1001-bomen-voor-Leuven

Posted by Polyglot on 5 March 2015 in Dutch (Nederlands)

Deze kaart laat toe om te zien hoeveel bomen en groenruimtes er zijn in Leuven. De data komt uit Openstreetmap.org.

Aan de linkerkant bovenaan kan u in/uitzoomen met de +/-, maar dat gaat eveneens met het muiswieltje.

Onder deze iconen staat een knop, waarmee u lagen kan selecteren. Klik eens op het oog van de heatmap-laag, om de 'concentratie' van de bomen te zien.

Dit is voorlopig niet echt representatief, aangezien niet elke boom al ingebracht werd in Openstreetmap. Wij hopen dat we dat wel kunnen realiseren met de hulp van de groendienst, de KU Leuven en u.

Voor sommige bomen werd species (soortnaam), genus en soms zelfs een foto toegevoegd. Als de foto op Mapillary.com staat, is het zelfs mogelijk om een virtueel bezoek aan die plaats te brengen. Dat wil zeggen dat er daar iemand is langsgeweest, die nog meer foto's gemaakt heeft van de omgeving van de boom.

U kan zelf bomen toevoegen aan de Openstreetmapdata. Klik op de knop More en dan de voorlaatste knop. Waarschijnlijk is het het eenvoudigste om daarvoor de iD-editor te gebruiken. Dan hoeft u geen extra software te installeren. Als u veel bomen wenst toe te voegen, geef ik graag een woordje uitleg over hoe dat efficiënt gedaan kan worden met JOSM.

U kan ook voorstellen formuleren, over waar er volgens u bomen of groenruimte kan worden toegevoegd. Daartoe kan u de knoppen aan de rechterkant gebruiken, nadat u op dat potloodje heeft geklikt om naar editeermodus te gaan.

Spijtig genoeg is het niet mogelijk om slechts bepaalde stukken van de kaart voor iedereen editeerbaar te maken. Het is alles of niets. Daarom is het nodig dat u eerst een account aanmaakt.

https://www.openstreetmap.org/user/new

En deze dan hier toevoegt: http://umap.openstreetmap.fr/nl/ (Inloggen / Log in)

Als u me dan een boodschap stuurt, voeg ik u toe als bewerker voor de kaart.

Wat ook kan, is hier oefenen. Hier kan alles aangepast worden, maar de consequentie is dat ook alles gewist kan worden door eender wie...

Oefenkaart

U moet dan wel nog even laten weten, waar u iets heeft toegevoegd. Het is waarschijnlijk ook het beste om daar steeds op een eigen nieuwe laag voor aan te maken.

Location: 50.879, 4.700

First Missing Maps in NL and Mapillary

Posted by Polyglot on 15 February 2015 in English (English)

After participating in the Missing Maps in Antwerpen, I went to the one in The Netherlands yesterday. Big thanks to Philip for making this possible for me!

The people from MSF/AZG/DWB/Red Cross had 2 Tasks in mind for us.

A though one in Africa where we had to add natural wood and rivers, as those are the (breeding) habitat for tse-tse flies, which cause sleeping disease. The beginners using iD had to abandon that one, as it's impractical to zoom in far enough to work and still see the bigger picture. With JOSM one can draw a general contour of the wood at low zoom, then refine it by using the Improve Way Accuracy mode zoomed in a lot further. It's still a daunting task to get it right though. What can be considered wood/forest? How does one recognise wetlands?

The other task was in Haiti. It was not easy either. A densely populated area and MSF wants to know how many buildings there are. (It's explained better in the description...)

It peaked my interest for 2 reasons:

  1. They used a drone to create superior aerial imagery
  2. The Red Cross drove around with a smartphone used as a dashcam. The pictures were uploaded to Mapillary. Press that little play button and enjoy the ride.

This allows to go and have a look around and see what's actually there. The width and state of the road, the state of completion of the buildings and so on.

Coverage is more limited than what can be seen from above, but it's possible to read the name of a school and then it's clear as well that it is a school.

The great thing about Mapillary is that the barrier of entry to contribute yourself is a lot lower than say, fly a drone to create excellent imagery...

All you need is a smartphone with a descent camera and GPS. Bonus points if it has a electronic compass, but that information can also be gotten from the GPX track. (electronic bread crumbs trail of where you've been).

I've been doing this for Geofort, somewhat surprised that I was the first one to do it. We've also covered a good part of the way over to Geofort from Belgium. Interrupted between Brasschaat and Breda because of a drizzle.

Only It's so Funny did the same. I had expected to see all the roads to the geofort from all directions. Now I feel bad for not having mentioned it on the Meetup group beforehand. Mapillary still seems less known than I had expected. Hence this diary entry.

Let's give Google Streetview some competition and do them one better! There are so many places their survey cars can't reach! And hopefully one day, they decide to be even nicer than they claim to be and give us permission to look at their Streetview to improve Openstreetmap. One can always be optimistic :-)

Location: 18.653, -72.268