In Flanders, all traffic signs are open data. This information is hugely useful. Well, it could be. Most of the traffic signs date from several years back and have not been updated since. The Flemish Verkeersborden.vlaanderen project intends to change that. Municipality by municipality, we see updates starting to happen. Several of them have done a complete update, or at least do occasional additions.

In a perfect world, when the municipality decides to change a traffic situation (a new speed limit, a new one-way restriction,…), they start to work in this database. First there’s a planned sign. Then when it is installed, it becomes a real sign. The real sign is offered to the OSM mapping community (and Waze, TomTom, Here, …) and they add the info to the affected street - almost in real time.

We’re not quite there just yet, but the edits in the database that do happen are still useful. Obviously because it makes for a better map. Less obviously because it saves everyone a lot of time. We often get a mail from municipalities: “hey, we have changed reality, can you now change your map”. We want to be able to say: “oh we know, we already updated it!””

So we’re building on a tool in good Road Completion tradition to make sure that if the government provides the data, we can guarantee that we’ll be up to date. This in turn might be a little incentive for more municipalities to keep their bit of the database online. Just like in Road Completion, we “accidentally review” the government data as well. When we map traffic signs, we spot errors. Often user error, sometimes logical errors. These can then help municipalities to improve their data quality or even local reality. OSM data users will be able to see how well we keep track of new traffic signs - they won’t have to trust us on our word that the data is good.

Traffic signs in Flanders: highlighted are from 2019 and more recent

In practice

We have a MapRoulette task online that is updated periodically. This task simply offers the most recent traffic signs as microtasks: the challenge is to map their effects.

How do we do that:

• we download all the traffic signs from the WFS service.
• we enrich the data with information about its name and effects. This is harvested from the JOSM plugin for traffic signs
• we keep only traffic signs that are “interesting” to us. For example, the traffic sign to say “you are on a priority road” has no effect on an OSM road. The sign “give way” does.
• we keep only traffic signs that are new since the previously processing date. N
• at the end of the process, a GeoJSON with relevant signs is put online. This is then harvested by MapRoulette
• The task is available at https://maproulette.org/browse/challenges/23548 . Please read the instructions carefully, and give it a shot. Make sure to select “nearby” not “random” when going to the next task; because you might end up mapping the effects of several nearby signs. Mapillary imagery might often be outdated with this hyper recent stuff. Please leave a comment if there is something wrong with the traffic sign data or if you have other remarks. Please refer to the wiki to learn which tags are associated with which traffic sign.

The code is available on Github. You can post issues there. A first proof of concept was built by Joost Schouppe. Ivan Diaz from TomTom rebuilt the entire workflow in Python and prepared a weekflow to ultimately run weekly updates completely automatically.

What’s next

• Activate automatic weekly updates with Github Actions.
• It might be possible to compare two dumps to find traffic signs that have been improved
• With our StreetLevel Imagery project, we create images that are uploaded to Mapillary. That platform automatically derives traffic signs. These signs are in turn detected by Osmose. This tool creates microtasks that are filtered for already mapped traffic signs. For example, it will only create a task for a max 30 sign if there is no road in OSM nearby where you can drive 30. The amount of tasks is still daunting. We intend to synchronize these tasks to MapRoulette, to “gamify” the mapping process in hopes of motivating more mappers to contribute.
• Find other source datasets to use this logic on. It is most interesting when there is an open, continously updated traffic sign database available. We did a first exploration in the Netherlands already, and it looks promising. If you know similar datasets, you’re invited to create a pull request or post an Issue.

Project key events

• 9/2021: first exploration of the dataset for mapping use.
• 11/2021: first filtering of traffic signs published on MapRoulette
• Since then the task has been updated occassionally and over 6000 new signs have been reviewed! We also passed along mistakes we saw to the managing government, who have used it to improve the source data.
• 10/2022: first contacts with TomTom about automating the analysis flow

*This Diary post is a copy from the new Project page at https://openstreetmap.be/en/projects/traffic-sign.html , for increased visibility and allowing comments.

Opening up streetlevel imagery

With OpenStreetMap Belgium and our umbrella Open Knowledge Belgium, we have been lobbying to get governments to collect their street level imagery in the open.

While there are many projects to collect images, almost allways the government does not retain ownership of the imagery. Instead of paying for data collection, they pay for access to the data. This means the price is low for the individual governement organization, but in our complicated political landscape of federal, regional, provincial, intercommunal, or communal organizations, many are buying access individually. In the end, the total cost for the tax payer is higher, without there being any open products that can benefit the rest of society.

We believe payment for the data collection should happen only once, and ownership of the imagery should be transferred to the government. This would result in the lowest cost overal. By releasing the imagery as open data, its value for society is increased even more. Access to extra services can incentivise governements to pay up for the base data collection.

Slow progress

While the intercommunal organization WVI (Dutch text) has contributed open 360° imagery for most of the industrial areas in West-Flanders, and three Flemish municipalities have shared 360° imagery taken by Vansteelandt, the vast majority of projects that we hear about do not result in open data.

This is why in 2022 we are launching the Open StreetLevel Imagery Project. We are scaling up our efforts to crowdsource open street level imagery at a low cost. We are investing a very small budget which we expect to have a significant impact.

The OpenStreetMap Belgium role

Our members have been contributing millions of openly licensed street level images in Belgium already. We do this because it helps us build the map. OpenStreetMap Belgium wants to support these contributors and increase the quality of their work. Most of the images are taken with smartphones or action cameras. But in order to convince more people that this kind of platform is the right place for them to contribute, we are investing in 360° cameras.

The images aren’t just useful to local mappers. As we publish them on the Mapillary platform, they can be used by anyone in the business of producing open data. More than that, the platform automatically generates derived data about traffic signs and infrastructure, which is in turn available as open data. As we retain full ownership of the images, we can also share them under other licenses and on other platforms. For example, we are looking into transfering the images to KartaView as well.

* Map of coverage from OSMbe members with their own gear and coverage with OSMbe gear

Crowdsourcing for the win

The crowd is more than just regular people. As the project grows, professionals start to use it, and start contributing. Open StreetLevel Imagery project contributors are also companies and governement organizations. The reasons are simple:

• It’s low cost. Since it takes no special skills to collect the imagery, you can work with volunteers or people who have to travel around anyway. Uploading is also quite easy
• It’s fast. Define a need today, grab your camera, and have the results available by the evening. Between large projects, ad-hoc work can fill the gap.
• It has unexpected benefits. You get derived data for free, can download the censored images for re-use, and your local OpenStreetMap data might get better faster.

While the low cost solution cannot offer the same benefits that advanced mobile mapping has, it can fill the gaps quite easily - possibly allowing you to leave more time between professional runs.

How can I contribute?

Thanks for asking! For our local audience:

• We’re always looking for volunteers who want to use the cameras. You don’t need to be crazy ambitious, but of course the cameras will go to people who can do most work first.
• Do you think there’s potential in your organization? We can set up a demo together and lend you a camera and accessories for testing purposes
• Do you have a use case for derived data? We can help set you up with the data and the tools.
• Would you like to see your local area mapped first? We can help you reach out to your network to find volunteers, and support them to collect the data.
• Want to see the project grow? Give us an earmarked donation and we will invest it in the project.

Internationally, we are simply looking for advice and exchange of experience. Ideally, we could just dump the images in a central repository and service providers can use them to create value. Right now we are building a simple backup solution for the images we took ourselves, so if this gets built, we can simply transfer everything we have collected.

Key events

• Early 2019. Work starts with a GoPro Hero 7 donated by Mapillary. Intensive use by contributor polyglot (many proposed cycle highways in Flanders get mapped) and Joost Schouppe
• Late 2019. As the Digitaal Vlaanderen “Mobile Mapping” project is stopped, we advocate for an open replacement.
• June 2020. The originally European open street level imagery company gets bought up by Facebook, as part of their OpenStreetMap centered maps strategy. Some context. This brings a lot of debate in the OSM community; temporary technical issues; and medium term stability.
• November 2020. OpenStreetCam becomes KartaView. Images from Mapillary can be automatically transfered to this similar platform, operated by Grab, a South-East Asian taxi company that contributes to and makes use of OpenStreetMap data.
• November 2021. The OSM Belgium membership decides to invest in 360 cameras.
• February 2022. The first camera is bought and gets heavy use in Lierde and Pepingen, by Dirk Dedoncker, a mobility consultant.
• March 2022. The camera goes to Saint-Georges-Sur-Meuse, where Pierre Serpe will use it for a project with the Gracq.
• March 2022. As the first camera is in heavy use, we receive an earmarked donation for a second camera. This camera goes to Westtoer, for experimental use in preparation of new cycle routes.

Note

This is a slightly rewritten version of the project page at OSM.be . You can read the full version there, in English, French and Dutch. OSM.be is geared towards explaining our projects to both mappers and outsiders.

The reason I’m writing this post is that my first MapComplete theme got merged into the official version. Yay! But first, an introduction.

Introduction

Having been freed from most OSMF duties, I got a bit of an OSM reboot as of late. I became more active in Belgium, where OSM seems to be booming. More people want to be involved, many more people care about what is on the map.

One of the projects that have made this a fun time, is MapComplete. I was always very enthused by projects like MapContrib - because they make it easy to create detailed thematic maps, while also being an editor for that specific theme.

I really believe thematic editors are the way forward for OpenStreetMap. Most of us here are people who care about “the map in general” (or at least a wide range of mappy topics). But most “normal” people don’t care enough about OSM data to invest in learning to use a fully fledged editor. Yet they might care more about data they really need - like whether they can store their bicycle safely, or if the power outlet is right to charge their car. Or they might be passionate about certain topics, like accessibility of paths in their neighborhood, public fruit trees, handicapped parking spots.

MapComplete is built to bring those people to OSM, instead of the plethora of POI databases that exist today. If you want to introduce MapComplete to non-OSM people, have a look at the project page on OSM.be (en, nl, fr). The project was started by PieterVDVN for a project to map the accessibility of neighborhood nature. During Open Summer of Code we expanded on it for the Brussels Region. Flemish municipalities are interested in it as well.

There’s a few OSM POI viewers around, some of them having editing functionality. Most of them thematic editors are build from scratch, I guess because they think MapContrib is too limited. MapComplete tries to offer more features, so that specialized editors can still share a platform. Some of the main features that make it stand out: - add an image to an object in just a few clicks - add reviews to any named object through an integration with Mangrove - all tags are translated to simple questions - it’s a progressive web-app, so you can install it on your smartphone home screen - it helps you become a mapper, by showing you more details and given you more rights as you start having more edits - easy interface for opening hours and orientation of objects - users can combine layers from all the themes - the theme can be stripped down to a simple webmap to use as a viewer on your website (example in the wild)

My own little theme: campersites!

It’s about one of my passions: travelling with a camper van. Put me in one, and within days I’ll start making plans to never having to return home. As a traveler in a camper, there are two things we often need: places to sleep, and places to dump waste and get provisions. There are several apps that collect info about these places, but none of them has an open data model. It always hurts to contribute to them - but it hurts just as much NOT to contribute to them either! So that’s where I started.

Because traveling isn’t the same if you haven’t got anything to contribute :)

Official places where you can spend the night with your camper are tagged with tourism=caravan_site. Places where you can dump chemical toilet waste and grey water (from washing the dishes, taking a shower) are tagged amenity=sanitary_dump_station. So those were the two layers I started with. I’ve been mapping these places for a while, so I know the data model pretty well. Of course, I had to make some choices, as some aspects of the data model are still under discussion. The code is readable enough for anyone to make suggestions for alternatives.

It’s been quite a learning curve to learn how to use the custom theme generator - Pieter’s patience with me has been impressive. The theme generator is only sufficient to create a rough draft. You have to clean the code it generates a bit before getting close to production. There’s no “real” coding involved. At some point, being able to host a JSON file somewhere is really useful though.

One of my favorite spots

Give it a spin, have a look at the other themes, add the app to your home screen, post an issue to suggest a theme, help with translations, or start building a theme from scratch. Both Pieter and me are on the new MapComplete Telegram channel, eager to help out!

Building the theme is of course only step one. I posted it to OverlandSphere, a Facebook community of overland travellers - with a pretty decent reception. If you know of other places where possible users hang out, do let me know.

Over the past years, the OpenStreetMap Foundation has worked hard to expand the membership. On the one hand, we want the core of the project to be run by people from all backgrounds. On the other hand, a larger membership makes it harder to buy your way into power.

Over the past year, MWG and Board worked on the implementation of the Active Contributor Membership (ACM). You no longer need to pay, or say you cannot pay to join. We now welcome anyone who is contributing significantly to the project by mapping or in other ways. The project was launched by the end of August 2020, and has already had a significant impact. A previous iteration of this was “the fee waiver program”. It was only available for people for whom the subscription cost of 15 pound is very high, or for whom making the payment is difficult (because of international banking issues). This program was launched around new years 2018.

Main impact

We compare the status at the end of the year 2018, with the membership right before the launch of the ACM and a few months later - mid November. Note that with the 90 day membership requirement for voting in the AGM/Board election, no snapshot exactly represents the voting population.

According to my count, membership rose from 1503 to 1939 in the period since the introduction of ACM. That’s over 400 new members or a growth of almost 30%. In this short time, the membership has grown more than in the almost two years before.

A lot of the motivation behind ACM is increasing geographic diversity. There has in fact been a significant shift. Where Africa was virtually unrepresented back in 2018, it is now at 5,2% of the total membership. Asia shows a steady growth. South America did not profit much from the fee waiver program, but has seen a significant rise recently.

Interestingly, Europe maintained a steady percentage in the first period, and a significant growth in the second. North America on the other hand, has shown a consistent decrease in the total. Note that North America’s decrease is entirely a US phenomenon. The share of non-US North American members stays almost exactly the same over the whole period.

It is quite obvious that the membership is very much concentrated in a relatively small part of the world.

The Active Contributor program comes with a simple rule: if you have mapped 42 days in the last year, you are (almost) automatically accepted. Now let’s take a look at the number of people who have mapped 42 days in the last year. Big shout-out to Pascal Neis, who provided the necessary data. Worldwide, there’s just 7971 user accounts that have mapped that much in the past 365 days! (this excludes accounts that are associated to paid mapping endeavours, as these contributions do not automatically make one qualify for ACM). That means that globally speaking, there’s a ratio of 24 members for every 100 of these heavy mappers. To get an idea of how well represented a country is within the OpenStreetMap Foundation, we can calculate this number for any country. If you have more than 24 OSMF members for every 100 mappers, you’re overrepresented; if you have less, you are underrepresented. This is not a perfect measure for at least two reasons:

• some countries might have a larger non-mapping community than others (more people building businesses around OSM, organizing events, writing software)
• the place people tend to map, is not necessarily the place they are from. Especially in countries with a relatively large amount of international remote mapping, that might make a significant difference.

There’s also a “political” dimension implied by using this benchmark. In a sense, it assumes “the OSMF should represent the community democratically based on community size”. But that goes somewhat against do-ocracy thinking - where OSMF at most can “level the playing field”, and power should ultimately be with the ones who actually do things. Also, it takes community size as a given - maybe it would be better for the project if the countries with fewer mappers punch above their weight in decision making, hopefully helping them to grow.

Looking at the continental statistics, it does help to understand the numbers more. We can now see that Asia is vastly underrepresented and that North America is vastly overrepresented when compared to the number of mappers. Hard to spot on the graph: Oceania is balanced, South America is seriously underrepresented. This graph also explains the North American exception: it was the only overrepresented continent (when compared to mappers).

Country comparisons

While these continental numbers allow for an easy overview, they hide a lot of nuance at the country level. Don’t worry, I’ll give you a nice table to play around with yourself. But first a bit of methodology. We’ve already said that 24 members for 100 mappers make you “average”. But that’s now. Since we grew a lot recently, a few months ago, you only needed 19 to be average! So if we want to compare two snapshots and see how a country has evolved when it comes to having few or many members, we need to compare to the average at that time.

So do that, we compare the real number of members to the “expected” number of members. So say a country has 100 mappers and 52 members. That means in the most recent snapshot, we expect 24 members. So that means there’s about 2.17 members for every expected member. Now if that country had 46 members previously, their previous number would have been 46 / 19 = 2.42 So even though this country gained 6 members, it is now not as overrepresented any more as it was before.

Using these numbers, we can make the graph below. Every country with members is a bubble. The size of the bubble shows how many potential active contributor members there are. If a country is to the right beyond the number 1 on the horizontal axis, it had more members than expected before the ACM role-out. If a country is above 1 on the vertical axis, it had more members than expected after the role-out. All countries under the diagonal line have lost some relative power, all those above have gained.

Some large countries get a label. Take for example Italy: it had only half the number of expected members before the role out, and now has an average result. Poland is still low, but has seen significant growth. Russia has seen some growth, but it remains vastly underrepresented.

Now go explore the full country list here: https://codepen.io/joostschouppe/full/oNzxxov All countries are there, here’s what you’ll find:

• Country: all countries with at least one private mapper with over 42 mapping days in the last year
• Active contributors: number of people with 42 mapping days (individual accounts only, data provided by Pascal Neis)
• OSMF members before ACM: number of members (of any type) just before the launch of Active Contributor Membership. -999 means “less than six, more than zero”
• OSMF members after ACM: number of members (of any type) when I started this article. -999 means “less than six, more than zero”
• Rate pre ACM: real number of members per active contributor, divided by global average number of members per active contributor. >1 means an “overrepresented country”, <1 is an underrepresented country. -999: underrepresented, 1-5 members; 999: overrepresented, 1-5 members
• Rate post ACM: same as above, but second measuring moment
• Members “needed”: number of members needed to reach the global average of members/mapper. Values below zero indicate the “excess” membership. -999 means there is “excess” membership, but current number 1-5. 999 means there’s extra members needed, but current number is very low. In most cases where current number is below the threshold, I have simply filled out the total number of expected members.

Call to action

We need a good 500 new members in underrepresented countries to make sure no country is vastly underrepresented anymore. Russia needs almost 100 members more. Indonesia, Japan, Poland and Ukraine need about 30 each. In most countries, we’re talking about smaller numbers.

Will you help achieve that goal? If you’re not a member, join now. If you are a member but know mappers in underrepresented countries, ask them to join. If you just want to help out: I understand that Pascal can give us a list of people with 42 mapping days, by country. If we combine that with a list of countries and a goal, we can start targeting mappers in specific countries. It’s not much work to set up a spreadsheet to track progress, nor to send out the messages. The most work is probably writing a good message and translating it to relevant languages. And of course, there is some processing required in the MWG. They are looking to automate things, but in the meantime you can offer your help!

Some time back, I wrote about my hopes and wishes for last year’s Board election. The reality has far surpassed my expectations. I’m impressed with how much has changed with the new set-up.

Even though ideologically, we remain just as diverse, this board is getting into much less conflict. Of course, we don’t agree on everything. But we seem to be much more capable to reach a common ground on things. This has made this Board, in my experience, much more productive than the previous one. What used to be an insurmountable hurdle, something small like switching from audio to video conferencing, was now quick, easy and straightforward.

Big part of that increased productivity, is that we have a new Chair (Allan Mustard), who takes the role much more as an active leadership role than the previous one. That is only possible because he happens to be recently retired, and willing and able to invest much more time in the role than any of the other Board members can.

We also have a new Treasurer - Guillaume was the only one to offer himself for the role, though he probably didn’t expect there to be no one else to volunteer. As he was starting the role, there was a crisis straight away: our bank kicked us out. The amount of time he has needed to invest in resolving this (and all the technical consequences of that) has been enormous, and beyond what can be expected of a volunteer. Thankfully, his free lancing was a little slow because of the corona crisis.

I also took on a new role myself: I volunteered for Secretary. Though most of the daily work is handled by Dorothea, it makes you responsible for the Local Chapters applications. This has been somewhat more work than I anticipated, and especially more exhausting than I expected. You need to get the volunteers of the other organisations to work on their application (knowing full well that running organisations is NOT why people generally like doing OSM). You need to find a consensus amongst the Board that their application is a good fit for the OSMF. And you need to find some sort of consensus in the community about the value of the application. This makes it prone to failure, especially as you may want to rush things a bit because the backlog of applications grows ever more. This feeling of not having the means (mostly simply “time”) for the task at hand is an immense energy drain for me. In the end, that makes it more likely that I actually start working on, say, a blog post about how the Board is functioning, rather than on the applications one should be working on! Even with the help of Rory and Dorothea, I still feel like the backlog is growing (note: in response to the first draft of this, Mikel offered help too).

Meanwhile the Microgrants project is running at full steam. I have joined all their meetings (weekly for a while) as a Board liaison. The first big hurdle was finding a way to ensure transparency, while also respecting the working conditions that the volunteers who run the project wish for. Next is making sure the Board and the community would be relatively happy with the result. All the while trying to keep track of issues to fix before the next round. Rather fun, but it does eat time!

The productiveness of this Board means more work for everyone. As another board member mentioned in response to the first draft, just keeping up with all the communications is exhausting. We have a chatroom which is much more active than it use to. Sign off for a few days and expect a backlog of a few hundred messages to catch up on. I “solve” this by simply not trying to follow all the conversations, but if there happen to be a few threads you care a lot about at the same time, than again, consider your time eaten.

We as a Board were fortunate to not have too many explosive situations on our hands. The biggest was probably the overload of the tile servers, and the risk an overheated infrastructure poses to the entire project. But mostly we’ve been able to work on the fundamentals of the organisation. How to set ourselves up for some necessary growth (the hiring consultation), how to deal with some of the recurrent conflicts (the iD and tagging consultation), how to grow into a globally representative organisation (the active contributor program), how to better engage our corporate members (monthly meetings) and local chapters (invite to the Board meetings). I think the current composition of the Board does a proper job in balancing all these interested parties.

But our openness to talk and our increased productivity is creating ever more work. All these consultations, all this input, all this policy making, all this practical implementation seems like much more work than there was last year. This also means much more things to criticise – this is to expected (and encouraged!). Both the tone and the merciless approach of some of the criticism of the Board, again, is exhausting.

We do have some fresh new volunteers on the Board, but to me, it feels like we are eating them. And though we are talking about some help for the sysadmins, we haven’t about more Board help. Dorothea is working at capacity already – and though the Board seems willing to give her more autonomy in her work for us, she would need to drop tasks in order to take this on. As it seems we as a Board and as a community are not ready for an executive director function, we will need to scale up our efforts. We might need someone to support Dorothea and we will need to find more volunteer help.

Burn-outs happen when you try and do the impossible for too long, instead of stepping back and saying: “hey, I might do this for a while, but we need a different approach to the issue”. I’m afraid we will eat through volunteers such as myself within a two year term. That would leave us with a Board where only these people can stay on to build experience: the recently retired; the self-employed who are willing to scale back their paid work; the single childless with a not too demanding job; the people who can do this as part of their paid job; the people who are magically immune to burn-out. If we want the Board to reflect the membership, IMHO, some change will be necessary.

The stakes are high. The value of the project is increasing on a daily basis. The OSMF needs to function as a stable and strong core in order to prevent being overpowered by a single player, or a forking or cloning of the project. I hope we can start fixing this sooner rather than later.

The OSMF is opening applications for the Microgrants project today. The Microgrants Committee explains what’s happening on the OpenStreetMap official blog. Read all that before you head over to the wiki to start your application!

---

The OSMF Microgrants project was set up by the OSMF Board and is run by the Microgrants Committee. If you have any questions, feel free to post them here or send them to microgrants (at) osmfoundation.org.

In these days of social isolation, at least here in Belgium, we’re still allowed to wander in our neighborhood. One way to make that a little more interesting: collect all the paths! I mentioned this on Twitter, and was asked to make a tutorial. So here goes.

The general idea: make a map showing all the paths in your neighborhood. Then go and visit them all, and visualize which ones you have visited. Over time, you’ll see what direction you should be heading to add more paths to your collection.

Collect the data

No dataset contains as many paths as OpenStreetMap. You can see them on your smartphone (e.g. on Osmand, on the main OpenStreetMap website, or on whatever style of map you like.

For this project, we want to download the data itself. Assuming we’re just downloading a little area, we can use the awesome Overpass Turbo for this.

I’ve prepared this sample query for you. Just move the map to your neighborhood, and click Run. Don’t zoom out too much, as loading all this data might crash your browser. Here’s a lighter query (in case you get a timeout; this one doesn’t exclude private roads).

Now you have a clear map of your data. But you can’t edit it here. So first, let’s download the data. Just click “Export”, then “Download as GeoJSON”.

Tell me more about this weird data set

Skip to the next chapter if you just want to make your own map :)

In OpenStreetMap, everyone (including you) can map anything they find interesting. Say you want to visit all the little chapels and shrines in your neighborhood, then you can just [query those] instead. A small sample of things we map can be found here.

In this example, I’ve selected line features that are a path (a generic path), footway (path dedicated to pedestrians), cycleway (dedicated to bicycles), bridleway (for horseriders), track (for agricultural purposes).

Create your own map

Now we want to have our own webmap to play around with. A wonderful open platform for doing that is umap. You can use this for all sorts of projects. Here, we’re aiming for the map below: just show all the paths in red, and the ones you’ve visited in blue.

While it isn’t hard to build your own umap, you can just re-use mine :) . Right click here and download this empty map file. Now, go to umap (thank you OSM France for hosting this). And click on the big “Create a map” button in the top right. You should now see an empty map. Click on the arrow pointing up on the right side of the screen. Select the file you just downloaded and click “Import”.

Voila, you now have a structure! Click on the upload button again, and choose the GeoJSON file you downloaded before. Now pay attention: there’s an option to choose the layer into which you want to upload the data. Make sure to choose “paths to visit”. Bang, you know have a map with all the paths in red!

Now’s a good time to save your map! Since we didn’t make an acocunt, a special edit link will be made. Copy and save that URL.

Edit your collection

Now to start marking paths as visited. Just click on a path, and a little pop-up with a pencil should appear. Click on the pencil. On the right, you will now get all the details.Change the layer of the path from “Paths to visit” to “Visited paths” and it will now show in blue. Just click anywhere on the map or on another path to stop editing. Again, don’t forget to save every once in a while.

But, but … your map is all wrong!

Really!? You’re lucky, you can now start another project! OpenStreetMap is only awesome because people like you fix mistakes. Go to openstreetmap.org and make an account . If you’re feeling adventurous, navigate the map to where you spotted the mistake and click Edit to start editing the database. Careful now: your edits will be applied to the real live database the very moment you save the edit! If you want to know a little more before diving in, check out this starters guide (many languages available!).

If you just want to let your friendly neighborhood mapper know that there is a mistake, but don’t feel like mapmaking yourself, you could make a Note. Lots of mappers keep an eye on these, and will try to fix them for you. Do be as specific as possible in your description of the problem - and be available for follow-up questions. On the main openstreetmap.org website, click on the text bubble to the right of the map. Zoom to the location, describe the problem and publish!

Want to learn more? Contact your local mapping community, they’ll probably be happy to help!

December is election season in the OpenStreetMap Foundation. The Board is planning the next elections already, so now seems like a good time to talk about my first year as a member of the Board.

Why being involved in OSMF matters

OpenStreetMap is a project of many hands. All around the world, people are doing amazing stuff: mapping, building community, inventing new data uses, building tools. Yet most of the visible innovation is happening on the fringes, not the core of the project. That’s normal for a community like OpenStreetMap. For example, new mappers are much more likely to convince someone to start mapping then regular ones. If only because us hard-core mappers have been stalking our friends for ages already. You won’t be getting many new open source routing planning experts on board - they’ve already seen the light. It is the people who are “on the fringes of OSM” that have the best view of where we can grow further.

And yet, OpenStreetMap isn’t run by people from the fringes, but by core volunteers. While people all over the world are developing new “osm welcoming tools”, we haven’t even started talking about integrating such a tool into osm.org. While vector tiles are now the industry standard, we don’t even have a roadmap on how to get there. Time and again, we see people ringing alarm bells about the lack of progress in the core. I believe part of the reason why this is happening, is that from the inside out, all looks like it’s going swell.

But I’m not sure things are swell. OpenStreetMap is not as revolutionary as it used to be. It is relatively easy to copy the idea - and improve it. I’m surprised nobody seems worried by how easy it’s been for Waze to build a community of mappers that is possibly larger than ours, in a fraction of the time. If some money and a decent app is all it takes to build a road network dataset that is as usable as OpenStreetMap roads, then that worries me. This is just one example, but I’m afraid there’s a pattern there. Our best counter-measure is getting back to the forefront. We need to be revolutionary again. We need to keep being unavoidable. I don’t just want the entire world to be mapped as well as Western Europe. I want to be astounded by creative solutions for problems we in Europe didn’t even know existed.

For this to happen, I believe the heart of OpenStreetMap needs fresh young blood. That means you! Even though OpenStreetMap grows at the fringes, central infrastructure and policy is made by a relatively small group of people - less then 2000 people make up the OpenStreetMap Foundation. They vote for the Board, who are the final responsible for the Foundation. And its the foundation that literally owns the project. A much smaller number make up the working groups and sysadmins - most of the practical decisions are made there. Now while the OSMF might not be the easiest environment to get things done, the small numbers do mean this: your joining us might have quite an impact.

It’s really easy to join the OSM Foundation. And while it used to require an international payment, there is a way out of that. We’ve just posted a blog about that, with a call to action to help us translate this info to more languages. And we’re proposing to make it even easier for contributors to join the Foundation.

Why being a candidate matters

The Board of the OSMF are seven volunteers. They come from different backgrounds, but they all love OpenStreetMap in their own different ways. Since we’re such a diverse bunch, it isn’t always easy to work together. Most things only move forward if someone takes the lead, and gets some more folks enthused. That is hard work, and our time is limited. On top of that, as it goes in “politics”: you can spend 10% of your time doing what you planned to do, but you’ll need 90% to deal with events. Those events make for times when your position demands more energy than you can sustain. After such a period, often we need some time to recuperate.

Over the year I’ve been on the board, those “time consuming events” were largely from before the elections: the massive signup of members from a single company, the Crimea boundary conflict. That translates to two areas of ongoing work:

• our relationship with the big corporations. We need to be immune to hostile takeover scenarios. We don’t want to -need- corporations for survival. We want the corporations to need us. Meanwhile we still would like them to be -happy- to need us.

• our way of organizing ourselves. Both big conflicts relate to how OSMF is run in a practical sense. From an organizational perspective, we have an immense way to go. I’m not a big fan of “procedures” myself. But we’re lacking a shared definition of how we want to work together. This makes people hesitant to take action. Especially since anything you do will be scrutinized in detail by the community. Sometimes we think there is a procedure, but there really isn’t. Sometimes we follow a procedure, but it’s not documented anywhere. And we seem to hardly have thought about “project management” at all. We are losing track of outstanding issues on a daily basis.

This big picture means that when there’s a difference of opinion about something trivial like how to run our e-mail server, it gets lost. It means when there are issues that no-one is really passionate about, it gets lost.

In this context, what I believe the qualities we need on the Board are (as many of the below as possible, in random order):

• being passionate about several OSM topics. The Board needs to deal with a wide range of topics, and if one member means one topic, then it is impossible to deal with the wide range of issues we face.

• high frustration tolerance. You will have to deal with disagreement, disinterest and confusion. You won’t even get to blame the others for that, because you won’t be able to actively participate in all topics either! If you’re a perfectionist, you will suffer. If you just want to try your best, please join :)

• good win-win detection capabilities. Sometimes your proposal might be shot down on first sight, because it seems to conflict with the other’s values. I’ve found that sometimes we think we know what the others think, but closer inspection reveals a lot of common ground. You don’t need to be “good at winning arguments”, you need to be good at changing your point of view while you’re changing someone else’s too.

• having organizational skills. As described above, our processes are a mess. We need people who can help devise simple processes to make sure as many issues as possible are addressed as soon as possible. People who know where the limits of “just working harder” are, and where organizing the work differently is what’s needed.

• having a strategic view. It would be nice if we have someone who can set up vector tiles. But it’s nicer if we have someone who can help others set up osm.org vector tiles. Or even better to have someone who can work on the root causes of why we still haven’t got them.

• be willing to trade a few hours of mapping per week for a few hours of organizational work

You might notice that nothing here says you need to be mapping every day. Or that you should be a developer. Or that you need to have run another organization before. Or that you need to be a GIS-related professional. While all of that might help, I’m not convinced it’s essential. So if you recognize yourself in the list above, but not in this paragraph: I’m looking forward to work with you!

The life of a board member

The life of a Board member is one with ebbs and flows. In times of crises, you might want to make a midnight call, or feel the need to send twenty mails in a day. Some Board members seem to be doing OSM 24/7, sometimes for the board, sometimes in a dozen different roles. Others schedule a block of hours, somewhere during their week. Others have a chaotic pattern of close engagement and then go quiet for a while. While this sometimes makes it complicated, it is also just the nature of things. It will get even more complicated as we gain board members from an even wider set of time zones.

That makes it hard to put a time estimate on total workload. In an ideal world, I would spend a working day every week on OSM-organizational stuff. But that would include OpenStreetMap-Belgium work too. In practice, the time spent is much less, amounting to a few hours weekly, or less.

Apart from that, in my opinion, as much face to face time as possible is absolutely worth the investment. The international State of the Map is an opportunity not to be missed. And I hope we can have the Board face to face meeting as soon as possible after the election. That means traveling from wherever you are to wherever the meeting is. Your travel is funded by the OSMF. It is not often that you get to spend two full days on discussing the challenges of OpenStreetMap. And it is an excellent opportunity to get to know the special people who are on the Board, and Dorothea of course. While we make few concrete decisions there, in practice, this is where the yearly agenda is set. Any decision on this will have to be taken by the new board, but I will try my best to make it as easy as possible to make it happen if the new Board decides to do it.

What’s in it for you? There will be the eternal ~~gratitude~~ scrutiny of the community. And while world fame and endless power is a bit too much to ask, it is something that will add your name to the history of this project. It is something that you can actually put on your resume. Being on the Board will never be easy, so it is OK to care about these things.

Note that the Board is not the vanguard of the OpenStreetMap project. The Foundation is there to support, not control the project. The Board is not generally the place where big projects are defined or executed. Still, there is support for the Board to fill in the gaps left by the community.

If you want to have a chat, with any questions about any of this, just send me an OSM message, or contact any of the Board members. And because “why not?”, I’ve created a public chatroom where you can Ask Me Anything: https://riot.im/app/#/room/#osm-joost:matrix.org

We all know the OSMF has a small membership compared to the mapping community. Worse, it is skewed towards certain countries. You might hear people say Germans and US Americans dominate the membership.

In a perfect world, all countries have a similar participation rate to OpenStreetMap, and mappers from all countries participate in OSMF by the same degree. The first is not something that is easily changed, but the second should clearly be our ambition. In France, a discussion about this led to a recruitment campaign with the explicit goal of rebalancing. This piqued my curiosity. Oh, and it increased membership with 90 people (from a mere 42).

But numbers without context are of little use. Guillaume (user Stereo) supported the French with membership statistics, and hooked me up with a list of membership by country for all countries. Now I’d say OSMF membership would ideally reflect the mapping community - skewed slightly for countries with a bigger data-user community. Unfortunately, no thorough statistics about mappers by country is available. Fortunately, there is something close to it: “the number of daily mappers” in a country. It is not a perfect measure:

• hdyc has a de facto “estimated home location”. This would allow to take some of the random noise out of the equasion

• “active mappers” (people with at least three months with a mapping day) might be a better measure

• in some cases, it’s impossible to find the country of people who only map abroad

But it’s close enough, and it’s available. Pascal Neis, deserving his last name as usual, sent me a list of all countries with the avarage number of daily mappers over the last 12 months.

Numbers with context

The easiest useful measure in this context is the membership rate, which allows to compare countries. For the world, you get 26 OSMF members for every 100 daily mappers. This rate varies from 0 in Tanzania to 94 in the US. On a map, it looks like this:

For greater detail, I made a table: First we have the number of active mappers and the number of OSMF members.
Then we look at the membership rate. The column “expected membership” gives you the number of members you would expect if all countries were the same. The representation rate is 100 when a country is perfectly average. Germany scores 156, which means that for every 100 expected OSMF members, they actually have 156. A country like Russia is heavily underrepresented with just 29 OSMF members for every 100 we would expect.

interactive version

Since a day without learning a new thing is a lost day, I played around a bit with interactive tables and found an example on CodePen. It was surprisingly easy (even without really understanding JavaScript) to adapt it to need. So you can now play with the table yourself! You can use it to easily identify over- and underrepresented countries, including your own.

For privacy purposes, countries with less then 1 or 2 OSMF members are shown with “-1” instead of their actual values.

One more thing

I should probably have been working on my position statement for the OSMF Board election, since I am one of the candidates. But since participation, growth and diversity are some of my core interests I just couldn’t stop myself.

• Update: corrected the table screenshot

OpenStreetMap Notes make it easy to share some info about mistakes or missing data in OpenStreetMap. It’s a very open system, allowing for posting and commenting from third party apps and even without logging in.

I’m working on an exhaustive analysis of how notes are used. Here are some prelimanry findings.

Notes are often personal

While the wiki doesn’t encourage using Notes as a personal to-do list, very many Notes get closed by the original poster. In fact, of all Notes that were posted by a logged in contributor, that are at least 90 days old, and are already closed, 50% were closed by the original mapper. The vast majority of those “self-closed” Notes (96%), never saw interaction. Notable though, is that IF there was some sort of interaction, there was a reaction by the original contributor in almost 90% of the cases.

One of the things that I want to investigate, is if these people who are closing so many of their own Notes, are also up to closing other Notes. What I would like to know is if the Maps.me surge in Notes created more note-closers, and if so if they were new to notes or not.

Maps.me is huge

When Maps.me implemented Notes into their app, the impact was huge. Basically, the number of Notes doubled overnight. The graph shows the monthly opened notes. In the first months of Maps.me Notes, the added notes were about the same as all other notes together!

Since its peak, the number of Maps.me Notes has been steadily going down, while the number of other notes is going up. My method of identifying Maps.me notes is looking for “#mapsme” or “Maps.me” in the opening comment, so maybe this string isn’t included anymore in some versions?

And we can’t keep up, right!?

Of course, this surge had a negative impact on the closing rates of Notes. However, the overall statistics that we are used to seeing, paint a somewhat overly negative image.

A tiny bit of methodology. I’m interested in how we handle incoming Notes. That means you need to look at rates, not raw numbers. One way to do that, is to define a goal, and see if we’re attaining it. For example, the graph below takes the goal “Notes should be closed within 90 days”. (note: I did not take account of the re-opening of Notes here.) That of course means you have to wait till a Note is at least 90 days old before you make a decision on the stat.

The graph shows that until the Maps.me surge, we had a slightly downward trend, but usually at least 60% of Notes were closed within 90 days. The “overall” line shows that this rate went steeply down as Maps.me surged. However, this is mostly caused by the Maps.me notes having a much worse performance. The closing rates of app-free notes did go down a bit, but in most months stayed close to 60%.

Those annoying Maps.me users?

The reason for a slower closing rate might be simple: Maps.me Notes usually aren’t made by people who close their own Notes. So a hypothesis I want to check is, if you split Notes between “made by people who tend to just work on their own Notes” and “Notes made by people who just want to share some information”, is there still a difference between Maps.me and other notes?

One of the main complaints about Maps.me users is that they don’t respond to comments. Of course, there can only be something to respond to, if their Note first gets commented. For that to happen, the Note has to be non-anonymous and commented by someone else who was logged in. To my surprise, this is a relatively rare situation. Less than 4% of all existing Notes were made by logged in users and responded to by other logged in users.

In this rare situation, there is a reaction from the original contributor in about 30% of the cases. If the Note was made through osm.org or an unknown app, this rises to about 40%. However, if the Note was made with Maps.me, response rate was only about 10%.

Since Navmii makes only anonymous notes, this rate can’t be counted for that app. The recent StreetComplete notes get a 26% response rate in similar circumstances. So there is clearly something going on with Maps.me. Some further measures to increase comparability might be necessary, for example excluding Notes from heavy mappers. But before we saddle the horses, don’t forget what the actual number is we’re talking about. In 2016, if Maps.me users were exactly like the general population, there would be just 2500 more reactions from Maps.me original contributors, on a total of 123.797 Maps.me notes that year.

Other explanations

So the lack of communication might be relatively minor compared to the “abuse” of Notes by people who don’t know what they are meant for. If someone would like to do some text analysis to see if automatic classification of usefulness is possible, I could offer a file that is ready for consumption. (the Notes dump is relatively easy to parse though)

A bit harder to analyse, would be the hypothesis that the difference is because Maps.me notes tend to contain some info that is hard to ignore (like “this place is closed now”), but hard to verify (“did you really mean Place X”? If you don’t trust the contributor, and they don’t answer, you have to go check).

In other news:

What on earth happened here? From May to July 2017, anonymous comments went absolutely through the roof, and then just as quickly went down to normal again. Most seemed to have been nonsense comments. Does anyone know what happened here?

Oh, and if you’re working with the Notes Dump yourself, be mindful of this minor bug I found.

Work in progress

I’ve been dreaming of a global database of local basic statistics about OpenStreetMap for years. That turned out to be a little over-ambitious for me. Hence my retreat to local statistics for Belgium only. Analyzing Notes is a kind of proof of concept. This article is a little side to the process of publishing statistics about Notes for the world, but also all continents, countries and “regions” in the world. Since these are the tools I’m fluent in, I wrote the analysis scripts in SPSS. They are available on Github. It should be relatively straightforward to translate to other, more open languages. Publishing will be on the Swing.eu platform. Unfortunately closed source payware, but they donated a version for OSM analysis purposes.

Shape this project

I usually start of projects like this with a clear goal in mind, then lose track of that goal completely because of all the interesting side streets I find. Working towards the Swing.eu platform helps shape that a bit. But what kind of questions would you like to see answered? I intend to make it easy to play around with the numbers yourself, but maybe there’s some aspects I’m missing entirely. Let me know! Feel free to post here or as a Github Issue.

This is my position statement for the December 2017 board election.

Where I’m coming from

Since joining OpenStreetMap, I’ve found myself on a slippery slope of ever stronger engagement to the project. Not only have I been mapping at least every other day, I’ve grown into being a community organizer. At first I was mostly interested in South America, where it felt like OSM has a much larger niche to fill than in Europe. I didn’t start off as an open source and open data enthusiast, but as someone crazy about maps. As a sociologist and data analyst I was fascinated by the data and the people behind it. I liked the way OpenStreetMap could solve real problems, and enjoyed being part of those solutions. Riding on the tails of Jorieke Vyncke (current Missing Maps coordinator) and Ben Abelshausen (awesome OSM routing developer and tireless organizer), it always seemed logical that we should build the map together.

Building OpenStreetMap Belgium, we worked with crafty local mappers, while supporting like-minded people around the world. We helped the development of what tools we could make with our local community, instead of complaining about the lack of global solutions. We had beers (and tea) together, to put faces to the usernames. We worked on humanitarian mapping to build consciousness about the project and to create a network of volunteers. We worked with open data - as just another tool to improve OSM. We worked with local government and other organizations to increase our visibility - using their networks for exposure, instead of having to build one from scratch. We worked on larger events - they would have been a failure without Ben’s sense of responsibility and the network of volunteers we grew during the mapathons. We worked on our online presence, with a nice website, a single point of contact for questions about OSM in Belgium, a newsletter. Finally, we are finalizing becoming a local chapter. We felt how more and more people started taking the project seriously.

This approach has been quite successful, and I would like to apply that experience to the entire project. It means focusing on growing the number of volunteers and letting them grow in their roles. It means sharing the work as much as possible, but still make sure things get done. I think I have been instrumental in that process, and I believe I could help the OSMF realize more of its plans. That’s why I am a candidate for the Board Elections.

Priorities as a Board Member

As an OSMF Board Member, I would devote most of my energy to growing the community. For me that means supporting local volunteers - which is why I am enthousiastic and impatient about microgrant and local chapter and event support plans. There are many relatively simple things that could help community builders everywhere (here’s a collection of ideas I worked on). As a board member I would focus on finding more such ideas - and turning them into realities. That would need growing the OSMF and the Working Groups, finding more people to share the work. And it requires helping the OSM community be a more friendly place - not just a fun place to map, but also a fun group to be part of.

1. Developing local communities. We should try to offer more to local organizers. Some basic tools and functionalities, maybe a network for best practices. Local mappers should have more tools available to monitor the successes and failures in their community. We need those organizers to come to SotM, even if they can’t afford to (and offer our help actively). At SotM, full focus on local successes and failures should be obvious. A small amount of money can go a long way - we could do more to help.

2. OpenStreetMap needs as diverse people as possible. We should take the time to think about solutions, even if they seem to be impossible. * How can we reach more non-English speakers, and how do we break the dominance of those that are fluent in English? In Belgium, a third language was part of the solution to surmount the language divide between French and Dutch speakers. But even so, we noticed we were excluding those who weren’t so good at foreign languages. So we include all three languages - it’s a lot of work, but it is necessary. * Communication isn’t just about languages, but also about bridging the gap between cultures. We should be aware of the gaping holes in our understanding of each other, be it based on education, culture or gender. We should be more active in helping people learn to do this, and avoid pointless arguing, especially on media that encourage that. * Even though OSM is a deadly serious thing, we should never forget that people contribute because it’s fun. A new mapper picks it up because they enjoy fixing that first mistake. Just as important is that an advanced mapper can keep enjoying the hard work they do. It should be just as enjoyable to become more active in the community.

3. We should realize we don’t know ourselves all that well. Still we have endless discussions based on assumptions, or just our own personal experience. To have more meaningful discussions, we need more facts. That means more analysis and research on how we function as a community. There are plenty of researchers both within and outside our community who are interested, let’s talk to them. My work experience might help here, as I work in government data research, mainly turning data into into actionable info. Between the mapping and organizing, I’ve devoted a lot of time to OpenStreetMap analysis (see my profile for an overview), I intend to expand that and actively work on the relation between OSM and science.

4. We should be aware of the risk of running out of steam. Every year, most people who picked up mapping, stop mapping. That’s normal, but it also means we need to keep growing in order to survive. The same people who enjoyed mapping from a blank slate, might not be the people who enjoy fixing mistakes or adding ever smaller details. We need to keep looking for new ways to engage people, to spot the new use cases that were unrealistic dreams just a few years ago. At the OSMF Board, we should actively seek out new use cases for OSM, especially if they have the potential to grow the mapping community.

That said, I don’t think we should change all that much. We are rightly proud of being able to achieve so much with so little structure. We should not sacrifice our criticism or our open way of working in exchange for anything else. There are almost always win-win compromises possible. It’s just a matter of creativity and good will to find them. There is still an enormous potential for growing our community. Most people still haven’t heard about us. As the project matures, many use cases are only now becoming possible. Many minds are only now opening up to the option of open collaboration. We can grow, and we can do it together.

For more about how we’re building OSM Belgium, check out OSM.be

For more about me, check out my OSM profile

Peter Mooney, Frank Ostermann and I first met at a workshop about Crowdsourcing in National Mapping in Leuven. There were people from national mapping agencies from around Europe, who came to talk about their experience with working with crowdsourcing. I talked about the crowdsourcer’s perspective. It was a bit frustrating to be the only OSM-community representative, as I know that we’re defined by many points of views. With Peter and Frank the conversation soon went to the science aspect of that same relation. Professional scientists find it hard to talk to OSM, and OSM people find it hard to talk to scientists. We believe we can do better. And we want to do something about it. Rather than just start doing stuff, we want to invite you to discuss this with us. Below is our line of thinking, written by the three of us together.

Problem

This initiative is based on our observations that there is room for improvement in the interactions between the academic research and OSM communities. On the one hand, the OSM community often learns late (or never) about research results generated from academic research on OSM. For example, the OSM wiki pages on academic research are likely not to be up-to-date (with the majority of entries from the years 2010/2011, and little after 2014), but nevertheless quite cluttered, containing many non-English entries, and therefore difficult to search effectively. On the other hand, the academic research community has often little information on what are important concerns for the OSM community. As a result, very often academic research is carried out on OSM in complete isolation from the OSM community itself. There has been substantial interest from the academic research community into OSM since at least 2006/2007. This interest shows no signs of abating. One must acknowledge that the incredible success story of OSM is an intriguing source of potential research for academics.

Objective:

Our initiative has therefore two main objectives:

1. Improve communication structurally and in a sustainable way between both the OSM community and the academic research community. This includes communication about research needs within OSM, communication of research results from the academic community to the OSM community, and shared goals and interests.
2. Learn about the interests and needs of the OSM community to enable co-created research

Approach:

Our approach has two stages: First, this blog post aims to deliver some basic information on what we plan, why we want to go forward with it, and how we hope to reach our objectives. Further, it aims to gather feedback from the OSM community through comments, and invites members of the OSM community to contribute, and propose ideas for research studies. As a second stage, we envision a more structured survey that proposes research ideas based on suggestions from this blog post’s comments , e.g. through voting or multiple-choice questions, that offers some open questions to allow for free-form comments, and asks for ideas on how to keep any wiki pages on research ideas and results more up-to-date. However, we are open for suggestions for different approaches!

Outcomes:

We aim for the following outcomes, to be shared with both academic research and OSM communities:

1. A short report or evaluation of the procedure itself.
2. Publish the highest voted or most often requested research ideas on the OSM community pages.
3. Establish a mechanism that allows the update of these ideas and feedback any results, e.g. through finding champions or supporters from both the OSM and the academic research communities, or linking with discussion board on OSM research.

Some more info on why are doing this: Academics/Researchers must write papers and do research as a key component of their ‘day job’. OSM community members want to continue to make the OSM map/database even better, map new things, write OSM software, etc. There surely exists some research problems that the OSM community is interested in investigating - these research problems could also be of great interest to the Academic/Research community. This provides great potential for a collaborative platform between Academia and the OSM community to work on problems of mutual interest. Moreover it provides the potential for a new form of collaboration where the results of the research are directed back to the OSM community for discussion and debate BEFORE they are published in academic journals or conferences. We believe that this vision of co-created research between the two communities will be of interest to everyone involved.

Joost Schouppe, OSM Belgium Frank Ostermann, ITC, Faculty of Geo-Information Science and Earth Observation, University of Twente, The Netherlands Peter Mooney, Dept of Computer Science, Maynooth University, Ireland.

Note: this post was announced on the talk mailing list here

Over the passed year, the Belgian community was involved in organizing 10 mapathons. It is an incredibly easy thing to do, once you have the documentation in order. And once you realize you should do as little as possible - just find people who have a location and a recruiting network.

Some time ago, Pascal Neis wrote an article about new mappers recruited through classic channels, Maps.me and humanitarian mapping. I asked and got a changeset dump of all the people who participated in our mapathons.

Here’s some stats about that.

Overal, 1925 unique mappers participated in our mapathons, of which 328 were new mappers.

First, did we manage to turn them into returning mappers? Well… As could have been predicted by Pascal’s depressing numbers: not really. The data used was from December 2016. You can clearly see that the percentage having more than one mapping day drops as we approach December. That simply means you need to wait a bit before you can do a decent analysis.

Say we give people 3 months, then we only look at the edits from September and before. We got 23% percent of people to map more than once! 10% mapped 3 days or more. Unfortunately, that’s even slightly worse than the international average. Maybe we just worked for a more difficult audience :)

We usually tell people to map something in their own neighborhood before starting on the mission. Less than 21 of them did so. And in fact, only 4 of the 328 have more than one Belgian mapping day. As a comparison, we had 2059 people mapping for the first time in Belgium in 2016.

Even if that all sounds thoroughly depressing, it should be noted that organizing mapathons still is a great way to build a community, even if it doesn’t show in these numbers. The mapathon movement was crucial in turning mappers into organizing volunteers. Especially the two interuniversity mapathons (with 200 participants last year and over 300 this year) are momentum-building moments. For the State of the Map in Brussels, we somehow managed to recruit 20 Belgian mappers to help out. That would have been impossible without the mapathons.

Apart from that, the constant confrontation with people who don’t have any idea about OpenStreetMap, is a stark reminder that we should all keep up the missionary work.

This wiki page has a nice collection of stats on editor popularity. The data is up to date, but the graphs aren’t. I’m not a big fan of the logarithmic scale either.

So here’s one graph to tell the main story.

I focused on “the big editors” to keep the graph simple. If you want more detail, just head over to the wiki page.

You can read the graph horizontally, showing first the distribution of changesets, then number of unique contributors, then total edits. On the left, market share. On the right absolute numbers.

full size

There’s some very clear patterns there. I really like how you can connect the dots for contributors of the “default editor” at the time: first Potlatch, then Potlatch2, then iD. All three of them reached 80% market share at their peak. But iD went down in relative terms because of Maps.me. That could only happen if Maps.me editors don’t use iD much. That’s a good thing, as it show they are new mappers. And it’s a bad thing, as it shows that we haven’t (yet) succeeded in getting them more deeply involved in OSM.

To make some of that more clear, here’s three more charts. Changesets per contributor show that JOSM users are quite productive. There’s also a very clear growth path for JOSM users. Merkaartor has a similar pattern. Maps.me hardly shows, with just 4 changesets per contributor.

Some changesets are bigger than other. JOSM changesets are the biggest. Potlatch2 are somewhere in the middel, and iD changesets are quite small. The average Maps.me changeset has only 2 changes.

So what’s the overall productivity of contributors? Here JOSM is quite extreme.

full size

Note that this doesn’t say anything about quality or amount of work. For example a JOSM changeset editing thousands of objects could have been made in minutes. Someone could have surveyed a day to collect ten POIs and map them with iD.

As one of the few remaining Potlatch users, I had to make this graph too:

As Potlatch2 lost the status of default editor, the remaining users became ever more productive. That makes sense, because “low engagement” contributors won’t find the way to that editor. So the only relevant numbers are those for 2011 and 2012. And compared to that, the low numbers for iD are striking. Low numbers may mean that more people with less motivation can be pushed to make at least one edit, so you can call that a success. This is the argument to call Maps.me a editing a huge success. But it can also mean that the editor isn’t as inviting to work on more stuff than just on the thing you wanted to do. Anyway, a much deeper analysis would be necessary to draw any conclusions on that. You’d have to take account of previous mapping experience, later shifts to JOSM, and possible differences between 2011 and 2016 newbies, to name just a few controls. Also: the numbers are rising every year, even as it remains the editor for new contributors.

And then there’s the good old Potlatch 1 of course. There’s only one reason to open that ugly duckling: go to a place where you think something was deleted, press U, and you can see and recover it. It is amazing that no other editor has a similar feature that makes this so simple.

You can download the cleaned up data here (dropbox).

When OpenStreetMap started, open geodata was basically unavailable. Some governments were quicker than others to release their data. And so some places had huge imports from the start. Whether that was a good idea or not is slowly becoming irrelevant: the map is too full for big new imports anyway. Imports are ever more exercises in conflation: merging sources and using them to validate and improve existing OSM data. The good news is that it means that often the same tools for the “initial” import can be used for keeping the data up to date. Continues synchronization between datasets changes the relation between data provider and OSM.

For a government, a complete and reliable OSM becomes a more valid tool for their projects. The synchronization processes we set up, can form the basis for an extra quality assurance (QA) channel for governments. It might even convince some agencies that there is little to be won by managing some of their data on their own.

To try and capture this changing relation, I started a thread on the talk mailing list. Mikel suggested creating a Wiki page on the subject: here it is. Meanwhile, several people have improved upon it!

During the course of the research for that page, I met Tomas Straupis. I wanted to share what he told me about what they do exactly with government data, and what their relationship is with the government.

Interview with Tomas Straupis

Here’s a general idea what we’re doing in Lithuania.

Government has datasets d1, d2… dn. OSM has one big dataset O which could be split into datasets o1, o2… om. We take datasets dx and oy which could be mapped (have similar data, like placenames, roads, lakes, rivers, etc.)

Automated importing to either direction is impossible (or not wanted by both sides). Government datasets need strict accountability (sources, documents) and responsibility. OSM has different data and simply overwriting it with government data would be bad in a lot of ways.

So the way integration between OSM and government (and actually any other datasets) is done is by synchronisation - checking for differences and taking action (mostly manual) on them on both datasets. By doing a comparison both government and OSM datasets are improved. The point here is that government datasets usually use official (document) source to update data. OSM uses local knowledge to update data. None of these methods are perfect, so synchronisation/comparison helps to get most/best of both. (as a separate note: here comes OSM strength that everything is in one layer - it is much harder to have a road going through a lake or building or having a street A with address B along it. Government datasets are usually separate and controlled by different institutions, so doing such topology checks is much more difficult there)

For this to work government must open datasets and appoint a working contact point where information about problems in government dataset could be sent and there this information is ACTUALLY used and feedback given.

Do you have more info on the projects, and the software/queries you use?

All info is in Lithuanian… Maybe google translate can help with the links to Lithuanian blog site I will provide below (if not - just tell me I will write the general idea in English).

All OSM data is imported to postgresql database using osm2pgsql and that is used for comparison/synchronisation.

We’re doing two types of comparison/synchronisation: 1. POI (point data, for some types of polygons centroid could be used) 2. Road (multi-vector data)

For POI synchronisation we have an ugly but functional universal comparison mechanism. We convert external data to xml file with lat, lon and some properties (or external source provides us information in xml for example via web-service). Then we provide mapping of this external data to OSM data. So having external data, mapping and OSM data we can create reports of differences.

Try automatic translating these two entries to get a general idea: https://blog.openmap.lt/2015/03/14/lankytinu-vietu-sinchronizavimas-i/ https://blog.openmap.lt/2015/04/18/lankytinu-vietu-sinchronizavimas-ii-dalis/

To compare road data, road shapes files are loaded to postgresql using shp2pgsql and then some queries are executed to find differences. Once again general idea is in this blog which you can try to translate: https://blog.openmap.lt/2015/09/22/keliu-numeriu-ir-dangu-lyginimas/

So basically we use postgresql/postgis and php. If you have more specific questions - I’m ready to answer them or send the code, just it is a dirty code as I’m a google copy/paste “programmer”… :-)

Does the government use your input, and how? Is there something structural? Or just mailing them and hoping they care?

Lithuania is a small country, everybody knows everybody :) Now we occasionally drink beer with “government” guys working with gis data. So we know they do change the data. They also give us feedback which data sets are “more important” for them, so we can prioritise comparing those. This way both sides are happy and thankful for help.

Additionally each month we take new/updated government data and do new comparison, so we can see that data has actually been updated.

From more or less “legal” perspective. This central government agency for gis data allows submitting error reports online for registered users (registration is free and open to anybody - http://www.geoportal.lt - created according to EU directive on spatial data). And they must check and give feedback in 20 days. We (OSM) are in somewhat different level - we mail directly to responsible group. One of the reasons for that is that they physically cannot fix all errors we report in 20 days, sometimes there are too many of problems, additionally they know report comes from a “trusted” source.

As per “structure”. For point type geometry (for example place names) we currently create a google doc online, where both sides write comments and status of errors. When everything is fixed - we take new updated government data and recreate that google doc.

For roads it is per-case mailing of coordinates and notes… But there is no reason why that could not be done in more “structural” way…

Maybe important point here is that OSM data could have some “bad/incorrect” data entered by mappers with not enough experience. And we do not want to make government gis people to sort/filter out such errors. So we go through all errors ourselves and only send those, which we think are really errors. This is the main reason why we cannot simply “automatically” run queries and send result to government people. There are no “technical/IT” problems to send mismatches automatically.

About amount of work

Initial comparisons of a specific dataset usually produces a large number of differences. Some of those are due to actual differences, some are because of different ways of entering data. So initial amount of work is usually high: both for updating data as well as fine-tuning comparison rules. After that only small amount of work is anticipated, because comparison simply notifies one side about the change in another sides data.

A note from Andrius Balčiūnas, Head of IT departament at GIS-Centras

Georeferenced data is created from ortophoto, but data changes much more often (than ortophotograpy is updated, currently each 4 years in Lithuania). OSM community notices the changes much faster. Therefore collaboration with OSM and their data usage for error checking, allows us to achieve higher data quality and relevancy. As this data is later used in national registries, cadastres, information systems - OSM community helps not only to improve the specific data set, but the whole national spacial data infrastructure content quality. Important thing to note here is that such a collaboration means that even small road segment or other improvement of OSM data by a community member could later appear in official government data.

A note on the ODbL license, and dealing with it. Government can use our error reports to start their own mapping process, but they can’t just copy our features. Do you know what they do at your government services?

Two points here:

1. Government is not using/copying any features from OSM. They get reports about problems and this simply attracts their attention on specific features in their datasets. By using their own sources they fix the problem. It cannot be done in any other way, because all changes/all data in official dataset must have an approved/reliable source. OSM triggers the process, OSM does not give any data.

2. Any database consists of numerous facts (features/records). Only the whole database can be protected by law. Single facts cannot be protected. If any database is publicly accessible, anybody can look at some facts (place name, street name, hotel name etc.) in that database. Then those facts become the facts they know/have in their brain. They can use it to update/insert such data in any other database irrespective of the permissions of original database. I’m not a lawyer. This is what I’ve heard from lawyers here in Lithuania. So in practice this means I can take this and that from ANY publicly accessible database (even google), until I do not take “too much” of the database that it is not just “some facts”, but “a considerable part of the database”. The big question here is only what is “considerable part of the database”…

P.S. 2nd point makes map “easter eggs” almost pointless…

Several people have written on the subject before: when you look at something like the evolution of road network length in OSM, the shape of the curve can tell you something about how complete the network is (on the condition that there are enough local mappers).

This graph shows this evolution for the main roads in Flanders.

full size

You can clearly see that the larger roads were mapped faster than the smaller roads. (note: there is a bug in the OSM-history-importer which prevents deleted objects from being removed from a snapshot. This could explain the continued slight growth of main roads. When people improve roads, they will often delete small portions of them.)

Assuming they are all kind of complete now, you can show the evolution of length as a percentage of current length. This shows quite clearly that there are “mapping priorities”: the 60% completion mark comes much sooner for motorways then it does for tertiaries.

full size

While this all sounds quite obvious, it really isn’t if you look at the map of road evolution in Flanders. From the very beginning of mapping, contributors have been interested in small roads as well as main roads.

Full size link. Colors: black: main roads, yellow: minor roads, green: slow roads.

If we extend our view to a wider range of roads, we can see that the main roads in general got mapped first, but minor roads soon came to dominate over them. Service roads, tracks and paths (footway, path, steps, bridleway, pedestrian) tell their own story.

full size

(Note: construction and proposed roads are removed from further graphs. I checked taginfo for alternative tagging styles, but they are also quite rare)

Because these last types of roads haven’t reached their final form yet, we’ll show the yearly growth rate. As this growth was explosive in the first years, we’ll start in 2012.

full size

The graph clearly shows that main roads and minor roads aren’t really growing anymore. However, the graphs for service roads, paths and tracks seem to level off in 2014. In fact, paths and tracks go up in 2016. In turn, that means there is a lot of mapping left to do. It is surprising to me that this holds for tracks too, as they can be mapped more easily from aerial imagery only. Open data sources of paths and high resolution aerial imagery (both provided by AGIV) could explain the upshot in the mapping of paths and tracks. Other explanations might be succesful relations with the GR and Trage Wegen organisations, or increased contribution triggered by data use.

Network growth versus amount of work

One more thing I do want to share now is the amount of work that is being done. While network completeness was achieved quite fast for main roads, that does not mean that people stopped caring after it was finished. In the animated map or primaries, trunks and motorways below, gray means “existing” and black means “been worked on this month”.

full size

These edits can be anything, but here are two examples: work on naming roads and on speed limits. From the beginning of the project, most residential roads were mapped with a name. Length of unnamed residentials started decreasing as soon as 2012. It will likely never reach zero, as many small bits and pieces are hard to assign to any one street. Also, there are in fact roads that do not have a name.

full size

For speed limits, the proportion that has a limit is much lower. Total length of untagged roads only started decreasing in 2014. This tagging is probably slower because it isn’t as important for routing and is sometimes seen as a consequence of road classification and location.

full size

Measuring edits

These graphs compare the added length for main road types (right) and the number of edits by road type (left). It is quite clear that mapping new roads peaked as early as 2008, but the amount of work done on these roads has in fact only gone up until 2014.

full size

(Note: here, the number of edits is the sum of the number of days a certain way has been edited. The category in which it shows is the last main tag for that day.)

These two graphs show the type of changes for primary and tertiary roads. Traditionally, geometry changes are the most important. As time goes by, their importance starts to lower, and editing tags becomes more important.

full size

In a more general sense, this holds true too. The amount of edits peaks much later than the adding of new roads. In fact, for most road types, it doesn’t seem to go down at all.

full size

What’s next?

As usual, I’m torn between answering more and more questions with the data, or scaling it up to more areas. Luckily, for your basic statistics needs, more and more options are finally popping up. See the road statistics provided by Mapbox, Steve Coast or the Missing Maps.

In the case of road network completeness, some efforts have been made to compare current OSM length to CIA stats to measure map completeness. This is problematic, because even if governments have decent stats, they are by their own local definition. Hence the comparison might be off. In the case of Flanders, we have a single, very good source for road lengths. One of the things I want to do next, is to compare local lengths in OSM and official data. This could show is where OSM is probably not finished yet. But you can also calculate this based on the shape of the curves we’ve seen before. If both approaches give similar results, that would clearly imply that you do not need external datasources to evaluate OSM data completeness.

Another thing is that we have noticed many new mappers first starting to map local paths. I’d like to see if this is a real evolution.

By focusing on road length, you measure both network completeness and level of detail. But neither very well. From a perspective of network completeness, you would have to discount things like cycleways that are mapped as separate ways, or only count dual carriageways once. An analysis detecting really new geometries would do that. I’m planning to do something like that “soon”. On the other hand, from a perspective of level of detail road length lacks subtlety. Take the example of cycleway networks. You would have to count all highway=cycleway, but also all the roads that have cycleway tags as part of the cycle network too.

But I told myself not to write articles that are too long to read in one go :) I might have failed.

Bonus: more animated maps

Because they are fun to make and to watch, here are some more animated maps.

Showing all major roads

Overlaying OSM on top of official road data (Wegenregister), to show where the map is complete

Focusing on “slow roads” (in green)

All data in this article copyright OpenStreetMap contributors, free to reproduce anywhere if source included. Download processed data here.

In my quest to understand the growth of OSM, I had a little fun today.

I took the 1/1/2017 full history dump for Brussels and I extracted a shapefile with all the versions of all the highway=* that ever existed.

Then I wanted to visualize it to see if there was a pattern in how the roads get mapped: “first real roads, then paths” or “everything all the time”. So I styled the paths clear green, the roads thin black and used a gray background for the current highways. Then I rendered a slide for every month.

link

It looks really cool, because it doesn’t just show the chaos of our growth. As the black roads are drawn slightly transparent and the monthly slide shows every version of the road in that month, “active areas” show up in heavy black. I think it’s really pretty.

On the occasion that it was a featured image in the Weekly OSM, I made a new version without a gray background and with a more logical image size.

link

You can download the individual images here and the shapefile (ugh, a shapefile) here

Community power

While building the program for State of the Map, the program committee had to say no to several people who wanted to talk about their local community – their successes and their challenges. As a kind of compensation, we added a local communities panel (video) and a local chapters congress to the program.

But during the preparation, I also got a lot of feedback from people who couldn’t make it to State of the Map: money, accidents, visa. I got feedback from Brian Pangle (UK), Felix Delattre (Nicarague), Clifford Snow (US/Seattle), Marco Antonio Frias (Bolivia), Redon Skikuli (Albania), Mohamet Lamine Ndiaye (Senegal), Yantisa Akhadi (Indonesia) and Michal Palenik (Slovakia). Most of them didn’t have a chance to be on the panel, or even make it all.

Some of their ideas did make it to the Local Chapters Congress, and helped put things in motion. For example, finally we have the option to follow comments on Diary posts! And there’s talk of putting some money into OSM.org website development for things like massive local messaging, which was a recurring theme there. That might involve helping Gravitystorm’s project to simplify the OSM.org codebase, as that would make contributing code that much easier. Also the idea to allow OSMF membership without payment was mentioned, which was an obvious frustration during the Local Chapters Congress.

What is important to me, is that it goes to show that focused community action can shift the focus of our dev team to issues that would otherwise be lower on their priorities list. I hope we can repeat efforts like this at the next SotM, hopefully even stronger.

This post does two things. First, it will give you, the local community builder, a lot of ideas about things you could do to work on a tighter and larger community. Second, it tries to set an agenda. It offers you several ideas which you could adapt, promote or realize.

Content

There are three subjects:

• What are our main dilemmas when organizing our communities

• What kind of tools do we need to build community

• What stuff are we doing now, that actually works

It was entirely built around the answers from the people mentioned above, plus our own experience here in Belgium.

Community builders’ dilemmas

Relatively little feedback on this, looks like we’re a confident bunch. But their are some interesting points.

• The challenge of mobilizing mappers: too soft vs too hard. We’re all volunteers, and if you push too hard, you’ll push people away. But if you don’t take action and keep it up, you’ll never get beyond three people at your activities.

• Building a local community means making decisions. Is it acceptable to offer financial rewards? Do we focus on finding the “mapping nerds” who create huge amounts of data? Or do we need to adapt to less obvious groups - people who often can’t even read a map, but have excellent local knowledge?

• Being local means embracing local culture. But we also want OSM to have a unified voice and a unified data model. And what do we do with well-intentioned outside help, who bring their own funding but also their own ideas and priorities?

Where the global community can help

In the answers, local communication needs were a top priority. The mailing lists, forums and IRC are good for reaching hard core mappers. But the large majority of contributors aren’t there. So how do you reach the local mapper who isn’t active anywhere on these channels?

We need an easy way to contact local mappers

When you want to organize a local activity, you need external tools like Pascal’s mappers around me. Or you could query Overpass and make a little list of who has been working on that area. Just collecting the info takes a long time, and then you have to send messages one by one. It is impossible to send a message to all your OSM contacts if you just have their username. Allowing otherwise is obviously not without risk, so some anti-spam measures have to be implemented from the start.

We need to connect the new mappers

It is very labour intensive to connect new mappers to their local communities. Several people running a program to send a message to every new mapper in their region have given up, even as this cool little website makes the work a bit easier. In Belgium, we use welcome.osm.be . It is a simple user interface which takes the New Mappers feed from Pascal Neis and makes it easy to send people a standard welcome message. One is defined as “Belgian” based on the location of their first changeset, which is good enough as a proxy for home region.

The message itself focuses on our communication channels, apart from giving some basic mapping tips. The advantage of using a tool is that you can share the workload, and can see who has been welcomed already. Of course, looking at changesets and giving some pointers is very useful – but a lot of work. It also thanks you for your contribution, and gives you someone to contact in case of doubt. It gives a human face to the map. This is something that could be entirely automated within the OSM.org ecosystem – a centralized system with the content provided by the local communities. This would not be an alternative to the Welcome Message you get on subscription, but a complementary message on first edit. Otherwise, it wouldn’t be possible to guess everyone’s location.

We need a lively community diary stream

Several of us commented on the impossibility of subscribing to comments on Diary posts, which leads to discussion rapidly dying down. This has now been implemented! Over a year ago, after some rather discouraging help, I opened a ticket on github to request this feature. Markus Heidelberg did make a Chrome/Firefox plugin to fix the same problem. It confused me a bit that someone would make an external tool, rather than fix the problem itself. Markus was kind enough to explain that it’s much more simple to write a separate bit of code than to integrate something into our osm.org website. Another argument for everyone to help modernize that codebase. But that won’t fix everything, because people do speak many different programming languages.

Anyway, the ticket remained open for almost a year, and it was only when the idea got wider support during SotM that we got the attention of our programmers. The pull request shows that even a “simple” feature like this is absolutely not straightforward to integrate. It looks like it took quite a bit of effort from Mikel, Ilya, Andy and Tom to do this. Thank you guys!

Still, we could do more to make communications easier. For example, you still need to be a bit of a nerd to find a way to follow the official blog. A subscribe button, anyone? But even to find this blog is a challenge. I find it strange that there are no direct links from the osm.org landing page to subdomains like help, forum., irc. and blog.osm.org .

We need to help new mappers gain experience

Becoming a mapper is not easy. When you often explain OSM to new mappers, you start to realize how many little things you’ve learned over the years. The more developed the map, the harder it will become. Attention for documentation, and making help easier to find will become ever more important. But a human touch might help too.

Godfather program A recurrent idea to help new mappers is to start a kind of “godfather” program. It might be as simple as sending a welcome message to new mappers, personalized with some tips about better mapping of what they added. But you could go further, and coach people as they grow. You would need some reward for that, because it would reduce your own mapping time. So imagine a HDYC not of your own mapping, but of the people you helped.

#reviewmychange OSM is easy for very confident people: you have to believe that little old me is capable of improving this big map made by so many people. At humanitarian mapathons, it is often a relief to people that their work will be reviewed. But why not add a simple feature to the iD editor to mark your own work as “please review”. It could be as simple as adding a hashtag #pleasereview to the changeset comment, and making a little tool that collects and geocodes these changesets into a simple website for follow-up.

A toolbox for local communities

This is a broad concept, but here are some examples of what that could mean:

• A little money can go a long way. In the US, it can help you set up a a local Meetup group. In Africa or Latin America, a microgrant would be enough to pay for internet access, a mapping device and transport costs. If we’re capable of getting free pizza for our mapathons, we should be able to do this too.

• A local web presence is something several people commented to as being very useful. Could we have a local community website starterkit, similar in ease to set up to a Maptime chapter?

• Could we build communication and tracking tools (new mappers, QA, stats) built on admin boundaries instead of bounding boxes?

  # Things that work

A central theme on the answers about things that work, is that none of them are easy. It takes time, it takes effort, and the impact can often be quite disappointing.

Some long-time mappers even believe that we’ve reached our potential: everyone who is interested in OpenStreetMap knows the project by now, so there is little to be won by reaching out. This is typical for a swarm organisation: it’s only those who are at the edges of the swarm that see the growth. It is the networks of the newer people that will help you grow – not your own.

All the more reason to learn about things that have worked for others. This chapter talks about how to grow your community, but also about community consolidation. You might have a lot of people working on the map, but who have never done anything but add info to the map. Minimal community engagement is necessary: how else will they keep their mapping habits in line with the wider community? And of course, they are the first place to look when you want to do stuff to grow your community.

The basics

When it comes to engaging existing mappers, there is no alternative for real life meetings. Even though we’re an online community, it is personal contacts that build ties. And these are the ties you need to turn mappers into organisers.

A good place to start, is by watching changesets and commenting on them. It’s one of the few ways of getting to know the people who add data but aren’t active anywhere else.

Adapting to different communication styles is essential. If you’re only using mailing lists, don’t be surprised that the level of engagement stays flat. Take the Bolivian talk e-mail list that had about two active members for years. Then Bolivia started a Telegram supergroup and suddenly there’s 40 members, of which at least a dozen are quite active. Here in Belgium we adopted Slack during the State of the Map, and it’s still quite active for more informal communication and quick questions.

But of course, having many channels makes things complicated. Especially if what works in one country doesn’t in the next. it will be a lot of work to find the right channel and to get people in the channel that’s best for them. An adapted welcome message makes it easier to integrate new mappers.

Where the local map is already relatively complete, there is little enthusiasm for mapping parties. The quaint model of going out collecting data and then mapping over a beer attracts much less people than other activities. But in places where the map is still quite basic, it can be very successful in building engagement and getting attention.

Doing exiting stuff, as Felix Delattre puts it, is effective to find new people. By doing something completely new and unheard of, you can create a lot of excitement about OpenStreetMap. In Nicaragua, being the first to create an online and paper map with all the bus routes in the capital can do that for you. The exposure this gives you, has an effect beyond the original mapping community that made the project possible in the first place.

Lacking big projects like this, showing real life use cases is an obvious way to connect to your audience once you get their attention. If you know your public, focus on what you know they could use. If you don’t, show the diversity of cool stuff you can do with OSM.

You need a way out of your inner circle. Engage outside organisations. You are basically tapping into existing networks, rather than building one from scratch. For example, connecting with “data science” people, but also local government, entrepreneurs, IT people. Working together with Trage Wegen has introduced many new mappers to OSM over the last two years in Belgium. This is an organisation focused on the threatened little paths and tracks that connects our messy towns and villages to the sparse open space. The people who support them are passionate about this subject, and it’s not that hard to take their passion for “slow roads” and turn it into a mapping passion, since a mapped path is harder to disappear.

Meetup

Especially in developed countries, Meetup seems to be a useful tool for creating events. Clifford Snow did an entire session on the subject (video). These events can be as small as a bar hangout, but it can also be used for much larger events. It is quite easy to start a group. As an organizer you have an idea how many people to expect, and Meetup does all the hard communication work for you (maintaining contact list, sending out reminders, thanking for showing up).

Meetup is very local: it will suggest groups to hang out with based on both your location and your other Meetup groups. So you will get a lot of subscriptions from people already active on Meetup, but not yet very interested in OSM. And you will almost automatically find meetup groups which have similar interests, where you might go and talk about OSM.

There are some challenges though. Meetup realizes the value of their network, and so you need to pay to be an organization on their website. Prices depend on the country (3 €/month in Belgium, 15 $ in the US). In practice, this is paid by the very motivated organizers themselves. As there is no free alternative, it might be an idea for central OSM organisations to provide this money instead. The impact is clear, and the investment is minimal. I would dare say that without Meetup, there would probably not have been a State of the Map in Belgium this year.

Humanitarian Mapathons

Both Belgium and Seattle talked about using Humanitarian Mapping as a recruitment tool. It helps attract people who would otherwise not be interested in OpenStreetMap, and gives you a chance to introduce them to the wider project too. It’s also a place to turn your hardcore mappers into volunteers. There are well defined tasks to do, like organizing, promoting, giving talks, making documentation, validating data or helping out individual mappers. That makes it easy to become a volunteer. The repetition of events gives them the opportunity to grow into ever more complex tasks.

Imports!

This will sound controversial to a lot of people, but imports can be a recruiting tool too. Clifford and Jeff Meyer talk about how they used an import to grow their community here. Imports aren’t easy, and having an ‘import party’ is usually a bad idea. But good imports are possible, and they provide an opportunity to recruit more technically oriented people who would balk at the idea of tracing thousands of buildings.

So, what else?

What dilemmas do you want to talk about? What do you think about the proposed needed tools? What worked for you or your local community? How can we make the life of new community builders easier?

And most of all, how do we keep the momentum we seemed to have during and after SotM 2016?  

Recently, I wrote about how you could use government road data to improve OpenStreetMap. Here’s a move in the other direction.

As an employee of the city of Antwerp, I was involved in the recent ‘validation’ of the Road Registry (Wegenregister) for our city. This registry is managed by the central Flemish government, but final responsibility for the content is with the municipality. Validation means the central government gives us a new dump for us to check for errors. This way of working is only a temporary situation: in the future, we will be live editing in the central database itself.

Some background

There’s an amazing amount of cleanup left to do, but we decided to focus on the completeness of the main road network. Before, we did this by comparing with our own city registry of roads. But that is not being updated anymore. So for the first time, we used OpenStreetMap for the validation. Using FME, we identified roads which exist in OSM, but not in the Road Registry. We excluded service roads and “slow roads” (paths, tracks, cycleways), as these are less of a priority right now.

Next time, we will also look at roads that are in the Road Registry, but not OSM. In some case, the lack of road in OSM is really an indication of an error in the Registry. For example when a road has been closed, and the government somehow missed that. This is more work, because the Road Registry contains a lot of little bits of “roads” that are really just driveways. Because nobody cares about them, they aren’t in OSM. But they are quite hard to filter out from the Registry data.

The results

The cleaned up dataset of roads that are in OSM and not in the Registry was really quite limited. Only 138 situations needed manual review. Of those cases, 32 were a simple matter of slightly different geometry. For example when OSM mapped the road as a polygon, which we didn’t really take into account. We identified 33 cases where the Road Registry was clearly wrong. Then there were 31 cases that looked like they shouldn’t have been in the selection anyway: they are private driveways, parking aisles, tramways. About half of those needed a fix in OSM. But the “tramways” were actually dedicated bus roads on top of tramways.

Most of the “mistakes” detected in OSM were caused by larger geometry issues. Sometimes the centerline of a road is debatable, but in most of these cases OSM could be improved, sometimes vastly. These were most often roads that hadn’t been touched in years. Only in a couple of cases was OSM really vastly wrong. This happened when the city reorganized streets, and somehow, nobody noticed. Most striking was the Troonplaats, which is a quite popular square. In several cases, OSM had already been corrected in the month or two between data download and final analysis (though to be honest, some of those were fixes of mine). A few mistakes were caused by errors in or outdated road classification.

There was one striking case (pictured above), where we were convinced OSM was wrong, but we apparently missed a big change in the road geometry. Fortunatly there was a [Mapillary sequence], of course one of the 1.1 million pictures uploaded by filipc. Even though the aerial photography in Flanders is excellent and recent, the only place this road shows up is on the OSM map.

Legal stuff (edit)

As Stereo pointed out in the comments, OSM cannot be copied by a non-ODbL source. I always translated the license of OSM as “if you merge your private data with OSM data, you have to open up your data”. But that’s not correct, it should be: “if you merge your data with OSM data, you have to open up your data AND prohibit anyone from ever making it private again”. In this case, the Flemish government allows (and explicitly wants) TomTom and Google to take official data and use it to improve their private data.

Because of that, us government workers are not allowed to copy features from OSM. But there is a precedent: the New York City government uses OSM to track changes to their buildings as imported into OSM. I’ll trust their research that ODbL does not exclude using OSM to detect errors, if you then proceed to do your own surveying before making changes to your own dataset. This is also what the License Working Group believes, as Simon Poole (thanks!) pointed out in the comments. I understand this bit of text was supposed to have landed in the Legal FAQ page, so I went ahead and did that. Please revert if this is inappropriate.

The ODbL always made sense to me, and it kind of still does. Say I was to download all of OSM to my own server, and redistribute it under a more open license. Then someone else could just take that data and close it off. But this case does help me understand those who aren’t very happy about this license a bit more. In the case of government, it means you can’t -really- integrate OSM into your processes. For example, you couldn’t take OSM, validate it with your own data and redistribute the result under the license of your choice.

Have a look

You can have a look at the cases here. There’s a bit of work left on the cases with a difference in geometry. The easiest way to get the Road Registry into your editor is with this (slightly outdated) WMTS:

https://api.mapbox.com/styles/v1/joostschouppe/cir6gwq2p0016cjlyx6e1b1cc/tiles/256/{z}/{x}/{y}?access_token=pk.eyJ1Ijoiam9vc3RzY2hvdXBwZSIsImEiOiJjaWh2djF1c2owMmJrdDNtMWV2c2Rld3QwIn0.9zXJJWZ4rOcspyFIdEC3Rw

You can contact me to get the FME models we used to identify these roads - they aren’t very complicated. You could easily do similar things in open source software.

TL;DR: Government road data, processed to help you map roads in Flanders, Belgium. All the tiled layers are available for use in your favorite editing software.

About the data

The Flemish government has a large project to measure most stuff you find in the public domain, the GRB (Dutch). The data is measured to incredible accuracy, but the project is not focused on maximum recency. Update frequency is once or twice a year. When it comes to roads, only those that need an official streetname are included.

That’s a bit limited for some purposes, so they started the Wegenregister (Registry of Roads). The idea is that all roads are included, also “slow roads” (paths and tracks), private roads and even future roads. They started of with the centerlines of roads from the GRB and enriched it with National Geographic Institute (NGI) data for smaller roads. It isn’t quite finished yet: a lot of local governments must still validate the data, and there is no automatic procedure in place to feed new GRB roads to the database. So you can expect some of the “future roads” to be quite present. The NGI data is also of varying quality: it is quite complete and has generally good geometry, but it can be quite outdated.

The scope of the Wegenregister is to offer a complete road network, not navigable data. It does not include anything like access restrictions, detailed lane info or max speeds. It does contain road surface information. It is divided into segments, which go from one junction to the next. Only if a new road is added, an existing segment will be split. That means segment ID’s are relatively stable. If a segment has a change of attribute somewhere, this is dealt with by dynamic segmentation. Basically, that means you have a table saying stuff like “from meter 0 to 100 asphalt, from meter 100 to end concrete”.

Finding missing roads

I did some quick visual checks in my own mapping neighbourhood, and I did find a LOT of missing roads. Some forest paths, several small alleys connecting backyards to the street, some graveyard paths, some driveways. I would say 95% of the missing paths/roads still existed, about 75% worth mapping in OSM.

Enough to warrant some closer inspection.

It is open data with an OSM compatible licence, which you can download through a website. First I tried FME, as we have processes in this software at my dayjob to do similar analysis that I could reuse. Alas, it didn’t scale well for larger data. QGIS, after some trial and error, did the job no problem. The main processing operations took about 36 hours on my not-fancy-at-all laptop.

First I took the OSM road data (as a shapefile, from Geofabrik), saved it in our local projection and buffered it by 7 meters. Then I used difference to find the parts of the Wegenregister that were outside of that buffer. Next I threw out segments of under 10 meters (unless they were entirely outside of the buffer). I also calculated the percentage outside of the buffer. The result are A LOT of segments (220.000 out of one million) , which are either missing in OSM or have a very different geometry.

Sharing the results

The result is still a shapefile of over 60 megabyte, so nothing you can just put on umap. Luckily, it is quite easy to make a TMS service from a shapefile using Mapbox Studio. These services can be used in a little leaflet map like this one, but can also be added in iD or JOSM.

Make sure you open the layers (button top right): you can use three background maps, see the whole Wegenregister, add Strava and see the OSM road network more clearly overlayed.

Mind you, I DO NOT want you to just get out your editor and start copying these features. There are several reasons why a road might be missing in OSM, some good, some bad:

• private roads in a forest: can’t see them, can’t survey them
• it doesn’t exist anymore
• it isn’t a road (but for example a dedicated tramway)
• it doesn’t exist yet (indicated in yellow in the complete dataset)
• the OSM road is newer than the extract I used
• geometry is so different that segments don’t match

[EDIT: thanks to tyr_asd you can now copy the URL to share your current view :)]

But you don’t need to go out surveying for every single change either. In the map I provided, you can combine the view of missing Wegenregister roads with aerial photgraphy, OSM gpx and Strava gpx layers. If they all point in the same direction, you can be quite sure that OSM is wrong and Wegenregister is right.

URLs for mapping

These URLs can be added in JOSM, iD and OsmAND. In iD, click the layers button (righthandside of the screen), then click on the magnifying glass next to Custom or ‘Aangepast’ to insert one of the URLs. To use this in Osmand, check my previous diary entry on Strava (only works for layers containg .png). If you use JOSM, you know things like this :)

Complete Wegenregister: > https://api.mapbox.com/styles/v1/joostschouppe/cir6gwq2p0016cjlyx6e1b1cc/tiles/256/{z}/{x}/{y}?access_token=pk.eyJ1Ijoiam9vc3RzY2hvdXBwZSIsImEiOiJjaWh2djF1c2owMmJrdDNtMWV2c2Rld3QwIn0.9zXJJWZ4rOcspyFIdEC3Rw

Wegenregister, missing roads only: > https://api.mapbox.com/styles/v1/joostschouppe/cirqcpmll003hh0ncb2wuv882/tiles/256/{z}/{x}/{y}?access_token=pk.eyJ1Ijoiam9vc3RzY2hvdXBwZSIsImEiOiJjaWh2djF1c2owMmJrdDNtMWV2c2Rld3QwIn0.9zXJJWZ4rOcspyFIdEC3Rw

Strava, all data: > http://globalheat.strava.com/tiles/both/color2/{z}/{x}/{y}.png

Strava, recent data only (seems to be hard to re-use) > http://d-yearheat.strava.com/tiles/both/color2/{z}/{x}/{y}.png?y=2015&v=6

Downloadables

You can download the entire dataset from the AGIV website (note: this link dies when they publish a new version. Just look for “wegenregister” in their catalog). And here is the entire dataset of missing Wegenregister roads as a shapefile. Use QGIS to extract your local area of interest. Save as GPX to add it to Osmand and go out mapping. Of course, you already have the Strava layer enabled in Osmand :)

I can also provide just the bits of Wegenregister that are outside of the buffer, just ask.

Better mapping practices

Now imagine you’ve checked your whole mapping neighborhood. The map will stay red, at least till the next update of the process. But what about the roads that you surveyed and concluded were invalid Wegenregister roads. They should be removed too. I’m not quite sure how to go about that.

• We could tell the government. And they might actually listen, but by the time the road is removed from the dataset, three more mappers might have analysed the same segment.
• We could build a list of “untrue” Wegenregister roads and remove these from analysis. There are quite stable unique identifiers available, but it would mean everybody should refer to the same list when marking something in Wegenregister as untrue.
• We could map non-existing roads in OSM (ooh, taboo!), analogous to the not:name tag that was used in the UK to mark that the official name for a road was wrong. I was tempted into something similar in this case, where a path is indefinitely closed off, but still quite existent (as seen from the street and aerial photography)

Seizing an opportunity

I know the Belgian heavy mappers like to work on stuff, but I think this might be a nice opportunity for expanding the community a little more. I’ve noticed how small paths and local trails are really something that can still attract new mappers. The Flemish Trage Wegen organisation is behind that for a large part, and I sense we could work together with them on a project like this. It is also very similar to the local “inventarisations” they do.

It is a very well defined task, it is repeatable, all the tools and pitfalls can be explained quite easily. Moreover, local governments could be contacted with a very clear proposal - to help them solve a problem they would have to solve themselves pretty soon anyway.

I see two main options, which are possible conflicting.

• Option one: a maproulette challenge or Canadian style crowdsourcing tool. It’s nice and easy, but it might be a little too simplistic for this task. The Canadian style tool would probably allow to generate a vast error report for the Flemish government, which is quite cool. Microtasking like this is not compatible with the extensive local surveying which we need when the reality isn’t very clear though. But it might make the job a little lighter for those working on Option Two.

• Option two: we set up a Belgian tasking manager (as in an instance of tasks.hotosm.org) and divide the job. It allows for very specific instructions, providing the analysed Wegenregister as imagery to people who have never used iD before and makes it really easy to track progress. Time-out for the tile you picked should probably changed from two hours to a couple of days though :)

One thing I’ve learned from working on Missing Maps, is that you need to use an existing network to recruit new mappers. You need an easy, repeatable task to make the work easier on OSM supporting volunteers. And you have an opportunity to take their passion (in this case “helping poor people”) and try to channel it into a passion for OpenStreetMap. Change MSF for local government, mapping buildings with mapping roads, and a passion for doing good with a passion for local paths, and there you are.

Working on it

To make such a project possible, we should probably set up an online service doing something similar to my analysis. So newly mapped roads in OSM are removed from the “to map” list, as well as invalidated Wegenregister roads.

My analysis is more a proof of concept than anything else. It would be interesting to go further. For example, one could make a map with just roads that have a different name in OSM than the official name. Or just focus on the planned roads. Or suggest surfacing information for inclusion.

It would of course be nice if it were easy to take the Wegenregister geometry and apply it to the OSM data, but that might be a little too much of a challenge right now.

If you feel like working on such a project, get in touch, start on your own, or come to the SOTM Hackathon in Brussels.