OpenStreetMap Notes make it easy to share some info about mistakes or missing data in OpenStreetMap. It’s a very open system, allowing for posting and commenting from third party apps and even without logging in.

I’m working on an exhaustive analysis of how notes are used. Here are some prelimanry findings.

Notes are often personal

While the wiki doesn’t encourage using Notes as a personal to-do list, very many Notes get closed by the original poster. In fact, of all Notes that were posted by a logged in contributor, that are at least 90 days old, and are already closed, 50% were closed by the original mapper. The vast majority of those “self-closed” Notes (96%), never saw interaction. Notable though, is that IF there was some sort of interaction, there was a reaction by the original contributor in almost 90% of the cases.

One of the things that I want to investigate, is if these people who are closing so many of their own Notes, are also up to closing other Notes. What I would like to know is if the Maps.me surge in Notes created more note-closers, and if so if they were new to notes or not.

Maps.me is huge

When Maps.me implemented Notes into their app, the impact was huge. Basically, the number of Notes doubled overnight. The graph shows the monthly opened notes. In the first months of Maps.me Notes, the added notes were about the same as all other notes together!

Since its peak, the number of Maps.me Notes has been steadily going down, while the number of other notes is going up. My method of identifying Maps.me notes is looking for “#mapsme” or “Maps.me” in the opening comment, so maybe this string isn’t included anymore in some versions?

And we can’t keep up, right!?

Of course, this surge had a negative impact on the closing rates of Notes. However, the overall statistics that we are used to seeing, paint a somewhat overly negative image.

A tiny bit of methodology. I’m interested in how we handle incoming Notes. That means you need to look at rates, not raw numbers. One way to do that, is to define a goal, and see if we’re attaining it. For example, the graph below takes the goal “Notes should be closed within 90 days”. (note: I did not take account of the re-opening of Notes here.) That of course means you have to wait till a Note is at least 90 days old before you make a decision on the stat.

The graph shows that until the Maps.me surge, we had a slightly downward trend, but usually at least 60% of Notes were closed within 90 days. The “overall” line shows that this rate went steeply down as Maps.me surged. However, this is mostly caused by the Maps.me notes having a much worse performance. The closing rates of app-free notes did go down a bit, but in most months stayed close to 60%.

Those annoying Maps.me users?

The reason for a slower closing rate might be simple: Maps.me Notes usually aren’t made by people who close their own Notes. So a hypothesis I want to check is, if you split Notes between “made by people who tend to just work on their own Notes” and “Notes made by people who just want to share some information”, is there still a difference between Maps.me and other notes?

One of the main complaints about Maps.me users is that they don’t respond to comments. Of course, there can only be something to respond to, if their Note first gets commented. For that to happen, the Note has to be non-anonymous and commented by someone else who was logged in. To my surprise, this is a relatively rare situation. Less than 4% of all existing Notes were made by logged in users and responded to by other logged in users.

In this rare situation, there is a reaction from the original contributor in about 30% of the cases. If the Note was made through osm.org or an unknown app, this rises to about 40%. However, if the Note was made with Maps.me, response rate was only about 10%.

Since Navmii makes only anonymous notes, this rate can’t be counted for that app. The recent StreetComplete notes get a 26% response rate in similar circumstances. So there is clearly something going on with Maps.me. Some further measures to increase comparability might be necessary, for example excluding Notes from heavy mappers. But before we saddle the horses, don’t forget what the actual number is we’re talking about. In 2016, if Maps.me users were exactly like the general population, there would be just 2500 more reactions from Maps.me original contributors, on a total of 123.797 Maps.me notes that year.

Other explanations

So the lack of communication might be relatively minor compared to the “abuse” of Notes by people who don’t know what they are meant for. If someone would like to do some text analysis to see if automatic classification of usefulness is possible, I could offer a file that is ready for consumption. (the Notes dump is relatively easy to parse though)

A bit harder to analyse, would be the hypothesis that the difference is because Maps.me notes tend to contain some info that is hard to ignore (like “this place is closed now”), but hard to verify (“did you really mean Place X”? If you don’t trust the contributor, and they don’t answer, you have to go check).

In other news:

What on earth happened here? From May to July 2017, anonymous comments went absolutely through the roof, and then just as quickly went down to normal again. Most seemed to have been nonsense comments. Does anyone know what happened here?

Oh, and if you’re working with the Notes Dump yourself, be mindful of this minor bug I found.

Work in progress

I’ve been dreaming of a global database of local basic statistics about OpenStreetMap for years. That turned out to be a little over-ambitious for me. Hence my retreat to local statistics for Belgium only. Analyzing Notes is a kind of proof of concept. This article is a little side to the process of publishing statistics about Notes for the world, but also all continents, countries and “regions” in the world. Since these are the tools I’m fluent in, I wrote the analysis scripts in SPSS. They are available on Github. It should be relatively straightforward to translate to other, more open languages. Publishing will be on the Swing.eu platform. Unfortunately closed source payware, but they donated a version for OSM analysis purposes.

Shape this project

I usually start of projects like this with a clear goal in mind, then lose track of that goal completely because of all the interesting side streets I find. Working towards the Swing.eu platform helps shape that a bit. But what kind of questions would you like to see answered? I intend to make it easy to play around with the numbers yourself, but maybe there’s some aspects I’m missing entirely. Let me know! Feel free to post here or as a Github Issue.