OpenStreetMap

How we apply map feedback

Posted by PlaneMad on 14 April 2015 in English.

At Mapbox, we’re continuously updating OpenStreetMap from user feedback - here’s how we go about the process.

Any user of a Mapbox map can flag wrong or missing information via an “Improve this map” link in the bottom right corner of the map. The link leads to a feedback page that allows a user to very quickly say what’s wrong on the map. Alternatively he/she can chose to edit the underlying data - OpenStreetMap - directly. Since we streamlined the feedback form in January we are receiving a much higher volume of end user feedback and we’ve formalized our process of handling this feedback.

Using “Improve this map” on a DuckDuckGo map.

What feedback we receive

The current break down of feedback we receive is roughly as follows:

  • 50 % does not contain sufficient information or is just spam
  • 30 % is wrong or missing information on map (this is the part that’s relevant for OpenStreetMap)
  • 10 % is feedback for the app the map is embedded in (not about the map at all)
  • 5 % is a geocoding issue (mostly an issue with the geocoding API or non-OpenStreetMap geocoding data)
  • 3 % is a directions issue (this is often also OpenStreetMap relevant)
  • 2 % is about map design or similar

Our process

Every map feedback submitted opens a ticket on an internal Github issue queue that our data team reviews daily.

Fresh map feedback in our issue queue. We’re using Zapier to turn form submissions in to Github tickets.

We review each map feedback ticket with the following workflow:

  1. Determine whether a ticket is relevant for OpenStreetMap data
  2. If yes, determine whether we can fix the issue with remote mapping
  3. If no, open a note on OpenStreetMap with details of the feedback.
  4. Verify feedback against a secondary source.
  5. If we can verify the feedback, we update OpenStreetMap.
  6. If no, open a note on OpenStreetMap with details of the feedback.

Secondary sources

We verify all map feedback against a second source before applying it to OpenStreetMap. To this end, we work with the following sources:

  1. Satellite imagery fit for use in OpenStreetMap (vast majority of feedback)
  2. Original information (like for instance a shop’s web site)
  3. Wikidata or similar appropriately licensed sources

Zeromap’s recent writeup of how we wound up accepting a misspelled location caused us to make the verification step a much stricter requirement. If we cannot verify information against a secondary source we will not apply it to OpenStreetMap but rather open a note. If the information between map feedback and the secondary source differs, we will use the secondary source.

Publishing feedback

The submission form on Mapbox maps allows for really fast feedback but with it also comes an expectation of privacy. For instance, some users submit personally identifiable information, such as their name together with their address. We take this expectation of privacy very very seriously and hence do not publish feedback we receive directly. When we pass on feedback in form of an OpenStreetMap note, we scrub it of any personally identifiable information and frame it as a question that enables participation from local mappers.

screenshot 2015-04-14 20 59 16

Your feedback

Our goal is to make OpenStreetMap the best map in the world. We see our work very complimentary to local community and are intent on making sure we only add solid information to OpenStreetMap.

Let us know where we can do better! Just comment here or send me a note through my profile.

Discussion

Comment from SimonPoole on 14 April 2015 at 21:27

The problem with the whole concept is that it doesn’t provide for the conversion of a map user with an issue to a map contributor who can fix an issue themselves.

Or put differently it removes one of the important incoming routes for recruiting new mappers.

As is, the whole process is very much at odds with the basic concept of OSM.

Comment from mikelmaron on 14 April 2015 at 21:40

There should be and are many levels to contribute to OSM, so this is a good concept, and thanks for the transparency. To pull out a productive idea, offering a path to users giving feedback (and notes), to get more involved in OSM, is a useful idea.

Comment from SimonPoole on 14 April 2015 at 23:40

@mikelmaron in case you hadn’t noticed, there has been a “Report a problem” page on openstreetmap.org for quite a while that anybody can link to from their map/website see http://www.openstreetmap.org/fixthemap It might profit from a more structured way of submitting notes, but that is a more general issue with the notes facility as it currently stands.

BTW there are good business reasons for MB to direct map feedback through their support group (lead generation, early detection of customer issues and so on) that does not mean it is necessarily a good thing for the project as a whole.

Comment from PlaneMad on 15 April 2015 at 03:35

@SimonPoole Thanks for bringing up that point. In after submitting feedback we prompt the user to directly edit the map using the live editor into OpenStreetMap.

screenshot 2015-04-15 08 52 03

We have noticed that users of apps like Runkeeper have consistently gone to edit OSM themselves after submitting feedback, thereby fixing the map even before we look into the issue. The conversion rate has been lower from other referrers, but we are definitely seeing new editors coming in through this process.

Even though iD makes it really simple to edit the map, for an end user on a mobile who does not have the inclination to learn a new tool while using his app, map feedback becomes the easiest way to contribute to OSM in a matter of seconds. Once a user has crossed this step, it becomes much easier to convert him into a map editor by showing him the way forward.

Let there be no doubt that we want to grow a strong OSM community, and map-feedback has been one of our approaches to help do that. Also note that this is still an experiment which has given very good results so far, and any ideas for improvement is definitely something we will factor in seriously and try out.

Comment from PlaneMad on 15 April 2015 at 03:43

Some stats since February when we started working on map-feedback: - 4,080 issues logged - 1,895 issues not actionable - 754 map issues fixed - 112 osm notes opened

Comment from SimonPoole on 15 April 2015 at 06:01

@PlaneMad offering the option -after- submitting a report is naturallly not really a solution, offering it as an option at least somewhere during the process would be.

Comment from PlaneMad on 15 April 2015 at 06:39

@SimonPoole The edit map option is available before as well.

screenshot 2015-04-15 12 04 50

Comment from lxbarth on 15 April 2015 at 16:39

Let there be no doubt that we want to grow a strong OSM community, and map-feedback has been one of our approaches to help do that.

Adding to this: a previous iteration of the map feedback page was flipped around: the first action was “contribute to OpenStreetMap”, the second action was “let us know what’s wrong”. This didn’t work as well as we wanted to. Only a minuscule amount of visitors wound up going to OpenStreetMap (we don’t know whether they actually edited).

We’ll continue to measure the impact of this page and iterate on it. Overall, driving editors to OpenStreetMap is the best thing for the project - we all agree on that.

Comment from lxbarth on 15 April 2015 at 16:52

Hit send too early, continuing:

We’ll also have to balance driving editors with the goal of knowing what’s wrong on the map. We only have a seconds of a user’s time to learn what’s going on, so we’re keeping this experience laser focused.

Since we flipped the form around and offer a fast submission first we’ve increased the number of reports we’re seeing at least by 10 x. I’m seeing we lost the piwik link on the edit button so I can’t report on users we drive (getting this one back in now).

Everybody’s OpenStreetMap relevant knowledge going through map-feedback winds up on the map: either as a direct edit by the Mapbox team, as a note or by a direct edit by the user.

Comment from Hedaja on 17 April 2015 at 18:18

I think having the contribute action later might be not that bad. If the user reporte a problem and corrected it by himself afterwards Mapbox still knows there was an issue an could control the changes the user made. (Don’t know if this is already happening) In my opinion the feedback possibility is quit useful for OSM.

Comment from trulysans on 26 July 2015 at 13:34

This feedback system is for osm based mapbox maps in places like duckduckgo and runkeeper only? And in India, I am not able to access Duckduckgo maps I think.

So this report problem feature is universal or specific to regions/applications/ users?

Comment from PlaneMad on 26 July 2015 at 13:47

@trulasans, its for any Mapbox map irrespective of location or application. Try it out on http://geojson.io/ , click the (i) at the bottom.

Comment from PlaneMad on 26 July 2015 at 13:48

@trulasans, its for any Mapbox map irrespective of location or application. Try it out on http://geojson.io/ , click the (i) at the bottom.

Comment from philippec on 23 February 2016 at 09:28

So, in short = Mapbox and others like me do work for people who comment but who do not give proof nor take a picture.

Comment from PlaneMad on 24 February 2016 at 09:48

@philippec not everyone has the leisure time to dedicate effort to learning OSM and mapping, and this is where experienced contributors like us can help bridge the gap.

Comment from Essin on 11 October 2016 at 08:06

While I understand why personal information needs to be removed, the current wording of the notes is too untransparent, I think.

I’m an experienced OSM mapper, but when MapBox notes started to appear in my area of interest (mostly Sweden), I didn’t understand at first that they were a coordinated effort. It was very hard to judge the reliability of the notes, and whether it would be worth taking a detour to check a note. My first guess was that the notes were from tourists who had noticed possible errors when passing through the area but didn’t remember it exactly when making the note. Follow-up questions for the notes (by me and other local mappers) often went unanswered for a long time. Only after starting to check the user pages of the people making similar notes, a pattern emerged. Even so, it was not obvious where the information came from. http://wiki.openstreetmap.org/wiki/Mapbox#Mapbox_Data_Team is linked from the user pages, but the link to this post from there is not prominent, and it took me some determined effort to find that it answered my question.

Tagging all MapBox notes with a link to this post, or at the very least a tag similar to Wisepilot or Navmii notes, would make the process more open without compromising the personal information of the original poster. This way, it would also be more obvious that a detailed reply doesn’t actually close the feedback loop, and might not be necessary as long as the map data itself is fixed.

Log in to leave a comment