OpenStreetMap
A workflow for using Overture places data in OSM
Posted by mikelmaron on 13 October 2023 in English.I want to use Overture Places to improve OSM, and want to share a few ideas on that.
First the data is worldwide, and there’s definitely added value there to mine. Here’s Brandon Liu’s map over Santo Domingo.
However it’s a mixed bag. Adjusting confidence score is helpful. But even then, some things are in right location, some are already in OSM, some are closed, or miscategorized. It takes careful analysis to find places that could added to OSM. Wille did a great job examining his local area
This is not a harsh critique – it’s tough data to manage. There’s value. I’d like a workflow that gets at deriving that value, fast. I think it’s a combination of something that enables the kind of analysis Wille did, along with an easy way to edit OSM.
I don’t think this needs to have a fancy entity matching process between Overture and OSM. Choose an area. Generate a list of features in Overture, with confidence threshold, and filtered to feature classes of interest. Show both Overture and OSM on the map. Work down the list, examine the map, take an action in OSM if necessary (adding, updating, or nothing), then mark the task with the action.
I guess this could be done through MapRoulette? Though tasks there seem typically driven by analysis of OSM objects, not 3rd party data to conflate. RapID? Not possible to create your own tasks. What about adding Overture as an overlay in iD? Another way?
Discussion
Comment from Friendly_Ghost on 14 October 2023 at 11:43
I use GeoNotes to generate GPX files to use as a background layer in JOSM. Sometimes that description is simply the name of a business that I passed by while I was travelling. The result is lines of text descriptions on top of the aerial imagery, and I can very easily make new map data based on the descriptions I wrote. I guess you can do the same with Overture Maps data.
Comment from tordans on 15 October 2023 at 04:36
Let me start by saying: I am general for adding all the details to OSM. It’s our USP. However, when it comes to Business POI, I am really hesitant and unshure if this is a dataset that we should strategically “invest” into (AKA push for more data and more usage). Why? Because it’s an area that – more than other – will attract fraud … and I believe our moderation and detection tools are way out of league for that. To get a feel for the issue, check out this podcast https://gimletmedia.com/shows/reply-all/o2ho87. There are also posts like https://www.theverge.com/2019/6/20/18693144/google-maps-fake-business-listings-investigation-report and recently https://latlong.blog/2023/08/places-on-google-maps-can-they-still-be-trusted
But putting that aside…
My general take on POI is: This is something to be mapped on the ground (not remotely). As in: You should stand in front of the shop to map it … or have great local knowledge of the shop to map it.
Which means: Whenever we look into data like this, we should look into mobile mapping tools.
POIs are also something that is very annoying to map. A lot of tags, a lot of micro decisions and – especially you start typing opening hours – a lot of time spent typing per edit.
Which means: To really roll out POI mapping, we need better tooling in our mobile editors to map those.
However, all of those are lacking features to validate external data and use a prepared external dataset as a basis for what the user maps.
There are several pieces missing in our toolchain…
We need an easy way to process external data against existing data. I wrote about that in https://github.com/facebook/Rapid/issues/585#issuecomment-1249994877 “Help with data preparation”. We have many external datasets that we could use to guide map updates. But processing them in “maybe needs to be deleted in OSM”, “maybe needs to be added to OSM” and “maybe needs to be updated in OSM” is too hard right now.
We need a tool that we can use as a shared Tasklist. This is needed so we can come together in a shared effort and in order to split up a big dataset into separate tasks. We also need a place outside of OSM to document our findings once we checked a task. The best tool we have is MapRoulette. However, we need a great mobile editor integration in order to use it for topics like the POI dataset – an IMO most other micro mapping datasets. I created https://github.com/maproulette/maproulette3/issues/1737 to track efforts in this area, but it is not a focussed topic for MapRoulette, yet.
And finally… - We need mobile editors to integrate MapRoulette as a tool to guide mappers to the next location “around the corner” which they then can update with minimal effort, due to the preparation of the external data in the MapRoulette Task. The issue above tracks my efforts to get MapRoulette inside editors. But we lack a shared understanding of the impact of such a workflow in order to get all the people needed to work on this.
All in all, we are slowly going in the right direction but have along way to go before we can efficiently update and add external datasets that require hyper local validation into OSM.
Comment from mikelmaron on 16 October 2023 at 01:06
I agree to a point – I think being familiar with an area is essential. I don’t think the mapping needs to happen standing right in front of it. I see Overture Places as a source for jogging my memory, or sparking my interest next time I travel in a direction. I wouldn’t use it as a source alone, but it’s useful as a source in combination with knowledge.
Comment from mvexel on 19 October 2023 at 16:04
To add to Tobias’ point: Local knowledge is what OSM is uniquely positioned to bring to the table when it comes to POI maintenance. Anecdotally I have found it much easier to get people interested in contributing to OSM talking about POIs and businesses. This is something everyone can relate to and are easy to add to OSM.
There have been a number of successful MapRoulette challenges that deal with keeping POI up-to-date. We currently have one ongoing in the Salt Lake City area for cleaning up old POI. (Side note, these challenges are quite easy to “clone” for your local area.)
You could apply the same principle to Overture data. With a bit of pre-processing using Overture metadata, geographic distance and string similarity, you could pre-filter the Overture places and feed those into a MapRoulette challenge asking mappers to use their local knowledge and / or internet sleuthing to decide if and how to add the POI to OSM.
Ideally these tasks would be included into an existing tool for mobile mapping, like OSMAnd or GoMap!!. MapRoulette already has the API for it, so integration with those apps is not some remote possibility. OSMAnd has done some work on integrating MapRoulette tasks already. I would not want to add yet another mobile OSM editing app to the mix, but it would be entirely possible to adapt the MapRoulette web front end to support mobile use cases as well.
Comment from mvexel on 21 October 2023 at 01:07
There’s some newer discussion on the mobile UX for MapRoulette in this Github ticket.