OpenStreetMap

Introducing ArcGIS Datasets in OSM Editors

Posted by Deane Kensok on 14 July 2020 in English (English)

During the Esri User Conference, Esri and Facebook today announced the availability of ArcGIS datasets in updated versions of two OSM editors, RapiD and JOSM. You can check out the Esri blog post and Facebook blog post for details, but I also wanted to provide a little more context and contact info for the OSM community.

In addition to the local knowledge and field surveys of OSM contributors, another source of OSM data is government data that has been released with appropriate licenses. Many government agencies in the United States and around the world are Esri customers, and they use ArcGIS software to create, publish, and share their GIS data.

Esri would like to help OSM mappers by providing more convenient access to this type of data where it would be useful in OSM and it can be provided with an appropriate license. Esri is encouraging our ArcGIS user community to share their data as open data, preferably with licenses that are compatible with OSM.

In the past few months, Esri has identified some open data (e.g. building footprints and address points) that can be used in OSM and processed that data so that it is more ready-to-use in OSM. To enable access to this data for OSM mappers, Esri has worked with developers at Facebook to expose this data in OSM editors through a new version of RapiD and an updated Map With AI plugin for JOSM.

As described in the Esri blog post, these tools will enable OSM mappers to choose a dataset they’d like to use for mapping, select a feature in the dataset that does not currently exist in OSM, review the geometry and available fields (tags) for the feature (such as below) and make adjustments based on local knowledge or other appropriate sources as needed, and then upload their changes to OSM.

Selected Feature for ArcGIS Dataset in RapiD

ArcGIS Datasets

Esri has identified several GIS datasets that contain data we believe would be useful additions to OSM and which are compatible with the OSM license. For those datasets, we have processed them into a more OSM-ready format (e.g. converting field names and values to those expected by local OSM community), and then published and shared them for access by OSM editors.

You can view the datasets in this Esri-curated group in ArcGIS Online, and get additional details on the datasets and plans for using them in this ArcGIS Datasets page on the OSM wiki. As described above and in the dataset pages, the intent is for OSM mappers to access and uses these datasets while doing manual editing in OSM editors such as RapiD and JOSM, and not to use them for bulk imports.

Going forward, Esri intends to assemble more types of data (e.g. roads, parks, POIs) that would be useful additions to OSM and make them available to OSM mappers. One of the primary ways we will do this is through the new data sharing option in the Esri Community Maps program, which will explicitly permit use of this data in OSM.

Many of us at Esri in the U.S. and around the world are OSM mappers and enjoy contributing to the project, both personally and professionally. We hope the availability of these ArcGIS datasets will be helpful to other OSM mappers and will help enhance OSM data for everyone.

How to Reach the Team

If you have questions on specific ArcGIS datasets, or suggestions for additional datasets, you can add them to the discussion on the respective ArcGIS Datasets wiki pages or send an email to osm@esri.com. If you have questions or suggestions on the new tools in RapiD and JOSM, you can tweet @MapWithAI or find the team in the #mapwithai_feedback channel of the OSM US Slack: OSM US Slack. To submit an issue, you may do so here for RapiD or here for the MapWithAI plugin for JOSM.

Location: Esri Inc., Lugonia Homes, Redlands, San Bernardino County, California, United States of America

Comment from ABZ_OSM on 15 July 2020 at 11:00

Thank you for your post. I did not realise all this very beneficial mapping work was going on at Facebook. I saw the linked video on your site which shows that Facebook work has populated hundereds of thousands of kilometers of roads in Thailand. This is good work!

At least a couple of us on here, discussing talks at a recent STOM 2020 Baltic, in Riga, Latvia, https://www.openstreetmap.org/user/Zkir/diary/392427 had subsequently tried the AI over trace plugin for JOSM. Neither of us could get it to auto over trace buildings, which would then be human verified as acceptable traces.

Auto over trace of buildings would be a signifcant technology change for mapping, as it would allow us to detail our local communities much much faster. The laborious nature of hand tracing is what is holding things up, as so many of us know.

If you know how auto tracing buildings can be done with JOSM, then perhaps you could make a video, demonstrating this, to make everything clear, or point me to a working resource, and I will make a youtube video and link it on the wiki and post the link up here on the diary forum.

Thank you again, this is very good work.

Comment from ABZ_OSM on 15 July 2020 at 12:35

Thank you again for your long post which was very informative and useful.

A youtube link to a screen cast video, I have made, of my experience using the “mapwithai-dev” plugin for the “josm-latest.jar” version of JOSM is contained within my diary entry linked here https://www.openstreetmap.org/user/ABZ_OSM/diary/393621

I did not find the plugin very useful, becasue, locally, here in Aberdeen, Scotland, Europe, all our roads are already in OSM.

What we need is for AI to suggest auto traces of buildings, for completing the next level of detail.

Surely all Facebook need to do is run the ai algorithim on a world map satelite imagery data set and release the building tracing files?

Thank you again for your long post which was very informative and useful.

Comment from RobJN on 15 July 2020 at 19:57

Hi ABZ_OSM, I might be wrong but I think the new datasets that are the topic of this diary post have come from local governments GIS team’s rather than from AI image detection. Nevertheless AI buildings is something that Microsoft has been releasing whilst Facebook focused on AI road detections.

As you rightly stated, we have good road coverage for the UK but would benefit from more buildings. It’s unlikely that we’ll get any “official” data as this is all done by Ordnance Survey and they clearly don’t want to release this as open data.

Manual or AI tracing of buildings is therefore our best bet. OSM UK has an option to access 12.5cm imagery (at a tiny cost for a company interested in OSM) so let’s discuss… :-D

Comment from ABZ_OSM on 16 July 2020 at 13:53

Hi @RobJN,

> “I might be wrong but I think the new datasets that are the topic of this diary post have come from local governments GIS team’s rather than from AI image detection.”

Do local government planning departments not derive all their data from OS data sets in any case, or do they have an agreement with OS to share ownership, given it is likley local authority surveyors make contributions. How does this work.

> “Nevertheless AI buildings is something that Microsoft has been releasing whilst Facebook focused on AI road detections.”

ah, that is interesting. Thank you. I was not aware of Microsoft making building trace contributions via AI.

> “.. we have good road coverage for the UK but would benefit from more buildings. It’s unlikely that we’ll get any “official” data as this is all done by Ordnance Survey and they clearly don’t want to release this as open data.”

Yes, I have gathered over the years that Ordnance Survey won’t release data as open data.
If I ever did know the reason, I can’t remember why.
I don’t know why they don’t.

/> OSM UK has an option to access 12.5cm imagery (at a tiny cost for a company interested in OSM) so let’s discuss… :-D

I don’t currently have access to any funds. Will let you know if that changes. If you have numbers message me, as it would be useful to have at hand.

Comment from RobJN on 16 July 2020 at 18:27

I think the OS are responsible for mapping the buildings although I’m no expert. The local authorities do feed some data in such as new street names and addresses. I think the recent changes that OS had made will make it easier for local authorities to publish their own geospatial data but there is a specific exclusion on building footprints.

The last comment was targeted at any corporate readers. Whilst we have what we believe is a good quote, the price is too high for any one individual to cover.

Comment from Deane Kensok on 16 July 2020 at 21:34

Hi @RobJN, thanks for adding your comments, they are very informative. You are correct about the type of GIS datasets that I was referencing in my post. We’ve been primarily working with local governments in the US initially but hope to add more internationally too.

You are probably more familiar with the OS data landscape than me, but I am checking with a couple of my friends and colleagues in the UK on potential open data there that we might be able to help make available.

Regarding the 12.5cm imagery that you mentioned, I would be interested to learn more about the quote for OSM UK. If you can share some more details with me, you can reach me by email at osm@esri.com.

Hi @ABZ_OSM, and thank you also for your comments. I’ll let you know if I can provide any additional suggestions for adding building data in the UK, either through JOSM, or using data from Microsoft, or perhaps some more local source. I will look into this more.

Comment from ABZ_OSM on 27 July 2020 at 11:05

Hi @RobJN,

I’ve had a think about 12.5cm maps.
The overtracing benefits need no explanation.

Here are suggestions that can potentially be taken forward, though I am out of time regards helping at the moment

  1. Communities Cabinet Secretary in Scotland’s Government Aileen Campbell is very helpful and might have some level of budget for maps for Scotland, and further afield, eg globally, if she can be shown how interconnectivity helps social cohesion. I beleive OSM is a great example of how people can be better connected, especially given the street level detail that can be filled out and custom maps that can be created using the many layers and renderings available.

A pitch about getting local communities involved in mapping I think would be especially attractive. Two good examples of improved connectivity might be
* placing numbered and named fairways on a local government run golf course. Often people living beside facilities have no idea what is next door. When it’s easy to see on a map whats around you everything changes. ps Golf is a sport for everyone in Scotland, Europe, and is not considered exclusive here. Understanding what’s around you opens the possibility of visiting and engaging. Getting people outdoors improves health and wellbeing including mental health. eg this example I completed the day before yesterday https://www.openstreetmap.org/#map=16/57.1614/-2.0835
* mapping a zoo with detail in Dundee, Scotland, Europe, so that the local community can see what is there. Again, understanding what’s around you opens the possibility of visiting and engaging. Getting people outdoors improves health and wellbeing including mental health. eg this example I initiated and completed https://www.openstreetmap.org/#map=18/56.48138/-3.04421

Paul Wheelhouse is also very approachable and is responsible for Energy, Connectivity and the Islands including 100% broadband.

I can recommend both as approachable (though very busy), serious and committed to the development and furtherance of Scotland’s interests, and wider interests, at home and abroad.

Bear in mind that at this time the covid priority is sucking all time and financial resources and therefore there might be no budget available or time to discuss the benefits of mapping at this time.

2 National Lottery. I am sure this avenue has been explored many times before.

Login to leave a comment