OpenStreetMap
How to create a translation file to convert a shp file to osm
Posted by baditaflorin on 31 October 2017 in English. Last updated on 1 November 2017.Obviously but very importantly:
- Before copying anything into OSM the license needs to be properly vetted.
- If you are planning to perform any kind of bulk update, the full extent of the Import Guidelines applies.
After comparing Tiger 2017 for 16 US states until now, using shp2cygnus, a tool that I written at Telenav to be able to compare big datasets in a automatic way, to identify roads that are present in the dataset, but not in OSM
After posting on the talk-us, some people told that it would be more useful to run this tool on other datasets, from state or local authorities.
In this article, I want to teach you how to create your own translation file, so that if you have a shp file, you can use Cygnus to compare this with OSM, using this link http://cygnus.improve-osm.org If you have a shp were the size is bigger then 50*50 km, you can send me a email and I can process it for you.
But you need to learn how to create your own translation file.
So, first step.
-
Find your data
We will use as a example Philadelphia
Each city or local authority should have a open data portal.
After a google search for “Philadelphia Open data”, I found this site
https://www.opendataphilly.org/
Search for either street or road
Click the data-set, and identify the shp, that you need to download
Extract the archive and open the shp into JOSM.
To be able to open shp in JOSM, you will need to install the OpenData Plugin
If we look at the tags, we see that they don`t have any connection with what we are used to, like highway=residential, tertiary, etc or oneway=yes
To be able to understand how to translate the shp to osm, you need the Street Centerline Metadata
Click Download, and you will see, at least for the Class, what is the highway type.
To see how you can create the translation for osm, you can look here for the OSM highway tags http://wiki.openstreetmap.org/wiki/Key:highway
After doing this, run this script, and then load the output.osm file into JOSM.
python ogr2osm.py -t translation_philadelphia.py –add-timestamp –add-version –positive-id input_shapefile.shp -o output.osm
It will not be perfect, this is after the first try.
Comparing this with osm data I see that I have to much yellow roads (tertiary)
Also, the trunk roads are actually motorway on OSM
We look at the shapefile and identify that Class=4 should be highway=residential, instead of highway=tertiary
We run again the converter
And we get now a more better result, more accurate with what we have in OSM.
If we are happy with the result, we need to save the osm XML file to PBF, so that we can use them with Cygnus.
We do this by installing the pbf plugin in JOSM
Save the file as Pbf
We next go to the http://cygnus.improve-osm.org website, and load the pbf file.
Also, you can select some tags of interest, for me, this was oneway.
In the end you should see something like this
You can download the translation file for Philadelphia from here
https://github.com/baditaflorin/ogr2osm-translations/blob/master/philadelphia_translation_file.py
This is a part of the final result.
## 1
The way 336945939 have 1500 nodes, and does not contain now any relevant information.
## 2 Buildings not sharing same connecting nodes
## 3 All the city had been imported 2 times
This means that every building have a duplicate, one of them should be deleted.
## 4 Duplicated Highway path
Way 370332689 and way 363957545 are the same.
## 5 Bad Import
Separate ways for each objects, when they should be connected and share the same way.
## 6 2 relations on 2 different ways
Way 164811803 and way 164682871 should be combined where they shere the same path, and one of them deleted, and mode the relation into the remaining way
## 7 Bad Import - Building and fence not sharing same way.
That is one of my imports. Ups.
## 8 - To much details
## 9 - Reduntant Nodes
The line is strait, so the information is redundant.
