Recent diary entries
This post is about some new tools I and others have made to facilitate community-based import of data into OpenStreetMap. These tools cover converting .csv files to .osm files and splitting .osm files for the Tasking Manager, and I am writing this post hoping they can be of help to others. The code is available in the mtl2osm GitHub project (“mtl” stands for “Montreal”).
(Please note: mostly if not all links here are to French web pages.)
The Montreal community import process
Here in Montreal we are slowly preparing to import open data provided the city. Said data is licensed under CC-BY 4.0, and we got the required legal authorization from the City to import them into OpenStreetMap.
Our import process is documented here. We have not started any data import yet. We are currently in the process of getting feedback from the local community through the OpenStreetMap Montréal user group.
Tools & dataset
I recently made a presentation about the various datasets provided by the Ville de Montréal at a Montréal Python meeting. The presentation was recorded and can be viewed on YouTube (the presentation is in French).
The datasets offered are various and many of them very pertinent for OpenStreetMap:
- address points
- bike parkings
- police & fire stations
- community gardens
- and many others
These datasets are provided mainly in three different formats: Shapefile, CSV and KML. Shapefile is the data format powering the ESRI GIS suite, and which QGIS can also work it. CSV is the well-known comma-separated values tabular format and KML is the XML format popularized by Google Earth.
The org2osm script can handle both Shapefile and KML. This program converts these files into .osm files that can be opened directly into JOSM. Using the concept of “translators”, it becomes possible to convert the tags and attributes of the original data source into values accepted by OpenStreetMap.
For example, for police stations, the data provided by the Ville de Montréal is a KML file that contains the name of the station, the centroid and the address formatted as HTML in the description field. The file was thought of for display purpose in a viewer and obviously said tags cannot be used as-is into OpenStreetMap. This is why I wrote a translator for ogr2osm to parse the data and output OSM compliant tags. The script is available here and instructions to run it available here.
csv2osm: a tool for converting .csv files
However, ogr2osm cannot easily read .csv files as an input, so I wrote a script that is similar to ogr2osm and provide the same translator concept, and work exclusively for .csv files. The script is called csv2osm and can be found here.
Translators for this script have the same entry point as ogr2osm (the filterTags function). For example, Montreal's open dataset for bike parkings is provided as a .csv file. The translator I wrote for this dataset parses the description of each bike parking to extract the capacity and provides the following tags for each points:
- capacity=(number of places)
- operator=Ville de Montréal
Using the HOT Task manager for community-driven import
Lastly, our import effort is going to be based on the HOT Tasking Manager. Our instance is going to be located at http://gt.osmqc.ca/ ("gt" standing for "gestionnaire de tâche", i.e. tasking manager).
To ease importing the .osm dataset into JOSM, we had to write a script that splits the .osm files into a series of sub-files, one per sub-task. Essentially the .osm file is split-up according to a grid defined in JSON format. This allows to see in JOSM only the part of the data for the task, instead of having to download the data for the whole city. Code is available here.
I hope these tools can be of benefit to the OpenStreetMap community. Do not hesitate to comment or write me if you have any question. I know the documentation on these tools is not 100% complete, so I'll certainly post more details if there is interest for them.
Nous avons été 8 personnes aujourd'hui à braver le froid, un dimanche après-midi de la fin du mois d'octobre, pour cartographier les commerces et les points d'intérêts du chemin de la Côte-des-Neiges dans le quartier du même nom à Montréal.
On sent bien que l'hiver est à nos portes, et qu'il sera froid!
La zone inventoriée est très dense en terme de commerces de toutes sortes, mais ces commerces étaient très peu représentés dans OpenStreetMap.
Voici le résultat. La carte avant:
La carte après:
La journée s'est terminée bien au chaud en mangeant de la tarte achetée dans un marché sur la rue, à introduire les données dans OpenStreetMap.
Merci à tous les participants!
PS: si vous vous intéressez aux activités se déroulant à Montréal, n'hésitez pas à visiter le site Web du groupe local: http://www.openstreetmap-montreal.org/
I have been a serious user of OpenStreetMap for less than six months, and I am proud to recently have achieved my one hundredth contribution to the project. I really love the OpenStreetMap project, and I would like to replace my daily usage of Google Maps with OpenStreetMap.
But it just seems I cannot.
I'll give a few concrete examples why, humbly hoping that my words can encourage changes to the main website.
First point: searching. I have OpenStreetMap zoomed in to some region of Montreal, Canada. I input "café", looking for a coffee shop. I get results from Nominatim, inviting me to visit a village in Brazil named "Café" or even the "Café point" in Antarctica. While these search results awaken my globetrotter's desire to explore the world, they frustrate me at the same time. Why couldn't Nominatim priorize results from the bounding box or surrounding? Why can't OpenStreetMap show me results on the map like the OverPass API does, performing a search on the tag amenity=cafe and showing the results on the map?
Second point: accessing POI information. I cannot click on point of interests (POI) to get more info about them. Why do we input address, business hours and phone numbers on shops and restaurants if the map cannot easily display this information to the user? Why do I have to show the map's data in order to have information on a point of interest?
Third point: maximum zoom level. Some area are densely populated, and OpenStreetMap's current zoom level is not enough to see all details of the map. This is really unfortunate. Example: http://osm.org/go/cIrNs6Qzp-- What are the restaurant surrounding the Hard Rock Café on this map? I have to use the editor to be able to zoom and see all data.
Fourth point: sharing a point of interest. There should be an easy way to do that. I have found a (complicated) way to do it, which is all but obvious to newcomers. Here is how:
- Using the layer icon at the top right of the map, I select "Browse Map Data";
- I select the object I want to share (which is not always possible; sometimes it is "hidden" behind a residential area or similar);
- I click on "Details"
- On the resulting page, I click on "View way on larger map"
- I get an URL similar to this that I can share: http://www.openstreetmap.org/?way=225637513
Fifth point: routing. Why is there absolutely no routing implemented on the main OpenStreetMap website? This is I concede a naïve question, as it might be simply because of limited server resources. Once we have our new servers, is this something we want to implement, as a community?
I really like the OpenStreetMap project, and I dream to be able to use it as a primary map instead of Google Maps. I feel resolving these issues would bring me many steps closer to making that dream come true.