OpenStreetMap

sabas88's diary

Recent diary entries

Surfacing Wikidata objects with coordinates to match them with OSM

Posted by sabas88 on 1 March 2017 in English (English)

Wikidata, as a project derived from Wikipedia, could be viewed as a crowdsourced database of VGI (Volunteered Geographical Information), of course less structured than OpenStreetMap spatially but at least comparable: we think that a cross-reference could be worthwhile for both projects. This work has already started from some years ago with the wikipedia tag (notably the WIWOSM project, and in Italy wtosm) but now the focus seems to be moving towards the use of Wikidata instead of Wikipedia.

In this post I would like to introduce our experiment in this direction, powered by the resources we have as a chapter of both the OSMF and the WMF.

We started from an existing OSM database replicated every half hour through osmosis, where all the tags are dumped in an hstore column and we added a table called wikidata and a view which gathers existing elements tagged with the wikidata key (UNION of nodes, ways and relations).

The wikidata table is populated by a script which parses the weekly Wikidata dump (~10 GB gzipped line delimited json): we get only the elements having a claim with the P625 property (an element with at least a coordinate) and we take only the ones in Italy (a "rough" point in polygon test). The objects are then saved with the most precise coordinate available, their id and a label (italian, english, serbo-croatian or the first available).

Why serbo-croatian you may ask? We noticed that the Wikipedia editors created a lot of stubs from Geonames which went to generate new Wikidata items having only the label in the sh iso code :-)

Now we have our brand new table and we can create our service: a map showing all the Wikidata elements colored by their OSM status. Green if already matched, Red if it’s an element which can't appear in OSM (an historical battle or structure for example), Grey if they still need to be processed. Each marker has its popup, linking to the object on Wikidata (and on OSM), the wikidata tag to copy, and two buttons: one to mark the object as non-mappable, the other to mark it temporarily done (it would -hopefully- become green on the next run).

Screenshot

The service is live at http://osmit3.wmflabs.org/wikidata/ and covers Italy.

Code: https://github.com/osmItalia/wikidata-geo-match

Daily local extracts for Italy

Posted by sabas88 on 23 December 2015 in English (English)

Screenshot

This week we (me, simone and sbiribizio) launched the new version of the 'Italian Local OpenStreetMap Extracts': a website from which users can download slices of the italy-latest.pbf provided by GeoFabrik on a regional or municipal level, converted also in different formats.

The live version is kindly hosted on a virtual instance at http://estratti.openstreetmap.it and the code is on Github

The web interface is written in php (with the flightphp microframework), allows multiple languages (currently italian and english) and queries a postgresql database popupulated by the extraction scripts. The scripts are a mix of bash, perl and python and require some binaries commonly available in Ubuntu. As background layer in the maps we use Wikimedia Maps

Location: ROXAN

Does it render?

Posted by sabas88 on 4 June 2014 in English (English)

Last year I did a quick demo application to mock people asking if an object will get rendered on the map, extracting some information from the stylesheet used for the map on osm.org (it was during the switch to the Carto version).

Today I reviewed the process of extracting that information and generated semi-automatically a complete list of key-value pairs, which powers now DoesItRender.

Code is here https://github.com/sabas/doesitrender and thanks to Pages, it's active on http://stefanosabatini.eu/doesitrender/

If a tag is in the style, it answers yes (http://stefanosabatini.eu/doesitrender/#amenity=bar). If not, it gives option (http://stefanosabatini.eu/doesitrender/#tourism=unicorns) to display it via overpass-turbo or xapi-viewer.

I mapped 100 countries in 100 days

Posted by sabas88 on 13 April 2014 in English (English)

World Map

If you for chance read my previous diary entry, I registered on GiveIt100 and documented with a video a changeset made each day in a different country. I followed some basic criteria: each day in a different continent (counterclockwise order); if I missed a day I'd map in the same continent for two days; map in a country where I didn't map before.

So, straight from January 1st, I got to April 10th and in the last video I did a changeset in San Mateo, CA, where lives the developer of GiveIt100 (I previously mapped in USA, I know!).

The complete project is visible on https://giveit100.com/@sabas/ro277m

Having the NaturalEarth shapefile half filled, I should paint it green completely some time...

Someone with a planet file at hand could do a ladder of users ranked by how many countries has been visited (similar to the hydc user page)..

Thanks to Karen and Finbarr for their work!

Mapping different countries, one a day

Posted by sabas88 on 6 January 2014 in English (English)

Imgur After seeing an article on The Verge - this one - about the launch of giveit100.com I wondered if there was something to do involving OSM. The mission of the social network is to try to make a resolution and stick to it for at least 100 days, filming a short 10 seconds video segment.

Well, I chose to make a small contribution in a different country each day, and today I did a changeset in Jamaica as my sixth country. Long way to go, but I'll try (to make viewable the video I make circa 6 minutes of editing, so between 50 and 150 edits usually). Here's how I work to do this:

  • the global map is a shapefile from Natural Earth rendered with qgis (3 pictures each lasting 15-16 frames)

  • I use recordmydesktop to do the recording (resulting in a ogv file)

  • ffmpeg to convert from ogv to avi

  • kdenlive to mix them up, the video is accelerated to fit 205 frames

Here's my project link (which was also selected as featured) https://giveit100.com/@sabas/ro277m

Location: OLETI, LIG, Italy

North Korea is strange

Posted by sabas88 on 15 August 2013 in English (English)

I was mapping something in North Korea when...

Imgur

I wonder if they understand they have to put map labels only on the map, not directly on the ground.

BTW some who understand Korean can translate? I'm going to put this on the map, there's another one near the dam on the right.

Location: Pongbul-ri, Tosan County, Hwanghaebuk-do, North Korea

Sardinian Coastlines

Posted by sabas88 on 21 June 2012 in English (English)

Today I finally ended checking the whole coastlines of Sardinia, except the Medio Campidano and Carbonia - Igliesias provinces. I started a year ago while editing my first area in West Sardinia and today I can say I've touched all the remaining parts where there was PGS or there was a bad definition.

Next I'd like to import the new borders for remaining provinces of Sassari and Olbia-Tempio.

Older Entries | Newer Entries