Bringing third-party data into OpenStreetMap has always been a pain point. Simple bulk imports can bulldoze existing community work and cohesion. Editors such as JOSM or even Potlatch 2 have extensive merging functionality but a steeper learning curve and little ability to remap/rewrite data internally. Yet third-party data can often be valuable - especially if it was originally hand-collected to the same standards that OSM volunteers work to.

OSM Live Conflation is a new tool and workflow that builds on the approach taken by OSM Conflator, Cygnus, the JOSM Conflation plugin and the Potlatch 2 merging tool. It remaps data from vector tiles into OSM data, and lets you bring items across one-by-one quickly and efficiently.

A picture is worth a thousand words:

Or if you want to play with a live instance, here’s a volunteer-collected dataset of cycling infrastructure in West Oxfordshire. (Don’t upload unless you actually want to make an edit to OSM, of course!)

User workflow

On the left is the source data. Clicking a feature shows the tags/attributes associated with it, and identifies a candidate in OSM for that feature: that’s highlighted in the map on the right, and the proposed changes listed.

You can choose to accept the change, or ignore the feature. If the candidate is wrong, then you can choose from several others, or choose to create a new OSM object. When you’re done, click ‘Upload’ to make the edit in OSM.

Project owner

That’s the user workflow. What about the project owner?

First you need to get your data into vector tiles. The easiest way to do this is to put the data in shapefiles (using ogr2ogr) then use tilemaker to create vector tiles. With tilemaker, you write a small Lua script to remap the shapefile attributes into OSM tags: here’s a simple example.

You then deploy an instance of the OSM Live Conflation site including your vector tiles. The site includes a small Ruby server to serve vector tiles, metadata, and static files. The main interface is, of course, Javascript.

iD integration

The vector tiles can be loaded directly into iD using a URL such as https://conflation.cycle.travel/{z}/{x}/{y}.pbf . I hope to be able to increase the integration once iD’s third-party features support lands.

Thanks and further work

Work on this project has been supported by the Open Data Institute via Oxfordshire County Council - thank you!

Development is at an early stage and I’m very happy to offer advice and support to anyone interested in experimenting with this tool. Leave a comment here, email me at richard@systemeD.net or find me on OSM’s IRC channels.

You can see an initial to-do list at https://github.com/systemed/conflation/issues/1 .

(Yes, I did write yet another editor :) )