I recently wrote about using a Tag-Fix MapRoulette Challenge to update OSM data in an efficient way, no editor required. Tag-Fix is one of two newer Challenge types MapRoulette has available. The other one is called Cooperative. They are called Cooperative because it’s a cooperation between you and the Challenge owner. The Challenge Owner provides the OSM changes, you validate them, tweak them as needed, and commit them to OSM. Cooperative challenges are ideal for when you may be thinking about an import, but you would like to have each feature manually verified by a mapper.

Today, we will look at how to set up a Cooperative Challenge using Microsoft Building Footprints open data. (In a future post we’ll look at how to work with the Challenge as a mapper.) This is not really a beginner’s tutorial; we will be working with PostGIS, QGIS and the command line tools imposm and ogr2ogr. If you have those tools installed and are just a little familiar with them, you should be able to follow along fine!

This is what the final result will look like:

These are the steps involved in creating this Challenge:

1. Download the source data
2. Data preparation
3. Challenge Creation

Let’s dive in.

Download the Source Data

We need both existing OSM data and Microsoft buildings data, so we can compare the two and only include MS Buildings in the Challenge where no OSM buildings exist yet. The OSM data is not hard to get a hold of, thanks to Geofabrik who make small areas available on their download website. You can also use a service like BBBike to download a smaller extract created on-demand.

Microsoft makes the data available on their website. You can access the various Github places where you can actually download the data as GeoJSON from there. The files can be quite large: for the United States, they are organized by state. I tend to pick places close to home for my tutorials, so I can more easily spot any weird things in the data or the process, so we will work with the Utah GeoJSON file, which is a little over 320MB when unzipped. For the Challenge, we will select a much smaller size area, which brings us to step 2!

Data Preparation

We’re going to load all the data into a PostGIS database. We will imposm to load the OSM data, and then ogr2ogr to load the MS Buildings data into the same database, in a separate table.

First we create the PostgreSQL database. You can follow the steps described in the imposm tutorial for this. If you’re on a Mac with Postgres.app you can skip the permissions / user / password steps and just do

$ createdb osm $ psql -d osm -c "CREATE EXTENSION postgis;"

(We do not need PostgreSQL’s hstore field type support for our purpose.)

We will use this simple mapping yaml file to tell imposm what to import and how. It will have imposm create one table osm_buildings with the building polygons in it, and another table osm_admin with administrative areas. We will use this table to define the area for our challenge.

With this mapping file and our database set up, we can run imposm to load the data:

$ imposm import -read utah-latest.osm.pbf -mapping building.yml -write -connection postgis://localhost/osm -optimize -deployproduction -overwritecache

See the imposm documentation for the meaning of all the flags; that is beyond the scope of this already long tutorial 😬.

As a final loading step, we use ogr2ogr to load the downloaded MS Buildings GeoJSON data into a table msbuildings in the same database:

$ ogr2ogr -f "PostgreSQL" PG:"dbname=osm" ~/osm/data/Utah.geojson -nln ms_buildings

Phew, that was the hard command-line part. There may very well be newer and / or easier ways to accomplish this. Let me know in the comments.

Now we’re ready for the fun part :) Let’s open QGIS and load the data: the osm_buildings and ms_buildings tables. (You may need to establish a connection with your database first.). Here’s a look with MS buildings in blue and OSM buildings in orange overlaid on top:

We also add the osm_admin layer that imposm created. I decided that I want to create this challenge for the city of Millcreek:

So I filter the admin layer by that OSM id:

(I don’t know why imposm uses negative ids when importing OSM data, but it does.)

Next, I use QGIS’s Extract by location tool to create new layers for the buildings within Millcreek’s boundaries:

Now finally we can extract the buildings from the Microsoft layer that are disjoint from (don’t share any geometry with) the buildings from OSM:

These are the building features we want to present in the Challenge. We save this layer as a GeoJSON file we can load in JOSM for the final data pre-processing step:

Using the Find dialog in JOSM, we select all the ways and add the building=yes tag to them. Finally, we save the layer as an .osm file.

We can use this .osm file as the input for mr-cli. mr-cli is a command line tool to turn .osm or OSM Change files into cooperative or tag-fix challenge JSON data which we can then load into MapRoulette. The tool is documented here. We use the following command to generate the Challenge JSON:

mr coop change --out ms_not_in_osm_millcreek.json ms_not_in_osm_millcreek.osm

This will give us ms_not_in_osm_millcreek.json, a GeoJSON file with specific attributes that MapRoulette recognizes as source data for a Cooperative Challenge. So let’s make the Challenge!

Challenge Creation

In MapRoulette, we navigate to “Create and Manage” in the user menu on the top right. We choose (or create) a Project that this Challenge will live under and select “Add Challenge” in the Project management screen. This takes us to the Challenge wizard. Here, we can drop or select the GeoJSON file we created and add a Title, Description and Instructions. Also important is to pre-select the most appropriate (most recent) aerial imagery for mappers to use, if you know. Adding a Changeset comment to be pre-filled is also highly recommended:

When we’re satisfied that everything looks good, we fire off the Challenge creation. This can take a while to complete, but you can leave the loader screen and start inspecting your work:

Here’s a sample task!

I actually created this Challenge, if you want to have a go, here it is.

Conclusion

This was a long tutorial, congrats for making it all the way down here ❤️.

I realize that this is not a super straightforward process. It’s not really meant to be: adding outside data to OSM should be done carefully, and all the steps we have taken here are to ensure that we end up with a Challenge that is correct, meaningful and hopefully fun to do. That said, if you would like to set up a similar Challenge with MS Building Footprints or other external data, ask a question in the OSM community forums and tag it with #maproulette, or join the #maproulette channel in the OSM US Slack.